2023-11-28T10:55:10.126 INFO:root:teuthology version: 0.0.1.dev209+g5ae09a5 2023-11-28T10:55:10.126 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --owner scheduled_neesingh@teuthology --name neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi --archive /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} -- /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957/orig.config.yaml 2023-11-28T10:55:10.168 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T10:55:10.261 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957 branch: wip-neesingh-testing-231123 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} email: neesingh@redhat.com first_in_suite: false job_id: '7469957' kernel: kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-neesingh-testing-231123 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - inode wrongly marked free - bad backtrace on inode - inode table repaired for inode - Scrub error on inode - Scrub error on dir - Metadata damage detected sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: issue_54557 sha1: e67b67438c1e5e13734a7203ff9e57e30b8e4d32 owner: scheduled_neesingh@teuthology priority: 10 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 9437 sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 sleep_before_teardown: 0 suite: fs suite_branch: issue_54557 suite_path: /home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa suite_relpath: qa suite_repo: https://github.com/neesingh-rh/ceph suite_sha1: e67b67438c1e5e13734a7203ff9e57e30b8e4d32 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi028.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= smithi059.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_forward_scrub teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 5ae09a5cc7064ca2bf994df19f3006b79794789a timestamp: 2023-11-28_10:48:03 tube: smithi user: neesingh verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-28T10:55:10.261 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa; will attempt to use it 2023-11-28T10:55:10.262 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks 2023-11-28T10:55:10.262 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-28T10:55:10.263 INFO:teuthology.task.internal:Checking packages... 2023-11-28T10:55:10.285 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '2f5f6028b893ad41357cd06e827373f0fe2a1232' 2023-11-28T10:55:10.285 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-28T10:55:10.286 INFO:teuthology.packaging:ref: None 2023-11-28T10:55:10.286 INFO:teuthology.packaging:tag: None 2023-11-28T10:55:10.286 INFO:teuthology.packaging:branch: wip-neesingh-testing-231123 2023-11-28T10:55:10.286 INFO:teuthology.packaging:sha1: 2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T10:55:10.286 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-neesingh-testing-231123 2023-11-28T10:55:10.498 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-2697.g2f5f6028 2023-11-28T10:55:10.498 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-28T10:55:10.504 INFO:teuthology.task.internal:no buildpackages task found 2023-11-28T10:55:10.505 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-28T10:55:10.538 INFO:teuthology.task.internal:Saving configuration 2023-11-28T10:55:10.554 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-28T10:55:10.564 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T10:55:10.584 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi028.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:50:16.898860', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T10:55:10.605 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:50:16.900710', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T10:55:10.605 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-28T10:55:10.612 INFO:teuthology.task.internal:roles: ubuntu@smithi028.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-28T10:55:10.612 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-28T10:55:10.612 INFO:teuthology.run_tasks:Running task console_log... 2023-11-28T10:55:10.706 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f01ac382a60>, signals=[15]) 2023-11-28T10:55:10.707 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-28T10:55:10.713 INFO:teuthology.task.internal:Opening connections... 2023-11-28T10:55:10.713 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T10:55:10.715 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:55:10.783 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T10:55:10.784 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T10:55:10.851 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-28T10:55:10.856 DEBUG:teuthology.orchestra.run.smithi028:> uname -m 2023-11-28T10:55:10.878 INFO:teuthology.orchestra.run.smithi028.stdout:x86_64 2023-11-28T10:55:10.878 DEBUG:teuthology.orchestra.run.smithi028:> cat /etc/os-release 2023-11-28T10:55:10.939 INFO:teuthology.orchestra.run.smithi028.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T10:55:10.941 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T10:55:10.941 INFO:teuthology.orchestra.run.smithi028.stdout:ID="rhel" 2023-11-28T10:55:10.941 INFO:teuthology.orchestra.run.smithi028.stdout:ID_LIKE="fedora" 2023-11-28T10:55:10.941 INFO:teuthology.orchestra.run.smithi028.stdout:VERSION_ID="8.6" 2023-11-28T10:55:10.941 INFO:teuthology.orchestra.run.smithi028.stdout:PLATFORM_ID="platform:el8" 2023-11-28T10:55:10.941 INFO:teuthology.orchestra.run.smithi028.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T10:55:10.941 INFO:teuthology.orchestra.run.smithi028.stdout:ANSI_COLOR="0;31" 2023-11-28T10:55:10.942 INFO:teuthology.orchestra.run.smithi028.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T10:55:10.942 INFO:teuthology.orchestra.run.smithi028.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T10:55:10.942 INFO:teuthology.orchestra.run.smithi028.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T10:55:10.942 INFO:teuthology.orchestra.run.smithi028.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T10:55:10.942 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T10:55:10.942 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T10:55:10.942 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T10:55:10.942 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T10:55:10.942 INFO:teuthology.orchestra.run.smithi028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T10:55:10.943 INFO:teuthology.lock.ops:Updating smithi028.front.sepia.ceph.com on lock server 2023-11-28T10:55:10.961 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2023-11-28T10:55:10.981 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2023-11-28T10:55:10.981 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2023-11-28T10:55:11.040 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T10:55:11.041 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="8.6 (Ootpa)" 2023-11-28T10:55:11.041 INFO:teuthology.orchestra.run.smithi059.stdout:ID="rhel" 2023-11-28T10:55:11.041 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="fedora" 2023-11-28T10:55:11.041 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="8.6" 2023-11-28T10:55:11.041 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el8" 2023-11-28T10:55:11.041 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-11-28T10:55:11.041 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2023-11-28T10:55:11.041 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-11-28T10:55:11.041 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T10:55:11.041 INFO:teuthology.orchestra.run.smithi059.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T10:55:11.041 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T10:55:11.041 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T10:55:11.042 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T10:55:11.042 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-11-28T10:55:11.042 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T10:55:11.042 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-11-28T10:55:11.042 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2023-11-28T10:55:11.061 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-28T10:55:11.070 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-28T10:55:11.105 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-28T10:55:11.105 DEBUG:teuthology.orchestra.run.smithi028:> test '!' -e /home/ubuntu/cephtest 2023-11-28T10:55:11.108 DEBUG:teuthology.orchestra.run.smithi059:> test '!' -e /home/ubuntu/cephtest 2023-11-28T10:55:11.127 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-28T10:55:11.132 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-28T10:55:11.133 DEBUG:teuthology.orchestra.run.smithi028:> test -z $(ls -A /var/lib/ceph) 2023-11-28T10:55:11.169 DEBUG:teuthology.orchestra.run.smithi059:> test -z $(ls -A /var/lib/ceph) 2023-11-28T10:55:11.196 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-28T10:55:11.249 INFO:teuthology.run_tasks:Running task kernel... 2023-11-28T10:55:11.264 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-11-28T10:55:11.264 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-11-28T10:55:11.264 DEBUG:teuthology.orchestra.run.smithi028:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T10:55:11.265 DEBUG:teuthology.orchestra.run.smithi059:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T10:55:11.283 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:55:11.284 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2023-11-28T10:55:11.284 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:55:11.285 DEBUG:teuthology.orchestra.run.smithi028:> uname -r 2023-11-28T10:55:11.344 INFO:teuthology.orchestra.run.smithi059.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:11.344 INFO:teuthology.task.kernel:Running kernel on smithi059: 4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:11.344 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum install -y kernel 2023-11-28T10:55:11.346 INFO:teuthology.orchestra.run.smithi028.stdout:4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:11.346 INFO:teuthology.task.kernel:Running kernel on smithi028: 4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:11.346 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum install -y kernel 2023-11-28T10:55:12.505 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:12.505 INFO:teuthology.orchestra.run.smithi059.stdout:Unable to read consumer identity 2023-11-28T10:55:12.558 INFO:teuthology.orchestra.run.smithi028.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:12.558 INFO:teuthology.orchestra.run.smithi028.stdout:Unable to read consumer identity 2023-11-28T10:55:13.801 INFO:teuthology.orchestra.run.smithi059.stdout:Copr repo for python3-asyncssh owned by ceph 22 kB/s | 3.5 kB 00:00 2023-11-28T10:55:14.172 INFO:teuthology.orchestra.run.smithi028.stdout:Copr repo for python3-asyncssh owned by ceph 9.0 kB/s | 3.5 kB 00:00 2023-11-28T10:55:14.490 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 24 MB/s | 16 MB 00:00 2023-11-28T10:55:14.872 INFO:teuthology.orchestra.run.smithi028.stdout:Extra Packages for Enterprise Linux 24 MB/s | 16 MB 00:00 2023-11-28T10:55:18.149 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 583 kB/s | 24 kB 00:00 2023-11-28T10:55:18.547 INFO:teuthology.orchestra.run.smithi028.stdout:lab-extras 583 kB/s | 24 kB 00:00 2023-11-28T10:55:18.835 INFO:teuthology.orchestra.run.smithi059.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-28T10:55:18.893 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2023-11-28T10:55:18.894 INFO:teuthology.orchestra.run.smithi059.stdout:Nothing to do. 2023-11-28T10:55:18.894 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2023-11-28T10:55:19.025 DEBUG:teuthology.orchestra.run.smithi059:> echo no | sudo yum reinstall kernel || true 2023-11-28T10:55:19.225 INFO:teuthology.orchestra.run.smithi028.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-11-28T10:55:19.271 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2023-11-28T10:55:19.272 INFO:teuthology.orchestra.run.smithi028.stdout:Nothing to do. 2023-11-28T10:55:19.272 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2023-11-28T10:55:19.376 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:19.376 INFO:teuthology.orchestra.run.smithi059.stdout:Unable to read consumer identity 2023-11-28T10:55:19.398 DEBUG:teuthology.orchestra.run.smithi028:> echo no | sudo yum reinstall kernel || true 2023-11-28T10:55:19.656 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:00:01 ago on Tue 28 Nov 2023 10:55:18 AM UTC. 2023-11-28T10:55:19.698 INFO:teuthology.orchestra.run.smithi059.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T10:55:19.727 INFO:teuthology.orchestra.run.smithi059.stderr:Error: No packages marked for reinstall. 2023-11-28T10:55:19.748 INFO:teuthology.orchestra.run.smithi028.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:19.749 INFO:teuthology.orchestra.run.smithi028.stdout:Unable to read consumer identity 2023-11-28T10:55:19.774 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum reinstall -y kernel || true 2023-11-28T10:55:20.021 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 10:55:18 AM UTC. 2023-11-28T10:55:20.068 INFO:teuthology.orchestra.run.smithi028.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T10:55:20.106 INFO:teuthology.orchestra.run.smithi028.stderr:Error: No packages marked for reinstall. 2023-11-28T10:55:20.123 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:20.123 INFO:teuthology.orchestra.run.smithi059.stdout:Unable to read consumer identity 2023-11-28T10:55:20.153 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum reinstall -y kernel || true 2023-11-28T10:55:20.396 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 10:55:18 AM UTC. 2023-11-28T10:55:20.438 INFO:teuthology.orchestra.run.smithi059.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T10:55:20.457 INFO:teuthology.orchestra.run.smithi059.stderr:Error: No packages marked for reinstall. 2023-11-28T10:55:20.501 INFO:teuthology.orchestra.run.smithi028.stdout:Updating Subscription Management repositories. 2023-11-28T10:55:20.502 INFO:teuthology.orchestra.run.smithi028.stdout:Unable to read consumer identity 2023-11-28T10:55:20.506 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T10:55:20.588 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:20.588 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:20.589 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-28T10:55:20.589 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-11-28T10:55:20.589 DEBUG:teuthology.orchestra.run.smithi059:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-28T10:55:20.626 INFO:teuthology.orchestra.run.smithi059.stdout:ttyS1 2023-11-28T10:55:20.638 DEBUG:teuthology.parallel:result is None 2023-11-28T10:55:20.774 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 10:55:18 AM UTC. 2023-11-28T10:55:20.815 INFO:teuthology.orchestra.run.smithi028.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-11-28T10:55:20.837 INFO:teuthology.orchestra.run.smithi028.stderr:Error: No packages marked for reinstall. 2023-11-28T10:55:20.884 DEBUG:teuthology.orchestra.run.smithi028:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T10:55:20.967 INFO:teuthology.orchestra.run.smithi028.stdout:kernel-4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:20.968 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-372.9.1.el8.x86_64 2023-11-28T10:55:20.968 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-11-28T10:55:20.968 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-11-28T10:55:20.968 DEBUG:teuthology.orchestra.run.smithi028:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-11-28T10:55:21.001 INFO:teuthology.orchestra.run.smithi028.stdout:ttyS1 2023-11-28T10:55:21.021 DEBUG:teuthology.parallel:result is None 2023-11-28T10:55:21.021 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-28T10:55:21.027 INFO:teuthology.task.internal:Creating test directory... 2023-11-28T10:55:21.028 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T10:55:21.063 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T10:55:21.081 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-28T10:55:21.088 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-28T10:55:21.112 INFO:teuthology.task.internal:Creating archive directory... 2023-11-28T10:55:21.113 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T10:55:21.121 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T10:55:21.146 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-28T10:55:21.153 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-28T10:55:21.153 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T10:55:21.178 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T10:55:21.211 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T10:55:21.222 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T10:55:21.233 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T10:55:21.244 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T10:55:21.246 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-28T10:55:21.253 INFO:teuthology.task.internal:Configuring sudo... 2023-11-28T10:55:21.253 DEBUG:teuthology.orchestra.run.smithi028:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T10:55:21.265 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T10:55:21.294 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-28T10:55:21.303 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-28T10:55:21.303 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T10:55:21.334 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T10:55:21.351 DEBUG:teuthology.orchestra.run.smithi028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T10:55:21.432 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T10:55:21.487 DEBUG:teuthology.orchestra.run.smithi028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T10:55:21.571 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T10:55:21.613 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T10:55:21.613 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T10:55:21.703 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T10:55:21.750 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T10:55:21.819 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T10:55:21.865 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T10:55:21.909 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T10:55:21.909 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T10:55:21.979 DEBUG:teuthology.orchestra.run.smithi028:> sudo service rsyslog restart 2023-11-28T10:55:21.981 DEBUG:teuthology.orchestra.run.smithi059:> sudo service rsyslog restart 2023-11-28T10:55:22.033 INFO:teuthology.orchestra.run.smithi028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T10:55:22.102 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T10:55:22.473 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-28T10:55:22.483 INFO:teuthology.task.internal:Starting timer... 2023-11-28T10:55:22.483 INFO:teuthology.run_tasks:Running task pcp... 2023-11-28T10:55:22.560 INFO:teuthology.run_tasks:Running task selinux... 2023-11-28T10:55:22.591 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-11-28T10:55:22.592 DEBUG:teuthology.orchestra.run.smithi028:> sudo service auditd rotate 2023-11-28T10:55:22.657 INFO:teuthology.orchestra.run.smithi028.stdout:Rotating logs: 2023-11-28T10:55:22.658 DEBUG:teuthology.orchestra.run.smithi059:> sudo service auditd rotate 2023-11-28T10:55:22.716 INFO:teuthology.orchestra.run.smithi059.stdout:Rotating logs: 2023-11-28T10:55:22.717 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-28T10:55:22.718 DEBUG:teuthology.orchestra.run.smithi028:> /usr/sbin/getenforce 2023-11-28T10:55:22.739 INFO:teuthology.orchestra.run.smithi028.stdout:Permissive 2023-11-28T10:55:22.739 DEBUG:teuthology.orchestra.run.smithi059:> /usr/sbin/getenforce 2023-11-28T10:55:22.765 INFO:teuthology.orchestra.run.smithi059.stdout:Permissive 2023-11-28T10:55:22.765 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi028.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi059.front.sepia.ceph.com': 'permissive'} 2023-11-28T10:55:22.765 DEBUG:teuthology.orchestra.run.smithi028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T10:55:22.807 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:55:22.808 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T10:55:22.837 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T10:55:22.838 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-28T10:55:22.838 DEBUG:teuthology.orchestra.run.smithi028:> sudo /usr/sbin/setenforce permissive 2023-11-28T10:55:22.876 DEBUG:teuthology.orchestra.run.smithi059:> sudo /usr/sbin/setenforce permissive 2023-11-28T10:55:22.922 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-28T10:55:22.932 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-11-28T10:55:22.932 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-28T10:55:22.947 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-28T10:55:22.948 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi028.front.sepia.ceph.com,smithi059.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-28T11:05:11.840 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi028.front.sepia.ceph.com'), Remote(name='ubuntu@smithi059.front.sepia.ceph.com')] 2023-11-28T11:05:11.841 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi028.front.sepia.ceph.com' 2023-11-28T11:05:11.842 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:05:11.913 DEBUG:teuthology.orchestra.run.smithi028:> true 2023-11-28T11:05:11.983 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi028.front.sepia.ceph.com' 2023-11-28T11:05:11.983 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2023-11-28T11:05:11.984 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T11:05:12.054 DEBUG:teuthology.orchestra.run.smithi059:> true 2023-11-28T11:05:12.123 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2023-11-28T11:05:12.124 INFO:teuthology.run_tasks:Running task clock... 2023-11-28T11:05:12.134 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-28T11:05:12.135 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T11:05:12.135 DEBUG:teuthology.orchestra.run.smithi028:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:05:12.139 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T11:05:12.139 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:05:12.174 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T11:05:12.191 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T11:05:12.196 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T11:05:12.211 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T11:05:12.279 INFO:teuthology.orchestra.run.smithi028.stderr:sudo: ntpd: command not found 2023-11-28T11:05:12.292 INFO:teuthology.orchestra.run.smithi028.stdout:506 Cannot talk to daemon 2023-11-28T11:05:12.296 INFO:teuthology.orchestra.run.smithi059.stderr:sudo: ntpd: command not found 2023-11-28T11:05:12.308 INFO:teuthology.orchestra.run.smithi059.stdout:506 Cannot talk to daemon 2023-11-28T11:05:12.309 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T11:05:12.323 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T11:05:12.324 INFO:teuthology.orchestra.run.smithi028.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T11:05:12.337 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T11:05:12.392 INFO:teuthology.orchestra.run.smithi028.stderr:bash: ntpq: command not found 2023-11-28T11:05:12.395 INFO:teuthology.orchestra.run.smithi028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:05:12.395 INFO:teuthology.orchestra.run.smithi028.stdout:=============================================================================== 2023-11-28T11:05:12.396 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:12.396 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:12.396 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:12.396 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:12.411 INFO:teuthology.orchestra.run.smithi059.stderr:bash: ntpq: command not found 2023-11-28T11:05:12.415 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:05:12.415 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2023-11-28T11:05:12.416 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:12.416 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:12.416 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:12.416 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:05:12.416 INFO:teuthology.run_tasks:Running task install... 2023-11-28T11:05:12.426 DEBUG:teuthology.task.install:project ceph 2023-11-28T11:05:12.426 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}} 2023-11-28T11:05:12.426 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'} 2023-11-28T11:05:12.426 INFO:teuthology.task.install:Using flavor: default 2023-11-28T11:05:12.433 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-28T11:05:12.433 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-28T11:05:12.434 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T11:05:12.434 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:05:12.436 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'tag': None, 'wait_for_package': False} 2023-11-28T11:05:12.437 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:05:12.587 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T11:05:12.587 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T11:05:12.592 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/ 2023-11-28T11:05:12.593 INFO:teuthology.task.install.rpm:Package version is 18.0.0-2697.g2f5f6028 2023-11-28T11:05:12.696 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T11:05:12.696 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:05:12.697 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T11:05:12.726 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T11:05:12.727 DEBUG:teuthology.orchestra.run.smithi059:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T11:05:12.796 DEBUG:teuthology.orchestra.run.smithi059:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T11:05:12.823 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-neesingh-testing-231123/2f5f6028b893ad41357cd06e827373f0fe2a1232/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T11:05:12.823 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:05:12.824 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T11:05:12.855 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-11-28T11:05:12.855 DEBUG:teuthology.orchestra.run.smithi028:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2f5f6028b893ad41357cd06e827373f0fe2a1232/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T11:05:12.876 DEBUG:teuthology.orchestra.run.smithi059:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T11:05:12.927 DEBUG:teuthology.orchestra.run.smithi028:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T11:05:12.944 INFO:teuthology.orchestra.run.smithi059.stdout:check_obsoletes = 1 2023-11-28T11:05:12.945 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum clean all 2023-11-28T11:05:13.009 DEBUG:teuthology.orchestra.run.smithi028:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T11:05:13.077 INFO:teuthology.orchestra.run.smithi028.stdout:check_obsoletes = 1 2023-11-28T11:05:13.079 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum clean all 2023-11-28T11:05:13.336 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-11-28T11:05:13.473 INFO:teuthology.orchestra.run.smithi028.stdout:Updating Subscription Management repositories. 2023-11-28T11:05:14.454 INFO:teuthology.orchestra.run.smithi059.stdout:44 files removed 2023-11-28T11:05:14.468 INFO:teuthology.orchestra.run.smithi028.stdout:44 files removed 2023-11-28T11:05:14.510 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T11:05:14.524 DEBUG:teuthology.orchestra.run.smithi028:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-11-28T11:05:14.862 INFO:teuthology.orchestra.run.smithi059.stdout:Updating Subscription Management repositories. 2023-11-28T11:05:14.876 INFO:teuthology.orchestra.run.smithi028.stdout:Updating Subscription Management repositories. 2023-11-28T11:05:16.199 INFO:teuthology.orchestra.run.smithi028.stdout:ceph packages for x86_64 229 kB/s | 80 kB 00:00 2023-11-28T11:05:16.454 INFO:teuthology.orchestra.run.smithi059.stdout:ceph packages for x86_64 163 kB/s | 80 kB 00:00 2023-11-28T11:05:16.499 INFO:teuthology.orchestra.run.smithi028.stdout:ceph noarch packages 35 kB/s | 9.4 kB 00:00 2023-11-28T11:05:16.747 INFO:teuthology.orchestra.run.smithi059.stdout:ceph noarch packages 36 kB/s | 9.4 kB 00:00 2023-11-28T11:05:16.784 INFO:teuthology.orchestra.run.smithi028.stdout:ceph source packages 5.7 kB/s | 1.5 kB 00:00 2023-11-28T11:05:16.918 INFO:teuthology.orchestra.run.smithi028.stdout:Copr repo for python3-asyncssh owned by ceph 31 kB/s | 3.5 kB 00:00 2023-11-28T11:05:17.089 INFO:teuthology.orchestra.run.smithi059.stdout:ceph source packages 4.7 kB/s | 1.5 kB 00:00 2023-11-28T11:05:17.232 INFO:teuthology.orchestra.run.smithi059.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-11-28T11:05:18.829 INFO:teuthology.orchestra.run.smithi028.stdout:Extra Packages for Enterprise Linux 8.4 MB/s | 16 MB 00:01 2023-11-28T11:05:22.277 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 3.2 MB/s | 16 MB 00:05 2023-11-28T11:05:22.568 INFO:teuthology.orchestra.run.smithi028.stdout:lab-extras 409 kB/s | 24 kB 00:00 2023-11-28T11:05:23.545 INFO:teuthology.orchestra.run.smithi028.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 53 MB/s | 47 MB 00:00 2023-11-28T11:05:26.042 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 434 kB/s | 24 kB 00:00 2023-11-28T11:05:27.068 INFO:teuthology.orchestra.run.smithi059.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 50 MB/s | 47 MB 00:00 2023-11-28T11:05:32.230 INFO:teuthology.orchestra.run.smithi028.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 49 MB/s | 53 MB 00:01 2023-11-28T11:05:35.769 INFO:teuthology.orchestra.run.smithi059.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 53 MB 00:01 2023-11-28T11:05:44.129 INFO:teuthology.orchestra.run.smithi028.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:05:44.130 INFO:teuthology.orchestra.run.smithi028.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:05:44.145 INFO:teuthology.orchestra.run.smithi028.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T11:05:44.146 INFO:teuthology.orchestra.run.smithi028.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T11:05:44.147 INFO:teuthology.orchestra.run.smithi028.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T11:05:44.148 INFO:teuthology.orchestra.run.smithi028.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T11:05:44.150 INFO:teuthology.orchestra.run.smithi028.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T11:05:44.153 INFO:teuthology.orchestra.run.smithi028.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T11:05:44.154 INFO:teuthology.orchestra.run.smithi028.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T11:05:44.300 INFO:teuthology.orchestra.run.smithi028.stdout:Dependencies resolved. 2023-11-28T11:05:44.314 INFO:teuthology.orchestra.run.smithi028.stdout:=========================================================================================================================== 2023-11-28T11:05:44.314 INFO:teuthology.orchestra.run.smithi028.stdout: Package Arch Version Repository Size 2023-11-28T11:05:44.314 INFO:teuthology.orchestra.run.smithi028.stdout:=========================================================================================================================== 2023-11-28T11:05:44.314 INFO:teuthology.orchestra.run.smithi028.stdout:Installing: 2023-11-28T11:05:44.314 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T11:05:44.314 INFO:teuthology.orchestra.run.smithi028.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T11:05:44.314 INFO:teuthology.orchestra.run.smithi028.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T11:05:44.315 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T11:05:44.315 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T11:05:44.315 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T11:05:44.315 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T11:05:44.315 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T11:05:44.315 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T11:05:44.315 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T11:05:44.315 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T11:05:44.315 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T11:05:44.315 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T11:05:44.315 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T11:05:44.316 INFO:teuthology.orchestra.run.smithi028.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T11:05:44.316 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T11:05:44.316 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T11:05:44.316 INFO:teuthology.orchestra.run.smithi028.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T11:05:44.316 INFO:teuthology.orchestra.run.smithi028.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T11:05:44.316 INFO:teuthology.orchestra.run.smithi028.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T11:05:44.316 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T11:05:44.316 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T11:05:44.316 INFO:teuthology.orchestra.run.smithi028.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T11:05:44.316 INFO:teuthology.orchestra.run.smithi028.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T11:05:44.316 INFO:teuthology.orchestra.run.smithi028.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T11:05:44.316 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T11:05:44.317 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T11:05:44.317 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T11:05:44.317 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T11:05:44.317 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T11:05:44.317 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T11:05:44.317 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T11:05:44.317 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T11:05:44.317 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T11:05:44.317 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T11:05:44.317 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T11:05:44.317 INFO:teuthology.orchestra.run.smithi028.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T11:05:44.317 INFO:teuthology.orchestra.run.smithi028.stdout:Upgrading: 2023-11-28T11:05:44.318 INFO:teuthology.orchestra.run.smithi028.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T11:05:44.318 INFO:teuthology.orchestra.run.smithi028.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T11:05:44.318 INFO:teuthology.orchestra.run.smithi028.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T11:05:44.318 INFO:teuthology.orchestra.run.smithi028.stdout:Installing dependencies: 2023-11-28T11:05:44.318 INFO:teuthology.orchestra.run.smithi028.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T11:05:44.318 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T11:05:44.318 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T11:05:44.319 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T11:05:44.319 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T11:05:44.319 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T11:05:44.319 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T11:05:44.319 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T11:05:44.319 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T11:05:44.319 INFO:teuthology.orchestra.run.smithi028.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T11:05:44.319 INFO:teuthology.orchestra.run.smithi028.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T11:05:44.319 INFO:teuthology.orchestra.run.smithi028.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T11:05:44.319 INFO:teuthology.orchestra.run.smithi028.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T11:05:44.319 INFO:teuthology.orchestra.run.smithi028.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T11:05:44.319 INFO:teuthology.orchestra.run.smithi028.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T11:05:44.320 INFO:teuthology.orchestra.run.smithi028.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T11:05:44.320 INFO:teuthology.orchestra.run.smithi028.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T11:05:44.320 INFO:teuthology.orchestra.run.smithi028.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T11:05:44.320 INFO:teuthology.orchestra.run.smithi028.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T11:05:44.320 INFO:teuthology.orchestra.run.smithi028.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T11:05:44.320 INFO:teuthology.orchestra.run.smithi028.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T11:05:44.320 INFO:teuthology.orchestra.run.smithi028.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T11:05:44.321 INFO:teuthology.orchestra.run.smithi028.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T11:05:44.321 INFO:teuthology.orchestra.run.smithi028.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T11:05:44.321 INFO:teuthology.orchestra.run.smithi028.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T11:05:44.321 INFO:teuthology.orchestra.run.smithi028.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T11:05:44.321 INFO:teuthology.orchestra.run.smithi028.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T11:05:44.321 INFO:teuthology.orchestra.run.smithi028.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T11:05:44.321 INFO:teuthology.orchestra.run.smithi028.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T11:05:44.321 INFO:teuthology.orchestra.run.smithi028.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T11:05:44.321 INFO:teuthology.orchestra.run.smithi028.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T11:05:44.321 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T11:05:44.321 INFO:teuthology.orchestra.run.smithi028.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T11:05:44.321 INFO:teuthology.orchestra.run.smithi028.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T11:05:44.322 INFO:teuthology.orchestra.run.smithi028.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T11:05:44.322 INFO:teuthology.orchestra.run.smithi028.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T11:05:44.322 INFO:teuthology.orchestra.run.smithi028.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T11:05:44.322 INFO:teuthology.orchestra.run.smithi028.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T11:05:44.322 INFO:teuthology.orchestra.run.smithi028.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T11:05:44.322 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T11:05:44.322 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T11:05:44.322 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T11:05:44.322 INFO:teuthology.orchestra.run.smithi028.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T11:05:44.322 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T11:05:44.322 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T11:05:44.322 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T11:05:44.322 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T11:05:44.323 INFO:teuthology.orchestra.run.smithi028.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T11:05:44.323 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T11:05:44.323 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T11:05:44.323 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T11:05:44.323 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T11:05:44.323 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T11:05:44.323 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:05:44.323 INFO:teuthology.orchestra.run.smithi028.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T11:05:44.323 INFO:teuthology.orchestra.run.smithi028.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T11:05:44.323 INFO:teuthology.orchestra.run.smithi028.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T11:05:44.323 INFO:teuthology.orchestra.run.smithi028.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T11:05:44.325 INFO:teuthology.orchestra.run.smithi028.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T11:05:44.325 INFO:teuthology.orchestra.run.smithi028.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T11:05:44.325 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T11:05:44.325 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T11:05:44.325 INFO:teuthology.orchestra.run.smithi028.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T11:05:44.325 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T11:05:44.325 INFO:teuthology.orchestra.run.smithi028.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T11:05:44.325 INFO:teuthology.orchestra.run.smithi028.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T11:05:44.325 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T11:05:44.325 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T11:05:44.326 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T11:05:44.326 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T11:05:44.326 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T11:05:44.326 INFO:teuthology.orchestra.run.smithi028.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T11:05:44.326 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:05:44.331 INFO:teuthology.orchestra.run.smithi028.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T11:05:44.331 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T11:05:44.331 INFO:teuthology.orchestra.run.smithi028.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T11:05:44.331 INFO:teuthology.orchestra.run.smithi028.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T11:05:44.331 INFO:teuthology.orchestra.run.smithi028.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T11:05:44.332 INFO:teuthology.orchestra.run.smithi028.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T11:05:44.332 INFO:teuthology.orchestra.run.smithi028.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T11:05:44.332 INFO:teuthology.orchestra.run.smithi028.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T11:05:44.332 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T11:05:44.332 INFO:teuthology.orchestra.run.smithi028.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T11:05:44.332 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T11:05:44.332 INFO:teuthology.orchestra.run.smithi028.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T11:05:44.332 INFO:teuthology.orchestra.run.smithi028.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T11:05:44.332 INFO:teuthology.orchestra.run.smithi028.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T11:05:44.332 INFO:teuthology.orchestra.run.smithi028.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T11:05:44.333 INFO:teuthology.orchestra.run.smithi028.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T11:05:44.333 INFO:teuthology.orchestra.run.smithi028.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T11:05:44.333 INFO:teuthology.orchestra.run.smithi028.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T11:05:44.333 INFO:teuthology.orchestra.run.smithi028.stdout:Enabling module streams: 2023-11-28T11:05:44.333 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql 10 2023-11-28T11:05:44.333 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:05:44.333 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction Summary 2023-11-28T11:05:44.333 INFO:teuthology.orchestra.run.smithi028.stdout:=========================================================================================================================== 2023-11-28T11:05:44.333 INFO:teuthology.orchestra.run.smithi028.stdout:Install 128 Packages 2023-11-28T11:05:44.333 INFO:teuthology.orchestra.run.smithi028.stdout:Upgrade 3 Packages 2023-11-28T11:05:44.333 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:05:44.335 INFO:teuthology.orchestra.run.smithi028.stdout:Total download size: 203 M 2023-11-28T11:05:44.335 INFO:teuthology.orchestra.run.smithi028.stdout:Downloading Packages: 2023-11-28T11:05:44.714 INFO:teuthology.orchestra.run.smithi028.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 18 kB/s | 6.4 kB 00:00 2023-11-28T11:05:45.881 INFO:teuthology.orchestra.run.smithi028.stdout:(2/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 725 kB/s | 845 kB 00:01 2023-11-28T11:05:46.081 INFO:teuthology.orchestra.run.smithi028.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 827 kB/s | 165 kB 00:00 2023-11-28T11:05:47.389 INFO:teuthology.orchestra.run.smithi028.stdout:(4/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 1.7 MB/s | 2.3 MB 00:01 2023-11-28T11:05:47.618 INFO:teuthology.orchestra.run.smithi059.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:05:47.619 INFO:teuthology.orchestra.run.smithi059.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T11:05:47.634 INFO:teuthology.orchestra.run.smithi059.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-11-28T11:05:47.636 INFO:teuthology.orchestra.run.smithi059.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-11-28T11:05:47.636 INFO:teuthology.orchestra.run.smithi059.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-11-28T11:05:47.638 INFO:teuthology.orchestra.run.smithi059.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T11:05:47.640 INFO:teuthology.orchestra.run.smithi059.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T11:05:47.643 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-11-28T11:05:47.644 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-11-28T11:05:47.790 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2023-11-28T11:05:47.803 INFO:teuthology.orchestra.run.smithi059.stdout:=========================================================================================================================== 2023-11-28T11:05:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2023-11-28T11:05:47.803 INFO:teuthology.orchestra.run.smithi059.stdout:=========================================================================================================================== 2023-11-28T11:05:47.803 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2023-11-28T11:05:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-initscripts-updown noarch 1:1.36.0-9.el8_6 rhel-8-for-x86_64-baseos-rpms 138 k 2023-11-28T11:05:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T11:05:47.803 INFO:teuthology.orchestra.run.smithi059.stdout: ceph x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.4 k 2023-11-28T11:05:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 5.4 M 2023-11-28T11:05:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 845 k 2023-11-28T11:05:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 165 k 2023-11-28T11:05:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 1.5 M 2023-11-28T11:05:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 123 k 2023-11-28T11:05:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 1.7 M 2023-11-28T11:05:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 7.4 M 2023-11-28T11:05:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 47 k 2023-11-28T11:05:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 13 M 2023-11-28T11:05:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 47 M 2023-11-28T11:05:47.804 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 263 k 2023-11-28T11:05:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 83 k 2023-11-28T11:05:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 257 k 2023-11-28T11:05:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 18 k 2023-11-28T11:05:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel x86_64 0.186-1.el8 rhel-8-for-x86_64-baseos-rpms 60 k 2023-11-28T11:05:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T11:05:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T11:05:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 30 k 2023-11-28T11:05:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 767 k 2023-11-28T11:05:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 139 k 2023-11-28T11:05:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T11:05:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel x86_64 2.32.1-35.el8 rhel-8-for-x86_64-baseos-rpms 98 k 2023-11-28T11:05:47.805 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel x86_64 1:1.1.1k-7.el8_6 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T11:05:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-11-28T11:05:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 808 k 2023-11-28T11:05:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server x86_64 10.21-2.module+el8.6.0+15342+53518fac rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-11-28T11:05:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 190 k 2023-11-28T11:05:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 371 k 2023-11-28T11:05:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 364 k 2023-11-28T11:05:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 107 k 2023-11-28T11:05:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 94 k 2023-11-28T11:05:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.4 M 2023-11-28T11:05:47.806 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 180 k 2023-11-28T11:05:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel x86_64 5.0.0-10.el8 rhel-8-for-x86_64-baseos-rpms 72 k 2023-11-28T11:05:47.807 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrading: 2023-11-28T11:05:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: iproute x86_64 5.15.0-4.el8_6.1 rhel-8-for-x86_64-baseos-rpms 798 k 2023-11-28T11:05:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: librados2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.7 M 2023-11-28T11:05:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 3.6 M 2023-11-28T11:05:47.807 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2023-11-28T11:05:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T11:05:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 22 M 2023-11-28T11:05:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 23 k 2023-11-28T11:05:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 2.3 M 2023-11-28T11:05:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 236 k 2023-11-28T11:05:47.807 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 4.6 M 2023-11-28T11:05:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 18 M 2023-11-28T11:05:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts noarch 2:18.0.0-2697.g2f5f6028.el8 ceph-noarch 15 k 2023-11-28T11:05:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 24 k 2023-11-28T11:05:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: fuse x86_64 2.9.7-15.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T11:05:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T11:05:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: jq x86_64 1.6-3.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-11-28T11:05:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T11:05:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: krb5-devel x86_64 1.18.2-14.el8 rhel-8-for-x86_64-baseos-rpms 560 k 2023-11-28T11:05:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 137 k 2023-11-28T11:05:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-11-28T11:05:47.808 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 644 k 2023-11-28T11:05:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T11:05:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: libpq x86_64 13.5-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-11-28T11:05:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath x86_64 8.5.0-10.1.el8_6 rhel-8-for-x86_64-baseos-rpms 171 k 2023-11-28T11:05:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T11:05:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 474 k 2023-11-28T11:05:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-11-28T11:05:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2 x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 6.5 M 2023-11-28T11:05:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T11:05:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-11-28T11:05:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T11:05:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T11:05:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T11:05:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T11:05:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: openblas x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T11:05:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-threads x86_64 0.3.15-3.el8 rhel-8-for-x86_64-appstream-rpms 4.9 M 2023-11-28T11:05:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T11:05:47.810 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T11:05:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T11:05:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-11-28T11:05:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T11:05:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T11:05:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T11:05:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T11:05:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 45 k 2023-11-28T11:05:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common x86_64 2:18.0.0-2697.g2f5f6028.el8 ceph 117 k 2023-11-28T11:05:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T11:05:47.811 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi x86_64 1.11.5-5.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-11-28T11:05:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T11:05:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T11:05:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography x86_64 3.2.1-5.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T11:05:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T11:05:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T11:05:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T11:05:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T11:05:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T11:05:47.812 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T11:05:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:05:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T11:05:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T11:05:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T11:05:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T11:05:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T11:05:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-11-28T11:05:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T11:05:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T11:05:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T11:05:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T11:05:47.813 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T11:05:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T11:05:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-11-28T11:05:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T11:05:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T11:05:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-11-28T11:05:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T11:05:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T11:05:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T11:05:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T11:05:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T11:05:47.814 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-11-28T11:05:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T11:05:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T11:05:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T11:05:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T11:05:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T11:05:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T11:05:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T11:05:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T11:05:47.815 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T11:05:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T11:05:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-11-28T11:05:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-11-28T11:05:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-11-28T11:05:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T11:05:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T11:05:47.816 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling module streams: 2023-11-28T11:05:47.816 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql 10 2023-11-28T11:05:47.817 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:05:47.817 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2023-11-28T11:05:47.817 INFO:teuthology.orchestra.run.smithi059.stdout:=========================================================================================================================== 2023-11-28T11:05:47.817 INFO:teuthology.orchestra.run.smithi059.stdout:Install 128 Packages 2023-11-28T11:05:47.817 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade 3 Packages 2023-11-28T11:05:47.817 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:05:47.818 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 203 M 2023-11-28T11:05:47.818 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2023-11-28T11:05:48.173 INFO:teuthology.orchestra.run.smithi028.stdout:(5/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 2.0 MB/s | 1.5 MB 00:00 2023-11-28T11:05:48.286 INFO:teuthology.orchestra.run.smithi059.stdout:(1/131): ceph-18.0.0-2697.g2f5f6028.el8.x86_64. 23 kB/s | 6.4 kB 00:00 2023-11-28T11:05:49.028 INFO:teuthology.orchestra.run.smithi059.stdout:(2/131): ceph-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.1 MB/s | 845 kB 00:00 2023-11-28T11:05:49.212 INFO:teuthology.orchestra.run.smithi059.stdout:(3/131): ceph-immutable-object-cache-18.0.0-269 904 kB/s | 165 kB 00:00 2023-11-28T11:05:50.131 INFO:teuthology.orchestra.run.smithi028.stdout:(6/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 954 kB/s | 5.4 MB 00:05 2023-11-28T11:05:50.923 INFO:teuthology.orchestra.run.smithi028.stdout:(7/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 1.7 MB/s | 4.6 MB 00:02 2023-11-28T11:05:51.178 INFO:teuthology.orchestra.run.smithi059.stdout:(4/131): ceph-mds-18.0.0-2697.g2f5f6028.el8.x86 1.2 MB/s | 2.3 MB 00:01 2023-11-28T11:05:52.162 INFO:teuthology.orchestra.run.smithi059.stdout:(5/131): ceph-mgr-18.0.0-2697.g2f5f6028.el8.x86 1.6 MB/s | 1.5 MB 00:00 2023-11-28T11:05:53.621 INFO:teuthology.orchestra.run.smithi059.stdout:(6/131): ceph-base-18.0.0-2697.g2f5f6028.el8.x8 980 kB/s | 5.4 MB 00:05 2023-11-28T11:05:54.382 INFO:teuthology.orchestra.run.smithi028.stdout:(8/131): ceph-common-18.0.0-2697.g2f5f6028.el8. 2.1 MB/s | 22 MB 00:10 2023-11-28T11:05:54.482 INFO:teuthology.orchestra.run.smithi028.stdout:(9/131): ceph-selinux-18.0.0-2697.g2f5f6028.el8 244 kB/s | 24 kB 00:00 2023-11-28T11:05:55.121 INFO:teuthology.orchestra.run.smithi059.stdout:(7/131): ceph-mon-18.0.0-2697.g2f5f6028.el8.x86 1.6 MB/s | 4.6 MB 00:02 2023-11-28T11:05:59.015 INFO:teuthology.orchestra.run.smithi028.stdout:(10/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el 1.6 MB/s | 13 MB 00:08 2023-11-28T11:05:59.215 INFO:teuthology.orchestra.run.smithi028.stdout:(11/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 1.3 MB/s | 257 kB 00:00 2023-11-28T11:05:59.332 INFO:teuthology.orchestra.run.smithi028.stdout:(12/131): libcephfs-devel-18.0.0-2697.g2f5f6028 261 kB/s | 30 kB 00:00 2023-11-28T11:05:59.799 INFO:teuthology.orchestra.run.smithi028.stdout:(13/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 1.6 MB/s | 767 kB 00:00 2023-11-28T11:05:59.916 INFO:teuthology.orchestra.run.smithi028.stdout:(14/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 1.2 MB/s | 137 kB 00:00 2023-11-28T11:06:00.383 INFO:teuthology.orchestra.run.smithi028.stdout:(15/131): librados-devel-18.0.0-2697.g2f5f6028. 298 kB/s | 139 kB 00:00 2023-11-28T11:06:00.758 INFO:teuthology.orchestra.run.smithi028.stdout:(16/131): libradosstriper1-18.0.0-2697.g2f5f602 1.2 MB/s | 474 kB 00:00 2023-11-28T11:06:01.287 INFO:teuthology.orchestra.run.smithi059.stdout:(8/131): ceph-common-18.0.0-2697.g2f5f6028.el8. 1.6 MB/s | 22 MB 00:13 2023-11-28T11:06:01.421 INFO:teuthology.orchestra.run.smithi059.stdout:(9/131): ceph-radosgw-18.0.0-2697.g2f5f6028.el8 2.1 MB/s | 13 MB 00:06 2023-11-28T11:06:01.446 INFO:teuthology.orchestra.run.smithi059.stdout:(10/131): ceph-selinux-18.0.0-2697.g2f5f6028.el 154 kB/s | 24 kB 00:00 2023-11-28T11:06:01.596 INFO:teuthology.orchestra.run.smithi059.stdout:(11/131): cephfs-mirror-18.0.0-2697.g2f5f6028.e 1.7 MB/s | 257 kB 00:00 2023-11-28T11:06:01.671 INFO:teuthology.orchestra.run.smithi059.stdout:(12/131): libcephfs-devel-18.0.0-2697.g2f5f6028 407 kB/s | 30 kB 00:00 2023-11-28T11:06:01.917 INFO:teuthology.orchestra.run.smithi028.stdout:(17/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 1.5 MB/s | 18 MB 00:11 2023-11-28T11:06:02.001 INFO:teuthology.orchestra.run.smithi028.stdout:(18/131): python3-ceph-argparse-18.0.0-2697.g2f 537 kB/s | 45 kB 00:00 2023-11-28T11:06:02.063 INFO:teuthology.orchestra.run.smithi059.stdout:(13/131): libcephfs2-18.0.0-2697.g2f5f6028.el8. 1.9 MB/s | 767 kB 00:00 2023-11-28T11:06:02.093 INFO:teuthology.orchestra.run.smithi028.stdout:(19/131): python3-ceph-common-18.0.0-2697.g2f5f 1.2 MB/s | 117 kB 00:00 2023-11-28T11:06:02.242 INFO:teuthology.orchestra.run.smithi059.stdout:(14/131): libcephsqlite-18.0.0-2697.g2f5f6028.e 772 kB/s | 137 kB 00:00 2023-11-28T11:06:02.243 INFO:teuthology.orchestra.run.smithi028.stdout:(20/131): python3-cephfs-18.0.0-2697.g2f5f6028. 1.2 MB/s | 190 kB 00:00 2023-11-28T11:06:02.380 INFO:teuthology.orchestra.run.smithi059.stdout:(15/131): librados-devel-18.0.0-2697.g2f5f6028. 1.0 MB/s | 139 kB 00:00 2023-11-28T11:06:02.485 INFO:teuthology.orchestra.run.smithi028.stdout:(21/131): python3-rados-18.0.0-2697.g2f5f6028.e 1.5 MB/s | 371 kB 00:00 2023-11-28T11:06:02.639 INFO:teuthology.orchestra.run.smithi059.stdout:(16/131): libradosstriper1-18.0.0-2697.g2f5f602 1.8 MB/s | 474 kB 00:00 2023-11-28T11:06:02.769 INFO:teuthology.orchestra.run.smithi028.stdout:(22/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 1.3 MB/s | 364 kB 00:00 2023-11-28T11:06:02.844 INFO:teuthology.orchestra.run.smithi028.stdout:(23/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 1.4 MB/s | 107 kB 00:00 2023-11-28T11:06:02.919 INFO:teuthology.orchestra.run.smithi028.stdout:(24/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.2 MB/s | 94 kB 00:00 2023-11-28T11:06:03.803 INFO:teuthology.orchestra.run.smithi028.stdout:(25/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 2.1 MB/s | 6.5 MB 00:03 2023-11-28T11:06:03.886 INFO:teuthology.orchestra.run.smithi028.stdout:(26/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 2.1 MB/s | 180 kB 00:00 2023-11-28T11:06:03.970 INFO:teuthology.orchestra.run.smithi028.stdout:(27/131): ceph-grafana-dashboards-18.0.0-2697.g 277 kB/s | 23 kB 00:00 2023-11-28T11:06:04.045 INFO:teuthology.orchestra.run.smithi028.stdout:(28/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 1.6 MB/s | 123 kB 00:00 2023-11-28T11:06:04.562 INFO:teuthology.orchestra.run.smithi028.stdout:(29/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 2.1 MB/s | 3.4 MB 00:01 2023-11-28T11:06:04.804 INFO:teuthology.orchestra.run.smithi028.stdout:(30/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 2.3 MB/s | 1.7 MB 00:00 2023-11-28T11:06:04.963 INFO:teuthology.orchestra.run.smithi028.stdout:(31/131): ceph-mgr-modules-core-18.0.0-2697.g2f 1.5 MB/s | 236 kB 00:00 2023-11-28T11:06:05.021 INFO:teuthology.orchestra.run.smithi028.stdout:(32/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 818 kB/s | 47 kB 00:00 2023-11-28T11:06:05.061 INFO:teuthology.orchestra.run.smithi059.stdout:(17/131): librgw2-18.0.0-2697.g2f5f6028.el8.x86 2.7 MB/s | 6.5 MB 00:02 2023-11-28T11:06:05.114 INFO:teuthology.orchestra.run.smithi028.stdout:(33/131): ceph-prometheus-alerts-18.0.0-2697.g2 158 kB/s | 15 kB 00:00 2023-11-28T11:06:05.132 INFO:teuthology.orchestra.run.smithi059.stdout:(18/131): python3-ceph-argparse-18.0.0-2697.g2f 642 kB/s | 45 kB 00:00 2023-11-28T11:06:05.207 INFO:teuthology.orchestra.run.smithi059.stdout:(19/131): python3-ceph-common-18.0.0-2697.g2f5f 1.5 MB/s | 117 kB 00:00 2023-11-28T11:06:05.239 INFO:teuthology.orchestra.run.smithi028.stdout:(34/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 2.1 MB/s | 263 kB 00:00 2023-11-28T11:06:05.290 INFO:teuthology.orchestra.run.smithi059.stdout:(20/131): python3-cephfs-18.0.0-2697.g2f5f6028. 2.2 MB/s | 190 kB 00:00 2023-11-28T11:06:05.297 INFO:teuthology.orchestra.run.smithi028.stdout:(35/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 1.4 MB/s | 83 kB 00:00 2023-11-28T11:06:05.373 INFO:teuthology.orchestra.run.smithi028.stdout:(36/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 241 kB/s | 18 kB 00:00 2023-11-28T11:06:05.460 INFO:teuthology.orchestra.run.smithi059.stdout:(21/131): ceph-osd-18.0.0-2697.g2f5f6028.el8.x8 1.5 MB/s | 18 MB 00:11 2023-11-28T11:06:05.482 INFO:teuthology.orchestra.run.smithi059.stdout:(22/131): python3-rados-18.0.0-2697.g2f5f6028.e 1.9 MB/s | 371 kB 00:00 2023-11-28T11:06:05.541 INFO:teuthology.orchestra.run.smithi059.stdout:(23/131): python3-rgw-18.0.0-2697.g2f5f6028.el8 1.8 MB/s | 107 kB 00:00 2023-11-28T11:06:05.566 INFO:teuthology.orchestra.run.smithi059.stdout:(24/131): python3-rbd-18.0.0-2697.g2f5f6028.el8 3.3 MB/s | 364 kB 00:00 2023-11-28T11:06:05.600 INFO:teuthology.orchestra.run.smithi059.stdout:(25/131): rbd-fuse-18.0.0-2697.g2f5f6028.el8.x8 1.6 MB/s | 94 kB 00:00 2023-11-28T11:06:05.700 INFO:teuthology.orchestra.run.smithi059.stdout:(26/131): rbd-nbd-18.0.0-2697.g2f5f6028.el8.x86 1.8 MB/s | 180 kB 00:00 2023-11-28T11:06:05.792 INFO:teuthology.orchestra.run.smithi059.stdout:(27/131): ceph-grafana-dashboards-18.0.0-2697.g 251 kB/s | 23 kB 00:00 2023-11-28T11:06:05.860 INFO:teuthology.orchestra.run.smithi059.stdout:(28/131): ceph-mgr-cephadm-18.0.0-2697.g2f5f602 1.8 MB/s | 123 kB 00:00 2023-11-28T11:06:06.115 INFO:teuthology.orchestra.run.smithi028.stdout:(37/131): gperftools-libs-2.7-9.el8.x86_64.rpm 413 kB/s | 306 kB 00:00 2023-11-28T11:06:06.248 INFO:teuthology.orchestra.run.smithi028.stdout:(38/131): liboath-2.6.2-3.el8.x86_64.rpm 442 kB/s | 59 kB 00:00 2023-11-28T11:06:06.251 INFO:teuthology.orchestra.run.smithi059.stdout:(29/131): ceph-mgr-dashboard-18.0.0-2697.g2f5f6 4.4 MB/s | 1.7 MB 00:00 2023-11-28T11:06:06.432 INFO:teuthology.orchestra.run.smithi028.stdout:(39/131): ceph-mgr-diskprediction-local-18.0.0- 4.0 MB/s | 7.4 MB 00:01 2023-11-28T11:06:06.449 INFO:teuthology.orchestra.run.smithi028.stdout:(40/131): libunwind-1.3.1-3.el8.x86_64.rpm 375 kB/s | 75 kB 00:00 2023-11-28T11:06:06.485 INFO:teuthology.orchestra.run.smithi059.stdout:(30/131): rbd-mirror-18.0.0-2697.g2f5f6028.el8. 3.8 MB/s | 3.4 MB 00:00 2023-11-28T11:06:06.552 INFO:teuthology.orchestra.run.smithi059.stdout:(31/131): ceph-mgr-modules-core-18.0.0-2697.g2f 3.5 MB/s | 236 kB 00:00 2023-11-28T11:06:06.574 INFO:teuthology.orchestra.run.smithi028.stdout:(41/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 349 kB/s | 44 kB 00:00 2023-11-28T11:06:06.594 INFO:teuthology.orchestra.run.smithi059.stdout:(32/131): ceph-mgr-rook-18.0.0-2697.g2f5f6028.e 1.1 MB/s | 47 kB 00:00 2023-11-28T11:06:06.644 INFO:teuthology.orchestra.run.smithi059.stdout:(33/131): ceph-prometheus-alerts-18.0.0-2697.g2 291 kB/s | 15 kB 00:00 2023-11-28T11:06:06.728 INFO:teuthology.orchestra.run.smithi059.stdout:(34/131): ceph-volume-18.0.0-2697.g2f5f6028.el8 3.1 MB/s | 263 kB 00:00 2023-11-28T11:06:06.733 INFO:teuthology.orchestra.run.smithi028.stdout:(42/131): python3-beautifulsoup4-4.6.3-2.el8.1. 1.1 MB/s | 185 kB 00:00 2023-11-28T11:06:06.778 INFO:teuthology.orchestra.run.smithi059.stdout:(35/131): cephadm-18.0.0-2697.g2f5f6028.el8.noa 1.6 MB/s | 83 kB 00:00 2023-11-28T11:06:06.828 INFO:teuthology.orchestra.run.smithi059.stdout:(36/131): cephfs-top-18.0.0-2697.g2f5f6028.el8. 361 kB/s | 18 kB 00:00 2023-11-28T11:06:06.858 INFO:teuthology.orchestra.run.smithi028.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 264 kB/s | 33 kB 00:00 2023-11-28T11:06:07.275 INFO:teuthology.orchestra.run.smithi028.stdout:(44/131): ceph-test-18.0.0-2697.g2f5f6028.el8.x 3.6 MB/s | 47 MB 00:12 2023-11-28T11:06:07.300 INFO:teuthology.orchestra.run.smithi028.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 37 kB/s | 16 kB 00:00 2023-11-28T11:06:07.312 INFO:teuthology.orchestra.run.smithi059.stdout:(37/131): ceph-mgr-diskprediction-local-18.0.0- 7.0 MB/s | 7.4 MB 00:01 2023-11-28T11:06:07.338 INFO:teuthology.orchestra.run.smithi059.stdout:(38/131): gperftools-libs-2.7-9.el8.x86_64.rpm 602 kB/s | 306 kB 00:00 2023-11-28T11:06:07.429 INFO:teuthology.orchestra.run.smithi059.stdout:(39/131): libunwind-1.3.1-3.el8.x86_64.rpm 821 kB/s | 75 kB 00:00 2023-11-28T11:06:07.509 INFO:teuthology.orchestra.run.smithi028.stdout:(46/131): python3-cherrypy-18.4.0-1.el8.noarch. 1.8 MB/s | 384 kB 00:00 2023-11-28T11:06:07.534 INFO:teuthology.orchestra.run.smithi028.stdout:(47/131): python3-asyncssh-2.7.0-2.el8.noarch.r 432 kB/s | 476 kB 00:01 2023-11-28T11:06:07.613 INFO:teuthology.orchestra.run.smithi059.stdout:(40/131): python3-asyncssh-2.7.0-2.el8.noarch.r 2.5 MB/s | 476 kB 00:00 2023-11-28T11:06:07.638 INFO:teuthology.orchestra.run.smithi059.stdout:(41/131): liboath-2.6.2-3.el8.x86_64.rpm 180 kB/s | 59 kB 00:00 2023-11-28T11:06:07.643 INFO:teuthology.orchestra.run.smithi028.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 608 kB/s | 81 kB 00:00 2023-11-28T11:06:07.668 INFO:teuthology.orchestra.run.smithi028.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 83 kB/s | 11 kB 00:00 2023-11-28T11:06:07.705 INFO:teuthology.orchestra.run.smithi059.stdout:(42/131): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 475 kB/s | 44 kB 00:00 2023-11-28T11:06:07.769 INFO:teuthology.orchestra.run.smithi028.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 140 kB/s | 18 kB 00:00 2023-11-28T11:06:07.806 INFO:teuthology.orchestra.run.smithi059.stdout:(43/131): python3-cachetools-3.1.1-4.el8.noarch 330 kB/s | 33 kB 00:00 2023-11-28T11:06:07.831 INFO:teuthology.orchestra.run.smithi059.stdout:(44/131): python3-beautifulsoup4-4.6.3-2.el8.1. 965 kB/s | 185 kB 00:00 2023-11-28T11:06:07.902 INFO:teuthology.orchestra.run.smithi028.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 366 kB/s | 49 kB 00:00 2023-11-28T11:06:07.906 INFO:teuthology.orchestra.run.smithi059.stdout:(45/131): python3-certifi-2018.10.15-7.el8.noar 163 kB/s | 16 kB 00:00 2023-11-28T11:06:07.923 INFO:teuthology.orchestra.run.smithi059.stdout:(46/131): python3-cheroot-8.5.2-1.el8.noarch.rp 1.8 MB/s | 173 kB 00:00 2023-11-28T11:06:07.927 INFO:teuthology.orchestra.run.smithi028.stdout:(52/131): python3-cheroot-8.5.2-1.el8.noarch.rp 265 kB/s | 173 kB 00:00 2023-11-28T11:06:08.007 INFO:teuthology.orchestra.run.smithi059.stdout:(47/131): python3-cherrypy-18.4.0-1.el8.noarch. 3.8 MB/s | 384 kB 00:00 2023-11-28T11:06:08.032 INFO:teuthology.orchestra.run.smithi059.stdout:(48/131): python3-google-auth-1.1.1-10.el8.noar 748 kB/s | 81 kB 00:00 2023-11-28T11:06:08.044 INFO:teuthology.orchestra.run.smithi028.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 413 kB/s | 59 kB 00:00 2023-11-28T11:06:08.086 INFO:teuthology.orchestra.run.smithi028.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 380 kB/s | 60 kB 00:00 2023-11-28T11:06:08.090 INFO:teuthology.orchestra.run.smithi059.stdout:(49/131): python3-jaraco-6.2-6.el8.noarch.rpm 133 kB/s | 11 kB 00:00 2023-11-28T11:06:08.116 INFO:teuthology.orchestra.run.smithi059.stdout:(50/131): python3-jaraco-functools-2.0-4.el8.no 211 kB/s | 18 kB 00:00 2023-11-28T11:06:08.212 INFO:teuthology.orchestra.run.smithi028.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 1.7 MB/s | 283 kB 00:00 2023-11-28T11:06:08.216 INFO:teuthology.orchestra.run.smithi059.stdout:(51/131): python3-logutils-0.3.5-11.el8.noarch. 487 kB/s | 49 kB 00:00 2023-11-28T11:06:08.228 INFO:teuthology.orchestra.run.smithi028.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 110 kB/s | 16 kB 00:00 2023-11-28T11:06:08.267 INFO:teuthology.orchestra.run.smithi059.stdout:(52/131): python3-kubernetes-11.0.0-6.el8.noarc 6.6 MB/s | 1.2 MB 00:00 2023-11-28T11:06:08.300 INFO:teuthology.orchestra.run.smithi059.stdout:(53/131): python3-more-itertools-7.2.0-3.el8.no 702 kB/s | 59 kB 00:00 2023-11-28T11:06:08.337 INFO:teuthology.orchestra.run.smithi028.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 268 kB/s | 34 kB 00:00 2023-11-28T11:06:08.360 INFO:teuthology.orchestra.run.smithi059.stdout:(54/131): python3-natsort-7.1.1-2.el8.noarch.rp 656 kB/s | 60 kB 00:00 2023-11-28T11:06:08.401 INFO:teuthology.orchestra.run.smithi059.stdout:(55/131): python3-pecan-1.3.2-9.el8.noarch.rpm 2.7 MB/s | 283 kB 00:00 2023-11-28T11:06:08.404 INFO:teuthology.orchestra.run.smithi028.stdout:(58/131): python3-kubernetes-11.0.0-6.el8.noarc 1.6 MB/s | 1.2 MB 00:00 2023-11-28T11:06:08.429 INFO:teuthology.orchestra.run.smithi028.stdout:(59/131): python3-routes-2.4.1-12.el8.noarch.rp 980 kB/s | 196 kB 00:00 2023-11-28T11:06:08.443 INFO:teuthology.orchestra.run.smithi059.stdout:(56/131): python3-portend-2.6-1.el8.noarch.rpm 186 kB/s | 16 kB 00:00 2023-11-28T11:06:08.471 INFO:teuthology.orchestra.run.smithi028.stdout:(60/131): python3-rsa-4.9-2.el8.noarch.rpm 480 kB/s | 64 kB 00:00 2023-11-28T11:06:08.485 INFO:teuthology.orchestra.run.smithi059.stdout:(57/131): python3-repoze-lru-0.7-6.el8.noarch.r 402 kB/s | 34 kB 00:00 2023-11-28T11:06:08.522 INFO:teuthology.orchestra.run.smithi028.stdout:(61/131): python3-simplegeneric-0.8.1-17.el8.no 160 kB/s | 19 kB 00:00 2023-11-28T11:06:08.543 INFO:teuthology.orchestra.run.smithi059.stdout:(58/131): python3-routes-2.4.1-12.el8.noarch.rp 1.9 MB/s | 196 kB 00:00 2023-11-28T11:06:08.547 INFO:teuthology.orchestra.run.smithi028.stdout:(62/131): python3-singledispatch-3.4.0.3-18.el8 204 kB/s | 24 kB 00:00 2023-11-28T11:06:08.589 INFO:teuthology.orchestra.run.smithi028.stdout:(63/131): python3-tempora-1.14.1-5.el8.noarch.r 249 kB/s | 29 kB 00:00 2023-11-28T11:06:08.602 INFO:teuthology.orchestra.run.smithi059.stdout:(59/131): python3-rsa-4.9-2.el8.noarch.rpm 549 kB/s | 64 kB 00:00 2023-11-28T11:06:08.636 INFO:teuthology.orchestra.run.smithi059.stdout:(60/131): python3-simplegeneric-0.8.1-17.el8.no 203 kB/s | 19 kB 00:00 2023-11-28T11:06:08.639 INFO:teuthology.orchestra.run.smithi028.stdout:(64/131): python3-trustme-0.6.0-4.el8.noarch.rp 235 kB/s | 27 kB 00:00 2023-11-28T11:06:08.695 INFO:teuthology.orchestra.run.smithi059.stdout:(61/131): python3-singledispatch-3.4.0.3-18.el8 261 kB/s | 24 kB 00:00 2023-11-28T11:06:08.728 INFO:teuthology.orchestra.run.smithi059.stdout:(62/131): python3-tempora-1.14.1-5.el8.noarch.r 318 kB/s | 29 kB 00:00 2023-11-28T11:06:08.748 INFO:teuthology.orchestra.run.smithi028.stdout:(65/131): python3-waitress-1.4.3-1.el8.noarch.r 1.2 MB/s | 255 kB 00:00 2023-11-28T11:06:08.787 INFO:teuthology.orchestra.run.smithi059.stdout:(63/131): python3-trustme-0.6.0-4.el8.noarch.rp 299 kB/s | 27 kB 00:00 2023-11-28T11:06:08.798 INFO:teuthology.orchestra.run.smithi028.stdout:(66/131): python3-webob-1.8.5-1.el8.1.noarch.rp 1.2 MB/s | 251 kB 00:00 2023-11-28T11:06:08.815 INFO:teuthology.orchestra.run.smithi028.stdout:(67/131): python3-websocket-client-0.56.0-5.el8 349 kB/s | 61 kB 00:00 2023-11-28T11:06:08.820 INFO:teuthology.orchestra.run.smithi059.stdout:(64/131): python3-waitress-1.4.3-1.el8.noarch.r 2.7 MB/s | 255 kB 00:00 2023-11-28T11:06:08.887 INFO:teuthology.orchestra.run.smithi059.stdout:(65/131): python3-webob-1.8.5-1.el8.1.noarch.rp 2.5 MB/s | 251 kB 00:00 2023-11-28T11:06:08.890 INFO:teuthology.orchestra.run.smithi028.stdout:(68/131): python3-webtest-2.0.33-1.el8.noarch.r 601 kB/s | 85 kB 00:00 2023-11-28T11:06:08.912 INFO:teuthology.orchestra.run.smithi059.stdout:(66/131): python3-websocket-client-0.56.0-5.el8 665 kB/s | 61 kB 00:00 2023-11-28T11:06:08.915 INFO:teuthology.orchestra.run.smithi028.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-28T11:06:08.941 INFO:teuthology.orchestra.run.smithi028.stdout:(70/131): python3-zc-lockfile-2.0-2.el8.noarch. 161 kB/s | 23 kB 00:00 2023-11-28T11:06:08.979 INFO:teuthology.orchestra.run.smithi059.stdout:(67/131): python3-webtest-2.0.33-1.el8.noarch.r 928 kB/s | 85 kB 00:00 2023-11-28T11:06:09.005 INFO:teuthology.orchestra.run.smithi059.stdout:(68/131): python3-zc-lockfile-2.0-2.el8.noarch. 249 kB/s | 23 kB 00:00 2023-11-28T11:06:09.016 INFO:teuthology.orchestra.run.smithi028.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.4 MB/s | 110 kB 00:00 2023-11-28T11:06:09.030 INFO:teuthology.orchestra.run.smithi059.stdout:(69/131): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.7 MB/s | 70 kB 00:00 2023-11-28T11:06:09.058 INFO:teuthology.orchestra.run.smithi028.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.3 MB/s | 54 kB 00:00 2023-11-28T11:06:09.108 INFO:teuthology.orchestra.run.smithi028.stdout:(73/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 5.1 MB/s | 259 kB 00:00 2023-11-28T11:06:09.161 INFO:teuthology.orchestra.run.smithi028.stdout:(74/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-11-28T11:06:09.201 INFO:teuthology.orchestra.run.smithi028.stdout:(75/131): python3-markupsafe-0.23-19.el8.x86_64 929 kB/s | 39 kB 00:00 2023-11-28T11:06:09.238 INFO:teuthology.orchestra.run.smithi059.stdout:(70/131): thrift-0.13.0-2.el8.x86_64.rpm 6.7 MB/s | 1.7 MB 00:00 2023-11-28T11:06:09.251 INFO:teuthology.orchestra.run.smithi028.stdout:(76/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-11-28T11:06:09.301 INFO:teuthology.orchestra.run.smithi028.stdout:(77/131): python3-jinja2-2.10.1-3.el8.noarch.rp 11 MB/s | 538 kB 00:00 2023-11-28T11:06:09.314 INFO:teuthology.orchestra.run.smithi059.stdout:(71/131): python3-pyasn1-modules-0.3.7-6.el8.no 1.4 MB/s | 110 kB 00:00 2023-11-28T11:06:09.343 INFO:teuthology.orchestra.run.smithi028.stdout:(78/131): python3-mako-1.0.6-13.el8.noarch.rpm 3.7 MB/s | 157 kB 00:00 2023-11-28T11:06:09.347 INFO:teuthology.orchestra.run.smithi059.stdout:(72/131): python3-pytz-2017.2-9.el8.noarch.rpm 1.6 MB/s | 54 kB 00:00 2023-11-28T11:06:09.385 INFO:teuthology.orchestra.run.smithi028.stdout:(79/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-11-28T11:06:09.436 INFO:teuthology.orchestra.run.smithi028.stdout:(80/131): automake-1.16.1-7.el8.noarch.rpm 14 MB/s | 713 kB 00:00 2023-11-28T11:06:09.498 INFO:teuthology.orchestra.run.smithi059.stdout:(73/131): python3-scipy-1.0.0-21.module+el8.5.0 29 MB/s | 14 MB 00:00 2023-11-28T11:06:09.523 INFO:teuthology.orchestra.run.smithi059.stdout:(74/131): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.4 MB/s | 259 kB 00:00 2023-11-28T11:06:09.548 INFO:teuthology.orchestra.run.smithi059.stdout:(75/131): socat-1.7.4.1-1.el8.x86_64.rpm 6.3 MB/s | 323 kB 00:00 2023-11-28T11:06:09.565 INFO:teuthology.orchestra.run.smithi059.stdout:(76/131): python3-markupsafe-0.23-19.el8.x86_64 927 kB/s | 39 kB 00:00 2023-11-28T11:06:09.594 INFO:teuthology.orchestra.run.smithi028.stdout:(81/131): python3-scipy-1.0.0-21.module+el8.5.0 20 MB/s | 14 MB 00:00 2023-11-28T11:06:09.599 INFO:teuthology.orchestra.run.smithi059.stdout:(77/131): bison-3.0.4-10.el8.x86_64.rpm 13 MB/s | 688 kB 00:00 2023-11-28T11:06:09.633 INFO:teuthology.orchestra.run.smithi059.stdout:(78/131): python3-jinja2-2.10.1-3.el8.noarch.rp 7.8 MB/s | 538 kB 00:00 2023-11-28T11:06:09.636 INFO:teuthology.orchestra.run.smithi028.stdout:(82/131): thrift-0.13.0-2.el8.x86_64.rpm 2.1 MB/s | 1.7 MB 00:00 2023-11-28T11:06:09.658 INFO:teuthology.orchestra.run.smithi059.stdout:(79/131): python3-mako-1.0.6-13.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-11-28T11:06:09.661 INFO:teuthology.orchestra.run.smithi028.stdout:(83/131): python3-jsonpatch-1.21-2.el8.noarch.r 398 kB/s | 27 kB 00:00 2023-11-28T11:06:09.675 INFO:teuthology.orchestra.run.smithi059.stdout:(80/131): oniguruma-6.8.2-2.el8.x86_64.rpm 4.4 MB/s | 187 kB 00:00 2023-11-28T11:06:09.703 INFO:teuthology.orchestra.run.smithi028.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 1.3 MB/s | 44 kB 00:00 2023-11-28T11:06:09.717 INFO:teuthology.orchestra.run.smithi059.stdout:(81/131): automake-1.16.1-7.el8.noarch.rpm 12 MB/s | 713 kB 00:00 2023-11-28T11:06:09.737 INFO:teuthology.orchestra.run.smithi028.stdout:(85/131): flex-2.6.1-9.el8.x86_64.rpm 3.1 MB/s | 320 kB 00:00 2023-11-28T11:06:09.750 INFO:teuthology.orchestra.run.smithi059.stdout:(82/131): python3-jsonpatch-1.21-2.el8.noarch.r 794 kB/s | 27 kB 00:00 2023-11-28T11:06:09.787 INFO:teuthology.orchestra.run.smithi028.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 8.9 MB/s | 457 kB 00:00 2023-11-28T11:06:09.792 INFO:teuthology.orchestra.run.smithi059.stdout:(83/131): flex-2.6.1-9.el8.x86_64.rpm 7.5 MB/s | 320 kB 00:00 2023-11-28T11:06:09.829 INFO:teuthology.orchestra.run.smithi028.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.3 MB/s | 225 kB 00:00 2023-11-28T11:06:09.843 INFO:teuthology.orchestra.run.smithi059.stdout:(84/131): python3-prettytable-0.7.2-14.el8.noar 876 kB/s | 44 kB 00:00 2023-11-28T11:06:09.863 INFO:teuthology.orchestra.run.smithi028.stdout:(88/131): uuid-1.6.2-43.el8.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2023-11-28T11:06:09.918 INFO:teuthology.orchestra.run.smithi059.stdout:(85/131): python3-babel-2.5.1-7.el8.noarch.rpm 20 MB/s | 4.8 MB 00:00 2023-11-28T11:06:09.930 INFO:teuthology.orchestra.run.smithi028.stdout:(89/131): python3-babel-2.5.1-7.el8.noarch.rpm 9.7 MB/s | 4.8 MB 00:00 2023-11-28T11:06:09.955 INFO:teuthology.orchestra.run.smithi028.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.3 MB/s | 126 kB 00:00 2023-11-28T11:06:09.981 INFO:teuthology.orchestra.run.smithi028.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 401 kB/s | 20 kB 00:00 2023-11-28T11:06:09.993 INFO:teuthology.orchestra.run.smithi059.stdout:(86/131): python3-werkzeug-0.12.2-4.el8.noarch. 6.0 MB/s | 457 kB 00:00 2023-11-28T11:06:10.005 INFO:teuthology.orchestra.run.smithi028.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-11-28T11:06:10.035 INFO:teuthology.orchestra.run.smithi059.stdout:(87/131): python3-numpy-f2py-1.14.3-10.el8.x86_ 5.2 MB/s | 225 kB 00:00 2023-11-28T11:06:10.064 INFO:teuthology.orchestra.run.smithi028.stdout:(93/131): python3-numpy-1.14.3-10.el8.x86_64.rp 10 MB/s | 3.7 MB 00:00 2023-11-28T11:06:10.098 INFO:teuthology.orchestra.run.smithi028.stdout:(94/131): libtool-2.4.6-25.el8.x86_64.rpm 5.9 MB/s | 709 kB 00:00 2023-11-28T11:06:10.102 INFO:teuthology.orchestra.run.smithi059.stdout:(88/131): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2023-11-28T11:06:10.123 INFO:teuthology.orchestra.run.smithi028.stdout:(95/131): libpq-13.5-1.el8.x86_64.rpm 3.3 MB/s | 198 kB 00:00 2023-11-28T11:06:10.127 INFO:teuthology.orchestra.run.smithi059.stdout:(89/131): uuid-1.6.2-43.el8.x86_64.rpm 696 kB/s | 64 kB 00:00 2023-11-28T11:06:10.153 INFO:teuthology.orchestra.run.smithi059.stdout:(90/131): python3-pyasn1-0.3.7-6.el8.noarch.rpm 2.4 MB/s | 126 kB 00:00 2023-11-28T11:06:10.165 INFO:teuthology.orchestra.run.smithi028.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.7 MB/s | 202 kB 00:00 2023-11-28T11:06:10.178 INFO:teuthology.orchestra.run.smithi059.stdout:(91/131): python3-jsonpointer-1.10-11.el8.noarc 399 kB/s | 20 kB 00:00 2023-11-28T11:06:10.203 INFO:teuthology.orchestra.run.smithi059.stdout:(92/131): python3-pyOpenSSL-19.0.0-1.el8.noarch 2.0 MB/s | 103 kB 00:00 2023-11-28T11:06:10.207 INFO:teuthology.orchestra.run.smithi028.stdout:(97/131): librdkafka-0.11.4-3.el8.x86_64.rpm 8.3 MB/s | 354 kB 00:00 2023-11-28T11:06:10.287 INFO:teuthology.orchestra.run.smithi059.stdout:(93/131): libtool-2.4.6-25.el8.x86_64.rpm 6.4 MB/s | 709 kB 00:00 2023-11-28T11:06:10.321 INFO:teuthology.orchestra.run.smithi028.stdout:(98/131): openblas-threads-0.3.15-3.el8.x86_64. 15 MB/s | 4.9 MB 00:00 2023-11-28T11:06:10.329 INFO:teuthology.orchestra.run.smithi059.stdout:(94/131): libpq-13.5-1.el8.x86_64.rpm 4.6 MB/s | 198 kB 00:00 2023-11-28T11:06:10.399 INFO:teuthology.orchestra.run.smithi028.stdout:(99/131): openblas-0.3.15-3.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-11-28T11:06:10.446 INFO:teuthology.orchestra.run.smithi059.stdout:(95/131): openblas-threads-0.3.15-3.el8.x86_64. 20 MB/s | 4.9 MB 00:00 2023-11-28T11:06:10.449 INFO:teuthology.orchestra.run.smithi028.stdout:(100/131): postgresql-contrib-10.21-2.module+el 6.2 MB/s | 808 kB 00:00 2023-11-28T11:06:10.483 INFO:teuthology.orchestra.run.smithi028.stdout:(101/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2023-11-28T11:06:10.488 INFO:teuthology.orchestra.run.smithi059.stdout:(96/131): jq-1.6-3.el8.x86_64.rpm 4.8 MB/s | 202 kB 00:00 2023-11-28T11:06:10.525 INFO:teuthology.orchestra.run.smithi028.stdout:(102/131): postgresql-10.21-2.module+el8.6.0+15 12 MB/s | 1.5 MB 00:00 2023-11-28T11:06:10.550 INFO:teuthology.orchestra.run.smithi028.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 2.9 MB/s | 200 kB 00:00 2023-11-28T11:06:10.555 INFO:teuthology.orchestra.run.smithi059.stdout:(97/131): openblas-0.3.15-3.el8.x86_64.rpm 21 MB/s | 4.7 MB 00:00 2023-11-28T11:06:10.576 INFO:teuthology.orchestra.run.smithi028.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 845 kB/s | 43 kB 00:00 2023-11-28T11:06:10.580 INFO:teuthology.orchestra.run.smithi059.stdout:(98/131): librdkafka-0.11.4-3.el8.x86_64.rpm 3.8 MB/s | 354 kB 00:00 2023-11-28T11:06:10.601 INFO:teuthology.orchestra.run.smithi028.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-11-28T11:06:10.627 INFO:teuthology.orchestra.run.smithi028.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 954 kB/s | 48 kB 00:00 2023-11-28T11:06:10.639 INFO:teuthology.orchestra.run.smithi059.stdout:(99/131): postgresql-contrib-10.21-2.module+el8 14 MB/s | 808 kB 00:00 2023-11-28T11:06:10.652 INFO:teuthology.orchestra.run.smithi028.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 1.7 MB/s | 87 kB 00:00 2023-11-28T11:06:10.677 INFO:teuthology.orchestra.run.smithi028.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 852 kB/s | 43 kB 00:00 2023-11-28T11:06:10.702 INFO:teuthology.orchestra.run.smithi028.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-11-28T11:06:10.722 INFO:teuthology.orchestra.run.smithi059.stdout:(100/131): postgresql-10.21-2.module+el8.6.0+15 18 MB/s | 1.5 MB 00:00 2023-11-28T11:06:10.728 INFO:teuthology.orchestra.run.smithi028.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-11-28T11:06:10.789 INFO:teuthology.orchestra.run.smithi059.stdout:(101/131): postgresql-server-10.21-2.module+el8 22 MB/s | 5.1 MB 00:00 2023-11-28T11:06:10.803 INFO:teuthology.orchestra.run.smithi028.stdout:(111/131): postgresql-server-10.21-2.module+el8 8.5 MB/s | 5.1 MB 00:00 2023-11-28T11:06:10.815 INFO:teuthology.orchestra.run.smithi059.stdout:(102/131): librabbitmq-0.9.0-3.el8.x86_64.rpm 508 kB/s | 47 kB 00:00 2023-11-28T11:06:10.828 INFO:teuthology.orchestra.run.smithi028.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 187 kB/s | 19 kB 00:00 2023-11-28T11:06:10.840 INFO:teuthology.orchestra.run.smithi059.stdout:(103/131): libselinux-devel-2.9-5.el8.x86_64.rp 3.9 MB/s | 200 kB 00:00 2023-11-28T11:06:10.854 INFO:teuthology.orchestra.run.smithi028.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 3.0 MB/s | 155 kB 00:00 2023-11-28T11:06:10.865 INFO:teuthology.orchestra.run.smithi059.stdout:(104/131): python3-jwt-1.6.1-2.el8.noarch.rpm 849 kB/s | 43 kB 00:00 2023-11-28T11:06:10.879 INFO:teuthology.orchestra.run.smithi028.stdout:(114/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 3.1 MB/s | 560 kB 00:00 2023-11-28T11:06:10.891 INFO:teuthology.orchestra.run.smithi059.stdout:(105/131): pcre2-utf32-10.32-2.el8.x86_64.rpm 4.3 MB/s | 220 kB 00:00 2023-11-28T11:06:10.904 INFO:teuthology.orchestra.run.smithi028.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 357 kB/s | 18 kB 00:00 2023-11-28T11:06:10.916 INFO:teuthology.orchestra.run.smithi059.stdout:(106/131): keyutils-libs-devel-1.5.10-9.el8.x86 956 kB/s | 48 kB 00:00 2023-11-28T11:06:10.933 INFO:teuthology.orchestra.run.smithi059.stdout:(107/131): libsepol-devel-2.9-3.el8.x86_64.rpm 2.0 MB/s | 87 kB 00:00 2023-11-28T11:06:10.946 INFO:teuthology.orchestra.run.smithi028.stdout:(116/131): python3-cryptography-3.2.1-5.el8.x86 4.6 MB/s | 559 kB 00:00 2023-11-28T11:06:10.958 INFO:teuthology.orchestra.run.smithi059.stdout:(108/131): python3-requests-oauthlib-1.0.0-1.el 1.0 MB/s | 43 kB 00:00 2023-11-28T11:06:10.979 INFO:teuthology.orchestra.run.smithi028.stdout:(117/131): pcre2-devel-10.32-2.el8.x86_64.rpm 5.9 MB/s | 605 kB 00:00 2023-11-28T11:06:10.983 INFO:teuthology.orchestra.run.smithi059.stdout:(109/131): python3-pycparser-2.14-14.el8.noarch 2.1 MB/s | 109 kB 00:00 2023-11-28T11:06:11.006 INFO:teuthology.orchestra.run.smithi028.stdout:(118/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 2.3 MB/s | 238 kB 00:00 2023-11-28T11:06:11.008 INFO:teuthology.orchestra.run.smithi059.stdout:(110/131): pcre2-utf16-10.32-2.el8.x86_64.rpm 4.4 MB/s | 229 kB 00:00 2023-11-28T11:06:11.030 INFO:teuthology.orchestra.run.smithi028.stdout:(119/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.4 MB/s | 72 kB 00:00 2023-11-28T11:06:11.042 INFO:teuthology.orchestra.run.smithi059.stdout:(111/131): krb5-devel-1.18.2-14.el8.x86_64.rpm 9.3 MB/s | 560 kB 00:00 2023-11-28T11:06:11.055 INFO:teuthology.orchestra.run.smithi028.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 1.6 MB/s | 83 kB 00:00 2023-11-28T11:06:11.068 INFO:teuthology.orchestra.run.smithi059.stdout:(112/131): libaio-devel-0.3.112-1.el8.x86_64.rp 318 kB/s | 19 kB 00:00 2023-11-28T11:06:11.097 INFO:teuthology.orchestra.run.smithi028.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 894 kB/s | 60 kB 00:00 2023-11-28T11:06:11.101 INFO:teuthology.orchestra.run.smithi059.stdout:(113/131): python3-oauthlib-2.1.0-1.el8.noarch. 2.6 MB/s | 155 kB 00:00 2023-11-28T11:06:11.123 INFO:teuthology.orchestra.run.smithi028.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 1.4 MB/s | 98 kB 00:00 2023-11-28T11:06:11.135 INFO:teuthology.orchestra.run.smithi059.stdout:(114/131): python3-cryptography-3.2.1-5.el8.x86 8.2 MB/s | 559 kB 00:00 2023-11-28T11:06:11.148 INFO:teuthology.orchestra.run.smithi028.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 1.1 MB/s | 58 kB 00:00 2023-11-28T11:06:11.160 INFO:teuthology.orchestra.run.smithi059.stdout:(115/131): libverto-devel-0.3.0-5.el8.x86_64.rp 306 kB/s | 18 kB 00:00 2023-11-28T11:06:11.173 INFO:teuthology.orchestra.run.smithi028.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 767 kB/s | 39 kB 00:00 2023-11-28T11:06:11.194 INFO:teuthology.orchestra.run.smithi059.stdout:(116/131): pcre2-devel-10.32-2.el8.x86_64.rpm 10 MB/s | 605 kB 00:00 2023-11-28T11:06:11.220 INFO:teuthology.orchestra.run.smithi059.stdout:(117/131): python3-cffi-1.11.5-5.el8.x86_64.rpm 4.0 MB/s | 238 kB 00:00 2023-11-28T11:06:11.223 INFO:teuthology.orchestra.run.smithi028.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 8.4 MB/s | 644 kB 00:00 2023-11-28T11:06:11.240 INFO:teuthology.orchestra.run.smithi028.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.5 MB/s | 171 kB 00:00 2023-11-28T11:06:11.261 INFO:teuthology.orchestra.run.smithi059.stdout:(118/131): xfsprogs-devel-5.0.0-10.el8.x86_64.r 1.7 MB/s | 72 kB 00:00 2023-11-28T11:06:11.266 INFO:teuthology.orchestra.run.smithi028.stdout:(127/131): NetworkManager-initscripts-updown-1. 3.2 MB/s | 138 kB 00:00 2023-11-28T11:06:11.320 INFO:teuthology.orchestra.run.smithi059.stdout:(119/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 19 MB/s | 2.3 MB 00:00 2023-11-28T11:06:11.324 INFO:teuthology.orchestra.run.smithi028.stdout:(128/131): openssl-devel-1.1.1k-7.el8_6.x86_64. 6.1 MB/s | 2.3 MB 00:00 2023-11-28T11:06:11.345 INFO:teuthology.orchestra.run.smithi059.stdout:(120/131): fuse-2.9.7-15.el8.x86_64.rpm 992 kB/s | 83 kB 00:00 2023-11-28T11:06:11.370 INFO:teuthology.orchestra.run.smithi059.stdout:(121/131): elfutils-libelf-devel-0.186-1.el8.x8 1.2 MB/s | 60 kB 00:00 2023-11-28T11:06:11.374 INFO:teuthology.orchestra.run.smithi028.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 16 MB/s | 798 kB 00:00 2023-11-28T11:06:11.529 INFO:teuthology.orchestra.run.smithi059.stdout:(122/131): libuuid-devel-2.32.1-35.el8.x86_64.r 534 kB/s | 98 kB 00:00 2023-11-28T11:06:11.554 INFO:teuthology.orchestra.run.smithi059.stdout:(123/131): zlib-devel-1.2.11-18.el8_5.x86_64.rp 314 kB/s | 58 kB 00:00 2023-11-28T11:06:11.580 INFO:teuthology.orchestra.run.smithi059.stdout:(124/131): libcom_err-devel-1.45.6-4.el8.x86_64 770 kB/s | 39 kB 00:00 2023-11-28T11:06:11.613 INFO:teuthology.orchestra.run.smithi059.stdout:(125/131): libgfortran-8.5.0-10.1.el8_6.x86_64. 11 MB/s | 644 kB 00:00 2023-11-28T11:06:11.639 INFO:teuthology.orchestra.run.smithi059.stdout:(126/131): libquadmath-8.5.0-10.1.el8_6.x86_64. 2.9 MB/s | 171 kB 00:00 2023-11-28T11:06:11.664 INFO:teuthology.orchestra.run.smithi059.stdout:(127/131): NetworkManager-initscripts-updown-1. 2.7 MB/s | 138 kB 00:00 2023-11-28T11:06:11.800 INFO:teuthology.orchestra.run.smithi028.stdout:(130/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 6.7 MB/s | 3.6 MB 00:00 2023-11-28T11:06:12.139 INFO:teuthology.orchestra.run.smithi059.stdout:(128/131): librados2-18.0.0-2697.g2f5f6028.el8. 7.3 MB/s | 3.7 MB 00:00 2023-11-28T11:06:12.198 INFO:teuthology.orchestra.run.smithi059.stdout:(129/131): iproute-5.15.0-4.el8_6.1.x86_64.rpm 13 MB/s | 798 kB 00:00 2023-11-28T11:06:12.315 INFO:teuthology.orchestra.run.smithi059.stdout:(130/131): librbd1-18.0.0-2697.g2f5f6028.el8.x8 5.5 MB/s | 3.6 MB 00:00 2023-11-28T11:06:12.342 INFO:teuthology.orchestra.run.smithi028.stdout:(131/131): librados2-18.0.0-2697.g2f5f6028.el8. 3.3 MB/s | 3.7 MB 00:01 2023-11-28T11:06:12.347 INFO:teuthology.orchestra.run.smithi028.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:06:12.347 INFO:teuthology.orchestra.run.smithi028.stdout:Total 7.2 MB/s | 203 MB 00:28 2023-11-28T11:06:13.218 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction check 2023-11-28T11:06:13.350 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction check succeeded. 2023-11-28T11:06:13.350 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction test 2023-11-28T11:06:14.514 INFO:teuthology.orchestra.run.smithi028.stdout:Transaction test succeeded. 2023-11-28T11:06:14.518 INFO:teuthology.orchestra.run.smithi028.stdout:Running transaction 2023-11-28T11:06:15.940 INFO:teuthology.orchestra.run.smithi059.stdout:(131/131): ceph-test-18.0.0-2697.g2f5f6028.el8. 3.2 MB/s | 47 MB 00:14 2023-11-28T11:06:15.943 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2023-11-28T11:06:15.943 INFO:teuthology.orchestra.run.smithi059.stdout:Total 7.2 MB/s | 203 MB 00:28 2023-11-28T11:06:16.790 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2023-11-28T11:06:16.922 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2023-11-28T11:06:16.922 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2023-11-28T11:06:18.096 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2023-11-28T11:06:18.100 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2023-11-28T11:06:19.486 INFO:teuthology.orchestra.run.smithi028.stdout: Preparing : 1/1 2023-11-28T11:06:19.661 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T11:06:19.813 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T11:06:20.005 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-28T11:06:20.155 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-28T11:06:20.286 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-28T11:06:20.496 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-28T11:06:20.920 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-28T11:06:21.154 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:06:21.556 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:06:21.730 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:06:21.858 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:06:22.029 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:06:22.124 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:06:22.333 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:06:22.587 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T11:06:22.828 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T11:06:23.239 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T11:06:23.261 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2023-11-28T11:06:23.463 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : thrift-0.13.0-2.el8.x86_64 1/134 2023-11-28T11:06:23.510 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T11:06:23.628 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : liboath-2.6.2-3.el8.x86_64 2/134 2023-11-28T11:06:23.688 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:06:23.791 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:06:23.815 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/134 2023-11-28T11:06:23.956 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-28T11:06:23.962 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:06:24.037 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:06:24.113 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 4/134 2023-11-28T11:06:24.179 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-28T11:06:24.306 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-28T11:06:24.331 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-28T11:06:24.479 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-28T11:06:24.716 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/134 2023-11-28T11:06:24.931 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:06:25.132 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-28T11:06:25.287 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-28T11:06:25.346 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:06:25.548 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:06:25.677 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:06:25.831 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:06:25.923 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-28T11:06:25.939 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:06:26.087 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-28T11:06:26.138 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:06:26.405 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libpq-13.5-1.el8.x86_64 10/134 2023-11-28T11:06:26.602 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/134 2023-11-28T11:06:26.839 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-28T11:06:27.046 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/134 2023-11-28T11:06:27.094 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-28T11:06:27.301 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-10.21-2.module+el8.6.0+15342+53518fac 13/134 2023-11-28T11:06:27.346 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-28T11:06:27.489 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:06:27.559 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-28T11:06:27.583 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 14/134 2023-11-28T11:06:27.727 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-28T11:06:27.764 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:06:27.843 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 15/134 2023-11-28T11:06:27.948 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-28T11:06:27.948 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-28T11:06:28.094 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libquadmath-8.5.0-10.1.el8_6.x86_64 16/134 2023-11-28T11:06:28.192 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T11:06:28.222 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-28T11:06:28.361 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T11:06:28.490 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T11:06:28.912 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libgfortran-8.5.0-10.1.el8_6.x86_64 17/134 2023-11-28T11:06:29.106 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-28T11:06:29.449 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T11:06:29.727 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openblas-threads-0.3.15-3.el8.x86_64 18/134 2023-11-28T11:06:29.785 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-28T11:06:29.922 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-0.3.15-3.el8.x86_64 19/134 2023-11-28T11:06:30.017 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-28T11:06:30.272 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-28T11:06:30.470 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-28T11:06:30.686 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openblas-0.3.15-3.el8.x86_64 19/134 2023-11-28T11:06:30.944 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 20/134 2023-11-28T11:06:31.163 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 21/134 2023-11-28T11:06:31.377 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 22/134 2023-11-28T11:06:31.563 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/134 2023-11-28T11:06:31.768 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 24/134 2023-11-28T11:06:32.029 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/134 2023-11-28T11:06:32.196 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 26/134 2023-11-28T11:06:32.326 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 27/134 2023-11-28T11:06:32.339 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-28T11:06:32.657 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-28T11:06:32.862 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T11:06:33.060 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T11:06:33.211 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T11:06:33.286 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 28/134 2023-11-28T11:06:33.367 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T11:06:33.604 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 29/134 2023-11-28T11:06:33.647 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-28T11:06:33.831 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-28T11:06:33.844 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 30/134 2023-11-28T11:06:33.998 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-28T11:06:34.167 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-28T11:06:34.263 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 31/134 2023-11-28T11:06:34.355 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-28T11:06:34.455 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-28T11:06:34.563 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-28T11:06:34.941 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-28T11:06:35.316 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-28T11:06:35.508 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-28T11:06:35.760 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-28T11:06:35.978 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-28T11:06:36.304 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-28T11:06:36.330 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 32/134 2023-11-28T11:06:36.553 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-28T11:06:36.641 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 33/134 2023-11-28T11:06:36.730 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-28T11:06:36.944 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-28T11:06:36.978 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rsa-4.9-2.el8.noarch 34/134 2023-11-28T11:06:37.118 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-28T11:06:37.184 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 35/134 2023-11-28T11:06:37.308 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-28T11:06:37.367 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 36/134 2023-11-28T11:06:37.535 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 37/134 2023-11-28T11:06:37.595 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-28T11:06:37.690 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 38/134 2023-11-28T11:06:37.841 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-28T11:06:37.870 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libuuid-devel-2.32.1-35.el8.x86_64 39/134 2023-11-28T11:06:38.032 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : fuse-2.9.7-15.el8.x86_64 40/134 2023-11-28T11:06:38.128 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-28T11:06:38.185 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 41/134 2023-11-28T11:06:38.343 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-28T11:06:38.399 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 42/134 2023-11-28T11:06:38.519 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-28T11:06:38.599 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 43/134 2023-11-28T11:06:38.663 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-28T11:06:38.852 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-28T11:06:38.912 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-28T11:06:38.956 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cffi-1.11.5-5.el8.x86_64 44/134 2023-11-28T11:06:39.146 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-28T11:06:39.217 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cryptography-3.2.1-5.el8.x86_64 45/134 2023-11-28T11:06:39.332 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-28T11:06:39.416 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 46/134 2023-11-28T11:06:39.527 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-28T11:06:39.604 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 47/134 2023-11-28T11:06:39.607 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-28T11:06:39.812 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-28T11:06:39.835 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 48/134 2023-11-28T11:06:39.980 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-28T11:06:40.137 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T11:06:40.153 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 49/134 2023-11-28T11:06:40.352 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T11:06:40.412 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 50/134 2023-11-28T11:06:40.589 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 51/134 2023-11-28T11:06:40.633 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T11:06:40.795 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 52/134 2023-11-28T11:06:40.952 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T11:06:40.952 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 53/134 2023-11-28T11:06:41.116 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T11:06:41.134 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 54/134 2023-11-28T11:06:41.257 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T11:06:41.439 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/134 2023-11-28T11:06:41.441 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T11:06:41.609 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T11:06:41.682 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/134 2023-11-28T11:06:41.791 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T11:06:41.944 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/134 2023-11-28T11:06:41.955 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T11:06:42.092 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T11:06:42.161 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : krb5-devel-1.18.2-14.el8.x86_64 58/134 2023-11-28T11:06:42.287 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T11:06:42.337 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 59/134 2023-11-28T11:06:42.491 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T11:06:42.515 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 60/134 2023-11-28T11:06:42.692 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T11:06:42.704 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : uuid-1.6.2-43.el8.x86_64 61/134 2023-11-28T11:06:42.766 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 61/134 2023-11-28T11:06:42.932 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T11:06:42.997 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 62/134 2023-11-28T11:06:43.169 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T11:06:43.200 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : automake-1.16.1-7.el8.noarch 63/134 2023-11-28T11:06:43.354 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-28T11:06:43.438 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 64/134 2023-11-28T11:06:43.630 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T11:06:43.637 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : jq-1.6-3.el8.x86_64 65/134 2023-11-28T11:06:43.814 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 66/134 2023-11-28T11:06:43.895 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T11:06:43.997 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 67/134 2023-11-28T11:06:44.186 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 68/134 2023-11-28T11:06:44.459 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 69/134 2023-11-28T11:06:44.670 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 70/134 2023-11-28T11:06:44.809 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 71/134 2023-11-28T11:06:44.958 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 72/134 2023-11-28T11:06:45.067 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T11:06:45.142 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 73/134 2023-11-28T11:06:45.316 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T11:06:45.361 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 74/134 2023-11-28T11:06:45.537 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T11:06:45.543 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 75/134 2023-11-28T11:06:45.699 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 76/134 2023-11-28T11:06:45.835 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 77/134 2023-11-28T11:06:45.864 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T11:06:46.071 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 78/134 2023-11-28T11:06:46.099 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T11:06:46.267 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T11:06:46.275 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 79/134 2023-11-28T11:06:46.468 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 80/134 2023-11-28T11:06:46.791 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-portend-2.6-1.el8.noarch 81/134 2023-11-28T11:06:47.020 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 82/134 2023-11-28T11:06:47.101 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T11:06:47.179 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 83/134 2023-11-28T11:06:47.301 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:06:47.375 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:06:47.377 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 84/134 2023-11-28T11:06:47.527 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T11:06:48.511 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 85/134 2023-11-28T11:06:48.750 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 86/134 2023-11-28T11:06:49.139 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 87/134 2023-11-28T11:06:49.432 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 88/134 2023-11-28T11:06:49.706 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 89/134 2023-11-28T11:06:49.886 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 90/134 2023-11-28T11:06:50.687 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 91/134 2023-11-28T11:06:50.853 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:06:50.947 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 92/134 2023-11-28T11:06:51.120 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 93/134 2023-11-28T11:06:51.403 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:51.593 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:52.207 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:52.470 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:06:52.579 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:06:52.700 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:06:55.170 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:55.369 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:55.982 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 94/134 2023-11-28T11:06:56.392 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:06:56.474 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 95/134 2023-11-28T11:06:56.660 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:07:12.174 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:07:12.174 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /sys 2023-11-28T11:07:12.174 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /proc 2023-11-28T11:07:12.174 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /dev 2023-11-28T11:07:12.174 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /run 2023-11-28T11:07:12.174 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /mnt 2023-11-28T11:07:12.175 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /var/tmp 2023-11-28T11:07:12.175 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /home 2023-11-28T11:07:12.175 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /tmp 2023-11-28T11:07:12.175 INFO:teuthology.orchestra.run.smithi028.stdout:skipping the directory /dev 2023-11-28T11:07:12.175 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:12.515 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:07:14.187 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:07:14.375 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:07:14.606 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:07:14.748 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:07:15.231 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:07:15.423 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:07:15.441 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:07:16.380 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 96/134 2023-11-28T11:07:16.381 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /sys 2023-11-28T11:07:16.381 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /proc 2023-11-28T11:07:16.381 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /dev 2023-11-28T11:07:16.381 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /run 2023-11-28T11:07:16.381 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /mnt 2023-11-28T11:07:16.381 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /var/tmp 2023-11-28T11:07:16.381 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /home 2023-11-28T11:07:16.381 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /tmp 2023-11-28T11:07:16.381 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /dev 2023-11-28T11:07:16.381 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:16.758 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:07:17.427 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:07:17.620 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:07:17.745 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T11:07:17.930 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T11:07:18.105 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:07:18.329 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:07:18.429 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 97/134 2023-11-28T11:07:18.589 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:07:18.608 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:07:18.819 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 98/134 2023-11-28T11:07:18.958 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:07:19.446 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 99/134 2023-11-28T11:07:19.633 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:07:19.666 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:07:19.684 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 100/134 2023-11-28T11:07:19.938 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:07:20.035 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:07:20.394 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T11:07:20.600 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:07:20.637 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:07:20.797 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:07:20.809 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:07:21.655 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:07:21.833 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 101/134 2023-11-28T11:07:21.956 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 102/134 2023-11-28T11:07:22.131 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 103/134 2023-11-28T11:07:22.150 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T11:07:22.307 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:07:22.372 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:07:22.407 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:07:22.529 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 104/134 2023-11-28T11:07:22.555 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:07:22.598 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:07:22.738 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:07:22.805 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:07:23.106 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:07:23.239 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:07:23.831 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 105/134 2023-11-28T11:07:23.990 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:07:24.085 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 106/134 2023-11-28T11:07:24.568 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 107/134 2023-11-28T11:07:24.730 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:07:24.767 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 108/134 2023-11-28T11:07:24.915 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:07:24.927 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 109/134 2023-11-28T11:07:26.221 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 110/134 2023-11-28T11:07:26.436 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:07:26.471 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 111/134 2023-11-28T11:07:26.657 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:07:26.700 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 112/134 2023-11-28T11:07:26.840 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:07:27.208 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 113/134 2023-11-28T11:07:27.382 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:07:27.951 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:07:28.221 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T11:07:28.371 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-28T11:07:28.576 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-28T11:07:29.524 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-28T11:07:29.859 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-28T11:07:30.030 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 119/134 2023-11-28T11:07:30.175 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-28T11:07:30.333 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T11:07:30.480 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:07:32.091 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 114/134 2023-11-28T11:07:32.402 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 115/134 2023-11-28T11:07:32.539 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libtool-2.4.6-25.el8.x86_64 116/134 2023-11-28T11:07:32.743 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 116/134 2023-11-28T11:07:33.008 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:07:33.220 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:07:33.239 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:07:33.379 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T11:07:33.672 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T11:07:33.711 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 117/134 2023-11-28T11:07:33.976 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T11:07:34.049 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openssl-devel-1:1.1.1k-7.el8_6.x86_64 118/134 2023-11-28T11:07:34.220 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 119/134 2023-11-28T11:07:34.247 INFO:teuthology.orchestra.run.smithi028.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T11:07:34.433 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:07:34.439 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:07:34.452 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-28T11:07:34.611 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 121/134 2023-11-28T11:07:34.634 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-28T11:07:34.740 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:07:34.787 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-28T11:07:34.876 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-28T11:07:35.062 INFO:teuthology.orchestra.run.smithi028.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-28T11:07:35.076 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-28T11:07:35.077 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:35.307 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:35.308 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:07:35.510 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:07:35.703 INFO:teuthology.orchestra.run.smithi028.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:36.239 INFO:teuthology.orchestra.run.smithi028.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:36.239 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T11:07:36.239 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T11:07:36.239 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T11:07:36.240 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T11:07:36.240 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T11:07:36.240 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:07:36.240 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:07:36.240 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:07:36.240 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:07:36.240 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T11:07:36.240 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T11:07:36.240 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T11:07:36.240 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T11:07:36.241 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T11:07:36.241 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T11:07:36.241 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T11:07:36.241 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T11:07:36.241 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T11:07:36.241 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T11:07:36.241 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T11:07:36.241 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T11:07:36.241 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T11:07:36.242 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T11:07:36.242 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T11:07:36.242 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T11:07:36.242 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T11:07:36.242 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T11:07:36.242 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T11:07:36.242 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T11:07:36.242 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T11:07:36.242 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T11:07:36.242 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T11:07:36.243 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T11:07:36.243 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T11:07:36.243 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T11:07:36.243 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T11:07:36.243 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T11:07:36.243 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T11:07:36.243 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T11:07:36.243 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T11:07:36.243 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T11:07:36.243 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T11:07:36.244 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T11:07:36.244 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T11:07:36.244 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T11:07:36.244 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T11:07:36.244 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T11:07:36.244 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T11:07:36.244 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T11:07:36.244 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T11:07:36.244 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T11:07:36.244 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T11:07:36.245 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T11:07:36.245 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T11:07:36.245 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T11:07:36.245 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T11:07:36.245 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T11:07:36.245 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T11:07:36.245 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T11:07:36.245 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T11:07:36.245 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T11:07:36.245 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T11:07:36.246 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T11:07:36.246 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T11:07:36.246 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T11:07:36.246 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T11:07:36.246 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T11:07:36.246 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T11:07:36.246 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T11:07:36.246 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T11:07:36.246 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T11:07:36.247 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-28T11:07:36.247 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-28T11:07:36.247 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-28T11:07:36.247 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-28T11:07:36.247 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-28T11:07:36.247 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-28T11:07:36.247 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-28T11:07:36.247 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-28T11:07:36.247 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-28T11:07:36.247 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-28T11:07:36.248 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-28T11:07:36.248 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-28T11:07:36.248 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-28T11:07:36.248 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-28T11:07:36.248 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-28T11:07:36.248 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-28T11:07:36.248 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-28T11:07:36.248 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-28T11:07:36.248 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-28T11:07:36.248 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-28T11:07:36.248 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-28T11:07:36.249 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-28T11:07:36.249 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-28T11:07:36.249 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-28T11:07:36.249 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-28T11:07:36.249 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-28T11:07:36.249 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-28T11:07:36.249 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-28T11:07:36.249 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-28T11:07:36.249 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-28T11:07:36.249 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-28T11:07:36.250 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-28T11:07:36.250 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-28T11:07:36.250 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-28T11:07:36.250 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-28T11:07:36.250 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-28T11:07:36.250 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-28T11:07:36.250 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-28T11:07:36.250 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-28T11:07:36.250 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-28T11:07:36.250 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-28T11:07:36.250 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-28T11:07:36.251 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-28T11:07:36.251 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-28T11:07:36.251 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-28T11:07:36.251 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-28T11:07:36.251 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-28T11:07:36.251 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-28T11:07:36.251 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-28T11:07:36.251 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-28T11:07:36.251 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:07:36.251 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-28T11:07:36.251 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-28T11:07:36.251 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-28T11:07:36.251 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-28T11:07:36.252 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-28T11:07:36.252 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:07:36.252 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T11:07:36.252 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T11:07:36.252 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T11:07:36.252 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:36.252 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T11:07:37.115 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:07:37.347 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:07:37.365 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-10.21-2.module+el8.6.0+15342+5 123/134 2023-11-28T11:07:37.728 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 124/134 2023-11-28T11:07:37.923 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 125/134 2023-11-28T11:07:38.206 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 126/134 2023-11-28T11:07:38.482 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : iproute-5.15.0-4.el8_6.1.x86_64 127/134 2023-11-28T11:07:38.656 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:07:38.664 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:07:38.831 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 129/134 2023-11-28T11:07:39.014 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flex-2.6.1-9.el8.x86_64 130/134 2023-11-28T11:07:39.024 INFO:teuthology.orchestra.run.smithi028.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:39.025 INFO:teuthology.orchestra.run.smithi028.stdout:Last metadata expiration check: 0:02:08 ago on Tue 28 Nov 2023 11:05:31 AM UTC. 2023-11-28T11:07:39.117 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 130/134 2023-11-28T11:07:39.338 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : bison-3.0.4-10.el8.x86_64 131/134 2023-11-28T11:07:39.353 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 131/134 2023-11-28T11:07:39.353 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:39.534 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:39.534 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:07:39.762 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 133/134 2023-11-28T11:07:39.938 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 1/134 2023-11-28T11:07:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2/134 2023-11-28T11:07:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 3/134 2023-11-28T11:07:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 4/134 2023-11-28T11:07:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6 5/134 2023-11-28T11:07:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 6/134 2023-11-28T11:07:40.511 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 7/134 2023-11-28T11:07:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 8/134 2023-11-28T11:07:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 9/134 2023-11-28T11:07:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 10/134 2023-11-28T11:07:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 11/134 2023-11-28T11:07:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 12/134 2023-11-28T11:07:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 13/134 2023-11-28T11:07:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_ 14/134 2023-11-28T11:07:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 15/134 2023-11-28T11:07:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 16/134 2023-11-28T11:07:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_6 17/134 2023-11-28T11:07:40.512 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86 18/134 2023-11-28T11:07:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 19/134 2023-11-28T11:07:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el 20/134 2023-11-28T11:07:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8. 21/134 2023-11-28T11:07:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_6 22/134 2023-11-28T11:07:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 23/134 2023-11-28T11:07:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 24/134 2023-11-28T11:07:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 25/134 2023-11-28T11:07:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 26/134 2023-11-28T11:07:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 27/134 2023-11-28T11:07:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 28/134 2023-11-28T11:07:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028. 29/134 2023-11-28T11:07:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noa 30/134 2023-11-28T11:07:40.513 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.n 31/134 2023-11-28T11:07:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5 32/134 2023-11-28T11:07:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el 33/134 2023-11-28T11:07:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 34/134 2023-11-28T11:07:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.e 35/134 2023-11-28T11:07:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 36/134 2023-11-28T11:07:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 37/134 2023-11-28T11:07:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 38/134 2023-11-28T11:07:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/134 2023-11-28T11:07:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/134 2023-11-28T11:07:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/134 2023-11-28T11:07:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/134 2023-11-28T11:07:40.514 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/134 2023-11-28T11:07:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/134 2023-11-28T11:07:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/134 2023-11-28T11:07:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/134 2023-11-28T11:07:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/134 2023-11-28T11:07:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/134 2023-11-28T11:07:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/134 2023-11-28T11:07:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/134 2023-11-28T11:07:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/134 2023-11-28T11:07:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/134 2023-11-28T11:07:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/134 2023-11-28T11:07:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/134 2023-11-28T11:07:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/134 2023-11-28T11:07:40.515 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/134 2023-11-28T11:07:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/134 2023-11-28T11:07:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/134 2023-11-28T11:07:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/134 2023-11-28T11:07:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/134 2023-11-28T11:07:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/134 2023-11-28T11:07:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/134 2023-11-28T11:07:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/134 2023-11-28T11:07:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/134 2023-11-28T11:07:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/134 2023-11-28T11:07:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/134 2023-11-28T11:07:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/134 2023-11-28T11:07:40.516 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/134 2023-11-28T11:07:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/134 2023-11-28T11:07:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/134 2023-11-28T11:07:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/134 2023-11-28T11:07:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 72/134 2023-11-28T11:07:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 73/134 2023-11-28T11:07:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 74/134 2023-11-28T11:07:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 75/134 2023-11-28T11:07:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 76/134 2023-11-28T11:07:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 77/134 2023-11-28T11:07:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/134 2023-11-28T11:07:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 79/134 2023-11-28T11:07:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 80/134 2023-11-28T11:07:40.517 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 81/134 2023-11-28T11:07:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : automake-1.16.1-7.el8.noarch 82/134 2023-11-28T11:07:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 83/134 2023-11-28T11:07:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 84/134 2023-11-28T11:07:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flex-2.6.1-9.el8.x86_64 85/134 2023-11-28T11:07:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 86/134 2023-11-28T11:07:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 87/134 2023-11-28T11:07:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 88/134 2023-11-28T11:07:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 89/134 2023-11-28T11:07:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 90/134 2023-11-28T11:07:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 91/134 2023-11-28T11:07:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 92/134 2023-11-28T11:07:40.518 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 93/134 2023-11-28T11:07:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 94/134 2023-11-28T11:07:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-threads-0.3.15-3.el8.x86_64 95/134 2023-11-28T11:07:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libpq-13.5-1.el8.x86_64 96/134 2023-11-28T11:07:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-0.3.15-3.el8.x86_64 97/134 2023-11-28T11:07:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : jq-1.6-3.el8.x86_64 98/134 2023-11-28T11:07:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 99/134 2023-11-28T11:07:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-server-10.21-2.module+el8.6.0+15342+5 100/134 2023-11-28T11:07:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-contrib-10.21-2.module+el8.6.0+15342+ 101/134 2023-11-28T11:07:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-10.21-2.module+el8.6.0+15342+53518fac 102/134 2023-11-28T11:07:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 103/134 2023-11-28T11:07:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 104/134 2023-11-28T11:07:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 105/134 2023-11-28T11:07:40.519 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 106/134 2023-11-28T11:07:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 107/134 2023-11-28T11:07:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 108/134 2023-11-28T11:07:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 109/134 2023-11-28T11:07:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 110/134 2023-11-28T11:07:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/134 2023-11-28T11:07:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : krb5-devel-1.18.2-14.el8.x86_64 112/134 2023-11-28T11:07:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 113/134 2023-11-28T11:07:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 114/134 2023-11-28T11:07:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cryptography-3.2.1-5.el8.x86_64 115/134 2023-11-28T11:07:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 116/134 2023-11-28T11:07:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 117/134 2023-11-28T11:07:40.520 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cffi-1.11.5-5.el8.x86_64 118/134 2023-11-28T11:07:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-devel-1:1.1.1k-7.el8_6.x86_64 119/134 2023-11-28T11:07:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-devel-5.0.0-10.el8.x86_64 120/134 2023-11-28T11:07:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : fuse-2.9.7-15.el8.x86_64 121/134 2023-11-28T11:07:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-devel-0.186-1.el8.x86_64 122/134 2023-11-28T11:07:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-devel-2.32.1-35.el8.x86_64 123/134 2023-11-28T11:07:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 124/134 2023-11-28T11:07:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 125/134 2023-11-28T11:07:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgfortran-8.5.0-10.1.el8_6.x86_64 126/134 2023-11-28T11:07:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libquadmath-8.5.0-10.1.el8_6.x86_64 127/134 2023-11-28T11:07:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-initscripts-updown-1:1.36.0-9.el8 128/134 2023-11-28T11:07:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 129/134 2023-11-28T11:07:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 130/134 2023-11-28T11:07:40.521 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 131/134 2023-11-28T11:07:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 132/134 2023-11-28T11:07:40.522 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : iproute-5.15.0-4.el8_6.1.x86_64 133/134 2023-11-28T11:07:42.056 INFO:teuthology.orchestra.run.smithi028.stdout:Installed products updated. 2023-11-28T11:07:42.347 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:42.347 INFO:teuthology.orchestra.run.smithi028.stdout:Upgraded: 2023-11-28T11:07:42.348 INFO:teuthology.orchestra.run.smithi028.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T11:07:42.348 INFO:teuthology.orchestra.run.smithi028.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.348 INFO:teuthology.orchestra.run.smithi028.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.348 INFO:teuthology.orchestra.run.smithi028.stdout:Installed: 2023-11-28T11:07:42.348 INFO:teuthology.orchestra.run.smithi028.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T11:07:42.348 INFO:teuthology.orchestra.run.smithi028.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T11:07:42.348 INFO:teuthology.orchestra.run.smithi028.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T11:07:42.348 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.348 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.348 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.348 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.349 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:42.349 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.349 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.349 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.349 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:42.349 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:42.349 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:42.349 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:42.349 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:42.349 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.349 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.349 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:42.350 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.350 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.350 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.351 INFO:teuthology.orchestra.run.smithi028.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:42.351 INFO:teuthology.orchestra.run.smithi028.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:42.351 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.351 INFO:teuthology.orchestra.run.smithi028.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:42.351 INFO:teuthology.orchestra.run.smithi028.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T11:07:42.351 INFO:teuthology.orchestra.run.smithi028.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T11:07:42.351 INFO:teuthology.orchestra.run.smithi028.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T11:07:42.351 INFO:teuthology.orchestra.run.smithi028.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T11:07:42.351 INFO:teuthology.orchestra.run.smithi028.stdout: jq-1.6-3.el8.x86_64 2023-11-28T11:07:42.351 INFO:teuthology.orchestra.run.smithi028.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T11:07:42.352 INFO:teuthology.orchestra.run.smithi028.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T11:07:42.352 INFO:teuthology.orchestra.run.smithi028.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T11:07:42.352 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.352 INFO:teuthology.orchestra.run.smithi028.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.352 INFO:teuthology.orchestra.run.smithi028.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.352 INFO:teuthology.orchestra.run.smithi028.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T11:07:42.352 INFO:teuthology.orchestra.run.smithi028.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:07:42.352 INFO:teuthology.orchestra.run.smithi028.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T11:07:42.352 INFO:teuthology.orchestra.run.smithi028.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T11:07:42.352 INFO:teuthology.orchestra.run.smithi028.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:07:42.352 INFO:teuthology.orchestra.run.smithi028.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T11:07:42.352 INFO:teuthology.orchestra.run.smithi028.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.353 INFO:teuthology.orchestra.run.smithi028.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.353 INFO:teuthology.orchestra.run.smithi028.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T11:07:42.353 INFO:teuthology.orchestra.run.smithi028.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.353 INFO:teuthology.orchestra.run.smithi028.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T11:07:42.353 INFO:teuthology.orchestra.run.smithi028.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T11:07:42.353 INFO:teuthology.orchestra.run.smithi028.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T11:07:42.353 INFO:teuthology.orchestra.run.smithi028.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T11:07:42.353 INFO:teuthology.orchestra.run.smithi028.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T11:07:42.353 INFO:teuthology.orchestra.run.smithi028.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T11:07:42.353 INFO:teuthology.orchestra.run.smithi028.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T11:07:42.353 INFO:teuthology.orchestra.run.smithi028.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T11:07:42.353 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T11:07:42.354 INFO:teuthology.orchestra.run.smithi028.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T11:07:42.354 INFO:teuthology.orchestra.run.smithi028.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T11:07:42.354 INFO:teuthology.orchestra.run.smithi028.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T11:07:42.354 INFO:teuthology.orchestra.run.smithi028.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T11:07:42.354 INFO:teuthology.orchestra.run.smithi028.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T11:07:42.354 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:42.354 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:42.354 INFO:teuthology.orchestra.run.smithi028.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:42.354 INFO:teuthology.orchestra.run.smithi028.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T11:07:42.354 INFO:teuthology.orchestra.run.smithi028.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T11:07:42.354 INFO:teuthology.orchestra.run.smithi028.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T11:07:42.354 INFO:teuthology.orchestra.run.smithi028.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T11:07:42.355 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T11:07:42.355 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.355 INFO:teuthology.orchestra.run.smithi028.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.355 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.355 INFO:teuthology.orchestra.run.smithi028.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T11:07:42.355 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T11:07:42.355 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T11:07:42.355 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T11:07:42.355 INFO:teuthology.orchestra.run.smithi028.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T11:07:42.355 INFO:teuthology.orchestra.run.smithi028.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T11:07:42.355 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T11:07:42.355 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T11:07:42.356 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T11:07:42.356 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T11:07:42.356 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T11:07:42.356 INFO:teuthology.orchestra.run.smithi028.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T11:07:42.356 INFO:teuthology.orchestra.run.smithi028.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T11:07:42.356 INFO:teuthology.orchestra.run.smithi028.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T11:07:42.356 INFO:teuthology.orchestra.run.smithi028.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T11:07:42.356 INFO:teuthology.orchestra.run.smithi028.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T11:07:42.356 INFO:teuthology.orchestra.run.smithi028.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T11:07:42.356 INFO:teuthology.orchestra.run.smithi028.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T11:07:42.356 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T11:07:42.356 INFO:teuthology.orchestra.run.smithi028.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T11:07:42.357 INFO:teuthology.orchestra.run.smithi028.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T11:07:42.357 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T11:07:42.357 INFO:teuthology.orchestra.run.smithi028.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T11:07:42.357 INFO:teuthology.orchestra.run.smithi028.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T11:07:42.357 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T11:07:42.357 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T11:07:42.357 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T11:07:42.357 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T11:07:42.357 INFO:teuthology.orchestra.run.smithi028.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T11:07:42.357 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.357 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.357 INFO:teuthology.orchestra.run.smithi028.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T11:07:42.358 INFO:teuthology.orchestra.run.smithi028.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T11:07:42.358 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.358 INFO:teuthology.orchestra.run.smithi028.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T11:07:42.358 INFO:teuthology.orchestra.run.smithi028.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T11:07:42.358 INFO:teuthology.orchestra.run.smithi028.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T11:07:42.358 INFO:teuthology.orchestra.run.smithi028.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T11:07:42.358 INFO:teuthology.orchestra.run.smithi028.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T11:07:42.358 INFO:teuthology.orchestra.run.smithi028.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T11:07:42.358 INFO:teuthology.orchestra.run.smithi028.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T11:07:42.358 INFO:teuthology.orchestra.run.smithi028.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T11:07:42.358 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T11:07:42.359 INFO:teuthology.orchestra.run.smithi028.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T11:07:42.359 INFO:teuthology.orchestra.run.smithi028.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T11:07:42.359 INFO:teuthology.orchestra.run.smithi028.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T11:07:42.359 INFO:teuthology.orchestra.run.smithi028.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T11:07:42.359 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.359 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.359 INFO:teuthology.orchestra.run.smithi028.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:42.359 INFO:teuthology.orchestra.run.smithi028.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T11:07:42.359 INFO:teuthology.orchestra.run.smithi028.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T11:07:42.359 INFO:teuthology.orchestra.run.smithi028.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T11:07:42.359 INFO:teuthology.orchestra.run.smithi028.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T11:07:42.359 INFO:teuthology.orchestra.run.smithi028.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T11:07:42.360 INFO:teuthology.orchestra.run.smithi028.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T11:07:42.360 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:42.360 INFO:teuthology.orchestra.run.smithi028.stdout:Complete! 2023-11-28T11:07:42.544 DEBUG:teuthology.parallel:result is None 2023-11-28T11:07:43.211 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 134/134 2023-11-28T11:07:43.211 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:02:08 ago on Tue 28 Nov 2023 11:05:35 AM UTC. 2023-11-28T11:07:48.487 INFO:teuthology.orchestra.run.smithi059.stdout:Installed products updated. 2023-11-28T11:07:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:48.731 INFO:teuthology.orchestra.run.smithi059.stdout:Upgraded: 2023-11-28T11:07:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: iproute-5.15.0-4.el8_6.1.x86_64 2023-11-28T11:07:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: librados2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.731 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2023-11-28T11:07:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-initscripts-updown-1:1.36.0-9.el8_6.noarch 2023-11-28T11:07:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T11:07:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T11:07:48.731 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:48.732 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top-2:18.0.0-2697.g2f5f6028.el8.noarch 2023-11-28T11:07:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel-0.186-1.el8.x86_64 2023-11-28T11:07:48.733 INFO:teuthology.orchestra.run.smithi059.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T11:07:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: fuse-2.9.7-15.el8.x86_64 2023-11-28T11:07:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T11:07:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: jq-1.6-3.el8.x86_64 2023-11-28T11:07:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-11-28T11:07:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: krb5-devel-1.18.2-14.el8.x86_64 2023-11-28T11:07:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T11:07:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-11-28T11:07:48.734 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:07:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T11:07:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: libpq-13.5-1.el8.x86_64 2023-11-28T11:07:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath-8.5.0-10.1.el8_6.x86_64 2023-11-28T11:07:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T11:07:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-11-28T11:07:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T11:07:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-11-28T11:07:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T11:07:48.735 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T11:07:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel-2.32.1-35.el8.x86_64 2023-11-28T11:07:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T11:07:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T11:07:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T11:07:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-0.3.15-3.el8.x86_64 2023-11-28T11:07:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-threads-0.3.15-3.el8.x86_64 2023-11-28T11:07:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel-1:1.1.1k-7.el8_6.x86_64 2023-11-28T11:07:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T11:07:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T11:07:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T11:07:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:48.736 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server-10.21-2.module+el8.6.0+15342+53518fac.x86_64 2023-11-28T11:07:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-11-28T11:07:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel-2.5.1-7.el8.noarch 2023-11-28T11:07:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T11:07:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T11:07:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T11:07:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T11:07:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi-1.11.5-5.el8.x86_64 2023-11-28T11:07:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T11:07:48.737 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T11:07:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography-3.2.1-5.el8.x86_64 2023-11-28T11:07:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T11:07:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T11:07:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T11:07:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-11-28T11:07:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T11:07:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T11:07:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T11:07:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T11:07:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T11:07:48.738 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T11:07:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T11:07:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T11:07:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-11-28T11:07:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-11-28T11:07:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-11-28T11:07:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T11:07:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T11:07:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T11:07:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T11:07:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-11-28T11:07:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T11:07:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T11:07:48.739 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser-2.14-14.el8.noarch 2023-11-28T11:07:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T11:07:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T11:07:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T11:07:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T11:07:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T11:07:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-11-28T11:07:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T11:07:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T11:07:48.740 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T11:07:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T11:07:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T11:07:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T11:07:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T11:07:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T11:07:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T11:07:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T11:07:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd-2:18.0.0-2697.g2f5f6028.el8.x86_64 2023-11-28T11:07:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: socat-1.7.4.1-1.el8.x86_64 2023-11-28T11:07:48.741 INFO:teuthology.orchestra.run.smithi059.stdout: thrift-0.13.0-2.el8.x86_64 2023-11-28T11:07:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: uuid-1.6.2-43.el8.x86_64 2023-11-28T11:07:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel-5.0.0-10.el8.x86_64 2023-11-28T11:07:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T11:07:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-11-28T11:07:48.742 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:48.742 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2023-11-28T11:07:48.912 DEBUG:teuthology.parallel:result is None 2023-11-28T11:07:48.913 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:07:49.078 DEBUG:teuthology.orchestra.run.smithi028:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T11:07:49.108 INFO:teuthology.orchestra.run.smithi028.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T11:07:49.109 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T11:07:49.109 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T11:07:49.109 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=2f5f6028b893ad41357cd06e827373f0fe2a1232 2023-11-28T11:07:49.351 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T11:07:49.377 INFO:teuthology.orchestra.run.smithi059.stdout:18.0.0-2697.g2f5f6028.el8 2023-11-28T11:07:49.377 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-2697.g2f5f6028.el8 2023-11-28T11:07:49.377 INFO:teuthology.task.install:The correct ceph version 18.0.0-2697.g2f5f6028 is installed. 2023-11-28T11:07:49.378 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-28T11:07:49.378 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:07:49.378 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T11:07:49.428 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:07:49.428 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T11:07:49.474 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-28T11:07:49.474 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:07:49.474 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T11:07:49.507 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T11:07:49.578 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:07:49.578 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T11:07:49.608 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T11:07:49.677 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-28T11:07:49.678 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:07:49.678 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T11:07:49.709 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T11:07:49.781 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:07:49.781 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T11:07:49.811 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T11:07:49.881 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-28T11:07:49.881 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:07:49.881 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T11:07:49.912 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T11:07:49.984 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:07:49.984 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T11:07:50.013 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T11:07:50.082 INFO:teuthology.run_tasks:Running task ceph... 2023-11-28T11:07:50.158 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-28T11:07:50.158 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 777 /var/log/ceph 2023-11-28T11:07:50.161 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 777 /var/log/ceph 2023-11-28T11:07:50.191 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-28T11:07:50.191 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T11:07:50.234 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T11:07:50.263 INFO:tasks.ceph:Creating extra log directories... 2023-11-28T11:07:50.263 DEBUG:teuthology.orchestra.run.smithi028:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T11:07:50.306 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T11:07:50.343 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-28T11:07:50.344 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-28T11:07:50.344 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-28T11:07:50.344 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:07:50.344 DEBUG:teuthology.orchestra.run.smithi028:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T11:07:50.395 DEBUG:teuthology.orchestra.run.smithi028:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T11:07:50.525 DEBUG:teuthology.orchestra.run.smithi028:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T11:07:50.555 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:07:50.555 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T11:07:50.574 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T11:07:50.725 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T11:07:50.761 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-28T11:07:50.761 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-28T11:07:50.761 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957', 'branch': 'wip-neesingh-testing-231123', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub}', 'email': 'neesingh@redhat.com', 'first_in_suite': False, 'job_id': '7469957', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-neesingh-testing-231123'}, 'ceph': {'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'inode wrongly marked free', 'bad backtrace on inode', 'inode table repaired for inode', 'Scrub error on inode', 'Scrub error on dir', 'Metadata damage detected'], 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'issue_54557', 'sha1': 'e67b67438c1e5e13734a7203ff9e57e30b8e4d32'}}, 'owner': 'scheduled_neesingh@teuthology', 'priority': 10, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 9437, 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'issue_54557', 'suite_path': '/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/neesingh-rh/ceph', 'suite_sha1': 'e67b67438c1e5e13734a7203ff9e57e30b8e4d32', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi028.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs=', 'smithi059.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '2f5f6028b893ad41357cd06e827373f0fe2a1232'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_forward_scrub']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '5ae09a5cc7064ca2bf994df19f3006b79794789a', 'timestamp': '2023-11-28_10:48:03', 'tube': 'smithi', 'user': 'neesingh', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-28T11:07:50.761 DEBUG:teuthology.orchestra.run.smithi028:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T11:07:50.764 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T11:07:50.813 DEBUG:teuthology.orchestra.run.smithi028:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T11:07:50.824 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T11:07:50.882 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:07:50.882 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T11:07:50.911 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T11:07:50.911 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_1 2023-11-28T11:07:50.970 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T11:07:50.970 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:50.970 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 6h/6d Inode: 387655 Links: 1 2023-11-28T11:07:50.970 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:50.970 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:50.970 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2023-11-28 11:07:35.914709476 +0000 2023-11-28T11:07:50.970 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2023-11-28 11:04:41.112202170 +0000 2023-11-28T11:07:50.971 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2023-11-28 11:04:41.112202170 +0000 2023-11-28T11:07:50.971 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: - 2023-11-28T11:07:50.971 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T11:07:51.037 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2023-11-28T11:07:51.037 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2023-11-28T11:07:51.038 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.00012235 s, 4.2 MB/s 2023-11-28T11:07:51.039 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T11:07:51.098 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_2 2023-11-28T11:07:51.156 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T11:07:51.157 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:51.157 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 6h/6d Inode: 392400 Links: 1 2023-11-28T11:07:51.157 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:51.157 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:51.157 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2023-11-28 11:07:35.915709448 +0000 2023-11-28T11:07:51.157 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2023-11-28 11:04:41.654186927 +0000 2023-11-28T11:07:51.157 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2023-11-28 11:04:41.654186927 +0000 2023-11-28T11:07:51.157 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: - 2023-11-28T11:07:51.157 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T11:07:51.225 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2023-11-28T11:07:51.225 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2023-11-28T11:07:51.225 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.00011508 s, 4.4 MB/s 2023-11-28T11:07:51.226 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T11:07:51.285 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_3 2023-11-28T11:07:51.344 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T11:07:51.344 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:51.344 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 6h/6d Inode: 394393 Links: 1 2023-11-28T11:07:51.344 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:51.344 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:51.344 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2023-11-28 11:07:35.915709448 +0000 2023-11-28T11:07:51.344 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2023-11-28 11:04:42.159172725 +0000 2023-11-28T11:07:51.344 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2023-11-28 11:04:42.159172725 +0000 2023-11-28T11:07:51.345 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: - 2023-11-28T11:07:51.345 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T11:07:51.411 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2023-11-28T11:07:51.411 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2023-11-28T11:07:51.412 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 0.000112102 s, 4.6 MB/s 2023-11-28T11:07:51.412 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T11:07:51.472 DEBUG:teuthology.orchestra.run.smithi028:> stat /dev/vg_nvme/lv_4 2023-11-28T11:07:51.530 INFO:teuthology.orchestra.run.smithi028.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T11:07:51.530 INFO:teuthology.orchestra.run.smithi028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:51.530 INFO:teuthology.orchestra.run.smithi028.stdout:Device: 6h/6d Inode: 391813 Links: 1 2023-11-28T11:07:51.530 INFO:teuthology.orchestra.run.smithi028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:51.531 INFO:teuthology.orchestra.run.smithi028.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:51.531 INFO:teuthology.orchestra.run.smithi028.stdout:Access: 2023-11-28 11:07:35.915709448 +0000 2023-11-28T11:07:51.531 INFO:teuthology.orchestra.run.smithi028.stdout:Modify: 2023-11-28 11:04:42.678158129 +0000 2023-11-28T11:07:51.531 INFO:teuthology.orchestra.run.smithi028.stdout:Change: 2023-11-28 11:04:42.678158129 +0000 2023-11-28T11:07:51.531 INFO:teuthology.orchestra.run.smithi028.stdout: Birth: - 2023-11-28T11:07:51.531 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T11:07:51.597 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records in 2023-11-28T11:07:51.598 INFO:teuthology.orchestra.run.smithi028.stderr:1+0 records out 2023-11-28T11:07:51.598 INFO:teuthology.orchestra.run.smithi028.stderr:512 bytes copied, 9.1464e-05 s, 5.6 MB/s 2023-11-28T11:07:51.599 DEBUG:teuthology.orchestra.run.smithi028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T11:07:51.658 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:51.659 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:07:51.659 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T11:07:51.680 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T11:07:51.680 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_1 2023-11-28T11:07:51.741 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T11:07:51.742 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:51.742 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 381793 Links: 1 2023-11-28T11:07:51.742 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:51.742 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:51.742 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2023-11-28 11:07:40.172870416 +0000 2023-11-28T11:07:51.742 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2023-11-28 11:04:34.146232351 +0000 2023-11-28T11:07:51.742 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2023-11-28 11:04:34.146232351 +0000 2023-11-28T11:07:51.742 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2023-11-28T11:07:51.743 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T11:07:51.812 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2023-11-28T11:07:51.813 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2023-11-28T11:07:51.813 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000279201 s, 1.8 MB/s 2023-11-28T11:07:51.814 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T11:07:51.875 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_2 2023-11-28T11:07:51.936 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T11:07:51.936 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:51.936 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 381911 Links: 1 2023-11-28T11:07:51.936 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:51.936 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:51.936 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2023-11-28 11:07:40.172870416 +0000 2023-11-28T11:07:51.937 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2023-11-28 11:04:34.715224887 +0000 2023-11-28T11:07:51.937 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2023-11-28 11:04:34.715224887 +0000 2023-11-28T11:07:51.937 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2023-11-28T11:07:51.937 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T11:07:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2023-11-28T11:07:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2023-11-28T11:07:52.006 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000230225 s, 2.2 MB/s 2023-11-28T11:07:52.007 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T11:07:52.069 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_3 2023-11-28T11:07:52.129 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T11:07:52.129 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:52.129 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 392588 Links: 1 2023-11-28T11:07:52.129 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:52.129 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:52.129 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2023-11-28 11:07:40.172870416 +0000 2023-11-28T11:07:52.130 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2023-11-28 11:04:35.256217790 +0000 2023-11-28T11:07:52.130 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2023-11-28 11:04:35.256217790 +0000 2023-11-28T11:07:52.130 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2023-11-28T11:07:52.130 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T11:07:52.202 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2023-11-28T11:07:52.203 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2023-11-28T11:07:52.204 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.00023122 s, 2.2 MB/s 2023-11-28T11:07:52.206 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T11:07:52.262 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_4 2023-11-28T11:07:52.321 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T11:07:52.322 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T11:07:52.322 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 6h/6d Inode: 395408 Links: 1 2023-11-28T11:07:52.322 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T11:07:52.322 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T11:07:52.322 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2023-11-28 11:07:40.172870416 +0000 2023-11-28T11:07:52.322 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2023-11-28 11:04:35.792210758 +0000 2023-11-28T11:07:52.322 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2023-11-28 11:04:35.792210758 +0000 2023-11-28T11:07:52.322 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: - 2023-11-28T11:07:52.322 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T11:07:52.390 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2023-11-28T11:07:52.390 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2023-11-28T11:07:52.390 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000184834 s, 2.8 MB/s 2023-11-28T11:07:52.391 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T11:07:52.450 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:52.450 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi028.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi059.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T11:07:52.450 INFO:tasks.ceph:Generating config... 2023-11-28T11:07:52.452 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-28T11:07:52.453 INFO:tasks.ceph:[client] debug client = 20 2023-11-28T11:07:52.453 INFO:tasks.ceph:[client] debug ms = 1 2023-11-28T11:07:52.453 INFO:tasks.ceph:[client] pid file = 2023-11-28T11:07:52.453 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-28T11:07:52.454 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-28T11:07:52.454 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-28T11:07:52.454 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-28T11:07:52.454 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-28T11:07:52.454 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-28T11:07:52.454 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-28T11:07:52.454 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-28T11:07:52.454 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-28T11:07:52.455 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-28T11:07:52.455 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-28T11:07:52.455 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-28T11:07:52.455 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-28T11:07:52.455 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-28T11:07:52.455 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-28T11:07:52.455 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-28T11:07:52.455 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-28T11:07:52.455 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-28T11:07:52.456 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-28T11:07:52.456 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-28T11:07:52.456 INFO:tasks.ceph:[osd] debug bluefs = 20 2023-11-28T11:07:52.456 INFO:tasks.ceph:[osd] debug bluestore = 20 2023-11-28T11:07:52.456 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-28T11:07:52.456 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-28T11:07:52.456 INFO:tasks.ceph:[osd] debug rocksdb = 10 2023-11-28T11:07:52.456 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-28T11:07:52.456 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-28T11:07:52.456 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-28T11:07:52.457 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-28T11:07:52.457 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-28T11:07:52.457 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-28T11:07:52.457 INFO:tasks.ceph:Setting up mon.a... 2023-11-28T11:07:52.457 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-28T11:07:52.524 INFO:teuthology.orchestra.run.smithi028.stdout:creating /etc/ceph/ceph.keyring 2023-11-28T11:07:52.527 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-28T11:07:52.577 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:07:52.606 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.28'), ('mon.b', '172.21.15.59'), ('mon.c', '[v2:172.21.15.59:3301,v1:172.21.15.59:6790]')] 2023-11-28T11:07:52.606 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.28,172.21.15.59,[v2:172.21.15.59:3301,v1:172.21.15.59:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-28T11:07:52.607 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:07:52.607 DEBUG:teuthology.orchestra.run.smithi028:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T11:07:52.664 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.28 --add b 172.21.15.59 --addv c '[v2:172.21.15.59:3301,v1:172.21.15.59:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:52.754 INFO:teuthology.orchestra.run.smithi028.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:52.755 INFO:teuthology.orchestra.run.smithi028.stdout:monmaptool: generated fsid 6e299adc-7a2a-4cf1-afa2-7ce104b29a51 2023-11-28T11:07:52.755 INFO:teuthology.orchestra.run.smithi028.stdout:setting min_mon_release = pacific 2023-11-28T11:07:52.755 INFO:teuthology.orchestra.run.smithi028.stdout:epoch 0 2023-11-28T11:07:52.755 INFO:teuthology.orchestra.run.smithi028.stdout:fsid 6e299adc-7a2a-4cf1-afa2-7ce104b29a51 2023-11-28T11:07:52.755 INFO:teuthology.orchestra.run.smithi028.stdout:last_changed 2023-11-28T11:07:52.754885+0000 2023-11-28T11:07:52.755 INFO:teuthology.orchestra.run.smithi028.stdout:created 2023-11-28T11:07:52.754885+0000 2023-11-28T11:07:52.755 INFO:teuthology.orchestra.run.smithi028.stdout:min_mon_release 16 (pacific) 2023-11-28T11:07:52.755 INFO:teuthology.orchestra.run.smithi028.stdout:election_strategy: 1 2023-11-28T11:07:52.755 INFO:teuthology.orchestra.run.smithi028.stdout:0: [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] mon.a 2023-11-28T11:07:52.755 INFO:teuthology.orchestra.run.smithi028.stdout:1: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.b 2023-11-28T11:07:52.755 INFO:teuthology.orchestra.run.smithi028.stdout:2: [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] mon.c 2023-11-28T11:07:52.756 INFO:teuthology.orchestra.run.smithi028.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-28T11:07:52.757 DEBUG:teuthology.orchestra.run.smithi028:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T11:07:52.773 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 6e299adc-7a2a-4cf1-afa2-7ce104b29a51... 2023-11-28T11:07:52.774 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T11:07:52.815 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T11:07:52.860 INFO:teuthology.orchestra.run.smithi028.stdout:[global] 2023-11-28T11:07:52.860 INFO:teuthology.orchestra.run.smithi028.stdout: chdir = "" 2023-11-28T11:07:52.860 INFO:teuthology.orchestra.run.smithi028.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T11:07:52.860 INFO:teuthology.orchestra.run.smithi028.stdout: auth supported = cephx 2023-11-28T11:07:52.860 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.860 INFO:teuthology.orchestra.run.smithi028.stdout: filestore xattr use omap = true 2023-11-28T11:07:52.861 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.861 INFO:teuthology.orchestra.run.smithi028.stdout: mon clock drift allowed = 1.000 2023-11-28T11:07:52.861 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.861 INFO:teuthology.orchestra.run.smithi028.stdout: osd crush chooseleaf type = 0 2023-11-28T11:07:52.861 INFO:teuthology.orchestra.run.smithi028.stdout: auth debug = true 2023-11-28T11:07:52.861 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.861 INFO:teuthology.orchestra.run.smithi028.stdout: ms die on old message = true 2023-11-28T11:07:52.861 INFO:teuthology.orchestra.run.smithi028.stdout: ms die on bug = true 2023-11-28T11:07:52.861 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.861 INFO:teuthology.orchestra.run.smithi028.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T11:07:52.861 INFO:teuthology.orchestra.run.smithi028.stdout: mon pg warn max object skew = 0 2023-11-28T11:07:52.861 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.862 INFO:teuthology.orchestra.run.smithi028.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T11:07:52.862 INFO:teuthology.orchestra.run.smithi028.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T11:07:52.862 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.862 INFO:teuthology.orchestra.run.smithi028.stdout: osd pool default size = 2 2023-11-28T11:07:52.862 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.862 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd allow primary affinity = true 2023-11-28T11:07:52.862 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd allow pg remap = true 2023-11-28T11:07:52.862 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on legacy crush tunables = false 2023-11-28T11:07:52.862 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on crush straw calc version zero = false 2023-11-28T11:07:52.862 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on no sortbitwise = false 2023-11-28T11:07:52.862 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on osd down out interval zero = false 2023-11-28T11:07:52.863 INFO:teuthology.orchestra.run.smithi028.stdout: mon warn on too few osds = false 2023-11-28T11:07:52.863 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T11:07:52.863 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T11:07:52.863 INFO:teuthology.orchestra.run.smithi028.stdout: mon_allow_pool_size_one = true 2023-11-28T11:07:52.863 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.863 INFO:teuthology.orchestra.run.smithi028.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T11:07:52.863 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.863 INFO:teuthology.orchestra.run.smithi028.stdout: osd default data pool replay window = 5 2023-11-28T11:07:52.863 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.863 INFO:teuthology.orchestra.run.smithi028.stdout: mon allow pool delete = true 2023-11-28T11:07:52.863 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.863 INFO:teuthology.orchestra.run.smithi028.stdout: mon cluster log file level = debug 2023-11-28T11:07:52.864 INFO:teuthology.orchestra.run.smithi028.stdout: debug asserts on shutdown = true 2023-11-28T11:07:52.864 INFO:teuthology.orchestra.run.smithi028.stdout: mon health detail to clog = false 2023-11-28T11:07:52.864 INFO:teuthology.orchestra.run.smithi028.stdout: mon host = "172.21.15.28,172.21.15.59,[v2:172.21.15.59:3301,v1:172.21.15.59:6790]" 2023-11-28T11:07:52.864 INFO:teuthology.orchestra.run.smithi028.stdout: fsid = 6e299adc-7a2a-4cf1-afa2-7ce104b29a51 2023-11-28T11:07:52.864 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.864 INFO:teuthology.orchestra.run.smithi028.stdout:[osd] 2023-11-28T11:07:52.864 INFO:teuthology.orchestra.run.smithi028.stdout: osd journal size = 100 2023-11-28T11:07:52.864 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.864 INFO:teuthology.orchestra.run.smithi028.stdout: osd scrub load threshold = 5.0 2023-11-28T11:07:52.864 INFO:teuthology.orchestra.run.smithi028.stdout: osd scrub max interval = 600 2023-11-28T11:07:52.864 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.864 INFO:teuthology.orchestra.run.smithi028.stdout: osd recover clone overlap = true 2023-11-28T11:07:52.865 INFO:teuthology.orchestra.run.smithi028.stdout: osd recovery max chunk = 1048576 2023-11-28T11:07:52.865 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.865 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug shutdown = true 2023-11-28T11:07:52.865 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug op order = true 2023-11-28T11:07:52.865 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug verify stray on activate = true 2023-11-28T11:07:52.865 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.865 INFO:teuthology.orchestra.run.smithi028.stdout: osd open classes on start = true 2023-11-28T11:07:52.865 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug pg log writeout = true 2023-11-28T11:07:52.865 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.865 INFO:teuthology.orchestra.run.smithi028.stdout: osd deep scrub update digest min age = 30 2023-11-28T11:07:52.865 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.865 INFO:teuthology.orchestra.run.smithi028.stdout: osd map max advance = 10 2023-11-28T11:07:52.866 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.866 INFO:teuthology.orchestra.run.smithi028.stdout: journal zero on create = true 2023-11-28T11:07:52.866 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.866 INFO:teuthology.orchestra.run.smithi028.stdout: filestore ondisk finisher threads = 3 2023-11-28T11:07:52.866 INFO:teuthology.orchestra.run.smithi028.stdout: filestore apply finisher threads = 3 2023-11-28T11:07:52.866 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.867 INFO:teuthology.orchestra.run.smithi028.stdout: bdev debug aio = true 2023-11-28T11:07:52.867 INFO:teuthology.orchestra.run.smithi028.stdout: osd debug misdirected ops = true 2023-11-28T11:07:52.867 INFO:teuthology.orchestra.run.smithi028.stdout: bluestore block size = 96636764160 2023-11-28T11:07:52.867 INFO:teuthology.orchestra.run.smithi028.stdout: bluestore fsck on mount = True 2023-11-28T11:07:52.867 INFO:teuthology.orchestra.run.smithi028.stdout: debug bluefs = 20 2023-11-28T11:07:52.867 INFO:teuthology.orchestra.run.smithi028.stdout: debug bluestore = 20 2023-11-28T11:07:52.867 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2023-11-28T11:07:52.868 INFO:teuthology.orchestra.run.smithi028.stdout: debug osd = 20 2023-11-28T11:07:52.868 INFO:teuthology.orchestra.run.smithi028.stdout: debug rocksdb = 10 2023-11-28T11:07:52.868 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T11:07:52.869 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd full ratio = 0.9 2023-11-28T11:07:52.869 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd nearfull ratio = 0.8 2023-11-28T11:07:52.869 INFO:teuthology.orchestra.run.smithi028.stdout: osd failsafe full ratio = 0.95 2023-11-28T11:07:52.869 INFO:teuthology.orchestra.run.smithi028.stdout: osd objectstore = bluestore 2023-11-28T11:07:52.869 INFO:teuthology.orchestra.run.smithi028.stdout: osd op complaint time = 180 2023-11-28T11:07:52.869 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.869 INFO:teuthology.orchestra.run.smithi028.stdout:[mgr] 2023-11-28T11:07:52.869 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2023-11-28T11:07:52.869 INFO:teuthology.orchestra.run.smithi028.stdout: debug mgr = 20 2023-11-28T11:07:52.869 INFO:teuthology.orchestra.run.smithi028.stdout: debug mon = 20 2023-11-28T11:07:52.869 INFO:teuthology.orchestra.run.smithi028.stdout: debug auth = 20 2023-11-28T11:07:52.869 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:07:52.870 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:07:52.870 INFO:teuthology.orchestra.run.smithi028.stdout: mgr/telemetry/nag = false 2023-11-28T11:07:52.870 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.870 INFO:teuthology.orchestra.run.smithi028.stdout:[mon] 2023-11-28T11:07:52.870 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2023-11-28T11:07:52.870 INFO:teuthology.orchestra.run.smithi028.stdout: debug mon = 20 2023-11-28T11:07:52.870 INFO:teuthology.orchestra.run.smithi028.stdout: debug paxos = 20 2023-11-28T11:07:52.870 INFO:teuthology.orchestra.run.smithi028.stdout: debug auth = 20 2023-11-28T11:07:52.870 INFO:teuthology.orchestra.run.smithi028.stdout: mon data avail warn = 5 2023-11-28T11:07:52.870 INFO:teuthology.orchestra.run.smithi028.stdout: mon mgr mkfs grace = 240 2023-11-28T11:07:52.870 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:07:52.871 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd reporter subtree level = osd 2023-11-28T11:07:52.871 INFO:teuthology.orchestra.run.smithi028.stdout: mon osd prime pg temp = true 2023-11-28T11:07:52.871 INFO:teuthology.orchestra.run.smithi028.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:07:52.871 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.871 INFO:teuthology.orchestra.run.smithi028.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T11:07:52.871 INFO:teuthology.orchestra.run.smithi028.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T11:07:52.871 INFO:teuthology.orchestra.run.smithi028.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T11:07:52.871 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.871 INFO:teuthology.orchestra.run.smithi028.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T11:07:52.871 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T11:07:52.871 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T11:07:52.871 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.872 INFO:teuthology.orchestra.run.smithi028.stdout: # 1m isn't quite enough 2023-11-28T11:07:52.872 INFO:teuthology.orchestra.run.smithi028.stdout: mon_down_mkfs_grace = 2m 2023-11-28T11:07:52.872 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.872 INFO:teuthology.orchestra.run.smithi028.stdout: mon_warn_on_filestore_osds = false 2023-11-28T11:07:52.872 INFO:teuthology.orchestra.run.smithi028.stdout: mon op complaint time = 120 2023-11-28T11:07:52.872 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:07:52.872 INFO:teuthology.orchestra.run.smithi028.stdout:[client] 2023-11-28T11:07:52.872 INFO:teuthology.orchestra.run.smithi028.stdout: rgw cache enabled = true 2023-11-28T11:07:52.872 INFO:teuthology.orchestra.run.smithi028.stdout: rgw enable ops log = true 2023-11-28T11:07:52.872 INFO:teuthology.orchestra.run.smithi028.stdout: rgw enable usage log = true 2023-11-28T11:07:52.872 INFO:teuthology.orchestra.run.smithi028.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T11:07:52.872 INFO:teuthology.orchestra.run.smithi028.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T11:07:52.873 INFO:teuthology.orchestra.run.smithi028.stdout: client mount timeout = 600 2023-11-28T11:07:52.873 INFO:teuthology.orchestra.run.smithi028.stdout: debug client = 20 2023-11-28T11:07:52.873 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2023-11-28T11:07:52.873 INFO:teuthology.orchestra.run.smithi028.stdout: pid file = "" 2023-11-28T11:07:52.873 INFO:teuthology.orchestra.run.smithi028.stdout: rados mon op timeout = 900 2023-11-28T11:07:52.873 INFO:teuthology.orchestra.run.smithi028.stdout: rados osd op timeout = 900 2023-11-28T11:07:52.873 INFO:teuthology.orchestra.run.smithi028.stdout:[mon.a] 2023-11-28T11:07:52.873 INFO:teuthology.orchestra.run.smithi028.stdout:[mon.b] 2023-11-28T11:07:52.873 INFO:teuthology.orchestra.run.smithi028.stdout:[mon.c] 2023-11-28T11:07:52.873 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.a] 2023-11-28T11:07:52.873 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.c] 2023-11-28T11:07:52.873 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.b] 2023-11-28T11:07:52.874 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.d] 2023-11-28T11:07:52.874 INFO:teuthology.orchestra.run.smithi028.stdout:[mds] 2023-11-28T11:07:52.874 INFO:teuthology.orchestra.run.smithi028.stdout: debug mds = 20 2023-11-28T11:07:52.874 INFO:teuthology.orchestra.run.smithi028.stdout: debug mds balancer = 20 2023-11-28T11:07:52.874 INFO:teuthology.orchestra.run.smithi028.stdout: debug ms = 1 2023-11-28T11:07:52.874 INFO:teuthology.orchestra.run.smithi028.stdout: mds debug frag = True 2023-11-28T11:07:52.874 INFO:teuthology.orchestra.run.smithi028.stdout: mds debug scatterstat = True 2023-11-28T11:07:52.874 INFO:teuthology.orchestra.run.smithi028.stdout: mds op complaint time = 180 2023-11-28T11:07:52.874 INFO:teuthology.orchestra.run.smithi028.stdout: mds verify scatter = True 2023-11-28T11:07:52.874 INFO:teuthology.orchestra.run.smithi028.stdout: osd op complaint time = 180 2023-11-28T11:07:52.874 INFO:teuthology.orchestra.run.smithi028.stdout: rados mon op timeout = 900 2023-11-28T11:07:52.874 INFO:teuthology.orchestra.run.smithi028.stdout: rados osd op timeout = 900 2023-11-28T11:07:52.875 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2023-11-28T11:07:52.875 INFO:teuthology.orchestra.run.smithi059.stdout: chdir = "" 2023-11-28T11:07:52.876 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T11:07:52.876 INFO:teuthology.orchestra.run.smithi059.stdout: auth supported = cephx 2023-11-28T11:07:52.876 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.876 INFO:teuthology.orchestra.run.smithi059.stdout: filestore xattr use omap = true 2023-11-28T11:07:52.876 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.876 INFO:teuthology.orchestra.run.smithi059.stdout: mon clock drift allowed = 1.000 2023-11-28T11:07:52.876 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.876 INFO:teuthology.orchestra.run.smithi059.stdout: osd crush chooseleaf type = 0 2023-11-28T11:07:52.876 INFO:teuthology.orchestra.run.smithi059.stdout: auth debug = true 2023-11-28T11:07:52.876 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.876 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on old message = true 2023-11-28T11:07:52.876 INFO:teuthology.orchestra.run.smithi059.stdout: ms die on bug = true 2023-11-28T11:07:52.877 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.877 INFO:teuthology.orchestra.run.smithi059.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T11:07:52.877 INFO:teuthology.orchestra.run.smithi059.stdout: mon pg warn max object skew = 0 2023-11-28T11:07:52.877 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.877 INFO:teuthology.orchestra.run.smithi059.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T11:07:52.877 INFO:teuthology.orchestra.run.smithi059.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T11:07:52.877 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.877 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default size = 2 2023-11-28T11:07:52.877 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.877 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow primary affinity = true 2023-11-28T11:07:52.877 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd allow pg remap = true 2023-11-28T11:07:52.877 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on legacy crush tunables = false 2023-11-28T11:07:52.878 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on crush straw calc version zero = false 2023-11-28T11:07:52.878 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on no sortbitwise = false 2023-11-28T11:07:52.878 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on osd down out interval zero = false 2023-11-28T11:07:52.878 INFO:teuthology.orchestra.run.smithi059.stdout: mon warn on too few osds = false 2023-11-28T11:07:52.878 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T11:07:52.878 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T11:07:52.878 INFO:teuthology.orchestra.run.smithi059.stdout: mon_allow_pool_size_one = true 2023-11-28T11:07:52.878 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.878 INFO:teuthology.orchestra.run.smithi059.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-11-28T11:07:52.878 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.878 INFO:teuthology.orchestra.run.smithi059.stdout: osd default data pool replay window = 5 2023-11-28T11:07:52.879 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.879 INFO:teuthology.orchestra.run.smithi059.stdout: mon allow pool delete = true 2023-11-28T11:07:52.879 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.879 INFO:teuthology.orchestra.run.smithi059.stdout: mon cluster log file level = debug 2023-11-28T11:07:52.879 INFO:teuthology.orchestra.run.smithi059.stdout: debug asserts on shutdown = true 2023-11-28T11:07:52.879 INFO:teuthology.orchestra.run.smithi059.stdout: mon health detail to clog = false 2023-11-28T11:07:52.879 INFO:teuthology.orchestra.run.smithi059.stdout: mon host = "172.21.15.28,172.21.15.59,[v2:172.21.15.59:3301,v1:172.21.15.59:6790]" 2023-11-28T11:07:52.879 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = 6e299adc-7a2a-4cf1-afa2-7ce104b29a51 2023-11-28T11:07:52.879 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.879 INFO:teuthology.orchestra.run.smithi059.stdout:[osd] 2023-11-28T11:07:52.879 INFO:teuthology.orchestra.run.smithi059.stdout: osd journal size = 100 2023-11-28T11:07:52.880 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.880 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub load threshold = 5.0 2023-11-28T11:07:52.880 INFO:teuthology.orchestra.run.smithi059.stdout: osd scrub max interval = 600 2023-11-28T11:07:52.880 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.880 INFO:teuthology.orchestra.run.smithi059.stdout: osd recover clone overlap = true 2023-11-28T11:07:52.880 INFO:teuthology.orchestra.run.smithi059.stdout: osd recovery max chunk = 1048576 2023-11-28T11:07:52.880 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.880 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug shutdown = true 2023-11-28T11:07:52.880 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug op order = true 2023-11-28T11:07:52.880 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug verify stray on activate = true 2023-11-28T11:07:52.880 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.880 INFO:teuthology.orchestra.run.smithi059.stdout: osd open classes on start = true 2023-11-28T11:07:52.881 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug pg log writeout = true 2023-11-28T11:07:52.881 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.881 INFO:teuthology.orchestra.run.smithi059.stdout: osd deep scrub update digest min age = 30 2023-11-28T11:07:52.881 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.881 INFO:teuthology.orchestra.run.smithi059.stdout: osd map max advance = 10 2023-11-28T11:07:52.881 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.881 INFO:teuthology.orchestra.run.smithi059.stdout: journal zero on create = true 2023-11-28T11:07:52.881 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.881 INFO:teuthology.orchestra.run.smithi059.stdout: filestore ondisk finisher threads = 3 2023-11-28T11:07:52.881 INFO:teuthology.orchestra.run.smithi059.stdout: filestore apply finisher threads = 3 2023-11-28T11:07:52.881 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.881 INFO:teuthology.orchestra.run.smithi059.stdout: bdev debug aio = true 2023-11-28T11:07:52.882 INFO:teuthology.orchestra.run.smithi059.stdout: osd debug misdirected ops = true 2023-11-28T11:07:52.882 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore block size = 96636764160 2023-11-28T11:07:52.882 INFO:teuthology.orchestra.run.smithi059.stdout: bluestore fsck on mount = True 2023-11-28T11:07:52.882 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluefs = 20 2023-11-28T11:07:52.882 INFO:teuthology.orchestra.run.smithi059.stdout: debug bluestore = 20 2023-11-28T11:07:52.882 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2023-11-28T11:07:52.883 INFO:teuthology.orchestra.run.smithi059.stdout: debug osd = 20 2023-11-28T11:07:52.883 INFO:teuthology.orchestra.run.smithi059.stdout: debug rocksdb = 10 2023-11-28T11:07:52.884 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T11:07:52.884 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd full ratio = 0.9 2023-11-28T11:07:52.884 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd nearfull ratio = 0.8 2023-11-28T11:07:52.884 INFO:teuthology.orchestra.run.smithi059.stdout: osd failsafe full ratio = 0.95 2023-11-28T11:07:52.884 INFO:teuthology.orchestra.run.smithi059.stdout: osd objectstore = bluestore 2023-11-28T11:07:52.884 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2023-11-28T11:07:52.884 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.884 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr] 2023-11-28T11:07:52.884 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2023-11-28T11:07:52.884 INFO:teuthology.orchestra.run.smithi059.stdout: debug mgr = 20 2023-11-28T11:07:52.884 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2023-11-28T11:07:52.884 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2023-11-28T11:07:52.885 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:07:52.885 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:07:52.885 INFO:teuthology.orchestra.run.smithi059.stdout: mgr/telemetry/nag = false 2023-11-28T11:07:52.885 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.885 INFO:teuthology.orchestra.run.smithi059.stdout:[mon] 2023-11-28T11:07:52.885 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2023-11-28T11:07:52.885 INFO:teuthology.orchestra.run.smithi059.stdout: debug mon = 20 2023-11-28T11:07:52.885 INFO:teuthology.orchestra.run.smithi059.stdout: debug paxos = 20 2023-11-28T11:07:52.885 INFO:teuthology.orchestra.run.smithi059.stdout: debug auth = 20 2023-11-28T11:07:52.885 INFO:teuthology.orchestra.run.smithi059.stdout: mon data avail warn = 5 2023-11-28T11:07:52.885 INFO:teuthology.orchestra.run.smithi059.stdout: mon mgr mkfs grace = 240 2023-11-28T11:07:52.886 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min pgs per osd = 4 2023-11-28T11:07:52.886 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd reporter subtree level = osd 2023-11-28T11:07:52.886 INFO:teuthology.orchestra.run.smithi059.stdout: mon osd prime pg temp = true 2023-11-28T11:07:52.886 INFO:teuthology.orchestra.run.smithi059.stdout: mon reweight min bytes per osd = 10 2023-11-28T11:07:52.886 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.886 INFO:teuthology.orchestra.run.smithi059.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T11:07:52.886 INFO:teuthology.orchestra.run.smithi059.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T11:07:52.886 INFO:teuthology.orchestra.run.smithi059.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T11:07:52.886 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.886 INFO:teuthology.orchestra.run.smithi059.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T11:07:52.886 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T11:07:52.886 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T11:07:52.887 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.887 INFO:teuthology.orchestra.run.smithi059.stdout: # 1m isn't quite enough 2023-11-28T11:07:52.887 INFO:teuthology.orchestra.run.smithi059.stdout: mon_down_mkfs_grace = 2m 2023-11-28T11:07:52.887 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.887 INFO:teuthology.orchestra.run.smithi059.stdout: mon_warn_on_filestore_osds = false 2023-11-28T11:07:52.887 INFO:teuthology.orchestra.run.smithi059.stdout: mon op complaint time = 120 2023-11-28T11:07:52.887 INFO:teuthology.orchestra.run.smithi059.stdout: 2023-11-28T11:07:52.887 INFO:teuthology.orchestra.run.smithi059.stdout:[client] 2023-11-28T11:07:52.887 INFO:teuthology.orchestra.run.smithi059.stdout: rgw cache enabled = true 2023-11-28T11:07:52.887 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable ops log = true 2023-11-28T11:07:52.887 INFO:teuthology.orchestra.run.smithi059.stdout: rgw enable usage log = true 2023-11-28T11:07:52.887 INFO:teuthology.orchestra.run.smithi059.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T11:07:52.888 INFO:teuthology.orchestra.run.smithi059.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T11:07:52.888 INFO:teuthology.orchestra.run.smithi059.stdout: client mount timeout = 600 2023-11-28T11:07:52.888 INFO:teuthology.orchestra.run.smithi059.stdout: debug client = 20 2023-11-28T11:07:52.888 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2023-11-28T11:07:52.888 INFO:teuthology.orchestra.run.smithi059.stdout: pid file = "" 2023-11-28T11:07:52.888 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2023-11-28T11:07:52.888 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2023-11-28T11:07:52.888 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.a] 2023-11-28T11:07:52.888 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.b] 2023-11-28T11:07:52.888 INFO:teuthology.orchestra.run.smithi059.stdout:[mon.c] 2023-11-28T11:07:52.888 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2023-11-28T11:07:52.888 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2023-11-28T11:07:52.889 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2023-11-28T11:07:52.889 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2023-11-28T11:07:52.889 INFO:teuthology.orchestra.run.smithi059.stdout:[mds] 2023-11-28T11:07:52.889 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds = 20 2023-11-28T11:07:52.889 INFO:teuthology.orchestra.run.smithi059.stdout: debug mds balancer = 20 2023-11-28T11:07:52.889 INFO:teuthology.orchestra.run.smithi059.stdout: debug ms = 1 2023-11-28T11:07:52.889 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug frag = True 2023-11-28T11:07:52.889 INFO:teuthology.orchestra.run.smithi059.stdout: mds debug scatterstat = True 2023-11-28T11:07:52.889 INFO:teuthology.orchestra.run.smithi059.stdout: mds op complaint time = 180 2023-11-28T11:07:52.890 INFO:teuthology.orchestra.run.smithi059.stdout: mds verify scatter = True 2023-11-28T11:07:52.890 INFO:teuthology.orchestra.run.smithi059.stdout: osd op complaint time = 180 2023-11-28T11:07:52.890 INFO:teuthology.orchestra.run.smithi059.stdout: rados mon op timeout = 900 2023-11-28T11:07:52.890 INFO:teuthology.orchestra.run.smithi059.stdout: rados osd op timeout = 900 2023-11-28T11:07:52.891 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-28T11:07:52.892 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-28T11:07:52.989 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-28T11:07:52.990 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:07:52.990 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-28T11:07:53.007 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:07:53.007 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-28T11:07:53.064 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:07:53.064 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:07:53.064 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T11:07:53.064 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:07:53.142 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:07:53.142 DEBUG:teuthology.orchestra.run.smithi028:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:53.199 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:07:53.200 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:07:53.200 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T11:07:53.200 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T11:07:53.245 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:07:53.246 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:53.305 INFO:tasks.ceph:Setting up mon nodes... 2023-11-28T11:07:53.306 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-28T11:07:53.306 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T11:07:53.361 INFO:teuthology.orchestra.run.smithi028.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T11:07:53.363 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T11:07:53.436 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T11:07:53.439 INFO:tasks.ceph:Setting up mds nodes... 2023-11-28T11:07:53.439 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T11:07:53.494 INFO:teuthology.orchestra.run.smithi028.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T11:07:53.497 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-28T11:07:53.524 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T11:07:53.622 INFO:teuthology.orchestra.run.smithi028.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T11:07:53.623 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-28T11:07:53.650 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T11:07:53.712 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T11:07:53.715 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-28T11:07:53.744 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T11:07:53.841 INFO:teuthology.orchestra.run.smithi059.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T11:07:53.844 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-28T11:07:53.871 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-28T11:07:53.872 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-28T11:07:53.919 INFO:teuthology.orchestra.run.smithi028.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-28T11:07:53.933 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-28T11:07:53.978 INFO:teuthology.orchestra.run.smithi028.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-28T11:07:53.991 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-28T11:07:54.036 INFO:teuthology.orchestra.run.smithi059.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-28T11:07:54.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-28T11:07:54.093 INFO:teuthology.orchestra.run.smithi059.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-28T11:07:54.107 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-28T11:07:54.107 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi028.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi059.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T11:07:54.107 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-28T11:07:54.135 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:54.135 INFO:tasks.ceph:role: osd.0 2023-11-28T11:07:54.136 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:07:54.136 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T11:07:54.202 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:54.202 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:54.203 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:54.203 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:54.203 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:54.203 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:54.203 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:54.204 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:54.204 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:54.204 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:54.225 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2023-11-28T11:07:54.227 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2023-11-28T11:07:54.227 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-28T11:07:54.309 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-28T11:07:54.381 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-28T11:07:54.450 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:54.450 INFO:tasks.ceph:role: osd.1 2023-11-28T11:07:54.450 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:07:54.450 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T11:07:54.517 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:54.517 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:54.518 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:54.518 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:54.518 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:54.518 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:54.518 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:54.518 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:54.519 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:54.519 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:54.540 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2023-11-28T11:07:54.541 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2023-11-28T11:07:54.542 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-28T11:07:54.623 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-28T11:07:54.695 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-28T11:07:54.762 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:54.762 INFO:tasks.ceph:role: osd.2 2023-11-28T11:07:54.762 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:07:54.762 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T11:07:54.830 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:54.830 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:54.831 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:54.831 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:54.831 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:54.831 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:54.831 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:54.831 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:54.831 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:54.831 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:54.852 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2023-11-28T11:07:54.853 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2023-11-28T11:07:54.853 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-28T11:07:54.933 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-28T11:07:55.005 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-28T11:07:55.073 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T11:07:55.073 INFO:tasks.ceph:role: osd.3 2023-11-28T11:07:55.073 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:07:55.073 DEBUG:teuthology.orchestra.run.smithi028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T11:07:55.140 INFO:teuthology.orchestra.run.smithi028.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:07:55.140 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:07:55.140 INFO:teuthology.orchestra.run.smithi028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:07:55.140 INFO:teuthology.orchestra.run.smithi028.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:07:55.140 INFO:teuthology.orchestra.run.smithi028.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:07:55.141 INFO:teuthology.orchestra.run.smithi028.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:07:55.141 INFO:teuthology.orchestra.run.smithi028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:07:55.141 INFO:teuthology.orchestra.run.smithi028.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:07:55.141 INFO:teuthology.orchestra.run.smithi028.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:07:55.141 INFO:teuthology.orchestra.run.smithi028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:07:55.163 INFO:teuthology.orchestra.run.smithi028.stdout:Discarding blocks...Done. 2023-11-28T11:07:55.165 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi028.front.sepia.ceph.com -o noatime 2023-11-28T11:07:55.165 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-28T11:07:55.245 DEBUG:teuthology.orchestra.run.smithi028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-28T11:07:55.316 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:55.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:07:55.611+0000 7f1d4b0d3640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-28T11:07:55.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:07:55.611+0000 7f1d4b0d3640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-28T11:07:55.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:07:55.611+0000 7f1d4b0d3640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:07:55.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:07:55.611+0000 7f1d4b0d3640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:07:55.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:07:55.611+0000 7f1d4b0d3640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T11:07:55.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:07:55.611+0000 7f1d4b0d3640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-28T11:07:58.477 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-28T11:07:58.510 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:07:58.604 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:07:58.603+0000 7f9c4693b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-28T11:07:58.604 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:07:58.603+0000 7f9c4693b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-28T11:07:58.604 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:07:58.603+0000 7f9c4693b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:07:58.604 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:07:58.603+0000 7f9c4693b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:07:58.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:07:58.603+0000 7f9c4693b640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T11:07:58.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:07:58.603+0000 7f9c4693b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-28T11:08:01.469 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-28T11:08:01.503 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:08:01.598 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:01.596+0000 7f73eecd6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-28T11:08:01.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:01.596+0000 7f73eecd6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-28T11:08:01.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:01.596+0000 7f73eecd6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:08:01.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:01.596+0000 7f73eecd6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:08:01.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:01.596+0000 7f73eecd6640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T11:08:01.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:01.597+0000 7f73eecd6640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-28T11:08:04.470 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-28T11:08:04.503 DEBUG:teuthology.orchestra.run.smithi028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:08:04.597 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:04.595+0000 7ff302146640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-28T11:08:08.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:04.596+0000 7ff302146640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-28T11:08:08.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:04.596+0000 7ff302146640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:08:08.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:04.596+0000 7ff302146640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:08:08.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:04.596+0000 7ff302146640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T11:08:08.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:04.596+0000 7ff302146640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-28T11:08:08.922 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-28T11:08:08.956 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-28T11:08:08.991 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:08:08.991 INFO:tasks.ceph:role: osd.4 2023-11-28T11:08:08.991 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:08:08.991 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T11:08:09.069 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:08:09.070 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:08:09.070 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:08:09.070 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:08:09.070 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:08:09.070 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:08:09.070 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:08:09.070 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:08:09.070 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:08:09.070 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:08:09.509 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2023-11-28T11:08:09.511 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2023-11-28T11:08:09.511 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-28T11:08:09.693 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-28T11:08:09.732 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-28T11:08:09.803 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:08:09.803 INFO:tasks.ceph:role: osd.5 2023-11-28T11:08:09.803 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:08:09.803 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T11:08:09.882 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:08:09.882 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:08:09.882 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:08:09.882 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:08:09.882 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:08:09.882 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:08:09.882 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:08:09.883 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:08:09.883 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:08:09.883 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:08:10.304 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2023-11-28T11:08:10.306 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2023-11-28T11:08:10.307 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-28T11:08:10.420 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-28T11:08:10.455 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-28T11:08:10.525 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:08:10.525 INFO:tasks.ceph:role: osd.6 2023-11-28T11:08:10.526 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:08:10.526 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T11:08:10.605 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:08:10.606 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:08:10.606 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:08:10.607 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:08:10.607 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:08:10.607 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:08:10.607 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:08:10.607 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:08:10.607 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:08:10.608 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:08:11.060 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2023-11-28T11:08:11.063 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2023-11-28T11:08:11.063 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-28T11:08:11.297 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-28T11:08:11.336 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-28T11:08:11.425 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T11:08:11.425 INFO:tasks.ceph:role: osd.7 2023-11-28T11:08:11.425 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:08:11.425 DEBUG:teuthology.orchestra.run.smithi059:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T11:08:11.512 INFO:teuthology.orchestra.run.smithi059.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T11:08:11.512 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T11:08:11.512 INFO:teuthology.orchestra.run.smithi059.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T11:08:11.512 INFO:teuthology.orchestra.run.smithi059.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-11-28T11:08:11.513 INFO:teuthology.orchestra.run.smithi059.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T11:08:11.513 INFO:teuthology.orchestra.run.smithi059.stdout: = sunit=0 swidth=0 blks 2023-11-28T11:08:11.513 INFO:teuthology.orchestra.run.smithi059.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T11:08:11.513 INFO:teuthology.orchestra.run.smithi059.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T11:08:11.513 INFO:teuthology.orchestra.run.smithi059.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T11:08:11.513 INFO:teuthology.orchestra.run.smithi059.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T11:08:11.944 INFO:teuthology.orchestra.run.smithi059.stdout:Discarding blocks...Done. 2023-11-28T11:08:11.946 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi059.front.sepia.ceph.com -o noatime 2023-11-28T11:08:11.947 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-28T11:08:12.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-28T11:08:12.087 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:08:12.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:12.213+0000 7f7150533640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-28T11:08:12.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:12.213+0000 7f7150533640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-28T11:08:12.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:12.213+0000 7f7150533640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:08:12.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:12.213+0000 7f7150533640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:08:12.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:12.213+0000 7f7150533640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T11:08:12.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:12.213+0000 7f7150533640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-28T11:08:15.086 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-28T11:08:15.120 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:08:15.214 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:15.213+0000 7f60e1145640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-28T11:08:15.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:15.213+0000 7f60e1145640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-28T11:08:15.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:15.213+0000 7f60e1145640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:08:15.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:15.213+0000 7f60e1145640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:08:15.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:15.213+0000 7f60e1145640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T11:08:15.215 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:15.213+0000 7f60e1145640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-28T11:08:18.081 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-28T11:08:18.113 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:08:18.210 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:18.208+0000 7f5aa49b2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-28T11:08:18.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:18.209+0000 7f5aa49b2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-28T11:08:18.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:18.209+0000 7f5aa49b2640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:08:18.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:18.209+0000 7f5aa49b2640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:08:18.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:18.209+0000 7f5aa49b2640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T11:08:18.211 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:18.209+0000 7f5aa49b2640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-28T11:08:20.762 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:08:20.766 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:08:21.085 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-28T11:08:21.118 DEBUG:teuthology.orchestra.run.smithi059:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:08:21.212 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:21.211+0000 7f3eab00b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-28T11:08:21.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:21.211+0000 7f3eab00b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-28T11:08:21.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:21.211+0000 7f3eab00b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:08:21.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:21.211+0000 7f3eab00b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:08:21.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:21.211+0000 7f3eab00b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T11:08:21.213 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:21.212+0000 7f3eab00b640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-28T11:08:24.084 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-28T11:08:24.117 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-28T11:08:24.117 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:08:24.117 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-28T11:08:24.151 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:08:24.152 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-28T11:08:24.224 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:08:24.225 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-28T11:08:24.297 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:08:24.298 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-28T11:08:24.370 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:08:24.370 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-28T11:08:24.442 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:08:24.442 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-28T11:08:24.513 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:08:24.513 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-28T11:08:24.583 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:08:24.584 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-28T11:08:24.617 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:08:24.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-28T11:08:24.690 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:08:24.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-28T11:08:24.761 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:08:24.761 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-28T11:08:24.831 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:08:24.831 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-28T11:08:24.901 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:08:24.901 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-28T11:08:24.971 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:08:24.971 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-28T11:08:25.040 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:08:25.040 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-28T11:08:25.061 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:08:25.061 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-28T11:08:25.122 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:08:25.122 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-28T11:08:25.140 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:08:25.140 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-28T11:08:25.199 INFO:tasks.ceph:Adding keys to all mons... 2023-11-28T11:08:25.199 DEBUG:teuthology.orchestra.run.smithi028:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T11:08:25.202 DEBUG:teuthology.orchestra.run.smithi059:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T11:08:25.242 INFO:teuthology.orchestra.run.smithi028.stdout:[mgr.y] 2023-11-28T11:08:25.242 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAJymVlyBOJFRAA6cjReBkYUD/3YOUHw7DHpg== 2023-11-28T11:08:25.242 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.a] 2023-11-28T11:08:25.243 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAJymVlx1F8HRAAqlv0qwBqCCSPbuXOK00uLQ== 2023-11-28T11:08:25.243 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.c] 2023-11-28T11:08:25.243 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAJymVlz/TyJBAAug4qbJp/xfFm3aHFNMSlBA== 2023-11-28T11:08:25.243 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.0] 2023-11-28T11:08:25.243 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQALymVlGI9xJBAAFY51VVoRxuhhNHGi8tlG5g== 2023-11-28T11:08:25.243 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.1] 2023-11-28T11:08:25.243 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAOymVl6bn5IxAAE02o425v2koATSJg/CKx2Q== 2023-11-28T11:08:25.243 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.2] 2023-11-28T11:08:25.243 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQARymVl0JCmIxAASw1qp+1l8/G+7uP2t2Gfcg== 2023-11-28T11:08:25.243 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.3] 2023-11-28T11:08:25.243 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAUymVlHpKYIxAAeEFLqUE4cWvE29wkIdb4xQ== 2023-11-28T11:08:25.243 INFO:teuthology.orchestra.run.smithi028.stdout:[mgr.x] 2023-11-28T11:08:25.244 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAJymVlZrj9GRAAXu1z37jFMZo3YXUJLRB2+w== 2023-11-28T11:08:25.244 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.b] 2023-11-28T11:08:25.244 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAJymVlqQd7KhAASvfFg9HJbZLZ48lOZ8rfdg== 2023-11-28T11:08:25.244 INFO:teuthology.orchestra.run.smithi028.stdout:[mds.d] 2023-11-28T11:08:25.244 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAJymVlJA0pMhAAgmUFSzlTLDsYT7SPRaS4hg== 2023-11-28T11:08:25.244 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.4] 2023-11-28T11:08:25.244 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAcymVlRw7IDBAACbu1rTbZMEVtwM33xdVPSA== 2023-11-28T11:08:25.244 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.5] 2023-11-28T11:08:25.244 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAfymVlDGXHDBAA5lGng1DglyCM+DnWCQ0iUg== 2023-11-28T11:08:25.244 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.6] 2023-11-28T11:08:25.244 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAiymVlfw+IDBAAX5cXoXUhrY8BX3tZFVyqCA== 2023-11-28T11:08:25.244 INFO:teuthology.orchestra.run.smithi028.stdout:[osd.7] 2023-11-28T11:08:25.244 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAlymVliBSsDBAACyShS6zNLoUAlVFbB5XEag== 2023-11-28T11:08:25.245 INFO:teuthology.orchestra.run.smithi028.stdout:[client.0] 2023-11-28T11:08:25.245 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAJymVluSHRNhAAmBrJ4shI0iFZ+s0TazdTrw== 2023-11-28T11:08:25.245 INFO:teuthology.orchestra.run.smithi028.stdout:[client.2] 2023-11-28T11:08:25.245 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAJymVlxqlTOhAAU7U9b//feat8r6WDQNx/Gg== 2023-11-28T11:08:25.245 INFO:teuthology.orchestra.run.smithi028.stdout:[client.1] 2023-11-28T11:08:25.245 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAKymVl2+otAhAATGAAuzHz3lKdV0enZTr6qA== 2023-11-28T11:08:25.245 INFO:teuthology.orchestra.run.smithi028.stdout:[client.3] 2023-11-28T11:08:25.245 INFO:teuthology.orchestra.run.smithi028.stdout: key = AQAKymVl592XBRAASQZMNs5hbHPm2CYq2ktYdg== 2023-11-28T11:08:25.267 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr.y] 2023-11-28T11:08:25.267 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAJymVlyBOJFRAA6cjReBkYUD/3YOUHw7DHpg== 2023-11-28T11:08:25.267 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a] 2023-11-28T11:08:25.267 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAJymVlx1F8HRAAqlv0qwBqCCSPbuXOK00uLQ== 2023-11-28T11:08:25.267 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c] 2023-11-28T11:08:25.267 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAJymVlz/TyJBAAug4qbJp/xfFm3aHFNMSlBA== 2023-11-28T11:08:25.268 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.0] 2023-11-28T11:08:25.268 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQALymVlGI9xJBAAFY51VVoRxuhhNHGi8tlG5g== 2023-11-28T11:08:25.268 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.1] 2023-11-28T11:08:25.268 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAOymVl6bn5IxAAE02o425v2koATSJg/CKx2Q== 2023-11-28T11:08:25.268 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.2] 2023-11-28T11:08:25.268 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQARymVl0JCmIxAASw1qp+1l8/G+7uP2t2Gfcg== 2023-11-28T11:08:25.268 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.3] 2023-11-28T11:08:25.268 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAUymVlHpKYIxAAeEFLqUE4cWvE29wkIdb4xQ== 2023-11-28T11:08:25.268 INFO:teuthology.orchestra.run.smithi059.stdout:[mgr.x] 2023-11-28T11:08:25.268 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAJymVlZrj9GRAAXu1z37jFMZo3YXUJLRB2+w== 2023-11-28T11:08:25.268 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b] 2023-11-28T11:08:25.268 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAJymVlqQd7KhAASvfFg9HJbZLZ48lOZ8rfdg== 2023-11-28T11:08:25.269 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d] 2023-11-28T11:08:25.269 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAJymVlJA0pMhAAgmUFSzlTLDsYT7SPRaS4hg== 2023-11-28T11:08:25.269 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.4] 2023-11-28T11:08:25.269 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAcymVlRw7IDBAACbu1rTbZMEVtwM33xdVPSA== 2023-11-28T11:08:25.269 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.5] 2023-11-28T11:08:25.269 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAfymVlDGXHDBAA5lGng1DglyCM+DnWCQ0iUg== 2023-11-28T11:08:25.269 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.6] 2023-11-28T11:08:25.269 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAiymVlfw+IDBAAX5cXoXUhrY8BX3tZFVyqCA== 2023-11-28T11:08:25.269 INFO:teuthology.orchestra.run.smithi059.stdout:[osd.7] 2023-11-28T11:08:25.269 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAlymVliBSsDBAACyShS6zNLoUAlVFbB5XEag== 2023-11-28T11:08:25.269 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2023-11-28T11:08:25.269 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAJymVluSHRNhAAmBrJ4shI0iFZ+s0TazdTrw== 2023-11-28T11:08:25.269 INFO:teuthology.orchestra.run.smithi059.stdout:[client.2] 2023-11-28T11:08:25.270 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAJymVlxqlTOhAAU7U9b//feat8r6WDQNx/Gg== 2023-11-28T11:08:25.270 INFO:teuthology.orchestra.run.smithi059.stdout:[client.1] 2023-11-28T11:08:25.270 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAKymVl2+otAhAATGAAuzHz3lKdV0enZTr6qA== 2023-11-28T11:08:25.270 INFO:teuthology.orchestra.run.smithi059.stdout:[client.3] 2023-11-28T11:08:25.270 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAKymVl592XBRAASQZMNs5hbHPm2CYq2ktYdg== 2023-11-28T11:08:25.271 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:08:25.289 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:08:25.368 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:25.370 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:25.428 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:25.431 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:25.488 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:25.490 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:25.544 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:25.546 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:25.600 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:25.602 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:25.657 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:25.659 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:25.713 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:08:25.715 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T11:08:25.769 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:25.776 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:25.834 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:25.837 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T11:08:25.894 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:25.896 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:25.950 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:25.953 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:26.007 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:26.009 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:26.063 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:26.065 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T11:08:26.120 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:26.122 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:26.179 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:26.182 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:26.239 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:26.242 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:26.300 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:26.303 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T11:08:26.360 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-28T11:08:26.360 DEBUG:teuthology.orchestra.run.smithi028:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-28T11:08:26.389 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:08:26.489 DEBUG:teuthology.orchestra.run.smithi028:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-28T11:08:26.517 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-28T11:08:26.545 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:08:26.645 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-28T11:08:26.682 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-28T11:08:26.772 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T11:08:26.879 DEBUG:teuthology.orchestra.run.smithi059:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-28T11:08:26.912 DEBUG:teuthology.orchestra.run.smithi028:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:08:26.917 DEBUG:teuthology.orchestra.run.smithi059:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T11:08:26.980 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-28T11:08:26.980 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-28T11:08:26.980 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-28T11:08:26.988 INFO:tasks.ceph.mon.a:Started 2023-11-28T11:08:26.988 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-28T11:08:26.989 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-28T11:08:27.024 INFO:tasks.ceph.mon.b:Started 2023-11-28T11:08:27.024 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-28T11:08:27.024 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-28T11:08:27.034 INFO:tasks.ceph.mon.c:Started 2023-11-28T11:08:27.034 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-28T11:08:27.034 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-28T11:08:27.034 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-28T11:08:27.037 INFO:tasks.ceph.mgr.y:Started 2023-11-28T11:08:27.037 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-28T11:08:27.037 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-28T11:08:27.044 INFO:tasks.ceph.mgr.x:Started 2023-11-28T11:08:27.044 INFO:tasks.ceph:Setting crush tunables to default 2023-11-28T11:08:27.044 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph osd crush tunables default 2023-11-28T11:08:27.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.178+0000 7ff929a35700 1 Processor -- start 2023-11-28T11:08:27.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.178+0000 7ff929a35700 1 -- start start 2023-11-28T11:08:27.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff929a35700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff924159500 0x7ff9240cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff929a35700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff92415dbf0 0x7ff9240ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff929a35700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff92415e040 con 0x7ff9240b1630 2023-11-28T11:08:27.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff929a35700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9240cede0 con 0x7ff92415dbf0 2023-11-28T11:08:27.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff929a35700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9240ceff0 con 0x7ff924159500 2023-11-28T11:08:27.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff9227fc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff92415dbf0 0x7ff9240ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff9227fc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff92415dbf0 0x7ff9240ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45768/0 (socket says 172.21.15.28:45768) 2023-11-28T11:08:27.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff9227fc700 1 -- 172.21.15.28:0/3998174154 learned_addr learned my addr 172.21.15.28:0/3998174154 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:27.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff922ffd700 1 --2- 172.21.15.28:0/3998174154 >> v2:172.21.15.59:3300/0 conn(0x7ff924159500 0x7ff9240cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff9227fc700 1 -- 172.21.15.28:0/3998174154 >> v2:172.21.15.28:3300/0 conn(0x7ff92415dbf0 msgr2=0x7ff9240ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T11:08:27.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff9227fc700 1 -- 172.21.15.28:0/3998174154 >> v2:172.21.15.28:3300/0 conn(0x7ff92415dbf0 msgr2=0x7ff9240ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:08:27.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff9227fc700 1 --2- 172.21.15.28:0/3998174154 >> v2:172.21.15.28:3300/0 conn(0x7ff92415dbf0 0x7ff9240ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:08:27.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff922ffd700 1 -- 172.21.15.28:0/3998174154 >> v2:172.21.15.59:3300/0 conn(0x7ff924159500 msgr2=0x7ff9240cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-11-28T11:08:27.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff922ffd700 1 -- 172.21.15.28:0/3998174154 >> v2:172.21.15.59:3300/0 conn(0x7ff924159500 msgr2=0x7ff9240cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T11:08:27.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff922ffd700 1 --2- 172.21.15.28:0/3998174154 >> v2:172.21.15.59:3300/0 conn(0x7ff924159500 0x7ff9240cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-11-28T11:08:27.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff9227fc700 1 --2- 172.21.15.28:0/3998174154 >> v2:172.21.15.28:3300/0 conn(0x7ff92415dbf0 0x7ff9240ce8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:08:27.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.179+0000 7ff922ffd700 1 --2- 172.21.15.28:0/3998174154 >> v2:172.21.15.59:3300/0 conn(0x7ff924159500 0x7ff9240cd140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-11-28T11:08:27.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.347+0000 7ff921ffb700 1 -- 172.21.15.28:0/3998174154 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3916994612 0 0) 0x7ff92415e040 con 0x7ff9240b1630 2023-11-28T11:08:27.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.347+0000 7ff921ffb700 1 -- 172.21.15.28:0/3998174154 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff90c0057c0 con 0x7ff9240b1630 2023-11-28T11:08:27.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.352+0000 7ff921ffb700 1 -- 172.21.15.28:0/3998174154 <== mon.1 v1:172.21.15.59:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1516511186 0 0) 0x7ff90c0057c0 con 0x7ff9240b1630 2023-11-28T11:08:27.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.352+0000 7ff921ffb700 1 -- 172.21.15.28:0/3998174154 >> v2:172.21.15.59:3300/0 conn(0x7ff924159500 msgr2=0x7ff9240cd140 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:08:27.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.352+0000 7ff921ffb700 1 --2- 172.21.15.28:0/3998174154 >> v2:172.21.15.59:3300/0 conn(0x7ff924159500 0x7ff9240cd140 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.352+0000 7ff921ffb700 1 -- 172.21.15.28:0/3998174154 >> v2:172.21.15.28:3300/0 conn(0x7ff92415dbf0 msgr2=0x7ff9240ce8a0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:08:27.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.352+0000 7ff921ffb700 1 --2- 172.21.15.28:0/3998174154 >> v2:172.21.15.28:3300/0 conn(0x7ff92415dbf0 0x7ff9240ce8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.352+0000 7ff921ffb700 1 -- 172.21.15.28:0/3998174154 --> v1:172.21.15.59:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9240cf2d0 con 0x7ff9240b1630 2023-11-28T11:08:27.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.352+0000 7ff921ffb700 1 -- 172.21.15.28:0/3998174154 <== mon.1 v1:172.21.15.59:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7ff91c0019f0 con 0x7ff9240b1630 2023-11-28T11:08:27.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff921ffb700 1 -- 172.21.15.28:0/3998174154 >> v1:172.21.15.59:6789/0 conn(0x7ff9240b1630 legacy=0x7ff92406bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff921ffb700 1 --2- 172.21.15.28:0/3998174154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff90c005c60 0x7ff90c006070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff921ffb700 1 --2- 172.21.15.28:0/3998174154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff90c006a60 0x7ff90c006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff921ffb700 1 --2- 172.21.15.28:0/3998174154 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff90c007410 0x7ff90c00dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff921ffb700 1 -- 172.21.15.28:0/3998174154 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9240cede0 con 0x7ff90c006a60 2023-11-28T11:08:27.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff921ffb700 1 -- 172.21.15.28:0/3998174154 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9240ceff0 con 0x7ff90c005c60 2023-11-28T11:08:27.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff921ffb700 1 -- 172.21.15.28:0/3998174154 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff90c00e320 con 0x7ff90c007410 2023-11-28T11:08:27.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff9227fc700 1 --2- 172.21.15.28:0/3998174154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff90c006a60 0x7ff90c006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff922ffd700 1 --2- 172.21.15.28:0/3998174154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff90c005c60 0x7ff90c006070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff9237fe700 1 --2- 172.21.15.28:0/3998174154 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff90c007410 0x7ff90c00dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff9227fc700 1 -- 172.21.15.28:0/3998174154 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff90c007410 msgr2=0x7ff90c00dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:27.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff9227fc700 1 --2- 172.21.15.28:0/3998174154 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff90c007410 0x7ff90c00dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff9227fc700 1 -- 172.21.15.28:0/3998174154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff90c005c60 msgr2=0x7ff90c006070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:27.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff9227fc700 1 --2- 172.21.15.28:0/3998174154 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff90c005c60 0x7ff90c006070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff9227fc700 1 -- 172.21.15.28:0/3998174154 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff90c0057c0 con 0x7ff90c006a60 2023-11-28T11:08:27.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff9237fe700 1 --2- 172.21.15.28:0/3998174154 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff90c007410 0x7ff90c00dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:27.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff9237fe700 1 -- 172.21.15.28:0/3998174154 reap_dead start 2023-11-28T11:08:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff9227fc700 1 --2- 172.21.15.28:0/3998174154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff90c006a60 0x7ff90c006ed0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff918009660 tx=0x7ff90c007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15f5a1ef0b161a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff921ffb700 1 -- 172.21.15.28:0/3998174154 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918004230 con 0x7ff90c006a60 2023-11-28T11:08:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff921ffb700 1 -- 172.21.15.28:0/3998174154 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9180043b0 con 0x7ff90c006a60 2023-11-28T11:08:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.353+0000 7ff921ffb700 1 -- 172.21.15.28:0/3998174154 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff918013950 con 0x7ff90c006a60 2023-11-28T11:08:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.354+0000 7ff929a35700 1 -- 172.21.15.28:0/3998174154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff90c006a60 msgr2=0x7ff90c006ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.354+0000 7ff929a35700 1 --2- 172.21.15.28:0/3998174154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff90c006a60 0x7ff90c006ed0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff918009660 tx=0x7ff90c007a90 comp rx=0 tx=0).stop 2023-11-28T11:08:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.354+0000 7ff929a35700 1 -- 172.21.15.28:0/3998174154 shutdown_connections 2023-11-28T11:08:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.354+0000 7ff929a35700 1 --2- 172.21.15.28:0/3998174154 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff90c006a60 0x7ff90c006ed0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.354+0000 7ff929a35700 1 -- 172.21.15.28:0/3998174154 >> 172.21.15.28:0/3998174154 conn(0x7ff9240cc630 msgr2=0x7ff9240cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:27.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.354+0000 7ff929a35700 1 -- 172.21.15.28:0/3998174154 shutdown_connections 2023-11-28T11:08:27.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.354+0000 7ff929a35700 1 -- 172.21.15.28:0/3998174154 wait complete. 2023-11-28T11:08:27.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.356+0000 7ff929a35700 1 Processor -- start 2023-11-28T11:08:27.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.356+0000 7ff929a35700 1 -- start start 2023-11-28T11:08:27.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.357+0000 7ff929a35700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff90c006a60 0x7ff924150bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.357+0000 7ff929a35700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff924152dc0 0x7ff924150370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.357+0000 7ff929a35700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9240b4cd0 0x7ff9240b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:27.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.357+0000 7ff929a35700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9240b5680 con 0x7ff9240b4cd0 2023-11-28T11:08:27.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.357+0000 7ff929a35700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff924152020 con 0x7ff90c006a60 2023-11-28T11:08:27.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.357+0000 7ff9237fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9240b4cd0 0x7ff9240b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.357+0000 7ff922ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff90c006a60 0x7ff924150bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.357+0000 7ff9227fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff924152dc0 0x7ff924150370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:27.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.357+0000 7ff922ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff90c006a60 0x7ff924150bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:33088/0 (socket says 172.21.15.28:33088) 2023-11-28T11:08:27.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.357+0000 7ff9237fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9240b4cd0 0x7ff9240b5140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45802/0 (socket says 172.21.15.28:45802) 2023-11-28T11:08:27.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.357+0000 7ff9227fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff924152dc0 0x7ff924150370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:51558/0 (socket says 172.21.15.28:51558) 2023-11-28T11:08:27.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.357+0000 7ff922ffd700 1 -- 172.21.15.28:0/1627278356 learned_addr learned my addr 172.21.15.28:0/1627278356 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:27.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.357+0000 7ff929a35700 1 -- 172.21.15.28:0/1627278356 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff924151410 con 0x7ff924152dc0 2023-11-28T11:08:27.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.357+0000 7ff9237fe700 1 -- 172.21.15.28:0/1627278356 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff924152dc0 msgr2=0x7ff924150370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.357+0000 7ff9237fe700 1 --2- 172.21.15.28:0/1627278356 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff924152dc0 0x7ff924150370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.358+0000 7ff9237fe700 1 -- 172.21.15.28:0/1627278356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff90c006a60 msgr2=0x7ff924150bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.358+0000 7ff9237fe700 1 --2- 172.21.15.28:0/1627278356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff90c006a60 0x7ff924150bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.358+0000 7ff9237fe700 1 -- 172.21.15.28:0/1627278356 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff92415d3f0 con 0x7ff9240b4cd0 2023-11-28T11:08:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.358+0000 7ff9237fe700 1 --2- 172.21.15.28:0/1627278356 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9240b4cd0 0x7ff9240b5140 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff924070f30 tx=0x7ff91c001ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.358+0000 7ff90bfff700 1 -- 172.21.15.28:0/1627278356 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c003c10 con 0x7ff9240b4cd0 2023-11-28T11:08:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.358+0000 7ff90bfff700 1 -- 172.21.15.28:0/1627278356 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91c003d70 con 0x7ff9240b4cd0 2023-11-28T11:08:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.358+0000 7ff90bfff700 1 -- 172.21.15.28:0/1627278356 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c005310 con 0x7ff9240b4cd0 2023-11-28T11:08:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.358+0000 7ff929a35700 1 -- 172.21.15.28:0/1627278356 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff92415d110 con 0x7ff9240b4cd0 2023-11-28T11:08:27.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.358+0000 7ff929a35700 1 -- 172.21.15.28:0/1627278356 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff92415d6d0 con 0x7ff9240b4cd0 2023-11-28T11:08:27.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.358+0000 7ff90bfff700 1 -- 172.21.15.28:0/1627278356 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ff91c005760 con 0x7ff9240b4cd0 2023-11-28T11:08:27.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.358+0000 7ff90bfff700 1 -- 172.21.15.28:0/1627278356 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff91c01d460 con 0x7ff9240b4cd0 2023-11-28T11:08:27.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.359+0000 7ff929a35700 1 -- 172.21.15.28:0/1627278356 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9240739f0 con 0x7ff9240b4cd0 2023-11-28T11:08:27.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.361+0000 7ff90bfff700 1 -- 172.21.15.28:0/1627278356 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7ff91c00c5e0 con 0x7ff9240b4cd0 2023-11-28T11:08:27.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.439+0000 7ff929a35700 1 -- 172.21.15.28:0/1627278356 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7ff924152160 con 0x7ff9240b4cd0 2023-11-28T11:08:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.441+0000 7ff90bfff700 1 -- 172.21.15.28:0/1627278356 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7ff91c00c780 con 0x7ff9240b4cd0 2023-11-28T11:08:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.442+0000 7ff929a35700 1 -- 172.21.15.28:0/1627278356 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9240b4cd0 msgr2=0x7ff9240b5140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.442+0000 7ff929a35700 1 --2- 172.21.15.28:0/1627278356 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9240b4cd0 0x7ff9240b5140 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff924070f30 tx=0x7ff91c001ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.442+0000 7ff929a35700 1 -- 172.21.15.28:0/1627278356 shutdown_connections 2023-11-28T11:08:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.442+0000 7ff929a35700 1 --2- 172.21.15.28:0/1627278356 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9240b4cd0 0x7ff9240b5140 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.442+0000 7ff929a35700 1 --2- 172.21.15.28:0/1627278356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff90c006a60 0x7ff924150bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.442+0000 7ff929a35700 1 --2- 172.21.15.28:0/1627278356 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff924152dc0 0x7ff924150370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.442+0000 7ff929a35700 1 -- 172.21.15.28:0/1627278356 >> 172.21.15.28:0/1627278356 conn(0x7ff9240cc630 msgr2=0x7ff9240afa10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.442+0000 7ff929a35700 1 -- 172.21.15.28:0/1627278356 shutdown_connections 2023-11-28T11:08:27.444 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:27.442+0000 7ff929a35700 1 -- 172.21.15.28:0/1627278356 wait complete. 2023-11-28T11:08:27.444 INFO:teuthology.orchestra.run.smithi028.stderr:adjusted tunables profile to default 2023-11-28T11:08:27.455 INFO:tasks.ceph:check_enable_crimson: False 2023-11-28T11:08:27.455 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-28T11:08:27.455 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:08:27.456 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-28T11:08:27.474 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:27.474+0000 7f23534f3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T11:08:27.475 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:27.475+0000 7f922613f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-11-28T11:08:27.488 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:08:27.488 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-28T11:08:27.543 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:27.543+0000 7f23534f3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T11:08:27.545 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:27.545+0000 7f922613f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T11:08:27.556 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:08:27.556 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-28T11:08:27.621 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:27.621+0000 7f23534f3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T11:08:27.625 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:08:27.625 DEBUG:teuthology.orchestra.run.smithi028:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-28T11:08:27.630 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:27.629+0000 7f922613f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T11:08:27.657 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:08:27.657 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-28T11:08:27.684 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:08:27.684 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-28T11:08:27.707 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:27.707+0000 7f23534f3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T11:08:27.716 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:27.716+0000 7f922613f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T11:08:27.734 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:08:27.734 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-28T11:08:27.801 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:08:27.802 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-28T11:08:27.861 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:27.861+0000 7f23534f3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T11:08:27.866 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:27.866+0000 7f922613f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T11:08:27.869 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new 5fc20989-bed6-49f3-875f-71acbd109417 0 2023-11-28T11:08:28.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.034+0000 7fa6f05d9700 1 Processor -- start 2023-11-28T11:08:28.035 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.034+0000 7fa6f05d9700 1 -- start start 2023-11-28T11:08:28.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.035+0000 7fa6f05d9700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa6e80b62d0 0x7fa6e80cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.035+0000 7fa6f05d9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa6e806f8b0 0x7fa6e80b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.035+0000 7fa6f05d9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e80b2fe0 con 0x7fa6e80b2bc0 2023-11-28T11:08:28.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.035+0000 7fa6f05d9700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e80cd2a0 con 0x7fa6e80b62d0 2023-11-28T11:08:28.037 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.035+0000 7fa6f05d9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e80d0400 con 0x7fa6e806f8b0 2023-11-28T11:08:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.035+0000 7fa6edb74700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa6e806f8b0 0x7fa6e80b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.035+0000 7fa6eeb76700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa6e80b2bc0 0x7fa6e80ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54914/0 (socket says 172.21.15.59:54914) 2023-11-28T11:08:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.035+0000 7fa6eeb76700 1 -- 172.21.15.59:0/4280651940 learned_addr learned my addr 172.21.15.59:0/4280651940 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-28T11:08:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.035+0000 7fa6ee375700 1 --2- 172.21.15.59:0/4280651940 >> v2:172.21.15.28:3300/0 conn(0x7fa6e80b62d0 0x7fa6e80cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.036+0000 7fa6edb74700 1 -- 172.21.15.59:0/4280651940 >> v2:172.21.15.28:3300/0 conn(0x7fa6e80b62d0 msgr2=0x7fa6e80cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.036+0000 7fa6edb74700 1 --2- 172.21.15.59:0/4280651940 >> v2:172.21.15.28:3300/0 conn(0x7fa6e80b62d0 0x7fa6e80cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.036+0000 7fa6edb74700 1 -- 172.21.15.59:0/4280651940 >> v1:172.21.15.59:6789/0 conn(0x7fa6e80b2bc0 legacy=0x7fa6e80ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.036+0000 7fa6ed373700 1 -- 172.21.15.59:0/4280651940 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3696518984 0 0) 0x7fa6e80b2fe0 con 0x7fa6e80b2bc0 2023-11-28T11:08:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.036+0000 7fa6edb74700 1 -- 172.21.15.59:0/4280651940 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e80d06e0 con 0x7fa6e806f8b0 2023-11-28T11:08:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.036+0000 7fa6edb74700 1 --2- 172.21.15.59:0/4280651940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e806f8b0 0x7fa6e80b2680 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80cc8e0 tx=0x7fa6d8009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b06812d23fe0a37 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.036+0000 7fa6ed373700 1 -- 172.21.15.59:0/4280651940 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8011830 con 0x7fa6e806f8b0 2023-11-28T11:08:28.038 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.036+0000 7fa6ed373700 1 -- 172.21.15.59:0/4280651940 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d8011990 con 0x7fa6e806f8b0 2023-11-28T11:08:28.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.036+0000 7fa6ed373700 1 -- 172.21.15.59:0/4280651940 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8011d20 con 0x7fa6e806f8b0 2023-11-28T11:08:28.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.037+0000 7fa6f05d9700 1 -- 172.21.15.59:0/4280651940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e806f8b0 msgr2=0x7fa6e80b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.037+0000 7fa6f05d9700 1 --2- 172.21.15.59:0/4280651940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e806f8b0 0x7fa6e80b2680 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fa6e80cc8e0 tx=0x7fa6d8009b20 comp rx=0 tx=0).stop 2023-11-28T11:08:28.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.037+0000 7fa6f05d9700 1 -- 172.21.15.59:0/4280651940 shutdown_connections 2023-11-28T11:08:28.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.037+0000 7fa6f05d9700 1 --2- 172.21.15.59:0/4280651940 >> v2:172.21.15.28:3300/0 conn(0x7fa6e80b62d0 0x7fa6e80cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.037+0000 7fa6f05d9700 1 --2- 172.21.15.59:0/4280651940 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e806f8b0 0x7fa6e80b2680 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.037+0000 7fa6f05d9700 1 -- 172.21.15.59:0/4280651940 >> 172.21.15.59:0/4280651940 conn(0x7fa6e80cd680 msgr2=0x7fa6e80cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.037+0000 7fa6f05d9700 1 -- 172.21.15.59:0/4280651940 shutdown_connections 2023-11-28T11:08:28.039 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.037+0000 7fa6f05d9700 1 -- 172.21.15.59:0/4280651940 wait complete. 2023-11-28T11:08:28.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.038+0000 7fa6f05d9700 1 Processor -- start 2023-11-28T11:08:28.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.038+0000 7fa6f05d9700 1 -- start start 2023-11-28T11:08:28.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6f05d9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6e806f8b0 0x7fa6e80c4730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6f05d9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6e80b2bc0 0x7fa6e814afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6f05d9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e80b62d0 0x7fa6e8157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6f05d9700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e81f46b0 con 0x7fa6e806f8b0 2023-11-28T11:08:28.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6f05d9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e81f3a30 con 0x7fa6e80b62d0 2023-11-28T11:08:28.042 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6f05d9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e81f3d10 con 0x7fa6e80b2bc0 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6eeb76700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e80b62d0 0x7fa6e8157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6edb74700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6e80b2bc0 0x7fa6e814afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6eeb76700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e80b62d0 0x7fa6e8157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53522/0 (socket says 172.21.15.59:53522) 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6eeb76700 1 -- 172.21.15.59:0/2135488260 learned_addr learned my addr 172.21.15.59:0/2135488260 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6ee375700 1 --2- 172.21.15.59:0/2135488260 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6e806f8b0 0x7fa6e80c4730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6eeb76700 1 -- 172.21.15.59:0/2135488260 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6e80b2bc0 msgr2=0x7fa6e814afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6eeb76700 1 --2- 172.21.15.59:0/2135488260 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6e80b2bc0 0x7fa6e814afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6eeb76700 1 -- 172.21.15.59:0/2135488260 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6e806f8b0 msgr2=0x7fa6e80c4730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6eeb76700 1 --2- 172.21.15.59:0/2135488260 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6e806f8b0 0x7fa6e80c4730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6eeb76700 1 -- 172.21.15.59:0/2135488260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d80094a0 con 0x7fa6e80b62d0 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.039+0000 7fa6eeb76700 1 --2- 172.21.15.59:0/2135488260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e80b62d0 0x7fa6e8157a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0002230 tx=0x7fa6e000a3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.043 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.040+0000 7fa6deffd700 1 -- 172.21.15.59:0/2135488260 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e00070d0 con 0x7fa6e80b62d0 2023-11-28T11:08:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.040+0000 7fa6deffd700 1 -- 172.21.15.59:0/2135488260 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6e0007250 con 0x7fa6e80b62d0 2023-11-28T11:08:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.040+0000 7fa6f05d9700 1 -- 172.21.15.59:0/2135488260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e806fd30 con 0x7fa6e80b62d0 2023-11-28T11:08:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.040+0000 7fa6deffd700 1 -- 172.21.15.59:0/2135488260 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e0007600 con 0x7fa6e80b62d0 2023-11-28T11:08:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.040+0000 7fa6f05d9700 1 -- 172.21.15.59:0/2135488260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e80cffb0 con 0x7fa6e80b62d0 2023-11-28T11:08:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.040+0000 7fa6deffd700 1 -- 172.21.15.59:0/2135488260 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fa6e0023360 con 0x7fa6e80b62d0 2023-11-28T11:08:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.040+0000 7fa6deffd700 1 -- 172.21.15.59:0/2135488260 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa6e0007a50 con 0x7fa6e80b62d0 2023-11-28T11:08:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.041+0000 7fa6f05d9700 1 -- 172.21.15.59:0/2135488260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e80b66a0 con 0x7fa6e80b62d0 2023-11-28T11:08:28.044 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.043+0000 7fa6deffd700 1 -- 172.21.15.59:0/2135488260 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fa6e0039b00 con 0x7fa6e80b62d0 2023-11-28T11:08:28.128 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.128+0000 7fa6f05d9700 1 -- 172.21.15.59:0/2135488260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5fc20989-bed6-49f3-875f-71acbd109417", "id": 0} v 0) v1 -- 0x7fa6e81f47f0 con 0x7fa6e80b62d0 2023-11-28T11:08:28.131 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.131+0000 7fa6deffd700 1 -- 172.21.15.59:0/2135488260 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5fc20989-bed6-49f3-875f-71acbd109417", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa6e81f47f0 con 0x7fa6e80b62d0 2023-11-28T11:08:28.132 INFO:teuthology.orchestra.run.smithi059.stdout:0 2023-11-28T11:08:28.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.132+0000 7fa6f05d9700 1 -- 172.21.15.59:0/2135488260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e80b62d0 msgr2=0x7fa6e8157a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.132+0000 7fa6f05d9700 1 --2- 172.21.15.59:0/2135488260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e80b62d0 0x7fa6e8157a80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0002230 tx=0x7fa6e000a3c0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.132+0000 7fa6f05d9700 1 -- 172.21.15.59:0/2135488260 shutdown_connections 2023-11-28T11:08:28.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.132+0000 7fa6f05d9700 1 --2- 172.21.15.59:0/2135488260 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6e806f8b0 0x7fa6e80c4730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.132+0000 7fa6f05d9700 1 --2- 172.21.15.59:0/2135488260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6e80b62d0 0x7fa6e8157a80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.132+0000 7fa6f05d9700 1 --2- 172.21.15.59:0/2135488260 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6e80b2bc0 0x7fa6e814afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.132+0000 7fa6f05d9700 1 -- 172.21.15.59:0/2135488260 >> 172.21.15.59:0/2135488260 conn(0x7fa6e80cd680 msgr2=0x7fa6e80b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.133 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.132+0000 7fa6f05d9700 1 -- 172.21.15.59:0/2135488260 shutdown_connections 2023-11-28T11:08:28.134 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.132+0000 7fa6f05d9700 1 -- 172.21.15.59:0/2135488260 wait complete. 2023-11-28T11:08:28.144 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new 7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7 1 2023-11-28T11:08:28.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.267+0000 7f6743d78700 1 Processor -- start 2023-11-28T11:08:28.268 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.267+0000 7f6743d78700 1 -- start start 2023-11-28T11:08:28.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.268+0000 7f6743d78700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f673c165670 0x7f673c167a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.268+0000 7f6743d78700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f673c0ce550 con 0x7f673c0cf5f0 2023-11-28T11:08:28.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.268+0000 7f6743d78700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f673c0ce830 con 0x7f673c0b1630 2023-11-28T11:08:28.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.268+0000 7f6743d78700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673c0ceb10 con 0x7f673c165670 2023-11-28T11:08:28.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.268+0000 7f6741b14700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f673c0b1630 0x7f673c0b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54922/0 (socket says 172.21.15.59:54922) 2023-11-28T11:08:28.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.268+0000 7f6742315700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f673c165670 0x7f673c167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.268+0000 7f6741b14700 1 -- 172.21.15.59:0/2167680272 learned_addr learned my addr 172.21.15.59:0/2167680272 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-28T11:08:28.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.269+0000 7f6740b12700 1 -- 172.21.15.59:0/2167680272 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125587253 0 0) 0x7f673c0ce830 con 0x7f673c0b1630 2023-11-28T11:08:28.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.269+0000 7f6740b12700 1 -- 172.21.15.59:0/2167680272 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67280035b0 con 0x7f673c0b1630 2023-11-28T11:08:28.270 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.269+0000 7f6742315700 1 -- 172.21.15.59:0/2167680272 >> v1:172.21.15.59:6789/0 conn(0x7f673c0b1630 legacy=0x7f673c0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.269+0000 7f6742315700 1 -- 172.21.15.59:0/2167680272 >> v1:172.21.15.28:6789/0 conn(0x7f673c0cf5f0 legacy=0x7f673c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.269+0000 7f6740b12700 1 -- 172.21.15.59:0/2167680272 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2470703010 0 0) 0x7f673c0ce550 con 0x7f673c0cf5f0 2023-11-28T11:08:28.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.269+0000 7f6742315700 1 -- 172.21.15.59:0/2167680272 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f673c0cedf0 con 0x7f673c165670 2023-11-28T11:08:28.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.269+0000 7f6742315700 1 --2- 172.21.15.59:0/2167680272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f673c165670 0x7f673c167a50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6734009df0 tx=0x7f673400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b04a718f0dad810 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.269+0000 7f6740b12700 1 -- 172.21.15.59:0/2167680272 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400e070 con 0x7f673c165670 2023-11-28T11:08:28.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.269+0000 7f6740b12700 1 -- 172.21.15.59:0/2167680272 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673400bae0 con 0x7f673c165670 2023-11-28T11:08:28.271 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.269+0000 7f6740b12700 1 -- 172.21.15.59:0/2167680272 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400be90 con 0x7f673c165670 2023-11-28T11:08:28.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.270+0000 7f6743d78700 1 -- 172.21.15.59:0/2167680272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f673c165670 msgr2=0x7f673c167a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.270+0000 7f6743d78700 1 --2- 172.21.15.59:0/2167680272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f673c165670 0x7f673c167a50 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6734009df0 tx=0x7f673400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:28.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.270+0000 7f6743d78700 1 -- 172.21.15.59:0/2167680272 shutdown_connections 2023-11-28T11:08:28.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.270+0000 7f6743d78700 1 --2- 172.21.15.59:0/2167680272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f673c165670 0x7f673c167a50 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.270+0000 7f6743d78700 1 -- 172.21.15.59:0/2167680272 >> 172.21.15.59:0/2167680272 conn(0x7f673c0cc630 msgr2=0x7f673c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.270+0000 7f6743d78700 1 -- 172.21.15.59:0/2167680272 shutdown_connections 2023-11-28T11:08:28.274 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.270+0000 7f6743d78700 1 -- 172.21.15.59:0/2167680272 wait complete. 2023-11-28T11:08:28.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.271+0000 7f6743d78700 1 Processor -- start 2023-11-28T11:08:28.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.271+0000 7f6743d78700 1 -- start start 2023-11-28T11:08:28.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6743d78700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673c0b1630 0x7f673c1535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6743d78700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f673c0cf5f0 0x7f673c0b2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6743d78700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f673c165670 0x7f673c1f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6743d78700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c0c8f40 con 0x7f673c0cf5f0 2023-11-28T11:08:28.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6743d78700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673c0c9220 con 0x7f673c0b1630 2023-11-28T11:08:28.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6743d78700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f673c0b46b0 con 0x7f673c165670 2023-11-28T11:08:28.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6742315700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f673c165670 0x7f673c1f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6741b14700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673c0b1630 0x7f673c1535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.275 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6741313700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f673c0cf5f0 0x7f673c0b2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6742315700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f673c165670 0x7f673c1f60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:44472/0 (socket says 172.21.15.59:44472) 2023-11-28T11:08:28.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6741b14700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673c0b1630 0x7f673c1535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53528/0 (socket says 172.21.15.59:53528) 2023-11-28T11:08:28.276 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6742315700 1 -- 172.21.15.59:0/3928105416 learned_addr learned my addr 172.21.15.59:0/3928105416 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-28T11:08:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6742315700 1 -- 172.21.15.59:0/3928105416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673c0b1630 msgr2=0x7f673c1535f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6742315700 1 --2- 172.21.15.59:0/3928105416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673c0b1630 0x7f673c1535f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6742315700 1 -- 172.21.15.59:0/3928105416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f673c0cf5f0 msgr2=0x7f673c0b2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6742315700 1 --2- 172.21.15.59:0/3928105416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f673c0cf5f0 0x7f673c0b2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6742315700 1 -- 172.21.15.59:0/3928105416 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6734009a70 con 0x7f673c165670 2023-11-28T11:08:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6741b14700 1 --2- 172.21.15.59:0/3928105416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673c0b1630 0x7f673c1535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.272+0000 7f6741313700 1 --2- 172.21.15.59:0/3928105416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f673c0cf5f0 0x7f673c0b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.273+0000 7f6742315700 1 --2- 172.21.15.59:0/3928105416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f673c165670 0x7f673c1f60a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6734014040 tx=0x7f6734004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.273+0000 7f67327fc700 1 -- 172.21.15.59:0/3928105416 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6734012bf0 con 0x7f673c165670 2023-11-28T11:08:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.273+0000 7f67327fc700 1 -- 172.21.15.59:0/3928105416 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6734012d70 con 0x7f673c165670 2023-11-28T11:08:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.273+0000 7f6743d78700 1 -- 172.21.15.59:0/3928105416 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673c0cb5b0 con 0x7f673c165670 2023-11-28T11:08:28.277 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.273+0000 7f67327fc700 1 -- 172.21.15.59:0/3928105416 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6734026630 con 0x7f673c165670 2023-11-28T11:08:28.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.273+0000 7f6743d78700 1 -- 172.21.15.59:0/3928105416 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673c0cf0d0 con 0x7f673c165670 2023-11-28T11:08:28.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.273+0000 7f67327fc700 1 -- 172.21.15.59:0/3928105416 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f673402c060 con 0x7f673c165670 2023-11-28T11:08:28.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.273+0000 7f67327fc700 1 -- 172.21.15.59:0/3928105416 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f673402eac0 con 0x7f673c165670 2023-11-28T11:08:28.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.274+0000 7f6743d78700 1 -- 172.21.15.59:0/3928105416 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673c0ce140 con 0x7f673c165670 2023-11-28T11:08:28.278 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.276+0000 7f67327fc700 1 -- 172.21.15.59:0/3928105416 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f673402ec80 con 0x7f673c165670 2023-11-28T11:08:28.364 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.363+0000 7f6743d78700 1 -- 172.21.15.59:0/3928105416 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7", "id": 1} v 0) v1 -- 0x7f673c067150 con 0x7f673c165670 2023-11-28T11:08:28.366 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.366+0000 7f67327fc700 1 -- 172.21.15.59:0/3928105416 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f673402ee20 con 0x7f673c165670 2023-11-28T11:08:28.367 INFO:teuthology.orchestra.run.smithi059.stdout:1 2023-11-28T11:08:28.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.367+0000 7f6743d78700 1 -- 172.21.15.59:0/3928105416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f673c165670 msgr2=0x7f673c1f60a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.367+0000 7f6743d78700 1 --2- 172.21.15.59:0/3928105416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f673c165670 0x7f673c1f60a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6734014040 tx=0x7f6734004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.367+0000 7f6743d78700 1 -- 172.21.15.59:0/3928105416 shutdown_connections 2023-11-28T11:08:28.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.367+0000 7f6743d78700 1 --2- 172.21.15.59:0/3928105416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f673c165670 0x7f673c1f60a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.367+0000 7f6743d78700 1 --2- 172.21.15.59:0/3928105416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f673c0b1630 0x7f673c1535f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.367+0000 7f6743d78700 1 --2- 172.21.15.59:0/3928105416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f673c0cf5f0 0x7f673c0b2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.367+0000 7f6743d78700 1 -- 172.21.15.59:0/3928105416 >> 172.21.15.59:0/3928105416 conn(0x7f673c0cc630 msgr2=0x7f673c0cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.367+0000 7f6743d78700 1 -- 172.21.15.59:0/3928105416 shutdown_connections 2023-11-28T11:08:28.368 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.367+0000 7f6743d78700 1 -- 172.21.15.59:0/3928105416 wait complete. 2023-11-28T11:08:28.379 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new f65f7a5b-99af-48a6-92ac-01403f42b390 2 2023-11-28T11:08:28.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.503+0000 7f84feba5700 1 Processor -- start 2023-11-28T11:08:28.504 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.504+0000 7f84feba5700 1 -- start start 2023-11-28T11:08:28.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.504+0000 7f84feba5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84f80cf5f0 0x7f84f80caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.506 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.504+0000 7f84feba5700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f84f80cb3e0 0x7f84f806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.504+0000 7f84feba5700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84f80ce550 con 0x7f84f80b1630 2023-11-28T11:08:28.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.504+0000 7f84feba5700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84f80cfa60 con 0x7f84f80cb3e0 2023-11-28T11:08:28.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.504+0000 7f84feba5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f80ce830 con 0x7f84f80cf5f0 2023-11-28T11:08:28.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.504+0000 7f84f7fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84f80cf5f0 0x7f84f80caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.505+0000 7f84f7fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84f80cf5f0 0x7f84f80caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:44476/0 (socket says 172.21.15.59:44476) 2023-11-28T11:08:28.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.505+0000 7f84f7fff700 1 -- 172.21.15.59:0/3469903208 learned_addr learned my addr 172.21.15.59:0/3469903208 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-28T11:08:28.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.505+0000 7f84fd142700 1 --2- 172.21.15.59:0/3469903208 >> v2:172.21.15.28:3300/0 conn(0x7f84f80cb3e0 0x7f84f806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.505+0000 7f84f7fff700 1 -- 172.21.15.59:0/3469903208 >> v2:172.21.15.28:3300/0 conn(0x7f84f80cb3e0 msgr2=0x7f84f806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.505+0000 7f84f7fff700 1 --2- 172.21.15.59:0/3469903208 >> v2:172.21.15.28:3300/0 conn(0x7f84f80cb3e0 0x7f84f806afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.505+0000 7f84f7fff700 1 -- 172.21.15.59:0/3469903208 >> v1:172.21.15.28:6789/0 conn(0x7f84f80b1630 legacy=0x7f84f80b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.507 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.505+0000 7f84f7fff700 1 -- 172.21.15.59:0/3469903208 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84f80ceb10 con 0x7f84f80cf5f0 2023-11-28T11:08:28.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.505+0000 7f84f7fff700 1 --2- 172.21.15.59:0/3469903208 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84f80cf5f0 0x7f84f80caea0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f84e40098f0 tx=0x7f84e4009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86a5fadb5b5ae9f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.505+0000 7f84f77fe700 1 -- 172.21.15.59:0/3469903208 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4011830 con 0x7f84f80cf5f0 2023-11-28T11:08:28.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.505+0000 7f84f77fe700 1 -- 172.21.15.59:0/3469903208 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84e4011990 con 0x7f84f80cf5f0 2023-11-28T11:08:28.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.505+0000 7f84f77fe700 1 -- 172.21.15.59:0/3469903208 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84e4011d20 con 0x7f84f80cf5f0 2023-11-28T11:08:28.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.506+0000 7f84feba5700 1 -- 172.21.15.59:0/3469903208 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84f80cf5f0 msgr2=0x7f84f80caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.506+0000 7f84feba5700 1 --2- 172.21.15.59:0/3469903208 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84f80cf5f0 0x7f84f80caea0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f84e40098f0 tx=0x7f84e4009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:28.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.506+0000 7f84feba5700 1 -- 172.21.15.59:0/3469903208 shutdown_connections 2023-11-28T11:08:28.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.506+0000 7f84feba5700 1 --2- 172.21.15.59:0/3469903208 >> v2:172.21.15.28:3300/0 conn(0x7f84f80cb3e0 0x7f84f806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.506+0000 7f84feba5700 1 --2- 172.21.15.59:0/3469903208 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84f80cf5f0 0x7f84f80caea0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.506+0000 7f84feba5700 1 -- 172.21.15.59:0/3469903208 >> 172.21.15.59:0/3469903208 conn(0x7f84f80cc630 msgr2=0x7f84f80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.506+0000 7f84feba5700 1 -- 172.21.15.59:0/3469903208 shutdown_connections 2023-11-28T11:08:28.508 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.506+0000 7f84feba5700 1 -- 172.21.15.59:0/3469903208 wait complete. 2023-11-28T11:08:28.511 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.507+0000 7f84feba5700 1 Processor -- start 2023-11-28T11:08:28.511 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.507+0000 7f84feba5700 1 -- start start 2023-11-28T11:08:28.511 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.508+0000 7f84feba5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f80b1630 0x7f84f8153160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.511 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.508+0000 7f84feba5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84f80cb3e0 0x7f84f80c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.511 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.508+0000 7f84feba5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84f80cf5f0 0x7f84f80c9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.511 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.508+0000 7f84feba5700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f81536a0 con 0x7f84f80cb3e0 2023-11-28T11:08:28.511 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.508+0000 7f84feba5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84f80c6120 con 0x7f84f80b1630 2023-11-28T11:08:28.511 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.508+0000 7f84feba5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84f80abfe0 con 0x7f84f80cf5f0 2023-11-28T11:08:28.511 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.508+0000 7f84fd142700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84f80cf5f0 0x7f84f80c9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.511 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.508+0000 7f84fc941700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f80b1630 0x7f84f8153160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.508+0000 7f84fd142700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84f80cf5f0 0x7f84f80c9030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:44484/0 (socket says 172.21.15.59:44484) 2023-11-28T11:08:28.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.508+0000 7f84fd142700 1 -- 172.21.15.59:0/428645112 learned_addr learned my addr 172.21.15.59:0/428645112 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-28T11:08:28.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.508+0000 7f84f7fff700 1 --2- 172.21.15.59:0/428645112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84f80cb3e0 0x7f84f80c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.508+0000 7f84fc941700 1 -- 172.21.15.59:0/428645112 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84f80cf5f0 msgr2=0x7f84f80c9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.508+0000 7f84fc941700 1 --2- 172.21.15.59:0/428645112 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84f80cf5f0 0x7f84f80c9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.508+0000 7f84fc941700 1 -- 172.21.15.59:0/428645112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84f80cb3e0 msgr2=0x7f84f80c8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.508+0000 7f84fc941700 1 --2- 172.21.15.59:0/428645112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84f80cb3e0 0x7f84f80c8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.509+0000 7f84fc941700 1 -- 172.21.15.59:0/428645112 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84e4009550 con 0x7f84f80b1630 2023-11-28T11:08:28.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.509+0000 7f84fd142700 1 --2- 172.21.15.59:0/428645112 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84f80cf5f0 0x7f84f80c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:28.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.509+0000 7f84f7fff700 1 --2- 172.21.15.59:0/428645112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84f80cb3e0 0x7f84f80c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:28.512 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.509+0000 7f84fc941700 1 --2- 172.21.15.59:0/428645112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f80b1630 0x7f84f8153160 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f84ec000f60 tx=0x7f84ec00c730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.509+0000 7f84f57fa700 1 -- 172.21.15.59:0/428645112 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec0052d0 con 0x7f84f80b1630 2023-11-28T11:08:28.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.509+0000 7f84f57fa700 1 -- 172.21.15.59:0/428645112 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec005430 con 0x7f84f80b1630 2023-11-28T11:08:28.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.509+0000 7f84feba5700 1 -- 172.21.15.59:0/428645112 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84f80cf0d0 con 0x7f84f80b1630 2023-11-28T11:08:28.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.509+0000 7f84f57fa700 1 -- 172.21.15.59:0/428645112 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec0038c0 con 0x7f84f80b1630 2023-11-28T11:08:28.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.509+0000 7f84feba5700 1 -- 172.21.15.59:0/428645112 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84f80cedf0 con 0x7f84f80b1630 2023-11-28T11:08:28.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.509+0000 7f84f57fa700 1 -- 172.21.15.59:0/428645112 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f84ec019030 con 0x7f84f80b1630 2023-11-28T11:08:28.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.509+0000 7f84f57fa700 1 -- 172.21.15.59:0/428645112 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f84ec011b40 con 0x7f84f80b1630 2023-11-28T11:08:28.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.510+0000 7f84feba5700 1 -- 172.21.15.59:0/428645112 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84f80ce140 con 0x7f84f80b1630 2023-11-28T11:08:28.513 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.512+0000 7f84f57fa700 1 -- 172.21.15.59:0/428645112 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f84ec017070 con 0x7f84f80b1630 2023-11-28T11:08:28.588 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:28.588+0000 7f922613f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T11:08:28.594 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:28.594+0000 7f23534f3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T11:08:28.596 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.596+0000 7f84feba5700 1 -- 172.21.15.59:0/428645112 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f65f7a5b-99af-48a6-92ac-01403f42b390", "id": 2} v 0) v1 -- 0x7f84f80739f0 con 0x7f84f80b1630 2023-11-28T11:08:28.599 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.599+0000 7f84f57fa700 1 -- 172.21.15.59:0/428645112 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f65f7a5b-99af-48a6-92ac-01403f42b390", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f84ec011da0 con 0x7f84f80b1630 2023-11-28T11:08:28.599 INFO:teuthology.orchestra.run.smithi059.stdout:2 2023-11-28T11:08:28.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.600+0000 7f84feba5700 1 -- 172.21.15.59:0/428645112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f80b1630 msgr2=0x7f84f8153160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.600+0000 7f84feba5700 1 --2- 172.21.15.59:0/428645112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f80b1630 0x7f84f8153160 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f84ec000f60 tx=0x7f84ec00c730 comp rx=0 tx=0).stop 2023-11-28T11:08:28.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.600+0000 7f84feba5700 1 -- 172.21.15.59:0/428645112 shutdown_connections 2023-11-28T11:08:28.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.600+0000 7f84feba5700 1 --2- 172.21.15.59:0/428645112 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84f80cf5f0 0x7f84f80c9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.600+0000 7f84feba5700 1 --2- 172.21.15.59:0/428645112 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84f80b1630 0x7f84f8153160 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.601 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.600+0000 7f84feba5700 1 --2- 172.21.15.59:0/428645112 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f84f80cb3e0 0x7f84f80c8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.600+0000 7f84feba5700 1 -- 172.21.15.59:0/428645112 >> 172.21.15.59:0/428645112 conn(0x7f84f80cc630 msgr2=0x7f84f80b49f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.600+0000 7f84feba5700 1 -- 172.21.15.59:0/428645112 shutdown_connections 2023-11-28T11:08:28.602 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.600+0000 7f84feba5700 1 -- 172.21.15.59:0/428645112 wait complete. 2023-11-28T11:08:28.613 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new 33cd8375-45c3-49b6-a4bb-fdcc1b2e4511 3 2023-11-28T11:08:28.668 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:28.667+0000 7f922613f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T11:08:28.677 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:28.677+0000 7f23534f3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T11:08:28.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.736+0000 7fe360d31700 1 Processor -- start 2023-11-28T11:08:28.737 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.737+0000 7fe360d31700 1 -- start start 2023-11-28T11:08:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.738+0000 7fe360d31700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe35c0b1630 0x7fe35c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.738+0000 7fe360d31700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe35c0b1a00 0x7fe35c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.738+0000 7fe360d31700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe35c0ce550 con 0x7fe35c0cb3e0 2023-11-28T11:08:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.738+0000 7fe360d31700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe35c0ce830 con 0x7fe35c0b1a00 2023-11-28T11:08:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.738+0000 7fe360d31700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0ceb10 con 0x7fe35c0b1630 2023-11-28T11:08:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.738+0000 7fe35a59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe35c0b1630 0x7fe35c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.738+0000 7fe35ad9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe35c0cb3e0 0x7fe35c160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54948/0 (socket says 172.21.15.59:54948) 2023-11-28T11:08:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.738+0000 7fe35ad9d700 1 -- 172.21.15.59:0/1814884403 learned_addr learned my addr 172.21.15.59:0/1814884403 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-28T11:08:28.740 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.738+0000 7fe359d9b700 1 --2- 172.21.15.59:0/1814884403 >> v2:172.21.15.28:3300/0 conn(0x7fe35c0b1a00 0x7fe35c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.739+0000 7fe35959a700 1 -- 172.21.15.59:0/1814884403 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2931602872 0 0) 0x7fe35c0ce550 con 0x7fe35c0cb3e0 2023-11-28T11:08:28.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.739+0000 7fe35a59c700 1 -- 172.21.15.59:0/1814884403 >> v2:172.21.15.28:3300/0 conn(0x7fe35c0b1a00 msgr2=0x7fe35c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:28.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.739+0000 7fe35a59c700 1 --2- 172.21.15.59:0/1814884403 >> v2:172.21.15.28:3300/0 conn(0x7fe35c0b1a00 0x7fe35c0caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.739+0000 7fe35a59c700 1 -- 172.21.15.59:0/1814884403 >> v1:172.21.15.59:6789/0 conn(0x7fe35c0cb3e0 legacy=0x7fe35c160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.739+0000 7fe35a59c700 1 -- 172.21.15.59:0/1814884403 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe35c0cedf0 con 0x7fe35c0b1630 2023-11-28T11:08:28.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.739+0000 7fe35a59c700 1 --2- 172.21.15.59:0/1814884403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe35c0b1630 0x7fe35c0cf5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009840 tx=0x7fe34c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=faf4ac8b3f091661 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.739+0000 7fe35959a700 1 -- 172.21.15.59:0/1814884403 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c011830 con 0x7fe35c0b1630 2023-11-28T11:08:28.741 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.739+0000 7fe35959a700 1 -- 172.21.15.59:0/1814884403 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c011990 con 0x7fe35c0b1630 2023-11-28T11:08:28.742 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.739+0000 7fe35959a700 1 -- 172.21.15.59:0/1814884403 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c011d20 con 0x7fe35c0b1630 2023-11-28T11:08:28.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.740+0000 7fe360d31700 1 -- 172.21.15.59:0/1814884403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe35c0b1630 msgr2=0x7fe35c0cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.740+0000 7fe360d31700 1 --2- 172.21.15.59:0/1814884403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe35c0b1630 0x7fe35c0cf5f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009840 tx=0x7fe34c009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:28.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.740+0000 7fe360d31700 1 -- 172.21.15.59:0/1814884403 shutdown_connections 2023-11-28T11:08:28.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.740+0000 7fe360d31700 1 --2- 172.21.15.59:0/1814884403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe35c0b1630 0x7fe35c0cf5f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.740+0000 7fe360d31700 1 --2- 172.21.15.59:0/1814884403 >> v2:172.21.15.28:3300/0 conn(0x7fe35c0b1a00 0x7fe35c0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.743 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.740+0000 7fe360d31700 1 -- 172.21.15.59:0/1814884403 >> 172.21.15.59:0/1814884403 conn(0x7fe35c0cc630 msgr2=0x7fe35c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.740+0000 7fe360d31700 1 -- 172.21.15.59:0/1814884403 shutdown_connections 2023-11-28T11:08:28.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.740+0000 7fe360d31700 1 -- 172.21.15.59:0/1814884403 wait complete. 2023-11-28T11:08:28.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.742+0000 7fe360d31700 1 Processor -- start 2023-11-28T11:08:28.744 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.742+0000 7fe360d31700 1 -- start start 2023-11-28T11:08:28.747 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.743+0000 7fe360d31700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe35c0b1630 0x7fe35c15b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.743+0000 7fe360d31700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe35c0b1a00 0x7fe35c15bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.743+0000 7fe360d31700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe35c0cb3e0 0x7fe35c0b4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:28.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.743+0000 7fe360d31700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c0c36e0 con 0x7fe35c0cb3e0 2023-11-28T11:08:28.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.743+0000 7fe360d31700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c15cfd0 con 0x7fe35c0b1a00 2023-11-28T11:08:28.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.743+0000 7fe360d31700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe35c15d2b0 con 0x7fe35c0b1630 2023-11-28T11:08:28.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe35a59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe35c0b1630 0x7fe35c15b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe359d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe35c0b1a00 0x7fe35c15bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe35a59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe35c0b1630 0x7fe35c15b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:44492/0 (socket says 172.21.15.59:44492) 2023-11-28T11:08:28.748 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe35a59c700 1 -- 172.21.15.59:0/3263106366 learned_addr learned my addr 172.21.15.59:0/3263106366 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-28T11:08:28.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe35ad9d700 1 --2- 172.21.15.59:0/3263106366 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe35c0cb3e0 0x7fe35c0b4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:28.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe35a59c700 1 -- 172.21.15.59:0/3263106366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe35c0b1a00 msgr2=0x7fe35c15bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe35a59c700 1 --2- 172.21.15.59:0/3263106366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe35c0b1a00 0x7fe35c15bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe35a59c700 1 -- 172.21.15.59:0/3263106366 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe35c0cb3e0 msgr2=0x7fe35c0b4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe35a59c700 1 --2- 172.21.15.59:0/3263106366 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe35c0cb3e0 0x7fe35c0b4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe35a59c700 1 -- 172.21.15.59:0/3263106366 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c0094a0 con 0x7fe35c0b1630 2023-11-28T11:08:28.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe359d9b700 1 --2- 172.21.15.59:0/3263106366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe35c0b1a00 0x7fe35c15bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:28.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe35a59c700 1 --2- 172.21.15.59:0/3263106366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe35c0b1630 0x7fe35c15b270 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009fd0 tx=0x7fe34c004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:28.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe34affd700 1 -- 172.21.15.59:0/3263106366 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c0036a0 con 0x7fe35c0b1630 2023-11-28T11:08:28.749 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe34affd700 1 -- 172.21.15.59:0/3263106366 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34c01f070 con 0x7fe35c0b1630 2023-11-28T11:08:28.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe360d31700 1 -- 172.21.15.59:0/3263106366 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe35c0b1e70 con 0x7fe35c0b1630 2023-11-28T11:08:28.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe34affd700 1 -- 172.21.15.59:0/3263106366 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34c003a40 con 0x7fe35c0b1630 2023-11-28T11:08:28.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.744+0000 7fe360d31700 1 -- 172.21.15.59:0/3263106366 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe35c0cf0d0 con 0x7fe35c0b1630 2023-11-28T11:08:28.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.745+0000 7fe34affd700 1 -- 172.21.15.59:0/3263106366 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fe34c00f050 con 0x7fe35c0b1630 2023-11-28T11:08:28.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.745+0000 7fe34affd700 1 -- 172.21.15.59:0/3263106366 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7fe34c021070 con 0x7fe35c0b1630 2023-11-28T11:08:28.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.746+0000 7fe360d31700 1 -- 172.21.15.59:0/3263106366 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe35c0ce140 con 0x7fe35c0b1630 2023-11-28T11:08:28.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.748+0000 7fe34affd700 1 -- 172.21.15.59:0/3263106366 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fe34c046b00 con 0x7fe35c0b1630 2023-11-28T11:08:28.770 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:28.770+0000 7f922613f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T11:08:28.784 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:28.784+0000 7f23534f3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T11:08:28.853 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.853+0000 7fe360d31700 1 -- 172.21.15.59:0/3263106366 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd new", "uuid": "33cd8375-45c3-49b6-a4bb-fdcc1b2e4511", "id": 3} v 0) v1 -- 0x7fe35c15f1f0 con 0x7fe35c0b1630 2023-11-28T11:08:28.859 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.858+0000 7fe34affd700 1 -- 172.21.15.59:0/3263106366 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "33cd8375-45c3-49b6-a4bb-fdcc1b2e4511", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe34c035030 con 0x7fe35c0b1630 2023-11-28T11:08:28.859 INFO:teuthology.orchestra.run.smithi059.stdout:3 2023-11-28T11:08:28.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.859+0000 7fe360d31700 1 -- 172.21.15.59:0/3263106366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe35c0b1630 msgr2=0x7fe35c15b270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:28.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.859+0000 7fe360d31700 1 --2- 172.21.15.59:0/3263106366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe35c0b1630 0x7fe35c15b270 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe34c009fd0 tx=0x7fe34c004750 comp rx=0 tx=0).stop 2023-11-28T11:08:28.860 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.859+0000 7fe360d31700 1 -- 172.21.15.59:0/3263106366 shutdown_connections 2023-11-28T11:08:28.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.859+0000 7fe360d31700 1 --2- 172.21.15.59:0/3263106366 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe35c0cb3e0 0x7fe35c0b4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.859+0000 7fe360d31700 1 --2- 172.21.15.59:0/3263106366 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe35c0b1630 0x7fe35c15b270 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.859+0000 7fe360d31700 1 --2- 172.21.15.59:0/3263106366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe35c0b1a00 0x7fe35c15bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:28.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.859+0000 7fe360d31700 1 -- 172.21.15.59:0/3263106366 >> 172.21.15.59:0/3263106366 conn(0x7fe35c0cc630 msgr2=0x7fe35c06bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:28.861 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.859+0000 7fe360d31700 1 -- 172.21.15.59:0/3263106366 shutdown_connections 2023-11-28T11:08:28.862 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.859+0000 7fe360d31700 1 -- 172.21.15.59:0/3263106366 wait complete. 2023-11-28T11:08:28.872 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new 8405f8e6-e7cf-4db6-9379-f177a3ae6acb 4 2023-11-28T11:08:28.913 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:28.912+0000 7f922613f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T11:08:28.936 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:28.936+0000 7f23534f3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T11:08:28.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.995+0000 7f990bccc700 1 Processor -- start 2023-11-28T11:08:28.996 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.995+0000 7f990bccc700 1 -- start start 2023-11-28T11:08:28.999 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.996+0000 7f990bccc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f99040ce560 0x7f990406f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.996+0000 7f990bccc700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990406e470 con 0x7f9904158d90 2023-11-28T11:08:29.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.996+0000 7f990bccc700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990406de60 con 0x7f99041589c0 2023-11-28T11:08:29.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.996+0000 7f990bccc700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99040ceb40 con 0x7f99040ce560 2023-11-28T11:08:29.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.996+0000 7f990a269700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f99040ce560 0x7f990406f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.996+0000 7f9909a68700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f99041589c0 0x7f990406aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54956/0 (socket says 172.21.15.59:54956) 2023-11-28T11:08:29.000 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.996+0000 7f9909a68700 1 -- 172.21.15.59:0/3067020288 learned_addr learned my addr 172.21.15.59:0/3067020288 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-28T11:08:29.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.997+0000 7f990a269700 1 -- 172.21.15.59:0/3067020288 >> v1:172.21.15.59:6789/0 conn(0x7f99041589c0 legacy=0x7f990406aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.997+0000 7f990a269700 1 -- 172.21.15.59:0/3067020288 >> v1:172.21.15.28:6789/0 conn(0x7f9904158d90 legacy=0x7f990406e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.997+0000 7f990a269700 1 -- 172.21.15.59:0/3067020288 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99040cee20 con 0x7f99040ce560 2023-11-28T11:08:29.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.997+0000 7f990a269700 1 --2- 172.21.15.59:0/3067020288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99040ce560 0x7f990406f010 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009d90 tx=0x7f98fc00b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d3ef86b6d55a0f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.997+0000 7f9908a66700 1 -- 172.21.15.59:0/3067020288 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc00e070 con 0x7f99040ce560 2023-11-28T11:08:29.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.997+0000 7f9908a66700 1 -- 172.21.15.59:0/3067020288 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98fc00bbd0 con 0x7f99040ce560 2023-11-28T11:08:29.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.997+0000 7f9908a66700 1 -- 172.21.15.59:0/3067020288 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98fc012430 con 0x7f99040ce560 2023-11-28T11:08:29.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.998+0000 7f990bccc700 1 -- 172.21.15.59:0/3067020288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99040ce560 msgr2=0x7f990406f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.001 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.998+0000 7f990bccc700 1 --2- 172.21.15.59:0/3067020288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99040ce560 0x7f990406f010 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f98fc009d90 tx=0x7f98fc00b330 comp rx=0 tx=0).stop 2023-11-28T11:08:29.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.998+0000 7f990bccc700 1 -- 172.21.15.59:0/3067020288 shutdown_connections 2023-11-28T11:08:29.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.998+0000 7f990bccc700 1 --2- 172.21.15.59:0/3067020288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99040ce560 0x7f990406f010 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.998+0000 7f990bccc700 1 -- 172.21.15.59:0/3067020288 >> 172.21.15.59:0/3067020288 conn(0x7f99040b0df0 msgr2=0x7f99040b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:29.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.998+0000 7f990bccc700 1 -- 172.21.15.59:0/3067020288 shutdown_connections 2023-11-28T11:08:29.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.998+0000 7f990bccc700 1 -- 172.21.15.59:0/3067020288 wait complete. 2023-11-28T11:08:29.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.999+0000 7f990bccc700 1 Processor -- start 2023-11-28T11:08:29.002 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:28.999+0000 7f990bccc700 1 -- start start 2023-11-28T11:08:29.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f990bccc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99040ce560 0x7f99040c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f990bccc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99041589c0 0x7f990413e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f990bccc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9904158d90 0x7f99040c3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f990bccc700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990413ece0 con 0x7f9904158d90 2023-11-28T11:08:29.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f990bccc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99040c4450 con 0x7f99041589c0 2023-11-28T11:08:29.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f990bccc700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99040c4730 con 0x7f99040ce560 2023-11-28T11:08:29.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f9909a68700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99040ce560 0x7f99040c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f9909267700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99041589c0 0x7f990413e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.005 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f9909267700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99041589c0 0x7f990413e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53566/0 (socket says 172.21.15.59:53566) 2023-11-28T11:08:29.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f9909a68700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99040ce560 0x7f99040c54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:44504/0 (socket says 172.21.15.59:44504) 2023-11-28T11:08:29.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f9909267700 1 -- 172.21.15.59:0/2967248942 learned_addr learned my addr 172.21.15.59:0/2967248942 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-28T11:08:29.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f990a269700 1 --2- 172.21.15.59:0/2967248942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9904158d90 0x7f99040c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f9909a68700 1 -- 172.21.15.59:0/2967248942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99041589c0 msgr2=0x7f990413e0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.006 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f9909a68700 1 --2- 172.21.15.59:0/2967248942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99041589c0 0x7f990413e0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f9909a68700 1 -- 172.21.15.59:0/2967248942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9904158d90 msgr2=0x7f99040c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f9909a68700 1 --2- 172.21.15.59:0/2967248942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9904158d90 0x7f99040c3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f9909a68700 1 -- 172.21.15.59:0/2967248942 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc009a70 con 0x7f99040ce560 2023-11-28T11:08:29.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f9909267700 1 --2- 172.21.15.59:0/2967248942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99041589c0 0x7f990413e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:29.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.000+0000 7f9909a68700 1 --2- 172.21.15.59:0/2967248942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99040ce560 0x7f99040c54f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f990000c500 tx=0x7f990000c810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.001+0000 7f98fa7fc700 1 -- 172.21.15.59:0/2967248942 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9900005d70 con 0x7f99040ce560 2023-11-28T11:08:29.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.001+0000 7f98fa7fc700 1 -- 172.21.15.59:0/2967248942 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99000070d0 con 0x7f99040ce560 2023-11-28T11:08:29.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.001+0000 7f990bccc700 1 -- 172.21.15.59:0/2967248942 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99040c9d90 con 0x7f99040ce560 2023-11-28T11:08:29.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.001+0000 7f98fa7fc700 1 -- 172.21.15.59:0/2967248942 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9900007440 con 0x7f99040ce560 2023-11-28T11:08:29.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.001+0000 7f990bccc700 1 -- 172.21.15.59:0/2967248942 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990406e100 con 0x7f99040ce560 2023-11-28T11:08:29.007 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.001+0000 7f98fa7fc700 1 -- 172.21.15.59:0/2967248942 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f9900019030 con 0x7f99040ce560 2023-11-28T11:08:29.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.001+0000 7f98fa7fc700 1 -- 172.21.15.59:0/2967248942 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7f990001b8c0 con 0x7f99040ce560 2023-11-28T11:08:29.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.002+0000 7f990bccc700 1 -- 172.21.15.59:0/2967248942 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9904163a40 con 0x7f99040ce560 2023-11-28T11:08:29.008 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.004+0000 7f98fa7fc700 1 -- 172.21.15.59:0/2967248942 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f9900021060 con 0x7f99040ce560 2023-11-28T11:08:29.089 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.089+0000 7f990bccc700 1 -- 172.21.15.59:0/2967248942 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd new", "uuid": "8405f8e6-e7cf-4db6-9379-f177a3ae6acb", "id": 4} v 0) v1 -- 0x7f99040739f0 con 0x7f99040ce560 2023-11-28T11:08:29.092 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.092+0000 7f98fa7fc700 1 -- 172.21.15.59:0/2967248942 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "8405f8e6-e7cf-4db6-9379-f177a3ae6acb", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9900017070 con 0x7f99040ce560 2023-11-28T11:08:29.093 INFO:teuthology.orchestra.run.smithi059.stdout:4 2023-11-28T11:08:29.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.093+0000 7f990bccc700 1 -- 172.21.15.59:0/2967248942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99040ce560 msgr2=0x7f99040c54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.093+0000 7f990bccc700 1 --2- 172.21.15.59:0/2967248942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99040ce560 0x7f99040c54f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f990000c500 tx=0x7f990000c810 comp rx=0 tx=0).stop 2023-11-28T11:08:29.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.093+0000 7f990bccc700 1 -- 172.21.15.59:0/2967248942 shutdown_connections 2023-11-28T11:08:29.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.093+0000 7f990bccc700 1 --2- 172.21.15.59:0/2967248942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9904158d90 0x7f99040c3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.094 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.093+0000 7f990bccc700 1 --2- 172.21.15.59:0/2967248942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99040ce560 0x7f99040c54f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.093+0000 7f990bccc700 1 --2- 172.21.15.59:0/2967248942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99041589c0 0x7f990413e0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.093+0000 7f990bccc700 1 -- 172.21.15.59:0/2967248942 >> 172.21.15.59:0/2967248942 conn(0x7f99040b0df0 msgr2=0x7f99040b0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:29.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.093+0000 7f990bccc700 1 -- 172.21.15.59:0/2967248942 shutdown_connections 2023-11-28T11:08:29.095 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.093+0000 7f990bccc700 1 -- 172.21.15.59:0/2967248942 wait complete. 2023-11-28T11:08:29.105 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new 10082fea-1dcf-4140-b3b5-b02ce671e800 5 2023-11-28T11:08:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.233+0000 7f519d11a700 1 Processor -- start 2023-11-28T11:08:29.233 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.233+0000 7f519d11a700 1 -- start start 2023-11-28T11:08:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.233+0000 7f519d11a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f519815c640 0x7f51980cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.233+0000 7f519d11a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f519806bf30 0x7f519806c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.233+0000 7f519d11a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f519806c890 con 0x7f51980d2820 2023-11-28T11:08:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.233+0000 7f519d11a700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519815ca10 con 0x7f519806bf30 2023-11-28T11:08:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.233+0000 7f519d11a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51980b6350 con 0x7f519815c640 2023-11-28T11:08:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.234+0000 7f5196d9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f519815c640 0x7f51980cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.234+0000 7f519659c700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f51980d2820 0x7f51980b5c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.59:54968/0 (socket says 172.21.15.59:54968) 2023-11-28T11:08:29.236 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.234+0000 7f519659c700 1 -- 172.21.15.59:0/3698362964 learned_addr learned my addr 172.21.15.59:0/3698362964 (peer_addr_for_me v1:172.21.15.59:0/0) 2023-11-28T11:08:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.234+0000 7f519759e700 1 --2- 172.21.15.59:0/3698362964 >> v2:172.21.15.28:3300/0 conn(0x7f519806bf30 0x7f519806c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.234+0000 7f5196d9d700 1 -- 172.21.15.59:0/3698362964 >> v2:172.21.15.28:3300/0 conn(0x7f519806bf30 msgr2=0x7f519806c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.234+0000 7f5196d9d700 1 --2- 172.21.15.59:0/3698362964 >> v2:172.21.15.28:3300/0 conn(0x7f519806bf30 0x7f519806c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.234+0000 7f5196d9d700 1 -- 172.21.15.59:0/3698362964 >> v1:172.21.15.59:6789/0 conn(0x7f51980d2820 legacy=0x7f51980b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.234+0000 7f5195d9b700 1 -- 172.21.15.59:0/3698362964 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4290882179 0 0) 0x7f519806c890 con 0x7f51980d2820 2023-11-28T11:08:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.234+0000 7f5196d9d700 1 -- 172.21.15.59:0/3698362964 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51980d1470 con 0x7f519815c640 2023-11-28T11:08:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.234+0000 7f5196d9d700 1 --2- 172.21.15.59:0/3698362964 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f519815c640 0x7f51980cf040 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5188009840 tx=0x7f5188009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e50441c6e0e753ed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.234+0000 7f5195d9b700 1 -- 172.21.15.59:0/3698362964 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188011830 con 0x7f519815c640 2023-11-28T11:08:29.237 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.234+0000 7f5195d9b700 1 -- 172.21.15.59:0/3698362964 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5188011990 con 0x7f519815c640 2023-11-28T11:08:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.234+0000 7f5195d9b700 1 -- 172.21.15.59:0/3698362964 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188011d20 con 0x7f519815c640 2023-11-28T11:08:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.235+0000 7f519d11a700 1 -- 172.21.15.59:0/3698362964 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f519815c640 msgr2=0x7f51980cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.235+0000 7f519d11a700 1 --2- 172.21.15.59:0/3698362964 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f519815c640 0x7f51980cf040 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5188009840 tx=0x7f5188009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.235+0000 7f519d11a700 1 -- 172.21.15.59:0/3698362964 shutdown_connections 2023-11-28T11:08:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.235+0000 7f519d11a700 1 --2- 172.21.15.59:0/3698362964 >> v2:172.21.15.28:3300/0 conn(0x7f519806bf30 0x7f519806c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.235+0000 7f519d11a700 1 --2- 172.21.15.59:0/3698362964 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f519815c640 0x7f51980cf040 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.235+0000 7f519d11a700 1 -- 172.21.15.59:0/3698362964 >> 172.21.15.59:0/3698362964 conn(0x7f51980d2df0 msgr2=0x7f51980d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.235+0000 7f519d11a700 1 -- 172.21.15.59:0/3698362964 shutdown_connections 2023-11-28T11:08:29.238 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.235+0000 7f519d11a700 1 -- 172.21.15.59:0/3698362964 wait complete. 2023-11-28T11:08:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.236+0000 7f519d11a700 1 Processor -- start 2023-11-28T11:08:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.236+0000 7f519d11a700 1 -- start start 2023-11-28T11:08:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.237+0000 7f519d11a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f519806bf30 0x7f5198157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.237+0000 7f519d11a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51980d2820 0x7f5198158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.237+0000 7f519d11a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f519815c640 0x7f5198154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.237+0000 7f519d11a700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51981586f0 con 0x7f51980d2820 2023-11-28T11:08:29.241 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.237+0000 7f519d11a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5198157ae0 con 0x7f519815c640 2023-11-28T11:08:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.237+0000 7f519d11a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5198157dc0 con 0x7f519806bf30 2023-11-28T11:08:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.237+0000 7f5196d9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f519806bf30 0x7f5198157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.237+0000 7f5196d9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f519806bf30 0x7f5198157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:44514/0 (socket says 172.21.15.59:44514) 2023-11-28T11:08:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.237+0000 7f5196d9d700 1 -- 172.21.15.59:0/1177661111 learned_addr learned my addr 172.21.15.59:0/1177661111 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-28T11:08:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.237+0000 7f519759e700 1 --2- 172.21.15.59:0/1177661111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f519815c640 0x7f5198154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.237+0000 7f519659c700 1 --2- 172.21.15.59:0/1177661111 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51980d2820 0x7f5198158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.238+0000 7f5196d9d700 1 -- 172.21.15.59:0/1177661111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f519815c640 msgr2=0x7f5198154760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.238+0000 7f5196d9d700 1 --2- 172.21.15.59:0/1177661111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f519815c640 0x7f5198154760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.238+0000 7f5196d9d700 1 -- 172.21.15.59:0/1177661111 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51980d2820 msgr2=0x7f5198158b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.238+0000 7f5196d9d700 1 --2- 172.21.15.59:0/1177661111 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51980d2820 0x7f5198158b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.238+0000 7f5196d9d700 1 -- 172.21.15.59:0/1177661111 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51880094a0 con 0x7f519806bf30 2023-11-28T11:08:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.238+0000 7f5196d9d700 1 --2- 172.21.15.59:0/1177661111 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f519806bf30 0x7f5198157400 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5188009b90 tx=0x7f5188004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.242 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.238+0000 7f517f7fe700 1 -- 172.21.15.59:0/1177661111 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51880036a0 con 0x7f519806bf30 2023-11-28T11:08:29.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.238+0000 7f517f7fe700 1 -- 172.21.15.59:0/1177661111 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518801e070 con 0x7f519806bf30 2023-11-28T11:08:29.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.238+0000 7f519d11a700 1 -- 172.21.15.59:0/1177661111 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51980cd7c0 con 0x7f519806bf30 2023-11-28T11:08:29.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.238+0000 7f517f7fe700 1 -- 172.21.15.59:0/1177661111 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5188003a60 con 0x7f519806bf30 2023-11-28T11:08:29.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.238+0000 7f519d11a700 1 -- 172.21.15.59:0/1177661111 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51980cd4e0 con 0x7f519806bf30 2023-11-28T11:08:29.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.238+0000 7f517f7fe700 1 -- 172.21.15.59:0/1177661111 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f518800f050 con 0x7f519806bf30 2023-11-28T11:08:29.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.238+0000 7f517f7fe700 1 -- 172.21.15.59:0/1177661111 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f518802dd90 con 0x7f519806bf30 2023-11-28T11:08:29.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.239+0000 7f519d11a700 1 -- 172.21.15.59:0/1177661111 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51980d1710 con 0x7f519806bf30 2023-11-28T11:08:29.243 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.241+0000 7f517f7fe700 1 -- 172.21.15.59:0/1177661111 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f5188020070 con 0x7f519806bf30 2023-11-28T11:08:29.321 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:29.321+0000 7f922613f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T11:08:29.327 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.327+0000 7f519d11a700 1 -- 172.21.15.59:0/1177661111 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd new", "uuid": "10082fea-1dcf-4140-b3b5-b02ce671e800", "id": 5} v 0) v1 -- 0x7f51981589d0 con 0x7f519806bf30 2023-11-28T11:08:29.331 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.330+0000 7f517f7fe700 1 -- 172.21.15.59:0/1177661111 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "10082fea-1dcf-4140-b3b5-b02ce671e800", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f5188015020 con 0x7f519806bf30 2023-11-28T11:08:29.331 INFO:teuthology.orchestra.run.smithi059.stdout:5 2023-11-28T11:08:29.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.331+0000 7f519d11a700 1 -- 172.21.15.59:0/1177661111 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f519806bf30 msgr2=0x7f5198157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.331+0000 7f519d11a700 1 --2- 172.21.15.59:0/1177661111 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f519806bf30 0x7f5198157400 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f5188009b90 tx=0x7f5188004750 comp rx=0 tx=0).stop 2023-11-28T11:08:29.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.331+0000 7f519d11a700 1 -- 172.21.15.59:0/1177661111 shutdown_connections 2023-11-28T11:08:29.332 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.331+0000 7f519d11a700 1 --2- 172.21.15.59:0/1177661111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f519815c640 0x7f5198154760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.331+0000 7f519d11a700 1 --2- 172.21.15.59:0/1177661111 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f519806bf30 0x7f5198157400 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.331+0000 7f519d11a700 1 --2- 172.21.15.59:0/1177661111 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f51980d2820 0x7f5198158b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.331+0000 7f519d11a700 1 -- 172.21.15.59:0/1177661111 >> 172.21.15.59:0/1177661111 conn(0x7f51980d2df0 msgr2=0x7f51980b7cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:29.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.331+0000 7f519d11a700 1 -- 172.21.15.59:0/1177661111 shutdown_connections 2023-11-28T11:08:29.333 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.331+0000 7f519d11a700 1 -- 172.21.15.59:0/1177661111 wait complete. 2023-11-28T11:08:29.344 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new 25e6b8ae-36a7-4713-8199-3916b0a4b217 6 2023-11-28T11:08:29.359 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:29.359+0000 7f23534f3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T11:08:29.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.468+0000 7fa02b056700 1 Processor -- start 2023-11-28T11:08:29.469 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.468+0000 7fa02b056700 1 -- start start 2023-11-28T11:08:29.470 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.469+0000 7fa02b056700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0240cd3e0 0x7fa02406a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.469+0000 7fa02b056700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa0240d0fe0 0x7fa0240d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.469+0000 7fa02b056700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02406bb90 con 0x7fa02415fa10 2023-11-28T11:08:29.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.469+0000 7fa02b056700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0240cfa50 con 0x7fa0240d0fe0 2023-11-28T11:08:29.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.469+0000 7fa02b056700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240d0ae0 con 0x7fa0240cd3e0 2023-11-28T11:08:29.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.469+0000 7fa023fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0240cd3e0 0x7fa02406a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.469+0000 7fa0295f3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa0240d0fe0 0x7fa0240d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.469+0000 7fa023fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0240cd3e0 0x7fa02406a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:44522/0 (socket says 172.21.15.59:44522) 2023-11-28T11:08:29.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.469+0000 7fa023fff700 1 -- 172.21.15.59:0/1007627309 learned_addr learned my addr 172.21.15.59:0/1007627309 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-28T11:08:29.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.470+0000 7fa023fff700 1 -- 172.21.15.59:0/1007627309 >> v2:172.21.15.59:3300/0 conn(0x7fa0240d0fe0 msgr2=0x7fa0240d1460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:29.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.470+0000 7fa023fff700 1 --2- 172.21.15.59:0/1007627309 >> v2:172.21.15.59:3300/0 conn(0x7fa0240d0fe0 0x7fa0240d1460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.471 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.470+0000 7fa023fff700 1 -- 172.21.15.59:0/1007627309 >> v1:172.21.15.59:6789/0 conn(0x7fa02415fa10 legacy=0x7fa02406a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.470+0000 7fa023fff700 1 -- 172.21.15.59:0/1007627309 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0240cfca0 con 0x7fa0240cd3e0 2023-11-28T11:08:29.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.470+0000 7fa0237fe700 1 -- 172.21.15.59:0/1007627309 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1464413963 0 0) 0x7fa02406bb90 con 0x7fa02415fa10 2023-11-28T11:08:29.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.470+0000 7fa0295f3700 1 --2- 172.21.15.59:0/1007627309 >> v2:172.21.15.59:3300/0 conn(0x7fa0240d0fe0 0x7fa0240d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:29.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.470+0000 7fa023fff700 1 --2- 172.21.15.59:0/1007627309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0240cd3e0 0x7fa02406a8f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0100098f0 tx=0x7fa010009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c135fc6ff9cb5d67 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.470+0000 7fa0237fe700 1 -- 172.21.15.59:0/1007627309 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010011830 con 0x7fa0240cd3e0 2023-11-28T11:08:29.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.470+0000 7fa0237fe700 1 -- 172.21.15.59:0/1007627309 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa010011990 con 0x7fa0240cd3e0 2023-11-28T11:08:29.472 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.470+0000 7fa0237fe700 1 -- 172.21.15.59:0/1007627309 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010011d20 con 0x7fa0240cd3e0 2023-11-28T11:08:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.471+0000 7fa02b056700 1 -- 172.21.15.59:0/1007627309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0240cd3e0 msgr2=0x7fa02406a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.473 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.471+0000 7fa02b056700 1 --2- 172.21.15.59:0/1007627309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0240cd3e0 0x7fa02406a8f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0100098f0 tx=0x7fa010009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.471+0000 7fa02b056700 1 -- 172.21.15.59:0/1007627309 shutdown_connections 2023-11-28T11:08:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.471+0000 7fa02b056700 1 --2- 172.21.15.59:0/1007627309 >> v2:172.21.15.59:3300/0 conn(0x7fa0240d0fe0 0x7fa0240d1460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.471+0000 7fa02b056700 1 --2- 172.21.15.59:0/1007627309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0240cd3e0 0x7fa02406a8f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.471+0000 7fa02b056700 1 -- 172.21.15.59:0/1007627309 >> 172.21.15.59:0/1007627309 conn(0x7fa02406ce40 msgr2=0x7fa0240b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.471+0000 7fa02b056700 1 -- 172.21.15.59:0/1007627309 shutdown_connections 2023-11-28T11:08:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.471+0000 7fa02b056700 1 -- 172.21.15.59:0/1007627309 wait complete. 2023-11-28T11:08:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.472+0000 7fa02b056700 1 Processor -- start 2023-11-28T11:08:29.474 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.472+0000 7fa02b056700 1 -- start start 2023-11-28T11:08:29.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.473+0000 7fa02b056700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0240cd3e0 0x7fa024154560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.476 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.473+0000 7fa02b056700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0240d0fe0 0x7fa024158800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.473+0000 7fa02b056700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa02415fa10 0x7fa024154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.473+0000 7fa02b056700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240c6aa0 con 0x7fa02415fa10 2023-11-28T11:08:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.473+0000 7fa02b056700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0240c6d80 con 0x7fa0240d0fe0 2023-11-28T11:08:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.473+0000 7fa02b056700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa024158000 con 0x7fa0240cd3e0 2023-11-28T11:08:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa028df2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0240cd3e0 0x7fa024154560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa023fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0240d0fe0 0x7fa024158800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa028df2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0240cd3e0 0x7fa024154560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:44526/0 (socket says 172.21.15.59:44526) 2023-11-28T11:08:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa028df2700 1 -- 172.21.15.59:0/640480293 learned_addr learned my addr 172.21.15.59:0/640480293 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-28T11:08:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa0295f3700 1 --2- 172.21.15.59:0/640480293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa02415fa10 0x7fa024154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa023fff700 1 -- 172.21.15.59:0/640480293 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0240cd3e0 msgr2=0x7fa024154560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.477 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa023fff700 1 --2- 172.21.15.59:0/640480293 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0240cd3e0 0x7fa024154560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa023fff700 1 -- 172.21.15.59:0/640480293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa02415fa10 msgr2=0x7fa024154b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa023fff700 1 --2- 172.21.15.59:0/640480293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa02415fa10 0x7fa024154b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa023fff700 1 -- 172.21.15.59:0/640480293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01c009a70 con 0x7fa0240d0fe0 2023-11-28T11:08:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa023fff700 1 --2- 172.21.15.59:0/640480293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0240d0fe0 0x7fa024158800 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa010016000 tx=0x7fa010004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa0217fa700 1 -- 172.21.15.59:0/640480293 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100044d0 con 0x7fa0240d0fe0 2023-11-28T11:08:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa0217fa700 1 -- 172.21.15.59:0/640480293 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa01001e070 con 0x7fa0240d0fe0 2023-11-28T11:08:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa02b056700 1 -- 172.21.15.59:0/640480293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa010009550 con 0x7fa0240d0fe0 2023-11-28T11:08:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa0217fa700 1 -- 172.21.15.59:0/640480293 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100038c0 con 0x7fa0240d0fe0 2023-11-28T11:08:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.474+0000 7fa02b056700 1 -- 172.21.15.59:0/640480293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0240cd850 con 0x7fa0240d0fe0 2023-11-28T11:08:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.475+0000 7fa0217fa700 1 -- 172.21.15.59:0/640480293 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fa01000f050 con 0x7fa0240d0fe0 2023-11-28T11:08:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.475+0000 7fa0217fa700 1 -- 172.21.15.59:0/640480293 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7fa01002dcc0 con 0x7fa0240d0fe0 2023-11-28T11:08:29.478 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.475+0000 7fa02b056700 1 -- 172.21.15.59:0/640480293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0240cff80 con 0x7fa0240d0fe0 2023-11-28T11:08:29.479 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.479+0000 7fa0217fa700 1 -- 172.21.15.59:0/640480293 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fa010020070 con 0x7fa0240d0fe0 2023-11-28T11:08:29.565 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.565+0000 7fa02b056700 1 -- 172.21.15.59:0/640480293 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd new", "uuid": "25e6b8ae-36a7-4713-8199-3916b0a4b217", "id": 6} v 0) v1 -- 0x7fa02415fde0 con 0x7fa0240d0fe0 2023-11-28T11:08:29.571 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.571+0000 7fa0217fa700 1 -- 172.21.15.59:0/640480293 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "25e6b8ae-36a7-4713-8199-3916b0a4b217", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa010015020 con 0x7fa0240d0fe0 2023-11-28T11:08:29.572 INFO:teuthology.orchestra.run.smithi059.stdout:6 2023-11-28T11:08:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.572+0000 7fa02b056700 1 -- 172.21.15.59:0/640480293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0240d0fe0 msgr2=0x7fa024158800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.572+0000 7fa02b056700 1 --2- 172.21.15.59:0/640480293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0240d0fe0 0x7fa024158800 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fa010016000 tx=0x7fa010004750 comp rx=0 tx=0).stop 2023-11-28T11:08:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.572+0000 7fa02b056700 1 -- 172.21.15.59:0/640480293 shutdown_connections 2023-11-28T11:08:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.572+0000 7fa02b056700 1 --2- 172.21.15.59:0/640480293 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa02415fa10 0x7fa024154b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.572+0000 7fa02b056700 1 --2- 172.21.15.59:0/640480293 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa0240cd3e0 0x7fa024154560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.572+0000 7fa02b056700 1 --2- 172.21.15.59:0/640480293 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0240d0fe0 0x7fa024158800 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.572+0000 7fa02b056700 1 -- 172.21.15.59:0/640480293 >> 172.21.15.59:0/640480293 conn(0x7fa02406ce40 msgr2=0x7fa0240b5bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:29.574 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.572+0000 7fa02b056700 1 -- 172.21.15.59:0/640480293 shutdown_connections 2023-11-28T11:08:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.572+0000 7fa02b056700 1 -- 172.21.15.59:0/640480293 wait complete. 2023-11-28T11:08:29.585 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --cluster ceph osd new 12eb3ca7-d8e0-4cfd-a68d-528bb95118d9 7 2023-11-28T11:08:29.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.750+0000 7fc9d1249700 1 Processor -- start 2023-11-28T11:08:29.750 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.750+0000 7fc9d1249700 1 -- start start 2023-11-28T11:08:29.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9d1249700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc9cc0b1630 0x7fc9cc0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.752 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9d1249700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9cc0b1a00 0x7fc9cc0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9d1249700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9cc0ce550 con 0x7fc9cc0cb3e0 2023-11-28T11:08:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9d1249700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9cc0ce830 con 0x7fc9cc0b1630 2023-11-28T11:08:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9d1249700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc0ceb10 con 0x7fc9cc0b1a00 2023-11-28T11:08:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9ca59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9cc0b1a00 0x7fc9cc0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9ca59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9cc0b1a00 0x7fc9cc0caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.59:44534/0 (socket says 172.21.15.59:44534) 2023-11-28T11:08:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9ca59c700 1 -- 172.21.15.59:0/4224602613 learned_addr learned my addr 172.21.15.59:0/4224602613 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-28T11:08:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9cad9d700 1 --2- 172.21.15.59:0/4224602613 >> v2:172.21.15.28:3300/0 conn(0x7fc9cc0b1630 0x7fc9cc0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9ca59c700 1 -- 172.21.15.59:0/4224602613 >> v2:172.21.15.28:3300/0 conn(0x7fc9cc0b1630 msgr2=0x7fc9cc0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9ca59c700 1 --2- 172.21.15.59:0/4224602613 >> v2:172.21.15.28:3300/0 conn(0x7fc9cc0b1630 0x7fc9cc0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9ca59c700 1 -- 172.21.15.59:0/4224602613 >> v1:172.21.15.28:6789/0 conn(0x7fc9cc0cb3e0 legacy=0x7fc9cc160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9ca59c700 1 -- 172.21.15.59:0/4224602613 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc0cedf0 con 0x7fc9cc0b1a00 2023-11-28T11:08:29.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9cad9d700 1 --2- 172.21.15.59:0/4224602613 >> v2:172.21.15.28:3300/0 conn(0x7fc9cc0b1630 0x7fc9cc0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:29.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9ca59c700 1 --2- 172.21.15.59:0/4224602613 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9cc0b1a00 0x7fc9cc0caea0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00098f0 tx=0x7fc9c0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb5c1b0e4a7b0b11 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9c9d9b700 1 -- 172.21.15.59:0/4224602613 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000b8a0 con 0x7fc9cc0b1a00 2023-11-28T11:08:29.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9c9d9b700 1 -- 172.21.15.59:0/4224602613 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c000ba00 con 0x7fc9cc0b1a00 2023-11-28T11:08:29.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.751+0000 7fc9c9d9b700 1 -- 172.21.15.59:0/4224602613 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000bd90 con 0x7fc9cc0b1a00 2023-11-28T11:08:29.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.752+0000 7fc9d1249700 1 -- 172.21.15.59:0/4224602613 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9cc0b1a00 msgr2=0x7fc9cc0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.752+0000 7fc9d1249700 1 --2- 172.21.15.59:0/4224602613 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9cc0b1a00 0x7fc9cc0caea0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc9c00098f0 tx=0x7fc9c0009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:29.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.752+0000 7fc9d1249700 1 -- 172.21.15.59:0/4224602613 shutdown_connections 2023-11-28T11:08:29.754 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.752+0000 7fc9d1249700 1 --2- 172.21.15.59:0/4224602613 >> v2:172.21.15.28:3300/0 conn(0x7fc9cc0b1630 0x7fc9cc0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.752+0000 7fc9d1249700 1 --2- 172.21.15.59:0/4224602613 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9cc0b1a00 0x7fc9cc0caea0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.752+0000 7fc9d1249700 1 -- 172.21.15.59:0/4224602613 >> 172.21.15.59:0/4224602613 conn(0x7fc9cc0cc630 msgr2=0x7fc9cc0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.752+0000 7fc9d1249700 1 -- 172.21.15.59:0/4224602613 shutdown_connections 2023-11-28T11:08:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.752+0000 7fc9d1249700 1 -- 172.21.15.59:0/4224602613 wait complete. 2023-11-28T11:08:29.757 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.754+0000 7fc9d1249700 1 Processor -- start 2023-11-28T11:08:29.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.754+0000 7fc9d1249700 1 -- start start 2023-11-28T11:08:29.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9d1249700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9cc0b1630 0x7fc9cc0ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9d1249700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9cc0b1a00 0x7fc9cc0c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9d1249700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9cc0cb3e0 0x7fc9cc0b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:29.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9d1249700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc0b5750 con 0x7fc9cc0b1a00 2023-11-28T11:08:29.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9d1249700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc0b5520 con 0x7fc9cc0b1630 2023-11-28T11:08:29.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9d1249700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9cc0ac450 con 0x7fc9cc0cb3e0 2023-11-28T11:08:29.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9cad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9cc0b1630 0x7fc9cc0ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.758 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9cb59e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9cc0cb3e0 0x7fc9cc0b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9cad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9cc0b1630 0x7fc9cc0ad210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53598/0 (socket says 172.21.15.59:53598) 2023-11-28T11:08:29.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9cad9d700 1 -- 172.21.15.59:0/346115309 learned_addr learned my addr 172.21.15.59:0/346115309 (peer_addr_for_me v2:172.21.15.59:0/0) 2023-11-28T11:08:29.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9ca59c700 1 --2- 172.21.15.59:0/346115309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9cc0b1a00 0x7fc9cc0c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:29.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9cb59e700 1 -- 172.21.15.59:0/346115309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9cc0b1630 msgr2=0x7fc9cc0ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9cb59e700 1 --2- 172.21.15.59:0/346115309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9cc0b1630 0x7fc9cc0ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9cb59e700 1 -- 172.21.15.59:0/346115309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9cc0b1a00 msgr2=0x7fc9cc0c5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9cb59e700 1 --2- 172.21.15.59:0/346115309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9cc0b1a00 0x7fc9cc0c5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9cb59e700 1 -- 172.21.15.59:0/346115309 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c0009550 con 0x7fc9cc0cb3e0 2023-11-28T11:08:29.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9cad9d700 1 --2- 172.21.15.59:0/346115309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9cc0b1630 0x7fc9cc0ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:29.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9cb59e700 1 --2- 172.21.15.59:0/346115309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9cc0cb3e0 0x7fc9cc0b4fe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400cb10 tx=0x7fc9c400ce20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:29.759 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.756+0000 7fc9b37fe700 1 -- 172.21.15.59:0/346115309 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c4003c00 con 0x7fc9cc0cb3e0 2023-11-28T11:08:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.757+0000 7fc9b37fe700 1 -- 172.21.15.59:0/346115309 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c4003d60 con 0x7fc9cc0cb3e0 2023-11-28T11:08:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.757+0000 7fc9d1249700 1 -- 172.21.15.59:0/346115309 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9cc0b1e70 con 0x7fc9cc0cb3e0 2023-11-28T11:08:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.757+0000 7fc9b37fe700 1 -- 172.21.15.59:0/346115309 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c400a9e0 con 0x7fc9cc0cb3e0 2023-11-28T11:08:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.757+0000 7fc9d1249700 1 -- 172.21.15.59:0/346115309 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9cc0cf0d0 con 0x7fc9cc0cb3e0 2023-11-28T11:08:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.757+0000 7fc9b37fe700 1 -- 172.21.15.59:0/346115309 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7fc9c4010050 con 0x7fc9cc0cb3e0 2023-11-28T11:08:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.757+0000 7fc9b37fe700 1 -- 172.21.15.59:0/346115309 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fc9c40057b0 con 0x7fc9cc0cb3e0 2023-11-28T11:08:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.757+0000 7fc9d1249700 1 -- 172.21.15.59:0/346115309 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9cc0ce140 con 0x7fc9cc0cb3e0 2023-11-28T11:08:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.760+0000 7fc9b37fe700 1 -- 172.21.15.59:0/346115309 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fc9c40050d0 con 0x7fc9cc0cb3e0 2023-11-28T11:08:29.890 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.890+0000 7fc9d1249700 1 -- 172.21.15.59:0/346115309 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd new", "uuid": "12eb3ca7-d8e0-4cfd-a68d-528bb95118d9", "id": 7} v 0) v1 -- 0x7fc9cc0739f0 con 0x7fc9cc0cb3e0 2023-11-28T11:08:29.898 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.897+0000 7fc9b37fe700 1 -- 172.21.15.59:0/346115309 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "12eb3ca7-d8e0-4cfd-a68d-528bb95118d9", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc9c4005270 con 0x7fc9cc0cb3e0 2023-11-28T11:08:29.898 INFO:teuthology.orchestra.run.smithi059.stdout:7 2023-11-28T11:08:29.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.898+0000 7fc9d1249700 1 -- 172.21.15.59:0/346115309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9cc0cb3e0 msgr2=0x7fc9cc0b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:29.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.898+0000 7fc9d1249700 1 --2- 172.21.15.59:0/346115309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9cc0cb3e0 0x7fc9cc0b4fe0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc9c400cb10 tx=0x7fc9c400ce20 comp rx=0 tx=0).stop 2023-11-28T11:08:29.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.898+0000 7fc9d1249700 1 -- 172.21.15.59:0/346115309 shutdown_connections 2023-11-28T11:08:29.899 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.898+0000 7fc9d1249700 1 --2- 172.21.15.59:0/346115309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9cc0cb3e0 0x7fc9cc0b4fe0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.898+0000 7fc9d1249700 1 --2- 172.21.15.59:0/346115309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9cc0b1630 0x7fc9cc0ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.898+0000 7fc9d1249700 1 --2- 172.21.15.59:0/346115309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9cc0b1a00 0x7fc9cc0c5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:29.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.898+0000 7fc9d1249700 1 -- 172.21.15.59:0/346115309 >> 172.21.15.59:0/346115309 conn(0x7fc9cc0cc630 msgr2=0x7fc9cc0cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:29.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.898+0000 7fc9d1249700 1 -- 172.21.15.59:0/346115309 shutdown_connections 2023-11-28T11:08:29.900 INFO:teuthology.orchestra.run.smithi059.stderr:2023-11-28T11:08:29.898+0000 7fc9d1249700 1 -- 172.21.15.59:0/346115309 wait complete. 2023-11-28T11:08:29.911 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-28T11:08:29.911 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-28T11:08:29.913 INFO:tasks.ceph.osd.0:Started 2023-11-28T11:08:29.913 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-28T11:08:29.913 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-28T11:08:29.915 INFO:tasks.ceph.osd.1:Started 2023-11-28T11:08:29.915 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-28T11:08:29.916 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-28T11:08:29.916 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:29.916+0000 7f922613f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T11:08:29.918 INFO:tasks.ceph.osd.2:Started 2023-11-28T11:08:29.918 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-28T11:08:29.918 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-28T11:08:29.920 INFO:tasks.ceph.osd.3:Started 2023-11-28T11:08:29.920 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-28T11:08:29.920 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-28T11:08:29.922 INFO:tasks.ceph.osd.4:Started 2023-11-28T11:08:29.922 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-28T11:08:29.922 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-28T11:08:29.924 INFO:tasks.ceph.osd.5:Started 2023-11-28T11:08:29.925 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-28T11:08:29.925 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-28T11:08:29.927 INFO:tasks.ceph.osd.6:Started 2023-11-28T11:08:29.927 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-28T11:08:29.927 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-28T11:08:29.931 INFO:tasks.ceph.osd.7:Started 2023-11-28T11:08:29.932 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:29.994 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:29.994+0000 7f23534f3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T11:08:30.020 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:30.019+0000 7f922613f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T11:08:30.080 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:30.079+0000 7f23534f3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T11:08:30.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.096+0000 7ff7049c5700 1 Processor -- start 2023-11-28T11:08:30.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.096+0000 7ff7049c5700 1 -- start start 2023-11-28T11:08:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff7049c5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff70015c990 0x7ff7000cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff7049c5700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff700161090 0x7ff7000ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff7049c5700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7000ce340 con 0x7ff7000caeb0 2023-11-28T11:08:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff7049c5700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7000cebe0 con 0x7ff700161090 2023-11-28T11:08:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff7049c5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000cb3f0 con 0x7ff70015c990 2023-11-28T11:08:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff6fdd9b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff700161090 0x7ff7000ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff6fed9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff7000caeb0 0x7ff7000cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:33888/0 (socket says 172.21.15.28:33888) 2023-11-28T11:08:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff6fed9d700 1 -- 172.21.15.28:0/2449230466 learned_addr learned my addr 172.21.15.28:0/2449230466 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff6fdd9b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff700161090 0x7ff7000ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45828/0 (socket says 172.21.15.28:45828) 2023-11-28T11:08:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff6fe59c700 1 --2- 172.21.15.28:0/2449230466 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff70015c990 0x7ff7000cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff6fe59c700 1 -- 172.21.15.28:0/2449230466 >> v2:172.21.15.28:3300/0 conn(0x7ff700161090 msgr2=0x7ff7000ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:30.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff6fe59c700 1 --2- 172.21.15.28:0/2449230466 >> v2:172.21.15.28:3300/0 conn(0x7ff700161090 0x7ff7000ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff6fe59c700 1 -- 172.21.15.28:0/2449230466 >> v1:172.21.15.28:6789/0 conn(0x7ff7000caeb0 legacy=0x7ff7000cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff6fe59c700 1 -- 172.21.15.28:0/2449230466 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7000ce590 con 0x7ff70015c990 2023-11-28T11:08:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff6fd59a700 1 -- 172.21.15.28:0/2449230466 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2501733324 0 0) 0x7ff7000ce340 con 0x7ff7000caeb0 2023-11-28T11:08:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff6fdd9b700 1 --2- 172.21.15.28:0/2449230466 >> v2:172.21.15.28:3300/0 conn(0x7ff700161090 0x7ff7000ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff6fe59c700 1 --2- 172.21.15.28:0/2449230466 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff70015c990 0x7ff7000cf070 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009840 tx=0x7ff6f0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a03bdbb9f2c021ab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.097+0000 7ff6fd59a700 1 -- 172.21.15.28:0/2449230466 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0011830 con 0x7ff70015c990 2023-11-28T11:08:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.098+0000 7ff6fd59a700 1 -- 172.21.15.28:0/2449230466 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f0011990 con 0x7ff70015c990 2023-11-28T11:08:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.098+0000 7ff6fd59a700 1 -- 172.21.15.28:0/2449230466 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f0011d20 con 0x7ff70015c990 2023-11-28T11:08:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.098+0000 7ff7049c5700 1 -- 172.21.15.28:0/2449230466 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff70015c990 msgr2=0x7ff7000cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.098+0000 7ff7049c5700 1 --2- 172.21.15.28:0/2449230466 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff70015c990 0x7ff7000cf070 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff6f0009840 tx=0x7ff6f0009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.098+0000 7ff7049c5700 1 -- 172.21.15.28:0/2449230466 shutdown_connections 2023-11-28T11:08:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.098+0000 7ff7049c5700 1 --2- 172.21.15.28:0/2449230466 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff70015c990 0x7ff7000cf070 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.098+0000 7ff7049c5700 1 --2- 172.21.15.28:0/2449230466 >> v2:172.21.15.28:3300/0 conn(0x7ff700161090 0x7ff7000ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.098+0000 7ff7049c5700 1 -- 172.21.15.28:0/2449230466 >> 172.21.15.28:0/2449230466 conn(0x7ff7000b1650 msgr2=0x7ff7000b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:30.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.098+0000 7ff7049c5700 1 -- 172.21.15.28:0/2449230466 shutdown_connections 2023-11-28T11:08:30.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.098+0000 7ff7049c5700 1 -- 172.21.15.28:0/2449230466 wait complete. 2023-11-28T11:08:30.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.099+0000 7ff7049c5700 1 Processor -- start 2023-11-28T11:08:30.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.100+0000 7ff7049c5700 1 -- start start 2023-11-28T11:08:30.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.100+0000 7ff7049c5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7000caeb0 0x7ff70015ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.100+0000 7ff7049c5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff70015c990 0x7ff7000b51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.100+0000 7ff7049c5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff700161090 0x7ff700150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:30.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.100+0000 7ff7049c5700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70015fdc0 con 0x7ff700161090 2023-11-28T11:08:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.100+0000 7ff7049c5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7000b5700 con 0x7ff7000caeb0 2023-11-28T11:08:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.100+0000 7ff7049c5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7001512d0 con 0x7ff70015c990 2023-11-28T11:08:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.100+0000 7ff6fed9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff700161090 0x7ff700150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.100+0000 7ff6fed9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff700161090 0x7ff700150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45836/0 (socket says 172.21.15.28:45836) 2023-11-28T11:08:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.100+0000 7ff6fed9d700 1 -- 172.21.15.28:0/3894132743 learned_addr learned my addr 172.21.15.28:0/3894132743 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.100+0000 7ff6fdd9b700 1 --2- 172.21.15.28:0/3894132743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff70015c990 0x7ff7000b51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.100+0000 7ff6fe59c700 1 --2- 172.21.15.28:0/3894132743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7000caeb0 0x7ff70015ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.101+0000 7ff6fed9d700 1 -- 172.21.15.28:0/3894132743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff70015c990 msgr2=0x7ff7000b51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.101+0000 7ff6fed9d700 1 --2- 172.21.15.28:0/3894132743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff70015c990 0x7ff7000b51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.101+0000 7ff6fed9d700 1 -- 172.21.15.28:0/3894132743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7000caeb0 msgr2=0x7ff70015ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.101+0000 7ff6fed9d700 1 --2- 172.21.15.28:0/3894132743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7000caeb0 0x7ff70015ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.101+0000 7ff6fed9d700 1 -- 172.21.15.28:0/3894132743 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f40094a0 con 0x7ff700161090 2023-11-28T11:08:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.101+0000 7ff6fed9d700 1 --2- 172.21.15.28:0/3894132743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff700161090 0x7ff700150d90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8002230 tx=0x7ff6f800cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.101+0000 7ff6eeffd700 1 -- 172.21.15.28:0/3894132743 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f800aac0 con 0x7ff700161090 2023-11-28T11:08:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.101+0000 7ff6eeffd700 1 -- 172.21.15.28:0/3894132743 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f800ac40 con 0x7ff700161090 2023-11-28T11:08:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.101+0000 7ff6eeffd700 1 -- 172.21.15.28:0/3894132743 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f80070d0 con 0x7ff700161090 2023-11-28T11:08:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.101+0000 7ff7049c5700 1 -- 172.21.15.28:0/3894132743 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f00094a0 con 0x7ff700161090 2023-11-28T11:08:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.101+0000 7ff7049c5700 1 -- 172.21.15.28:0/3894132743 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7000cf5b0 con 0x7ff700161090 2023-11-28T11:08:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.101+0000 7ff6eeffd700 1 -- 172.21.15.28:0/3894132743 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7ff6f8010070 con 0x7ff700161090 2023-11-28T11:08:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.101+0000 7ff6eeffd700 1 -- 172.21.15.28:0/3894132743 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7ff6f801ba30 con 0x7ff700161090 2023-11-28T11:08:30.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.102+0000 7ff7049c5700 1 -- 172.21.15.28:0/3894132743 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7000ce870 con 0x7ff700161090 2023-11-28T11:08:30.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.104+0000 7ff6eeffd700 1 -- 172.21.15.28:0/3894132743 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7ff6f8018070 con 0x7ff700161090 2023-11-28T11:08:30.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.175+0000 7ff7049c5700 1 -- 172.21.15.28:0/3894132743 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7000cf740 con 0x7ff700161090 2023-11-28T11:08:30.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.176+0000 7ff6eeffd700 1 -- 172.21.15.28:0/3894132743 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7ff6f801bcd0 con 0x7ff700161090 2023-11-28T11:08:30.176 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:08:30.176 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":10,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:29.891204+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:30.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.176+0000 7ff7049c5700 1 -- 172.21.15.28:0/3894132743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff700161090 msgr2=0x7ff700150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:30.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.176+0000 7ff7049c5700 1 --2- 172.21.15.28:0/3894132743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff700161090 0x7ff700150d90 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8002230 tx=0x7ff6f800cf50 comp rx=0 tx=0).stop 2023-11-28T11:08:30.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.177+0000 7ff7049c5700 1 -- 172.21.15.28:0/3894132743 shutdown_connections 2023-11-28T11:08:30.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.177+0000 7ff7049c5700 1 --2- 172.21.15.28:0/3894132743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff700161090 0x7ff700150d90 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.177+0000 7ff7049c5700 1 --2- 172.21.15.28:0/3894132743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7000caeb0 0x7ff70015ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.177+0000 7ff7049c5700 1 --2- 172.21.15.28:0/3894132743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff70015c990 0x7ff7000b51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:30.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.177+0000 7ff7049c5700 1 -- 172.21.15.28:0/3894132743 >> 172.21.15.28:0/3894132743 conn(0x7ff7000b1650 msgr2=0x7ff7000c8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:30.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.177+0000 7ff7049c5700 1 -- 172.21.15.28:0/3894132743 shutdown_connections 2023-11-28T11:08:30.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:30.177+0000 7ff7049c5700 1 -- 172.21.15.28:0/3894132743 wait complete. 2023-11-28T11:08:30.189 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-28T11:08:30.189 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-28T11:08:30.296 INFO:tasks.ceph.osd.0.smithi028.stderr:2023-11-28T11:08:30.295+0000 7f5584d7a640 -1 Falling back to public interface 2023-11-28T11:08:30.536 INFO:tasks.ceph.osd.5.smithi059.stderr:2023-11-28T11:08:30.535+0000 7fbcdd0fc640 -1 Falling back to public interface 2023-11-28T11:08:30.536 INFO:tasks.ceph.osd.4.smithi059.stderr:2023-11-28T11:08:30.535+0000 7f46276ee640 -1 Falling back to public interface 2023-11-28T11:08:30.536 INFO:tasks.ceph.osd.5.smithi059.stderr:2023-11-28T11:08:30.535+0000 7fbcd44ba700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6801/0: (98) Address already in use 2023-11-28T11:08:30.539 INFO:tasks.ceph.osd.6.smithi059.stderr:2023-11-28T11:08:30.539+0000 7f6b822ba640 -1 Falling back to public interface 2023-11-28T11:08:30.551 INFO:tasks.ceph.osd.3.smithi028.stderr:2023-11-28T11:08:30.551+0000 7f93a52c4640 -1 Falling back to public interface 2023-11-28T11:08:30.556 INFO:tasks.ceph.osd.2.smithi028.stderr:2023-11-28T11:08:30.555+0000 7f9b94a69640 -1 Falling back to public interface 2023-11-28T11:08:30.561 INFO:tasks.ceph.osd.1.smithi028.stderr:2023-11-28T11:08:30.561+0000 7f1541356640 -1 Falling back to public interface 2023-11-28T11:08:30.563 INFO:tasks.ceph.osd.7.smithi059.stderr:2023-11-28T11:08:30.563+0000 7f5560209640 -1 Falling back to public interface 2023-11-28T11:08:30.750 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:30.750+0000 7f922613f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:08:30.791 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:30.790+0000 7f23534f3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:08:30.832 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:30.832+0000 7f922613f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T11:08:30.872 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:30.871+0000 7f23534f3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-11-28T11:08:30.979 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:30.979+0000 7f922613f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:08:30.995 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:31.017 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:31.017+0000 7f23534f3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T11:08:31.111 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.110+0000 7f5012e4b700 1 Processor -- start 2023-11-28T11:08:31.111 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.110+0000 7f5012e4b700 1 -- start start 2023-11-28T11:08:31.113 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.111+0000 7f5012e4b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f500c157e60 0x7f500c15b550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.113 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.111+0000 7f5012e4b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f500c15e8a0 0x7f500c0ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.113 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.111+0000 7f5012e4b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f500c070270 0x7f500c0706d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.113 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.111+0000 7f5012e4b700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500c0cb7c0 con 0x7f500c070270 2023-11-28T11:08:31.113 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.111+0000 7f5012e4b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f500c070c10 con 0x7f500c15e8a0 2023-11-28T11:08:31.113 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.111+0000 7f5012e4b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500c0cb200 con 0x7f500c157e60 2023-11-28T11:08:31.114 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f50113e8700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f500c070270 0x7f500c0706d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.114 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f500bfff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f500c15e8a0 0x7f500c0ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.114 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f5010be7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f500c157e60 0x7f500c15b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.114 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f5010be7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f500c157e60 0x7f500c15b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:51596/0 (socket says 172.21.15.28:51596) 2023-11-28T11:08:31.114 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f500bfff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f500c15e8a0 0x7f500c0ccc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:33130/0 (socket says 172.21.15.28:33130) 2023-11-28T11:08:31.114 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f50113e8700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f500c070270 0x7f500c0706d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:45842/0 (socket says 172.21.15.28:45842) 2023-11-28T11:08:31.114 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f5010be7700 1 -- 172.21.15.28:0/3657384587 learned_addr learned my addr 172.21.15.28:0/3657384587 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:31.114 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f50113e8700 1 -- 172.21.15.28:0/3657384587 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f500c157e60 msgr2=0x7f500c15b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:31.114 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f50113e8700 1 --2- 172.21.15.28:0/3657384587 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f500c157e60 0x7f500c15b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.114 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f50113e8700 1 -- 172.21.15.28:0/3657384587 >> v2:172.21.15.59:3300/0 conn(0x7f500c15e8a0 msgr2=0x7f500c0ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:31.114 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f50113e8700 1 --2- 172.21.15.28:0/3657384587 >> v2:172.21.15.59:3300/0 conn(0x7f500c15e8a0 0x7f500c0ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.115 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f50113e8700 1 -- 172.21.15.28:0/3657384587 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f500c0cb4e0 con 0x7f500c070270 2023-11-28T11:08:31.115 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f500bfff700 1 --2- 172.21.15.28:0/3657384587 >> v2:172.21.15.59:3300/0 conn(0x7f500c15e8a0 0x7f500c0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:31.115 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f50113e8700 1 --2- 172.21.15.28:0/3657384587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500c070270 0x7f500c0706d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f500c0b36c0 tx=0x7f500400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb828447fd6de257 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.115 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f500b7fe700 1 -- 172.21.15.28:0/3657384587 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f500400e070 con 0x7f500c070270 2023-11-28T11:08:31.115 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f500b7fe700 1 -- 172.21.15.28:0/3657384587 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f500400bb20 con 0x7f500c070270 2023-11-28T11:08:31.115 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.112+0000 7f500b7fe700 1 -- 172.21.15.28:0/3657384587 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5004012430 con 0x7f500c070270 2023-11-28T11:08:31.118 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.113+0000 7f5012e4b700 1 -- 172.21.15.28:0/3657384587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500c070270 msgr2=0x7f500c0706d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.118 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.113+0000 7f5012e4b700 1 --2- 172.21.15.28:0/3657384587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500c070270 0x7f500c0706d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f500c0b36c0 tx=0x7f500400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:31.118 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.113+0000 7f5012e4b700 1 -- 172.21.15.28:0/3657384587 shutdown_connections 2023-11-28T11:08:31.118 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.113+0000 7f5012e4b700 1 --2- 172.21.15.28:0/3657384587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500c070270 0x7f500c0706d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.118 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.113+0000 7f5012e4b700 1 --2- 172.21.15.28:0/3657384587 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f500c157e60 0x7f500c15b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.118 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.113+0000 7f5012e4b700 1 --2- 172.21.15.28:0/3657384587 >> v2:172.21.15.59:3300/0 conn(0x7f500c15e8a0 0x7f500c0ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.118 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.113+0000 7f5012e4b700 1 -- 172.21.15.28:0/3657384587 >> 172.21.15.28:0/3657384587 conn(0x7f500c0d0b10 msgr2=0x7f500c0d0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:31.119 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.113+0000 7f5012e4b700 1 -- 172.21.15.28:0/3657384587 shutdown_connections 2023-11-28T11:08:31.119 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.113+0000 7f5012e4b700 1 -- 172.21.15.28:0/3657384587 wait complete. 2023-11-28T11:08:31.119 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.115+0000 7f5012e4b700 1 Processor -- start 2023-11-28T11:08:31.119 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.115+0000 7f5012e4b700 1 -- start start 2023-11-28T11:08:31.119 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.115+0000 7f5012e4b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500c070270 0x7f500c0b0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.119 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.115+0000 7f5012e4b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f500c157e60 0x7f500c0b1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.119 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.115+0000 7f5012e4b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f500c15e8a0 0x7f500c14c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:31.119 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f5012e4b700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c0b17d0 con 0x7f500c070270 2023-11-28T11:08:31.119 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f5012e4b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f500c14c6c0 con 0x7f500c15e8a0 2023-11-28T11:08:31.119 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f5012e4b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f500c0b0b50 con 0x7f500c157e60 2023-11-28T11:08:31.120 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f5010be7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500c070270 0x7f500c0b0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.120 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f500bfff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f500c157e60 0x7f500c0b1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.120 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f500bfff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f500c157e60 0x7f500c0b1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:51604/0 (socket says 172.21.15.28:51604) 2023-11-28T11:08:31.120 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f500bfff700 1 -- 172.21.15.28:0/3089731893 learned_addr learned my addr 172.21.15.28:0/3089731893 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:31.120 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f50113e8700 1 --2- 172.21.15.28:0/3089731893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f500c15e8a0 0x7f500c14c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:31.120 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f500bfff700 1 -- 172.21.15.28:0/3089731893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f500c15e8a0 msgr2=0x7f500c14c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.120 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f500bfff700 1 --2- 172.21.15.28:0/3089731893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f500c15e8a0 0x7f500c14c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.120 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f500bfff700 1 -- 172.21.15.28:0/3089731893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500c070270 msgr2=0x7f500c0b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.120 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f500bfff700 1 --2- 172.21.15.28:0/3089731893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500c070270 0x7f500c0b0570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.120 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f500bfff700 1 -- 172.21.15.28:0/3089731893 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ff80094a0 con 0x7f500c157e60 2023-11-28T11:08:31.120 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f5010be7700 1 --2- 172.21.15.28:0/3089731893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500c070270 0x7f500c0b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:31.121 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f500bfff700 1 --2- 172.21.15.28:0/3089731893 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f500c157e60 0x7f500c0b1bf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8010ef0 tx=0x7f4ff800b420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:31.121 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f50097fa700 1 -- 172.21.15.28:0/3089731893 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8015070 con 0x7f500c157e60 2023-11-28T11:08:31.121 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f50097fa700 1 -- 172.21.15.28:0/3089731893 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ff800bb90 con 0x7f500c157e60 2023-11-28T11:08:31.121 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f5012e4b700 1 -- 172.21.15.28:0/3089731893 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5004009a70 con 0x7f500c157e60 2023-11-28T11:08:31.121 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f50097fa700 1 -- 172.21.15.28:0/3089731893 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ff8019410 con 0x7f500c157e60 2023-11-28T11:08:31.121 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.116+0000 7f5012e4b700 1 -- 172.21.15.28:0/3089731893 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f500c15e380 con 0x7f500c157e60 2023-11-28T11:08:31.121 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.117+0000 7f50097fa700 1 -- 172.21.15.28:0/3089731893 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 1) v1 ==== 656+0+0 (secure 0 0 0) 0x7f4ff8020030 con 0x7f500c157e60 2023-11-28T11:08:31.121 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.117+0000 7f50097fa700 1 -- 172.21.15.28:0/3089731893 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4ff8019570 con 0x7f500c157e60 2023-11-28T11:08:31.121 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.117+0000 7f5012e4b700 1 -- 172.21.15.28:0/3089731893 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f500c15e0a0 con 0x7f500c157e60 2023-11-28T11:08:31.122 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.122+0000 7f50097fa700 1 -- 172.21.15.28:0/3089731893 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f4ff801e070 con 0x7f500c157e60 2023-11-28T11:08:31.193 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.193+0000 7f5012e4b700 1 -- 172.21.15.28:0/3089731893 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f500c158230 con 0x7f500c157e60 2023-11-28T11:08:31.194 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.194+0000 7f50097fa700 1 -- 172.21.15.28:0/3089731893 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f4ff80064e0 con 0x7f500c157e60 2023-11-28T11:08:31.195 INFO:teuthology.misc.health.smithi028.stdout: 2023-11-28T11:08:31.195 INFO:teuthology.misc.health.smithi028.stdout:{"epoch":10,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:29.891204+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:31.196 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.195+0000 7f5012e4b700 1 -- 172.21.15.28:0/3089731893 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f500c157e60 msgr2=0x7f500c0b1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:31.196 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.195+0000 7f5012e4b700 1 --2- 172.21.15.28:0/3089731893 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f500c157e60 0x7f500c0b1bf0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f4ff8010ef0 tx=0x7f4ff800b420 comp rx=0 tx=0).stop 2023-11-28T11:08:31.196 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.195+0000 7f5012e4b700 1 -- 172.21.15.28:0/3089731893 shutdown_connections 2023-11-28T11:08:31.196 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.195+0000 7f5012e4b700 1 --2- 172.21.15.28:0/3089731893 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f500c070270 0x7f500c0b0570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.196 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.195+0000 7f5012e4b700 1 --2- 172.21.15.28:0/3089731893 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f500c15e8a0 0x7f500c14c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.196 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.195+0000 7f5012e4b700 1 --2- 172.21.15.28:0/3089731893 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f500c157e60 0x7f500c0b1bf0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:31.196 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.195+0000 7f5012e4b700 1 -- 172.21.15.28:0/3089731893 >> 172.21.15.28:0/3089731893 conn(0x7f500c0d0b10 msgr2=0x7f500c15cb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:31.196 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.195+0000 7f5012e4b700 1 -- 172.21.15.28:0/3089731893 shutdown_connections 2023-11-28T11:08:31.196 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:31.195+0000 7f5012e4b700 1 -- 172.21.15.28:0/3089731893 wait complete. 2023-11-28T11:08:31.207 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-28T11:08:31.294 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:31.294+0000 7f922613f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T11:08:31.329 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:31.329+0000 7f23534f3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T11:08:31.434 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:31.433+0000 7f922613f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T11:08:31.470 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:31.470+0000 7f23534f3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T11:08:31.660 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:31.660+0000 7f922613f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T11:08:31.696 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:31.696+0000 7f23534f3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T11:08:31.824 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:31.824+0000 7f922613f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T11:08:31.858 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:31.858+0000 7f23534f3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T11:08:31.946 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:31.946+0000 7f922613f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T11:08:31.979 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:31.978+0000 7f23534f3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T11:08:32.283 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:32.282+0000 7f922613f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T11:08:32.313 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:32.312+0000 7f23534f3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T11:08:32.386 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:32.386+0000 7f922613f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T11:08:32.416 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:32.416+0000 7f23534f3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T11:08:32.579 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:32.578+0000 7f922613f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T11:08:32.608 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:32.608+0000 7f23534f3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T11:08:33.141 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:33.141+0000 7f922613f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T11:08:33.162 INFO:tasks.ceph.mgr.x.smithi059.stderr:2023-11-28T11:08:33.162+0000 7f23534f3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T11:08:33.832 INFO:tasks.ceph.osd.0.smithi028.stderr:2023-11-28T11:08:33.832+0000 7f5584d7a640 -1 osd.0 0 log_to_monitors true 2023-11-28T11:08:34.061 INFO:tasks.ceph.osd.6.smithi059.stderr:2023-11-28T11:08:34.061+0000 7f6b822ba640 -1 osd.6 0 log_to_monitors true 2023-11-28T11:08:34.062 INFO:tasks.ceph.osd.4.smithi059.stderr:2023-11-28T11:08:34.062+0000 7f46276ee640 -1 osd.4 0 log_to_monitors true 2023-11-28T11:08:34.069 INFO:tasks.ceph.osd.5.smithi059.stderr:2023-11-28T11:08:34.069+0000 7fbcdd0fc640 -1 osd.5 0 log_to_monitors true 2023-11-28T11:08:34.077 INFO:tasks.ceph.osd.1.smithi028.stderr:2023-11-28T11:08:34.077+0000 7f1541356640 -1 osd.1 0 log_to_monitors true 2023-11-28T11:08:34.101 INFO:tasks.ceph.osd.7.smithi059.stderr:2023-11-28T11:08:34.101+0000 7f5560209640 -1 osd.7 0 log_to_monitors true 2023-11-28T11:08:34.112 INFO:tasks.ceph.osd.3.smithi028.stderr:2023-11-28T11:08:34.112+0000 7f93a52c4640 -1 osd.3 0 log_to_monitors true 2023-11-28T11:08:34.116 INFO:tasks.ceph.osd.2.smithi028.stderr:2023-11-28T11:08:34.116+0000 7f9b94a69640 -1 osd.2 0 log_to_monitors true 2023-11-28T11:08:34.165 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:34.164+0000 7f91dd36d700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-28T11:08:34.167 INFO:tasks.ceph.mgr.y.smithi028.stderr:2023-11-28T11:08:34.167+0000 7f91dd36d700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-28T11:08:36.551 INFO:tasks.ceph.osd.2.smithi028.stderr:2023-11-28T11:08:36.550+0000 7f9b88e21700 -1 osd.2 0 waiting for initial osdmap 2023-11-28T11:08:36.568 INFO:tasks.ceph.osd.2.smithi028.stderr:2023-11-28T11:08:36.568+0000 7f9b809fb700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:36.583 INFO:tasks.ceph.osd.1.smithi028.stderr:2023-11-28T11:08:36.583+0000 7f153570e700 -1 osd.1 0 waiting for initial osdmap 2023-11-28T11:08:36.593 INFO:tasks.ceph.osd.0.smithi028.stderr:2023-11-28T11:08:36.592+0000 7f5579132700 -1 osd.0 0 waiting for initial osdmap 2023-11-28T11:08:36.596 INFO:tasks.ceph.osd.3.smithi028.stderr:2023-11-28T11:08:36.595+0000 7f939967c700 -1 osd.3 0 waiting for initial osdmap 2023-11-28T11:08:36.598 INFO:tasks.ceph.osd.1.smithi028.stderr:2023-11-28T11:08:36.598+0000 7f152d2e8700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:36.610 INFO:tasks.ceph.osd.0.smithi028.stderr:2023-11-28T11:08:36.609+0000 7f5570d0c700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:36.612 INFO:tasks.ceph.osd.3.smithi028.stderr:2023-11-28T11:08:36.612+0000 7f9391256700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:37.217 INFO:tasks.ceph.osd.7.smithi059.stderr:2023-11-28T11:08:37.216+0000 7f55545c1700 -1 osd.7 0 waiting for initial osdmap 2023-11-28T11:08:37.241 INFO:tasks.ceph.osd.7.smithi059.stderr:2023-11-28T11:08:37.240+0000 7f554c19b700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:37.242 INFO:tasks.ceph.osd.6.smithi059.stderr:2023-11-28T11:08:37.242+0000 7f6b76672700 -1 osd.6 0 waiting for initial osdmap 2023-11-28T11:08:37.259 INFO:tasks.ceph.osd.6.smithi059.stderr:2023-11-28T11:08:37.259+0000 7f6b6e24c700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:37.266 INFO:tasks.ceph.osd.4.smithi059.stderr:2023-11-28T11:08:37.266+0000 7f461baa6700 -1 osd.4 0 waiting for initial osdmap 2023-11-28T11:08:37.278 INFO:tasks.ceph.osd.4.smithi059.stderr:2023-11-28T11:08:37.278+0000 7f4613680700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:37.279 INFO:tasks.ceph.osd.5.smithi059.stderr:2023-11-28T11:08:37.278+0000 7fbcd14b4700 -1 osd.5 0 waiting for initial osdmap 2023-11-28T11:08:37.293 INFO:tasks.ceph.osd.5.smithi059.stderr:2023-11-28T11:08:37.293+0000 7fbcc908e700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T11:08:38.014 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:38.144 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.142+0000 7efcbbadd700 1 Processor -- start 2023-11-28T11:08:38.144 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.142+0000 7efcbbadd700 1 -- start start 2023-11-28T11:08:38.146 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.143+0000 7efcbbadd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efcb415a7a0 0x7efcb406eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.147 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.143+0000 7efcbbadd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efcb40d0630 0x7efcb40b4990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.147 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.143+0000 7efcbbadd700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efcb406f3e0 con 0x7efcb40b4ed0 2023-11-28T11:08:38.147 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.143+0000 7efcbbadd700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb40b5370 con 0x7efcb415a7a0 2023-11-28T11:08:38.147 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.143+0000 7efcbbadd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb406f6c0 con 0x7efcb40d0630 2023-11-28T11:08:38.147 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.143+0000 7efcb9879700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efcb415a7a0 0x7efcb406eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.147 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.143+0000 7efcb9879700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7efcb415a7a0 0x7efcb406eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46078/0 (socket says 172.21.15.28:46078) 2023-11-28T11:08:38.147 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.143+0000 7efcb9879700 1 -- 172.21.15.28:0/2339902323 learned_addr learned my addr 172.21.15.28:0/2339902323 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:38.148 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.143+0000 7efcb9078700 1 --2- 172.21.15.28:0/2339902323 >> v2:172.21.15.59:3300/0 conn(0x7efcb40d0630 0x7efcb40b4990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.148 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.143+0000 7efcb9879700 1 -- 172.21.15.28:0/2339902323 >> v2:172.21.15.59:3300/0 conn(0x7efcb40d0630 msgr2=0x7efcb40b4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:38.148 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.143+0000 7efcb9879700 1 --2- 172.21.15.28:0/2339902323 >> v2:172.21.15.59:3300/0 conn(0x7efcb40d0630 0x7efcb40b4990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.148 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.143+0000 7efcb9879700 1 -- 172.21.15.28:0/2339902323 >> v1:172.21.15.59:6789/0 conn(0x7efcb40b4ed0 legacy=0x7efcb40b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.148 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.144+0000 7efcb9879700 1 -- 172.21.15.28:0/2339902323 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb40caa60 con 0x7efcb415a7a0 2023-11-28T11:08:38.148 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.144+0000 7efcb9879700 1 --2- 172.21.15.28:0/2339902323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcb415a7a0 0x7efcb406eb90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efcb406a800 tx=0x7efca4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8915f6a5ddc7282 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:38.148 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.144+0000 7efcb8877700 1 -- 172.21.15.28:0/2339902323 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca400b870 con 0x7efcb415a7a0 2023-11-28T11:08:38.148 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.144+0000 7efcb8877700 1 -- 172.21.15.28:0/2339902323 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca400b9d0 con 0x7efcb415a7a0 2023-11-28T11:08:38.149 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.144+0000 7efcb8877700 1 -- 172.21.15.28:0/2339902323 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca400bd60 con 0x7efcb415a7a0 2023-11-28T11:08:38.149 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.144+0000 7efcbbadd700 1 -- 172.21.15.28:0/2339902323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcb415a7a0 msgr2=0x7efcb406eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.149 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.144+0000 7efcbbadd700 1 --2- 172.21.15.28:0/2339902323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcb415a7a0 0x7efcb406eb90 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efcb406a800 tx=0x7efca4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.149 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.144+0000 7efcbbadd700 1 -- 172.21.15.28:0/2339902323 shutdown_connections 2023-11-28T11:08:38.149 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.144+0000 7efcbbadd700 1 --2- 172.21.15.28:0/2339902323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcb415a7a0 0x7efcb406eb90 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.149 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.144+0000 7efcbbadd700 1 --2- 172.21.15.28:0/2339902323 >> v2:172.21.15.59:3300/0 conn(0x7efcb40d0630 0x7efcb40b4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.150 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.144+0000 7efcbbadd700 1 -- 172.21.15.28:0/2339902323 >> 172.21.15.28:0/2339902323 conn(0x7efcb40b3500 msgr2=0x7efcb40cf9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:38.150 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.145+0000 7efcbbadd700 1 -- 172.21.15.28:0/2339902323 shutdown_connections 2023-11-28T11:08:38.150 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.145+0000 7efcbbadd700 1 -- 172.21.15.28:0/2339902323 wait complete. 2023-11-28T11:08:38.152 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.146+0000 7efcbbadd700 1 Processor -- start 2023-11-28T11:08:38.153 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.146+0000 7efcbbadd700 1 -- start start 2023-11-28T11:08:38.153 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.146+0000 7efcbbadd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcb40b4ed0 0x7efcb4151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.153 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcbbadd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb40d0630 0x7efcb40c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.153 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcbbadd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efcb415a7a0 0x7efcb40ae530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.153 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcbbadd700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb4151f80 con 0x7efcb40b4ed0 2023-11-28T11:08:38.153 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcbbadd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb415d3d0 con 0x7efcb40d0630 2023-11-28T11:08:38.154 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcbbadd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb415d6b0 con 0x7efcb415a7a0 2023-11-28T11:08:38.154 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcb9879700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcb40b4ed0 0x7efcb4151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.154 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcb9879700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcb40b4ed0 0x7efcb4151a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46084/0 (socket says 172.21.15.28:46084) 2023-11-28T11:08:38.154 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcb9879700 1 -- 172.21.15.28:0/2355461428 learned_addr learned my addr 172.21.15.28:0/2355461428 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:38.154 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcb9078700 1 --2- 172.21.15.28:0/2355461428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb40d0630 0x7efcb40c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.155 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcba07a700 1 --2- 172.21.15.28:0/2355461428 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efcb415a7a0 0x7efcb40ae530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.155 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcb9879700 1 -- 172.21.15.28:0/2355461428 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efcb415a7a0 msgr2=0x7efcb40ae530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.155 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcb9879700 1 --2- 172.21.15.28:0/2355461428 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efcb415a7a0 0x7efcb40ae530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.155 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcb9879700 1 -- 172.21.15.28:0/2355461428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb40d0630 msgr2=0x7efcb40c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.155 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcb9879700 1 --2- 172.21.15.28:0/2355461428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb40d0630 0x7efcb40c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.155 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcb9879700 1 -- 172.21.15.28:0/2355461428 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efca4009550 con 0x7efcb40b4ed0 2023-11-28T11:08:38.155 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcb9879700 1 --2- 172.21.15.28:0/2355461428 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcb40b4ed0 0x7efcb4151a40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efca4000c00 tx=0x7efca4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:38.155 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcaa7fc700 1 -- 172.21.15.28:0/2355461428 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca4004520 con 0x7efcb40b4ed0 2023-11-28T11:08:38.156 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcaa7fc700 1 -- 172.21.15.28:0/2355461428 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efca401e070 con 0x7efcb40b4ed0 2023-11-28T11:08:38.156 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcaa7fc700 1 -- 172.21.15.28:0/2355461428 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efca40038e0 con 0x7efcb40b4ed0 2023-11-28T11:08:38.156 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcbbadd700 1 -- 172.21.15.28:0/2355461428 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb40cb020 con 0x7efcb40b4ed0 2023-11-28T11:08:38.156 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.147+0000 7efcbbadd700 1 -- 172.21.15.28:0/2355461428 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb40cad40 con 0x7efcb40b4ed0 2023-11-28T11:08:38.156 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.148+0000 7efcbbadd700 1 -- 172.21.15.28:0/2355461428 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb406f0d0 con 0x7efcb40b4ed0 2023-11-28T11:08:38.156 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.149+0000 7efcaa7fc700 1 -- 172.21.15.28:0/2355461428 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7efca400f050 con 0x7efcb40b4ed0 2023-11-28T11:08:38.156 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.149+0000 7efcaa7fc700 1 --2- 172.21.15.28:0/2355461428 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efc980682a0 0x7efc9806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.157 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.149+0000 7efcaa7fc700 1 -- 172.21.15.28:0/2355461428 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(13..13 src has 1..13) v4 ==== 4471+0+0 (secure 0 0 0) 0x7efca4020070 con 0x7efcb40b4ed0 2023-11-28T11:08:38.157 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.149+0000 7efcb9078700 1 --2- 172.21.15.28:0/2355461428 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efc980682a0 0x7efc9806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.157 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.150+0000 7efcb9078700 1 --2- 172.21.15.28:0/2355461428 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efc980682a0 0x7efc9806a750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efcb40af1d0 tx=0x7efcb0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:38.157 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.154+0000 7efcaa7fc700 1 -- 172.21.15.28:0/2355461428 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efca40539e0 con 0x7efcb40b4ed0 2023-11-28T11:08:38.353 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.351+0000 7efcbbadd700 1 -- 172.21.15.28:0/2355461428 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efcb40675a0 con 0x7efcb40b4ed0 2023-11-28T11:08:38.353 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.352+0000 7efcaa7fc700 1 -- 172.21.15.28:0/2355461428 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v13) v1 ==== 74+0+10488 (secure 0 0 0) 0x7efca402a090 con 0x7efcb40b4ed0 2023-11-28T11:08:38.354 INFO:teuthology.misc.health.smithi028.stdout: 2023-11-28T11:08:38.354 INFO:teuthology.misc.health.smithi028.stdout:{"epoch":13,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:37.353927+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:38.356 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.354+0000 7efcbbadd700 1 -- 172.21.15.28:0/2355461428 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efc980682a0 msgr2=0x7efc9806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.356 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.354+0000 7efcbbadd700 1 --2- 172.21.15.28:0/2355461428 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efc980682a0 0x7efc9806a750 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7efcb40af1d0 tx=0x7efcb0008040 comp rx=0 tx=0).stop 2023-11-28T11:08:38.356 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.354+0000 7efcbbadd700 1 -- 172.21.15.28:0/2355461428 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcb40b4ed0 msgr2=0x7efcb4151a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.356 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.354+0000 7efcbbadd700 1 --2- 172.21.15.28:0/2355461428 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcb40b4ed0 0x7efcb4151a40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7efca4000c00 tx=0x7efca4004750 comp rx=0 tx=0).stop 2023-11-28T11:08:38.357 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.354+0000 7efcbbadd700 1 -- 172.21.15.28:0/2355461428 shutdown_connections 2023-11-28T11:08:38.357 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.354+0000 7efcbbadd700 1 --2- 172.21.15.28:0/2355461428 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efc980682a0 0x7efc9806a750 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.357 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.354+0000 7efcbbadd700 1 --2- 172.21.15.28:0/2355461428 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efcb40b4ed0 0x7efcb4151a40 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.357 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.354+0000 7efcbbadd700 1 --2- 172.21.15.28:0/2355461428 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efcb415a7a0 0x7efcb40ae530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.357 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.354+0000 7efcbbadd700 1 --2- 172.21.15.28:0/2355461428 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efcb40d0630 0x7efcb40c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.357 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.354+0000 7efcbbadd700 1 -- 172.21.15.28:0/2355461428 >> 172.21.15.28:0/2355461428 conn(0x7efcb40b3500 msgr2=0x7efcb40adae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:38.357 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.354+0000 7efcbbadd700 1 -- 172.21.15.28:0/2355461428 shutdown_connections 2023-11-28T11:08:38.358 INFO:teuthology.misc.health.smithi028.stderr:2023-11-28T11:08:38.354+0000 7efcbbadd700 1 -- 172.21.15.28:0/2355461428 wait complete. 2023-11-28T11:08:38.367 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-28T11:08:38.367 INFO:tasks.ceph:Creating RBD pool 2023-11-28T11:08:38.367 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-28T11:08:38.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.489+0000 7f1f8d142700 1 Processor -- start 2023-11-28T11:08:38.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.490+0000 7f1f8d142700 1 -- start start 2023-11-28T11:08:38.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.490+0000 7f1f8d142700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1f880b1d40 0x7f1f880cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.490+0000 7f1f8d142700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f8815dbf0 0x7f1f880ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.490+0000 7f1f8d142700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f880ceff0 con 0x7f1f88159500 2023-11-28T11:08:38.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.490+0000 7f1f8d142700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f8815e090 con 0x7f1f880b1d40 2023-11-28T11:08:38.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.490+0000 7f1f8d142700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f880cede0 con 0x7f1f8815dbf0 2023-11-28T11:08:38.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.490+0000 7f1f8659c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1f880b1d40 0x7f1f880cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.490+0000 7f1f86d9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1f88159500 0x7f1f880b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34158/0 (socket says 172.21.15.28:34158) 2023-11-28T11:08:38.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.490+0000 7f1f8659c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1f880b1d40 0x7f1f880cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46102/0 (socket says 172.21.15.28:46102) 2023-11-28T11:08:38.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.490+0000 7f1f86d9d700 1 -- 172.21.15.28:0/2321770085 learned_addr learned my addr 172.21.15.28:0/2321770085 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:38.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.491+0000 7f1f8759e700 1 --2- 172.21.15.28:0/2321770085 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f8815dbf0 0x7f1f880ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.491+0000 7f1f8659c700 1 -- 172.21.15.28:0/2321770085 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f8815dbf0 msgr2=0x7f1f880ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:38.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.491+0000 7f1f8659c700 1 --2- 172.21.15.28:0/2321770085 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f8815dbf0 0x7f1f880ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.491+0000 7f1f8659c700 1 -- 172.21.15.28:0/2321770085 >> v1:172.21.15.28:6789/0 conn(0x7f1f88159500 legacy=0x7f1f880b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.491+0000 7f1f85d9b700 1 -- 172.21.15.28:0/2321770085 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4263417455 0 0) 0x7f1f880ceff0 con 0x7f1f88159500 2023-11-28T11:08:38.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.491+0000 7f1f8659c700 1 -- 172.21.15.28:0/2321770085 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f880cf2d0 con 0x7f1f880b1d40 2023-11-28T11:08:38.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.491+0000 7f1f8759e700 1 --2- 172.21.15.28:0/2321770085 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f8815dbf0 0x7f1f880ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:38.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.491+0000 7f1f8659c700 1 --2- 172.21.15.28:0/2321770085 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f880b1d40 0x7f1f880cd140 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009840 tx=0x7f1f70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=116ee8cb4b2fc0dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:38.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.491+0000 7f1f85d9b700 1 -- 172.21.15.28:0/2321770085 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70011830 con 0x7f1f880b1d40 2023-11-28T11:08:38.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.491+0000 7f1f85d9b700 1 -- 172.21.15.28:0/2321770085 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f70011990 con 0x7f1f880b1d40 2023-11-28T11:08:38.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.491+0000 7f1f85d9b700 1 -- 172.21.15.28:0/2321770085 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f70011d20 con 0x7f1f880b1d40 2023-11-28T11:08:38.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.492+0000 7f1f8d142700 1 -- 172.21.15.28:0/2321770085 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f880b1d40 msgr2=0x7f1f880cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.492+0000 7f1f8d142700 1 --2- 172.21.15.28:0/2321770085 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f880b1d40 0x7f1f880cd140 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009840 tx=0x7f1f70009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:38.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.492+0000 7f1f8d142700 1 -- 172.21.15.28:0/2321770085 shutdown_connections 2023-11-28T11:08:38.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.492+0000 7f1f8d142700 1 --2- 172.21.15.28:0/2321770085 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f8815dbf0 0x7f1f880ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.492+0000 7f1f8d142700 1 --2- 172.21.15.28:0/2321770085 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f880b1d40 0x7f1f880cd140 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.492+0000 7f1f8d142700 1 -- 172.21.15.28:0/2321770085 >> 172.21.15.28:0/2321770085 conn(0x7f1f880cc630 msgr2=0x7f1f880cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:38.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.492+0000 7f1f8d142700 1 -- 172.21.15.28:0/2321770085 shutdown_connections 2023-11-28T11:08:38.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.492+0000 7f1f8d142700 1 -- 172.21.15.28:0/2321770085 wait complete. 2023-11-28T11:08:38.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.493+0000 7f1f8d142700 1 Processor -- start 2023-11-28T11:08:38.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.493+0000 7f1f8d142700 1 -- start start 2023-11-28T11:08:38.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.494+0000 7f1f8d142700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f880b1d40 0x7f1f88144c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.494+0000 7f1f8d142700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f88159500 0x7f1f8806bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.494+0000 7f1f8d142700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f8815dbf0 0x7f1f8806c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.494+0000 7f1f8d142700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8806b480 con 0x7f1f880b1d40 2023-11-28T11:08:38.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.494+0000 7f1f8d142700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8806b760 con 0x7f1f88159500 2023-11-28T11:08:38.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.494+0000 7f1f8d142700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f881f21b0 con 0x7f1f8815dbf0 2023-11-28T11:08:38.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.494+0000 7f1f86d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f880b1d40 0x7f1f88144c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.494+0000 7f1f86d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f880b1d40 0x7f1f88144c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46106/0 (socket says 172.21.15.28:46106) 2023-11-28T11:08:38.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.494+0000 7f1f86d9d700 1 -- 172.21.15.28:0/1873461983 learned_addr learned my addr 172.21.15.28:0/1873461983 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:38.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.494+0000 7f1f86d9d700 1 -- 172.21.15.28:0/1873461983 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f8815dbf0 msgr2=0x7f1f8806c130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.494+0000 7f1f86d9d700 1 --2- 172.21.15.28:0/1873461983 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f8815dbf0 0x7f1f8806c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.494+0000 7f1f86d9d700 1 -- 172.21.15.28:0/1873461983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f88159500 msgr2=0x7f1f8806bbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:38.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.494+0000 7f1f86d9d700 1 --2- 172.21.15.28:0/1873461983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f88159500 0x7f1f8806bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:38.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.494+0000 7f1f86d9d700 1 -- 172.21.15.28:0/1873461983 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f700094a0 con 0x7f1f880b1d40 2023-11-28T11:08:38.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.495+0000 7f1f86d9d700 1 --2- 172.21.15.28:0/1873461983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f880b1d40 0x7f1f88144c20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800c750 tx=0x7f1f7800cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:38.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.495+0000 7f1f7f7fe700 1 -- 172.21.15.28:0/1873461983 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78004c80 con 0x7f1f880b1d40 2023-11-28T11:08:38.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.495+0000 7f1f7f7fe700 1 -- 172.21.15.28:0/1873461983 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f78004e00 con 0x7f1f880b1d40 2023-11-28T11:08:38.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.495+0000 7f1f7f7fe700 1 -- 172.21.15.28:0/1873461983 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78007310 con 0x7f1f880b1d40 2023-11-28T11:08:38.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.495+0000 7f1f8d142700 1 -- 172.21.15.28:0/1873461983 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8815d3f0 con 0x7f1f880b1d40 2023-11-28T11:08:38.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.495+0000 7f1f8d142700 1 -- 172.21.15.28:0/1873461983 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8815d110 con 0x7f1f880b1d40 2023-11-28T11:08:38.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.496+0000 7f1f8d142700 1 -- 172.21.15.28:0/1873461983 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f8815d6d0 con 0x7f1f880b1d40 2023-11-28T11:08:38.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.496+0000 7f1f7f7fe700 1 -- 172.21.15.28:0/1873461983 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f1f7801a020 con 0x7f1f880b1d40 2023-11-28T11:08:38.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.496+0000 7f1f7f7fe700 1 --2- 172.21.15.28:0/1873461983 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f680682a0 0x7f1f6806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:38.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.497+0000 7f1f7f7fe700 1 -- 172.21.15.28:0/1873461983 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4471+0+0 (secure 0 0 0) 0x7f1f78085290 con 0x7f1f880b1d40 2023-11-28T11:08:38.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.497+0000 7f1f8659c700 1 --2- 172.21.15.28:0/1873461983 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f680682a0 0x7f1f6806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:38.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.497+0000 7f1f8659c700 1 --2- 172.21.15.28:0/1873461983 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f680682a0 0x7f1f6806a750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009840 tx=0x7f1f70010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:38.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.501+0000 7f1f7f7fe700 1 -- 172.21.15.28:0/1873461983 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1f78052220 con 0x7f1f880b1d40 2023-11-28T11:08:38.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:38.696+0000 7f1f8d142700 1 -- 172.21.15.28:0/1873461983 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f1f880739f0 con 0x7f1f880b1d40 2023-11-28T11:08:39.377 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.376+0000 7f1f7f7fe700 1 -- 172.21.15.28:0/1873461983 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v15) v1 ==== 109+0+0 (secure 0 0 0) 0x7f1f78055840 con 0x7f1f880b1d40 2023-11-28T11:08:39.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.379+0000 7f1f8d142700 1 -- 172.21.15.28:0/1873461983 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f680682a0 msgr2=0x7f1f6806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.379+0000 7f1f8d142700 1 --2- 172.21.15.28:0/1873461983 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f680682a0 0x7f1f6806a750 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1f70009840 tx=0x7f1f70010000 comp rx=0 tx=0).stop 2023-11-28T11:08:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.379+0000 7f1f8d142700 1 -- 172.21.15.28:0/1873461983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f880b1d40 msgr2=0x7f1f88144c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.379+0000 7f1f8d142700 1 --2- 172.21.15.28:0/1873461983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f880b1d40 0x7f1f88144c20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f1f7800c750 tx=0x7f1f7800cb10 comp rx=0 tx=0).stop 2023-11-28T11:08:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.379+0000 7f1f8d142700 1 -- 172.21.15.28:0/1873461983 shutdown_connections 2023-11-28T11:08:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.379+0000 7f1f8d142700 1 --2- 172.21.15.28:0/1873461983 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f680682a0 0x7f1f6806a750 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.379+0000 7f1f8d142700 1 --2- 172.21.15.28:0/1873461983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f880b1d40 0x7f1f88144c20 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.379+0000 7f1f8d142700 1 --2- 172.21.15.28:0/1873461983 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f8815dbf0 0x7f1f8806c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.379+0000 7f1f8d142700 1 --2- 172.21.15.28:0/1873461983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f88159500 0x7f1f8806bbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.379+0000 7f1f8d142700 1 -- 172.21.15.28:0/1873461983 >> 172.21.15.28:0/1873461983 conn(0x7f1f880cc630 msgr2=0x7f1f88156e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:39.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.379+0000 7f1f8d142700 1 -- 172.21.15.28:0/1873461983 shutdown_connections 2023-11-28T11:08:39.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.379+0000 7f1f8d142700 1 -- 172.21.15.28:0/1873461983 wait complete. 2023-11-28T11:08:39.382 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'rbd' created 2023-11-28T11:08:39.395 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-28T11:08:39.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.526+0000 7f66f65bf700 1 Processor -- start 2023-11-28T11:08:39.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.526+0000 7f66f65bf700 1 -- start start 2023-11-28T11:08:39.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.527+0000 7f66f65bf700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f66f015c560 0x7f66f006f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:39.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.527+0000 7f66f65bf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f66f00b2660 0x7f66f00ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:39.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.527+0000 7f66f65bf700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f00b30b0 con 0x7f66f00cecf0 2023-11-28T11:08:39.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.527+0000 7f66f65bf700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f006c020 con 0x7f66f015c560 2023-11-28T11:08:39.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.527+0000 7f66f65bf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f00cf110 con 0x7f66f00b2660 2023-11-28T11:08:39.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.527+0000 7f66effff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f66f015c560 0x7f66f006f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:39.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.527+0000 7f66effff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f66f015c560 0x7f66f006f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46152/0 (socket says 172.21.15.28:46152) 2023-11-28T11:08:39.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.527+0000 7f66effff700 1 -- 172.21.15.28:0/3875436573 learned_addr learned my addr 172.21.15.28:0/3875436573 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:39.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.528+0000 7f66ef7fe700 1 --2- 172.21.15.28:0/3875436573 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f66f00b2660 0x7f66f00ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:39.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.528+0000 7f66effff700 1 -- 172.21.15.28:0/3875436573 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f66f00b2660 msgr2=0x7f66f00ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:39.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.528+0000 7f66effff700 1 --2- 172.21.15.28:0/3875436573 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f66f00b2660 0x7f66f00ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.528+0000 7f66effff700 1 -- 172.21.15.28:0/3875436573 >> v1:172.21.15.28:6789/0 conn(0x7f66f00cecf0 legacy=0x7f66f0160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.528+0000 7f66effff700 1 -- 172.21.15.28:0/3875436573 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f00d09c0 con 0x7f66f015c560 2023-11-28T11:08:39.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.528+0000 7f66eeffd700 1 -- 172.21.15.28:0/3875436573 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1477195494 0 0) 0x7f66f00b30b0 con 0x7f66f00cecf0 2023-11-28T11:08:39.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.528+0000 7f66effff700 1 --2- 172.21.15.28:0/3875436573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f015c560 0x7f66f006f8b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f66e00098f0 tx=0x7f66e0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76da9681b635c9bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:39.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.528+0000 7f66eeffd700 1 -- 172.21.15.28:0/3875436573 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e000b8a0 con 0x7f66f015c560 2023-11-28T11:08:39.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.528+0000 7f66eeffd700 1 -- 172.21.15.28:0/3875436573 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e000ba00 con 0x7f66f015c560 2023-11-28T11:08:39.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.528+0000 7f66eeffd700 1 -- 172.21.15.28:0/3875436573 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e000bd90 con 0x7f66f015c560 2023-11-28T11:08:39.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.529+0000 7f66f65bf700 1 -- 172.21.15.28:0/3875436573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f015c560 msgr2=0x7f66f006f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.529+0000 7f66f65bf700 1 --2- 172.21.15.28:0/3875436573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f015c560 0x7f66f006f8b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f66e00098f0 tx=0x7f66e0009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:39.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.529+0000 7f66f65bf700 1 -- 172.21.15.28:0/3875436573 shutdown_connections 2023-11-28T11:08:39.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.529+0000 7f66f65bf700 1 --2- 172.21.15.28:0/3875436573 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f015c560 0x7f66f006f8b0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.529+0000 7f66f65bf700 1 --2- 172.21.15.28:0/3875436573 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f66f00b2660 0x7f66f00ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.529+0000 7f66f65bf700 1 -- 172.21.15.28:0/3875436573 >> 172.21.15.28:0/3875436573 conn(0x7f66f00cd660 msgr2=0x7f66f00cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:39.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.529+0000 7f66f65bf700 1 -- 172.21.15.28:0/3875436573 shutdown_connections 2023-11-28T11:08:39.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.529+0000 7f66f65bf700 1 -- 172.21.15.28:0/3875436573 wait complete. 2023-11-28T11:08:39.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.530+0000 7f66f65bf700 1 Processor -- start 2023-11-28T11:08:39.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.530+0000 7f66f65bf700 1 -- start start 2023-11-28T11:08:39.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66f65bf700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f00b2660 0x7f66f00ab9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:39.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66f65bf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f66f00cecf0 0x7f66f00b5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:39.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66f65bf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66f015c560 0x7f66f00c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:39.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66f65bf700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f00af790 con 0x7f66f00b2660 2023-11-28T11:08:39.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66f65bf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f00c5760 con 0x7f66f015c560 2023-11-28T11:08:39.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66f65bf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f01f28c0 con 0x7f66f00cecf0 2023-11-28T11:08:39.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66effff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f00b2660 0x7f66f00ab9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:39.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66effff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f00b2660 0x7f66f00ab9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46158/0 (socket says 172.21.15.28:46158) 2023-11-28T11:08:39.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66effff700 1 -- 172.21.15.28:0/2932730641 learned_addr learned my addr 172.21.15.28:0/2932730641 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:39.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66effff700 1 -- 172.21.15.28:0/2932730641 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f66f00cecf0 msgr2=0x7f66f00b5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66ef7fe700 1 --2- 172.21.15.28:0/2932730641 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f66f00cecf0 0x7f66f00b5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:39.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66f4b5c700 1 --2- 172.21.15.28:0/2932730641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66f015c560 0x7f66f00c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:39.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66effff700 1 --2- 172.21.15.28:0/2932730641 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f66f00cecf0 0x7f66f00b5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66effff700 1 -- 172.21.15.28:0/2932730641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66f015c560 msgr2=0x7f66f00c7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:39.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66effff700 1 --2- 172.21.15.28:0/2932730641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66f015c560 0x7f66f00c7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:39.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66effff700 1 -- 172.21.15.28:0/2932730641 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e0009550 con 0x7f66f00b2660 2023-11-28T11:08:39.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66effff700 1 --2- 172.21.15.28:0/2932730641 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f00b2660 0x7f66f00ab9c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f66e0000c00 tx=0x7f66e0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:39.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66ecff9700 1 -- 172.21.15.28:0/2932730641 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e00038a0 con 0x7f66f00b2660 2023-11-28T11:08:39.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66ecff9700 1 -- 172.21.15.28:0/2932730641 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e001e070 con 0x7f66f00b2660 2023-11-28T11:08:39.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.531+0000 7f66ecff9700 1 -- 172.21.15.28:0/2932730641 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e0003c20 con 0x7f66f00b2660 2023-11-28T11:08:39.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.532+0000 7f66f65bf700 1 -- 172.21.15.28:0/2932730641 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f00b2af0 con 0x7f66f00b2660 2023-11-28T11:08:39.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.532+0000 7f66f65bf700 1 -- 172.21.15.28:0/2932730641 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f006fdf0 con 0x7f66f00b2660 2023-11-28T11:08:39.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.532+0000 7f66f65bf700 1 -- 172.21.15.28:0/2932730641 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f00b2dd0 con 0x7f66f00b2660 2023-11-28T11:08:39.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.533+0000 7f66ecff9700 1 -- 172.21.15.28:0/2932730641 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f66e000f050 con 0x7f66f00b2660 2023-11-28T11:08:39.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.533+0000 7f66ecff9700 1 --2- 172.21.15.28:0/2932730641 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f66dc0682a0 0x7f66dc06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:39.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.534+0000 7f66ef7fe700 1 --2- 172.21.15.28:0/2932730641 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f66dc0682a0 0x7f66dc06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:39.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.534+0000 7f66ecff9700 1 -- 172.21.15.28:0/2932730641 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 5188+0+0 (secure 0 0 0) 0x7f66e0020070 con 0x7f66f00b2660 2023-11-28T11:08:39.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.534+0000 7f66ef7fe700 1 --2- 172.21.15.28:0/2932730641 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f66dc0682a0 0x7f66dc06a750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66f006fc90 tx=0x7f66e4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:39.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.538+0000 7f66ecff9700 1 -- 172.21.15.28:0/2932730641 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f66e0053b20 con 0x7f66f00b2660 2023-11-28T11:08:39.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:39.735+0000 7f66f65bf700 1 -- 172.21.15.28:0/2932730641 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f66f00b6880 con 0x7f66f00b2660 2023-11-28T11:08:40.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.378+0000 7f66ecff9700 1 -- 172.21.15.28:0/2932730641 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v16) v1 ==== 173+0+0 (secure 0 0 0) 0x7f66e00150c0 con 0x7f66f00b2660 2023-11-28T11:08:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.381+0000 7f66f65bf700 1 -- 172.21.15.28:0/2932730641 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f66dc0682a0 msgr2=0x7f66dc06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.381+0000 7f66f65bf700 1 --2- 172.21.15.28:0/2932730641 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f66dc0682a0 0x7f66dc06a750 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66f006fc90 tx=0x7f66e4008040 comp rx=0 tx=0).stop 2023-11-28T11:08:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.381+0000 7f66f65bf700 1 -- 172.21.15.28:0/2932730641 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f00b2660 msgr2=0x7f66f00ab9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.381+0000 7f66f65bf700 1 --2- 172.21.15.28:0/2932730641 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f00b2660 0x7f66f00ab9c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f66e0000c00 tx=0x7f66e0004750 comp rx=0 tx=0).stop 2023-11-28T11:08:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.381+0000 7f66f65bf700 1 -- 172.21.15.28:0/2932730641 shutdown_connections 2023-11-28T11:08:40.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.381+0000 7f66f65bf700 1 --2- 172.21.15.28:0/2932730641 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f66dc0682a0 0x7f66dc06a750 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.381+0000 7f66f65bf700 1 --2- 172.21.15.28:0/2932730641 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f66f00b2660 0x7f66f00ab9c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.381+0000 7f66f65bf700 1 --2- 172.21.15.28:0/2932730641 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66f015c560 0x7f66f00c7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.381+0000 7f66f65bf700 1 --2- 172.21.15.28:0/2932730641 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f66f00cecf0 0x7f66f00b5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.381+0000 7f66f65bf700 1 -- 172.21.15.28:0/2932730641 >> 172.21.15.28:0/2932730641 conn(0x7f66f00cd660 msgr2=0x7f66f00b4290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:40.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.382+0000 7f66f65bf700 1 -- 172.21.15.28:0/2932730641 shutdown_connections 2023-11-28T11:08:40.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.382+0000 7f66f65bf700 1 -- 172.21.15.28:0/2932730641 wait complete. 2023-11-28T11:08:40.384 INFO:teuthology.orchestra.run.smithi028.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-28T11:08:40.400 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-28T11:08:40.401 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:08:40.401 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:08:40.403 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:08:40.403 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:08:40.403 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:08:40.405 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:08:40.405 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:08:40.406 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:08:40.408 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:08:40.408 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:08:40.408 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:08:40.410 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:08:40.410 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-28T11:08:40.411 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:40.503 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2023-11-28T11:08:40.504 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2023-11-28T11:08:40.506 INFO:tasks.ceph.mds.b.smithi059.stdout:starting mds.b at 2023-11-28T11:08:40.507 INFO:tasks.ceph.mds.d.smithi059.stdout:starting mds.d at 2023-11-28T11:08:40.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.560+0000 7f3adecf0700 1 Processor -- start 2023-11-28T11:08:40.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.561+0000 7f3adecf0700 1 -- start start 2023-11-28T11:08:40.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.561+0000 7f3adecf0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ad80b2660 0x7f3ad80b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.561+0000 7f3adecf0700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad80cfb90 con 0x7f3ad815a6c0 2023-11-28T11:08:40.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.561+0000 7f3adecf0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ad80cb680 con 0x7f3ad80cb230 2023-11-28T11:08:40.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.561+0000 7f3adecf0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80b3020 con 0x7f3ad80b2660 2023-11-28T11:08:40.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.561+0000 7f3adca8c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3ad815a6c0 0x7f3ad80d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34270/0 (socket says 172.21.15.28:34270) 2023-11-28T11:08:40.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.561+0000 7f3adca8c700 1 -- 172.21.15.28:0/29524095 learned_addr learned my addr 172.21.15.28:0/29524095 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:40.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.562+0000 7f3add28d700 1 --2- 172.21.15.28:0/29524095 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ad80b2660 0x7f3ad80b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.562+0000 7f3ad77fe700 1 -- 172.21.15.28:0/29524095 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2085862231 0 0) 0x7f3ad80cfb90 con 0x7f3ad815a6c0 2023-11-28T11:08:40.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.562+0000 7f3ad77fe700 1 -- 172.21.15.28:0/29524095 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ac80035b0 con 0x7f3ad815a6c0 2023-11-28T11:08:40.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.562+0000 7f3ad77fe700 1 -- 172.21.15.28:0/29524095 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3166171651 0 0) 0x7f3ad80cb680 con 0x7f3ad80cb230 2023-11-28T11:08:40.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.562+0000 7f3ad77fe700 1 -- 172.21.15.28:0/29524095 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ad80cfb90 con 0x7f3ad80cb230 2023-11-28T11:08:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.562+0000 7f3ad77fe700 1 -- 172.21.15.28:0/29524095 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3914855693 0 0) 0x7f3ac80035b0 con 0x7f3ad815a6c0 2023-11-28T11:08:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.562+0000 7f3add28d700 1 -- 172.21.15.28:0/29524095 >> v1:172.21.15.59:6789/0 conn(0x7f3ad80cb230 legacy=0x7f3ad80ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.562+0000 7f3add28d700 1 -- 172.21.15.28:0/29524095 >> v1:172.21.15.28:6789/0 conn(0x7f3ad815a6c0 legacy=0x7f3ad80d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.562+0000 7f3add28d700 1 -- 172.21.15.28:0/29524095 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad80cb930 con 0x7f3ad80b2660 2023-11-28T11:08:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.562+0000 7f3add28d700 1 --2- 172.21.15.28:0/29524095 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ad80b2660 0x7f3ad80b2ae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009dc0 tx=0x7f3ad000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65c5c6991fcd63dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.562+0000 7f3ad77fe700 1 -- 172.21.15.28:0/29524095 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad000e070 con 0x7f3ad80b2660 2023-11-28T11:08:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.562+0000 7f3ad77fe700 1 -- 172.21.15.28:0/29524095 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad0003a20 con 0x7f3ad80b2660 2023-11-28T11:08:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.562+0000 7f3ad77fe700 1 -- 172.21.15.28:0/29524095 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc001670 con 0x7f3ad80b2660 2023-11-28T11:08:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.563+0000 7f3adecf0700 1 -- 172.21.15.28:0/29524095 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ad80b2660 msgr2=0x7f3ad80b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.563+0000 7f3adecf0700 1 --2- 172.21.15.28:0/29524095 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ad80b2660 0x7f3ad80b2ae0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009dc0 tx=0x7f3ad000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:40.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.563+0000 7f3adecf0700 1 -- 172.21.15.28:0/29524095 shutdown_connections 2023-11-28T11:08:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.563+0000 7f3adecf0700 1 --2- 172.21.15.28:0/29524095 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ad80b2660 0x7f3ad80b2ae0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.563+0000 7f3adecf0700 1 -- 172.21.15.28:0/29524095 >> 172.21.15.28:0/29524095 conn(0x7f3ad80cd660 msgr2=0x7f3ad80cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.563+0000 7f3adecf0700 1 -- 172.21.15.28:0/29524095 shutdown_connections 2023-11-28T11:08:40.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.563+0000 7f3adecf0700 1 -- 172.21.15.28:0/29524095 wait complete. 2023-11-28T11:08:40.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.564+0000 7f3adecf0700 1 Processor -- start 2023-11-28T11:08:40.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.564+0000 7f3adecf0700 1 -- start start 2023-11-28T11:08:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.565+0000 7f3adecf0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ad80b2660 0x7f3ad815ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.565+0000 7f3adecf0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ad80cb230 0x7f3ad8153770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.565+0000 7f3adecf0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ad815a6c0 0x7f3ad81efef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.565+0000 7f3adecf0700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80c7cb0 con 0x7f3ad80b2660 2023-11-28T11:08:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.565+0000 7f3adecf0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80c87f0 con 0x7f3ad80cb230 2023-11-28T11:08:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.565+0000 7f3adecf0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad80c7ec0 con 0x7f3ad815a6c0 2023-11-28T11:08:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.565+0000 7f3adca8c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ad80b2660 0x7f3ad815ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.565+0000 7f3adca8c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ad80b2660 0x7f3ad815ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46218/0 (socket says 172.21.15.28:46218) 2023-11-28T11:08:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.565+0000 7f3adca8c700 1 -- 172.21.15.28:0/1581658776 learned_addr learned my addr 172.21.15.28:0/1581658776 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.565+0000 7f3add28d700 1 --2- 172.21.15.28:0/1581658776 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ad815a6c0 0x7f3ad81efef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.565+0000 7f3ad7fff700 1 --2- 172.21.15.28:0/1581658776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ad80cb230 0x7f3ad8153770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.565+0000 7f3adca8c700 1 -- 172.21.15.28:0/1581658776 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ad815a6c0 msgr2=0x7f3ad81efef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.565+0000 7f3adca8c700 1 --2- 172.21.15.28:0/1581658776 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ad815a6c0 0x7f3ad81efef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.565+0000 7f3adca8c700 1 -- 172.21.15.28:0/1581658776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ad80cb230 msgr2=0x7f3ad8153770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.565+0000 7f3adca8c700 1 --2- 172.21.15.28:0/1581658776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ad80cb230 0x7f3ad8153770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.566+0000 7f3adca8c700 1 -- 172.21.15.28:0/1581658776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad0009a70 con 0x7f3ad80b2660 2023-11-28T11:08:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.566+0000 7f3ad7fff700 1 --2- 172.21.15.28:0/1581658776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ad80cb230 0x7f3ad8153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.566+0000 7f3adca8c700 1 --2- 172.21.15.28:0/1581658776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ad80b2660 0x7f3ad815ceb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00c770 tx=0x7f3acc00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.566+0000 7f3ad57fa700 1 -- 172.21.15.28:0/1581658776 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc007560 con 0x7f3ad80b2660 2023-11-28T11:08:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.566+0000 7f3ad57fa700 1 -- 172.21.15.28:0/1581658776 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3acc0076c0 con 0x7f3ad80b2660 2023-11-28T11:08:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.566+0000 7f3adecf0700 1 -- 172.21.15.28:0/1581658776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad80cf540 con 0x7f3ad80b2660 2023-11-28T11:08:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.566+0000 7f3ad57fa700 1 -- 172.21.15.28:0/1581658776 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3acc004a70 con 0x7f3ad80b2660 2023-11-28T11:08:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.566+0000 7f3adecf0700 1 -- 172.21.15.28:0/1581658776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad80cf260 con 0x7f3ad80b2660 2023-11-28T11:08:40.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.567+0000 7f3adecf0700 1 -- 172.21.15.28:0/1581658776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad80cf820 con 0x7f3ad80b2660 2023-11-28T11:08:40.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.567+0000 7f3ad57fa700 1 -- 172.21.15.28:0/1581658776 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f3acc010070 con 0x7f3ad80b2660 2023-11-28T11:08:40.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.568+0000 7f3ad57fa700 1 --2- 172.21.15.28:0/1581658776 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3abc0682a0 0x7f3abc06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.568+0000 7f3ad57fa700 1 -- 172.21.15.28:0/1581658776 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f3acc084b60 con 0x7f3ad80b2660 2023-11-28T11:08:40.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.568+0000 7f3ad7fff700 1 --2- 172.21.15.28:0/1581658776 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3abc0682a0 0x7f3abc06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.568+0000 7f3ad7fff700 1 --2- 172.21.15.28:0/1581658776 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3abc0682a0 0x7f3abc06a750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4007fd0 tx=0x7f3ac4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.572+0000 7f3ad57fa700 1 -- 172.21.15.28:0/1581658776 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3acc051680 con 0x7f3ad80b2660 2023-11-28T11:08:40.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.776+0000 7f3adecf0700 1 -- 172.21.15.28:0/1581658776 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ad80c77e0 con 0x7f3ad80b2660 2023-11-28T11:08:40.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.777+0000 7f3ad57fa700 1 -- 172.21.15.28:0/1581658776 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+14185 (secure 0 0 0) 0x7f3acc018070 con 0x7f3ad80b2660 2023-11-28T11:08:40.779 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:08:40.779 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":16,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:40.375400+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:40.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.779+0000 7f3adecf0700 1 -- 172.21.15.28:0/1581658776 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3abc0682a0 msgr2=0x7f3abc06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.779+0000 7f3adecf0700 1 --2- 172.21.15.28:0/1581658776 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3abc0682a0 0x7f3abc06a750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3ac4007fd0 tx=0x7f3ac4009040 comp rx=0 tx=0).stop 2023-11-28T11:08:40.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.779+0000 7f3adecf0700 1 -- 172.21.15.28:0/1581658776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ad80b2660 msgr2=0x7f3ad815ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.779+0000 7f3adecf0700 1 --2- 172.21.15.28:0/1581658776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ad80b2660 0x7f3ad815ceb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f3acc00c770 tx=0x7f3acc00cb30 comp rx=0 tx=0).stop 2023-11-28T11:08:40.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.779+0000 7f3adecf0700 1 -- 172.21.15.28:0/1581658776 shutdown_connections 2023-11-28T11:08:40.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.779+0000 7f3adecf0700 1 --2- 172.21.15.28:0/1581658776 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3abc0682a0 0x7f3abc06a750 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.779+0000 7f3adecf0700 1 --2- 172.21.15.28:0/1581658776 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ad80b2660 0x7f3ad815ceb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.779+0000 7f3adecf0700 1 --2- 172.21.15.28:0/1581658776 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ad815a6c0 0x7f3ad81efef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.779+0000 7f3adecf0700 1 --2- 172.21.15.28:0/1581658776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ad80cb230 0x7f3ad8153770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.779+0000 7f3adecf0700 1 -- 172.21.15.28:0/1581658776 >> 172.21.15.28:0/1581658776 conn(0x7f3ad80cd660 msgr2=0x7f3ad80b46b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:40.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.779+0000 7f3adecf0700 1 -- 172.21.15.28:0/1581658776 shutdown_connections 2023-11-28T11:08:40.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.779+0000 7f3adecf0700 1 -- 172.21.15.28:0/1581658776 wait complete. 2023-11-28T11:08:40.797 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:08:40.798 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:40.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.930+0000 7f83f2ad9700 1 Processor -- start 2023-11-28T11:08:40.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.930+0000 7f83f2ad9700 1 -- start start 2023-11-28T11:08:40.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.931+0000 7f83f2ad9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83ec0b62d0 0x7f83ec0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.931+0000 7f83f2ad9700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ec0cd2a0 con 0x7f83ec06f8b0 2023-11-28T11:08:40.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.931+0000 7f83f2ad9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83ec0d0400 con 0x7f83ec0b2d90 2023-11-28T11:08:40.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.931+0000 7f83f2ad9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83ec0d06e0 con 0x7f83ec0b62d0 2023-11-28T11:08:40.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.931+0000 7f83ebfff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f83ec06f8b0 0x7f83ec0b2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34290/0 (socket says 172.21.15.28:34290) 2023-11-28T11:08:40.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.931+0000 7f83ebfff700 1 -- 172.21.15.28:0/773587180 learned_addr learned my addr 172.21.15.28:0/773587180 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:40.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.932+0000 7f83f0875700 1 --2- 172.21.15.28:0/773587180 >> v2:172.21.15.59:3300/0 conn(0x7f83ec0b62d0 0x7f83ec0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.932+0000 7f83eb7fe700 1 -- 172.21.15.28:0/773587180 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4136570149 0 0) 0x7f83ec0cd2a0 con 0x7f83ec06f8b0 2023-11-28T11:08:40.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.932+0000 7f83eb7fe700 1 -- 172.21.15.28:0/773587180 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83dc0035b0 con 0x7f83ec06f8b0 2023-11-28T11:08:40.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.932+0000 7f83f0875700 1 -- 172.21.15.28:0/773587180 >> v1:172.21.15.59:6789/0 conn(0x7f83ec0b2d90 legacy=0x7f83ec0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.932+0000 7f83f0875700 1 -- 172.21.15.28:0/773587180 >> v1:172.21.15.28:6789/0 conn(0x7f83ec06f8b0 legacy=0x7f83ec0b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.932+0000 7f83f0875700 1 -- 172.21.15.28:0/773587180 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83ec0cffb0 con 0x7f83ec0b62d0 2023-11-28T11:08:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.932+0000 7f83f0875700 1 --2- 172.21.15.28:0/773587180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec0b62d0 0x7f83ec0cb630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009840 tx=0x7f83e0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c841e8b39d34a43 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.932+0000 7f83eb7fe700 1 -- 172.21.15.28:0/773587180 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0011830 con 0x7f83ec0b62d0 2023-11-28T11:08:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.932+0000 7f83eb7fe700 1 -- 172.21.15.28:0/773587180 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e0011990 con 0x7f83ec0b62d0 2023-11-28T11:08:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.932+0000 7f83eb7fe700 1 -- 172.21.15.28:0/773587180 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0011d20 con 0x7f83ec0b62d0 2023-11-28T11:08:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.933+0000 7f83f2ad9700 1 -- 172.21.15.28:0/773587180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec0b62d0 msgr2=0x7f83ec0cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.933+0000 7f83f2ad9700 1 --2- 172.21.15.28:0/773587180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec0b62d0 0x7f83ec0cb630 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009840 tx=0x7f83e0009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.933+0000 7f83f2ad9700 1 -- 172.21.15.28:0/773587180 shutdown_connections 2023-11-28T11:08:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.933+0000 7f83f2ad9700 1 --2- 172.21.15.28:0/773587180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec0b62d0 0x7f83ec0cb630 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.933+0000 7f83f2ad9700 1 -- 172.21.15.28:0/773587180 >> 172.21.15.28:0/773587180 conn(0x7f83ec0cd680 msgr2=0x7f83ec0cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.933+0000 7f83f2ad9700 1 -- 172.21.15.28:0/773587180 shutdown_connections 2023-11-28T11:08:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.933+0000 7f83f2ad9700 1 -- 172.21.15.28:0/773587180 wait complete. 2023-11-28T11:08:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.934+0000 7f83f2ad9700 1 Processor -- start 2023-11-28T11:08:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.934+0000 7f83f2ad9700 1 -- start start 2023-11-28T11:08:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f2ad9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec06f8b0 0x7f83ec154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f2ad9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83ec0b2d90 0x7f83ec158e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f2ad9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83ec0b62d0 0x7f83ec153c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f2ad9700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec159590 con 0x7f83ec0b62d0 2023-11-28T11:08:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f2ad9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec159360 con 0x7f83ec06f8b0 2023-11-28T11:08:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f2ad9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83ec1e45d0 con 0x7f83ec0b2d90 2023-11-28T11:08:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f1076700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83ec0b62d0 0x7f83ec153c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f1076700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83ec0b62d0 0x7f83ec153c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46242/0 (socket says 172.21.15.28:46242) 2023-11-28T11:08:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f1076700 1 -- 172.21.15.28:0/3461358290 learned_addr learned my addr 172.21.15.28:0/3461358290 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83ebfff700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83ec0b2d90 0x7f83ec158e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f0875700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec06f8b0 0x7f83ec154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f1076700 1 -- 172.21.15.28:0/3461358290 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83ec0b2d90 msgr2=0x7f83ec158e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f1076700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83ec0b2d90 0x7f83ec158e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f1076700 1 -- 172.21.15.28:0/3461358290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec06f8b0 msgr2=0x7f83ec154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f1076700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec06f8b0 0x7f83ec154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f1076700 1 -- 172.21.15.28:0/3461358290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e00094a0 con 0x7f83ec0b62d0 2023-11-28T11:08:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83ebfff700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83ec0b2d90 0x7f83ec158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.935+0000 7f83f0875700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec06f8b0 0x7f83ec154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.936+0000 7f83f1076700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83ec0b62d0 0x7f83ec153c40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f83e400cc50 tx=0x7f83e400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.936+0000 7f83e97fa700 1 -- 172.21.15.28:0/3461358290 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e400ad80 con 0x7f83ec0b62d0 2023-11-28T11:08:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.936+0000 7f83e97fa700 1 -- 172.21.15.28:0/3461358290 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e400ce50 con 0x7f83ec0b62d0 2023-11-28T11:08:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.936+0000 7f83f2ad9700 1 -- 172.21.15.28:0/3461358290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83ec0b66a0 con 0x7f83ec0b62d0 2023-11-28T11:08:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.936+0000 7f83e97fa700 1 -- 172.21.15.28:0/3461358290 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4003c40 con 0x7f83ec0b62d0 2023-11-28T11:08:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.936+0000 7f83f2ad9700 1 -- 172.21.15.28:0/3461358290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83ec06fd30 con 0x7f83ec0b62d0 2023-11-28T11:08:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.937+0000 7f83f2ad9700 1 -- 172.21.15.28:0/3461358290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83ec0cce50 con 0x7f83ec0b62d0 2023-11-28T11:08:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.937+0000 7f83e97fa700 1 -- 172.21.15.28:0/3461358290 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f83e4010050 con 0x7f83ec0b62d0 2023-11-28T11:08:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.938+0000 7f83e97fa700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83d00682a0 0x7f83d006a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.938+0000 7f83e97fa700 1 -- 172.21.15.28:0/3461358290 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f83e4085860 con 0x7f83ec0b62d0 2023-11-28T11:08:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.938+0000 7f83f0875700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83d00682a0 0x7f83d006a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.938+0000 7f83f0875700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83d00682a0 0x7f83d006a750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0b4340 tx=0x7f83e0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:40.942+0000 7f83e97fa700 1 -- 172.21.15.28:0/3461358290 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f83e4052460 con 0x7f83ec0b62d0 2023-11-28T11:08:41.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.142+0000 7f83f2ad9700 1 -- 172.21.15.28:0/3461358290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f83ec1e4710 con 0x7f83ec0b62d0 2023-11-28T11:08:41.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.143+0000 7f83e97fa700 1 -- 172.21.15.28:0/3461358290 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v16) v1 ==== 92+0+10 (secure 0 0 0) 0x7f83e4055a80 con 0x7f83ec0b62d0 2023-11-28T11:08:41.144 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:08:41.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.145+0000 7f83f2ad9700 1 -- 172.21.15.28:0/3461358290 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83d00682a0 msgr2=0x7f83d006a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.145+0000 7f83f2ad9700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83d00682a0 0x7f83d006a750 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f83ec0b4340 tx=0x7f83e0008030 comp rx=0 tx=0).stop 2023-11-28T11:08:41.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.145+0000 7f83f2ad9700 1 -- 172.21.15.28:0/3461358290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83ec0b62d0 msgr2=0x7f83ec153c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.145+0000 7f83f2ad9700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83ec0b62d0 0x7f83ec153c40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f83e400cc50 tx=0x7f83e400a340 comp rx=0 tx=0).stop 2023-11-28T11:08:41.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.145+0000 7f83f2ad9700 1 -- 172.21.15.28:0/3461358290 shutdown_connections 2023-11-28T11:08:41.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.145+0000 7f83f2ad9700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83d00682a0 0x7f83d006a750 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.145+0000 7f83f2ad9700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83ec0b62d0 0x7f83ec153c40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.145+0000 7f83f2ad9700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83ec06f8b0 0x7f83ec154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.145+0000 7f83f2ad9700 1 --2- 172.21.15.28:0/3461358290 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83ec0b2d90 0x7f83ec158e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.145+0000 7f83f2ad9700 1 -- 172.21.15.28:0/3461358290 >> 172.21.15.28:0/3461358290 conn(0x7f83ec0cd680 msgr2=0x7f83ec0ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:41.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.145+0000 7f83f2ad9700 1 -- 172.21.15.28:0/3461358290 shutdown_connections 2023-11-28T11:08:41.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.145+0000 7f83f2ad9700 1 -- 172.21.15.28:0/3461358290 wait complete. 2023-11-28T11:08:41.158 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:41.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.285+0000 7fd695640700 1 Processor -- start 2023-11-28T11:08:41.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.286+0000 7fd695640700 1 -- start start 2023-11-28T11:08:41.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.286+0000 7fd695640700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd69015fc00 0x7fd69006eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.286+0000 7fd695640700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69006f940 con 0x7fd69006f3e0 2023-11-28T11:08:41.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.286+0000 7fd695640700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd69006fc20 con 0x7fd6900b2d90 2023-11-28T11:08:41.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.286+0000 7fd695640700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6900cb3f0 con 0x7fd69015fc00 2023-11-28T11:08:41.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.286+0000 7fd68e7fc700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd69006f3e0 0x7fd6900b2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34314/0 (socket says 172.21.15.28:34314) 2023-11-28T11:08:41.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.287+0000 7fd68e7fc700 1 -- 172.21.15.28:0/482758749 learned_addr learned my addr 172.21.15.28:0/482758749 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.287+0000 7fd68effd700 1 --2- 172.21.15.28:0/482758749 >> v2:172.21.15.59:3300/0 conn(0x7fd69015fc00 0x7fd69006eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.287+0000 7fd68dffb700 1 -- 172.21.15.28:0/482758749 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3342204194 0 0) 0x7fd69006f940 con 0x7fd69006f3e0 2023-11-28T11:08:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.287+0000 7fd68dffb700 1 -- 172.21.15.28:0/482758749 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6780035b0 con 0x7fd69006f3e0 2023-11-28T11:08:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.287+0000 7fd68dffb700 1 -- 172.21.15.28:0/482758749 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 529710626 0 0) 0x7fd69006fc20 con 0x7fd6900b2d90 2023-11-28T11:08:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.287+0000 7fd68dffb700 1 -- 172.21.15.28:0/482758749 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd69006f940 con 0x7fd6900b2d90 2023-11-28T11:08:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.287+0000 7fd68effd700 1 -- 172.21.15.28:0/482758749 >> v1:172.21.15.59:6789/0 conn(0x7fd6900b2d90 legacy=0x7fd6900cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.287+0000 7fd68effd700 1 -- 172.21.15.28:0/482758749 >> v1:172.21.15.28:6789/0 conn(0x7fd69006f3e0 legacy=0x7fd6900b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.287+0000 7fd68effd700 1 -- 172.21.15.28:0/482758749 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6900cb6d0 con 0x7fd69015fc00 2023-11-28T11:08:41.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.287+0000 7fd68effd700 1 --2- 172.21.15.28:0/482758749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd69015fc00 0x7fd69006eea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd680009840 tx=0x7fd680009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5a8754fb6b9ca6a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.287+0000 7fd68dffb700 1 -- 172.21.15.28:0/482758749 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd680011830 con 0x7fd69015fc00 2023-11-28T11:08:41.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.287+0000 7fd68dffb700 1 -- 172.21.15.28:0/482758749 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd680011990 con 0x7fd69015fc00 2023-11-28T11:08:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.287+0000 7fd68dffb700 1 -- 172.21.15.28:0/482758749 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd680011d00 con 0x7fd69015fc00 2023-11-28T11:08:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.288+0000 7fd695640700 1 -- 172.21.15.28:0/482758749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd69015fc00 msgr2=0x7fd69006eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.288+0000 7fd695640700 1 --2- 172.21.15.28:0/482758749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd69015fc00 0x7fd69006eea0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd680009840 tx=0x7fd680009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.288+0000 7fd695640700 1 -- 172.21.15.28:0/482758749 shutdown_connections 2023-11-28T11:08:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.288+0000 7fd695640700 1 --2- 172.21.15.28:0/482758749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd69015fc00 0x7fd69006eea0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.288+0000 7fd695640700 1 -- 172.21.15.28:0/482758749 >> 172.21.15.28:0/482758749 conn(0x7fd6900cd680 msgr2=0x7fd6900cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.288+0000 7fd695640700 1 -- 172.21.15.28:0/482758749 shutdown_connections 2023-11-28T11:08:41.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.288+0000 7fd695640700 1 -- 172.21.15.28:0/482758749 wait complete. 2023-11-28T11:08:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.289+0000 7fd695640700 1 Processor -- start 2023-11-28T11:08:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.289+0000 7fd695640700 1 -- start start 2023-11-28T11:08:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd695640700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd69006f3e0 0x7fd6900b5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd695640700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6900b2d90 0x7fd690159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd695640700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd69015fc00 0x7fd69015a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd695640700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69015a8b0 con 0x7fd6900b2d90 2023-11-28T11:08:41.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd695640700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69015c400 con 0x7fd69015fc00 2023-11-28T11:08:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd695640700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6900c7e00 con 0x7fd69006f3e0 2023-11-28T11:08:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd68e7fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6900b2d90 0x7fd690159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd68e7fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6900b2d90 0x7fd690159e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46262/0 (socket says 172.21.15.28:46262) 2023-11-28T11:08:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd68e7fc700 1 -- 172.21.15.28:0/2749140450 learned_addr learned my addr 172.21.15.28:0/2749140450 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd68f7fe700 1 --2- 172.21.15.28:0/2749140450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd69015fc00 0x7fd69015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd68effd700 1 --2- 172.21.15.28:0/2749140450 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd69006f3e0 0x7fd6900b5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd68e7fc700 1 -- 172.21.15.28:0/2749140450 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd69006f3e0 msgr2=0x7fd6900b5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd68e7fc700 1 --2- 172.21.15.28:0/2749140450 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd69006f3e0 0x7fd6900b5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd68e7fc700 1 -- 172.21.15.28:0/2749140450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd69015fc00 msgr2=0x7fd69015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd68e7fc700 1 --2- 172.21.15.28:0/2749140450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd69015fc00 0x7fd69015a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.290+0000 7fd68e7fc700 1 -- 172.21.15.28:0/2749140450 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6800094a0 con 0x7fd6900b2d90 2023-11-28T11:08:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.291+0000 7fd68e7fc700 1 --2- 172.21.15.28:0/2749140450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6900b2d90 0x7fd690159e30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd68400c700 tx=0x7fd68400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.291+0000 7fd6777fe700 1 -- 172.21.15.28:0/2749140450 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684005d70 con 0x7fd6900b2d90 2023-11-28T11:08:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.291+0000 7fd6777fe700 1 -- 172.21.15.28:0/2749140450 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6840070d0 con 0x7fd6900b2d90 2023-11-28T11:08:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.291+0000 7fd695640700 1 -- 172.21.15.28:0/2749140450 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6900ce340 con 0x7fd6900b2d90 2023-11-28T11:08:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.291+0000 7fd6777fe700 1 -- 172.21.15.28:0/2749140450 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd684007440 con 0x7fd6900b2d90 2023-11-28T11:08:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.291+0000 7fd695640700 1 -- 172.21.15.28:0/2749140450 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd69015ffd0 con 0x7fd6900b2d90 2023-11-28T11:08:41.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.292+0000 7fd695640700 1 -- 172.21.15.28:0/2749140450 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6900cc300 con 0x7fd6900b2d90 2023-11-28T11:08:41.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.292+0000 7fd6777fe700 1 -- 172.21.15.28:0/2749140450 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7fd684019030 con 0x7fd6900b2d90 2023-11-28T11:08:41.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.292+0000 7fd6777fe700 1 --2- 172.21.15.28:0/2749140450 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd67c0682a0 0x7fd67c06a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.293+0000 7fd68effd700 1 --2- 172.21.15.28:0/2749140450 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd67c0682a0 0x7fd67c06a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.293+0000 7fd6777fe700 1 -- 172.21.15.28:0/2749140450 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd684084a30 con 0x7fd6900b2d90 2023-11-28T11:08:41.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.293+0000 7fd68effd700 1 --2- 172.21.15.28:0/2749140450 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd67c0682a0 0x7fd67c06a750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd6800090b0 tx=0x7fd68001c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.297+0000 7fd6777fe700 1 -- 172.21.15.28:0/2749140450 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd68404b5e0 con 0x7fd6900b2d90 2023-11-28T11:08:41.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.494+0000 7fd695640700 1 -- 172.21.15.28:0/2749140450 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6900739f0 con 0x7fd6900b2d90 2023-11-28T11:08:41.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.494+0000 7fd6777fe700 1 -- 172.21.15.28:0/2749140450 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd684017070 con 0x7fd6900b2d90 2023-11-28T11:08:41.496 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:08:41.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.496+0000 7fd695640700 1 -- 172.21.15.28:0/2749140450 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd67c0682a0 msgr2=0x7fd67c06a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.496+0000 7fd695640700 1 --2- 172.21.15.28:0/2749140450 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd67c0682a0 0x7fd67c06a750 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fd6800090b0 tx=0x7fd68001c040 comp rx=0 tx=0).stop 2023-11-28T11:08:41.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.496+0000 7fd695640700 1 -- 172.21.15.28:0/2749140450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6900b2d90 msgr2=0x7fd690159e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.496+0000 7fd695640700 1 --2- 172.21.15.28:0/2749140450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6900b2d90 0x7fd690159e30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd68400c700 tx=0x7fd68400ca10 comp rx=0 tx=0).stop 2023-11-28T11:08:41.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.496+0000 7fd695640700 1 -- 172.21.15.28:0/2749140450 shutdown_connections 2023-11-28T11:08:41.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.496+0000 7fd695640700 1 --2- 172.21.15.28:0/2749140450 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd67c0682a0 0x7fd67c06a750 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.496+0000 7fd695640700 1 --2- 172.21.15.28:0/2749140450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd69015fc00 0x7fd69015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.496+0000 7fd695640700 1 --2- 172.21.15.28:0/2749140450 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd69006f3e0 0x7fd6900b5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.496+0000 7fd695640700 1 --2- 172.21.15.28:0/2749140450 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6900b2d90 0x7fd690159e30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.496+0000 7fd695640700 1 -- 172.21.15.28:0/2749140450 >> 172.21.15.28:0/2749140450 conn(0x7fd6900cd680 msgr2=0x7fd6900b2120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:41.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.496+0000 7fd695640700 1 -- 172.21.15.28:0/2749140450 shutdown_connections 2023-11-28T11:08:41.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.496+0000 7fd695640700 1 -- 172.21.15.28:0/2749140450 wait complete. 2023-11-28T11:08:41.510 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:08:41.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.634+0000 7faa7dd12700 1 Processor -- start 2023-11-28T11:08:41.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.634+0000 7faa7dd12700 1 -- start start 2023-11-28T11:08:41.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.635+0000 7faa7dd12700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7faa780cb0a0 0x7faa780ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.635+0000 7faa7dd12700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa7806c140 con 0x7faa7806bb80 2023-11-28T11:08:41.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.635+0000 7faa7dd12700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa780c9ab0 con 0x7faa780cb470 2023-11-28T11:08:41.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.635+0000 7faa7dd12700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa780caad0 con 0x7faa780cb0a0 2023-11-28T11:08:41.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.635+0000 7faa77fff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7faa7806bb80 0x7faa7814fea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34338/0 (socket says 172.21.15.28:34338) 2023-11-28T11:08:41.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.635+0000 7faa77fff700 1 -- 172.21.15.28:0/3540266537 learned_addr learned my addr 172.21.15.28:0/3540266537 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:41.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.635+0000 7faa777fe700 1 --2- 172.21.15.28:0/3540266537 >> v2:172.21.15.59:3300/0 conn(0x7faa780cb0a0 0x7faa780ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.636+0000 7faa767fc700 1 -- 172.21.15.28:0/3540266537 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 206065687 0 0) 0x7faa7806c140 con 0x7faa7806bb80 2023-11-28T11:08:41.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.636+0000 7faa767fc700 1 -- 172.21.15.28:0/3540266537 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa640035b0 con 0x7faa7806bb80 2023-11-28T11:08:41.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.636+0000 7faa777fe700 1 -- 172.21.15.28:0/3540266537 >> v1:172.21.15.59:6789/0 conn(0x7faa780cb470 legacy=0x7faa7815a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.636+0000 7faa777fe700 1 -- 172.21.15.28:0/3540266537 >> v1:172.21.15.28:6789/0 conn(0x7faa7806bb80 legacy=0x7faa7814fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.636+0000 7faa777fe700 1 -- 172.21.15.28:0/3540266537 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa780c9d90 con 0x7faa780cb0a0 2023-11-28T11:08:41.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.636+0000 7faa777fe700 1 --2- 172.21.15.28:0/3540266537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa780cb0a0 0x7faa780ca590 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faa68009840 tx=0x7faa68009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f31988bdd65e8c6b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.636+0000 7faa767fc700 1 -- 172.21.15.28:0/3540266537 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa68011830 con 0x7faa780cb0a0 2023-11-28T11:08:41.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.636+0000 7faa767fc700 1 -- 172.21.15.28:0/3540266537 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa68011990 con 0x7faa780cb0a0 2023-11-28T11:08:41.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.636+0000 7faa767fc700 1 -- 172.21.15.28:0/3540266537 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa68011d00 con 0x7faa780cb0a0 2023-11-28T11:08:41.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.637+0000 7faa7dd12700 1 -- 172.21.15.28:0/3540266537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa780cb0a0 msgr2=0x7faa780ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.637+0000 7faa7dd12700 1 --2- 172.21.15.28:0/3540266537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa780cb0a0 0x7faa780ca590 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faa68009840 tx=0x7faa68009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:41.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.637+0000 7faa7dd12700 1 -- 172.21.15.28:0/3540266537 shutdown_connections 2023-11-28T11:08:41.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.637+0000 7faa7dd12700 1 --2- 172.21.15.28:0/3540266537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa780cb0a0 0x7faa780ca590 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.637+0000 7faa7dd12700 1 -- 172.21.15.28:0/3540266537 >> 172.21.15.28:0/3540266537 conn(0x7faa780b0dd0 msgr2=0x7faa780b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:41.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.637+0000 7faa7dd12700 1 -- 172.21.15.28:0/3540266537 shutdown_connections 2023-11-28T11:08:41.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.637+0000 7faa7dd12700 1 -- 172.21.15.28:0/3540266537 wait complete. 2023-11-28T11:08:41.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.638+0000 7faa7dd12700 1 Processor -- start 2023-11-28T11:08:41.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.638+0000 7faa7dd12700 1 -- start start 2023-11-28T11:08:41.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa7dd12700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa7806bb80 0x7faa780c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa7dd12700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa780cb0a0 0x7faa7814eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa7dd12700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa780cb470 0x7faa780c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa7dd12700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa780ce2e0 con 0x7faa780cb470 2023-11-28T11:08:41.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa7dd12700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa7814f530 con 0x7faa7806bb80 2023-11-28T11:08:41.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa7dd12700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa780cd940 con 0x7faa780cb0a0 2023-11-28T11:08:41.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa77fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa780cb470 0x7faa780c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa777fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa7806bb80 0x7faa780c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa777fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa7806bb80 0x7faa780c63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:33572/0 (socket says 172.21.15.28:33572) 2023-11-28T11:08:41.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa777fe700 1 -- 172.21.15.28:0/1751206587 learned_addr learned my addr 172.21.15.28:0/1751206587 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:41.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa77fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa780cb470 0x7faa780c8070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46286/0 (socket says 172.21.15.28:46286) 2023-11-28T11:08:41.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa76ffd700 1 --2- 172.21.15.28:0/1751206587 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa780cb0a0 0x7faa7814eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa777fe700 1 -- 172.21.15.28:0/1751206587 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa780cb0a0 msgr2=0x7faa7814eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa777fe700 1 --2- 172.21.15.28:0/1751206587 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa780cb0a0 0x7faa7814eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa777fe700 1 -- 172.21.15.28:0/1751206587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa780cb470 msgr2=0x7faa780c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa777fe700 1 --2- 172.21.15.28:0/1751206587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa780cb470 0x7faa780c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa777fe700 1 -- 172.21.15.28:0/1751206587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa680094a0 con 0x7faa7806bb80 2023-11-28T11:08:41.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.639+0000 7faa77fff700 1 --2- 172.21.15.28:0/1751206587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa780cb470 0x7faa780c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:41.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.640+0000 7faa777fe700 1 --2- 172.21.15.28:0/1751206587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa7806bb80 0x7faa780c63b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faa68009fd0 tx=0x7faa68004750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.640+0000 7faa6ffff700 1 -- 172.21.15.28:0/1751206587 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa68004550 con 0x7faa7806bb80 2023-11-28T11:08:41.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.640+0000 7faa6ffff700 1 -- 172.21.15.28:0/1751206587 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa6801f070 con 0x7faa7806bb80 2023-11-28T11:08:41.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.640+0000 7faa6ffff700 1 -- 172.21.15.28:0/1751206587 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa680038c0 con 0x7faa7806bb80 2023-11-28T11:08:41.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.640+0000 7faa7dd12700 1 -- 172.21.15.28:0/1751206587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa781505b0 con 0x7faa7806bb80 2023-11-28T11:08:41.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.640+0000 7faa7dd12700 1 -- 172.21.15.28:0/1751206587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa780ca070 con 0x7faa7806bb80 2023-11-28T11:08:41.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.641+0000 7faa7dd12700 1 -- 172.21.15.28:0/1751206587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa78150740 con 0x7faa7806bb80 2023-11-28T11:08:41.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.641+0000 7faa6ffff700 1 -- 172.21.15.28:0/1751206587 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7faa6800f050 con 0x7faa7806bb80 2023-11-28T11:08:41.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.642+0000 7faa6ffff700 1 --2- 172.21.15.28:0/1751206587 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faa580682a0 0x7faa5806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:41.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.642+0000 7faa6ffff700 1 -- 172.21.15.28:0/1751206587 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7faa68021070 con 0x7faa7806bb80 2023-11-28T11:08:41.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.642+0000 7faa76ffd700 1 --2- 172.21.15.28:0/1751206587 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faa580682a0 0x7faa5806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:41.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.642+0000 7faa76ffd700 1 --2- 172.21.15.28:0/1751206587 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faa580682a0 0x7faa5806a750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faa7814fee0 tx=0x7faa60009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:41.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.646+0000 7faa6ffff700 1 -- 172.21.15.28:0/1751206587 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faa68053d20 con 0x7faa7806bb80 2023-11-28T11:08:41.892 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.891+0000 7faa7dd12700 1 -- 172.21.15.28:0/1751206587 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa780675a0 con 0x7faa7806bb80 2023-11-28T11:08:41.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.891+0000 7faa6ffff700 1 -- 172.21.15.28:0/1751206587 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7faa68015de0 con 0x7faa7806bb80 2023-11-28T11:08:41.893 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:08:41.893 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/3112877467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3112877467},{"type":"v1","addr":"172.21.15.59:6834","nonce":3112877467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3448236489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3448236489},{"type":"v1","addr":"172.21.15.28:6836","nonce":3448236489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4232269940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4232269940},{"type":"v1","addr":"172.21.15.28:6837","nonce":4232269940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4296,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3293053261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3293053261},{"type":"v1","addr":"172.21.15.59:6835","nonce":3293053261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-28T11:08:41.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.893+0000 7faa7dd12700 1 -- 172.21.15.28:0/1751206587 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faa580682a0 msgr2=0x7faa5806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.893+0000 7faa7dd12700 1 --2- 172.21.15.28:0/1751206587 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faa580682a0 0x7faa5806a750 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7faa7814fee0 tx=0x7faa60009040 comp rx=0 tx=0).stop 2023-11-28T11:08:41.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.893+0000 7faa7dd12700 1 -- 172.21.15.28:0/1751206587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa7806bb80 msgr2=0x7faa780c63b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:41.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.893+0000 7faa7dd12700 1 --2- 172.21.15.28:0/1751206587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa7806bb80 0x7faa780c63b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7faa68009fd0 tx=0x7faa68004750 comp rx=0 tx=0).stop 2023-11-28T11:08:41.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.893+0000 7faa7dd12700 1 -- 172.21.15.28:0/1751206587 shutdown_connections 2023-11-28T11:08:41.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.893+0000 7faa7dd12700 1 --2- 172.21.15.28:0/1751206587 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faa580682a0 0x7faa5806a750 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.893+0000 7faa7dd12700 1 --2- 172.21.15.28:0/1751206587 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faa780cb470 0x7faa780c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.893+0000 7faa7dd12700 1 --2- 172.21.15.28:0/1751206587 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faa7806bb80 0x7faa780c63b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.893+0000 7faa7dd12700 1 --2- 172.21.15.28:0/1751206587 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faa780cb0a0 0x7faa7814eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:41.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.893+0000 7faa7dd12700 1 -- 172.21.15.28:0/1751206587 >> 172.21.15.28:0/1751206587 conn(0x7faa780b0dd0 msgr2=0x7faa78071e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:41.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.894+0000 7faa7dd12700 1 -- 172.21.15.28:0/1751206587 shutdown_connections 2023-11-28T11:08:41.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:41.894+0000 7faa7dd12700 1 -- 172.21.15.28:0/1751206587 wait complete. 2023-11-28T11:08:41.896 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 3 2023-11-28T11:08:41.908 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:42.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.030+0000 7f2b3b9b1700 1 Processor -- start 2023-11-28T11:08:42.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.031+0000 7f2b3b9b1700 1 -- start start 2023-11-28T11:08:42.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.031+0000 7f2b3b9b1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2b34161090 0x7f2b340ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.031+0000 7f2b3b9b1700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b340ce340 con 0x7f2b3415c990 2023-11-28T11:08:42.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.031+0000 7f2b3b9b1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b340cebe0 con 0x7f2b340caeb0 2023-11-28T11:08:42.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.031+0000 7f2b3b9b1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b340cf780 con 0x7f2b34161090 2023-11-28T11:08:42.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.032+0000 7f2b3974d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2b3415c990 0x7f2b340cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34348/0 (socket says 172.21.15.28:34348) 2023-11-28T11:08:42.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.032+0000 7f2b3974d700 1 -- 172.21.15.28:0/639768122 learned_addr learned my addr 172.21.15.28:0/639768122 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:42.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.032+0000 7f2b38f4c700 1 --2- 172.21.15.28:0/639768122 >> v2:172.21.15.59:3300/0 conn(0x7f2b34161090 0x7f2b340ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.032+0000 7f2b27fff700 1 -- 172.21.15.28:0/639768122 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2893972378 0 0) 0x7f2b340ce340 con 0x7f2b3415c990 2023-11-28T11:08:42.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.032+0000 7f2b27fff700 1 -- 172.21.15.28:0/639768122 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b200035b0 con 0x7f2b3415c990 2023-11-28T11:08:42.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.032+0000 7f2b27fff700 1 -- 172.21.15.28:0/639768122 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2767096296 0 0) 0x7f2b340cebe0 con 0x7f2b340caeb0 2023-11-28T11:08:42.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.032+0000 7f2b27fff700 1 -- 172.21.15.28:0/639768122 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b340ce340 con 0x7f2b340caeb0 2023-11-28T11:08:42.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.032+0000 7f2b38f4c700 1 -- 172.21.15.28:0/639768122 >> v1:172.21.15.59:6789/0 conn(0x7f2b340caeb0 legacy=0x7f2b340cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.032+0000 7f2b38f4c700 1 -- 172.21.15.28:0/639768122 >> v1:172.21.15.28:6789/0 conn(0x7f2b3415c990 legacy=0x7f2b340cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.032+0000 7f2b38f4c700 1 -- 172.21.15.28:0/639768122 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b340cb3f0 con 0x7f2b34161090 2023-11-28T11:08:42.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.032+0000 7f2b38f4c700 1 --2- 172.21.15.28:0/639768122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b34161090 0x7f2b340ca970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009840 tx=0x7f2b28009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=97922fb01eab3fe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:42.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.032+0000 7f2b27fff700 1 -- 172.21.15.28:0/639768122 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b28011830 con 0x7f2b34161090 2023-11-28T11:08:42.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.032+0000 7f2b27fff700 1 -- 172.21.15.28:0/639768122 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b28011990 con 0x7f2b34161090 2023-11-28T11:08:42.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.032+0000 7f2b27fff700 1 -- 172.21.15.28:0/639768122 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b28011d00 con 0x7f2b34161090 2023-11-28T11:08:42.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.033+0000 7f2b3b9b1700 1 -- 172.21.15.28:0/639768122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b34161090 msgr2=0x7f2b340ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.033+0000 7f2b3b9b1700 1 --2- 172.21.15.28:0/639768122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b34161090 0x7f2b340ca970 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009840 tx=0x7f2b28009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:42.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.033+0000 7f2b3b9b1700 1 -- 172.21.15.28:0/639768122 shutdown_connections 2023-11-28T11:08:42.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.033+0000 7f2b3b9b1700 1 --2- 172.21.15.28:0/639768122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b34161090 0x7f2b340ca970 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.033+0000 7f2b3b9b1700 1 -- 172.21.15.28:0/639768122 >> 172.21.15.28:0/639768122 conn(0x7f2b340b1650 msgr2=0x7f2b340b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:42.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.033+0000 7f2b3b9b1700 1 -- 172.21.15.28:0/639768122 shutdown_connections 2023-11-28T11:08:42.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.033+0000 7f2b3b9b1700 1 -- 172.21.15.28:0/639768122 wait complete. 2023-11-28T11:08:42.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.034+0000 7f2b3b9b1700 1 Processor -- start 2023-11-28T11:08:42.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.034+0000 7f2b3b9b1700 1 -- start start 2023-11-28T11:08:42.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b3b9b1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b340caeb0 0x7f2b341f2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b3b9b1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3415c990 0x7f2b34159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b3b9b1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b34161090 0x7f2b340b4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b3b9b1700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b341f3260 con 0x7f2b3415c990 2023-11-28T11:08:42.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b3b9b1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b340adbb0 con 0x7f2b34161090 2023-11-28T11:08:42.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b3b9b1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b34159ea0 con 0x7f2b340caeb0 2023-11-28T11:08:42.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b38f4c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3415c990 0x7f2b34159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b38f4c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3415c990 0x7f2b34159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46298/0 (socket says 172.21.15.28:46298) 2023-11-28T11:08:42.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b38f4c700 1 -- 172.21.15.28:0/377731032 learned_addr learned my addr 172.21.15.28:0/377731032 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:42.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b3974d700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b340caeb0 0x7f2b341f2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b39f4e700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b34161090 0x7f2b340b4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b38f4c700 1 -- 172.21.15.28:0/377731032 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b340caeb0 msgr2=0x7f2b341f2d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b38f4c700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b340caeb0 0x7f2b341f2d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b38f4c700 1 -- 172.21.15.28:0/377731032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b34161090 msgr2=0x7f2b340b4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b38f4c700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b34161090 0x7f2b340b4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b38f4c700 1 -- 172.21.15.28:0/377731032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b280094a0 con 0x7f2b3415c990 2023-11-28T11:08:42.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b3974d700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b340caeb0 0x7f2b341f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:42.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.035+0000 7f2b39f4e700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b34161090 0x7f2b340b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:42.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.036+0000 7f2b38f4c700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3415c990 0x7f2b34159960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009fd0 tx=0x7f2b28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:42.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.036+0000 7f2b25ffb700 1 -- 172.21.15.28:0/377731032 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b280045d0 con 0x7f2b3415c990 2023-11-28T11:08:42.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.036+0000 7f2b25ffb700 1 -- 172.21.15.28:0/377731032 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b2801f070 con 0x7f2b3415c990 2023-11-28T11:08:42.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.036+0000 7f2b25ffb700 1 -- 172.21.15.28:0/377731032 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b280038c0 con 0x7f2b3415c990 2023-11-28T11:08:42.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.036+0000 7f2b3b9b1700 1 -- 172.21.15.28:0/377731032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b340ce870 con 0x7f2b3415c990 2023-11-28T11:08:42.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.036+0000 7f2b3b9b1700 1 -- 172.21.15.28:0/377731032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b340ce590 con 0x7f2b3415c990 2023-11-28T11:08:42.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.037+0000 7f2b3b9b1700 1 -- 172.21.15.28:0/377731032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b340cca20 con 0x7f2b3415c990 2023-11-28T11:08:42.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.037+0000 7f2b25ffb700 1 -- 172.21.15.28:0/377731032 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 4) v1 ==== 86375+0+0 (secure 0 0 0) 0x7f2b2800f050 con 0x7f2b3415c990 2023-11-28T11:08:42.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.038+0000 7f2b25ffb700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2b180682a0 0x7f2b1806a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.038+0000 7f2b25ffb700 1 -- 172.21.15.28:0/377731032 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f2b28021070 con 0x7f2b3415c990 2023-11-28T11:08:42.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.038+0000 7f2b3974d700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2b180682a0 0x7f2b1806a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.038+0000 7f2b3974d700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2b180682a0 0x7f2b1806a750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b30000f60 tx=0x7f2b30009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:42.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.043+0000 7f2b25ffb700 1 -- 172.21.15.28:0/377731032 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2b28053d00 con 0x7f2b3415c990 2023-11-28T11:08:42.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.097+0000 7f2b25ffb700 1 -- 172.21.15.28:0/377731032 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2b280540d0 con 0x7f2b3415c990 2023-11-28T11:08:42.239 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.238+0000 7f2b3b9b1700 1 -- 172.21.15.28:0/377731032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b340cdc50 con 0x7f2b3415c990 2023-11-28T11:08:42.240 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.239+0000 7f2b25ffb700 1 -- 172.21.15.28:0/377731032 <== mon.0 v2:172.21.15.28:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+14159 (secure 0 0 0) 0x7f2b2806eed0 con 0x7f2b3415c990 2023-11-28T11:08:42.240 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:08:42.240 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:41.384627+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:42.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.240+0000 7f2b3b9b1700 1 -- 172.21.15.28:0/377731032 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2b180682a0 msgr2=0x7f2b1806a750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.240+0000 7f2b3b9b1700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2b180682a0 0x7f2b1806a750 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f2b30000f60 tx=0x7f2b30009040 comp rx=0 tx=0).stop 2023-11-28T11:08:42.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.240+0000 7f2b3b9b1700 1 -- 172.21.15.28:0/377731032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3415c990 msgr2=0x7f2b34159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.240+0000 7f2b3b9b1700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3415c990 0x7f2b34159960 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2b28009fd0 tx=0x7f2b28004750 comp rx=0 tx=0).stop 2023-11-28T11:08:42.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.241+0000 7f2b3b9b1700 1 -- 172.21.15.28:0/377731032 shutdown_connections 2023-11-28T11:08:42.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.241+0000 7f2b3b9b1700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2b180682a0 0x7f2b1806a750 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.241+0000 7f2b3b9b1700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b34161090 0x7f2b340b4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.241+0000 7f2b3b9b1700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b340caeb0 0x7f2b341f2d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.241+0000 7f2b3b9b1700 1 --2- 172.21.15.28:0/377731032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b3415c990 0x7f2b34159960 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.241+0000 7f2b3b9b1700 1 -- 172.21.15.28:0/377731032 >> 172.21.15.28:0/377731032 conn(0x7f2b340b1650 msgr2=0x7f2b340af0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:42.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.241+0000 7f2b3b9b1700 1 -- 172.21.15.28:0/377731032 shutdown_connections 2023-11-28T11:08:42.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.241+0000 7f2b3b9b1700 1 -- 172.21.15.28:0/377731032 wait complete. 2023-11-28T11:08:42.254 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:08:42.255 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:42.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.381+0000 7fd9a26d8700 1 Processor -- start 2023-11-28T11:08:42.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.381+0000 7fd9a26d8700 1 -- start start 2023-11-28T11:08:42.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.381+0000 7fd9a26d8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd99c0b1630 0x7fd99c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.381+0000 7fd9a26d8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd99c0b1a00 0x7fd99c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.381+0000 7fd9a26d8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd99c0ce550 con 0x7fd99c0cb3e0 2023-11-28T11:08:42.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.381+0000 7fd9a26d8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd99c0ce830 con 0x7fd99c0b1a00 2023-11-28T11:08:42.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.381+0000 7fd9a26d8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c0ceb10 con 0x7fd99c0b1630 2023-11-28T11:08:42.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.382+0000 7fd99b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd99c0b1a00 0x7fd99c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.382+0000 7fd9a0c75700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd99c0cb3e0 0x7fd99c160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:45152/0 (socket says 172.21.15.28:45152) 2023-11-28T11:08:42.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.382+0000 7fd9a0c75700 1 -- 172.21.15.28:0/942590314 learned_addr learned my addr 172.21.15.28:0/942590314 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:42.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.382+0000 7fd99b7fe700 1 --2- 172.21.15.28:0/942590314 >> v2:172.21.15.59:3300/0 conn(0x7fd99c0b1a00 0x7fd99c0caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:33594/0 (socket says 172.21.15.28:33594) 2023-11-28T11:08:42.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.382+0000 7fd99bfff700 1 --2- 172.21.15.28:0/942590314 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd99c0b1630 0x7fd99c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.382+0000 7fd99affd700 1 -- 172.21.15.28:0/942590314 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1643378897 0 0) 0x7fd99c0ce550 con 0x7fd99c0cb3e0 2023-11-28T11:08:42.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.382+0000 7fd99affd700 1 -- 172.21.15.28:0/942590314 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9840035b0 con 0x7fd99c0cb3e0 2023-11-28T11:08:42.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.382+0000 7fd99bfff700 1 -- 172.21.15.28:0/942590314 >> v2:172.21.15.59:3300/0 conn(0x7fd99c0b1a00 msgr2=0x7fd99c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:42.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.382+0000 7fd99bfff700 1 --2- 172.21.15.28:0/942590314 >> v2:172.21.15.59:3300/0 conn(0x7fd99c0b1a00 0x7fd99c0caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.382+0000 7fd99bfff700 1 -- 172.21.15.28:0/942590314 >> v1:172.21.15.59:6789/0 conn(0x7fd99c0cb3e0 legacy=0x7fd99c160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.382+0000 7fd99bfff700 1 -- 172.21.15.28:0/942590314 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd99c0cedf0 con 0x7fd99c0b1630 2023-11-28T11:08:42.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.382+0000 7fd99bfff700 1 --2- 172.21.15.28:0/942590314 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd99c0b1630 0x7fd99c0cf5f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009840 tx=0x7fd98c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f8b32b6993218f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:42.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.383+0000 7fd99affd700 1 -- 172.21.15.28:0/942590314 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c011830 con 0x7fd99c0b1630 2023-11-28T11:08:42.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.383+0000 7fd99affd700 1 -- 172.21.15.28:0/942590314 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c011990 con 0x7fd99c0b1630 2023-11-28T11:08:42.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.383+0000 7fd99affd700 1 -- 172.21.15.28:0/942590314 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c011d00 con 0x7fd99c0b1630 2023-11-28T11:08:42.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.383+0000 7fd9a26d8700 1 -- 172.21.15.28:0/942590314 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd99c0b1630 msgr2=0x7fd99c0cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.383+0000 7fd9a26d8700 1 --2- 172.21.15.28:0/942590314 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd99c0b1630 0x7fd99c0cf5f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009840 tx=0x7fd98c009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:42.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.383+0000 7fd9a26d8700 1 -- 172.21.15.28:0/942590314 shutdown_connections 2023-11-28T11:08:42.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.383+0000 7fd9a26d8700 1 --2- 172.21.15.28:0/942590314 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd99c0b1630 0x7fd99c0cf5f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.383+0000 7fd9a26d8700 1 --2- 172.21.15.28:0/942590314 >> v2:172.21.15.59:3300/0 conn(0x7fd99c0b1a00 0x7fd99c0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.383+0000 7fd9a26d8700 1 -- 172.21.15.28:0/942590314 >> 172.21.15.28:0/942590314 conn(0x7fd99c0cc630 msgr2=0x7fd99c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:42.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.383+0000 7fd9a26d8700 1 -- 172.21.15.28:0/942590314 shutdown_connections 2023-11-28T11:08:42.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.383+0000 7fd9a26d8700 1 -- 172.21.15.28:0/942590314 wait complete. 2023-11-28T11:08:42.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.384+0000 7fd9a26d8700 1 Processor -- start 2023-11-28T11:08:42.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.385+0000 7fd9a26d8700 1 -- start start 2023-11-28T11:08:42.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.385+0000 7fd9a26d8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd99c0b1630 0x7fd99c14e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.385+0000 7fd9a26d8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd99c0b1a00 0x7fd99c0c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.385+0000 7fd9a26d8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd99c0cb3e0 0x7fd99c0c36e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.385+0000 7fd9a26d8700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c14e600 con 0x7fd99c0b1630 2023-11-28T11:08:42.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.385+0000 7fd9a26d8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c0c4c40 con 0x7fd99c0b1a00 2023-11-28T11:08:42.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.385+0000 7fd9a26d8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd99c0c4030 con 0x7fd99c0cb3e0 2023-11-28T11:08:42.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.385+0000 7fd99bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd99c0b1630 0x7fd99c14e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.385+0000 7fd99bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd99c0b1630 0x7fd99c14e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46310/0 (socket says 172.21.15.28:46310) 2023-11-28T11:08:42.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.385+0000 7fd99bfff700 1 -- 172.21.15.28:0/2294547405 learned_addr learned my addr 172.21.15.28:0/2294547405 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:42.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.385+0000 7fd99b7fe700 1 --2- 172.21.15.28:0/2294547405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd99c0b1a00 0x7fd99c0c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.385+0000 7fd9a0c75700 1 --2- 172.21.15.28:0/2294547405 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd99c0cb3e0 0x7fd99c0c36e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.386+0000 7fd99bfff700 1 -- 172.21.15.28:0/2294547405 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd99c0cb3e0 msgr2=0x7fd99c0c36e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.386+0000 7fd99bfff700 1 --2- 172.21.15.28:0/2294547405 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd99c0cb3e0 0x7fd99c0c36e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.386+0000 7fd99bfff700 1 -- 172.21.15.28:0/2294547405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd99c0b1a00 msgr2=0x7fd99c0c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.386+0000 7fd99bfff700 1 --2- 172.21.15.28:0/2294547405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd99c0b1a00 0x7fd99c0c50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.386+0000 7fd99bfff700 1 -- 172.21.15.28:0/2294547405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd98c0094a0 con 0x7fd99c0b1630 2023-11-28T11:08:42.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.386+0000 7fd99bfff700 1 --2- 172.21.15.28:0/2294547405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd99c0b1630 0x7fd99c14e0c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009fd0 tx=0x7fd98c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:42.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.386+0000 7fd998ff9700 1 -- 172.21.15.28:0/2294547405 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c004550 con 0x7fd99c0b1630 2023-11-28T11:08:42.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.386+0000 7fd998ff9700 1 -- 172.21.15.28:0/2294547405 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd98c01e070 con 0x7fd99c0b1630 2023-11-28T11:08:42.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.386+0000 7fd998ff9700 1 -- 172.21.15.28:0/2294547405 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0038e0 con 0x7fd99c0b1630 2023-11-28T11:08:42.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.386+0000 7fd9a26d8700 1 -- 172.21.15.28:0/2294547405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd99c0b1e70 con 0x7fd99c0b1630 2023-11-28T11:08:42.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.386+0000 7fd9a26d8700 1 -- 172.21.15.28:0/2294547405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd99c0cf0d0 con 0x7fd99c0b1630 2023-11-28T11:08:42.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.387+0000 7fd9a26d8700 1 -- 172.21.15.28:0/2294547405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd99c0ce140 con 0x7fd99c0b1630 2023-11-28T11:08:42.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.387+0000 7fd998ff9700 1 -- 172.21.15.28:0/2294547405 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd98c00f050 con 0x7fd99c0b1630 2023-11-28T11:08:42.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.388+0000 7fd998ff9700 1 --2- 172.21.15.28:0/2294547405 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd9880682f0 0x7fd98806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.388+0000 7fd998ff9700 1 -- 172.21.15.28:0/2294547405 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7fd98c086750 con 0x7fd99c0b1630 2023-11-28T11:08:42.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.388+0000 7fd99b7fe700 1 --2- 172.21.15.28:0/2294547405 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd9880682f0 0x7fd98806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.388+0000 7fd99b7fe700 1 --2- 172.21.15.28:0/2294547405 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd9880682f0 0x7fd98806a7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0ad0a0 tx=0x7fd990009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:42.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.392+0000 7fd998ff9700 1 -- 172.21.15.28:0/2294547405 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd98c053300 con 0x7fd99c0b1630 2023-11-28T11:08:42.589 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.587+0000 7fd9a26d8700 1 -- 172.21.15.28:0/2294547405 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd99c0c4f20 con 0x7fd99c0b1630 2023-11-28T11:08:42.589 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.588+0000 7fd998ff9700 1 -- 172.21.15.28:0/2294547405 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd98c0200a0 con 0x7fd99c0b1630 2023-11-28T11:08:42.589 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:08:42.591 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.590+0000 7fd9a26d8700 1 -- 172.21.15.28:0/2294547405 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd9880682f0 msgr2=0x7fd98806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.590+0000 7fd9a26d8700 1 --2- 172.21.15.28:0/2294547405 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd9880682f0 0x7fd98806a7a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd99c0ad0a0 tx=0x7fd990009210 comp rx=0 tx=0).stop 2023-11-28T11:08:42.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.590+0000 7fd9a26d8700 1 -- 172.21.15.28:0/2294547405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd99c0b1630 msgr2=0x7fd99c14e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.590+0000 7fd9a26d8700 1 --2- 172.21.15.28:0/2294547405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd99c0b1630 0x7fd99c14e0c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009fd0 tx=0x7fd98c004750 comp rx=0 tx=0).stop 2023-11-28T11:08:42.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.590+0000 7fd9a26d8700 1 -- 172.21.15.28:0/2294547405 shutdown_connections 2023-11-28T11:08:42.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.590+0000 7fd9a26d8700 1 --2- 172.21.15.28:0/2294547405 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd9880682f0 0x7fd98806a7a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.590+0000 7fd9a26d8700 1 --2- 172.21.15.28:0/2294547405 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd99c0b1630 0x7fd99c14e0c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.590+0000 7fd9a26d8700 1 --2- 172.21.15.28:0/2294547405 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd99c0cb3e0 0x7fd99c0c36e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.590+0000 7fd9a26d8700 1 --2- 172.21.15.28:0/2294547405 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd99c0b1a00 0x7fd99c0c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.590+0000 7fd9a26d8700 1 -- 172.21.15.28:0/2294547405 >> 172.21.15.28:0/2294547405 conn(0x7fd99c0cc630 msgr2=0x7fd99c0b5390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:42.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.590+0000 7fd9a26d8700 1 -- 172.21.15.28:0/2294547405 shutdown_connections 2023-11-28T11:08:42.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.590+0000 7fd9a26d8700 1 -- 172.21.15.28:0/2294547405 wait complete. 2023-11-28T11:08:42.603 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:42.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.729+0000 7f6043e58700 1 Processor -- start 2023-11-28T11:08:42.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.729+0000 7f6043e58700 1 -- start start 2023-11-28T11:08:42.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.730+0000 7f6043e58700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f603c15fa10 0x7f603c0cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.730+0000 7f6043e58700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f603c06ad60 0x7f603c0d0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.730+0000 7f6043e58700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f603c06bb90 con 0x7f603c06a1e0 2023-11-28T11:08:42.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.730+0000 7f6043e58700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f603c0cd920 con 0x7f603c06ad60 2023-11-28T11:08:42.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.730+0000 7f6043e58700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603c0cfa50 con 0x7f603c15fa10 2023-11-28T11:08:42.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.731+0000 7f60423f5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f603c06ad60 0x7f603c0d0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.731+0000 7f6041bf4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f603c15fa10 0x7f603c0cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.731+0000 7f60413f3700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f603c06a1e0 0x7f603c06a650 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:45164/0 (socket says 172.21.15.28:45164) 2023-11-28T11:08:42.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.731+0000 7f60413f3700 1 -- 172.21.15.28:0/296264755 learned_addr learned my addr 172.21.15.28:0/296264755 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:42.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.731+0000 7f6040bf2700 1 -- 172.21.15.28:0/296264755 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3068462266 0 0) 0x7f603c06bb90 con 0x7f603c06a1e0 2023-11-28T11:08:42.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.731+0000 7f6040bf2700 1 -- 172.21.15.28:0/296264755 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60280035b0 con 0x7f603c06a1e0 2023-11-28T11:08:42.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.731+0000 7f6041bf4700 1 -- 172.21.15.28:0/296264755 >> v2:172.21.15.59:3300/0 conn(0x7f603c06ad60 msgr2=0x7f603c0d0fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:42.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.731+0000 7f6041bf4700 1 --2- 172.21.15.28:0/296264755 >> v2:172.21.15.59:3300/0 conn(0x7f603c06ad60 0x7f603c0d0fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.731+0000 7f6041bf4700 1 -- 172.21.15.28:0/296264755 >> v1:172.21.15.59:6789/0 conn(0x7f603c06a1e0 legacy=0x7f603c06a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.731+0000 7f6041bf4700 1 -- 172.21.15.28:0/296264755 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f603c0d0ae0 con 0x7f603c15fa10 2023-11-28T11:08:42.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.731+0000 7f60423f5700 1 --2- 172.21.15.28:0/296264755 >> v2:172.21.15.59:3300/0 conn(0x7f603c06ad60 0x7f603c0d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:42.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.731+0000 7f6041bf4700 1 --2- 172.21.15.28:0/296264755 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f603c15fa10 0x7f603c0cd3e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f602c009840 tx=0x7f602c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b11dddf1fb37c0d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:42.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.731+0000 7f6040bf2700 1 -- 172.21.15.28:0/296264755 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c011830 con 0x7f603c15fa10 2023-11-28T11:08:42.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.731+0000 7f6040bf2700 1 -- 172.21.15.28:0/296264755 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f602c011990 con 0x7f603c15fa10 2023-11-28T11:08:42.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.731+0000 7f6040bf2700 1 -- 172.21.15.28:0/296264755 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f602c011d20 con 0x7f603c15fa10 2023-11-28T11:08:42.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.732+0000 7f6043e58700 1 -- 172.21.15.28:0/296264755 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f603c15fa10 msgr2=0x7f603c0cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.732+0000 7f6043e58700 1 --2- 172.21.15.28:0/296264755 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f603c15fa10 0x7f603c0cd3e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f602c009840 tx=0x7f602c009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:42.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.732+0000 7f6043e58700 1 -- 172.21.15.28:0/296264755 shutdown_connections 2023-11-28T11:08:42.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.732+0000 7f6043e58700 1 --2- 172.21.15.28:0/296264755 >> v2:172.21.15.59:3300/0 conn(0x7f603c06ad60 0x7f603c0d0fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.732+0000 7f6043e58700 1 --2- 172.21.15.28:0/296264755 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f603c15fa10 0x7f603c0cd3e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.732+0000 7f6043e58700 1 -- 172.21.15.28:0/296264755 >> 172.21.15.28:0/296264755 conn(0x7f603c06ce40 msgr2=0x7f603c0b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:42.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.732+0000 7f6043e58700 1 -- 172.21.15.28:0/296264755 shutdown_connections 2023-11-28T11:08:42.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.732+0000 7f6043e58700 1 -- 172.21.15.28:0/296264755 wait complete. 2023-11-28T11:08:42.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.733+0000 7f6043e58700 1 Processor -- start 2023-11-28T11:08:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.733+0000 7f6043e58700 1 -- start start 2023-11-28T11:08:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f6043e58700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f603c06a1e0 0x7f603c158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f6043e58700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f603c06ad60 0x7f603c0cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f6043e58700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f603c15fa10 0x7f603c155c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f6043e58700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c148d10 con 0x7f603c06ad60 2023-11-28T11:08:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f6043e58700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f603c157d70 con 0x7f603c15fa10 2023-11-28T11:08:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f6043e58700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f603c158050 con 0x7f603c06a1e0 2023-11-28T11:08:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f60413f3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f603c06ad60 0x7f603c0cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f60413f3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f603c06ad60 0x7f603c0cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46326/0 (socket says 172.21.15.28:46326) 2023-11-28T11:08:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f60413f3700 1 -- 172.21.15.28:0/2460150691 learned_addr learned my addr 172.21.15.28:0/2460150691 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f60423f5700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f603c15fa10 0x7f603c155c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f6041bf4700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f603c06a1e0 0x7f603c158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f60413f3700 1 -- 172.21.15.28:0/2460150691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f603c06a1e0 msgr2=0x7f603c158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f60413f3700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f603c06a1e0 0x7f603c158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f60413f3700 1 -- 172.21.15.28:0/2460150691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f603c15fa10 msgr2=0x7f603c155c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f60413f3700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f603c15fa10 0x7f603c155c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f60413f3700 1 -- 172.21.15.28:0/2460150691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6034009a70 con 0x7f603c06ad60 2023-11-28T11:08:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f6041bf4700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f603c06a1e0 0x7f603c158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.734+0000 7f60423f5700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f603c15fa10 0x7f603c155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.735+0000 7f60413f3700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f603c06ad60 0x7f603c0cb5f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6038001eb0 tx=0x7f603800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.735+0000 7f60327fc700 1 -- 172.21.15.28:0/2460150691 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038005d70 con 0x7f603c06ad60 2023-11-28T11:08:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.735+0000 7f60327fc700 1 -- 172.21.15.28:0/2460150691 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60380070d0 con 0x7f603c06ad60 2023-11-28T11:08:42.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.735+0000 7f6043e58700 1 -- 172.21.15.28:0/2460150691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602c0094a0 con 0x7f603c06ad60 2023-11-28T11:08:42.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.735+0000 7f60327fc700 1 -- 172.21.15.28:0/2460150691 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6038007460 con 0x7f603c06ad60 2023-11-28T11:08:42.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.735+0000 7f6043e58700 1 -- 172.21.15.28:0/2460150691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f603c0cff80 con 0x7f603c06ad60 2023-11-28T11:08:42.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.736+0000 7f6043e58700 1 -- 172.21.15.28:0/2460150691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f603c0cfca0 con 0x7f603c06ad60 2023-11-28T11:08:42.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.736+0000 7f60327fc700 1 -- 172.21.15.28:0/2460150691 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6038019030 con 0x7f603c06ad60 2023-11-28T11:08:42.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.737+0000 7f60327fc700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60200682f0 0x7f602006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:42.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.737+0000 7f60327fc700 1 -- 172.21.15.28:0/2460150691 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f6038084da0 con 0x7f603c06ad60 2023-11-28T11:08:42.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.737+0000 7f6041bf4700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60200682f0 0x7f602006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:42.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.737+0000 7f6041bf4700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60200682f0 0x7f602006a7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f602c009b90 tx=0x7f602c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:42.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.741+0000 7f60327fc700 1 -- 172.21.15.28:0/2460150691 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f60380518c0 con 0x7f603c06ad60 2023-11-28T11:08:42.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.933+0000 7f6043e58700 1 -- 172.21.15.28:0/2460150691 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f603c15fde0 con 0x7f603c06ad60 2023-11-28T11:08:42.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.933+0000 7f60327fc700 1 -- 172.21.15.28:0/2460150691 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6038017020 con 0x7f603c06ad60 2023-11-28T11:08:42.935 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:08:42.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.935+0000 7f6043e58700 1 -- 172.21.15.28:0/2460150691 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60200682f0 msgr2=0x7f602006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.935+0000 7f6043e58700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60200682f0 0x7f602006a7a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f602c009b90 tx=0x7f602c008030 comp rx=0 tx=0).stop 2023-11-28T11:08:42.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.935+0000 7f6043e58700 1 -- 172.21.15.28:0/2460150691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f603c06ad60 msgr2=0x7f603c0cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:42.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.935+0000 7f6043e58700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f603c06ad60 0x7f603c0cb5f0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6038001eb0 tx=0x7f603800c860 comp rx=0 tx=0).stop 2023-11-28T11:08:42.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.935+0000 7f6043e58700 1 -- 172.21.15.28:0/2460150691 shutdown_connections 2023-11-28T11:08:42.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.935+0000 7f6043e58700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60200682f0 0x7f602006a7a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.935+0000 7f6043e58700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f603c15fa10 0x7f603c155c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.935+0000 7f6043e58700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f603c06a1e0 0x7f603c158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.935+0000 7f6043e58700 1 --2- 172.21.15.28:0/2460150691 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f603c06ad60 0x7f603c0cb5f0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:42.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.935+0000 7f6043e58700 1 -- 172.21.15.28:0/2460150691 >> 172.21.15.28:0/2460150691 conn(0x7f603c06ce40 msgr2=0x7f603c15c690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:42.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.935+0000 7f6043e58700 1 -- 172.21.15.28:0/2460150691 shutdown_connections 2023-11-28T11:08:42.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:42.935+0000 7f6043e58700 1 -- 172.21.15.28:0/2460150691 wait complete. 2023-11-28T11:08:42.949 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:08:43.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.071+0000 7f3480150700 1 Processor -- start 2023-11-28T11:08:43.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.071+0000 7f3480150700 1 -- start start 2023-11-28T11:08:43.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.072+0000 7f3480150700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3478161090 0x7f34780ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.072+0000 7f3480150700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34780ce340 con 0x7f34780caeb0 2023-11-28T11:08:43.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.072+0000 7f3480150700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34780cebe0 con 0x7f347815c990 2023-11-28T11:08:43.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.072+0000 7f3480150700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34780cf780 con 0x7f3478161090 2023-11-28T11:08:43.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.072+0000 7f347d6eb700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3478161090 0x7f34780ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.072+0000 7f347e6ed700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f34780caeb0 0x7f34780cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34394/0 (socket says 172.21.15.28:34394) 2023-11-28T11:08:43.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.072+0000 7f347e6ed700 1 -- 172.21.15.28:0/3393004131 learned_addr learned my addr 172.21.15.28:0/3393004131 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:43.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.073+0000 7f347d6eb700 1 -- 172.21.15.28:0/3393004131 >> v1:172.21.15.59:6789/0 conn(0x7f347815c990 legacy=0x7f34780cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:43.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.073+0000 7f347d6eb700 1 -- 172.21.15.28:0/3393004131 >> v1:172.21.15.28:6789/0 conn(0x7f34780caeb0 legacy=0x7f34780cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.073+0000 7f347d6eb700 1 -- 172.21.15.28:0/3393004131 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34780cb3f0 con 0x7f3478161090 2023-11-28T11:08:43.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.073+0000 7f347ceea700 1 -- 172.21.15.28:0/3393004131 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 323631740 0 0) 0x7f34780ce340 con 0x7f34780caeb0 2023-11-28T11:08:43.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.073+0000 7f347d6eb700 1 --2- 172.21.15.28:0/3393004131 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3478161090 0x7f34780ca970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3474009840 tx=0x7f3474009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ae127fc1c8c3220 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:43.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.073+0000 7f347ceea700 1 -- 172.21.15.28:0/3393004131 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474004230 con 0x7f3478161090 2023-11-28T11:08:43.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.073+0000 7f347ceea700 1 -- 172.21.15.28:0/3393004131 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3474004390 con 0x7f3478161090 2023-11-28T11:08:43.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.073+0000 7f347ceea700 1 -- 172.21.15.28:0/3393004131 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34740036a0 con 0x7f3478161090 2023-11-28T11:08:43.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.073+0000 7f3480150700 1 -- 172.21.15.28:0/3393004131 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3478161090 msgr2=0x7f34780ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.073+0000 7f3480150700 1 --2- 172.21.15.28:0/3393004131 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3478161090 0x7f34780ca970 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3474009840 tx=0x7f3474009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:43.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.073+0000 7f3480150700 1 -- 172.21.15.28:0/3393004131 shutdown_connections 2023-11-28T11:08:43.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.073+0000 7f3480150700 1 --2- 172.21.15.28:0/3393004131 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3478161090 0x7f34780ca970 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.073+0000 7f3480150700 1 -- 172.21.15.28:0/3393004131 >> 172.21.15.28:0/3393004131 conn(0x7f34780b1650 msgr2=0x7f34780b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:43.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.074+0000 7f3480150700 1 -- 172.21.15.28:0/3393004131 shutdown_connections 2023-11-28T11:08:43.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.074+0000 7f3480150700 1 -- 172.21.15.28:0/3393004131 wait complete. 2023-11-28T11:08:43.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.075+0000 7f3480150700 1 Processor -- start 2023-11-28T11:08:43.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.075+0000 7f3480150700 1 -- start start 2023-11-28T11:08:43.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f3480150700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34780caeb0 0x7f34781586e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f3480150700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f347815c990 0x7f34780ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f3480150700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3478161090 0x7f34780abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f3480150700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34780ada30 con 0x7f3478161090 2023-11-28T11:08:43.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f3480150700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34780add10 con 0x7f347815c990 2023-11-28T11:08:43.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f3480150700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34780adff0 con 0x7f34780caeb0 2023-11-28T11:08:43.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f347e6ed700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3478161090 0x7f34780abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f347e6ed700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3478161090 0x7f34780abe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46342/0 (socket says 172.21.15.28:46342) 2023-11-28T11:08:43.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f347e6ed700 1 -- 172.21.15.28:0/744141234 learned_addr learned my addr 172.21.15.28:0/744141234 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:43.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f347d6eb700 1 --2- 172.21.15.28:0/744141234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f347815c990 0x7f34780ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f347deec700 1 --2- 172.21.15.28:0/744141234 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34780caeb0 0x7f34781586e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f347e6ed700 1 -- 172.21.15.28:0/744141234 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34780caeb0 msgr2=0x7f34781586e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f347e6ed700 1 --2- 172.21.15.28:0/744141234 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34780caeb0 0x7f34781586e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f347e6ed700 1 -- 172.21.15.28:0/744141234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f347815c990 msgr2=0x7f34780ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f347e6ed700 1 --2- 172.21.15.28:0/744141234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f347815c990 0x7f34780ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.076+0000 7f347e6ed700 1 -- 172.21.15.28:0/744141234 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34740094a0 con 0x7f3478161090 2023-11-28T11:08:43.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.077+0000 7f347e6ed700 1 --2- 172.21.15.28:0/744141234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3478161090 0x7f34780abe00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f347000cca0 tx=0x7f347000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:43.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.077+0000 7f346e7fc700 1 -- 172.21.15.28:0/744141234 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f347000cea0 con 0x7f3478161090 2023-11-28T11:08:43.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.077+0000 7f346e7fc700 1 -- 172.21.15.28:0/744141234 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f347000ade0 con 0x7f3478161090 2023-11-28T11:08:43.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.077+0000 7f346e7fc700 1 -- 172.21.15.28:0/744141234 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3470007310 con 0x7f3478161090 2023-11-28T11:08:43.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.077+0000 7f3480150700 1 -- 172.21.15.28:0/744141234 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34780ce870 con 0x7f3478161090 2023-11-28T11:08:43.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.077+0000 7f3480150700 1 -- 172.21.15.28:0/744141234 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34780ce590 con 0x7f3478161090 2023-11-28T11:08:43.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.078+0000 7f3480150700 1 -- 172.21.15.28:0/744141234 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34780cca20 con 0x7f3478161090 2023-11-28T11:08:43.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.078+0000 7f346e7fc700 1 -- 172.21.15.28:0/744141234 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3470010070 con 0x7f3478161090 2023-11-28T11:08:43.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.079+0000 7f346e7fc700 1 --2- 172.21.15.28:0/744141234 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f345c0682f0 0x7f345c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.079+0000 7f346e7fc700 1 -- 172.21.15.28:0/744141234 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f34700854f0 con 0x7f3478161090 2023-11-28T11:08:43.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.079+0000 7f347deec700 1 --2- 172.21.15.28:0/744141234 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f345c0682f0 0x7f345c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.079+0000 7f347deec700 1 --2- 172.21.15.28:0/744141234 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f345c0682f0 0x7f345c06a7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3468004f30 tx=0x7f3468009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:43.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.083+0000 7f346e7fc700 1 -- 172.21.15.28:0/744141234 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f34700520a0 con 0x7f3478161090 2023-11-28T11:08:43.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.281+0000 7f3480150700 1 -- 172.21.15.28:0/744141234 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f34780ae5e0 con 0x7f3478161090 2023-11-28T11:08:43.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.281+0000 7f346e7fc700 1 -- 172.21.15.28:0/744141234 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+126 (secure 0 0 0) 0x7f3470018020 con 0x7f3478161090 2023-11-28T11:08:43.283 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:08:43.283 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2023-11-28T11:08:43.283 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:08:43.283 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 1, 2023-11-28T11:08:43.283 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": "rbd" 2023-11-28T11:08:43.283 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:08:43.283 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:08:43.283 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 2, 2023-11-28T11:08:43.283 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": ".mgr" 2023-11-28T11:08:43.284 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:08:43.284 INFO:teuthology.orchestra.run.smithi028.stdout:] 2023-11-28T11:08:43.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.283+0000 7f3480150700 1 -- 172.21.15.28:0/744141234 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f345c0682f0 msgr2=0x7f345c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.283+0000 7f3480150700 1 --2- 172.21.15.28:0/744141234 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f345c0682f0 0x7f345c06a7a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3468004f30 tx=0x7f3468009040 comp rx=0 tx=0).stop 2023-11-28T11:08:43.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.283+0000 7f3480150700 1 -- 172.21.15.28:0/744141234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3478161090 msgr2=0x7f34780abe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.283+0000 7f3480150700 1 --2- 172.21.15.28:0/744141234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3478161090 0x7f34780abe00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f347000cca0 tx=0x7f347000a340 comp rx=0 tx=0).stop 2023-11-28T11:08:43.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.283+0000 7f3480150700 1 -- 172.21.15.28:0/744141234 shutdown_connections 2023-11-28T11:08:43.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.283+0000 7f3480150700 1 --2- 172.21.15.28:0/744141234 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f345c0682f0 0x7f345c06a7a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.283+0000 7f3480150700 1 --2- 172.21.15.28:0/744141234 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3478161090 0x7f34780abe00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.283+0000 7f3480150700 1 --2- 172.21.15.28:0/744141234 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34780caeb0 0x7f34781586e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.283+0000 7f3480150700 1 --2- 172.21.15.28:0/744141234 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f347815c990 0x7f34780ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.283+0000 7f3480150700 1 -- 172.21.15.28:0/744141234 >> 172.21.15.28:0/744141234 conn(0x7f34780b1650 msgr2=0x7f347806ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.283+0000 7f3480150700 1 -- 172.21.15.28:0/744141234 shutdown_connections 2023-11-28T11:08:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.283+0000 7f3480150700 1 -- 172.21.15.28:0/744141234 wait complete. 2023-11-28T11:08:43.297 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:08:43.297 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:08:43.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.424+0000 7f32bc062700 1 Processor -- start 2023-11-28T11:08:43.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.424+0000 7f32bc062700 1 -- start start 2023-11-28T11:08:43.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.425+0000 7f32bc062700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32b40cc830 0x7f32b415caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.425+0000 7f32bc062700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32b415d030 0x7f32b40cdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.425+0000 7f32bc062700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32b415d620 con 0x7f32b40cc460 2023-11-28T11:08:43.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.425+0000 7f32bc062700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32b4160110 con 0x7f32b40cc830 2023-11-28T11:08:43.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.425+0000 7f32bc062700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b415d900 con 0x7f32b415d030 2023-11-28T11:08:43.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.425+0000 7f32b95fd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32b40cc830 0x7f32b415caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.425+0000 7f32b95fd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32b40cc830 0x7f32b415caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46348/0 (socket says 172.21.15.28:46348) 2023-11-28T11:08:43.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.425+0000 7f32b95fd700 1 -- 172.21.15.28:0/996860842 learned_addr learned my addr 172.21.15.28:0/996860842 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:43.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.426+0000 7f32ba5ff700 1 --2- 172.21.15.28:0/996860842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32b415d030 0x7f32b40cdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.426+0000 7f32b95fd700 1 -- 172.21.15.28:0/996860842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32b415d030 msgr2=0x7f32b40cdb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:43.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.426+0000 7f32b95fd700 1 --2- 172.21.15.28:0/996860842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32b415d030 0x7f32b40cdb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.426+0000 7f32b95fd700 1 -- 172.21.15.28:0/996860842 >> v1:172.21.15.59:6789/0 conn(0x7f32b40cc460 legacy=0x7f32b4160af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.426+0000 7f32b95fd700 1 -- 172.21.15.28:0/996860842 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32b41602f0 con 0x7f32b40cc830 2023-11-28T11:08:43.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.426+0000 7f32b95fd700 1 --2- 172.21.15.28:0/996860842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32b40cc830 0x7f32b415caf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f32a80098f0 tx=0x7f32a8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=257548fdd842408 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:43.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.426+0000 7f32b8dfc700 1 -- 172.21.15.28:0/996860842 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a800b8a0 con 0x7f32b40cc830 2023-11-28T11:08:43.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.426+0000 7f32b8dfc700 1 -- 172.21.15.28:0/996860842 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32a800ba00 con 0x7f32b40cc830 2023-11-28T11:08:43.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.426+0000 7f32b8dfc700 1 -- 172.21.15.28:0/996860842 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32a800bd90 con 0x7f32b40cc830 2023-11-28T11:08:43.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.427+0000 7f32bc062700 1 -- 172.21.15.28:0/996860842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32b40cc830 msgr2=0x7f32b415caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.427+0000 7f32bc062700 1 --2- 172.21.15.28:0/996860842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32b40cc830 0x7f32b415caf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f32a80098f0 tx=0x7f32a8009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:43.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.427+0000 7f32bc062700 1 -- 172.21.15.28:0/996860842 shutdown_connections 2023-11-28T11:08:43.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.427+0000 7f32bc062700 1 --2- 172.21.15.28:0/996860842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32b415d030 0x7f32b40cdb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.427+0000 7f32bc062700 1 --2- 172.21.15.28:0/996860842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32b40cc830 0x7f32b415caf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.427+0000 7f32bc062700 1 -- 172.21.15.28:0/996860842 >> 172.21.15.28:0/996860842 conn(0x7f32b406dae0 msgr2=0x7f32b406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:43.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.427+0000 7f32bc062700 1 -- 172.21.15.28:0/996860842 shutdown_connections 2023-11-28T11:08:43.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.427+0000 7f32bc062700 1 -- 172.21.15.28:0/996860842 wait complete. 2023-11-28T11:08:43.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.428+0000 7f32bc062700 1 Processor -- start 2023-11-28T11:08:43.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.428+0000 7f32bc062700 1 -- start start 2023-11-28T11:08:43.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32bc062700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32b40cc460 0x7f32b41f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32bc062700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32b40cc830 0x7f32b406c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32bc062700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32b415d030 0x7f32b406ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32bc062700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b41f6b80 con 0x7f32b415d030 2023-11-28T11:08:43.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32bc062700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32b41f5f70 con 0x7f32b40cc830 2023-11-28T11:08:43.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32bc062700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32b41f6250 con 0x7f32b40cc460 2023-11-28T11:08:43.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32ba5ff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32b415d030 0x7f32b406ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32ba5ff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32b415d030 0x7f32b406ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46356/0 (socket says 172.21.15.28:46356) 2023-11-28T11:08:43.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32ba5ff700 1 -- 172.21.15.28:0/1900953276 learned_addr learned my addr 172.21.15.28:0/1900953276 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:43.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32b95fd700 1 --2- 172.21.15.28:0/1900953276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32b40cc830 0x7f32b406c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32b9dfe700 1 --2- 172.21.15.28:0/1900953276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32b40cc460 0x7f32b41f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32ba5ff700 1 -- 172.21.15.28:0/1900953276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32b40cc460 msgr2=0x7f32b41f5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32ba5ff700 1 --2- 172.21.15.28:0/1900953276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32b40cc460 0x7f32b41f5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32ba5ff700 1 -- 172.21.15.28:0/1900953276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32b40cc830 msgr2=0x7f32b406c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:43.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32ba5ff700 1 --2- 172.21.15.28:0/1900953276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32b40cc830 0x7f32b406c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:43.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32ba5ff700 1 -- 172.21.15.28:0/1900953276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a8009550 con 0x7f32b415d030 2023-11-28T11:08:43.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32b9dfe700 1 --2- 172.21.15.28:0/1900953276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32b40cc460 0x7f32b41f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:43.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32ba5ff700 1 --2- 172.21.15.28:0/1900953276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32b415d030 0x7f32b406ccc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f32ac007e40 tx=0x7f32ac00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:43.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.429+0000 7f32a67fc700 1 -- 172.21.15.28:0/1900953276 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac00ede0 con 0x7f32b415d030 2023-11-28T11:08:43.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.430+0000 7f32a67fc700 1 -- 172.21.15.28:0/1900953276 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ac014420 con 0x7f32b415d030 2023-11-28T11:08:43.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.430+0000 7f32a67fc700 1 -- 172.21.15.28:0/1900953276 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ac0147d0 con 0x7f32b415d030 2023-11-28T11:08:43.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.430+0000 7f32bc062700 1 -- 172.21.15.28:0/1900953276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32b40ccca0 con 0x7f32b415d030 2023-11-28T11:08:43.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.430+0000 7f32bc062700 1 -- 172.21.15.28:0/1900953276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32b41605d0 con 0x7f32b415d030 2023-11-28T11:08:43.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.430+0000 7f32bc062700 1 -- 172.21.15.28:0/1900953276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32b40cb270 con 0x7f32b415d030 2023-11-28T11:08:43.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.431+0000 7f32a67fc700 1 -- 172.21.15.28:0/1900953276 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f32ac012050 con 0x7f32b415d030 2023-11-28T11:08:43.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.431+0000 7f32a67fc700 1 --2- 172.21.15.28:0/1900953276 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32980682f0 0x7f329806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:43.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.431+0000 7f32a67fc700 1 -- 172.21.15.28:0/1900953276 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5210+0+0 (secure 0 0 0) 0x7f32ac085f70 con 0x7f32b415d030 2023-11-28T11:08:43.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.431+0000 7f32b9dfe700 1 --2- 172.21.15.28:0/1900953276 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32980682f0 0x7f329806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:43.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.432+0000 7f32b9dfe700 1 --2- 172.21.15.28:0/1900953276 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32980682f0 0x7f329806a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32b0000f60 tx=0x7f32b0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:43.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.436+0000 7f32a67fc700 1 -- 172.21.15.28:0/1900953276 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f32ac052b20 con 0x7f32b415d030 2023-11-28T11:08:43.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:43.630+0000 7f32bc062700 1 -- 172.21.15.28:0/1900953276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f32b40739f0 con 0x7f32b415d030 2023-11-28T11:08:44.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.491+0000 7f32a67fc700 1 -- 172.21.15.28:0/1900953276 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7f32ac056140 con 0x7f32b415d030 2023-11-28T11:08:44.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.495+0000 7f32bc062700 1 -- 172.21.15.28:0/1900953276 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32980682f0 msgr2=0x7f329806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.495+0000 7f32bc062700 1 --2- 172.21.15.28:0/1900953276 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32980682f0 0x7f329806a7a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32b0000f60 tx=0x7f32b0009040 comp rx=0 tx=0).stop 2023-11-28T11:08:44.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.495+0000 7f32bc062700 1 -- 172.21.15.28:0/1900953276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32b415d030 msgr2=0x7f32b406ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.495+0000 7f32bc062700 1 --2- 172.21.15.28:0/1900953276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32b415d030 0x7f32b406ccc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f32ac007e40 tx=0x7f32ac00e3c0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.496+0000 7f32bc062700 1 -- 172.21.15.28:0/1900953276 shutdown_connections 2023-11-28T11:08:44.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.496+0000 7f32bc062700 1 --2- 172.21.15.28:0/1900953276 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32980682f0 0x7f329806a7a0 secure :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f32b0000f60 tx=0x7f32b0009040 comp rx=0 tx=0).stop 2023-11-28T11:08:44.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.496+0000 7f32bc062700 1 --2- 172.21.15.28:0/1900953276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32b415d030 0x7f32b406ccc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.496+0000 7f32bc062700 1 --2- 172.21.15.28:0/1900953276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32b40cc460 0x7f32b41f5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.496+0000 7f32bc062700 1 --2- 172.21.15.28:0/1900953276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32b40cc830 0x7f32b406c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.496+0000 7f32bc062700 1 -- 172.21.15.28:0/1900953276 >> 172.21.15.28:0/1900953276 conn(0x7f32b406dae0 msgr2=0x7f32b415f8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:44.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.496+0000 7f32bc062700 1 -- 172.21.15.28:0/1900953276 shutdown_connections 2023-11-28T11:08:44.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.496+0000 7f32bc062700 1 -- 172.21.15.28:0/1900953276 wait complete. 2023-11-28T11:08:44.500 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' created 2023-11-28T11:08:44.514 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-11-28T11:08:44.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.651+0000 7f00beb13700 1 Processor -- start 2023-11-28T11:08:44.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.651+0000 7f00beb13700 1 -- start start 2023-11-28T11:08:44.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.651+0000 7f00beb13700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f00b80b62d0 0x7f00b80cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.651+0000 7f00beb13700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b80cd2a0 con 0x7f00b80b2d90 2023-11-28T11:08:44.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.651+0000 7f00beb13700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00b80d0400 con 0x7f00b806f8b0 2023-11-28T11:08:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.651+0000 7f00beb13700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00b80d06e0 con 0x7f00b80b62d0 2023-11-28T11:08:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.652+0000 7f00bc8af700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f00b80b62d0 0x7f00b80cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.652+0000 7f00bd0b0700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f00b80b2d90 0x7f00b80ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34418/0 (socket says 172.21.15.28:34418) 2023-11-28T11:08:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.652+0000 7f00bc8af700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f00b80b62d0 0x7f00b80cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46360/0 (socket says 172.21.15.28:46360) 2023-11-28T11:08:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.652+0000 7f00bd0b0700 1 -- 172.21.15.28:0/3805254355 learned_addr learned my addr 172.21.15.28:0/3805254355 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.652+0000 7f00bc8af700 1 -- 172.21.15.28:0/3805254355 >> v1:172.21.15.59:6789/0 conn(0x7f00b806f8b0 legacy=0x7f00b80b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.652+0000 7f00bc8af700 1 -- 172.21.15.28:0/3805254355 >> v1:172.21.15.28:6789/0 conn(0x7f00b80b2d90 legacy=0x7f00b80ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.652+0000 7f00bc8af700 1 -- 172.21.15.28:0/3805254355 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00b80cffb0 con 0x7f00b80b62d0 2023-11-28T11:08:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.652+0000 7f00b77fe700 1 -- 172.21.15.28:0/3805254355 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468043361 0 0) 0x7f00b80cd2a0 con 0x7f00b80b2d90 2023-11-28T11:08:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.652+0000 7f00bc8af700 1 --2- 172.21.15.28:0/3805254355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b80b62d0 0x7f00b80cb630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009840 tx=0x7f00ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed1542cc6972656 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.652+0000 7f00b77fe700 1 -- 172.21.15.28:0/3805254355 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac011830 con 0x7f00b80b62d0 2023-11-28T11:08:44.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.652+0000 7f00b77fe700 1 -- 172.21.15.28:0/3805254355 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ac011990 con 0x7f00b80b62d0 2023-11-28T11:08:44.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.652+0000 7f00b77fe700 1 -- 172.21.15.28:0/3805254355 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ac011d20 con 0x7f00b80b62d0 2023-11-28T11:08:44.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.653+0000 7f00beb13700 1 -- 172.21.15.28:0/3805254355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b80b62d0 msgr2=0x7f00b80cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.653+0000 7f00beb13700 1 --2- 172.21.15.28:0/3805254355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b80b62d0 0x7f00b80cb630 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009840 tx=0x7f00ac009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:44.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.653+0000 7f00beb13700 1 -- 172.21.15.28:0/3805254355 shutdown_connections 2023-11-28T11:08:44.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.653+0000 7f00beb13700 1 --2- 172.21.15.28:0/3805254355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b80b62d0 0x7f00b80cb630 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.653+0000 7f00beb13700 1 -- 172.21.15.28:0/3805254355 >> 172.21.15.28:0/3805254355 conn(0x7f00b80cd680 msgr2=0x7f00b80cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:44.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.653+0000 7f00beb13700 1 -- 172.21.15.28:0/3805254355 shutdown_connections 2023-11-28T11:08:44.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.653+0000 7f00beb13700 1 -- 172.21.15.28:0/3805254355 wait complete. 2023-11-28T11:08:44.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.654+0000 7f00beb13700 1 Processor -- start 2023-11-28T11:08:44.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.654+0000 7f00beb13700 1 -- start start 2023-11-28T11:08:44.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00beb13700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00b806f8b0 0x7f00b81f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00beb13700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b80b2d90 0x7f00b80c7cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00beb13700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b80b62d0 0x7f00b80c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00beb13700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b80c48d0 con 0x7f00b80b62d0 2023-11-28T11:08:44.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00beb13700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00b80c8210 con 0x7f00b80b2d90 2023-11-28T11:08:44.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00beb13700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00b80c8cf0 con 0x7f00b806f8b0 2023-11-28T11:08:44.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00bd0b0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b80b62d0 0x7f00b80c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00bd0b0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b80b62d0 0x7f00b80c6c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46366/0 (socket says 172.21.15.28:46366) 2023-11-28T11:08:44.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00bd0b0700 1 -- 172.21.15.28:0/880890098 learned_addr learned my addr 172.21.15.28:0/880890098 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:44.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00b7fff700 1 --2- 172.21.15.28:0/880890098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b80b2d90 0x7f00b80c7cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00bc8af700 1 --2- 172.21.15.28:0/880890098 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00b806f8b0 0x7f00b81f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00bd0b0700 1 -- 172.21.15.28:0/880890098 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00b806f8b0 msgr2=0x7f00b81f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00bd0b0700 1 --2- 172.21.15.28:0/880890098 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00b806f8b0 0x7f00b81f3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00bd0b0700 1 -- 172.21.15.28:0/880890098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b80b2d90 msgr2=0x7f00b80c7cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:44.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00bd0b0700 1 --2- 172.21.15.28:0/880890098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b80b2d90 0x7f00b80c7cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:44.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00bd0b0700 1 -- 172.21.15.28:0/880890098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac0094a0 con 0x7f00b80b62d0 2023-11-28T11:08:44.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.655+0000 7f00b7fff700 1 --2- 172.21.15.28:0/880890098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b80b2d90 0x7f00b80c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:44.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.656+0000 7f00bd0b0700 1 --2- 172.21.15.28:0/880890098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b80b62d0 0x7f00b80c6c60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f00b000ccd0 tx=0x7f00b000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.656+0000 7f00b57fa700 1 -- 172.21.15.28:0/880890098 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b000ad40 con 0x7f00b80b62d0 2023-11-28T11:08:44.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.656+0000 7f00b57fa700 1 -- 172.21.15.28:0/880890098 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00b00070d0 con 0x7f00b80b62d0 2023-11-28T11:08:44.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.656+0000 7f00beb13700 1 -- 172.21.15.28:0/880890098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00b80b66a0 con 0x7f00b80b62d0 2023-11-28T11:08:44.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.656+0000 7f00b57fa700 1 -- 172.21.15.28:0/880890098 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00b000aea0 con 0x7f00b80b62d0 2023-11-28T11:08:44.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.656+0000 7f00beb13700 1 -- 172.21.15.28:0/880890098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00b806fd30 con 0x7f00b80b62d0 2023-11-28T11:08:44.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.657+0000 7f00beb13700 1 -- 172.21.15.28:0/880890098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00b80cce50 con 0x7f00b80b62d0 2023-11-28T11:08:44.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.657+0000 7f00b57fa700 1 -- 172.21.15.28:0/880890098 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f00b001a030 con 0x7f00b80b62d0 2023-11-28T11:08:44.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.657+0000 7f00b57fa700 1 --2- 172.21.15.28:0/880890098 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f009c0682f0 0x7f009c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:44.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.658+0000 7f00b57fa700 1 -- 172.21.15.28:0/880890098 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5580+0+0 (secure 0 0 0) 0x7f00b0085830 con 0x7f00b80b62d0 2023-11-28T11:08:44.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.658+0000 7f00bc8af700 1 --2- 172.21.15.28:0/880890098 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f009c0682f0 0x7f009c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:44.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.658+0000 7f00bc8af700 1 --2- 172.21.15.28:0/880890098 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f009c0682f0 0x7f009c06a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009b90 tx=0x7f00ac008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:44.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.662+0000 7f00b57fa700 1 -- 172.21.15.28:0/880890098 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f00b0052270 con 0x7f00b80b62d0 2023-11-28T11:08:44.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:44.858+0000 7f00beb13700 1 -- 172.21.15.28:0/880890098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f00b80c4a10 con 0x7f00b80b62d0 2023-11-28T11:08:45.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.497+0000 7f00b57fa700 1 -- 172.21.15.28:0/880890098 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v19) v1 ==== 171+0+0 (secure 0 0 0) 0x7f00b80c4a10 con 0x7f00b80b62d0 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.505+0000 7f00beb13700 1 -- 172.21.15.28:0/880890098 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f009c0682f0 msgr2=0x7f009c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.505+0000 7f00beb13700 1 --2- 172.21.15.28:0/880890098 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f009c0682f0 0x7f009c06a7a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009b90 tx=0x7f00ac008030 comp rx=0 tx=0).stop 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.505+0000 7f00beb13700 1 -- 172.21.15.28:0/880890098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b80b62d0 msgr2=0x7f00b80c6c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.505+0000 7f00beb13700 1 --2- 172.21.15.28:0/880890098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b80b62d0 0x7f00b80c6c60 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f00b000ccd0 tx=0x7f00b000a340 comp rx=0 tx=0).stop 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.505+0000 7f00beb13700 1 -- 172.21.15.28:0/880890098 shutdown_connections 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.505+0000 7f00beb13700 1 --2- 172.21.15.28:0/880890098 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f009c0682f0 0x7f009c06a7a0 secure :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f00ac009b90 tx=0x7f00ac008030 comp rx=0 tx=0).stop 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.505+0000 7f00beb13700 1 --2- 172.21.15.28:0/880890098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00b80b62d0 0x7f00b80c6c60 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.505+0000 7f00beb13700 1 --2- 172.21.15.28:0/880890098 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00b806f8b0 0x7f00b81f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.505+0000 7f00beb13700 1 --2- 172.21.15.28:0/880890098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00b80b2d90 0x7f00b80c7cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.505+0000 7f00beb13700 1 -- 172.21.15.28:0/880890098 >> 172.21.15.28:0/880890098 conn(0x7f00b80cd680 msgr2=0x7f00b80b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:45.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.506+0000 7f00beb13700 1 -- 172.21.15.28:0/880890098 shutdown_connections 2023-11-28T11:08:45.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.509+0000 7f00beb13700 1 -- 172.21.15.28:0/880890098 wait complete. 2023-11-28T11:08:45.523 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' created 2023-11-28T11:08:45.535 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:08:45.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.668+0000 7f62e75cb700 1 Processor -- start 2023-11-28T11:08:45.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.669+0000 7f62e75cb700 1 -- start start 2023-11-28T11:08:45.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.669+0000 7f62e75cb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f62e00b1d40 0x7f62e00cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.669+0000 7f62e75cb700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f62e015dbf0 0x7f62e00ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.669+0000 7f62e75cb700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e00ceff0 con 0x7f62e0159500 2023-11-28T11:08:45.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.669+0000 7f62e75cb700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e015e090 con 0x7f62e015dbf0 2023-11-28T11:08:45.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.669+0000 7f62e75cb700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e00cede0 con 0x7f62e00b1d40 2023-11-28T11:08:45.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.670+0000 7f62e5b68700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f62e015dbf0 0x7f62e00ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.670+0000 7f62e5b68700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f62e015dbf0 0x7f62e00ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46374/0 (socket says 172.21.15.28:46374) 2023-11-28T11:08:45.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.670+0000 7f62e5b68700 1 -- 172.21.15.28:0/389365500 learned_addr learned my addr 172.21.15.28:0/389365500 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:45.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.670+0000 7f62e4b66700 1 --2- 172.21.15.28:0/389365500 >> v2:172.21.15.59:3300/0 conn(0x7f62e00b1d40 0x7f62e00cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.670+0000 7f62e5b68700 1 -- 172.21.15.28:0/389365500 >> v2:172.21.15.59:3300/0 conn(0x7f62e00b1d40 msgr2=0x7f62e00cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:45.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.670+0000 7f62e5b68700 1 --2- 172.21.15.28:0/389365500 >> v2:172.21.15.59:3300/0 conn(0x7f62e00b1d40 0x7f62e00cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.670+0000 7f62e5b68700 1 -- 172.21.15.28:0/389365500 >> v1:172.21.15.59:6789/0 conn(0x7f62e0159500 legacy=0x7f62e00b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:45.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.670+0000 7f62e5b68700 1 -- 172.21.15.28:0/389365500 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e00cf2d0 con 0x7f62e015dbf0 2023-11-28T11:08:45.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.670+0000 7f62e5b68700 1 --2- 172.21.15.28:0/389365500 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e015dbf0 0x7f62e00ce8a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f62e00d0c50 tx=0x7f62d800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8bd0692a104f0c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.670+0000 7f62d7fff700 1 -- 172.21.15.28:0/389365500 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d8010070 con 0x7f62e015dbf0 2023-11-28T11:08:45.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.670+0000 7f62d7fff700 1 -- 172.21.15.28:0/389365500 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d800dd50 con 0x7f62e015dbf0 2023-11-28T11:08:45.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.670+0000 7f62d7fff700 1 -- 172.21.15.28:0/389365500 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d800b310 con 0x7f62e015dbf0 2023-11-28T11:08:45.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.671+0000 7f62e75cb700 1 -- 172.21.15.28:0/389365500 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e015dbf0 msgr2=0x7f62e00ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.671+0000 7f62e75cb700 1 --2- 172.21.15.28:0/389365500 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e015dbf0 0x7f62e00ce8a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f62e00d0c50 tx=0x7f62d800d430 comp rx=0 tx=0).stop 2023-11-28T11:08:45.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.671+0000 7f62e75cb700 1 -- 172.21.15.28:0/389365500 shutdown_connections 2023-11-28T11:08:45.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.671+0000 7f62e75cb700 1 --2- 172.21.15.28:0/389365500 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e015dbf0 0x7f62e00ce8a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.671+0000 7f62e75cb700 1 --2- 172.21.15.28:0/389365500 >> v2:172.21.15.59:3300/0 conn(0x7f62e00b1d40 0x7f62e00cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.671+0000 7f62e75cb700 1 -- 172.21.15.28:0/389365500 >> 172.21.15.28:0/389365500 conn(0x7f62e00cc630 msgr2=0x7f62e00cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:45.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.671+0000 7f62e75cb700 1 -- 172.21.15.28:0/389365500 shutdown_connections 2023-11-28T11:08:45.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.671+0000 7f62e75cb700 1 -- 172.21.15.28:0/389365500 wait complete. 2023-11-28T11:08:45.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.672+0000 7f62e75cb700 1 Processor -- start 2023-11-28T11:08:45.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.672+0000 7f62e75cb700 1 -- start start 2023-11-28T11:08:45.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.673+0000 7f62e75cb700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e00b1d40 0x7f62e00adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.673+0000 7f62e75cb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62e0159500 0x7f62e00b4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.673+0000 7f62e75cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e015dbf0 0x7f62e00accc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.673+0000 7f62e75cb700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e006bed0 con 0x7f62e00b1d40 2023-11-28T11:08:45.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.673+0000 7f62e75cb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00b58e0 con 0x7f62e015dbf0 2023-11-28T11:08:45.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.673+0000 7f62e75cb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00b5320 con 0x7f62e0159500 2023-11-28T11:08:45.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.673+0000 7f62e5367700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e00b1d40 0x7f62e00adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62e5367700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e00b1d40 0x7f62e00adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46376/0 (socket says 172.21.15.28:46376) 2023-11-28T11:08:45.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62e5367700 1 -- 172.21.15.28:0/2393954186 learned_addr learned my addr 172.21.15.28:0/2393954186 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:45.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62e5b68700 1 --2- 172.21.15.28:0/2393954186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e015dbf0 0x7f62e00accc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62e4b66700 1 --2- 172.21.15.28:0/2393954186 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62e0159500 0x7f62e00b4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62e5367700 1 -- 172.21.15.28:0/2393954186 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62e0159500 msgr2=0x7f62e00b4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62e5367700 1 --2- 172.21.15.28:0/2393954186 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62e0159500 0x7f62e00b4a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62e5367700 1 -- 172.21.15.28:0/2393954186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e015dbf0 msgr2=0x7f62e00accc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62e5367700 1 --2- 172.21.15.28:0/2393954186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e015dbf0 0x7f62e00accc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62e5367700 1 -- 172.21.15.28:0/2393954186 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d800bd40 con 0x7f62e00b1d40 2023-11-28T11:08:45.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62e5b68700 1 --2- 172.21.15.28:0/2393954186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e015dbf0 0x7f62e00accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:45.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62e5367700 1 --2- 172.21.15.28:0/2393954186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e00b1d40 0x7f62e00adfa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c4d0 tx=0x7f62d000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62d5ffb700 1 -- 172.21.15.28:0/2393954186 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0011790 con 0x7f62e00b1d40 2023-11-28T11:08:45.679 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62d5ffb700 1 -- 172.21.15.28:0/2393954186 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d00118f0 con 0x7f62e00b1d40 2023-11-28T11:08:45.679 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62d5ffb700 1 -- 172.21.15.28:0/2393954186 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0011ca0 con 0x7f62e00b1d40 2023-11-28T11:08:45.679 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62e75cb700 1 -- 172.21.15.28:0/2393954186 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e015d3f0 con 0x7f62e00b1d40 2023-11-28T11:08:45.679 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.674+0000 7f62e75cb700 1 -- 172.21.15.28:0/2393954186 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e015d110 con 0x7f62e00b1d40 2023-11-28T11:08:45.679 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.675+0000 7f62e75cb700 1 -- 172.21.15.28:0/2393954186 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e015d6d0 con 0x7f62e00b1d40 2023-11-28T11:08:45.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.676+0000 7f62d5ffb700 1 -- 172.21.15.28:0/2393954186 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f62d0010050 con 0x7f62e00b1d40 2023-11-28T11:08:45.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.676+0000 7f62d5ffb700 1 --2- 172.21.15.28:0/2393954186 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f62c40682f0 0x7f62c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:45.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.676+0000 7f62d5ffb700 1 -- 172.21.15.28:0/2393954186 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5962+0+0 (secure 0 0 0) 0x7f62d0085920 con 0x7f62e00b1d40 2023-11-28T11:08:45.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.676+0000 7f62e4b66700 1 --2- 172.21.15.28:0/2393954186 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f62c40682f0 0x7f62c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:45.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.677+0000 7f62e4b66700 1 --2- 172.21.15.28:0/2393954186 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f62c40682f0 0x7f62c406a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f62e00cc5b0 tx=0x7f62dc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:45.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.681+0000 7f62d5ffb700 1 -- 172.21.15.28:0/2393954186 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f62d00521e0 con 0x7f62e00b1d40 2023-11-28T11:08:45.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.922+0000 7f62e75cb700 1 -- 172.21.15.28:0/2393954186 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f62e00739f0 con 0x7f62e00b1d40 2023-11-28T11:08:45.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.927+0000 7f62d5ffb700 1 -- 172.21.15.28:0/2393954186 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f62d0085020 con 0x7f62e00b1d40 2023-11-28T11:08:45.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.933+0000 7f62e75cb700 1 -- 172.21.15.28:0/2393954186 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f62c40682f0 msgr2=0x7f62c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.933+0000 7f62e75cb700 1 --2- 172.21.15.28:0/2393954186 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f62c40682f0 0x7f62c406a7a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f62e00cc5b0 tx=0x7f62dc008040 comp rx=0 tx=0).stop 2023-11-28T11:08:45.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.933+0000 7f62e75cb700 1 -- 172.21.15.28:0/2393954186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e00b1d40 msgr2=0x7f62e00adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:45.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.933+0000 7f62e75cb700 1 --2- 172.21.15.28:0/2393954186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e00b1d40 0x7f62e00adfa0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c4d0 tx=0x7f62d000c890 comp rx=0 tx=0).stop 2023-11-28T11:08:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.937+0000 7f62e75cb700 1 -- 172.21.15.28:0/2393954186 shutdown_connections 2023-11-28T11:08:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.937+0000 7f62e75cb700 1 --2- 172.21.15.28:0/2393954186 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f62c40682f0 0x7f62c406a7a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.937+0000 7f62e75cb700 1 --2- 172.21.15.28:0/2393954186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62e00b1d40 0x7f62e00adfa0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.937+0000 7f62e75cb700 1 --2- 172.21.15.28:0/2393954186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62e015dbf0 0x7f62e00accc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.937+0000 7f62e75cb700 1 --2- 172.21.15.28:0/2393954186 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f62e0159500 0x7f62e00b4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:45.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.937+0000 7f62e75cb700 1 -- 172.21.15.28:0/2393954186 >> 172.21.15.28:0/2393954186 conn(0x7f62e00cc630 msgr2=0x7f62e0157140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:45.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.938+0000 7f62e75cb700 1 -- 172.21.15.28:0/2393954186 shutdown_connections 2023-11-28T11:08:45.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:45.938+0000 7f62e75cb700 1 -- 172.21.15.28:0/2393954186 wait complete. 2023-11-28T11:08:45.948 INFO:teuthology.orchestra.run.smithi028.stderr:new fs with metadata pool 3 and data pool 4 2023-11-28T11:08:45.964 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-11-28T11:08:45.965 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-11-28T11:08:46.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.092+0000 7f5c86aa6700 1 Processor -- start 2023-11-28T11:08:46.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.093+0000 7f5c86aa6700 1 -- start start 2023-11-28T11:08:46.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.094+0000 7f5c86aa6700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5c80159590 0x7f5c8006e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.094+0000 7f5c86aa6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c800ce890 0x7f5c800ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.094+0000 7f5c86aa6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c8015d120 con 0x7f5c8015db70 2023-11-28T11:08:46.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.094+0000 7f5c86aa6700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c800cf250 con 0x7f5c80159590 2023-11-28T11:08:46.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.094+0000 7f5c86aa6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8015d6e0 con 0x7f5c800ce890 2023-11-28T11:08:46.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.094+0000 7f5c84842700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5c80159590 0x7f5c8006e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.094+0000 7f5c84842700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5c80159590 0x7f5c8006e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46384/0 (socket says 172.21.15.28:46384) 2023-11-28T11:08:46.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.094+0000 7f5c84842700 1 -- 172.21.15.28:0/453206346 learned_addr learned my addr 172.21.15.28:0/453206346 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:46.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.094+0000 7f5c85043700 1 --2- 172.21.15.28:0/453206346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c800ce890 0x7f5c800ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.094+0000 7f5c84842700 1 -- 172.21.15.28:0/453206346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c800ce890 msgr2=0x7f5c800ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:46.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.094+0000 7f5c84842700 1 --2- 172.21.15.28:0/453206346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c800ce890 0x7f5c800ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.094+0000 7f5c84842700 1 -- 172.21.15.28:0/453206346 >> v1:172.21.15.59:6789/0 conn(0x7f5c8015db70 legacy=0x7f5c800ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.094+0000 7f5c84842700 1 -- 172.21.15.28:0/453206346 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c800aef60 con 0x7f5c80159590 2023-11-28T11:08:46.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.094+0000 7f5c84842700 1 --2- 172.21.15.28:0/453206346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c80159590 0x7f5c8006e400 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c740098f0 tx=0x7f5c74009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd678d9ebd6b2307 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.094+0000 7f5c7f7fe700 1 -- 172.21.15.28:0/453206346 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7400b8a0 con 0x7f5c80159590 2023-11-28T11:08:46.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.095+0000 7f5c7f7fe700 1 -- 172.21.15.28:0/453206346 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7400ba00 con 0x7f5c80159590 2023-11-28T11:08:46.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.095+0000 7f5c7f7fe700 1 -- 172.21.15.28:0/453206346 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7400bd90 con 0x7f5c80159590 2023-11-28T11:08:46.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.095+0000 7f5c86aa6700 1 -- 172.21.15.28:0/453206346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c80159590 msgr2=0x7f5c8006e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.095+0000 7f5c86aa6700 1 --2- 172.21.15.28:0/453206346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c80159590 0x7f5c8006e400 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5c740098f0 tx=0x7f5c74009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:46.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.095+0000 7f5c86aa6700 1 -- 172.21.15.28:0/453206346 shutdown_connections 2023-11-28T11:08:46.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.095+0000 7f5c86aa6700 1 --2- 172.21.15.28:0/453206346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c80159590 0x7f5c8006e400 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.095+0000 7f5c86aa6700 1 --2- 172.21.15.28:0/453206346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c800ce890 0x7f5c800ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.095+0000 7f5c86aa6700 1 -- 172.21.15.28:0/453206346 >> 172.21.15.28:0/453206346 conn(0x7f5c800b1650 msgr2=0x7f5c800b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:46.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.095+0000 7f5c86aa6700 1 -- 172.21.15.28:0/453206346 shutdown_connections 2023-11-28T11:08:46.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.095+0000 7f5c86aa6700 1 -- 172.21.15.28:0/453206346 wait complete. 2023-11-28T11:08:46.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.096+0000 7f5c86aa6700 1 Processor -- start 2023-11-28T11:08:46.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.096+0000 7f5c86aa6700 1 -- start start 2023-11-28T11:08:46.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.097+0000 7f5c86aa6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c800ce890 0x7f5c80155040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.097+0000 7f5c86aa6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c80159590 0x7f5c800c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.097+0000 7f5c86aa6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c8015db70 0x7f5c80156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.097+0000 7f5c86aa6700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c80156c80 con 0x7f5c8015db70 2023-11-28T11:08:46.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.097+0000 7f5c86aa6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c800b5210 con 0x7f5c80159590 2023-11-28T11:08:46.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.097+0000 7f5c86aa6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c801556a0 con 0x7f5c800ce890 2023-11-28T11:08:46.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.097+0000 7f5c85043700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c8015db70 0x7f5c80156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.097+0000 7f5c85043700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c8015db70 0x7f5c80156740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46394/0 (socket says 172.21.15.28:46394) 2023-11-28T11:08:46.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.097+0000 7f5c85043700 1 -- 172.21.15.28:0/2769643623 learned_addr learned my addr 172.21.15.28:0/2769643623 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:46.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.098+0000 7f5c84842700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c800ce890 0x7f5c80155040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.098+0000 7f5c85043700 1 -- 172.21.15.28:0/2769643623 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c800ce890 msgr2=0x7f5c80155040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.098+0000 7f5c7ffff700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c80159590 0x7f5c800c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.098+0000 7f5c85043700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c800ce890 0x7f5c80155040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.098+0000 7f5c85043700 1 -- 172.21.15.28:0/2769643623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c80159590 msgr2=0x7f5c800c9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.098+0000 7f5c85043700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c80159590 0x7f5c800c9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.098+0000 7f5c85043700 1 -- 172.21.15.28:0/2769643623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c74009550 con 0x7f5c8015db70 2023-11-28T11:08:46.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.098+0000 7f5c84842700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c800ce890 0x7f5c80155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:46.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.098+0000 7f5c7ffff700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c80159590 0x7f5c800c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:46.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.098+0000 7f5c85043700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c8015db70 0x7f5c80156740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5c78007e40 tx=0x7f5c7800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.098+0000 7f5c7d7fa700 1 -- 172.21.15.28:0/2769643623 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7800eea0 con 0x7f5c8015db70 2023-11-28T11:08:46.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.098+0000 7f5c7d7fa700 1 -- 172.21.15.28:0/2769643623 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c78014420 con 0x7f5c8015db70 2023-11-28T11:08:46.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.098+0000 7f5c86aa6700 1 -- 172.21.15.28:0/2769643623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8015d370 con 0x7f5c8015db70 2023-11-28T11:08:46.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.098+0000 7f5c7d7fa700 1 -- 172.21.15.28:0/2769643623 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c780147b0 con 0x7f5c8015db70 2023-11-28T11:08:46.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.098+0000 7f5c86aa6700 1 -- 172.21.15.28:0/2769643623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c800af240 con 0x7f5c8015db70 2023-11-28T11:08:46.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.099+0000 7f5c86aa6700 1 -- 172.21.15.28:0/2769643623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8015e010 con 0x7f5c8015db70 2023-11-28T11:08:46.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.099+0000 7f5c7d7fa700 1 -- 172.21.15.28:0/2769643623 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5c78012050 con 0x7f5c8015db70 2023-11-28T11:08:46.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.100+0000 7f5c7d7fa700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5c640682f0 0x7f5c6406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:46.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.100+0000 7f5c84842700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5c640682f0 0x7f5c6406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:46.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.100+0000 7f5c7d7fa700 1 -- 172.21.15.28:0/2769643623 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6062+0+0 (secure 0 0 0) 0x7f5c78086370 con 0x7f5c8015db70 2023-11-28T11:08:46.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.100+0000 7f5c84842700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5c640682f0 0x7f5c6406a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5c740098f0 tx=0x7f5c74008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:46.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.104+0000 7f5c7d7fa700 1 -- 172.21.15.28:0/2769643623 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5c78052b40 con 0x7f5c8015db70 2023-11-28T11:08:46.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.308+0000 7f5c86aa6700 1 -- 172.21.15.28:0/2769643623 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f5c80155a90 con 0x7f5c8015db70 2023-11-28T11:08:46.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.929+0000 7f5c7d7fa700 1 -- 172.21.15.28:0/2769643623 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v21) v1 ==== 157+0+0 (secure 0 0 0) 0x7f5c78056160 con 0x7f5c8015db70 2023-11-28T11:08:46.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.932+0000 7f5c86aa6700 1 -- 172.21.15.28:0/2769643623 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5c640682f0 msgr2=0x7f5c6406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.932+0000 7f5c86aa6700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5c640682f0 0x7f5c6406a7a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5c740098f0 tx=0x7f5c74008030 comp rx=0 tx=0).stop 2023-11-28T11:08:46.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.932+0000 7f5c86aa6700 1 -- 172.21.15.28:0/2769643623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c8015db70 msgr2=0x7f5c80156740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:46.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.932+0000 7f5c86aa6700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c8015db70 0x7f5c80156740 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f5c78007e40 tx=0x7f5c7800e3c0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.932+0000 7f5c86aa6700 1 -- 172.21.15.28:0/2769643623 shutdown_connections 2023-11-28T11:08:46.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.932+0000 7f5c86aa6700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5c640682f0 0x7f5c6406a7a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.932+0000 7f5c86aa6700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c8015db70 0x7f5c80156740 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.932+0000 7f5c86aa6700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c800ce890 0x7f5c80155040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.932+0000 7f5c86aa6700 1 --2- 172.21.15.28:0/2769643623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c80159590 0x7f5c800c9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:46.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.932+0000 7f5c86aa6700 1 -- 172.21.15.28:0/2769643623 >> 172.21.15.28:0/2769643623 conn(0x7f5c800b1650 msgr2=0x7f5c8006d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:46.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.932+0000 7f5c86aa6700 1 -- 172.21.15.28:0/2769643623 shutdown_connections 2023-11-28T11:08:46.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:46.932+0000 7f5c86aa6700 1 -- 172.21.15.28:0/2769643623 wait complete. 2023-11-28T11:08:46.951 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:08:47.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.092+0000 7f759d39e700 1 Processor -- start 2023-11-28T11:08:47.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.092+0000 7f759d39e700 1 -- start start 2023-11-28T11:08:47.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.092+0000 7f759d39e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f75980b6510 0x7f75980cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.092+0000 7f759d39e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75980ccfe0 0x7f7598070520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.092+0000 7f759d39e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75980ce9b0 con 0x7f7598070a60 2023-11-28T11:08:47.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.092+0000 7f759d39e700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7598070f00 con 0x7f75980b6510 2023-11-28T11:08:47.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.092+0000 7f759d39e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75980b0d30 con 0x7f75980ccfe0 2023-11-28T11:08:47.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.093+0000 7f7596ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f75980b6510 0x7f75980cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.093+0000 7f7596ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f75980b6510 0x7f75980cb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46398/0 (socket says 172.21.15.28:46398) 2023-11-28T11:08:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.093+0000 7f7596ffd700 1 -- 172.21.15.28:0/957699601 learned_addr learned my addr 172.21.15.28:0/957699601 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.093+0000 7f75967fc700 1 --2- 172.21.15.28:0/957699601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75980ccfe0 0x7f7598070520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.093+0000 7f7596ffd700 1 -- 172.21.15.28:0/957699601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75980ccfe0 msgr2=0x7f7598070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.093+0000 7f7596ffd700 1 --2- 172.21.15.28:0/957699601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75980ccfe0 0x7f7598070520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.093+0000 7f7596ffd700 1 -- 172.21.15.28:0/957699601 >> v1:172.21.15.59:6789/0 conn(0x7f7598070a60 legacy=0x7f759815df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.093+0000 7f7596ffd700 1 -- 172.21.15.28:0/957699601 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75980b0fd0 con 0x7f75980b6510 2023-11-28T11:08:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.093+0000 7f7596ffd700 1 --2- 172.21.15.28:0/957699601 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75980b6510 0x7f75980cb220 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f75880098f0 tx=0x7f7588009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8acb034a01886e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.093+0000 7f7595ffb700 1 -- 172.21.15.28:0/957699601 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588004230 con 0x7f75980b6510 2023-11-28T11:08:47.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.093+0000 7f7595ffb700 1 -- 172.21.15.28:0/957699601 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7588004390 con 0x7f75980b6510 2023-11-28T11:08:47.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.093+0000 7f7595ffb700 1 -- 172.21.15.28:0/957699601 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75880036a0 con 0x7f75980b6510 2023-11-28T11:08:47.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.094+0000 7f759d39e700 1 -- 172.21.15.28:0/957699601 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75980b6510 msgr2=0x7f75980cb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.094+0000 7f759d39e700 1 --2- 172.21.15.28:0/957699601 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75980b6510 0x7f75980cb220 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f75880098f0 tx=0x7f7588009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:47.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.094+0000 7f759d39e700 1 -- 172.21.15.28:0/957699601 shutdown_connections 2023-11-28T11:08:47.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.094+0000 7f759d39e700 1 --2- 172.21.15.28:0/957699601 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75980b6510 0x7f75980cb220 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.094+0000 7f759d39e700 1 --2- 172.21.15.28:0/957699601 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75980ccfe0 0x7f7598070520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.094+0000 7f759d39e700 1 -- 172.21.15.28:0/957699601 >> 172.21.15.28:0/957699601 conn(0x7f75980d0bd0 msgr2=0x7f75980d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:47.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.094+0000 7f759d39e700 1 -- 172.21.15.28:0/957699601 shutdown_connections 2023-11-28T11:08:47.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.094+0000 7f759d39e700 1 -- 172.21.15.28:0/957699601 wait complete. 2023-11-28T11:08:47.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.095+0000 7f759d39e700 1 Processor -- start 2023-11-28T11:08:47.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.095+0000 7f759d39e700 1 -- start start 2023-11-28T11:08:47.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f759d39e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7598070a60 0x7f75981509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f759d39e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75980b6510 0x7f75980c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f759d39e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75980ccfe0 0x7f75980b5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f759d39e700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7598150f10 con 0x7f7598070a60 2023-11-28T11:08:47.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f759d39e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7598155280 con 0x7f75980ccfe0 2023-11-28T11:08:47.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f759d39e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75980c7d40 con 0x7f75980b6510 2023-11-28T11:08:47.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f7596ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7598070a60 0x7f75981509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f7596ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7598070a60 0x7f75981509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46404/0 (socket says 172.21.15.28:46404) 2023-11-28T11:08:47.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f7596ffd700 1 -- 172.21.15.28:0/2074761425 learned_addr learned my addr 172.21.15.28:0/2074761425 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:47.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f75967fc700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75980b6510 0x7f75980c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f75977fe700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75980ccfe0 0x7f75980b5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f7596ffd700 1 -- 172.21.15.28:0/2074761425 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75980b6510 msgr2=0x7f75980c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f7596ffd700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75980b6510 0x7f75980c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f7596ffd700 1 -- 172.21.15.28:0/2074761425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75980ccfe0 msgr2=0x7f75980b5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:47.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f7596ffd700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75980ccfe0 0x7f75980b5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:47.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f7596ffd700 1 -- 172.21.15.28:0/2074761425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7588009550 con 0x7f7598070a60 2023-11-28T11:08:47.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f75967fc700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75980b6510 0x7f75980c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:47.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f75977fe700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75980ccfe0 0x7f75980b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:47.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.096+0000 7f7596ffd700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7598070a60 0x7f75981509d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f759806ae00 tx=0x7f7588004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.097+0000 7f757f7fe700 1 -- 172.21.15.28:0/2074761425 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7588003e70 con 0x7f7598070a60 2023-11-28T11:08:47.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.097+0000 7f757f7fe700 1 -- 172.21.15.28:0/2074761425 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758801f070 con 0x7f7598070a60 2023-11-28T11:08:47.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.097+0000 7f759d39e700 1 -- 172.21.15.28:0/2074761425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75980cb760 con 0x7f7598070a60 2023-11-28T11:08:47.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.097+0000 7f757f7fe700 1 -- 172.21.15.28:0/2074761425 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758800b8c0 con 0x7f7598070a60 2023-11-28T11:08:47.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.097+0000 7f759d39e700 1 -- 172.21.15.28:0/2074761425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75980b12b0 con 0x7f7598070a60 2023-11-28T11:08:47.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.098+0000 7f759d39e700 1 -- 172.21.15.28:0/2074761425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75980cfe50 con 0x7f7598070a60 2023-11-28T11:08:47.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.098+0000 7f757f7fe700 1 -- 172.21.15.28:0/2074761425 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f758802a020 con 0x7f7598070a60 2023-11-28T11:08:47.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.098+0000 7f757f7fe700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75840682f0 0x7f758406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:47.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.099+0000 7f757f7fe700 1 -- 172.21.15.28:0/2074761425 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6288+0+0 (secure 0 0 0) 0x7f7588087580 con 0x7f7598070a60 2023-11-28T11:08:47.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.099+0000 7f75967fc700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75840682f0 0x7f758406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:47.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.099+0000 7f75967fc700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75840682f0 0x7f758406a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f758c006fd0 tx=0x7f758c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:47.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.103+0000 7f757f7fe700 1 -- 172.21.15.28:0/2074761425 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7588053d00 con 0x7f7598070a60 2023-11-28T11:08:47.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:47.298+0000 7f759d39e700 1 -- 172.21.15.28:0/2074761425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f75980cfe50 con 0x7f7598070a60 2023-11-28T11:08:49.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.097+0000 7f757f7fe700 1 -- 172.21.15.28:0/2074761425 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v23) v1 ==== 228+0+0 (secure 0 0 0) 0x7f7588057320 con 0x7f7598070a60 2023-11-28T11:08:49.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.103+0000 7f759d39e700 1 -- 172.21.15.28:0/2074761425 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75840682f0 msgr2=0x7f758406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.103+0000 7f759d39e700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75840682f0 0x7f758406a7a0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f758c006fd0 tx=0x7f758c008040 comp rx=0 tx=0).stop 2023-11-28T11:08:49.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.103+0000 7f759d39e700 1 -- 172.21.15.28:0/2074761425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7598070a60 msgr2=0x7f75981509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.103+0000 7f759d39e700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7598070a60 0x7f75981509d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f759806ae00 tx=0x7f7588004750 comp rx=0 tx=0).stop 2023-11-28T11:08:49.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.104+0000 7f759d39e700 1 -- 172.21.15.28:0/2074761425 shutdown_connections 2023-11-28T11:08:49.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.104+0000 7f759d39e700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75840682f0 0x7f758406a7a0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.104+0000 7f759d39e700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7598070a60 0x7f75981509d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.104+0000 7f759d39e700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75980ccfe0 0x7f75980b5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.104+0000 7f759d39e700 1 --2- 172.21.15.28:0/2074761425 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75980b6510 0x7f75980c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.104+0000 7f759d39e700 1 -- 172.21.15.28:0/2074761425 >> 172.21.15.28:0/2074761425 conn(0x7f75980d0bd0 msgr2=0x7f759806e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:49.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.104+0000 7f759d39e700 1 -- 172.21.15.28:0/2074761425 shutdown_connections 2023-11-28T11:08:49.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.106+0000 7f759d39e700 1 -- 172.21.15.28:0/2074761425 wait complete. 2023-11-28T11:08:49.110 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data_ec' created 2023-11-28T11:08:49.125 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-11-28T11:08:49.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.259+0000 7f99d1033700 1 Processor -- start 2023-11-28T11:08:49.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.259+0000 7f99d1033700 1 -- start start 2023-11-28T11:08:49.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.260+0000 7f99d1033700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99cc0cefa0 0x7f99cc15db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.260+0000 7f99d1033700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99cc15e0b0 con 0x7f99cc159590 2023-11-28T11:08:49.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.260+0000 7f99d1033700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99cc15d120 con 0x7f99cc06e400 2023-11-28T11:08:49.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.260+0000 7f99d1033700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc15d6e0 con 0x7f99cc0cefa0 2023-11-28T11:08:49.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.260+0000 7f99cad9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f99cc159590 0x7f99cc0ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34470/0 (socket says 172.21.15.28:34470) 2023-11-28T11:08:49.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.260+0000 7f99cad9d700 1 -- 172.21.15.28:0/212238454 learned_addr learned my addr 172.21.15.28:0/212238454 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:49.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.260+0000 7f99c9d9b700 1 -- 172.21.15.28:0/212238454 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2526935456 0 0) 0x7f99c0001110 con 0x7f99cc159590 2023-11-28T11:08:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.260+0000 7f99cb59e700 1 --2- 172.21.15.28:0/212238454 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99cc0cefa0 0x7f99cc15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.260+0000 7f99c9d9b700 1 -- 172.21.15.28:0/212238454 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f99b40035b0 con 0x7f99cc159590 2023-11-28T11:08:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99c9d9b700 1 -- 172.21.15.28:0/212238454 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 651886401 0 0) 0x7f99b40035b0 con 0x7f99cc159590 2023-11-28T11:08:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99c9d9b700 1 -- 172.21.15.28:0/212238454 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99cc0cefa0 msgr2=0x7f99cc15db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99c9d9b700 1 --2- 172.21.15.28:0/212238454 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99cc0cefa0 0x7f99cc15db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99c9d9b700 1 -- 172.21.15.28:0/212238454 >> v1:172.21.15.59:6789/0 conn(0x7f99cc06e400 legacy=0x7f99cc0ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99c9d9b700 1 -- 172.21.15.28:0/212238454 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc0aef60 con 0x7f99cc159590 2023-11-28T11:08:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99c9d9b700 1 -- 172.21.15.28:0/212238454 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f99c0001800 con 0x7f99cc159590 2023-11-28T11:08:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99c9d9b700 1 -- 172.21.15.28:0/212238454 >> v1:172.21.15.28:6789/0 conn(0x7f99cc159590 legacy=0x7f99cc0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99c9d9b700 1 --2- 172.21.15.28:0/212238454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99b4003a30 0x7f99b4003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99c9d9b700 1 --2- 172.21.15.28:0/212238454 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99cc0cefa0 0x7f99b4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99c9d9b700 1 --2- 172.21.15.28:0/212238454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99b4004d90 0x7f99b4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99c9d9b700 1 -- 172.21.15.28:0/212238454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc15d6e0 con 0x7f99b4004d90 2023-11-28T11:08:49.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99c9d9b700 1 -- 172.21.15.28:0/212238454 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99b400bac0 con 0x7f99b4003a30 2023-11-28T11:08:49.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99c9d9b700 1 -- 172.21.15.28:0/212238454 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99b400bc80 con 0x7f99cc0cefa0 2023-11-28T11:08:49.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99cb59e700 1 --2- 172.21.15.28:0/212238454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99b4004d90 0x7f99b4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99ca59c700 1 --2- 172.21.15.28:0/212238454 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99cc0cefa0 0x7f99b4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99cad9d700 1 --2- 172.21.15.28:0/212238454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99b4003a30 0x7f99b4003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99cb59e700 1 -- 172.21.15.28:0/212238454 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99cc0cefa0 msgr2=0x7f99b4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:49.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99cb59e700 1 --2- 172.21.15.28:0/212238454 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99cc0cefa0 0x7f99b4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99cb59e700 1 -- 172.21.15.28:0/212238454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99b4003a30 msgr2=0x7f99b4003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:49.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99cb59e700 1 --2- 172.21.15.28:0/212238454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99b4003a30 0x7f99b4003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99cb59e700 1 -- 172.21.15.28:0/212238454 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99b40035b0 con 0x7f99b4004d90 2023-11-28T11:08:49.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99cad9d700 1 --2- 172.21.15.28:0/212238454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99b4003a30 0x7f99b4003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99cb59e700 1 --2- 172.21.15.28:0/212238454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99b4004d90 0x7f99b4005200 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009f90 tx=0x7f99b4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f83beb7c6ae0da3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.261+0000 7f99c9d9b700 1 -- 172.21.15.28:0/212238454 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c400ec30 con 0x7f99b4004d90 2023-11-28T11:08:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.262+0000 7f99c9d9b700 1 -- 172.21.15.28:0/212238454 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99c400f040 con 0x7f99b4004d90 2023-11-28T11:08:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.262+0000 7f99c9d9b700 1 -- 172.21.15.28:0/212238454 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99c4014480 con 0x7f99b4004d90 2023-11-28T11:08:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.262+0000 7f99d1033700 1 -- 172.21.15.28:0/212238454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99b4004d90 msgr2=0x7f99b4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.262+0000 7f99d1033700 1 --2- 172.21.15.28:0/212238454 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99b4004d90 0x7f99b4005200 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f99c4009f90 tx=0x7f99b4005880 comp rx=0 tx=0).stop 2023-11-28T11:08:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.262+0000 7f99cb59e700 1 -- 172.21.15.28:0/212238454 reap_dead start 2023-11-28T11:08:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.262+0000 7f99d1033700 1 -- 172.21.15.28:0/212238454 shutdown_connections 2023-11-28T11:08:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.262+0000 7f99d1033700 1 -- 172.21.15.28:0/212238454 >> 172.21.15.28:0/212238454 conn(0x7f99cc0b1650 msgr2=0x7f99cc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.262+0000 7f99d1033700 1 -- 172.21.15.28:0/212238454 shutdown_connections 2023-11-28T11:08:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.262+0000 7f99d1033700 1 -- 172.21.15.28:0/212238454 wait complete. 2023-11-28T11:08:49.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.264+0000 7f99d1033700 1 Processor -- start 2023-11-28T11:08:49.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.264+0000 7f99d1033700 1 -- start start 2023-11-28T11:08:49.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.264+0000 7f99d1033700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99b4004d90 0x7f99cc154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.264+0000 7f99d1033700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99cc0ac910 0x7f99cc0acd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.264+0000 7f99d1033700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99cc0b2950 0x7f99cc0b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.264+0000 7f99d1033700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc0b3300 con 0x7f99cc0ac910 2023-11-28T11:08:49.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.264+0000 7f99d1033700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc0adfa0 con 0x7f99b4004d90 2023-11-28T11:08:49.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.264+0000 7f99d1033700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99cc0ad390 con 0x7f99cc0b2950 2023-11-28T11:08:49.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.264+0000 7f99ca59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99cc0ac910 0x7f99cc0acd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.264+0000 7f99ca59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99cc0ac910 0x7f99cc0acd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46426/0 (socket says 172.21.15.28:46426) 2023-11-28T11:08:49.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.264+0000 7f99ca59c700 1 -- 172.21.15.28:0/4092320206 learned_addr learned my addr 172.21.15.28:0/4092320206 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.265+0000 7f99cad9d700 1 --2- 172.21.15.28:0/4092320206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99b4004d90 0x7f99cc154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.265+0000 7f99cb59e700 1 --2- 172.21.15.28:0/4092320206 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99cc0b2950 0x7f99cc0b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.265+0000 7f99ca59c700 1 -- 172.21.15.28:0/4092320206 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99cc0b2950 msgr2=0x7f99cc0b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.265+0000 7f99ca59c700 1 --2- 172.21.15.28:0/4092320206 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99cc0b2950 0x7f99cc0b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.265+0000 7f99ca59c700 1 -- 172.21.15.28:0/4092320206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99b4004d90 msgr2=0x7f99cc154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.265+0000 7f99ca59c700 1 --2- 172.21.15.28:0/4092320206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99b4004d90 0x7f99cc154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.265+0000 7f99ca59c700 1 -- 172.21.15.28:0/4092320206 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99cc15d370 con 0x7f99cc0ac910 2023-11-28T11:08:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.265+0000 7f99ca59c700 1 --2- 172.21.15.28:0/4092320206 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99cc0ac910 0x7f99cc0acd80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00f820 tx=0x7f99bc00fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.265+0000 7f99b37fe700 1 -- 172.21.15.28:0/4092320206 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0052e0 con 0x7f99cc0ac910 2023-11-28T11:08:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.265+0000 7f99b37fe700 1 -- 172.21.15.28:0/4092320206 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99bc005440 con 0x7f99cc0ac910 2023-11-28T11:08:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.265+0000 7f99d1033700 1 -- 172.21.15.28:0/4092320206 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99cc0af240 con 0x7f99cc0ac910 2023-11-28T11:08:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.265+0000 7f99b37fe700 1 -- 172.21.15.28:0/4092320206 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99bc0038c0 con 0x7f99cc0ac910 2023-11-28T11:08:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.265+0000 7f99d1033700 1 -- 172.21.15.28:0/4092320206 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99cc159960 con 0x7f99cc0ac910 2023-11-28T11:08:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.266+0000 7f99d1033700 1 -- 172.21.15.28:0/4092320206 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99cc0739f0 con 0x7f99cc0ac910 2023-11-28T11:08:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.267+0000 7f99b37fe700 1 -- 172.21.15.28:0/4092320206 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f99bc003a20 con 0x7f99cc0ac910 2023-11-28T11:08:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.267+0000 7f99b37fe700 1 --2- 172.21.15.28:0/4092320206 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f99b80682f0 0x7f99b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.267+0000 7f99b37fe700 1 -- 172.21.15.28:0/4092320206 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f99bc085fa0 con 0x7f99cc0ac910 2023-11-28T11:08:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.267+0000 7f99cad9d700 1 --2- 172.21.15.28:0/4092320206 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f99b80682f0 0x7f99b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.267+0000 7f99cad9d700 1 --2- 172.21.15.28:0/4092320206 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f99b80682f0 0x7f99b806a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99c0000fb0 tx=0x7f99c0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.271+0000 7f99b37fe700 1 -- 172.21.15.28:0/4092320206 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f99bc052530 con 0x7f99cc0ac910 2023-11-28T11:08:49.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:49.464+0000 7f99d1033700 1 -- 172.21.15.28:0/4092320206 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f99cc06c170 con 0x7f99cc0ac910 2023-11-28T11:08:50.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.103+0000 7f99b37fe700 1 -- 172.21.15.28:0/4092320206 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 5 allow_ec_overwrites to true v24) v1 ==== 169+0+0 (secure 0 0 0) 0x7f99bc003cb0 con 0x7f99cc0ac910 2023-11-28T11:08:50.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.109+0000 7f99d1033700 1 -- 172.21.15.28:0/4092320206 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f99b80682f0 msgr2=0x7f99b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.109+0000 7f99d1033700 1 --2- 172.21.15.28:0/4092320206 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f99b80682f0 0x7f99b806a7a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f99c0000fb0 tx=0x7f99c0009040 comp rx=0 tx=0).stop 2023-11-28T11:08:50.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.109+0000 7f99d1033700 1 -- 172.21.15.28:0/4092320206 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99cc0ac910 msgr2=0x7f99cc0acd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.109+0000 7f99d1033700 1 --2- 172.21.15.28:0/4092320206 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99cc0ac910 0x7f99cc0acd80 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f99bc00f820 tx=0x7f99bc00fbe0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.110+0000 7f99d1033700 1 -- 172.21.15.28:0/4092320206 shutdown_connections 2023-11-28T11:08:50.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.110+0000 7f99d1033700 1 --2- 172.21.15.28:0/4092320206 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f99b80682f0 0x7f99b806a7a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.110+0000 7f99d1033700 1 --2- 172.21.15.28:0/4092320206 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f99cc0b2950 0x7f99cc0b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.110+0000 7f99d1033700 1 --2- 172.21.15.28:0/4092320206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f99b4004d90 0x7f99cc154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.110+0000 7f99d1033700 1 --2- 172.21.15.28:0/4092320206 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f99cc0ac910 0x7f99cc0acd80 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.110+0000 7f99d1033700 1 -- 172.21.15.28:0/4092320206 >> 172.21.15.28:0/4092320206 conn(0x7f99cc0b1650 msgr2=0x7f99cc0cde00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:50.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.111+0000 7f99d1033700 1 -- 172.21.15.28:0/4092320206 shutdown_connections 2023-11-28T11:08:50.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.115+0000 7f99d1033700 1 -- 172.21.15.28:0/4092320206 wait complete. 2023-11-28T11:08:50.128 INFO:teuthology.orchestra.run.smithi028.stderr:set pool 5 allow_ec_overwrites to true 2023-11-28T11:08:50.144 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-11-28T11:08:50.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.273+0000 7f3fd08b0700 1 Processor -- start 2023-11-28T11:08:50.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.273+0000 7f3fd08b0700 1 -- start start 2023-11-28T11:08:50.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.274+0000 7f3fd08b0700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3fcc15fc00 0x7f3fcc06eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.274+0000 7f3fd08b0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3fcc0b2d90 0x7f3fcc0cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.274+0000 7f3fd08b0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fcc06f940 con 0x7f3fcc06f3e0 2023-11-28T11:08:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.274+0000 7f3fd08b0700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fcc06fc20 con 0x7f3fcc15fc00 2023-11-28T11:08:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.274+0000 7f3fd08b0700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fcc0cb3f0 con 0x7f3fcc0b2d90 2023-11-28T11:08:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.275+0000 7f3fca59c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3fcc15fc00 0x7f3fcc06eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.275+0000 7f3fca59c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3fcc15fc00 0x7f3fcc06eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46432/0 (socket says 172.21.15.28:46432) 2023-11-28T11:08:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.275+0000 7f3fca59c700 1 -- 172.21.15.28:0/41747350 learned_addr learned my addr 172.21.15.28:0/41747350 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.275+0000 7f3fcad9d700 1 --2- 172.21.15.28:0/41747350 >> v2:172.21.15.59:3300/0 conn(0x7f3fcc0b2d90 0x7f3fcc0cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.275+0000 7f3fca59c700 1 -- 172.21.15.28:0/41747350 >> v2:172.21.15.59:3300/0 conn(0x7f3fcc0b2d90 msgr2=0x7f3fcc0cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.275+0000 7f3fca59c700 1 --2- 172.21.15.28:0/41747350 >> v2:172.21.15.59:3300/0 conn(0x7f3fcc0b2d90 0x7f3fcc0cbbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.275+0000 7f3fca59c700 1 -- 172.21.15.28:0/41747350 >> v1:172.21.15.59:6789/0 conn(0x7f3fcc06f3e0 legacy=0x7f3fcc0b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.275+0000 7f3fca59c700 1 -- 172.21.15.28:0/41747350 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fcc0cb6d0 con 0x7f3fcc15fc00 2023-11-28T11:08:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.275+0000 7f3fca59c700 1 --2- 172.21.15.28:0/41747350 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fcc15fc00 0x7f3fcc06eea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0b6b30 tx=0x7f3fc0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf18e271993cadc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.275+0000 7f3fc959a700 1 -- 172.21.15.28:0/41747350 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0011830 con 0x7f3fcc15fc00 2023-11-28T11:08:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.275+0000 7f3fc959a700 1 -- 172.21.15.28:0/41747350 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc0011990 con 0x7f3fcc15fc00 2023-11-28T11:08:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.275+0000 7f3fc959a700 1 -- 172.21.15.28:0/41747350 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0011d20 con 0x7f3fcc15fc00 2023-11-28T11:08:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.276+0000 7f3fd08b0700 1 -- 172.21.15.28:0/41747350 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fcc15fc00 msgr2=0x7f3fcc06eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.276+0000 7f3fd08b0700 1 --2- 172.21.15.28:0/41747350 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fcc15fc00 0x7f3fcc06eea0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc0b6b30 tx=0x7f3fc0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.276+0000 7f3fd08b0700 1 -- 172.21.15.28:0/41747350 shutdown_connections 2023-11-28T11:08:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.276+0000 7f3fd08b0700 1 --2- 172.21.15.28:0/41747350 >> v2:172.21.15.59:3300/0 conn(0x7f3fcc0b2d90 0x7f3fcc0cbbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.276+0000 7f3fd08b0700 1 --2- 172.21.15.28:0/41747350 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fcc15fc00 0x7f3fcc06eea0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.276+0000 7f3fd08b0700 1 -- 172.21.15.28:0/41747350 >> 172.21.15.28:0/41747350 conn(0x7f3fcc0cd680 msgr2=0x7f3fcc0cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.276+0000 7f3fd08b0700 1 -- 172.21.15.28:0/41747350 shutdown_connections 2023-11-28T11:08:50.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.276+0000 7f3fd08b0700 1 -- 172.21.15.28:0/41747350 wait complete. 2023-11-28T11:08:50.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.277+0000 7f3fd08b0700 1 Processor -- start 2023-11-28T11:08:50.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.277+0000 7f3fd08b0700 1 -- start start 2023-11-28T11:08:50.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.278+0000 7f3fd08b0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fcc06f3e0 0x7f3fcc0c58f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.278+0000 7f3fd08b0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fcc0b2d90 0x7f3fcc0c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.278+0000 7f3fd08b0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3fcc15fc00 0x7f3fcc1f2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.278+0000 7f3fd08b0700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc0c6650 con 0x7f3fcc0b2d90 2023-11-28T11:08:50.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.278+0000 7f3fd08b0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1f37e0 con 0x7f3fcc06f3e0 2023-11-28T11:08:50.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.278+0000 7f3fd08b0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fcc1f3ac0 con 0x7f3fcc15fc00 2023-11-28T11:08:50.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.278+0000 7f3fc9d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fcc0b2d90 0x7f3fcc0c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.278+0000 7f3fc9d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fcc0b2d90 0x7f3fcc0c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46440/0 (socket says 172.21.15.28:46440) 2023-11-28T11:08:50.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.278+0000 7f3fc9d9b700 1 -- 172.21.15.28:0/1868391526 learned_addr learned my addr 172.21.15.28:0/1868391526 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:50.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.278+0000 7f3fca59c700 1 --2- 172.21.15.28:0/1868391526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fcc06f3e0 0x7f3fcc0c58f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.278+0000 7f3fcad9d700 1 --2- 172.21.15.28:0/1868391526 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3fcc15fc00 0x7f3fcc1f2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.279+0000 7f3fc9d9b700 1 -- 172.21.15.28:0/1868391526 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3fcc15fc00 msgr2=0x7f3fcc1f2520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.279+0000 7f3fc9d9b700 1 --2- 172.21.15.28:0/1868391526 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3fcc15fc00 0x7f3fcc1f2520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.279+0000 7f3fc9d9b700 1 -- 172.21.15.28:0/1868391526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fcc06f3e0 msgr2=0x7f3fcc0c58f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.279+0000 7f3fc9d9b700 1 --2- 172.21.15.28:0/1868391526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fcc06f3e0 0x7f3fcc0c58f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.279+0000 7f3fc9d9b700 1 -- 172.21.15.28:0/1868391526 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc0009550 con 0x7f3fcc0b2d90 2023-11-28T11:08:50.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.279+0000 7f3fca59c700 1 --2- 172.21.15.28:0/1868391526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fcc06f3e0 0x7f3fcc0c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:50.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.279+0000 7f3fc9d9b700 1 --2- 172.21.15.28:0/1868391526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fcc0b2d90 0x7f3fcc0c5e30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00c4e0 tx=0x7f3fbc00c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:50.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.279+0000 7f3fbaffd700 1 -- 172.21.15.28:0/1868391526 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc0052d0 con 0x7f3fcc0b2d90 2023-11-28T11:08:50.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.279+0000 7f3fbaffd700 1 -- 172.21.15.28:0/1868391526 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc005430 con 0x7f3fcc0b2d90 2023-11-28T11:08:50.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.279+0000 7f3fbaffd700 1 -- 172.21.15.28:0/1868391526 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc0038c0 con 0x7f3fcc0b2d90 2023-11-28T11:08:50.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.279+0000 7f3fd08b0700 1 -- 172.21.15.28:0/1868391526 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fcc0ce340 con 0x7f3fcc0b2d90 2023-11-28T11:08:50.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.279+0000 7f3fd08b0700 1 -- 172.21.15.28:0/1868391526 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fcc15ffd0 con 0x7f3fcc0b2d90 2023-11-28T11:08:50.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.280+0000 7f3fd08b0700 1 -- 172.21.15.28:0/1868391526 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fcc0cc130 con 0x7f3fcc0b2d90 2023-11-28T11:08:50.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.280+0000 7f3fbaffd700 1 -- 172.21.15.28:0/1868391526 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3fbc019030 con 0x7f3fcc0b2d90 2023-11-28T11:08:50.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.281+0000 7f3fbaffd700 1 --2- 172.21.15.28:0/1868391526 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3fac0682f0 0x7f3fac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.281+0000 7f3fbaffd700 1 -- 172.21.15.28:0/1868391526 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6781+0+0 (secure 0 0 0) 0x7f3fbc0852c0 con 0x7f3fcc0b2d90 2023-11-28T11:08:50.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.281+0000 7f3fca59c700 1 --2- 172.21.15.28:0/1868391526 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3fac0682f0 0x7f3fac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.281+0000 7f3fca59c700 1 --2- 172.21.15.28:0/1868391526 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3fac0682f0 0x7f3fac06a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0014040 tx=0x7f3fc0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:50.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.285+0000 7f3fbaffd700 1 -- 172.21.15.28:0/1868391526 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3fbc051850 con 0x7f3fcc0b2d90 2023-11-28T11:08:50.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.530+0000 7f3fd08b0700 1 -- 172.21.15.28:0/1868391526 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7f3fcc0cbbf0 con 0x7f3fcc0b2d90 2023-11-28T11:08:50.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.533+0000 7f3fbaffd700 1 -- 172.21.15.28:0/1868391526 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 5 to fsmap v7) v1 ==== 137+0+0 (secure 0 0 0) 0x7f3fbc054e70 con 0x7f3fcc0b2d90 2023-11-28T11:08:50.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.535+0000 7f3fd08b0700 1 -- 172.21.15.28:0/1868391526 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3fac0682f0 msgr2=0x7f3fac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.535+0000 7f3fd08b0700 1 --2- 172.21.15.28:0/1868391526 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3fac0682f0 0x7f3fac06a7a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0014040 tx=0x7f3fc0008030 comp rx=0 tx=0).stop 2023-11-28T11:08:50.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.536+0000 7f3fd08b0700 1 -- 172.21.15.28:0/1868391526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fcc0b2d90 msgr2=0x7f3fcc0c5e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.536+0000 7f3fd08b0700 1 --2- 172.21.15.28:0/1868391526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fcc0b2d90 0x7f3fcc0c5e30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00c4e0 tx=0x7f3fbc00c8a0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.536+0000 7f3fd08b0700 1 -- 172.21.15.28:0/1868391526 shutdown_connections 2023-11-28T11:08:50.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.536+0000 7f3fd08b0700 1 --2- 172.21.15.28:0/1868391526 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3fac0682f0 0x7f3fac06a7a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.536+0000 7f3fd08b0700 1 --2- 172.21.15.28:0/1868391526 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3fcc15fc00 0x7f3fcc1f2520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.536+0000 7f3fd08b0700 1 --2- 172.21.15.28:0/1868391526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fcc06f3e0 0x7f3fcc0c58f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.536+0000 7f3fd08b0700 1 --2- 172.21.15.28:0/1868391526 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fcc0b2d90 0x7f3fcc0c5e30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.536+0000 7f3fd08b0700 1 -- 172.21.15.28:0/1868391526 >> 172.21.15.28:0/1868391526 conn(0x7f3fcc0cd680 msgr2=0x7f3fcc15a7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:50.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.536+0000 7f3fd08b0700 1 -- 172.21.15.28:0/1868391526 shutdown_connections 2023-11-28T11:08:50.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.536+0000 7f3fd08b0700 1 -- 172.21.15.28:0/1868391526 wait complete. 2023-11-28T11:08:50.540 INFO:teuthology.orchestra.run.smithi028.stderr:added data pool 5 to fsmap 2023-11-28T11:08:50.549 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:08:50.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.672+0000 7f49d33b8700 1 Processor -- start 2023-11-28T11:08:50.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.672+0000 7f49d33b8700 1 -- start start 2023-11-28T11:08:50.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d33b8700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f49cc15c990 0x7f49cc0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d33b8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49cc161090 0x7f49cc0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d33b8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f49cc0caeb0 0x7f49cc0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d33b8700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49cc0ce340 con 0x7f49cc15c990 2023-11-28T11:08:50.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d33b8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49cc0cebe0 con 0x7f49cc0caeb0 2023-11-28T11:08:50.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d33b8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc0cb3f0 con 0x7f49cc161090 2023-11-28T11:08:50.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d1154700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f49cc15c990 0x7f49cc0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d1154700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f49cc15c990 0x7f49cc0cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46446/0 (socket says 172.21.15.28:46446) 2023-11-28T11:08:50.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d1154700 1 -- 172.21.15.28:0/2018294963 learned_addr learned my addr 172.21.15.28:0/2018294963 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d1955700 1 --2- 172.21.15.28:0/2018294963 >> v2:172.21.15.59:3300/0 conn(0x7f49cc0caeb0 0x7f49cc0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d0953700 1 --2- 172.21.15.28:0/2018294963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49cc161090 0x7f49cc0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d1154700 1 -- 172.21.15.28:0/2018294963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49cc161090 msgr2=0x7f49cc0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d1154700 1 --2- 172.21.15.28:0/2018294963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49cc161090 0x7f49cc0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d1154700 1 -- 172.21.15.28:0/2018294963 >> v2:172.21.15.59:3300/0 conn(0x7f49cc0caeb0 msgr2=0x7f49cc0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d1154700 1 --2- 172.21.15.28:0/2018294963 >> v2:172.21.15.59:3300/0 conn(0x7f49cc0caeb0 0x7f49cc0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.673+0000 7f49d1154700 1 -- 172.21.15.28:0/2018294963 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc0ce590 con 0x7f49cc15c990 2023-11-28T11:08:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.674+0000 7f49d1154700 1 --2- 172.21.15.28:0/2018294963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f49cc15c990 0x7f49cc0cf070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0ca080 tx=0x7f49bc009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2d440bb8e7e1172 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.674+0000 7f49c3fff700 1 -- 172.21.15.28:0/2018294963 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc011830 con 0x7f49cc15c990 2023-11-28T11:08:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.674+0000 7f49c3fff700 1 -- 172.21.15.28:0/2018294963 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc011990 con 0x7f49cc15c990 2023-11-28T11:08:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.674+0000 7f49c3fff700 1 -- 172.21.15.28:0/2018294963 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc011d20 con 0x7f49cc15c990 2023-11-28T11:08:50.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.674+0000 7f49d33b8700 1 -- 172.21.15.28:0/2018294963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f49cc15c990 msgr2=0x7f49cc0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.674+0000 7f49d33b8700 1 --2- 172.21.15.28:0/2018294963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f49cc15c990 0x7f49cc0cf070 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0ca080 tx=0x7f49bc009bd0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.674+0000 7f49d33b8700 1 -- 172.21.15.28:0/2018294963 shutdown_connections 2023-11-28T11:08:50.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.674+0000 7f49d33b8700 1 --2- 172.21.15.28:0/2018294963 >> v2:172.21.15.59:3300/0 conn(0x7f49cc0caeb0 0x7f49cc0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.674+0000 7f49d33b8700 1 --2- 172.21.15.28:0/2018294963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f49cc15c990 0x7f49cc0cf070 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.674+0000 7f49d33b8700 1 --2- 172.21.15.28:0/2018294963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49cc161090 0x7f49cc0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.674+0000 7f49d33b8700 1 -- 172.21.15.28:0/2018294963 >> 172.21.15.28:0/2018294963 conn(0x7f49cc0b1650 msgr2=0x7f49cc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:50.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.674+0000 7f49d33b8700 1 -- 172.21.15.28:0/2018294963 shutdown_connections 2023-11-28T11:08:50.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.674+0000 7f49d33b8700 1 -- 172.21.15.28:0/2018294963 wait complete. 2023-11-28T11:08:50.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.676+0000 7f49d33b8700 1 Processor -- start 2023-11-28T11:08:50.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.676+0000 7f49d33b8700 1 -- start start 2023-11-28T11:08:50.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.676+0000 7f49d33b8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49cc0caeb0 0x7f49cc0ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.676+0000 7f49d33b8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49cc15c990 0x7f49cc0c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.676+0000 7f49d33b8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f49cc161090 0x7f49cc0c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.676+0000 7f49d33b8700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc0c43a0 con 0x7f49cc161090 2023-11-28T11:08:50.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.676+0000 7f49d33b8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc14b1d0 con 0x7f49cc15c990 2023-11-28T11:08:50.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.676+0000 7f49d33b8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc0c9a30 con 0x7f49cc0caeb0 2023-11-28T11:08:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.676+0000 7f49d1955700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f49cc161090 0x7f49cc0c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.677+0000 7f49d1955700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f49cc161090 0x7f49cc0c5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46456/0 (socket says 172.21.15.28:46456) 2023-11-28T11:08:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.677+0000 7f49d1955700 1 -- 172.21.15.28:0/2409701240 learned_addr learned my addr 172.21.15.28:0/2409701240 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.677+0000 7f49d0953700 1 --2- 172.21.15.28:0/2409701240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49cc15c990 0x7f49cc0c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.677+0000 7f49d1154700 1 --2- 172.21.15.28:0/2409701240 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49cc0caeb0 0x7f49cc0ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.677+0000 7f49d1955700 1 -- 172.21.15.28:0/2409701240 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49cc0caeb0 msgr2=0x7f49cc0ae2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.677+0000 7f49d1955700 1 --2- 172.21.15.28:0/2409701240 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49cc0caeb0 0x7f49cc0ae2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.677+0000 7f49d1955700 1 -- 172.21.15.28:0/2409701240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49cc15c990 msgr2=0x7f49cc0c5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.677+0000 7f49d1955700 1 --2- 172.21.15.28:0/2409701240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49cc15c990 0x7f49cc0c5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.677+0000 7f49d1955700 1 -- 172.21.15.28:0/2409701240 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc009550 con 0x7f49cc161090 2023-11-28T11:08:50.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.677+0000 7f49d1955700 1 --2- 172.21.15.28:0/2409701240 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f49cc161090 0x7f49cc0c5660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f49c400eed0 tx=0x7f49c400b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:50.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.677+0000 7f49c1ffb700 1 -- 172.21.15.28:0/2409701240 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4010040 con 0x7f49cc161090 2023-11-28T11:08:50.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.677+0000 7f49c1ffb700 1 -- 172.21.15.28:0/2409701240 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c400bd30 con 0x7f49cc161090 2023-11-28T11:08:50.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.677+0000 7f49d33b8700 1 -- 172.21.15.28:0/2409701240 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49cc0cf5b0 con 0x7f49cc161090 2023-11-28T11:08:50.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.677+0000 7f49c1ffb700 1 -- 172.21.15.28:0/2409701240 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4009ea0 con 0x7f49cc161090 2023-11-28T11:08:50.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.677+0000 7f49d33b8700 1 -- 172.21.15.28:0/2409701240 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc0ce870 con 0x7f49cc161090 2023-11-28T11:08:50.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.678+0000 7f49d33b8700 1 -- 172.21.15.28:0/2409701240 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49cc0cf740 con 0x7f49cc161090 2023-11-28T11:08:50.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.679+0000 7f49c1ffb700 1 -- 172.21.15.28:0/2409701240 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f49c4014640 con 0x7f49cc161090 2023-11-28T11:08:50.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.679+0000 7f49c1ffb700 1 --2- 172.21.15.28:0/2409701240 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f49b00682f0 0x7f49b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:50.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.679+0000 7f49c1ffb700 1 -- 172.21.15.28:0/2409701240 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f49c4086730 con 0x7f49cc161090 2023-11-28T11:08:50.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.679+0000 7f49d1154700 1 --2- 172.21.15.28:0/2409701240 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f49b00682f0 0x7f49b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:50.684 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.680+0000 7f49d1154700 1 --2- 172.21.15.28:0/2409701240 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f49b00682f0 0x7f49b006a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009c10 tx=0x7f49bc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:50.684 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.683+0000 7f49c1ffb700 1 -- 172.21.15.28:0/2409701240 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f49c4052ca0 con 0x7f49cc161090 2023-11-28T11:08:50.841 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:08:50.844 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:08:50.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.920+0000 7f49d33b8700 1 -- 172.21.15.28:0/2409701240 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49cc0675a0 con 0x7f49cc161090 2023-11-28T11:08:50.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.920+0000 7f49c1ffb700 1 -- 172.21.15.28:0/2409701240 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4644 (secure 0 0 0) 0x7f49c40562c0 con 0x7f49cc161090 2023-11-28T11:08:50.922 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:08:50.922 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/3112877467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3112877467},{"type":"v1","addr":"172.21.15.59:6834","nonce":3112877467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3448236489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3448236489},{"type":"v1","addr":"172.21.15.28:6836","nonce":3448236489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4232269940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4232269940},{"type":"v1","addr":"172.21.15.28:6837","nonce":4232269940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:45.924816+0000","modified":"2023-11-28T11:08:50.531944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/3293053261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3293053261},{"type":"v1","addr":"172.21.15.59:6835","nonce":3293053261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-28T11:08:50.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.922+0000 7f49d33b8700 1 -- 172.21.15.28:0/2409701240 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f49b00682f0 msgr2=0x7f49b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.922+0000 7f49d33b8700 1 --2- 172.21.15.28:0/2409701240 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f49b00682f0 0x7f49b006a7a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f49bc009c10 tx=0x7f49bc008030 comp rx=0 tx=0).stop 2023-11-28T11:08:50.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.922+0000 7f49d33b8700 1 -- 172.21.15.28:0/2409701240 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f49cc161090 msgr2=0x7f49cc0c5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:50.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.922+0000 7f49d33b8700 1 --2- 172.21.15.28:0/2409701240 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f49cc161090 0x7f49cc0c5660 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f49c400eed0 tx=0x7f49c400b390 comp rx=0 tx=0).stop 2023-11-28T11:08:50.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.922+0000 7f49d33b8700 1 -- 172.21.15.28:0/2409701240 shutdown_connections 2023-11-28T11:08:50.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.922+0000 7f49d33b8700 1 --2- 172.21.15.28:0/2409701240 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f49b00682f0 0x7f49b006a7a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.922+0000 7f49d33b8700 1 --2- 172.21.15.28:0/2409701240 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f49cc161090 0x7f49cc0c5660 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.922+0000 7f49d33b8700 1 --2- 172.21.15.28:0/2409701240 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f49cc0caeb0 0x7f49cc0ae2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.922+0000 7f49d33b8700 1 --2- 172.21.15.28:0/2409701240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49cc15c990 0x7f49cc0c5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:50.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.923+0000 7f49d33b8700 1 -- 172.21.15.28:0/2409701240 >> 172.21.15.28:0/2409701240 conn(0x7f49cc0b1650 msgr2=0x7f49cc0b0ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:50.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.923+0000 7f49d33b8700 1 -- 172.21.15.28:0/2409701240 shutdown_connections 2023-11-28T11:08:50.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:50.923+0000 7f49d33b8700 1 -- 172.21.15.28:0/2409701240 wait complete. 2023-11-28T11:08:50.926 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 7 2023-11-28T11:08:50.936 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:51.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.099+0000 7f1af0966700 1 Processor -- start 2023-11-28T11:08:51.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.099+0000 7f1af0966700 1 -- start start 2023-11-28T11:08:51.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.100+0000 7f1af0966700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1aec15c990 0x7f1aec0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.100+0000 7f1af0966700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aec0ce340 con 0x7f1aec161090 2023-11-28T11:08:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.100+0000 7f1af0966700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aec0cebe0 con 0x7f1aec0cc310 2023-11-28T11:08:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.100+0000 7f1af0966700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aec0cb3f0 con 0x7f1aec15c990 2023-11-28T11:08:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.100+0000 7f1ae9d9b700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1aec161090 0x7f1aec0ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34518/0 (socket says 172.21.15.28:34518) 2023-11-28T11:08:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.101+0000 7f1ae9d9b700 1 -- 172.21.15.28:0/2366659369 learned_addr learned my addr 172.21.15.28:0/2366659369 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.101+0000 7f1aea59c700 1 --2- 172.21.15.28:0/2366659369 >> v2:172.21.15.59:3300/0 conn(0x7f1aec15c990 0x7f1aec0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.101+0000 7f1aea59c700 1 -- 172.21.15.28:0/2366659369 >> v1:172.21.15.59:6789/0 conn(0x7f1aec0cc310 legacy=0x7f1aec0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.101+0000 7f1aea59c700 1 -- 172.21.15.28:0/2366659369 >> v1:172.21.15.28:6789/0 conn(0x7f1aec161090 legacy=0x7f1aec0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.101+0000 7f1aea59c700 1 -- 172.21.15.28:0/2366659369 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec0ce590 con 0x7f1aec15c990 2023-11-28T11:08:51.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.101+0000 7f1aea59c700 1 --2- 172.21.15.28:0/2366659369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec15c990 0x7f1aec0cf070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009840 tx=0x7f1adc009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6aa78a351028c7f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:51.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.104+0000 7f1ae959a700 1 -- 172.21.15.28:0/2366659369 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc011830 con 0x7f1aec15c990 2023-11-28T11:08:51.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.104+0000 7f1ae959a700 1 -- 172.21.15.28:0/2366659369 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1adc011990 con 0x7f1aec15c990 2023-11-28T11:08:51.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.104+0000 7f1ae959a700 1 -- 172.21.15.28:0/2366659369 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1adc011d20 con 0x7f1aec15c990 2023-11-28T11:08:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.104+0000 7f1af0966700 1 -- 172.21.15.28:0/2366659369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec15c990 msgr2=0x7f1aec0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.104+0000 7f1af0966700 1 --2- 172.21.15.28:0/2366659369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec15c990 0x7f1aec0cf070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1adc009840 tx=0x7f1adc009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.104+0000 7f1af0966700 1 -- 172.21.15.28:0/2366659369 shutdown_connections 2023-11-28T11:08:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.104+0000 7f1af0966700 1 --2- 172.21.15.28:0/2366659369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec15c990 0x7f1aec0cf070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.104+0000 7f1af0966700 1 -- 172.21.15.28:0/2366659369 >> 172.21.15.28:0/2366659369 conn(0x7f1aec0b1650 msgr2=0x7f1aec0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:51.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.104+0000 7f1af0966700 1 -- 172.21.15.28:0/2366659369 shutdown_connections 2023-11-28T11:08:51.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.104+0000 7f1af0966700 1 -- 172.21.15.28:0/2366659369 wait complete. 2023-11-28T11:08:51.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.106+0000 7f1af0966700 1 Processor -- start 2023-11-28T11:08:51.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.106+0000 7f1af0966700 1 -- start start 2023-11-28T11:08:51.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1af0966700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1aec0cc310 0x7f1aec0c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1af0966700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aec15c990 0x7f1aec0c7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1af0966700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec161090 0x7f1aec0b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1af0966700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec0b5810 con 0x7f1aec15c990 2023-11-28T11:08:51.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1af0966700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec0b4c00 con 0x7f1aec161090 2023-11-28T11:08:51.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1af0966700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec0b4ee0 con 0x7f1aec0cc310 2023-11-28T11:08:51.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1ae9d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aec15c990 0x7f1aec0c7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1ae9d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aec15c990 0x7f1aec0c7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46468/0 (socket says 172.21.15.28:46468) 2023-11-28T11:08:51.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1ae9d9b700 1 -- 172.21.15.28:0/4282719035 learned_addr learned my addr 172.21.15.28:0/4282719035 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:51.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1aea59c700 1 --2- 172.21.15.28:0/4282719035 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1aec0cc310 0x7f1aec0c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1aead9d700 1 --2- 172.21.15.28:0/4282719035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec161090 0x7f1aec0b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1ae9d9b700 1 -- 172.21.15.28:0/4282719035 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1aec0cc310 msgr2=0x7f1aec0c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1ae9d9b700 1 --2- 172.21.15.28:0/4282719035 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1aec0cc310 0x7f1aec0c8970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1ae9d9b700 1 -- 172.21.15.28:0/4282719035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec161090 msgr2=0x7f1aec0b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1ae9d9b700 1 --2- 172.21.15.28:0/4282719035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec161090 0x7f1aec0b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.107+0000 7f1ae9d9b700 1 -- 172.21.15.28:0/4282719035 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1adc0094a0 con 0x7f1aec15c990 2023-11-28T11:08:51.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.108+0000 7f1ae9d9b700 1 --2- 172.21.15.28:0/4282719035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aec15c990 0x7f1aec0c7340 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000c520 tx=0x7f1ae000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:51.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.108+0000 7f1adaffd700 1 -- 172.21.15.28:0/4282719035 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0005d70 con 0x7f1aec15c990 2023-11-28T11:08:51.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.108+0000 7f1adaffd700 1 -- 172.21.15.28:0/4282719035 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae00070d0 con 0x7f1aec15c990 2023-11-28T11:08:51.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.108+0000 7f1af0966700 1 -- 172.21.15.28:0/4282719035 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aec0cb080 con 0x7f1aec15c990 2023-11-28T11:08:51.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.108+0000 7f1adaffd700 1 -- 172.21.15.28:0/4282719035 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae0007440 con 0x7f1aec15c990 2023-11-28T11:08:51.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.108+0000 7f1af0966700 1 -- 172.21.15.28:0/4282719035 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aec0ce870 con 0x7f1aec15c990 2023-11-28T11:08:51.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.109+0000 7f1af0966700 1 -- 172.21.15.28:0/4282719035 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aec0cf5b0 con 0x7f1aec15c990 2023-11-28T11:08:51.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.109+0000 7f1adaffd700 1 -- 172.21.15.28:0/4282719035 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1ae0019030 con 0x7f1aec15c990 2023-11-28T11:08:51.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.110+0000 7f1adaffd700 1 --2- 172.21.15.28:0/4282719035 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1acc0682f0 0x7f1acc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.110+0000 7f1adaffd700 1 -- 172.21.15.28:0/4282719035 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1ae0085220 con 0x7f1aec15c990 2023-11-28T11:08:51.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.110+0000 7f1aea59c700 1 --2- 172.21.15.28:0/4282719035 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1acc0682f0 0x7f1acc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.110+0000 7f1aea59c700 1 --2- 172.21.15.28:0/4282719035 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1acc0682f0 0x7f1acc06a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1adc0090b0 tx=0x7f1adc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:51.115 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.114+0000 7f1adaffd700 1 -- 172.21.15.28:0/4282719035 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1ae004b5e0 con 0x7f1aec15c990 2023-11-28T11:08:51.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.309+0000 7f1af0966700 1 -- 172.21.15.28:0/4282719035 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1aec0675a0 con 0x7f1aec15c990 2023-11-28T11:08:51.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.310+0000 7f1adaffd700 1 -- 172.21.15.28:0/4282719035 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f1ae0051840 con 0x7f1aec15c990 2023-11-28T11:08:51.312 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:08:51.312 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:50.531733+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:51.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.313+0000 7f1af0966700 1 -- 172.21.15.28:0/4282719035 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1acc0682f0 msgr2=0x7f1acc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.313+0000 7f1af0966700 1 --2- 172.21.15.28:0/4282719035 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1acc0682f0 0x7f1acc06a7a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1adc0090b0 tx=0x7f1adc008030 comp rx=0 tx=0).stop 2023-11-28T11:08:51.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.313+0000 7f1af0966700 1 -- 172.21.15.28:0/4282719035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aec15c990 msgr2=0x7f1aec0c7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.313+0000 7f1af0966700 1 --2- 172.21.15.28:0/4282719035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aec15c990 0x7f1aec0c7340 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1ae000c520 tx=0x7f1ae000c8e0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.313+0000 7f1af0966700 1 -- 172.21.15.28:0/4282719035 shutdown_connections 2023-11-28T11:08:51.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.313+0000 7f1af0966700 1 --2- 172.21.15.28:0/4282719035 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1acc0682f0 0x7f1acc06a7a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.313+0000 7f1af0966700 1 --2- 172.21.15.28:0/4282719035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec161090 0x7f1aec0b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.313+0000 7f1af0966700 1 --2- 172.21.15.28:0/4282719035 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1aec0cc310 0x7f1aec0c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.313+0000 7f1af0966700 1 --2- 172.21.15.28:0/4282719035 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aec15c990 0x7f1aec0c7340 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.313+0000 7f1af0966700 1 -- 172.21.15.28:0/4282719035 >> 172.21.15.28:0/4282719035 conn(0x7f1aec0b1650 msgr2=0x7f1aec0ccee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:51.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.313+0000 7f1af0966700 1 -- 172.21.15.28:0/4282719035 shutdown_connections 2023-11-28T11:08:51.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.313+0000 7f1af0966700 1 -- 172.21.15.28:0/4282719035 wait complete. 2023-11-28T11:08:51.327 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:51.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.452+0000 7fdf33999700 1 Processor -- start 2023-11-28T11:08:51.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.452+0000 7fdf33999700 1 -- start start 2023-11-28T11:08:51.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.453+0000 7fdf33999700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf2c15c990 0x7fdf2c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.453+0000 7fdf33999700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdf2c161090 0x7fdf2c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.453+0000 7fdf33999700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf2c0ce340 con 0x7fdf2c0caeb0 2023-11-28T11:08:51.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.453+0000 7fdf33999700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf2c0cebe0 con 0x7fdf2c161090 2023-11-28T11:08:51.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.453+0000 7fdf33999700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0cb3f0 con 0x7fdf2c15c990 2023-11-28T11:08:51.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.453+0000 7fdf30f34700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdf2c161090 0x7fdf2c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.453+0000 7fdf30f34700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdf2c161090 0x7fdf2c0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46476/0 (socket says 172.21.15.28:46476) 2023-11-28T11:08:51.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.453+0000 7fdf30f34700 1 -- 172.21.15.28:0/1361853755 learned_addr learned my addr 172.21.15.28:0/1361853755 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:51.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.453+0000 7fdf31735700 1 --2- 172.21.15.28:0/1361853755 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf2c15c990 0x7fdf2c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.454+0000 7fdf30f34700 1 -- 172.21.15.28:0/1361853755 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf2c15c990 msgr2=0x7fdf2c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:51.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.454+0000 7fdf30f34700 1 --2- 172.21.15.28:0/1361853755 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf2c15c990 0x7fdf2c0cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.454+0000 7fdf30f34700 1 -- 172.21.15.28:0/1361853755 >> v1:172.21.15.59:6789/0 conn(0x7fdf2c0caeb0 legacy=0x7fdf2c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.454+0000 7fdf30f34700 1 -- 172.21.15.28:0/1361853755 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2c0ce590 con 0x7fdf2c161090 2023-11-28T11:08:51.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.454+0000 7fdf30f34700 1 --2- 172.21.15.28:0/1361853755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf2c161090 0x7fdf2c0ca970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdf280098f0 tx=0x7fdf28009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=413015ff7329222c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:51.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.454+0000 7fdf23fff700 1 -- 172.21.15.28:0/1361853755 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2800b8a0 con 0x7fdf2c161090 2023-11-28T11:08:51.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.454+0000 7fdf23fff700 1 -- 172.21.15.28:0/1361853755 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2800ba00 con 0x7fdf2c161090 2023-11-28T11:08:51.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.454+0000 7fdf23fff700 1 -- 172.21.15.28:0/1361853755 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2800bd90 con 0x7fdf2c161090 2023-11-28T11:08:51.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.454+0000 7fdf33999700 1 -- 172.21.15.28:0/1361853755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf2c161090 msgr2=0x7fdf2c0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.454+0000 7fdf33999700 1 --2- 172.21.15.28:0/1361853755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf2c161090 0x7fdf2c0ca970 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdf280098f0 tx=0x7fdf28009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:51.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.454+0000 7fdf33999700 1 -- 172.21.15.28:0/1361853755 shutdown_connections 2023-11-28T11:08:51.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.454+0000 7fdf33999700 1 --2- 172.21.15.28:0/1361853755 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf2c15c990 0x7fdf2c0cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.454+0000 7fdf33999700 1 --2- 172.21.15.28:0/1361853755 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf2c161090 0x7fdf2c0ca970 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.454+0000 7fdf33999700 1 -- 172.21.15.28:0/1361853755 >> 172.21.15.28:0/1361853755 conn(0x7fdf2c0b1650 msgr2=0x7fdf2c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:51.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.455+0000 7fdf33999700 1 -- 172.21.15.28:0/1361853755 shutdown_connections 2023-11-28T11:08:51.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.455+0000 7fdf33999700 1 -- 172.21.15.28:0/1361853755 wait complete. 2023-11-28T11:08:51.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.456+0000 7fdf33999700 1 Processor -- start 2023-11-28T11:08:51.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.456+0000 7fdf33999700 1 -- start start 2023-11-28T11:08:51.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.457+0000 7fdf33999700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf2c0caeb0 0x7fdf2c159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.457+0000 7fdf33999700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf2c15c990 0x7fdf2c0c5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.457+0000 7fdf33999700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf2c161090 0x7fdf2c0c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.457+0000 7fdf33999700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c0c4b00 con 0x7fdf2c0caeb0 2023-11-28T11:08:51.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.457+0000 7fdf33999700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c159ea0 con 0x7fdf2c161090 2023-11-28T11:08:51.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.457+0000 7fdf33999700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf2c160830 con 0x7fdf2c15c990 2023-11-28T11:08:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.457+0000 7fdf31735700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf2c0caeb0 0x7fdf2c159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.457+0000 7fdf31735700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf2c0caeb0 0x7fdf2c159960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46480/0 (socket says 172.21.15.28:46480) 2023-11-28T11:08:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.457+0000 7fdf31735700 1 -- 172.21.15.28:0/3184861630 learned_addr learned my addr 172.21.15.28:0/3184861630 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.457+0000 7fdf30f34700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf2c15c990 0x7fdf2c0c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.458+0000 7fdf31f36700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf2c161090 0x7fdf2c0c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.458+0000 7fdf31735700 1 -- 172.21.15.28:0/3184861630 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf2c15c990 msgr2=0x7fdf2c0c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.458+0000 7fdf31735700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf2c15c990 0x7fdf2c0c5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.458+0000 7fdf31735700 1 -- 172.21.15.28:0/3184861630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf2c161090 msgr2=0x7fdf2c0c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.458+0000 7fdf31735700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf2c161090 0x7fdf2c0c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.458+0000 7fdf31735700 1 -- 172.21.15.28:0/3184861630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf28009550 con 0x7fdf2c0caeb0 2023-11-28T11:08:51.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.458+0000 7fdf30f34700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf2c15c990 0x7fdf2c0c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:51.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.458+0000 7fdf31f36700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf2c161090 0x7fdf2c0c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:51.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.458+0000 7fdf31735700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf2c0caeb0 0x7fdf2c159960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00e810 tx=0x7fdf1c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:51.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.458+0000 7fdf21ffb700 1 -- 172.21.15.28:0/3184861630 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c013610 con 0x7fdf2c0caeb0 2023-11-28T11:08:51.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.458+0000 7fdf21ffb700 1 -- 172.21.15.28:0/3184861630 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1c013770 con 0x7fdf2c0caeb0 2023-11-28T11:08:51.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.458+0000 7fdf21ffb700 1 -- 172.21.15.28:0/3184861630 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1c013b20 con 0x7fdf2c0caeb0 2023-11-28T11:08:51.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.458+0000 7fdf33999700 1 -- 172.21.15.28:0/3184861630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf2c0cf5b0 con 0x7fdf2c0caeb0 2023-11-28T11:08:51.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.458+0000 7fdf33999700 1 -- 172.21.15.28:0/3184861630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf2c0ce870 con 0x7fdf2c0caeb0 2023-11-28T11:08:51.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.459+0000 7fdf33999700 1 -- 172.21.15.28:0/3184861630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf2c0cf740 con 0x7fdf2c0caeb0 2023-11-28T11:08:51.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.459+0000 7fdf21ffb700 1 -- 172.21.15.28:0/3184861630 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdf1c023070 con 0x7fdf2c0caeb0 2023-11-28T11:08:51.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.460+0000 7fdf21ffb700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdf100682f0 0x7fdf1006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.460+0000 7fdf21ffb700 1 -- 172.21.15.28:0/3184861630 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fdf1c085c90 con 0x7fdf2c0caeb0 2023-11-28T11:08:51.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.460+0000 7fdf30f34700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdf100682f0 0x7fdf1006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.460+0000 7fdf30f34700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdf100682f0 0x7fdf1006a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c1614c0 tx=0x7fdf28008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:51.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.464+0000 7fdf21ffb700 1 -- 172.21.15.28:0/3184861630 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdf1c052180 con 0x7fdf2c0caeb0 2023-11-28T11:08:51.585 INFO:tasks.ceph.mds.d.smithi059.stderr:2023-11-28T11:08:51.584+0000 7f532f296700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:08:51.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.661+0000 7fdf33999700 1 -- 172.21.15.28:0/3184861630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf2c0cf3d0 con 0x7fdf2c0caeb0 2023-11-28T11:08:51.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.662+0000 7fdf21ffb700 1 -- 172.21.15.28:0/3184861630 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19642 (secure 0 0 0) 0x7fdf1c0557a0 con 0x7fdf2c0caeb0 2023-11-28T11:08:51.664 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:08:51.664 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:50.531733+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:51.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.664+0000 7fdf33999700 1 -- 172.21.15.28:0/3184861630 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdf100682f0 msgr2=0x7fdf1006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.664+0000 7fdf33999700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdf100682f0 0x7fdf1006a7a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c1614c0 tx=0x7fdf28008030 comp rx=0 tx=0).stop 2023-11-28T11:08:51.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.664+0000 7fdf33999700 1 -- 172.21.15.28:0/3184861630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf2c0caeb0 msgr2=0x7fdf2c159960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.664+0000 7fdf33999700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf2c0caeb0 0x7fdf2c159960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c00e810 tx=0x7fdf1c00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.664+0000 7fdf33999700 1 -- 172.21.15.28:0/3184861630 shutdown_connections 2023-11-28T11:08:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.664+0000 7fdf33999700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdf100682f0 0x7fdf1006a7a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.664+0000 7fdf33999700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdf2c0caeb0 0x7fdf2c159960 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.664+0000 7fdf33999700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf2c161090 0x7fdf2c0c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.664+0000 7fdf33999700 1 --2- 172.21.15.28:0/3184861630 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdf2c15c990 0x7fdf2c0c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.664+0000 7fdf33999700 1 -- 172.21.15.28:0/3184861630 >> 172.21.15.28:0/3184861630 conn(0x7fdf2c0b1650 msgr2=0x7fdf2c0ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.664+0000 7fdf33999700 1 -- 172.21.15.28:0/3184861630 shutdown_connections 2023-11-28T11:08:51.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.664+0000 7fdf33999700 1 -- 172.21.15.28:0/3184861630 wait complete. 2023-11-28T11:08:51.693 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:08:51.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.815+0000 7fe79a7cb700 1 Processor -- start 2023-11-28T11:08:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.816+0000 7fe79a7cb700 1 -- start start 2023-11-28T11:08:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.816+0000 7fe79a7cb700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe7941619c0 0x7fe794163da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.816+0000 7fe79a7cb700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79406ed80 con 0x7fe79406aa70 2023-11-28T11:08:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.816+0000 7fe79a7cb700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe79406f060 con 0x7fe794158a30 2023-11-28T11:08:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.816+0000 7fe79a7cb700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7940cd950 con 0x7fe7941619c0 2023-11-28T11:08:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.816+0000 7fe798d68700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe7941619c0 0x7fe794163da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.817+0000 7fe7937fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe79406aa70 0x7fe7940ce860 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34548/0 (socket says 172.21.15.28:34548) 2023-11-28T11:08:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.817+0000 7fe7937fe700 1 -- 172.21.15.28:0/35387743 learned_addr learned my addr 172.21.15.28:0/35387743 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.817+0000 7fe798d68700 1 -- 172.21.15.28:0/35387743 >> v1:172.21.15.59:6789/0 conn(0x7fe794158a30 legacy=0x7fe7940ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.817+0000 7fe798d68700 1 -- 172.21.15.28:0/35387743 >> v1:172.21.15.28:6789/0 conn(0x7fe79406aa70 legacy=0x7fe7940ce860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.817+0000 7fe798d68700 1 -- 172.21.15.28:0/35387743 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7940cdc30 con 0x7fe7941619c0 2023-11-28T11:08:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.817+0000 7fe792ffd700 1 -- 172.21.15.28:0/35387743 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4150208784 0 0) 0x7fe79406ed80 con 0x7fe79406aa70 2023-11-28T11:08:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.817+0000 7fe798d68700 1 --2- 172.21.15.28:0/35387743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7941619c0 0x7fe794163da0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00bfa0 tx=0x7fe78c00d330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d917661fba3908 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.817+0000 7fe792ffd700 1 -- 172.21.15.28:0/35387743 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c010070 con 0x7fe7941619c0 2023-11-28T11:08:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.817+0000 7fe792ffd700 1 -- 172.21.15.28:0/35387743 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c00b2e0 con 0x7fe7941619c0 2023-11-28T11:08:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.817+0000 7fe792ffd700 1 -- 172.21.15.28:0/35387743 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00dcf0 con 0x7fe7941619c0 2023-11-28T11:08:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.818+0000 7fe79a7cb700 1 -- 172.21.15.28:0/35387743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7941619c0 msgr2=0x7fe794163da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.818+0000 7fe79a7cb700 1 --2- 172.21.15.28:0/35387743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7941619c0 0x7fe794163da0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe78c00bfa0 tx=0x7fe78c00d330 comp rx=0 tx=0).stop 2023-11-28T11:08:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.818+0000 7fe79a7cb700 1 -- 172.21.15.28:0/35387743 shutdown_connections 2023-11-28T11:08:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.818+0000 7fe79a7cb700 1 --2- 172.21.15.28:0/35387743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7941619c0 0x7fe794163da0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.818+0000 7fe79a7cb700 1 -- 172.21.15.28:0/35387743 >> 172.21.15.28:0/35387743 conn(0x7fe7940b0dd0 msgr2=0x7fe7940b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:51.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.818+0000 7fe79a7cb700 1 -- 172.21.15.28:0/35387743 shutdown_connections 2023-11-28T11:08:51.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.818+0000 7fe79a7cb700 1 -- 172.21.15.28:0/35387743 wait complete. 2023-11-28T11:08:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.819+0000 7fe79a7cb700 1 Processor -- start 2023-11-28T11:08:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.819+0000 7fe79a7cb700 1 -- start start 2023-11-28T11:08:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe79a7cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe79406aa70 0x7fe79414b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe79a7cb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe794158a30 0x7fe794157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe79a7cb700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7941619c0 0x7fe79414f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe79a7cb700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe794156d90 con 0x7fe7941619c0 2023-11-28T11:08:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe79a7cb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7941561c0 con 0x7fe79406aa70 2023-11-28T11:08:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe79a7cb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe794156460 con 0x7fe794158a30 2023-11-28T11:08:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe798d68700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7941619c0 0x7fe79414f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe798d68700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7941619c0 0x7fe79414f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46498/0 (socket says 172.21.15.28:46498) 2023-11-28T11:08:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe798d68700 1 -- 172.21.15.28:0/3092504000 learned_addr learned my addr 172.21.15.28:0/3092504000 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe7937fe700 1 --2- 172.21.15.28:0/3092504000 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe794158a30 0x7fe794157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe793fff700 1 --2- 172.21.15.28:0/3092504000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe79406aa70 0x7fe79414b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe798d68700 1 -- 172.21.15.28:0/3092504000 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe794158a30 msgr2=0x7fe794157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe798d68700 1 --2- 172.21.15.28:0/3092504000 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe794158a30 0x7fe794157220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe798d68700 1 -- 172.21.15.28:0/3092504000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe79406aa70 msgr2=0x7fe79414b0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe798d68700 1 --2- 172.21.15.28:0/3092504000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe79406aa70 0x7fe79414b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe798d68700 1 -- 172.21.15.28:0/3092504000 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe78c00bc80 con 0x7fe7941619c0 2023-11-28T11:08:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.820+0000 7fe798d68700 1 --2- 172.21.15.28:0/3092504000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7941619c0 0x7fe79414f660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe79406d2f0 tx=0x7fe78c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.821+0000 7fe790ff9700 1 -- 172.21.15.28:0/3092504000 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c00dea0 con 0x7fe7941619c0 2023-11-28T11:08:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.821+0000 7fe790ff9700 1 -- 172.21.15.28:0/3092504000 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78c01d650 con 0x7fe7941619c0 2023-11-28T11:08:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.821+0000 7fe79a7cb700 1 -- 172.21.15.28:0/3092504000 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe79406e9c0 con 0x7fe7941619c0 2023-11-28T11:08:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.821+0000 7fe790ff9700 1 -- 172.21.15.28:0/3092504000 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78c01da00 con 0x7fe7941619c0 2023-11-28T11:08:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.821+0000 7fe79a7cb700 1 -- 172.21.15.28:0/3092504000 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79406f340 con 0x7fe7941619c0 2023-11-28T11:08:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.822+0000 7fe79a7cb700 1 -- 172.21.15.28:0/3092504000 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe79406eb50 con 0x7fe7941619c0 2023-11-28T11:08:51.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.822+0000 7fe790ff9700 1 -- 172.21.15.28:0/3092504000 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe78c02f070 con 0x7fe7941619c0 2023-11-28T11:08:51.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.822+0000 7fe790ff9700 1 --2- 172.21.15.28:0/3092504000 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe7800682f0 0x7fe78006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:51.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.823+0000 7fe790ff9700 1 -- 172.21.15.28:0/3092504000 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe78c089b90 con 0x7fe7941619c0 2023-11-28T11:08:51.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.823+0000 7fe793fff700 1 --2- 172.21.15.28:0/3092504000 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe7800682f0 0x7fe78006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:51.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.823+0000 7fe793fff700 1 --2- 172.21.15.28:0/3092504000 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe7800682f0 0x7fe78006a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe784000f60 tx=0x7fe784009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:51.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:51.827+0000 7fe790ff9700 1 -- 172.21.15.28:0/3092504000 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe78c055f90 con 0x7fe7941619c0 2023-11-28T11:08:52.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.065+0000 7fe79a7cb700 1 -- 172.21.15.28:0/3092504000 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe7940675a0 con 0x7fe7941619c0 2023-11-28T11:08:52.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.066+0000 7fe790ff9700 1 -- 172.21.15.28:0/3092504000 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) v1 ==== 93+0+4644 (secure 0 0 0) 0x7fe78c023090 con 0x7fe7941619c0 2023-11-28T11:08:52.068 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:08:52.068 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":7,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/3112877467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3112877467},{"type":"v1","addr":"172.21.15.59:6834","nonce":3112877467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3448236489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3448236489},{"type":"v1","addr":"172.21.15.28:6836","nonce":3448236489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4232269940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4232269940},{"type":"v1","addr":"172.21.15.28:6837","nonce":4232269940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":7,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:45.924816+0000","modified":"2023-11-28T11:08:50.531944+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/3293053261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3293053261},{"type":"v1","addr":"172.21.15.59:6835","nonce":3293053261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-11-28T11:08:52.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.068+0000 7fe79a7cb700 1 -- 172.21.15.28:0/3092504000 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe7800682f0 msgr2=0x7fe78006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.068+0000 7fe79a7cb700 1 --2- 172.21.15.28:0/3092504000 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe7800682f0 0x7fe78006a7a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe784000f60 tx=0x7fe784009040 comp rx=0 tx=0).stop 2023-11-28T11:08:52.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.068+0000 7fe79a7cb700 1 -- 172.21.15.28:0/3092504000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7941619c0 msgr2=0x7fe79414f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.068+0000 7fe79a7cb700 1 --2- 172.21.15.28:0/3092504000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7941619c0 0x7fe79414f660 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe79406d2f0 tx=0x7fe78c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.068+0000 7fe79a7cb700 1 -- 172.21.15.28:0/3092504000 shutdown_connections 2023-11-28T11:08:52.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.068+0000 7fe79a7cb700 1 --2- 172.21.15.28:0/3092504000 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe7800682f0 0x7fe78006a7a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.068+0000 7fe79a7cb700 1 --2- 172.21.15.28:0/3092504000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe7941619c0 0x7fe79414f660 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.068+0000 7fe79a7cb700 1 --2- 172.21.15.28:0/3092504000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe79406aa70 0x7fe79414b0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.068+0000 7fe79a7cb700 1 --2- 172.21.15.28:0/3092504000 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe794158a30 0x7fe794157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.068+0000 7fe79a7cb700 1 -- 172.21.15.28:0/3092504000 >> 172.21.15.28:0/3092504000 conn(0x7fe7940b0dd0 msgr2=0x7fe79414c770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.068+0000 7fe79a7cb700 1 -- 172.21.15.28:0/3092504000 shutdown_connections 2023-11-28T11:08:52.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.068+0000 7fe79a7cb700 1 -- 172.21.15.28:0/3092504000 wait complete. 2023-11-28T11:08:52.073 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 7 2023-11-28T11:08:52.484 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 7, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:08:45.924816+0000', 'modified': '2023-11-28T11:08:50.531944+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4296': {'gid': 4296, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6835/3293053261', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6833', 'nonce': 3293053261}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 3293053261}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-11-28T11:08:52.484 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:08:52.485 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T11:08:52.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.620+0000 7fbbab047700 1 Processor -- start 2023-11-28T11:08:52.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.620+0000 7fbbab047700 1 -- start start 2023-11-28T11:08:52.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.620+0000 7fbbab047700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbba40cb470 0x7fbba40ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.620+0000 7fbbab047700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbba406bb80 0x7fbba414fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.620+0000 7fbbab047700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbba406c140 con 0x7fbba40cb0a0 2023-11-28T11:08:52.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.620+0000 7fbbab047700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbba40caad0 con 0x7fbba40cb470 2023-11-28T11:08:52.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.620+0000 7fbbab047700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba40c9ab0 con 0x7fbba406bb80 2023-11-28T11:08:52.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.621+0000 7fbba3fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbba40cb470 0x7fbba40ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.621+0000 7fbba8de3700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbba40cb0a0 0x7fbba415a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34560/0 (socket says 172.21.15.28:34560) 2023-11-28T11:08:52.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.621+0000 7fbba8de3700 1 -- 172.21.15.28:0/300977443 learned_addr learned my addr 172.21.15.28:0/300977443 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:52.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.621+0000 7fbba95e4700 1 --2- 172.21.15.28:0/300977443 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbba406bb80 0x7fbba414fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.621+0000 7fbba3fff700 1 -- 172.21.15.28:0/300977443 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbba406bb80 msgr2=0x7fbba414fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.621+0000 7fbba3fff700 1 --2- 172.21.15.28:0/300977443 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbba406bb80 0x7fbba414fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.621+0000 7fbba3fff700 1 -- 172.21.15.28:0/300977443 >> v1:172.21.15.28:6789/0 conn(0x7fbba40cb0a0 legacy=0x7fbba415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.621+0000 7fbba3fff700 1 -- 172.21.15.28:0/300977443 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba40c9d90 con 0x7fbba40cb470 2023-11-28T11:08:52.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.621+0000 7fbba37fe700 1 -- 172.21.15.28:0/300977443 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570809677 0 0) 0x7fbba406c140 con 0x7fbba40cb0a0 2023-11-28T11:08:52.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.621+0000 7fbba3fff700 1 --2- 172.21.15.28:0/300977443 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbba40cb470 0x7fbba40ca590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000bae0 tx=0x7fbb9000bdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3896c60d8320e74 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.621+0000 7fbba37fe700 1 -- 172.21.15.28:0/300977443 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9000dd30 con 0x7fbba40cb470 2023-11-28T11:08:52.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.621+0000 7fbba37fe700 1 -- 172.21.15.28:0/300977443 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9000de90 con 0x7fbba40cb470 2023-11-28T11:08:52.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.621+0000 7fbba37fe700 1 -- 172.21.15.28:0/300977443 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb90010690 con 0x7fbba40cb470 2023-11-28T11:08:52.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.622+0000 7fbbab047700 1 -- 172.21.15.28:0/300977443 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbba40cb470 msgr2=0x7fbba40ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.622+0000 7fbbab047700 1 --2- 172.21.15.28:0/300977443 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbba40cb470 0x7fbba40ca590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000bae0 tx=0x7fbb9000bdf0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.622+0000 7fbbab047700 1 -- 172.21.15.28:0/300977443 shutdown_connections 2023-11-28T11:08:52.629 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.622+0000 7fbbab047700 1 --2- 172.21.15.28:0/300977443 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbba406bb80 0x7fbba414fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.622+0000 7fbbab047700 1 --2- 172.21.15.28:0/300977443 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbba40cb470 0x7fbba40ca590 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.622+0000 7fbbab047700 1 -- 172.21.15.28:0/300977443 >> 172.21.15.28:0/300977443 conn(0x7fbba40b0dd0 msgr2=0x7fbba40b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.622+0000 7fbbab047700 1 -- 172.21.15.28:0/300977443 shutdown_connections 2023-11-28T11:08:52.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.622+0000 7fbbab047700 1 -- 172.21.15.28:0/300977443 wait complete. 2023-11-28T11:08:52.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.623+0000 7fbbab047700 1 Processor -- start 2023-11-28T11:08:52.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.623+0000 7fbbab047700 1 -- start start 2023-11-28T11:08:52.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.624+0000 7fbbab047700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbba406bb80 0x7fbba40b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.624+0000 7fbbab047700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbba40cb0a0 0x7fbba41ece60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.624+0000 7fbbab047700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbba40cb470 0x7fbba41ed3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.624+0000 7fbbab047700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba40ced00 con 0x7fbba406bb80 2023-11-28T11:08:52.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.624+0000 7fbbab047700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbba40ce740 con 0x7fbba40cb470 2023-11-28T11:08:52.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.624+0000 7fbbab047700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbba40cea20 con 0x7fbba40cb0a0 2023-11-28T11:08:52.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.624+0000 7fbba8de3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbba406bb80 0x7fbba40b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.624+0000 7fbba8de3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbba406bb80 0x7fbba40b2380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46508/0 (socket says 172.21.15.28:46508) 2023-11-28T11:08:52.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.624+0000 7fbba8de3700 1 -- 172.21.15.28:0/553664917 learned_addr learned my addr 172.21.15.28:0/553664917 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:52.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.624+0000 7fbba95e4700 1 --2- 172.21.15.28:0/553664917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbba40cb470 0x7fbba41ed3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.624+0000 7fbba3fff700 1 --2- 172.21.15.28:0/553664917 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbba40cb0a0 0x7fbba41ece60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.625+0000 7fbba8de3700 1 -- 172.21.15.28:0/553664917 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbba40cb0a0 msgr2=0x7fbba41ece60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.625+0000 7fbba8de3700 1 --2- 172.21.15.28:0/553664917 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbba40cb0a0 0x7fbba41ece60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.625+0000 7fbba8de3700 1 -- 172.21.15.28:0/553664917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbba40cb470 msgr2=0x7fbba41ed3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.625+0000 7fbba8de3700 1 --2- 172.21.15.28:0/553664917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbba40cb470 0x7fbba41ed3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.625+0000 7fbba8de3700 1 -- 172.21.15.28:0/553664917 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb9000b740 con 0x7fbba406bb80 2023-11-28T11:08:52.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.625+0000 7fbba8de3700 1 --2- 172.21.15.28:0/553664917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbba406bb80 0x7fbba40b2380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800c750 tx=0x7fbb9800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.625+0000 7fbba17fa700 1 -- 172.21.15.28:0/553664917 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98004c80 con 0x7fbba406bb80 2023-11-28T11:08:52.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.625+0000 7fbba17fa700 1 -- 172.21.15.28:0/553664917 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb98004e00 con 0x7fbba406bb80 2023-11-28T11:08:52.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.625+0000 7fbbab047700 1 -- 172.21.15.28:0/553664917 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbba41503e0 con 0x7fbba406bb80 2023-11-28T11:08:52.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.625+0000 7fbba17fa700 1 -- 172.21.15.28:0/553664917 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb98007310 con 0x7fbba406bb80 2023-11-28T11:08:52.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.625+0000 7fbbab047700 1 -- 172.21.15.28:0/553664917 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbba40ca070 con 0x7fbba406bb80 2023-11-28T11:08:52.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.626+0000 7fbba17fa700 1 -- 172.21.15.28:0/553664917 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbb9801a020 con 0x7fbba406bb80 2023-11-28T11:08:52.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.627+0000 7fbba17fa700 1 --2- 172.21.15.28:0/553664917 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbb880682f0 0x7fbb8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.627+0000 7fbba17fa700 1 -- 172.21.15.28:0/553664917 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbb98085a60 con 0x7fbba406bb80 2023-11-28T11:08:52.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.627+0000 7fbba3fff700 1 --2- 172.21.15.28:0/553664917 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbb880682f0 0x7fbb8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.627+0000 7fbba3fff700 1 --2- 172.21.15.28:0/553664917 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbb880682f0 0x7fbb8806a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000b2c0 tx=0x7fbb9001c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.629+0000 7fbbb71d0b80 1 Processor -- start 2023-11-28T11:08:52.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.629+0000 7fbbb71d0b80 1 -- start start 2023-11-28T11:08:52.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbbb71d0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba4810 0x564871b7aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbbb71d0b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba2130 0x564871ba4be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbbb71d0b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc0770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbbb71d0b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x564871a64bc0 con 0x564871ba5120 2023-11-28T11:08:52.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbbb71d0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x564871ad3350 con 0x564871ba4810 2023-11-28T11:08:52.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbbb71d0b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x564871acc880 con 0x564871ba2130 2023-11-28T11:08:52.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbba95e4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc0770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbba95e4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc0770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46520/0 (socket says 172.21.15.28:46520) 2023-11-28T11:08:52.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbba95e4700 1 -- 172.21.15.28:0/1114263013 learned_addr learned my addr 172.21.15.28:0/1114263013 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:52.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbba8de3700 1 --2- 172.21.15.28:0/1114263013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba4810 0x564871b7aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbba3fff700 1 --2- 172.21.15.28:0/1114263013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba2130 0x564871ba4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbba95e4700 1 -- 172.21.15.28:0/1114263013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba2130 msgr2=0x564871ba4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbba95e4700 1 --2- 172.21.15.28:0/1114263013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba2130 0x564871ba4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbba95e4700 1 -- 172.21.15.28:0/1114263013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba4810 msgr2=0x564871b7aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbba95e4700 1 --2- 172.21.15.28:0/1114263013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba4810 0x564871b7aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbba95e4700 1 -- 172.21.15.28:0/1114263013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56487194e5e0 con 0x564871ba5120 2023-11-28T11:08:52.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.630+0000 7fbba8de3700 1 --2- 172.21.15.28:0/1114263013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba4810 0x564871b7aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:52.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.631+0000 7fbba95e4700 1 --2- 172.21.15.28:0/1114263013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc0770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c003aa0 tx=0x7fbb9c003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5a6a10a8696582d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.631+0000 7fbb8e7fc700 1 -- 172.21.15.28:0/1114263013 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c004800 con 0x564871ba5120 2023-11-28T11:08:52.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.631+0000 7fbb8e7fc700 1 -- 172.21.15.28:0/1114263013 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9c015420 con 0x564871ba5120 2023-11-28T11:08:52.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.631+0000 7fbb8e7fc700 1 -- 172.21.15.28:0/1114263013 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c015790 con 0x564871ba5120 2023-11-28T11:08:52.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.631+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/1114263013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 msgr2=0x564871bc0770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.631+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/1114263013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc0770 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c003aa0 tx=0x7fbb9c003db0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.631+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/1114263013 shutdown_connections 2023-11-28T11:08:52.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.631+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/1114263013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc0770 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.631+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/1114263013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba4810 0x564871b7aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.631+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/1114263013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba2130 0x564871ba4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.631+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/1114263013 >> 172.21.15.28:0/1114263013 conn(0x564871b77d60 msgr2=0x564871b786f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.631+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/1114263013 shutdown_connections 2023-11-28T11:08:52.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.631+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/1114263013 wait complete. 2023-11-28T11:08:52.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.632+0000 7fbbb71d0b80 1 -- start start 2023-11-28T11:08:52.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.632+0000 7fbbb71d0b80 1 Processor -- start 2023-11-28T11:08:52.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbbb71d0b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba2130 0x564871ba7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbbb71d0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba4810 0x564871bb77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbbb71d0b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbbb71d0b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x5648719508d0 con 0x564871ba5120 2023-11-28T11:08:52.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbbb71d0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x564871bb8270 con 0x564871ba4810 2023-11-28T11:08:52.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbbb71d0b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x564871bc4130 con 0x564871ba2130 2023-11-28T11:08:52.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbbb71d0b80 10 client.0 _ll_register_callbacks cb 0x564871972240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:08:52.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbba95e4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbba95e4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46526/0 (socket says 172.21.15.28:46526) 2023-11-28T11:08:52.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbba95e4700 1 -- 172.21.15.28:0/937726337 learned_addr learned my addr 172.21.15.28:0/937726337 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:52.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbba3fff700 1 --2- 172.21.15.28:0/937726337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba4810 0x564871bb77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbba8de3700 1 --2- 172.21.15.28:0/937726337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba2130 0x564871ba7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbba95e4700 1 -- 172.21.15.28:0/937726337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba2130 msgr2=0x564871ba7de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbba95e4700 1 --2- 172.21.15.28:0/937726337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba2130 0x564871ba7de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbba95e4700 1 -- 172.21.15.28:0/937726337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba4810 msgr2=0x564871bb77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbba95e4700 1 --2- 172.21.15.28:0/937726337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba4810 0x564871bb77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbba95e4700 1 -- 172.21.15.28:0/937726337 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb9c011040 con 0x564871ba5120 2023-11-28T11:08:52.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbba95e4700 1 --2- 172.21.15.28:0/937726337 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc5420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c000f80 tx=0x7fbb9c004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7c35e0d379d3427 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbb837fe700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:08:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbb837fe700 1 -- 172.21.15.28:0/937726337 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c015890 con 0x564871ba5120 2023-11-28T11:08:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbb837fe700 1 -- 172.21.15.28:0/937726337 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbb9c0159f0 con 0x564871ba5120 2023-11-28T11:08:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/937726337 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564871a9a140 con 0x564871ba5120 2023-11-28T11:08:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbb837fe700 1 -- 172.21.15.28:0/937726337 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb9c015bb0 con 0x564871ba5120 2023-11-28T11:08:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbbb71d0b80 10 client.4428 fetch_fsmap learned FSMap version 7 2023-11-28T11:08:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.634+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/937726337 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x5648718fad90 con 0x564871ba5120 2023-11-28T11:08:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.635+0000 7fbb837fe700 1 -- 172.21.15.28:0/937726337 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564871a9a140 con 0x564871ba5120 2023-11-28T11:08:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.635+0000 7fbb837fe700 1 -- 172.21.15.28:0/937726337 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fbb9c015d10 con 0x564871ba5120 2023-11-28T11:08:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.635+0000 7fbbb71d0b80 10 client.4428 fetch_fsmap finished waiting for FSMap version 7 2023-11-28T11:08:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.635+0000 7fbbb71d0b80 10 client.4428 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:08:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.635+0000 7fbbb71d0b80 20 client.4428 populate_metadata read hostname 'smithi028' 2023-11-28T11:08:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.635+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/937726337 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x564871bb7d30 0x564871b8a330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.635+0000 7fbbb71d0b80 4 client.4428 mds_command: new command op to 4296 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:08:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.635+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/937726337 --> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564871950f10 con 0x564871bb7d30 2023-11-28T11:08:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.635+0000 7fbba8de3700 1 --2- 172.21.15.28:0/937726337 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x564871bb7d30 0x564871b8a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.635+0000 7fbb837fe700 10 client.4428 ms_handle_connect on v2:172.21.15.59:6833/3293053261 2023-11-28T11:08:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.635+0000 7fbba8de3700 1 --2- 172.21.15.28:0/937726337 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x564871bb7d30 0x564871b8a330 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be9029bf5b0be084 server_cookie=9a24aa1e51a931f9 in_seq=0 out_seq=0 2023-11-28T11:08:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.636+0000 7fbb837fe700 1 -- 172.21.15.28:0/937726337 <== mds.0 v2:172.21.15.59:6833/3293053261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x564871950f10 con 0x564871bb7d30 2023-11-28T11:08:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.636+0000 7fbb837fe700 10 client.4428 handle_command_reply: tid=0 2023-11-28T11:08:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.636+0000 7fbbb71d0b80 1 client.4428 shutdown 2023-11-28T11:08:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.636+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/937726337 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 msgr2=0x564871bc5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.636+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/937726337 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc5420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c000f80 tx=0x7fbb9c004b40 comp rx=0 tx=0).stop 2023-11-28T11:08:52.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.636+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/937726337 shutdown_connections 2023-11-28T11:08:52.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.636+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/937726337 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x564871bb7d30 0x564871b8a330 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.636+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/937726337 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc5420 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.636+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/937726337 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba2130 0x564871ba7de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.636+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/937726337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba4810 0x564871bb77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.636+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/937726337 >> 172.21.15.28:0/937726337 conn(0x564871b77d60 msgr2=0x564871b7a990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.636+0000 7fbb837fe700 0 client.4428 ms_handle_reset on v2:172.21.15.59:6833/3293053261 2023-11-28T11:08:52.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.637+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/937726337 shutdown_connections 2023-11-28T11:08:52.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.637+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/937726337 wait complete. 2023-11-28T11:08:52.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.637+0000 7fbbb71d0b80 20 client.4428 trim_cache size 0 max 16384 2023-11-28T11:08:52.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.654+0000 7fbbb71d0b80 1 Processor -- start 2023-11-28T11:08:52.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.654+0000 7fbbb71d0b80 1 -- start start 2023-11-28T11:08:52.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbbb71d0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba2130 0x564871ba1b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbbb71d0b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba4810 0x564871bb7b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbbb71d0b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc1880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbbb71d0b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x564871b79500 con 0x564871ba5120 2023-11-28T11:08:52.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbbb71d0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x564871bc60a0 con 0x564871ba2130 2023-11-28T11:08:52.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbbb71d0b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x564871b77980 con 0x564871ba4810 2023-11-28T11:08:52.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbba95e4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbba95e4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc1880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46534/0 (socket says 172.21.15.28:46534) 2023-11-28T11:08:52.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbba95e4700 1 -- 172.21.15.28:0/2671572720 learned_addr learned my addr 172.21.15.28:0/2671572720 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:52.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbba8de3700 1 --2- 172.21.15.28:0/2671572720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba2130 0x564871ba1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbba3fff700 1 --2- 172.21.15.28:0/2671572720 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba4810 0x564871bb7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbba95e4700 1 -- 172.21.15.28:0/2671572720 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba4810 msgr2=0x564871bb7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbba95e4700 1 --2- 172.21.15.28:0/2671572720 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba4810 0x564871bb7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbba95e4700 1 -- 172.21.15.28:0/2671572720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba2130 msgr2=0x564871ba1b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbba95e4700 1 --2- 172.21.15.28:0/2671572720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba2130 0x564871ba1b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.655+0000 7fbba95e4700 1 -- 172.21.15.28:0/2671572720 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564871a93140 con 0x564871ba5120 2023-11-28T11:08:52.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.656+0000 7fbba3fff700 1 --2- 172.21.15.28:0/2671572720 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba4810 0x564871bb7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:52.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.656+0000 7fbba95e4700 1 --2- 172.21.15.28:0/2671572720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc1880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0288e0 tx=0x7fbb9c020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46e6413b51f15b60 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.656+0000 7fbb8effd700 1 -- 172.21.15.28:0/2671572720 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0285e0 con 0x564871ba5120 2023-11-28T11:08:52.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.656+0000 7fbb8effd700 1 -- 172.21.15.28:0/2671572720 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb9c028960 con 0x564871ba5120 2023-11-28T11:08:52.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.656+0000 7fbb8effd700 1 -- 172.21.15.28:0/2671572720 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c028cd0 con 0x564871ba5120 2023-11-28T11:08:52.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.656+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/2671572720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 msgr2=0x564871bc1880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.656+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/2671572720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc1880 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0288e0 tx=0x7fbb9c020000 comp rx=0 tx=0).stop 2023-11-28T11:08:52.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.656+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/2671572720 shutdown_connections 2023-11-28T11:08:52.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.656+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/2671572720 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba5120 0x564871bc1880 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.656+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/2671572720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba2130 0x564871ba1b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.656+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/2671572720 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba4810 0x564871bb7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.656+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/2671572720 >> 172.21.15.28:0/2671572720 conn(0x564871b77d60 msgr2=0x564871bc5400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.656+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/2671572720 shutdown_connections 2023-11-28T11:08:52.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.656+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/2671572720 wait complete. 2023-11-28T11:08:52.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.657+0000 7fbbb71d0b80 1 -- start start 2023-11-28T11:08:52.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.657+0000 7fbbb71d0b80 1 Processor -- start 2023-11-28T11:08:52.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbbb71d0b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba2130 0x564871b7abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbbb71d0b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba4810 0x564871b892a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbbb71d0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba5120 0x564871bb53b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbbb71d0b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x564871b73d20 con 0x564871ba4810 2023-11-28T11:08:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbbb71d0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x564871b73450 con 0x564871ba5120 2023-11-28T11:08:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbbb71d0b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x564871c281f0 con 0x564871ba2130 2023-11-28T11:08:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbbb71d0b80 10 client.0 _ll_register_callbacks cb 0x564871972240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:08:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbba3fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba4810 0x564871b892a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbba3fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba4810 0x564871b892a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46538/0 (socket says 172.21.15.28:46538) 2023-11-28T11:08:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbba3fff700 1 -- 172.21.15.28:0/3283462677 learned_addr learned my addr 172.21.15.28:0/3283462677 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbba8de3700 1 --2- 172.21.15.28:0/3283462677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba2130 0x564871b7abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbba95e4700 1 --2- 172.21.15.28:0/3283462677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba5120 0x564871bb53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbba3fff700 1 -- 172.21.15.28:0/3283462677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba2130 msgr2=0x564871b7abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbba3fff700 1 --2- 172.21.15.28:0/3283462677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba2130 0x564871b7abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbba3fff700 1 -- 172.21.15.28:0/3283462677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba5120 msgr2=0x564871bb53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbba3fff700 1 --2- 172.21.15.28:0/3283462677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba5120 0x564871bb53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.658+0000 7fbba3fff700 1 -- 172.21.15.28:0/3283462677 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5648718fad90 con 0x564871ba4810 2023-11-28T11:08:52.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbba3fff700 1 --2- 172.21.15.28:0/3283462677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba4810 0x564871b892a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbb90006dd0 tx=0x7fbb900047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea0a54f4f079b006 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbb837fe700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:08:52.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbb837fe700 1 -- 172.21.15.28:0/3283462677 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb9001f040 con 0x564871ba4810 2023-11-28T11:08:52.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/3283462677 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564871b88520 con 0x564871ba4810 2023-11-28T11:08:52.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbb837fe700 1 -- 172.21.15.28:0/3283462677 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbb90010470 con 0x564871ba4810 2023-11-28T11:08:52.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbb837fe700 1 -- 172.21.15.28:0/3283462677 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=7) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb90003e90 con 0x564871ba4810 2023-11-28T11:08:52.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbbb71d0b80 10 client.4434 fetch_fsmap learned FSMap version 7 2023-11-28T11:08:52.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/3283462677 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=7,monmap=2+}) v3 -- 0x564871921200 con 0x564871ba4810 2023-11-28T11:08:52.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbb837fe700 1 -- 172.21.15.28:0/3283462677 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564871b88520 con 0x564871ba4810 2023-11-28T11:08:52.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbb837fe700 1 -- 172.21.15.28:0/3283462677 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 7) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fbb90010630 con 0x564871ba4810 2023-11-28T11:08:52.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbbb71d0b80 10 client.4434 fetch_fsmap finished waiting for FSMap version 7 2023-11-28T11:08:52.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbbb71d0b80 10 client.4434 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:08:52.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbbb71d0b80 20 client.4434 populate_metadata read hostname 'smithi028' 2023-11-28T11:08:52.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/3283462677 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x564871bb63f0 0x564871bb67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbbb71d0b80 4 client.4434 mds_command: new command op to 4296 tid=0[{"prefix": "status"}] 2023-11-28T11:08:52.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/3283462677 --> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564871a8a160 con 0x564871bb63f0 2023-11-28T11:08:52.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.659+0000 7fbba8de3700 1 --2- 172.21.15.28:0/3283462677 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x564871bb63f0 0x564871bb67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbba8de3700 1 --2- 172.21.15.28:0/3283462677 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x564871bb63f0 0x564871bb67c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa7fd953352d0712 server_cookie=5e29befb14cd4e63 in_seq=0 out_seq=0 2023-11-28T11:08:52.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbb837fe700 10 client.4434 ms_handle_connect on v2:172.21.15.59:6833/3293053261 2023-11-28T11:08:52.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbb837fe700 1 -- 172.21.15.28:0/3283462677 <== mds.0 v2:172.21.15.59:6833/3293053261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x564871a8a160 con 0x564871bb63f0 2023-11-28T11:08:52.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbb837fe700 10 client.4434 handle_command_reply: tid=0 2023-11-28T11:08:52.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbbb71d0b80 1 client.4434 shutdown 2023-11-28T11:08:52.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/3283462677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba4810 msgr2=0x564871b892a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/3283462677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba4810 0x564871b892a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fbb90006dd0 tx=0x7fbb900047c0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/3283462677 shutdown_connections 2023-11-28T11:08:52.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/3283462677 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x564871bb63f0 0x564871bb67c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/3283462677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564871ba5120 0x564871bb53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/3283462677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x564871ba2130 0x564871b7abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbb837fe700 0 client.4434 ms_handle_reset on v2:172.21.15.59:6833/3293053261 2023-11-28T11:08:52.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbbb71d0b80 1 --2- 172.21.15.28:0/3283462677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x564871ba4810 0x564871b892a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/3283462677 >> 172.21.15.28:0/3283462677 conn(0x564871b77d60 msgr2=0x564871bc5390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/3283462677 shutdown_connections 2023-11-28T11:08:52.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbbb71d0b80 1 -- 172.21.15.28:0/3283462677 wait complete. 2023-11-28T11:08:52.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.660+0000 7fbbb71d0b80 20 client.4434 trim_cache size 0 max 16384 2023-11-28T11:08:52.670 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:08:52.670 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "6e299adc-7a2a-4cf1-afa2-7ce104b29a51", 2023-11-28T11:08:52.670 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2023-11-28T11:08:52.670 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4296, 2023-11-28T11:08:52.670 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2023-11-28T11:08:52.670 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2023-11-28T11:08:52.670 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2023-11-28T11:08:52.671 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 6.7348300249999999, 2023-11-28T11:08:52.671 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 7, 2023-11-28T11:08:52.671 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 25, 2023-11-28T11:08:52.671 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 24, 2023-11-28T11:08:52.671 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 12.152545129 2023-11-28T11:08:52.671 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:08:52.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.661+0000 7fbbab047700 1 -- 172.21.15.28:0/553664917 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbb880682f0 msgr2=0x7fbb8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.661+0000 7fbbab047700 1 --2- 172.21.15.28:0/553664917 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbb880682f0 0x7fbb8806a7a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fbb9000b2c0 tx=0x7fbb9001c040 comp rx=0 tx=0).stop 2023-11-28T11:08:52.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.661+0000 7fbbab047700 1 -- 172.21.15.28:0/553664917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbba406bb80 msgr2=0x7fbba40b2380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.661+0000 7fbbab047700 1 --2- 172.21.15.28:0/553664917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbba406bb80 0x7fbba40b2380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb9800c750 tx=0x7fbb9800ca60 comp rx=0 tx=0).stop 2023-11-28T11:08:52.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.661+0000 7fbbab047700 1 -- 172.21.15.28:0/553664917 shutdown_connections 2023-11-28T11:08:52.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.661+0000 7fbbab047700 1 --2- 172.21.15.28:0/553664917 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbb880682f0 0x7fbb8806a7a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.661+0000 7fbbab047700 1 --2- 172.21.15.28:0/553664917 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbba406bb80 0x7fbba40b2380 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.661+0000 7fbbab047700 1 --2- 172.21.15.28:0/553664917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbba40cb470 0x7fbba41ed3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.661+0000 7fbbab047700 1 --2- 172.21.15.28:0/553664917 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbba40cb0a0 0x7fbba41ece60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.661+0000 7fbbab047700 1 -- 172.21.15.28:0/553664917 >> 172.21.15.28:0/553664917 conn(0x7fbba40b0dd0 msgr2=0x7fbba40b0650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.661+0000 7fbbab047700 1 -- 172.21.15.28:0/553664917 shutdown_connections 2023-11-28T11:08:52.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.661+0000 7fbbab047700 1 -- 172.21.15.28:0/553664917 wait complete. 2023-11-28T11:08:52.674 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:08:52.674 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:08:52.675 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:08:52.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.797+0000 7fad1d72e700 1 Processor -- start 2023-11-28T11:08:52.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.797+0000 7fad1d72e700 1 -- start start 2023-11-28T11:08:52.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.798+0000 7fad1d72e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fad18161090 0x7fad180ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.798+0000 7fad1d72e700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad180ce340 con 0x7fad180caeb0 2023-11-28T11:08:52.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.798+0000 7fad1d72e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad180cebe0 con 0x7fad1815c990 2023-11-28T11:08:52.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.798+0000 7fad1d72e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad180cf780 con 0x7fad18161090 2023-11-28T11:08:52.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.798+0000 7fad177fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fad180caeb0 0x7fad180cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34606/0 (socket says 172.21.15.28:34606) 2023-11-28T11:08:52.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.798+0000 7fad177fe700 1 -- 172.21.15.28:0/3237275186 learned_addr learned my addr 172.21.15.28:0/3237275186 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:52.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.798+0000 7fad167fc700 1 --2- 172.21.15.28:0/3237275186 >> v2:172.21.15.59:3300/0 conn(0x7fad18161090 0x7fad180ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.798+0000 7fad15ffb700 1 -- 172.21.15.28:0/3237275186 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689966811 0 0) 0x7fad180ce340 con 0x7fad180caeb0 2023-11-28T11:08:52.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad15ffb700 1 -- 172.21.15.28:0/3237275186 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad000035b0 con 0x7fad180caeb0 2023-11-28T11:08:52.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad15ffb700 1 -- 172.21.15.28:0/3237275186 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 822814358 0 0) 0x7fad000035b0 con 0x7fad180caeb0 2023-11-28T11:08:52.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad15ffb700 1 -- 172.21.15.28:0/3237275186 >> v2:172.21.15.59:3300/0 conn(0x7fad18161090 msgr2=0x7fad180ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad15ffb700 1 --2- 172.21.15.28:0/3237275186 >> v2:172.21.15.59:3300/0 conn(0x7fad18161090 0x7fad180ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad15ffb700 1 -- 172.21.15.28:0/3237275186 >> v1:172.21.15.59:6789/0 conn(0x7fad1815c990 legacy=0x7fad180cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad15ffb700 1 -- 172.21.15.28:0/3237275186 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad180cb3f0 con 0x7fad180caeb0 2023-11-28T11:08:52.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad15ffb700 1 -- 172.21.15.28:0/3237275186 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fad100019f0 con 0x7fad180caeb0 2023-11-28T11:08:52.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad15ffb700 1 -- 172.21.15.28:0/3237275186 >> v1:172.21.15.28:6789/0 conn(0x7fad180caeb0 legacy=0x7fad180cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad15ffb700 1 --2- 172.21.15.28:0/3237275186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad00003a00 0x7fad00003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad15ffb700 1 --2- 172.21.15.28:0/3237275186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad00004850 0x7fad00004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad15ffb700 1 --2- 172.21.15.28:0/3237275186 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad00005200 0x7fad0000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad15ffb700 1 -- 172.21.15.28:0/3237275186 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad180cf780 con 0x7fad00004850 2023-11-28T11:08:52.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad15ffb700 1 -- 172.21.15.28:0/3237275186 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0000c190 con 0x7fad00003a00 2023-11-28T11:08:52.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad15ffb700 1 -- 172.21.15.28:0/3237275186 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad0000c350 con 0x7fad00005200 2023-11-28T11:08:52.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad167fc700 1 --2- 172.21.15.28:0/3237275186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad00004850 0x7fad00004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad16ffd700 1 --2- 172.21.15.28:0/3237275186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad00003a00 0x7fad00003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad177fe700 1 --2- 172.21.15.28:0/3237275186 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad00005200 0x7fad0000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad167fc700 1 -- 172.21.15.28:0/3237275186 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad00005200 msgr2=0x7fad0000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad167fc700 1 --2- 172.21.15.28:0/3237275186 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad00005200 0x7fad0000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad167fc700 1 -- 172.21.15.28:0/3237275186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad00003a00 msgr2=0x7fad00003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:52.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad167fc700 1 --2- 172.21.15.28:0/3237275186 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad00003a00 0x7fad00003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad177fe700 1 -- 172.21.15.28:0/3237275186 reap_dead start 2023-11-28T11:08:52.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad167fc700 1 -- 172.21.15.28:0/3237275186 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad000035b0 con 0x7fad00004850 2023-11-28T11:08:52.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.799+0000 7fad167fc700 1 --2- 172.21.15.28:0/3237275186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad00004850 0x7fad00004cc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fad08009b30 tx=0x7fad00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21ba35b609504e81 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.800+0000 7fad15ffb700 1 -- 172.21.15.28:0/3237275186 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0800e6c0 con 0x7fad00004850 2023-11-28T11:08:52.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.800+0000 7fad15ffb700 1 -- 172.21.15.28:0/3237275186 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad0800e820 con 0x7fad00004850 2023-11-28T11:08:52.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.800+0000 7fad15ffb700 1 -- 172.21.15.28:0/3237275186 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0800ebb0 con 0x7fad00004850 2023-11-28T11:08:52.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.800+0000 7fad1d72e700 1 -- 172.21.15.28:0/3237275186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad00004850 msgr2=0x7fad00004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.800+0000 7fad1d72e700 1 --2- 172.21.15.28:0/3237275186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad00004850 0x7fad00004cc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fad08009b30 tx=0x7fad00005880 comp rx=0 tx=0).stop 2023-11-28T11:08:52.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.800+0000 7fad1d72e700 1 -- 172.21.15.28:0/3237275186 shutdown_connections 2023-11-28T11:08:52.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.800+0000 7fad1d72e700 1 --2- 172.21.15.28:0/3237275186 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad00004850 0x7fad00004cc0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.800+0000 7fad1d72e700 1 -- 172.21.15.28:0/3237275186 >> 172.21.15.28:0/3237275186 conn(0x7fad180b1650 msgr2=0x7fad180b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:52.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.800+0000 7fad1d72e700 1 -- 172.21.15.28:0/3237275186 shutdown_connections 2023-11-28T11:08:52.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.800+0000 7fad1d72e700 1 -- 172.21.15.28:0/3237275186 wait complete. 2023-11-28T11:08:52.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.802+0000 7fad1d72e700 1 Processor -- start 2023-11-28T11:08:52.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.802+0000 7fad1d72e700 1 -- start start 2023-11-28T11:08:52.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.802+0000 7fad1d72e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad00004850 0x7fad1815a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.802+0000 7fad1d72e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad180c3470 0x7fad180b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.802+0000 7fad1d72e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad180b3490 0x7fad180b3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.802+0000 7fad1d72e700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad180b57c0 con 0x7fad180c3470 2023-11-28T11:08:52.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.802+0000 7fad1d72e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad180b5190 con 0x7fad180b3490 2023-11-28T11:08:52.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.802+0000 7fad1d72e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad180b3040 con 0x7fad00004850 2023-11-28T11:08:52.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.802+0000 7fad167fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad180c3470 0x7fad180b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.802+0000 7fad167fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad180c3470 0x7fad180b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46558/0 (socket says 172.21.15.28:46558) 2023-11-28T11:08:52.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.802+0000 7fad167fc700 1 -- 172.21.15.28:0/3521567341 learned_addr learned my addr 172.21.15.28:0/3521567341 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:52.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.803+0000 7fad16ffd700 1 --2- 172.21.15.28:0/3521567341 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad00004850 0x7fad1815a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.803+0000 7fad177fe700 1 --2- 172.21.15.28:0/3521567341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad180b3490 0x7fad180b3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.803+0000 7fad167fc700 1 -- 172.21.15.28:0/3521567341 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad00004850 msgr2=0x7fad1815a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.803+0000 7fad167fc700 1 --2- 172.21.15.28:0/3521567341 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad00004850 0x7fad1815a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.803+0000 7fad167fc700 1 -- 172.21.15.28:0/3521567341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad180b3490 msgr2=0x7fad180b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:52.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.803+0000 7fad167fc700 1 --2- 172.21.15.28:0/3521567341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad180b3490 0x7fad180b3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:52.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.803+0000 7fad167fc700 1 -- 172.21.15.28:0/3521567341 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad180ce870 con 0x7fad180c3470 2023-11-28T11:08:52.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.803+0000 7fad167fc700 1 --2- 172.21.15.28:0/3521567341 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad180c3470 0x7fad180b2b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fad080090b0 tx=0x7fad080047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.803+0000 7facff7fe700 1 -- 172.21.15.28:0/3521567341 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad0800e6c0 con 0x7fad180c3470 2023-11-28T11:08:52.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.803+0000 7facff7fe700 1 -- 172.21.15.28:0/3521567341 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad080038a0 con 0x7fad180c3470 2023-11-28T11:08:52.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.803+0000 7facff7fe700 1 -- 172.21.15.28:0/3521567341 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad08003c50 con 0x7fad180c3470 2023-11-28T11:08:52.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.803+0000 7fad1d72e700 1 -- 172.21.15.28:0/3521567341 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad180ce590 con 0x7fad180c3470 2023-11-28T11:08:52.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.803+0000 7fad1d72e700 1 -- 172.21.15.28:0/3521567341 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad180ca970 con 0x7fad180c3470 2023-11-28T11:08:52.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.804+0000 7fad1d72e700 1 -- 172.21.15.28:0/3521567341 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad18067150 con 0x7fad180c3470 2023-11-28T11:08:52.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.804+0000 7facff7fe700 1 -- 172.21.15.28:0/3521567341 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fad08019030 con 0x7fad180c3470 2023-11-28T11:08:52.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.805+0000 7facff7fe700 1 --2- 172.21.15.28:0/3521567341 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fad040682f0 0x7fad0406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:52.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.805+0000 7facff7fe700 1 -- 172.21.15.28:0/3521567341 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fad080875c0 con 0x7fad180c3470 2023-11-28T11:08:52.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.805+0000 7fad16ffd700 1 --2- 172.21.15.28:0/3521567341 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fad040682f0 0x7fad0406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:52.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.805+0000 7fad16ffd700 1 --2- 172.21.15.28:0/3521567341 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fad040682f0 0x7fad0406a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad1806d120 tx=0x7fad0c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:52.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:52.809+0000 7facff7fe700 1 -- 172.21.15.28:0/3521567341 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fad08053b30 con 0x7fad180c3470 2023-11-28T11:08:53.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:53.050+0000 7fad1d72e700 1 -- 172.21.15.28:0/3521567341 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fad180675a0 con 0x7fad180c3470 2023-11-28T11:08:53.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:53.050+0000 7facff7fe700 1 -- 172.21.15.28:0/3521567341 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v7) v1 ==== 53+0+88 (secure 0 0 0) 0x7fad08057150 con 0x7fad180c3470 2023-11-28T11:08:53.052 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T11:08:53.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:53.052+0000 7fad1d72e700 1 -- 172.21.15.28:0/3521567341 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fad040682f0 msgr2=0x7fad0406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:53.052+0000 7fad1d72e700 1 --2- 172.21.15.28:0/3521567341 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fad040682f0 0x7fad0406a7a0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fad1806d120 tx=0x7fad0c009040 comp rx=0 tx=0).stop 2023-11-28T11:08:53.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:53.052+0000 7fad1d72e700 1 -- 172.21.15.28:0/3521567341 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad180c3470 msgr2=0x7fad180b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:53.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:53.052+0000 7fad1d72e700 1 --2- 172.21.15.28:0/3521567341 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad180c3470 0x7fad180b2b00 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fad080090b0 tx=0x7fad080047c0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:53.052+0000 7fad1d72e700 1 -- 172.21.15.28:0/3521567341 shutdown_connections 2023-11-28T11:08:53.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:53.052+0000 7fad1d72e700 1 --2- 172.21.15.28:0/3521567341 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fad040682f0 0x7fad0406a7a0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:53.052+0000 7fad1d72e700 1 --2- 172.21.15.28:0/3521567341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fad180b3490 0x7fad180b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:53.052+0000 7fad1d72e700 1 --2- 172.21.15.28:0/3521567341 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fad00004850 0x7fad1815a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:53.052+0000 7fad1d72e700 1 --2- 172.21.15.28:0/3521567341 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fad180c3470 0x7fad180b2b00 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:53.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:53.052+0000 7fad1d72e700 1 -- 172.21.15.28:0/3521567341 >> 172.21.15.28:0/3521567341 conn(0x7fad180b1650 msgr2=0x7fad1806d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:53.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:53.052+0000 7fad1d72e700 1 -- 172.21.15.28:0/3521567341 shutdown_connections 2023-11-28T11:08:53.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:53.052+0000 7fad1d72e700 1 -- 172.21.15.28:0/3521567341 wait complete. 2023-11-28T11:08:53.066 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T11:08:53.066 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:08:53.066 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2023-11-28T11:08:53.067 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-11-28T11:08:53.067 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:08:53.067 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T11:08:53.067 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:08:53.067 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:08:53.067 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-11-28T11:08:53.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:53.068 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2023-11-28T11:08:53.085 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:08:53.085 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:08:53.085 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:08:53.086 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:53.086 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2023-11-28T11:08:53.086 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:53.086 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:53.086 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:53.086 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:53.086 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:53.086 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:08:53.086 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:53.087 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T11:08:53.087 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42325sec preferred_lft 42325sec 2023-11-28T11:08:53.087 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2023-11-28T11:08:53.087 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:08:53.087 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:08:53.087 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2023-11-28T11:08:53.088 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T11:08:53.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:53.088 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:08:53.088 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:08:53.089 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add name ceph-brx type bridge 2023-11-28T11:08:53.089 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip addr flush dev ceph-brx 2023-11-28T11:08:53.089 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set ceph-brx up 2023-11-28T11:08:53.089 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T11:08:53.089 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:08:53.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:53.258 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:08:53.287 INFO:teuthology.orchestra.run.smithi028.stdout:1 2023-11-28T11:08:53.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:53.288 DEBUG:teuthology.orchestra.run.smithi028:> route 2023-11-28T11:08:53.362 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2023-11-28T11:08:53.362 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:08:53.362 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T11:08:53.362 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T11:08:53.362 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:08:53.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:53.363 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:08:53.363 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:08:53.363 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T11:08:53.363 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T11:08:53.363 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T11:08:53.363 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:08:53.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:53.636 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2023-11-28T11:08:53.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:53.653 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list-id 2023-11-28T11:08:53.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:53.711 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:08:53.711 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:08:53.711 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T11:08:53.711 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-11-28T11:08:53.711 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:08:53.806 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-11-28T11:08:53.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:53.807 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:08:53.807 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:08:53.807 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-11-28T11:08:53.807 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T11:08:53.807 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-11-28T11:08:53.807 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-11-28T11:08:53.807 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-11-28T11:08:53.807 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:08:54.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:54.021 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:08:54.021 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:08:54.021 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2023-11-28T11:08:54.021 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T11:08:54.021 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:08:54.127 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-11-28T11:08:54.127 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:08:54.127 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:08:54.149 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-11-28T11:08:54.149 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:08:54.149 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:08:54.206 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2023-11-28T11:08:54.274 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:08:54.274 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:08:54.341 INFO:teuthology.orchestra.run.smithi028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:08:54.342 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:08:54.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:54.343 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2023-11-28T11:08:54.400 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T11:08:54.400 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_mountpoint=/ --client_fs=cephfs) 2023-11-28T11:08:54.442 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2023-11-28T11:08:54.470 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:08:54.470 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:08:54.536 INFO:teuthology.orchestra.run.smithi028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:08:54.537 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:08:54.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:54.538 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2023-11-28T11:08:54.570 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi028.stderr:2023-11-28T11:08:54.568+0000 7fa95c7a15c0 -1 init, newargv = 0x556a09ab5300 newargc=15 2023-11-28T11:08:54.570 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi028.stderr:ceph-fuse[110727]: starting ceph client 2023-11-28T11:08:54.576 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi028.stderr:ceph-fuse[110727]: starting fuse 2023-11-28T11:08:54.586 INFO:teuthology.orchestra.run.smithi028.stdout:47 2023-11-28T11:08:54.587 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T11:08:54.587 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:08:54.587 DEBUG:teuthology.orchestra.run.smithi028:> import glob 2023-11-28T11:08:54.587 DEBUG:teuthology.orchestra.run.smithi028:> import re 2023-11-28T11:08:54.587 DEBUG:teuthology.orchestra.run.smithi028:> import os 2023-11-28T11:08:54.587 DEBUG:teuthology.orchestra.run.smithi028:> import subprocess 2023-11-28T11:08:54.587 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:08:54.587 DEBUG:teuthology.orchestra.run.smithi028:> def _find_admin_socket(client_name): 2023-11-28T11:08:54.588 DEBUG:teuthology.orchestra.run.smithi028:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-11-28T11:08:54.588 DEBUG:teuthology.orchestra.run.smithi028:> files = glob.glob(asok_path) 2023-11-28T11:08:54.588 DEBUG:teuthology.orchestra.run.smithi028:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-11-28T11:08:54.588 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:08:54.588 DEBUG:teuthology.orchestra.run.smithi028:> # Given a non-glob path, it better be there 2023-11-28T11:08:54.588 DEBUG:teuthology.orchestra.run.smithi028:> if "*" not in asok_path: 2023-11-28T11:08:54.588 DEBUG:teuthology.orchestra.run.smithi028:> assert(len(files) == 1) 2023-11-28T11:08:54.588 DEBUG:teuthology.orchestra.run.smithi028:> return files[0] 2023-11-28T11:08:54.588 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:08:54.588 DEBUG:teuthology.orchestra.run.smithi028:> for f in files: 2023-11-28T11:08:54.588 DEBUG:teuthology.orchestra.run.smithi028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T11:08:54.588 DEBUG:teuthology.orchestra.run.smithi028:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T11:08:54.588 DEBUG:teuthology.orchestra.run.smithi028:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T11:08:54.589 DEBUG:teuthology.orchestra.run.smithi028:> contents = proc_f.read() 2023-11-28T11:08:54.589 DEBUG:teuthology.orchestra.run.smithi028:> if mountpoint in contents: 2023-11-28T11:08:54.589 DEBUG:teuthology.orchestra.run.smithi028:> return f 2023-11-28T11:08:54.589 DEBUG:teuthology.orchestra.run.smithi028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T11:08:54.589 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:08:54.589 DEBUG:teuthology.orchestra.run.smithi028:> print(_find_admin_socket("client.admin")) 2023-11-28T11:08:54.589 DEBUG:teuthology.orchestra.run.smithi028:> ' 2023-11-28T11:08:54.712 INFO:teuthology.orchestra.run.smithi028.stdout:/var/run/ceph/ceph-client.admin.110727.asok 2023-11-28T11:08:54.894 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.110727.asok 2023-11-28T11:08:54.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:54.895 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.110727.asok status 2023-11-28T11:08:55.056 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:08:55.057 INFO:teuthology.orchestra.run.smithi028.stdout: "metadata": { 2023-11-28T11:08:55.057 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T11:08:55.057 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T11:08:55.057 INFO:teuthology.orchestra.run.smithi028.stdout: "entity_id": "admin", 2023-11-28T11:08:55.057 INFO:teuthology.orchestra.run.smithi028.stdout: "hostname": "smithi028", 2023-11-28T11:08:55.057 INFO:teuthology.orchestra.run.smithi028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-11-28T11:08:55.057 INFO:teuthology.orchestra.run.smithi028.stdout: "pid": "110727", 2023-11-28T11:08:55.057 INFO:teuthology.orchestra.run.smithi028.stdout: "root": "/" 2023-11-28T11:08:55.057 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:08:55.057 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_count": 0, 2023-11-28T11:08:55.057 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_pinned_count": 0, 2023-11-28T11:08:55.058 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4446, 2023-11-28T11:08:55.058 INFO:teuthology.orchestra.run.smithi028.stdout: "inst": { 2023-11-28T11:08:55.058 INFO:teuthology.orchestra.run.smithi028.stdout: "name": { 2023-11-28T11:08:55.058 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "client", 2023-11-28T11:08:55.058 INFO:teuthology.orchestra.run.smithi028.stdout: "num": 4446 2023-11-28T11:08:55.058 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:08:55.058 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2023-11-28T11:08:55.058 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2023-11-28T11:08:55.058 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2023-11-28T11:08:55.058 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1721483265 2023-11-28T11:08:55.058 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1721483265 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi028.stdout: "inst_str": "client.4446 192.168.0.1:0/1721483265", 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi028.stdout: "addr_str": "192.168.0.1:0/1721483265", 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi028.stdout: "inode_count": 1, 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_epoch": 7, 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch": 25, 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch_barrier": 0, 2023-11-28T11:08:55.059 INFO:teuthology.orchestra.run.smithi028.stdout: "blocklisted": false, 2023-11-28T11:08:55.060 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs" 2023-11-28T11:08:55.060 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:08:55.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:55.066 DEBUG:teuthology.orchestra.run.smithi028:> stat --file-system '--printf=%T 2023-11-28T11:08:55.066 DEBUG:teuthology.orchestra.run.smithi028:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:08:55.083 INFO:teuthology.orchestra.run.smithi028.stdout:fuseblk 2023-11-28T11:08:55.084 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:08:55.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:55.084 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:08:55.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:08:55.156 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-11-28T11:08:55.262 INFO:teuthology.orchestra.run.smithi028.stdout:# file: . 2023-11-28T11:08:55.262 INFO:teuthology.orchestra.run.smithi028.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-11-28T11:08:55.262 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:08:55.263 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:08:55.263 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:55.283 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:55.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.447+0000 7f8a3d51d700 1 Processor -- start 2023-11-28T11:08:55.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.447+0000 7f8a3d51d700 1 -- start start 2023-11-28T11:08:55.450 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.447+0000 7f8a3d51d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a3815df40 0x7f8a380adf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.447+0000 7f8a3d51d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8a38071070 0x7f8a380cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.447+0000 7f8a3d51d700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a3815a7e0 con 0x7f8a3815a410 2023-11-28T11:08:55.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.447+0000 7f8a3d51d700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a3815e3c0 con 0x7f8a38071070 2023-11-28T11:08:55.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.447+0000 7f8a3d51d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a3815e6a0 con 0x7f8a3815df40 2023-11-28T11:08:55.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.448+0000 7f8a377fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8a38071070 0x7f8a380cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.448+0000 7f8a36ffd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8a3815a410 0x7f8a38070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34636/0 (socket says 172.21.15.28:34636) 2023-11-28T11:08:55.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.448+0000 7f8a36ffd700 1 -- 172.21.15.28:0/3273188863 learned_addr learned my addr 172.21.15.28:0/3273188863 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:55.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.448+0000 7f8a367fc700 1 --2- 172.21.15.28:0/3273188863 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a3815df40 0x7f8a380adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.448+0000 7f8a377fe700 1 -- 172.21.15.28:0/3273188863 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a3815df40 msgr2=0x7f8a380adf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:55.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.448+0000 7f8a377fe700 1 --2- 172.21.15.28:0/3273188863 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a3815df40 0x7f8a380adf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.448+0000 7f8a377fe700 1 -- 172.21.15.28:0/3273188863 >> v1:172.21.15.28:6789/0 conn(0x7f8a3815a410 legacy=0x7f8a38070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.448+0000 7f8a377fe700 1 -- 172.21.15.28:0/3273188863 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3815e980 con 0x7f8a38071070 2023-11-28T11:08:55.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.448+0000 7f8a35ffb700 1 -- 172.21.15.28:0/3273188863 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 104479840 0 0) 0x7f8a3815a7e0 con 0x7f8a3815a410 2023-11-28T11:08:55.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.448+0000 7f8a377fe700 1 --2- 172.21.15.28:0/3273188863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a38071070 0x7f8a380cc990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009d90 tx=0x7f8a3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3fbe509ed2f19e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.448+0000 7f8a35ffb700 1 -- 172.21.15.28:0/3273188863 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3000e070 con 0x7f8a38071070 2023-11-28T11:08:55.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.448+0000 7f8a35ffb700 1 -- 172.21.15.28:0/3273188863 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3000bbd0 con 0x7f8a38071070 2023-11-28T11:08:55.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.448+0000 7f8a35ffb700 1 -- 172.21.15.28:0/3273188863 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a30012430 con 0x7f8a38071070 2023-11-28T11:08:55.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.449+0000 7f8a3d51d700 1 -- 172.21.15.28:0/3273188863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a38071070 msgr2=0x7f8a380cc990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.449+0000 7f8a3d51d700 1 --2- 172.21.15.28:0/3273188863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a38071070 0x7f8a380cc990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f8a30009d90 tx=0x7f8a3000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:55.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.449+0000 7f8a3d51d700 1 -- 172.21.15.28:0/3273188863 shutdown_connections 2023-11-28T11:08:55.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.449+0000 7f8a3d51d700 1 --2- 172.21.15.28:0/3273188863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a38071070 0x7f8a380cc990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.449+0000 7f8a3d51d700 1 --2- 172.21.15.28:0/3273188863 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a3815df40 0x7f8a380adf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.449+0000 7f8a3d51d700 1 -- 172.21.15.28:0/3273188863 >> 172.21.15.28:0/3273188863 conn(0x7f8a380ce9e0 msgr2=0x7f8a380cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.449+0000 7f8a3d51d700 1 -- 172.21.15.28:0/3273188863 shutdown_connections 2023-11-28T11:08:55.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.449+0000 7f8a3d51d700 1 -- 172.21.15.28:0/3273188863 wait complete. 2023-11-28T11:08:55.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.451+0000 7f8a3d51d700 1 Processor -- start 2023-11-28T11:08:55.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.451+0000 7f8a3d51d700 1 -- start start 2023-11-28T11:08:55.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a3d51d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a38071070 0x7f8a38155600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a3d51d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a3815a410 0x7f8a380c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a3d51d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a380c7990 0x7f8a380c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a3d51d700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a381e5a60 con 0x7f8a38071070 2023-11-28T11:08:55.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a3d51d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a38156110 con 0x7f8a380c7990 2023-11-28T11:08:55.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a3d51d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a381e4de0 con 0x7f8a3815a410 2023-11-28T11:08:55.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a36ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a38071070 0x7f8a38155600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a36ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a38071070 0x7f8a38155600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46584/0 (socket says 172.21.15.28:46584) 2023-11-28T11:08:55.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a36ffd700 1 -- 172.21.15.28:0/259095903 learned_addr learned my addr 172.21.15.28:0/259095903 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:55.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a367fc700 1 --2- 172.21.15.28:0/259095903 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a3815a410 0x7f8a380c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a377fe700 1 --2- 172.21.15.28:0/259095903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a380c7990 0x7f8a380c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a36ffd700 1 -- 172.21.15.28:0/259095903 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a3815a410 msgr2=0x7f8a380c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a36ffd700 1 --2- 172.21.15.28:0/259095903 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a3815a410 0x7f8a380c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a36ffd700 1 -- 172.21.15.28:0/259095903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a380c7990 msgr2=0x7f8a380c7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a36ffd700 1 --2- 172.21.15.28:0/259095903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a380c7990 0x7f8a380c7e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a36ffd700 1 -- 172.21.15.28:0/259095903 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a30009a70 con 0x7f8a38071070 2023-11-28T11:08:55.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a36ffd700 1 --2- 172.21.15.28:0/259095903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a38071070 0x7f8a38155600 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8a280011e0 tx=0x7f8a2800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a1f7fe700 1 -- 172.21.15.28:0/259095903 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28004c80 con 0x7f8a38071070 2023-11-28T11:08:55.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a1f7fe700 1 -- 172.21.15.28:0/259095903 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a28004de0 con 0x7f8a38071070 2023-11-28T11:08:55.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a3d51d700 1 -- 172.21.15.28:0/259095903 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a380adbd0 con 0x7f8a38071070 2023-11-28T11:08:55.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.452+0000 7f8a1f7fe700 1 -- 172.21.15.28:0/259095903 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a28007310 con 0x7f8a38071070 2023-11-28T11:08:55.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.453+0000 7f8a3d51d700 1 -- 172.21.15.28:0/259095903 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a3815ec60 con 0x7f8a38071070 2023-11-28T11:08:55.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.453+0000 7f8a3d51d700 1 -- 172.21.15.28:0/259095903 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a380add60 con 0x7f8a38071070 2023-11-28T11:08:55.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.454+0000 7f8a1f7fe700 1 -- 172.21.15.28:0/259095903 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8a2801a020 con 0x7f8a38071070 2023-11-28T11:08:55.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.454+0000 7f8a1f7fe700 1 --2- 172.21.15.28:0/259095903 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8a240682f0 0x7f8a2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.454+0000 7f8a1f7fe700 1 -- 172.21.15.28:0/259095903 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8a28085b80 con 0x7f8a38071070 2023-11-28T11:08:55.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.454+0000 7f8a367fc700 1 --2- 172.21.15.28:0/259095903 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8a240682f0 0x7f8a2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.455+0000 7f8a367fc700 1 --2- 172.21.15.28:0/259095903 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8a240682f0 0x7f8a2406a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8a380b4290 tx=0x7f8a2c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.459+0000 7f8a1f7fe700 1 -- 172.21.15.28:0/259095903 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8a280520f0 con 0x7f8a38071070 2023-11-28T11:08:55.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.653+0000 7f8a3d51d700 1 -- 172.21.15.28:0/259095903 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a38068570 con 0x7f8a38071070 2023-11-28T11:08:55.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.655+0000 7f8a1f7fe700 1 -- 172.21.15.28:0/259095903 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f8a28055710 con 0x7f8a38071070 2023-11-28T11:08:55.656 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:08:55.656 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:50.531733+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.656+0000 7f8a3d51d700 1 -- 172.21.15.28:0/259095903 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8a240682f0 msgr2=0x7f8a2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.656+0000 7f8a3d51d700 1 --2- 172.21.15.28:0/259095903 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8a240682f0 0x7f8a2406a7a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8a380b4290 tx=0x7f8a2c008040 comp rx=0 tx=0).stop 2023-11-28T11:08:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.656+0000 7f8a3d51d700 1 -- 172.21.15.28:0/259095903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a38071070 msgr2=0x7f8a38155600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.656+0000 7f8a3d51d700 1 --2- 172.21.15.28:0/259095903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a38071070 0x7f8a38155600 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8a280011e0 tx=0x7f8a2800ca30 comp rx=0 tx=0).stop 2023-11-28T11:08:55.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.656+0000 7f8a3d51d700 1 -- 172.21.15.28:0/259095903 shutdown_connections 2023-11-28T11:08:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.656+0000 7f8a3d51d700 1 --2- 172.21.15.28:0/259095903 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8a240682f0 0x7f8a2406a7a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.656+0000 7f8a3d51d700 1 --2- 172.21.15.28:0/259095903 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a38071070 0x7f8a38155600 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.656+0000 7f8a3d51d700 1 --2- 172.21.15.28:0/259095903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a380c7990 0x7f8a380c7e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.656+0000 7f8a3d51d700 1 --2- 172.21.15.28:0/259095903 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a3815a410 0x7f8a380c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.656+0000 7f8a3d51d700 1 -- 172.21.15.28:0/259095903 >> 172.21.15.28:0/259095903 conn(0x7f8a380ce9e0 msgr2=0x7f8a380af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.657+0000 7f8a3d51d700 1 -- 172.21.15.28:0/259095903 shutdown_connections 2023-11-28T11:08:55.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.657+0000 7f8a3d51d700 1 -- 172.21.15.28:0/259095903 wait complete. 2023-11-28T11:08:55.671 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:08:55.672 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:55.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.798+0000 7fd6aacb9700 1 Processor -- start 2023-11-28T11:08:55.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.799+0000 7fd6aacb9700 1 -- start start 2023-11-28T11:08:55.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.799+0000 7fd6aacb9700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd6a40b1630 0x7fd6a40cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.799+0000 7fd6aacb9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd6a406afc0 0x7fd6a406b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.799+0000 7fd6aacb9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6a40ce550 con 0x7fd6a40b1a00 2023-11-28T11:08:55.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.799+0000 7fd6aacb9700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a40ce830 con 0x7fd6a40b1630 2023-11-28T11:08:55.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.799+0000 7fd6aacb9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6a40ceb10 con 0x7fd6a406afc0 2023-11-28T11:08:55.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.799+0000 7fd6a8a55700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd6a40b1630 0x7fd6a40cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.800+0000 7fd6a8a55700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd6a40b1630 0x7fd6a40cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46592/0 (socket says 172.21.15.28:46592) 2023-11-28T11:08:55.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.800+0000 7fd6a8a55700 1 -- 172.21.15.28:0/868514391 learned_addr learned my addr 172.21.15.28:0/868514391 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:55.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.800+0000 7fd6a9256700 1 --2- 172.21.15.28:0/868514391 >> v2:172.21.15.59:3300/0 conn(0x7fd6a406afc0 0x7fd6a406b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.800+0000 7fd6a8a55700 1 -- 172.21.15.28:0/868514391 >> v2:172.21.15.59:3300/0 conn(0x7fd6a406afc0 msgr2=0x7fd6a406b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:55.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.800+0000 7fd6a8a55700 1 --2- 172.21.15.28:0/868514391 >> v2:172.21.15.59:3300/0 conn(0x7fd6a406afc0 0x7fd6a406b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.800+0000 7fd6a8a55700 1 -- 172.21.15.28:0/868514391 >> v1:172.21.15.59:6789/0 conn(0x7fd6a40b1a00 legacy=0x7fd6a40caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.800+0000 7fd6a8a55700 1 -- 172.21.15.28:0/868514391 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6a40cedf0 con 0x7fd6a40b1630 2023-11-28T11:08:55.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.800+0000 7fd6a8a55700 1 --2- 172.21.15.28:0/868514391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6a40b1630 0x7fd6a40cf5f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd6980098f0 tx=0x7fd698009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed30e4c51536b561 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.800+0000 7fd6a37fe700 1 -- 172.21.15.28:0/868514391 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698011830 con 0x7fd6a40b1630 2023-11-28T11:08:55.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.800+0000 7fd6a37fe700 1 -- 172.21.15.28:0/868514391 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd698011990 con 0x7fd6a40b1630 2023-11-28T11:08:55.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.800+0000 7fd6a37fe700 1 -- 172.21.15.28:0/868514391 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd698011d20 con 0x7fd6a40b1630 2023-11-28T11:08:55.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.801+0000 7fd6aacb9700 1 -- 172.21.15.28:0/868514391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6a40b1630 msgr2=0x7fd6a40cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.801+0000 7fd6aacb9700 1 --2- 172.21.15.28:0/868514391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6a40b1630 0x7fd6a40cf5f0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fd6980098f0 tx=0x7fd698009c00 comp rx=0 tx=0).stop 2023-11-28T11:08:55.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.801+0000 7fd6aacb9700 1 -- 172.21.15.28:0/868514391 shutdown_connections 2023-11-28T11:08:55.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.801+0000 7fd6aacb9700 1 --2- 172.21.15.28:0/868514391 >> v2:172.21.15.59:3300/0 conn(0x7fd6a406afc0 0x7fd6a406b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.801+0000 7fd6aacb9700 1 --2- 172.21.15.28:0/868514391 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6a40b1630 0x7fd6a40cf5f0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.801+0000 7fd6aacb9700 1 -- 172.21.15.28:0/868514391 >> 172.21.15.28:0/868514391 conn(0x7fd6a40cc630 msgr2=0x7fd6a40cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:55.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.801+0000 7fd6aacb9700 1 -- 172.21.15.28:0/868514391 shutdown_connections 2023-11-28T11:08:55.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.801+0000 7fd6aacb9700 1 -- 172.21.15.28:0/868514391 wait complete. 2023-11-28T11:08:55.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.802+0000 7fd6aacb9700 1 Processor -- start 2023-11-28T11:08:55.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.802+0000 7fd6aacb9700 1 -- start start 2023-11-28T11:08:55.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6aacb9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6a406afc0 0x7fd6a40b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6aacb9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6a40b1630 0x7fd6a40b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6aacb9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6a40b1a00 0x7fd6a41f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6aacb9700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a41f39c0 con 0x7fd6a40b1a00 2023-11-28T11:08:55.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6aacb9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a40b51e0 con 0x7fd6a40b1630 2023-11-28T11:08:55.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6aacb9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6a41f3bd0 con 0x7fd6a406afc0 2023-11-28T11:08:55.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6a9256700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6a40b1a00 0x7fd6a41f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6a9256700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6a40b1a00 0x7fd6a41f2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46602/0 (socket says 172.21.15.28:46602) 2023-11-28T11:08:55.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6a9256700 1 -- 172.21.15.28:0/1045766008 learned_addr learned my addr 172.21.15.28:0/1045766008 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:55.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6a8a55700 1 --2- 172.21.15.28:0/1045766008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6a406afc0 0x7fd6a40b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6a3fff700 1 --2- 172.21.15.28:0/1045766008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6a40b1630 0x7fd6a40b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6a9256700 1 -- 172.21.15.28:0/1045766008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6a406afc0 msgr2=0x7fd6a40b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6a9256700 1 --2- 172.21.15.28:0/1045766008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6a406afc0 0x7fd6a40b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6a9256700 1 -- 172.21.15.28:0/1045766008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6a40b1630 msgr2=0x7fd6a40b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:55.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6a9256700 1 --2- 172.21.15.28:0/1045766008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6a40b1630 0x7fd6a40b49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:55.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6a9256700 1 -- 172.21.15.28:0/1045766008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd698009550 con 0x7fd6a40b1a00 2023-11-28T11:08:55.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.803+0000 7fd6a3fff700 1 --2- 172.21.15.28:0/1045766008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6a40b1630 0x7fd6a40b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:55.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.804+0000 7fd6a9256700 1 --2- 172.21.15.28:0/1045766008 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6a40b1a00 0x7fd6a41f2800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00efc0 tx=0x7fd69c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.804+0000 7fd6a17fa700 1 -- 172.21.15.28:0/1045766008 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c009ea0 con 0x7fd6a40b1a00 2023-11-28T11:08:55.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.804+0000 7fd6a17fa700 1 -- 172.21.15.28:0/1045766008 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd69c00f040 con 0x7fd6a40b1a00 2023-11-28T11:08:55.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.804+0000 7fd6a17fa700 1 -- 172.21.15.28:0/1045766008 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd69c014480 con 0x7fd6a40b1a00 2023-11-28T11:08:55.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.804+0000 7fd6aacb9700 1 -- 172.21.15.28:0/1045766008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6a40b1ea0 con 0x7fd6a40b1a00 2023-11-28T11:08:55.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.804+0000 7fd6aacb9700 1 -- 172.21.15.28:0/1045766008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6a40cf0d0 con 0x7fd6a40b1a00 2023-11-28T11:08:55.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.805+0000 7fd6aacb9700 1 -- 172.21.15.28:0/1045766008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6a40cb5b0 con 0x7fd6a40b1a00 2023-11-28T11:08:55.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.805+0000 7fd6a17fa700 1 -- 172.21.15.28:0/1045766008 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd69c0145e0 con 0x7fd6a40b1a00 2023-11-28T11:08:55.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.805+0000 7fd6a17fa700 1 --2- 172.21.15.28:0/1045766008 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd6880682f0 0x7fd68806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:55.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.806+0000 7fd6a17fa700 1 -- 172.21.15.28:0/1045766008 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd69c0869d0 con 0x7fd6a40b1a00 2023-11-28T11:08:55.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.806+0000 7fd6a8a55700 1 --2- 172.21.15.28:0/1045766008 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd6880682f0 0x7fd68806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:55.810 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.806+0000 7fd6a8a55700 1 --2- 172.21.15.28:0/1045766008 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd6880682f0 0x7fd68806a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd698009c40 tx=0x7fd698008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:55.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:55.810+0000 7fd6a17fa700 1 -- 172.21.15.28:0/1045766008 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd69c052f40 con 0x7fd6a40b1a00 2023-11-28T11:08:56.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.003+0000 7fd6aacb9700 1 -- 172.21.15.28:0/1045766008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd6a40739f0 con 0x7fd6a40b1a00 2023-11-28T11:08:56.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.003+0000 7fd6a17fa700 1 -- 172.21.15.28:0/1045766008 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd69c056560 con 0x7fd6a40b1a00 2023-11-28T11:08:56.005 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:08:56.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.005+0000 7fd6aacb9700 1 -- 172.21.15.28:0/1045766008 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd6880682f0 msgr2=0x7fd68806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.005+0000 7fd6aacb9700 1 --2- 172.21.15.28:0/1045766008 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd6880682f0 0x7fd68806a7a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fd698009c40 tx=0x7fd698008030 comp rx=0 tx=0).stop 2023-11-28T11:08:56.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.005+0000 7fd6aacb9700 1 -- 172.21.15.28:0/1045766008 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6a40b1a00 msgr2=0x7fd6a41f2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.005+0000 7fd6aacb9700 1 --2- 172.21.15.28:0/1045766008 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6a40b1a00 0x7fd6a41f2800 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fd69c00efc0 tx=0x7fd69c00c3a0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.005+0000 7fd6aacb9700 1 -- 172.21.15.28:0/1045766008 shutdown_connections 2023-11-28T11:08:56.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.005+0000 7fd6aacb9700 1 --2- 172.21.15.28:0/1045766008 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd6880682f0 0x7fd68806a7a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.005+0000 7fd6aacb9700 1 --2- 172.21.15.28:0/1045766008 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6a40b1a00 0x7fd6a41f2800 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.005+0000 7fd6aacb9700 1 --2- 172.21.15.28:0/1045766008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6a406afc0 0x7fd6a40b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.005+0000 7fd6aacb9700 1 --2- 172.21.15.28:0/1045766008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6a40b1630 0x7fd6a40b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.005+0000 7fd6aacb9700 1 -- 172.21.15.28:0/1045766008 >> 172.21.15.28:0/1045766008 conn(0x7fd6a40cc630 msgr2=0x7fd6a40ab950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.005+0000 7fd6aacb9700 1 -- 172.21.15.28:0/1045766008 shutdown_connections 2023-11-28T11:08:56.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.005+0000 7fd6aacb9700 1 -- 172.21.15.28:0/1045766008 wait complete. 2023-11-28T11:08:56.019 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:56.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.144+0000 7f1140a74700 1 Processor -- start 2023-11-28T11:08:56.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.144+0000 7f1140a74700 1 -- start start 2023-11-28T11:08:56.148 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.145+0000 7f1140a74700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f113c0b62d0 0x7f113c0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.148 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.145+0000 7f1140a74700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f113c06f8b0 0x7f113c0b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.145+0000 7f1140a74700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f113c0b3020 con 0x7f113c0b2bc0 2023-11-28T11:08:56.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.145+0000 7f1140a74700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f113c0cd2a0 con 0x7f113c06f8b0 2023-11-28T11:08:56.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.145+0000 7f1140a74700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113c0d0400 con 0x7f113c0b62d0 2023-11-28T11:08:56.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.145+0000 7f1139d9b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f113c06f8b0 0x7f113c0b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.145+0000 7f113ad9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f113c0b2bc0 0x7f113c0ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34668/0 (socket says 172.21.15.28:34668) 2023-11-28T11:08:56.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.145+0000 7f113ad9d700 1 -- 172.21.15.28:0/3174087103 learned_addr learned my addr 172.21.15.28:0/3174087103 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:56.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.146+0000 7f113a59c700 1 --2- 172.21.15.28:0/3174087103 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f113c0b62d0 0x7f113c0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.146+0000 7f1139d9b700 1 -- 172.21.15.28:0/3174087103 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f113c0b62d0 msgr2=0x7f113c0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.146+0000 7f1139d9b700 1 --2- 172.21.15.28:0/3174087103 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f113c0b62d0 0x7f113c0cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.146+0000 7f1139d9b700 1 -- 172.21.15.28:0/3174087103 >> v1:172.21.15.28:6789/0 conn(0x7f113c0b2bc0 legacy=0x7f113c0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.146+0000 7f1139d9b700 1 -- 172.21.15.28:0/3174087103 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f113c0d06e0 con 0x7f113c06f8b0 2023-11-28T11:08:56.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.146+0000 7f113959a700 1 -- 172.21.15.28:0/3174087103 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3584977303 0 0) 0x7f113c0b3020 con 0x7f113c0b2bc0 2023-11-28T11:08:56.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.146+0000 7f1139d9b700 1 --2- 172.21.15.28:0/3174087103 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f113c06f8b0 0x7f113c0b2680 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f112c009840 tx=0x7f112c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbc7e1b7b04b870f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.146+0000 7f113959a700 1 -- 172.21.15.28:0/3174087103 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c011830 con 0x7f113c06f8b0 2023-11-28T11:08:56.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.146+0000 7f113959a700 1 -- 172.21.15.28:0/3174087103 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c011990 con 0x7f113c06f8b0 2023-11-28T11:08:56.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.146+0000 7f113959a700 1 -- 172.21.15.28:0/3174087103 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c011d20 con 0x7f113c06f8b0 2023-11-28T11:08:56.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.147+0000 7f1140a74700 1 -- 172.21.15.28:0/3174087103 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f113c06f8b0 msgr2=0x7f113c0b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.147+0000 7f1140a74700 1 --2- 172.21.15.28:0/3174087103 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f113c06f8b0 0x7f113c0b2680 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f112c009840 tx=0x7f112c009b50 comp rx=0 tx=0).stop 2023-11-28T11:08:56.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.147+0000 7f1140a74700 1 -- 172.21.15.28:0/3174087103 shutdown_connections 2023-11-28T11:08:56.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.147+0000 7f1140a74700 1 --2- 172.21.15.28:0/3174087103 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f113c0b62d0 0x7f113c0cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.147+0000 7f1140a74700 1 --2- 172.21.15.28:0/3174087103 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f113c06f8b0 0x7f113c0b2680 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.147+0000 7f1140a74700 1 -- 172.21.15.28:0/3174087103 >> 172.21.15.28:0/3174087103 conn(0x7f113c0cd680 msgr2=0x7f113c0cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.147+0000 7f1140a74700 1 -- 172.21.15.28:0/3174087103 shutdown_connections 2023-11-28T11:08:56.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.147+0000 7f1140a74700 1 -- 172.21.15.28:0/3174087103 wait complete. 2023-11-28T11:08:56.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.148+0000 7f1140a74700 1 Processor -- start 2023-11-28T11:08:56.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.148+0000 7f1140a74700 1 -- start start 2023-11-28T11:08:56.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f1140a74700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f113c06f8b0 0x7f113c0ca910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f1140a74700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f113c0b2bc0 0x7f113c0a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f1140a74700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f113c0b62d0 0x7f113c157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f1140a74700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c0a7b10 con 0x7f113c06f8b0 2023-11-28T11:08:56.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f1140a74700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f113c1580e0 con 0x7f113c0b2bc0 2023-11-28T11:08:56.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f1140a74700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f113c0c89a0 con 0x7f113c0b62d0 2023-11-28T11:08:56.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f113a59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f113c06f8b0 0x7f113c0ca910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f113a59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f113c06f8b0 0x7f113c0ca910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46612/0 (socket says 172.21.15.28:46612) 2023-11-28T11:08:56.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f113a59c700 1 -- 172.21.15.28:0/1773297669 learned_addr learned my addr 172.21.15.28:0/1773297669 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:56.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f113ad9d700 1 --2- 172.21.15.28:0/1773297669 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f113c0b62d0 0x7f113c157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f1139d9b700 1 --2- 172.21.15.28:0/1773297669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f113c0b2bc0 0x7f113c0a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f113a59c700 1 -- 172.21.15.28:0/1773297669 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f113c0b62d0 msgr2=0x7f113c157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f113a59c700 1 --2- 172.21.15.28:0/1773297669 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f113c0b62d0 0x7f113c157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f113a59c700 1 -- 172.21.15.28:0/1773297669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f113c0b2bc0 msgr2=0x7f113c0a75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f113a59c700 1 --2- 172.21.15.28:0/1773297669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f113c0b2bc0 0x7f113c0a75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f113a59c700 1 -- 172.21.15.28:0/1773297669 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112c0094a0 con 0x7f113c06f8b0 2023-11-28T11:08:56.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.149+0000 7f113ad9d700 1 --2- 172.21.15.28:0/1773297669 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f113c0b62d0 0x7f113c157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:56.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.150+0000 7f113a59c700 1 --2- 172.21.15.28:0/1773297669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f113c06f8b0 0x7f113c0ca910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f113000d800 tx=0x7f113000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.150+0000 7f112affd700 1 -- 172.21.15.28:0/1773297669 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11300125f0 con 0x7f113c06f8b0 2023-11-28T11:08:56.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.150+0000 7f112affd700 1 -- 172.21.15.28:0/1773297669 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1130012750 con 0x7f113c06f8b0 2023-11-28T11:08:56.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.150+0000 7f1140a74700 1 -- 172.21.15.28:0/1773297669 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f113c0b66a0 con 0x7f113c06f8b0 2023-11-28T11:08:56.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.150+0000 7f112affd700 1 -- 172.21.15.28:0/1773297669 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1130012b00 con 0x7f113c06f8b0 2023-11-28T11:08:56.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.150+0000 7f1140a74700 1 -- 172.21.15.28:0/1773297669 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f113c0cffb0 con 0x7f113c06f8b0 2023-11-28T11:08:56.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.151+0000 7f1140a74700 1 -- 172.21.15.28:0/1773297669 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f113c06fd20 con 0x7f113c06f8b0 2023-11-28T11:08:56.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.151+0000 7f112affd700 1 -- 172.21.15.28:0/1773297669 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1130012c60 con 0x7f113c06f8b0 2023-11-28T11:08:56.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.152+0000 7f112affd700 1 --2- 172.21.15.28:0/1773297669 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f111c0682f0 0x7f111c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.152+0000 7f112affd700 1 -- 172.21.15.28:0/1773297669 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f11300864e0 con 0x7f113c06f8b0 2023-11-28T11:08:56.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.152+0000 7f1139d9b700 1 --2- 172.21.15.28:0/1773297669 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f111c0682f0 0x7f111c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.152+0000 7f1139d9b700 1 --2- 172.21.15.28:0/1773297669 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f111c0682f0 0x7f111c06a7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f113c0c9eb0 tx=0x7f112c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.156+0000 7f112affd700 1 -- 172.21.15.28:0/1773297669 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1130052a50 con 0x7f113c06f8b0 2023-11-28T11:08:56.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.351+0000 7f1140a74700 1 -- 172.21.15.28:0/1773297669 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f113c0c9890 con 0x7f113c06f8b0 2023-11-28T11:08:56.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.351+0000 7f112affd700 1 -- 172.21.15.28:0/1773297669 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1130056070 con 0x7f113c06f8b0 2023-11-28T11:08:56.353 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:08:56.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.353+0000 7f1140a74700 1 -- 172.21.15.28:0/1773297669 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f111c0682f0 msgr2=0x7f111c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.353+0000 7f1140a74700 1 --2- 172.21.15.28:0/1773297669 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f111c0682f0 0x7f111c06a7a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f113c0c9eb0 tx=0x7f112c008030 comp rx=0 tx=0).stop 2023-11-28T11:08:56.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.353+0000 7f1140a74700 1 -- 172.21.15.28:0/1773297669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f113c06f8b0 msgr2=0x7f113c0ca910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.353+0000 7f1140a74700 1 --2- 172.21.15.28:0/1773297669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f113c06f8b0 0x7f113c0ca910 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f113000d800 tx=0x7f113000dbc0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.353+0000 7f1140a74700 1 -- 172.21.15.28:0/1773297669 shutdown_connections 2023-11-28T11:08:56.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.353+0000 7f1140a74700 1 --2- 172.21.15.28:0/1773297669 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f111c0682f0 0x7f111c06a7a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.353+0000 7f1140a74700 1 --2- 172.21.15.28:0/1773297669 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f113c06f8b0 0x7f113c0ca910 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.353+0000 7f1140a74700 1 --2- 172.21.15.28:0/1773297669 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f113c0b62d0 0x7f113c157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.353+0000 7f1140a74700 1 --2- 172.21.15.28:0/1773297669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f113c0b2bc0 0x7f113c0a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.353+0000 7f1140a74700 1 -- 172.21.15.28:0/1773297669 >> 172.21.15.28:0/1773297669 conn(0x7f113c0cd680 msgr2=0x7f113c0cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.353+0000 7f1140a74700 1 -- 172.21.15.28:0/1773297669 shutdown_connections 2023-11-28T11:08:56.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.353+0000 7f1140a74700 1 -- 172.21.15.28:0/1773297669 wait complete. 2023-11-28T11:08:56.367 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:08:56.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.493+0000 7f712257b700 1 Processor -- start 2023-11-28T11:08:56.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.493+0000 7f712257b700 1 -- start start 2023-11-28T11:08:56.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.494+0000 7f712257b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f711c0ce560 0x7f711c06f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.494+0000 7f712257b700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711c06e470 con 0x7f711c1589c0 2023-11-28T11:08:56.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.494+0000 7f712257b700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711c06de60 con 0x7f711c158d90 2023-11-28T11:08:56.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.494+0000 7f712257b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f711c0ceb40 con 0x7f711c0ce560 2023-11-28T11:08:56.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.494+0000 7f711bfff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f711c1589c0 0x7f711c06aa70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34678/0 (socket says 172.21.15.28:34678) 2023-11-28T11:08:56.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.494+0000 7f711bfff700 1 -- 172.21.15.28:0/3770504039 learned_addr learned my addr 172.21.15.28:0/3770504039 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:56.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.495+0000 7f7120b18700 1 --2- 172.21.15.28:0/3770504039 >> v2:172.21.15.59:3300/0 conn(0x7f711c0ce560 0x7f711c06f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.495+0000 7f711affd700 1 -- 172.21.15.28:0/3770504039 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4093621880 0 0) 0x7f711c06e470 con 0x7f711c1589c0 2023-11-28T11:08:56.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.495+0000 7f711affd700 1 -- 172.21.15.28:0/3770504039 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71080035b0 con 0x7f711c1589c0 2023-11-28T11:08:56.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.495+0000 7f711affd700 1 -- 172.21.15.28:0/3770504039 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1460879040 0 0) 0x7f711c06de60 con 0x7f711c158d90 2023-11-28T11:08:56.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.495+0000 7f711affd700 1 -- 172.21.15.28:0/3770504039 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f711c06e470 con 0x7f711c158d90 2023-11-28T11:08:56.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.495+0000 7f711affd700 1 -- 172.21.15.28:0/3770504039 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3406747526 0 0) 0x7f71080035b0 con 0x7f711c1589c0 2023-11-28T11:08:56.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.495+0000 7f7120b18700 1 -- 172.21.15.28:0/3770504039 >> v1:172.21.15.59:6789/0 conn(0x7f711c158d90 legacy=0x7f711c06e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.495+0000 7f7120b18700 1 -- 172.21.15.28:0/3770504039 >> v1:172.21.15.28:6789/0 conn(0x7f711c1589c0 legacy=0x7f711c06aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.495+0000 7f7120b18700 1 -- 172.21.15.28:0/3770504039 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f711c0cee20 con 0x7f711c0ce560 2023-11-28T11:08:56.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.495+0000 7f7120b18700 1 --2- 172.21.15.28:0/3770504039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f711c0ce560 0x7f711c06f010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7114009d90 tx=0x7f711400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a3d68e4f26da4515 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.495+0000 7f711affd700 1 -- 172.21.15.28:0/3770504039 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711400e070 con 0x7f711c0ce560 2023-11-28T11:08:56.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.495+0000 7f711affd700 1 -- 172.21.15.28:0/3770504039 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7114003a20 con 0x7f711c0ce560 2023-11-28T11:08:56.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.495+0000 7f711affd700 1 -- 172.21.15.28:0/3770504039 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710c001670 con 0x7f711c0ce560 2023-11-28T11:08:56.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.496+0000 7f712257b700 1 -- 172.21.15.28:0/3770504039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f711c0ce560 msgr2=0x7f711c06f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.496+0000 7f712257b700 1 --2- 172.21.15.28:0/3770504039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f711c0ce560 0x7f711c06f010 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7114009d90 tx=0x7f711400b330 comp rx=0 tx=0).stop 2023-11-28T11:08:56.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.496+0000 7f712257b700 1 -- 172.21.15.28:0/3770504039 shutdown_connections 2023-11-28T11:08:56.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.496+0000 7f712257b700 1 --2- 172.21.15.28:0/3770504039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f711c0ce560 0x7f711c06f010 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.496+0000 7f712257b700 1 -- 172.21.15.28:0/3770504039 >> 172.21.15.28:0/3770504039 conn(0x7f711c0b0df0 msgr2=0x7f711c0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.496+0000 7f712257b700 1 -- 172.21.15.28:0/3770504039 shutdown_connections 2023-11-28T11:08:56.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.496+0000 7f712257b700 1 -- 172.21.15.28:0/3770504039 wait complete. 2023-11-28T11:08:56.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.497+0000 7f712257b700 1 Processor -- start 2023-11-28T11:08:56.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.497+0000 7f712257b700 1 -- start start 2023-11-28T11:08:56.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f712257b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f711c0ce560 0x7f711c159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f712257b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f711c1589c0 0x7f711c15b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f712257b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f711c158d90 0x7f711c0c63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f712257b700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c15a6e0 con 0x7f711c1589c0 2023-11-28T11:08:56.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f712257b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711c159a60 con 0x7f711c0ce560 2023-11-28T11:08:56.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f712257b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f711c159d40 con 0x7f711c158d90 2023-11-28T11:08:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f711b7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f711c1589c0 0x7f711c15b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f711b7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f711c1589c0 0x7f711c15b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46628/0 (socket says 172.21.15.28:46628) 2023-11-28T11:08:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f711b7fe700 1 -- 172.21.15.28:0/4159744276 learned_addr learned my addr 172.21.15.28:0/4159744276 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f711bfff700 1 --2- 172.21.15.28:0/4159744276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f711c0ce560 0x7f711c159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f7120b18700 1 --2- 172.21.15.28:0/4159744276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f711c158d90 0x7f711c0c63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f711b7fe700 1 -- 172.21.15.28:0/4159744276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f711c158d90 msgr2=0x7f711c0c63d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f711b7fe700 1 --2- 172.21.15.28:0/4159744276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f711c158d90 0x7f711c0c63d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f711b7fe700 1 -- 172.21.15.28:0/4159744276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f711c0ce560 msgr2=0x7f711c159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f711b7fe700 1 --2- 172.21.15.28:0/4159744276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f711c0ce560 0x7f711c159210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.498+0000 7f711b7fe700 1 -- 172.21.15.28:0/4159744276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7114009a70 con 0x7f711c1589c0 2023-11-28T11:08:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.499+0000 7f711b7fe700 1 --2- 172.21.15.28:0/4159744276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f711c1589c0 0x7f711c15b990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f710400c4d0 tx=0x7f710400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.499+0000 7f7118ff9700 1 -- 172.21.15.28:0/4159744276 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71040052d0 con 0x7f711c1589c0 2023-11-28T11:08:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.499+0000 7f7118ff9700 1 -- 172.21.15.28:0/4159744276 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7104005430 con 0x7f711c1589c0 2023-11-28T11:08:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.499+0000 7f712257b700 1 -- 172.21.15.28:0/4159744276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711c0c9d90 con 0x7f711c1589c0 2023-11-28T11:08:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.499+0000 7f7118ff9700 1 -- 172.21.15.28:0/4159744276 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71040038a0 con 0x7f711c1589c0 2023-11-28T11:08:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.499+0000 7f712257b700 1 -- 172.21.15.28:0/4159744276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711c06e100 con 0x7f711c1589c0 2023-11-28T11:08:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.500+0000 7f712257b700 1 -- 172.21.15.28:0/4159744276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711c06a1e0 con 0x7f711c1589c0 2023-11-28T11:08:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.500+0000 7f7118ff9700 1 -- 172.21.15.28:0/4159744276 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7104019030 con 0x7f711c1589c0 2023-11-28T11:08:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.501+0000 7f7118ff9700 1 --2- 172.21.15.28:0/4159744276 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70fc0682f0 0x7f70fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.501+0000 7f7118ff9700 1 -- 172.21.15.28:0/4159744276 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f71040852c0 con 0x7f711c1589c0 2023-11-28T11:08:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.501+0000 7f711bfff700 1 --2- 172.21.15.28:0/4159744276 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70fc0682f0 0x7f70fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.501+0000 7f711bfff700 1 --2- 172.21.15.28:0/4159744276 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70fc0682f0 0x7f70fc06a7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f710c0011e0 tx=0x7f710c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.505+0000 7f7118ff9700 1 -- 172.21.15.28:0/4159744276 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7104051830 con 0x7f711c1589c0 2023-11-28T11:08:56.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.695+0000 7f712257b700 1 -- 172.21.15.28:0/4159744276 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f711c15a820 con 0x7f711c1589c0 2023-11-28T11:08:56.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.695+0000 7f7118ff9700 1 -- 172.21.15.28:0/4159744276 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f711c15a820 con 0x7f711c1589c0 2023-11-28T11:08:56.697 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:08:56.700 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.698+0000 7f712257b700 1 -- 172.21.15.28:0/4159744276 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70fc0682f0 msgr2=0x7f70fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.700 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.698+0000 7f712257b700 1 --2- 172.21.15.28:0/4159744276 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70fc0682f0 0x7f70fc06a7a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f710c0011e0 tx=0x7f710c009040 comp rx=0 tx=0).stop 2023-11-28T11:08:56.700 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.698+0000 7f712257b700 1 -- 172.21.15.28:0/4159744276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f711c1589c0 msgr2=0x7f711c15b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.700 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.698+0000 7f712257b700 1 --2- 172.21.15.28:0/4159744276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f711c1589c0 0x7f711c15b990 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f710400c4d0 tx=0x7f710400c890 comp rx=0 tx=0).stop 2023-11-28T11:08:56.700 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.698+0000 7f712257b700 1 -- 172.21.15.28:0/4159744276 shutdown_connections 2023-11-28T11:08:56.700 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.698+0000 7f712257b700 1 --2- 172.21.15.28:0/4159744276 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70fc0682f0 0x7f70fc06a7a0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.698+0000 7f712257b700 1 --2- 172.21.15.28:0/4159744276 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f711c158d90 0x7f711c0c63d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.698+0000 7f712257b700 1 --2- 172.21.15.28:0/4159744276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f711c0ce560 0x7f711c159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.698+0000 7f712257b700 1 --2- 172.21.15.28:0/4159744276 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f711c1589c0 0x7f711c15b990 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.698+0000 7f712257b700 1 -- 172.21.15.28:0/4159744276 >> 172.21.15.28:0/4159744276 conn(0x7f711c0b0df0 msgr2=0x7f711c06a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.698+0000 7f712257b700 1 -- 172.21.15.28:0/4159744276 shutdown_connections 2023-11-28T11:08:56.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.698+0000 7f712257b700 1 -- 172.21.15.28:0/4159744276 wait complete. 2023-11-28T11:08:56.711 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:08:56.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.838+0000 7fbbc5b81700 1 Processor -- start 2023-11-28T11:08:56.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.838+0000 7fbbc5b81700 1 -- start start 2023-11-28T11:08:56.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.839+0000 7fbbc5b81700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbbc00b1630 0x7fbbc00cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.839+0000 7fbbc5b81700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbbc006afc0 0x7fbbc006b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.839+0000 7fbbc5b81700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbc00ce550 con 0x7fbbc00b1a00 2023-11-28T11:08:56.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.839+0000 7fbbc5b81700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc00ce830 con 0x7fbbc006afc0 2023-11-28T11:08:56.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.839+0000 7fbbc5b81700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbc00ceb10 con 0x7fbbc00b1630 2023-11-28T11:08:56.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.839+0000 7fbbbeffd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbbc00b1a00 0x7fbbc00caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34692/0 (socket says 172.21.15.28:34692) 2023-11-28T11:08:56.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.839+0000 7fbbbeffd700 1 -- 172.21.15.28:0/3360781126 learned_addr learned my addr 172.21.15.28:0/3360781126 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:56.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.839+0000 7fbbbffff700 1 --2- 172.21.15.28:0/3360781126 >> v2:172.21.15.28:3300/0 conn(0x7fbbc006afc0 0x7fbbc006b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.839+0000 7fbbbe7fc700 1 -- 172.21.15.28:0/3360781126 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3212708229 0 0) 0x7fbbc00ce550 con 0x7fbbc00b1a00 2023-11-28T11:08:56.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.839+0000 7fbbbf7fe700 1 --2- 172.21.15.28:0/3360781126 >> v2:172.21.15.59:3300/0 conn(0x7fbbc00b1630 0x7fbbc00cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.839+0000 7fbbbe7fc700 1 -- 172.21.15.28:0/3360781126 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbbac0035b0 con 0x7fbbc00b1a00 2023-11-28T11:08:56.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.840+0000 7fbbbffff700 1 -- 172.21.15.28:0/3360781126 >> v2:172.21.15.59:3300/0 conn(0x7fbbc00b1630 msgr2=0x7fbbc00cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:56.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.840+0000 7fbbbffff700 1 --2- 172.21.15.28:0/3360781126 >> v2:172.21.15.59:3300/0 conn(0x7fbbc00b1630 0x7fbbc00cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.840+0000 7fbbbffff700 1 -- 172.21.15.28:0/3360781126 >> v1:172.21.15.28:6789/0 conn(0x7fbbc00b1a00 legacy=0x7fbbc00caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.840+0000 7fbbbffff700 1 -- 172.21.15.28:0/3360781126 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbc00cedf0 con 0x7fbbc006afc0 2023-11-28T11:08:56.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.840+0000 7fbbbf7fe700 1 --2- 172.21.15.28:0/3360781126 >> v2:172.21.15.59:3300/0 conn(0x7fbbc00b1630 0x7fbbc00cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:56.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.840+0000 7fbbbffff700 1 --2- 172.21.15.28:0/3360781126 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbc006afc0 0x7fbbc006b420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009d90 tx=0x7fbbb800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8bec42ce5bb72 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.840+0000 7fbbbe7fc700 1 -- 172.21.15.28:0/3360781126 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb800e070 con 0x7fbbc006afc0 2023-11-28T11:08:56.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.840+0000 7fbbbe7fc700 1 -- 172.21.15.28:0/3360781126 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb800bae0 con 0x7fbbc006afc0 2023-11-28T11:08:56.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.840+0000 7fbbbe7fc700 1 -- 172.21.15.28:0/3360781126 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb800be90 con 0x7fbbc006afc0 2023-11-28T11:08:56.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.840+0000 7fbbc5b81700 1 -- 172.21.15.28:0/3360781126 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbc006afc0 msgr2=0x7fbbc006b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.840+0000 7fbbc5b81700 1 --2- 172.21.15.28:0/3360781126 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbc006afc0 0x7fbbc006b420 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009d90 tx=0x7fbbb800b330 comp rx=0 tx=0).stop 2023-11-28T11:08:56.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.841+0000 7fbbc5b81700 1 -- 172.21.15.28:0/3360781126 shutdown_connections 2023-11-28T11:08:56.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.841+0000 7fbbc5b81700 1 --2- 172.21.15.28:0/3360781126 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbc006afc0 0x7fbbc006b420 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.841+0000 7fbbc5b81700 1 --2- 172.21.15.28:0/3360781126 >> v2:172.21.15.59:3300/0 conn(0x7fbbc00b1630 0x7fbbc00cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.841+0000 7fbbc5b81700 1 -- 172.21.15.28:0/3360781126 >> 172.21.15.28:0/3360781126 conn(0x7fbbc00cc630 msgr2=0x7fbbc00cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:56.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.841+0000 7fbbc5b81700 1 -- 172.21.15.28:0/3360781126 shutdown_connections 2023-11-28T11:08:56.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.841+0000 7fbbc5b81700 1 -- 172.21.15.28:0/3360781126 wait complete. 2023-11-28T11:08:56.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.842+0000 7fbbc5b81700 1 Processor -- start 2023-11-28T11:08:56.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.842+0000 7fbbc5b81700 1 -- start start 2023-11-28T11:08:56.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbc5b81700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbc006afc0 0x7fbbc00c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbc5b81700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbc00b1630 0x7fbbc01f2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbc5b81700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbbc00b1a00 0x7fbbc015b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbc5b81700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc015db20 con 0x7fbbc006afc0 2023-11-28T11:08:56.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbc5b81700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc015cea0 con 0x7fbbc00b1630 2023-11-28T11:08:56.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbc5b81700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbc015d180 con 0x7fbbc00b1a00 2023-11-28T11:08:56.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbbf7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbc006afc0 0x7fbbc00c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbbf7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbc006afc0 0x7fbbc00c95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46640/0 (socket says 172.21.15.28:46640) 2023-11-28T11:08:56.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbbf7fe700 1 -- 172.21.15.28:0/1011363212 learned_addr learned my addr 172.21.15.28:0/1011363212 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:56.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbbeffd700 1 --2- 172.21.15.28:0/1011363212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbc00b1630 0x7fbbc01f2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbbffff700 1 --2- 172.21.15.28:0/1011363212 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbbc00b1a00 0x7fbbc015b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbbf7fe700 1 -- 172.21.15.28:0/1011363212 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbbc00b1a00 msgr2=0x7fbbc015b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbbf7fe700 1 --2- 172.21.15.28:0/1011363212 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbbc00b1a00 0x7fbbc015b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbbf7fe700 1 -- 172.21.15.28:0/1011363212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbc00b1630 msgr2=0x7fbbc01f2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:56.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbbf7fe700 1 --2- 172.21.15.28:0/1011363212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbc00b1630 0x7fbbc01f2740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:56.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbbf7fe700 1 -- 172.21.15.28:0/1011363212 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb8009a70 con 0x7fbbc006afc0 2023-11-28T11:08:56.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.843+0000 7fbbbf7fe700 1 --2- 172.21.15.28:0/1011363212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbc006afc0 0x7fbbc00c95f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbba800e810 tx=0x7fbba800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.844+0000 7fbbb7fff700 1 -- 172.21.15.28:0/1011363212 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba800c070 con 0x7fbbc006afc0 2023-11-28T11:08:56.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.844+0000 7fbbb7fff700 1 -- 172.21.15.28:0/1011363212 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba8004230 con 0x7fbbc006afc0 2023-11-28T11:08:56.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.844+0000 7fbbc5b81700 1 -- 172.21.15.28:0/1011363212 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbc00b1ea0 con 0x7fbbc006afc0 2023-11-28T11:08:56.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.844+0000 7fbbb7fff700 1 -- 172.21.15.28:0/1011363212 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba80036a0 con 0x7fbbc006afc0 2023-11-28T11:08:56.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.844+0000 7fbbc5b81700 1 -- 172.21.15.28:0/1011363212 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbc00cf0d0 con 0x7fbbc006afc0 2023-11-28T11:08:56.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.845+0000 7fbbc5b81700 1 -- 172.21.15.28:0/1011363212 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbc00cb5b0 con 0x7fbbc006afc0 2023-11-28T11:08:56.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.845+0000 7fbbb7fff700 1 -- 172.21.15.28:0/1011363212 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbba8019020 con 0x7fbbc006afc0 2023-11-28T11:08:56.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.845+0000 7fbbb7fff700 1 --2- 172.21.15.28:0/1011363212 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbba00682f0 0x7fbba006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:56.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.846+0000 7fbbb7fff700 1 -- 172.21.15.28:0/1011363212 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbba8085ba0 con 0x7fbbc006afc0 2023-11-28T11:08:56.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.846+0000 7fbbbeffd700 1 --2- 172.21.15.28:0/1011363212 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbba00682f0 0x7fbba006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:56.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.846+0000 7fbbbeffd700 1 --2- 172.21.15.28:0/1011363212 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbba00682f0 0x7fbba006a7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00c6420 tx=0x7fbbb0007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:56.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:56.850+0000 7fbbb7fff700 1 -- 172.21.15.28:0/1011363212 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbba8052110 con 0x7fbbc006afc0 2023-11-28T11:08:57.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.043+0000 7fbbc5b81700 1 -- 172.21.15.28:0/1011363212 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbbc00739f0 con 0x7fbbc006afc0 2023-11-28T11:08:57.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.043+0000 7fbbb7fff700 1 -- 172.21.15.28:0/1011363212 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbba800f040 con 0x7fbbc006afc0 2023-11-28T11:08:57.045 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:08:57.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.045+0000 7fbbc5b81700 1 -- 172.21.15.28:0/1011363212 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbba00682f0 msgr2=0x7fbba006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.045+0000 7fbbc5b81700 1 --2- 172.21.15.28:0/1011363212 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbba00682f0 0x7fbba006a7a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fbbc00c6420 tx=0x7fbbb0007dc0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.045+0000 7fbbc5b81700 1 -- 172.21.15.28:0/1011363212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbc006afc0 msgr2=0x7fbbc00c95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.045+0000 7fbbc5b81700 1 --2- 172.21.15.28:0/1011363212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbc006afc0 0x7fbbc00c95f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fbba800e810 tx=0x7fbba800eb20 comp rx=0 tx=0).stop 2023-11-28T11:08:57.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.045+0000 7fbbc5b81700 1 -- 172.21.15.28:0/1011363212 shutdown_connections 2023-11-28T11:08:57.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.045+0000 7fbbc5b81700 1 --2- 172.21.15.28:0/1011363212 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbba00682f0 0x7fbba006a7a0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.045+0000 7fbbc5b81700 1 --2- 172.21.15.28:0/1011363212 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbc006afc0 0x7fbbc00c95f0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.045+0000 7fbbc5b81700 1 --2- 172.21.15.28:0/1011363212 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbbc00b1a00 0x7fbbc015b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.045+0000 7fbbc5b81700 1 --2- 172.21.15.28:0/1011363212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbc00b1630 0x7fbbc01f2740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.045+0000 7fbbc5b81700 1 -- 172.21.15.28:0/1011363212 >> 172.21.15.28:0/1011363212 conn(0x7fbbc00cc630 msgr2=0x7fbbc00ae300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.045+0000 7fbbc5b81700 1 -- 172.21.15.28:0/1011363212 shutdown_connections 2023-11-28T11:08:57.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.045+0000 7fbbc5b81700 1 -- 172.21.15.28:0/1011363212 wait complete. 2023-11-28T11:08:57.059 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:08:57.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.185+0000 7f05be4e9700 1 Processor -- start 2023-11-28T11:08:57.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.185+0000 7f05be4e9700 1 -- start start 2023-11-28T11:08:57.189 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.186+0000 7f05be4e9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05b80b1630 0x7f05b80cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.189 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.186+0000 7f05be4e9700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f05b806afc0 0x7f05b806b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.186+0000 7f05be4e9700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b80ce550 con 0x7f05b80b1a00 2023-11-28T11:08:57.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.186+0000 7f05be4e9700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b80ce830 con 0x7f05b806afc0 2023-11-28T11:08:57.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.186+0000 7f05be4e9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b80ceb10 con 0x7f05b80b1630 2023-11-28T11:08:57.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.186+0000 7f05bca86700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f05b806afc0 0x7f05b806b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.186+0000 7f05b77fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f05b80b1a00 0x7f05b80caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34708/0 (socket says 172.21.15.28:34708) 2023-11-28T11:08:57.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.186+0000 7f05bca86700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f05b806afc0 0x7f05b806b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46652/0 (socket says 172.21.15.28:46652) 2023-11-28T11:08:57.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.186+0000 7f05b77fe700 1 -- 172.21.15.28:0/67668585 learned_addr learned my addr 172.21.15.28:0/67668585 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:57.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.187+0000 7f05b7fff700 1 --2- 172.21.15.28:0/67668585 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05b80b1630 0x7f05b80cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.187+0000 7f05bca86700 1 -- 172.21.15.28:0/67668585 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05b80b1630 msgr2=0x7f05b80cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.187+0000 7f05bca86700 1 --2- 172.21.15.28:0/67668585 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05b80b1630 0x7f05b80cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.187+0000 7f05bca86700 1 -- 172.21.15.28:0/67668585 >> v1:172.21.15.28:6789/0 conn(0x7f05b80b1a00 legacy=0x7f05b80caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.187+0000 7f05bca86700 1 -- 172.21.15.28:0/67668585 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b80cedf0 con 0x7f05b806afc0 2023-11-28T11:08:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.187+0000 7f05b6ffd700 1 -- 172.21.15.28:0/67668585 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1332127289 0 0) 0x7f05b80ce550 con 0x7f05b80b1a00 2023-11-28T11:08:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.187+0000 7f05bca86700 1 --2- 172.21.15.28:0/67668585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05b806afc0 0x7f05b806b440 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009d90 tx=0x7f05b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2014db5ff285292 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.187+0000 7f05b6ffd700 1 -- 172.21.15.28:0/67668585 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b000e070 con 0x7f05b806afc0 2023-11-28T11:08:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.187+0000 7f05b6ffd700 1 -- 172.21.15.28:0/67668585 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b000bbd0 con 0x7f05b806afc0 2023-11-28T11:08:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.187+0000 7f05b6ffd700 1 -- 172.21.15.28:0/67668585 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0012430 con 0x7f05b806afc0 2023-11-28T11:08:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.187+0000 7f05be4e9700 1 -- 172.21.15.28:0/67668585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05b806afc0 msgr2=0x7f05b806b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.187+0000 7f05be4e9700 1 --2- 172.21.15.28:0/67668585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05b806afc0 0x7f05b806b440 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f05b0009d90 tx=0x7f05b000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:57.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.187+0000 7f05be4e9700 1 -- 172.21.15.28:0/67668585 shutdown_connections 2023-11-28T11:08:57.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.187+0000 7f05be4e9700 1 --2- 172.21.15.28:0/67668585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05b806afc0 0x7f05b806b440 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.188+0000 7f05be4e9700 1 --2- 172.21.15.28:0/67668585 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05b80b1630 0x7f05b80cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.188+0000 7f05be4e9700 1 -- 172.21.15.28:0/67668585 >> 172.21.15.28:0/67668585 conn(0x7f05b80cc630 msgr2=0x7f05b80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.188+0000 7f05be4e9700 1 -- 172.21.15.28:0/67668585 shutdown_connections 2023-11-28T11:08:57.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.188+0000 7f05be4e9700 1 -- 172.21.15.28:0/67668585 wait complete. 2023-11-28T11:08:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.189+0000 7f05be4e9700 1 Processor -- start 2023-11-28T11:08:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.189+0000 7f05be4e9700 1 -- start start 2023-11-28T11:08:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.190+0000 7f05be4e9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05b806afc0 0x7f05b80c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.190+0000 7f05be4e9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05b80b1630 0x7f05b80b54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.190+0000 7f05be4e9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05b80b1a00 0x7f05b80c9500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.190+0000 7f05be4e9700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b80c4ca0 con 0x7f05b80b1a00 2023-11-28T11:08:57.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.190+0000 7f05be4e9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b80c4f80 con 0x7f05b80b1630 2023-11-28T11:08:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.190+0000 7f05be4e9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b80c5260 con 0x7f05b806afc0 2023-11-28T11:08:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.190+0000 7f05bca86700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05b80b1a00 0x7f05b80c9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.190+0000 7f05bca86700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05b80b1a00 0x7f05b80c9500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46658/0 (socket says 172.21.15.28:46658) 2023-11-28T11:08:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.190+0000 7f05bca86700 1 -- 172.21.15.28:0/415281217 learned_addr learned my addr 172.21.15.28:0/415281217 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.191+0000 7f05b7fff700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05b806afc0 0x7f05b80c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.191+0000 7f05b77fe700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05b80b1630 0x7f05b80b54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.191+0000 7f05bca86700 1 -- 172.21.15.28:0/415281217 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05b806afc0 msgr2=0x7f05b80c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.191+0000 7f05bca86700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05b806afc0 0x7f05b80c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.191+0000 7f05bca86700 1 -- 172.21.15.28:0/415281217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05b80b1630 msgr2=0x7f05b80b54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.191+0000 7f05bca86700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05b80b1630 0x7f05b80b54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.191+0000 7f05bca86700 1 -- 172.21.15.28:0/415281217 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b0009a70 con 0x7f05b80b1a00 2023-11-28T11:08:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.191+0000 7f05b7fff700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05b806afc0 0x7f05b80c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.191+0000 7f05b77fe700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05b80b1630 0x7f05b80b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.191+0000 7f05bca86700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05b80b1a00 0x7f05b80c9500 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f05b000b370 tx=0x7f05b0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.191+0000 7f05b4ff9700 1 -- 172.21.15.28:0/415281217 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0012430 con 0x7f05b80b1a00 2023-11-28T11:08:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.191+0000 7f05b4ff9700 1 -- 172.21.15.28:0/415281217 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b0020070 con 0x7f05b80b1a00 2023-11-28T11:08:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.191+0000 7f05be4e9700 1 -- 172.21.15.28:0/415281217 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b80b1e70 con 0x7f05b80b1a00 2023-11-28T11:08:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.191+0000 7f05b4ff9700 1 -- 172.21.15.28:0/415281217 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0004260 con 0x7f05b80b1a00 2023-11-28T11:08:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.191+0000 7f05be4e9700 1 -- 172.21.15.28:0/415281217 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b80cf0d0 con 0x7f05b80b1a00 2023-11-28T11:08:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.192+0000 7f05be4e9700 1 -- 172.21.15.28:0/415281217 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b80cb5b0 con 0x7f05b80b1a00 2023-11-28T11:08:57.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.192+0000 7f05b4ff9700 1 -- 172.21.15.28:0/415281217 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f05b002e070 con 0x7f05b80b1a00 2023-11-28T11:08:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.193+0000 7f05b4ff9700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f05980682f0 0x7f059806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.193+0000 7f05b4ff9700 1 -- 172.21.15.28:0/415281217 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f05b0022070 con 0x7f05b80b1a00 2023-11-28T11:08:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.193+0000 7f05b7fff700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f05980682f0 0x7f059806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.193+0000 7f05b7fff700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f05980682f0 0x7f059806a7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05b80c66e0 tx=0x7f05a8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.197+0000 7f05b4ff9700 1 -- 172.21.15.28:0/415281217 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f05b0054370 con 0x7f05b80b1a00 2023-11-28T11:08:57.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.391+0000 7f05be4e9700 1 -- 172.21.15.28:0/415281217 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f05b80739f0 con 0x7f05b80b1a00 2023-11-28T11:08:57.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.391+0000 7f05b4ff9700 1 -- 172.21.15.28:0/415281217 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f05b002c020 con 0x7f05b80b1a00 2023-11-28T11:08:57.393 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:08:57.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.394+0000 7f05be4e9700 1 -- 172.21.15.28:0/415281217 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f05980682f0 msgr2=0x7f059806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.394+0000 7f05be4e9700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f05980682f0 0x7f059806a7a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f05b80c66e0 tx=0x7f05a8008040 comp rx=0 tx=0).stop 2023-11-28T11:08:57.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.394+0000 7f05be4e9700 1 -- 172.21.15.28:0/415281217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05b80b1a00 msgr2=0x7f05b80c9500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.394+0000 7f05be4e9700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05b80b1a00 0x7f05b80c9500 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f05b000b370 tx=0x7f05b0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.394+0000 7f05be4e9700 1 -- 172.21.15.28:0/415281217 shutdown_connections 2023-11-28T11:08:57.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.394+0000 7f05be4e9700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f05980682f0 0x7f059806a7a0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.394+0000 7f05be4e9700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05b80b1a00 0x7f05b80c9500 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.394+0000 7f05be4e9700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05b806afc0 0x7f05b80c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.394+0000 7f05be4e9700 1 --2- 172.21.15.28:0/415281217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05b80b1630 0x7f05b80b54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.394+0000 7f05be4e9700 1 -- 172.21.15.28:0/415281217 >> 172.21.15.28:0/415281217 conn(0x7f05b80cc630 msgr2=0x7f05b80caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.394+0000 7f05be4e9700 1 -- 172.21.15.28:0/415281217 shutdown_connections 2023-11-28T11:08:57.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.394+0000 7f05be4e9700 1 -- 172.21.15.28:0/415281217 wait complete. 2023-11-28T11:08:57.407 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:08:57.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.531+0000 7fcd3edee700 1 Processor -- start 2023-11-28T11:08:57.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.531+0000 7fcd3edee700 1 -- start start 2023-11-28T11:08:57.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.532+0000 7fcd3edee700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcd380ceec0 0x7fcd38167590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.532+0000 7fcd3edee700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd380b30b0 con 0x7fcd380b2660 2023-11-28T11:08:57.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.532+0000 7fcd3edee700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd3806c020 con 0x7fcd3815c560 2023-11-28T11:08:57.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.532+0000 7fcd3edee700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd380d09c0 con 0x7fcd380ceec0 2023-11-28T11:08:57.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.532+0000 7fcd3d38b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcd380ceec0 0x7fcd38167590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.532+0000 7fcd37fff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fcd380b2660 0x7fcd380ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34722/0 (socket says 172.21.15.28:34722) 2023-11-28T11:08:57.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.532+0000 7fcd37fff700 1 -- 172.21.15.28:0/2272385911 learned_addr learned my addr 172.21.15.28:0/2272385911 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:57.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.533+0000 7fcd3d38b700 1 -- 172.21.15.28:0/2272385911 >> v1:172.21.15.59:6789/0 conn(0x7fcd3815c560 legacy=0x7fcd3806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.533+0000 7fcd3d38b700 1 -- 172.21.15.28:0/2272385911 >> v1:172.21.15.28:6789/0 conn(0x7fcd380b2660 legacy=0x7fcd380ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.533+0000 7fcd377fe700 1 -- 172.21.15.28:0/2272385911 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309668832 0 0) 0x7fcd380b30b0 con 0x7fcd380b2660 2023-11-28T11:08:57.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.533+0000 7fcd3d38b700 1 -- 172.21.15.28:0/2272385911 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd380b2af0 con 0x7fcd380ceec0 2023-11-28T11:08:57.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.533+0000 7fcd3d38b700 1 --2- 172.21.15.28:0/2272385911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd380ceec0 0x7fcd38167590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009d90 tx=0x7fcd3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3135e0d2cf7d87 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.533+0000 7fcd377fe700 1 -- 172.21.15.28:0/2272385911 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd3000e070 con 0x7fcd380ceec0 2023-11-28T11:08:57.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.533+0000 7fcd377fe700 1 -- 172.21.15.28:0/2272385911 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd3000bbd0 con 0x7fcd380ceec0 2023-11-28T11:08:57.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.533+0000 7fcd377fe700 1 -- 172.21.15.28:0/2272385911 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd30012430 con 0x7fcd380ceec0 2023-11-28T11:08:57.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.534+0000 7fcd3edee700 1 -- 172.21.15.28:0/2272385911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd380ceec0 msgr2=0x7fcd38167590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.534+0000 7fcd3edee700 1 --2- 172.21.15.28:0/2272385911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd380ceec0 0x7fcd38167590 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fcd30009d90 tx=0x7fcd3000b330 comp rx=0 tx=0).stop 2023-11-28T11:08:57.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.534+0000 7fcd3edee700 1 -- 172.21.15.28:0/2272385911 shutdown_connections 2023-11-28T11:08:57.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.534+0000 7fcd3edee700 1 --2- 172.21.15.28:0/2272385911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd380ceec0 0x7fcd38167590 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.534+0000 7fcd3edee700 1 -- 172.21.15.28:0/2272385911 >> 172.21.15.28:0/2272385911 conn(0x7fcd380cd660 msgr2=0x7fcd380cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.534+0000 7fcd3edee700 1 -- 172.21.15.28:0/2272385911 shutdown_connections 2023-11-28T11:08:57.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.534+0000 7fcd3edee700 1 -- 172.21.15.28:0/2272385911 wait complete. 2023-11-28T11:08:57.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.535+0000 7fcd3edee700 1 Processor -- start 2023-11-28T11:08:57.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.535+0000 7fcd3edee700 1 -- start start 2023-11-28T11:08:57.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3edee700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd380b2660 0x7fcd38154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3edee700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcd380ceec0 0x7fcd3815ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3edee700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd3815c560 0x7fcd38158480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3edee700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3815ec40 con 0x7fcd380b2660 2023-11-28T11:08:57.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3edee700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd3815fa80 con 0x7fcd3815c560 2023-11-28T11:08:57.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3edee700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd380c70d0 con 0x7fcd380ceec0 2023-11-28T11:08:57.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3cb8a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd380b2660 0x7fcd38154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3cb8a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd380b2660 0x7fcd38154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46668/0 (socket says 172.21.15.28:46668) 2023-11-28T11:08:57.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3cb8a700 1 -- 172.21.15.28:0/3024590661 learned_addr learned my addr 172.21.15.28:0/3024590661 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:57.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3d38b700 1 --2- 172.21.15.28:0/3024590661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd3815c560 0x7fcd38158480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd37fff700 1 --2- 172.21.15.28:0/3024590661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcd380ceec0 0x7fcd3815ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3cb8a700 1 -- 172.21.15.28:0/3024590661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcd380ceec0 msgr2=0x7fcd3815ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3cb8a700 1 --2- 172.21.15.28:0/3024590661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcd380ceec0 0x7fcd3815ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3cb8a700 1 -- 172.21.15.28:0/3024590661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd3815c560 msgr2=0x7fcd38158480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3cb8a700 1 --2- 172.21.15.28:0/3024590661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd3815c560 0x7fcd38158480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3cb8a700 1 -- 172.21.15.28:0/3024590661 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd30009a70 con 0x7fcd380b2660 2023-11-28T11:08:57.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.536+0000 7fcd3cb8a700 1 --2- 172.21.15.28:0/3024590661 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd380b2660 0x7fcd38154c70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00c510 tx=0x7fcd2c00c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.537+0000 7fcd357fa700 1 -- 172.21.15.28:0/3024590661 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0052d0 con 0x7fcd380b2660 2023-11-28T11:08:57.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.537+0000 7fcd357fa700 1 -- 172.21.15.28:0/3024590661 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd2c005430 con 0x7fcd380b2660 2023-11-28T11:08:57.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.537+0000 7fcd357fa700 1 -- 172.21.15.28:0/3024590661 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd2c0038c0 con 0x7fcd380b2660 2023-11-28T11:08:57.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.537+0000 7fcd3edee700 1 -- 172.21.15.28:0/3024590661 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd3806bcb0 con 0x7fcd380b2660 2023-11-28T11:08:57.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.537+0000 7fcd3edee700 1 -- 172.21.15.28:0/3024590661 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd380b2dd0 con 0x7fcd380b2660 2023-11-28T11:08:57.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.538+0000 7fcd3edee700 1 -- 172.21.15.28:0/3024590661 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd3815c930 con 0x7fcd380b2660 2023-11-28T11:08:57.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.538+0000 7fcd357fa700 1 -- 172.21.15.28:0/3024590661 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcd2c019030 con 0x7fcd380b2660 2023-11-28T11:08:57.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.538+0000 7fcd357fa700 1 --2- 172.21.15.28:0/3024590661 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcd1c0682f0 0x7fcd1c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.538+0000 7fcd357fa700 1 -- 172.21.15.28:0/3024590661 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fcd2c085180 con 0x7fcd380b2660 2023-11-28T11:08:57.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.538+0000 7fcd37fff700 1 --2- 172.21.15.28:0/3024590661 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcd1c0682f0 0x7fcd1c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.539+0000 7fcd37fff700 1 --2- 172.21.15.28:0/3024590661 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcd1c0682f0 0x7fcd1c06a7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd380c7320 tx=0x7fcd24004a50 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.543+0000 7fcd357fa700 1 -- 172.21.15.28:0/3024590661 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcd2c0516f0 con 0x7fcd380b2660 2023-11-28T11:08:57.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.752+0000 7fcd3edee700 1 -- 172.21.15.28:0/3024590661 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcd3815fd60 con 0x7fcd380b2660 2023-11-28T11:08:57.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.753+0000 7fcd357fa700 1 -- 172.21.15.28:0/3024590661 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcd2c054d10 con 0x7fcd380b2660 2023-11-28T11:08:57.756 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.755+0000 7fcd3edee700 1 -- 172.21.15.28:0/3024590661 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcd1c0682f0 msgr2=0x7fcd1c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.756 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.755+0000 7fcd3edee700 1 --2- 172.21.15.28:0/3024590661 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcd1c0682f0 0x7fcd1c06a7a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcd380c7320 tx=0x7fcd24004a50 comp rx=0 tx=0).stop 2023-11-28T11:08:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.755+0000 7fcd3edee700 1 -- 172.21.15.28:0/3024590661 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd380b2660 msgr2=0x7fcd38154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.755+0000 7fcd3edee700 1 --2- 172.21.15.28:0/3024590661 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd380b2660 0x7fcd38154c70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcd2c00c510 tx=0x7fcd2c00c820 comp rx=0 tx=0).stop 2023-11-28T11:08:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.755+0000 7fcd3edee700 1 -- 172.21.15.28:0/3024590661 shutdown_connections 2023-11-28T11:08:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.755+0000 7fcd3edee700 1 --2- 172.21.15.28:0/3024590661 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcd1c0682f0 0x7fcd1c06a7a0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.755+0000 7fcd3edee700 1 --2- 172.21.15.28:0/3024590661 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd380b2660 0x7fcd38154c70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.755+0000 7fcd3edee700 1 --2- 172.21.15.28:0/3024590661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd3815c560 0x7fcd38158480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.755+0000 7fcd3edee700 1 --2- 172.21.15.28:0/3024590661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcd380ceec0 0x7fcd3815ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.755+0000 7fcd3edee700 1 -- 172.21.15.28:0/3024590661 >> 172.21.15.28:0/3024590661 conn(0x7fcd380cd660 msgr2=0x7fcd380b3a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.755+0000 7fcd3edee700 1 -- 172.21.15.28:0/3024590661 shutdown_connections 2023-11-28T11:08:57.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.755+0000 7fcd3edee700 1 -- 172.21.15.28:0/3024590661 wait complete. 2023-11-28T11:08:57.758 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2023-11-28T11:08:57.768 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:08:57.768 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:08:57.786 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:08:57.951 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.949+0000 7fa514b9c700 1 Processor -- start 2023-11-28T11:08:57.951 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.949+0000 7fa514b9c700 1 -- start start 2023-11-28T11:08:57.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.950+0000 7fa514b9c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa51015a6c0 0x7fa5100cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.950+0000 7fa514b9c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa5100d0020 0x7fa5100ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.950+0000 7fa514b9c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5100cfb90 con 0x7fa5100b2660 2023-11-28T11:08:57.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.950+0000 7fa514b9c700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5100cb930 con 0x7fa51015a6c0 2023-11-28T11:08:57.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.950+0000 7fa514b9c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5100cf260 con 0x7fa5100d0020 2023-11-28T11:08:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.950+0000 7fa50e59c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa51015a6c0 0x7fa5100cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.951+0000 7fa50e59c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa51015a6c0 0x7fa5100cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46676/0 (socket says 172.21.15.28:46676) 2023-11-28T11:08:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.951+0000 7fa50e59c700 1 -- 172.21.15.28:0/2395352376 learned_addr learned my addr 172.21.15.28:0/2395352376 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.951+0000 7fa50dd9b700 1 --2- 172.21.15.28:0/2395352376 >> v2:172.21.15.59:3300/0 conn(0x7fa5100d0020 0x7fa5100ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.951+0000 7fa50e59c700 1 -- 172.21.15.28:0/2395352376 >> v2:172.21.15.59:3300/0 conn(0x7fa5100d0020 msgr2=0x7fa5100ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.951+0000 7fa50e59c700 1 --2- 172.21.15.28:0/2395352376 >> v2:172.21.15.59:3300/0 conn(0x7fa5100d0020 0x7fa5100ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.951+0000 7fa50e59c700 1 -- 172.21.15.28:0/2395352376 >> v1:172.21.15.59:6789/0 conn(0x7fa5100b2660 legacy=0x7fa5100b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.951+0000 7fa50e59c700 1 -- 172.21.15.28:0/2395352376 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5100cf540 con 0x7fa51015a6c0 2023-11-28T11:08:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.951+0000 7fa50e59c700 1 --2- 172.21.15.28:0/2395352376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa51015a6c0 0x7fa5100cb230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa51015dff0 tx=0x7fa50800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb17a355cdcd83 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.951+0000 7fa50d59a700 1 -- 172.21.15.28:0/2395352376 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50800e070 con 0x7fa51015a6c0 2023-11-28T11:08:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.951+0000 7fa50d59a700 1 -- 172.21.15.28:0/2395352376 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa50800b910 con 0x7fa51015a6c0 2023-11-28T11:08:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.951+0000 7fa50d59a700 1 -- 172.21.15.28:0/2395352376 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50800bcc0 con 0x7fa51015a6c0 2023-11-28T11:08:57.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.952+0000 7fa514b9c700 1 -- 172.21.15.28:0/2395352376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa51015a6c0 msgr2=0x7fa5100cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.952+0000 7fa514b9c700 1 --2- 172.21.15.28:0/2395352376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa51015a6c0 0x7fa5100cb230 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fa51015dff0 tx=0x7fa50800b330 comp rx=0 tx=0).stop 2023-11-28T11:08:57.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.952+0000 7fa514b9c700 1 -- 172.21.15.28:0/2395352376 shutdown_connections 2023-11-28T11:08:57.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.952+0000 7fa514b9c700 1 --2- 172.21.15.28:0/2395352376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa51015a6c0 0x7fa5100cb230 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.952+0000 7fa514b9c700 1 --2- 172.21.15.28:0/2395352376 >> v2:172.21.15.59:3300/0 conn(0x7fa5100d0020 0x7fa5100ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.952+0000 7fa514b9c700 1 -- 172.21.15.28:0/2395352376 >> 172.21.15.28:0/2395352376 conn(0x7fa5100cd660 msgr2=0x7fa5100cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:57.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.952+0000 7fa514b9c700 1 -- 172.21.15.28:0/2395352376 shutdown_connections 2023-11-28T11:08:57.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.952+0000 7fa514b9c700 1 -- 172.21.15.28:0/2395352376 wait complete. 2023-11-28T11:08:57.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.953+0000 7fa514b9c700 1 Processor -- start 2023-11-28T11:08:57.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.953+0000 7fa514b9c700 1 -- start start 2023-11-28T11:08:57.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa514b9c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5100b2660 0x7fa51015aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa514b9c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5100d0020 0x7fa51015afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa514b9c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa51015a6c0 0x7fa51015b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa514b9c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5100c9650 con 0x7fa5100b2660 2023-11-28T11:08:57.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa514b9c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5100ca6b0 con 0x7fa5100d0020 2023-11-28T11:08:57.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa514b9c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5100c98f0 con 0x7fa51015a6c0 2023-11-28T11:08:57.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa50e59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5100b2660 0x7fa51015aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa50e59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5100b2660 0x7fa51015aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46682/0 (socket says 172.21.15.28:46682) 2023-11-28T11:08:57.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa50e59c700 1 -- 172.21.15.28:0/1226862585 learned_addr learned my addr 172.21.15.28:0/1226862585 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:57.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa50dd9b700 1 --2- 172.21.15.28:0/1226862585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5100d0020 0x7fa51015afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa50e59c700 1 -- 172.21.15.28:0/1226862585 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa51015a6c0 msgr2=0x7fa51015b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa50ed9d700 1 --2- 172.21.15.28:0/1226862585 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa51015a6c0 0x7fa51015b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa50e59c700 1 --2- 172.21.15.28:0/1226862585 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa51015a6c0 0x7fa51015b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa50e59c700 1 -- 172.21.15.28:0/1226862585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5100d0020 msgr2=0x7fa51015afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:57.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa50e59c700 1 --2- 172.21.15.28:0/1226862585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5100d0020 0x7fa51015afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:57.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.954+0000 7fa50e59c700 1 -- 172.21.15.28:0/1226862585 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa508009b30 con 0x7fa5100b2660 2023-11-28T11:08:57.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.955+0000 7fa50e59c700 1 --2- 172.21.15.28:0/1226862585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5100b2660 0x7fa51015aa90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa508000f80 tx=0x7fa508004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.955+0000 7fa506ffd700 1 -- 172.21.15.28:0/1226862585 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5080092e0 con 0x7fa5100b2660 2023-11-28T11:08:57.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.955+0000 7fa506ffd700 1 -- 172.21.15.28:0/1226862585 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5080099b0 con 0x7fa5100b2660 2023-11-28T11:08:57.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.955+0000 7fa514b9c700 1 -- 172.21.15.28:0/1226862585 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5100cd3e0 con 0x7fa5100b2660 2023-11-28T11:08:57.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.955+0000 7fa506ffd700 1 -- 172.21.15.28:0/1226862585 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa50801b650 con 0x7fa5100b2660 2023-11-28T11:08:57.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.955+0000 7fa514b9c700 1 -- 172.21.15.28:0/1226862585 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5100cf820 con 0x7fa5100b2660 2023-11-28T11:08:57.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.956+0000 7fa514b9c700 1 -- 172.21.15.28:0/1226862585 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5100d0470 con 0x7fa5100b2660 2023-11-28T11:08:57.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.956+0000 7fa506ffd700 1 -- 172.21.15.28:0/1226862585 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa508021030 con 0x7fa5100b2660 2023-11-28T11:08:57.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.956+0000 7fa506ffd700 1 --2- 172.21.15.28:0/1226862585 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa4f00682f0 0x7fa4f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:57.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.957+0000 7fa506ffd700 1 -- 172.21.15.28:0/1226862585 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa5080874f0 con 0x7fa5100b2660 2023-11-28T11:08:57.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.957+0000 7fa50dd9b700 1 --2- 172.21.15.28:0/1226862585 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa4f00682f0 0x7fa4f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:57.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.957+0000 7fa50dd9b700 1 --2- 172.21.15.28:0/1226862585 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa4f00682f0 0x7fa4f006a7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa5100b0f00 tx=0x7fa4f8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:57.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:57.961+0000 7fa506ffd700 1 -- 172.21.15.28:0/1226862585 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa508053a60 con 0x7fa5100b2660 2023-11-28T11:08:58.160 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.159+0000 7fa514b9c700 1 -- 172.21.15.28:0/1226862585 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa510067150 con 0x7fa5100b2660 2023-11-28T11:08:58.162 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.160+0000 7fa506ffd700 1 -- 172.21.15.28:0/1226862585 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19642 (secure 0 0 0) 0x7fa50801f020 con 0x7fa5100b2660 2023-11-28T11:08:58.162 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:08:58.163 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:50.531733+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:08:58.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.162+0000 7fa514b9c700 1 -- 172.21.15.28:0/1226862585 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa4f00682f0 msgr2=0x7fa4f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.162+0000 7fa514b9c700 1 --2- 172.21.15.28:0/1226862585 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa4f00682f0 0x7fa4f006a7a0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa5100b0f00 tx=0x7fa4f8008040 comp rx=0 tx=0).stop 2023-11-28T11:08:58.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.162+0000 7fa514b9c700 1 -- 172.21.15.28:0/1226862585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5100b2660 msgr2=0x7fa51015aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.162+0000 7fa514b9c700 1 --2- 172.21.15.28:0/1226862585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5100b2660 0x7fa51015aa90 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa508000f80 tx=0x7fa508004ad0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.162+0000 7fa514b9c700 1 -- 172.21.15.28:0/1226862585 shutdown_connections 2023-11-28T11:08:58.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.162+0000 7fa514b9c700 1 --2- 172.21.15.28:0/1226862585 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa4f00682f0 0x7fa4f006a7a0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.162+0000 7fa514b9c700 1 --2- 172.21.15.28:0/1226862585 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa5100b2660 0x7fa51015aa90 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.162+0000 7fa514b9c700 1 --2- 172.21.15.28:0/1226862585 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa51015a6c0 0x7fa51015b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.162+0000 7fa514b9c700 1 --2- 172.21.15.28:0/1226862585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa5100d0020 0x7fa51015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.162+0000 7fa514b9c700 1 -- 172.21.15.28:0/1226862585 >> 172.21.15.28:0/1226862585 conn(0x7fa5100cd660 msgr2=0x7fa5100b1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.162+0000 7fa514b9c700 1 -- 172.21.15.28:0/1226862585 shutdown_connections 2023-11-28T11:08:58.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.162+0000 7fa514b9c700 1 -- 172.21.15.28:0/1226862585 wait complete. 2023-11-28T11:08:58.177 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:08:58.178 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:08:58.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.305+0000 7ff6186af700 1 Processor -- start 2023-11-28T11:08:58.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.305+0000 7ff6186af700 1 -- start start 2023-11-28T11:08:58.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.306+0000 7ff6186af700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100cf5f0 0x7ff6100caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.306+0000 7ff6186af700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff6100cb3e0 0x7ff61006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.306+0000 7ff6186af700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6100ce550 con 0x7ff6100b1630 2023-11-28T11:08:58.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.306+0000 7ff6186af700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6100cfa60 con 0x7ff6100cb3e0 2023-11-28T11:08:58.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.306+0000 7ff6186af700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100ce830 con 0x7ff6100cf5f0 2023-11-28T11:08:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.306+0000 7ff616c4c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff6100cb3e0 0x7ff61006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.307+0000 7ff616c4c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff6100cb3e0 0x7ff61006afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46694/0 (socket says 172.21.15.28:46694) 2023-11-28T11:08:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.307+0000 7ff616c4c700 1 -- 172.21.15.28:0/941403325 learned_addr learned my addr 172.21.15.28:0/941403325 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.307+0000 7ff615c4a700 1 --2- 172.21.15.28:0/941403325 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100cf5f0 0x7ff6100caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.307+0000 7ff616c4c700 1 -- 172.21.15.28:0/941403325 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100cf5f0 msgr2=0x7ff6100caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.307+0000 7ff616c4c700 1 --2- 172.21.15.28:0/941403325 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100cf5f0 0x7ff6100caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.307+0000 7ff616c4c700 1 -- 172.21.15.28:0/941403325 >> v1:172.21.15.59:6789/0 conn(0x7ff6100b1630 legacy=0x7ff6100b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.307+0000 7ff616c4c700 1 -- 172.21.15.28:0/941403325 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6100ceb10 con 0x7ff6100cb3e0 2023-11-28T11:08:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.307+0000 7ff616c4c700 1 --2- 172.21.15.28:0/941403325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6100cb3e0 0x7ff61006afc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff608009e50 tx=0x7ff60800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd0ab618404a4dbf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.307+0000 7ff615449700 1 -- 172.21.15.28:0/941403325 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60800e070 con 0x7ff6100cb3e0 2023-11-28T11:08:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.307+0000 7ff615449700 1 -- 172.21.15.28:0/941403325 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60800b910 con 0x7ff6100cb3e0 2023-11-28T11:08:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.307+0000 7ff615449700 1 -- 172.21.15.28:0/941403325 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60800bcc0 con 0x7ff6100cb3e0 2023-11-28T11:08:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.308+0000 7ff6186af700 1 -- 172.21.15.28:0/941403325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6100cb3e0 msgr2=0x7ff61006afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.308+0000 7ff6186af700 1 --2- 172.21.15.28:0/941403325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6100cb3e0 0x7ff61006afc0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff608009e50 tx=0x7ff60800b330 comp rx=0 tx=0).stop 2023-11-28T11:08:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.308+0000 7ff6186af700 1 -- 172.21.15.28:0/941403325 shutdown_connections 2023-11-28T11:08:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.308+0000 7ff6186af700 1 --2- 172.21.15.28:0/941403325 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6100cb3e0 0x7ff61006afc0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.308+0000 7ff6186af700 1 --2- 172.21.15.28:0/941403325 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100cf5f0 0x7ff6100caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.308+0000 7ff6186af700 1 -- 172.21.15.28:0/941403325 >> 172.21.15.28:0/941403325 conn(0x7ff6100cc630 msgr2=0x7ff6100cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.308+0000 7ff6186af700 1 -- 172.21.15.28:0/941403325 shutdown_connections 2023-11-28T11:08:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.308+0000 7ff6186af700 1 -- 172.21.15.28:0/941403325 wait complete. 2023-11-28T11:08:58.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.309+0000 7ff6186af700 1 Processor -- start 2023-11-28T11:08:58.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.309+0000 7ff6186af700 1 -- start start 2023-11-28T11:08:58.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff6186af700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6100b1630 0x7ff61015af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff6186af700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6100cb3e0 0x7ff61015a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff6186af700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100cf5f0 0x7ff6100c5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff6186af700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100c36e0 con 0x7ff6100cb3e0 2023-11-28T11:08:58.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff6186af700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61015d730 con 0x7ff6100b1630 2023-11-28T11:08:58.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff6186af700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61015a820 con 0x7ff6100cf5f0 2023-11-28T11:08:58.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff615c4a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6100cb3e0 0x7ff61015a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff615c4a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6100cb3e0 0x7ff61015a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46698/0 (socket says 172.21.15.28:46698) 2023-11-28T11:08:58.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff615c4a700 1 -- 172.21.15.28:0/3337784613 learned_addr learned my addr 172.21.15.28:0/3337784613 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:58.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff61644b700 1 --2- 172.21.15.28:0/3337784613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6100b1630 0x7ff61015af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff616c4c700 1 --2- 172.21.15.28:0/3337784613 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100cf5f0 0x7ff6100c5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff615c4a700 1 -- 172.21.15.28:0/3337784613 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100cf5f0 msgr2=0x7ff6100c5b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff615c4a700 1 --2- 172.21.15.28:0/3337784613 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100cf5f0 0x7ff6100c5b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff615c4a700 1 -- 172.21.15.28:0/3337784613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6100b1630 msgr2=0x7ff61015af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff615c4a700 1 --2- 172.21.15.28:0/3337784613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6100b1630 0x7ff61015af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff615c4a700 1 -- 172.21.15.28:0/3337784613 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff608009b30 con 0x7ff6100cb3e0 2023-11-28T11:08:58.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.310+0000 7ff615c4a700 1 --2- 172.21.15.28:0/3337784613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6100cb3e0 0x7ff61015a2e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00e8d0 tx=0x7ff60c00ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.311+0000 7ff606ffd700 1 -- 172.21.15.28:0/3337784613 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c009590 con 0x7ff6100cb3e0 2023-11-28T11:08:58.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.311+0000 7ff606ffd700 1 -- 172.21.15.28:0/3337784613 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff60c013680 con 0x7ff6100cb3e0 2023-11-28T11:08:58.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.311+0000 7ff6186af700 1 -- 172.21.15.28:0/3337784613 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6100cf0d0 con 0x7ff6100cb3e0 2023-11-28T11:08:58.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.311+0000 7ff606ffd700 1 -- 172.21.15.28:0/3337784613 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff60c0139f0 con 0x7ff6100cb3e0 2023-11-28T11:08:58.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.311+0000 7ff6186af700 1 -- 172.21.15.28:0/3337784613 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6100cedf0 con 0x7ff6100cb3e0 2023-11-28T11:08:58.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.312+0000 7ff6186af700 1 -- 172.21.15.28:0/3337784613 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6100ce140 con 0x7ff6100cb3e0 2023-11-28T11:08:58.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.312+0000 7ff606ffd700 1 -- 172.21.15.28:0/3337784613 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff60c019030 con 0x7ff6100cb3e0 2023-11-28T11:08:58.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.312+0000 7ff606ffd700 1 --2- 172.21.15.28:0/3337784613 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff5f40682f0 0x7ff5f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.313+0000 7ff606ffd700 1 -- 172.21.15.28:0/3337784613 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff60c085c30 con 0x7ff6100cb3e0 2023-11-28T11:08:58.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.313+0000 7ff61644b700 1 --2- 172.21.15.28:0/3337784613 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff5f40682f0 0x7ff5f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.313+0000 7ff61644b700 1 --2- 172.21.15.28:0/3337784613 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff5f40682f0 0x7ff5f406a7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff61015fc80 tx=0x7ff600009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.317+0000 7ff606ffd700 1 -- 172.21.15.28:0/3337784613 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff60c0521a0 con 0x7ff6100cb3e0 2023-11-28T11:08:58.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.507+0000 7ff6186af700 1 -- 172.21.15.28:0/3337784613 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff61015da10 con 0x7ff6100cb3e0 2023-11-28T11:08:58.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.508+0000 7ff606ffd700 1 -- 172.21.15.28:0/3337784613 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff60c0557c0 con 0x7ff6100cb3e0 2023-11-28T11:08:58.510 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:08:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.510+0000 7ff6186af700 1 -- 172.21.15.28:0/3337784613 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff5f40682f0 msgr2=0x7ff5f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.510+0000 7ff6186af700 1 --2- 172.21.15.28:0/3337784613 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff5f40682f0 0x7ff5f406a7a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff61015fc80 tx=0x7ff600009040 comp rx=0 tx=0).stop 2023-11-28T11:08:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.510+0000 7ff6186af700 1 -- 172.21.15.28:0/3337784613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6100cb3e0 msgr2=0x7ff61015a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.510+0000 7ff6186af700 1 --2- 172.21.15.28:0/3337784613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6100cb3e0 0x7ff61015a2e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ff60c00e8d0 tx=0x7ff60c00ebe0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.510+0000 7ff6186af700 1 -- 172.21.15.28:0/3337784613 shutdown_connections 2023-11-28T11:08:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.510+0000 7ff6186af700 1 --2- 172.21.15.28:0/3337784613 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff5f40682f0 0x7ff5f406a7a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.510+0000 7ff6186af700 1 --2- 172.21.15.28:0/3337784613 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100cf5f0 0x7ff6100c5b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.510+0000 7ff6186af700 1 --2- 172.21.15.28:0/3337784613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff6100b1630 0x7ff61015af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.510+0000 7ff6186af700 1 --2- 172.21.15.28:0/3337784613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff6100cb3e0 0x7ff61015a2e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.510+0000 7ff6186af700 1 -- 172.21.15.28:0/3337784613 >> 172.21.15.28:0/3337784613 conn(0x7ff6100cc630 msgr2=0x7ff6100b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.510+0000 7ff6186af700 1 -- 172.21.15.28:0/3337784613 shutdown_connections 2023-11-28T11:08:58.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.510+0000 7ff6186af700 1 -- 172.21.15.28:0/3337784613 wait complete. 2023-11-28T11:08:58.524 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:08:58.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.649+0000 7fb4e3b6d700 1 Processor -- start 2023-11-28T11:08:58.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.649+0000 7fb4e3b6d700 1 -- start start 2023-11-28T11:08:58.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.650+0000 7fb4e3b6d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4dc06dd40 0x7fb4dc06f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.650+0000 7fb4e3b6d700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4dc15a860 con 0x7fb4dc0b2150 2023-11-28T11:08:58.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.650+0000 7fb4e3b6d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4dc06b510 con 0x7fb4dc15a490 2023-11-28T11:08:58.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.650+0000 7fb4e3b6d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc0c9a80 con 0x7fb4dc06dd40 2023-11-28T11:08:58.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.650+0000 7fb4e1108700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb4dc0b2150 0x7fb4dc0ca520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34764/0 (socket says 172.21.15.28:34764) 2023-11-28T11:08:58.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.650+0000 7fb4e1108700 1 -- 172.21.15.28:0/3852016162 learned_addr learned my addr 172.21.15.28:0/3852016162 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:58.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e210a700 1 --2- 172.21.15.28:0/3852016162 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4dc06dd40 0x7fb4dc06f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4276529763 0 0) 0x7fb4dc15a860 con 0x7fb4dc0b2150 2023-11-28T11:08:58.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4c80035b0 con 0x7fb4dc0b2150 2023-11-28T11:08:58.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2938008926 0 0) 0x7fb4dc06b510 con 0x7fb4dc15a490 2023-11-28T11:08:58.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4dc15a860 con 0x7fb4dc15a490 2023-11-28T11:08:58.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3349925653 0 0) 0x7fb4c80035b0 con 0x7fb4dc0b2150 2023-11-28T11:08:58.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4dc06dd40 msgr2=0x7fb4dc06f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:58.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 --2- 172.21.15.28:0/3852016162 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4dc06dd40 0x7fb4dc06f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 >> v1:172.21.15.59:6789/0 conn(0x7fb4dc15a490 legacy=0x7fb4dc06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc06b230 con 0x7fb4dc0b2150 2023-11-28T11:08:58.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fb4d8001670 con 0x7fb4dc0b2150 2023-11-28T11:08:58.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e210a700 1 --2- 172.21.15.28:0/3852016162 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4dc06dd40 0x7fb4dc06f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:58.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 >> v1:172.21.15.28:6789/0 conn(0x7fb4dc0b2150 legacy=0x7fb4dc0ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 --2- 172.21.15.28:0/3852016162 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4dc06dd40 0x7fb4c8003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 --2- 172.21.15.28:0/3852016162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c8004150 0x7fb4c8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 --2- 172.21.15.28:0/3852016162 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4c8004d90 0x7fb4c8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc0c9a80 con 0x7fb4c8004d90 2023-11-28T11:08:58.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c800bba0 con 0x7fb4c8004150 2023-11-28T11:08:58.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4c800bd60 con 0x7fb4dc06dd40 2023-11-28T11:08:58.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e210a700 1 --2- 172.21.15.28:0/3852016162 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4c8004d90 0x7fb4c8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e1909700 1 --2- 172.21.15.28:0/3852016162 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4dc06dd40 0x7fb4c8003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e1108700 1 --2- 172.21.15.28:0/3852016162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c8004150 0x7fb4c8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e210a700 1 -- 172.21.15.28:0/3852016162 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4dc06dd40 msgr2=0x7fb4c8003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:58.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e210a700 1 --2- 172.21.15.28:0/3852016162 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4dc06dd40 0x7fb4c8003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e210a700 1 -- 172.21.15.28:0/3852016162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c8004150 msgr2=0x7fb4c8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:58.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e210a700 1 --2- 172.21.15.28:0/3852016162 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4c8004150 0x7fb4c8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.651+0000 7fb4e210a700 1 -- 172.21.15.28:0/3852016162 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d4009a70 con 0x7fb4c8004d90 2023-11-28T11:08:58.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.652+0000 7fb4e210a700 1 --2- 172.21.15.28:0/3852016162 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4c8004d90 0x7fb4c8005200 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0ae7b0 tx=0x7fb4c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=422fba240e0d84e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.652+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400edb0 con 0x7fb4c8004d90 2023-11-28T11:08:58.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.652+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d400f040 con 0x7fb4c8004d90 2023-11-28T11:08:58.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.652+0000 7fb4e0907700 1 -- 172.21.15.28:0/3852016162 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4003c40 con 0x7fb4c8004d90 2023-11-28T11:08:58.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.652+0000 7fb4e3b6d700 1 -- 172.21.15.28:0/3852016162 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4c8004d90 msgr2=0x7fb4c8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.652+0000 7fb4e3b6d700 1 --2- 172.21.15.28:0/3852016162 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4c8004d90 0x7fb4c8005200 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7fb4dc0ae7b0 tx=0x7fb4c8005880 comp rx=0 tx=0).stop 2023-11-28T11:08:58.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.652+0000 7fb4e210a700 1 -- 172.21.15.28:0/3852016162 reap_dead start 2023-11-28T11:08:58.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.652+0000 7fb4e3b6d700 1 -- 172.21.15.28:0/3852016162 shutdown_connections 2023-11-28T11:08:58.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.652+0000 7fb4e3b6d700 1 -- 172.21.15.28:0/3852016162 >> 172.21.15.28:0/3852016162 conn(0x7fb4dc0b0df0 msgr2=0x7fb4dc0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.652+0000 7fb4e3b6d700 1 -- 172.21.15.28:0/3852016162 shutdown_connections 2023-11-28T11:08:58.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.652+0000 7fb4e3b6d700 1 -- 172.21.15.28:0/3852016162 wait complete. 2023-11-28T11:08:58.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.654+0000 7fb4e3b6d700 1 Processor -- start 2023-11-28T11:08:58.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.654+0000 7fb4e3b6d700 1 -- start start 2023-11-28T11:08:58.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e3b6d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4dc0b4170 0x7fb4dc157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e3b6d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4dc1f01a0 0x7fb4dc0c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e3b6d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4dc0b4540 0x7fb4dc0c4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e3b6d700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc0c7320 con 0x7fb4dc1f01a0 2023-11-28T11:08:58.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e3b6d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc0c7a80 con 0x7fb4dc0b4170 2023-11-28T11:08:58.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e3b6d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc0c7d20 con 0x7fb4dc0b4540 2023-11-28T11:08:58.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e1108700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4dc1f01a0 0x7fb4dc0c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e1108700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4dc1f01a0 0x7fb4dc0c6de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46718/0 (socket says 172.21.15.28:46718) 2023-11-28T11:08:58.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e1108700 1 -- 172.21.15.28:0/1834821785 learned_addr learned my addr 172.21.15.28:0/1834821785 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:58.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e210a700 1 --2- 172.21.15.28:0/1834821785 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4dc0b4540 0x7fb4dc0c4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e1909700 1 --2- 172.21.15.28:0/1834821785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4dc0b4170 0x7fb4dc157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e1108700 1 -- 172.21.15.28:0/1834821785 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4dc0b4540 msgr2=0x7fb4dc0c4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e1108700 1 --2- 172.21.15.28:0/1834821785 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4dc0b4540 0x7fb4dc0c4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e1108700 1 -- 172.21.15.28:0/1834821785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4dc0b4170 msgr2=0x7fb4dc157600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e1108700 1 --2- 172.21.15.28:0/1834821785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4dc0b4170 0x7fb4dc157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e1108700 1 -- 172.21.15.28:0/1834821785 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4dc0ca000 con 0x7fb4dc1f01a0 2023-11-28T11:08:58.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.655+0000 7fb4e210a700 1 --2- 172.21.15.28:0/1834821785 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4dc0b4540 0x7fb4dc0c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:08:58.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.656+0000 7fb4e1108700 1 --2- 172.21.15.28:0/1834821785 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4dc1f01a0 0x7fb4dc0c6de0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8007060 tx=0x7fb4d8007420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.656+0000 7fb4d27fc700 1 -- 172.21.15.28:0/1834821785 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d800f5e0 con 0x7fb4dc1f01a0 2023-11-28T11:08:58.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.656+0000 7fb4d27fc700 1 -- 172.21.15.28:0/1834821785 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4d800f780 con 0x7fb4dc1f01a0 2023-11-28T11:08:58.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.656+0000 7fb4e3b6d700 1 -- 172.21.15.28:0/1834821785 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4dc0c9d20 con 0x7fb4dc1f01a0 2023-11-28T11:08:58.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.656+0000 7fb4d27fc700 1 -- 172.21.15.28:0/1834821785 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d800fb30 con 0x7fb4dc1f01a0 2023-11-28T11:08:58.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.656+0000 7fb4e3b6d700 1 -- 172.21.15.28:0/1834821785 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4dc06ae90 con 0x7fb4dc1f01a0 2023-11-28T11:08:58.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.657+0000 7fb4e3b6d700 1 -- 172.21.15.28:0/1834821785 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4dc0739f0 con 0x7fb4dc1f01a0 2023-11-28T11:08:58.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.657+0000 7fb4d27fc700 1 -- 172.21.15.28:0/1834821785 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb4d800fc90 con 0x7fb4dc1f01a0 2023-11-28T11:08:58.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.658+0000 7fb4d27fc700 1 --2- 172.21.15.28:0/1834821785 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb4c00682f0 0x7fb4c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:58.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.658+0000 7fb4d27fc700 1 -- 172.21.15.28:0/1834821785 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb4d80860e0 con 0x7fb4dc1f01a0 2023-11-28T11:08:58.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.658+0000 7fb4e1909700 1 --2- 172.21.15.28:0/1834821785 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb4c00682f0 0x7fb4c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:58.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.658+0000 7fb4e1909700 1 --2- 172.21.15.28:0/1834821785 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb4c00682f0 0x7fb4c006a7a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc000c00 tx=0x7fb4cc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:58.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.662+0000 7fb4d27fc700 1 -- 172.21.15.28:0/1834821785 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb4d80525d0 con 0x7fb4dc1f01a0 2023-11-28T11:08:58.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.853+0000 7fb4e3b6d700 1 -- 172.21.15.28:0/1834821785 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb4dc0cba50 con 0x7fb4dc1f01a0 2023-11-28T11:08:58.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.853+0000 7fb4d27fc700 1 -- 172.21.15.28:0/1834821785 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb4d8055bf0 con 0x7fb4dc1f01a0 2023-11-28T11:08:58.855 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:08:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.855+0000 7fb4e3b6d700 1 -- 172.21.15.28:0/1834821785 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb4c00682f0 msgr2=0x7fb4c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.855+0000 7fb4e3b6d700 1 --2- 172.21.15.28:0/1834821785 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb4c00682f0 0x7fb4c006a7a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc000c00 tx=0x7fb4cc009040 comp rx=0 tx=0).stop 2023-11-28T11:08:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.855+0000 7fb4e3b6d700 1 -- 172.21.15.28:0/1834821785 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4dc1f01a0 msgr2=0x7fb4dc0c6de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.855+0000 7fb4e3b6d700 1 --2- 172.21.15.28:0/1834821785 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4dc1f01a0 0x7fb4dc0c6de0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fb4d8007060 tx=0x7fb4d8007420 comp rx=0 tx=0).stop 2023-11-28T11:08:58.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.855+0000 7fb4e3b6d700 1 -- 172.21.15.28:0/1834821785 shutdown_connections 2023-11-28T11:08:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.855+0000 7fb4e3b6d700 1 --2- 172.21.15.28:0/1834821785 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb4c00682f0 0x7fb4c006a7a0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.855+0000 7fb4e3b6d700 1 --2- 172.21.15.28:0/1834821785 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb4dc0b4540 0x7fb4dc0c4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.855+0000 7fb4e3b6d700 1 --2- 172.21.15.28:0/1834821785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4dc0b4170 0x7fb4dc157600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.855+0000 7fb4e3b6d700 1 --2- 172.21.15.28:0/1834821785 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4dc1f01a0 0x7fb4dc0c6de0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.855+0000 7fb4e3b6d700 1 -- 172.21.15.28:0/1834821785 >> 172.21.15.28:0/1834821785 conn(0x7fb4dc0b0df0 msgr2=0x7fb4dc0b2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.855+0000 7fb4e3b6d700 1 -- 172.21.15.28:0/1834821785 shutdown_connections 2023-11-28T11:08:58.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.856+0000 7fb4e3b6d700 1 -- 172.21.15.28:0/1834821785 wait complete. 2023-11-28T11:08:58.869 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:08:58.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.996+0000 7f6916106700 1 Processor -- start 2023-11-28T11:08:58.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.996+0000 7f6916106700 1 -- start start 2023-11-28T11:08:59.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.997+0000 7f6916106700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69100b2af0 0x7f69100cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.997+0000 7f6916106700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691006f940 con 0x7f69100b2680 2023-11-28T11:08:59.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.997+0000 7f6916106700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f691006fc20 con 0x7f691015fc00 2023-11-28T11:08:59.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.997+0000 7f6916106700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69100cb3f0 con 0x7f69100b2af0 2023-11-28T11:08:59.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.997+0000 7f690effd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f69100b2680 0x7f69100cbbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34784/0 (socket says 172.21.15.28:34784) 2023-11-28T11:08:59.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.997+0000 7f690effd700 1 -- 172.21.15.28:0/2761935442 learned_addr learned my addr 172.21.15.28:0/2761935442 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:59.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690ffff700 1 --2- 172.21.15.28:0/2761935442 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69100b2af0 0x7f69100cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690e7fc700 1 -- 172.21.15.28:0/2761935442 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3301765505 0 0) 0x7f691006f940 con 0x7f69100b2680 2023-11-28T11:08:59.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690e7fc700 1 -- 172.21.15.28:0/2761935442 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68fc0035b0 con 0x7f69100b2680 2023-11-28T11:08:59.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690e7fc700 1 -- 172.21.15.28:0/2761935442 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2078004131 0 0) 0x7f68fc0035b0 con 0x7f69100b2680 2023-11-28T11:08:59.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690e7fc700 1 -- 172.21.15.28:0/2761935442 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69100b2af0 msgr2=0x7f69100cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:59.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690e7fc700 1 --2- 172.21.15.28:0/2761935442 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69100b2af0 0x7f69100cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690e7fc700 1 -- 172.21.15.28:0/2761935442 >> v1:172.21.15.59:6789/0 conn(0x7f691015fc00 legacy=0x7f691006eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690e7fc700 1 -- 172.21.15.28:0/2761935442 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69100cb6d0 con 0x7f69100b2680 2023-11-28T11:08:59.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690e7fc700 1 -- 172.21.15.28:0/2761935442 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f68f8001670 con 0x7f69100b2680 2023-11-28T11:08:59.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690e7fc700 1 -- 172.21.15.28:0/2761935442 >> v1:172.21.15.28:6789/0 conn(0x7f69100b2680 legacy=0x7f69100cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690e7fc700 1 --2- 172.21.15.28:0/2761935442 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68fc003a30 0x7f68fc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690e7fc700 1 --2- 172.21.15.28:0/2761935442 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69100b2af0 0x7f68fc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690e7fc700 1 --2- 172.21.15.28:0/2761935442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68fc004d90 0x7f68fc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690e7fc700 1 -- 172.21.15.28:0/2761935442 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69100cb3f0 con 0x7f68fc003a30 2023-11-28T11:08:59.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690e7fc700 1 -- 172.21.15.28:0/2761935442 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc00ba10 con 0x7f68fc004d90 2023-11-28T11:08:59.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690e7fc700 1 -- 172.21.15.28:0/2761935442 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68fc00bba0 con 0x7f69100b2af0 2023-11-28T11:08:59.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690f7fe700 1 --2- 172.21.15.28:0/2761935442 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68fc003a30 0x7f68fc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690effd700 1 --2- 172.21.15.28:0/2761935442 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69100b2af0 0x7f68fc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690ffff700 1 --2- 172.21.15.28:0/2761935442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68fc004d90 0x7f68fc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690f7fe700 1 -- 172.21.15.28:0/2761935442 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69100b2af0 msgr2=0x7f68fc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:59.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690f7fe700 1 --2- 172.21.15.28:0/2761935442 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69100b2af0 0x7f68fc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690f7fe700 1 -- 172.21.15.28:0/2761935442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68fc004d90 msgr2=0x7f68fc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:59.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690f7fe700 1 --2- 172.21.15.28:0/2761935442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68fc004d90 0x7f68fc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690f7fe700 1 -- 172.21.15.28:0/2761935442 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68fc0035b0 con 0x7f68fc003a30 2023-11-28T11:08:59.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690effd700 1 --2- 172.21.15.28:0/2761935442 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69100b2af0 0x7f68fc004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:59.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.998+0000 7f690f7fe700 1 --2- 172.21.15.28:0/2761935442 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68fc003a30 0x7f68fc003e00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f69000018b0 tx=0x7f68fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35530904679ea2cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.999+0000 7f690e7fc700 1 -- 172.21.15.28:0/2761935442 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900001370 con 0x7f68fc003a30 2023-11-28T11:08:59.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.999+0000 7f690e7fc700 1 -- 172.21.15.28:0/2761935442 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690000c6b0 con 0x7f68fc003a30 2023-11-28T11:08:59.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.999+0000 7f690e7fc700 1 -- 172.21.15.28:0/2761935442 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f690000ca60 con 0x7f68fc003a30 2023-11-28T11:08:59.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.999+0000 7f6916106700 1 -- 172.21.15.28:0/2761935442 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68fc003a30 msgr2=0x7f68fc003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.999+0000 7f6916106700 1 --2- 172.21.15.28:0/2761935442 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68fc003a30 0x7f68fc003e00 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f69000018b0 tx=0x7f68fc005880 comp rx=0 tx=0).stop 2023-11-28T11:08:59.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.999+0000 7f690ffff700 1 -- 172.21.15.28:0/2761935442 reap_dead start 2023-11-28T11:08:59.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.999+0000 7f6916106700 1 -- 172.21.15.28:0/2761935442 shutdown_connections 2023-11-28T11:08:59.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.999+0000 7f6916106700 1 -- 172.21.15.28:0/2761935442 >> 172.21.15.28:0/2761935442 conn(0x7f69100cd680 msgr2=0x7f69100cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.999+0000 7f6916106700 1 -- 172.21.15.28:0/2761935442 shutdown_connections 2023-11-28T11:08:59.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:58.999+0000 7f6916106700 1 -- 172.21.15.28:0/2761935442 wait complete. 2023-11-28T11:08:59.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.000+0000 7f6916106700 1 Processor -- start 2023-11-28T11:08:59.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.001+0000 7f6916106700 1 -- start start 2023-11-28T11:08:59.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.001+0000 7f6916106700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68fc003a30 0x7f69101ef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.001+0000 7f6916106700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69101f2f80 0x7f69100c46f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.001+0000 7f6916106700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f691015e160 0x7f691015e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.001+0000 7f6916106700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f691015eb40 con 0x7f68fc003a30 2023-11-28T11:08:59.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.001+0000 7f6916106700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69100c6090 con 0x7f691015e160 2023-11-28T11:08:59.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.001+0000 7f6916106700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69100c4ff0 con 0x7f69101f2f80 2023-11-28T11:08:59.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.001+0000 7f690f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68fc003a30 0x7f69101ef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.001+0000 7f690f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68fc003a30 0x7f69101ef8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46736/0 (socket says 172.21.15.28:46736) 2023-11-28T11:08:59.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.001+0000 7f690f7fe700 1 -- 172.21.15.28:0/4200085018 learned_addr learned my addr 172.21.15.28:0/4200085018 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:59.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.002+0000 7f690ffff700 1 --2- 172.21.15.28:0/4200085018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f691015e160 0x7f691015e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.002+0000 7f690effd700 1 --2- 172.21.15.28:0/4200085018 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69101f2f80 0x7f69100c46f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.002+0000 7f690f7fe700 1 -- 172.21.15.28:0/4200085018 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69101f2f80 msgr2=0x7f69100c46f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.002+0000 7f690f7fe700 1 --2- 172.21.15.28:0/4200085018 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69101f2f80 0x7f69100c46f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.002+0000 7f690f7fe700 1 -- 172.21.15.28:0/4200085018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f691015e160 msgr2=0x7f691015e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.002+0000 7f690f7fe700 1 --2- 172.21.15.28:0/4200085018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f691015e160 0x7f691015e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.002+0000 7f690f7fe700 1 -- 172.21.15.28:0/4200085018 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f691015ffd0 con 0x7f68fc003a30 2023-11-28T11:08:59.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.002+0000 7f690ffff700 1 --2- 172.21.15.28:0/4200085018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f691015e160 0x7f691015e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:59.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.002+0000 7f690f7fe700 1 --2- 172.21.15.28:0/4200085018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68fc003a30 0x7f69101ef8a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f690000c460 tx=0x7f6900001110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.002+0000 7f6907fff700 1 -- 172.21.15.28:0/4200085018 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900001370 con 0x7f68fc003a30 2023-11-28T11:08:59.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.002+0000 7f6907fff700 1 -- 172.21.15.28:0/4200085018 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f690000cc30 con 0x7f68fc003a30 2023-11-28T11:08:59.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.002+0000 7f6907fff700 1 -- 172.21.15.28:0/4200085018 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900003680 con 0x7f68fc003a30 2023-11-28T11:08:59.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.002+0000 7f6916106700 1 -- 172.21.15.28:0/4200085018 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69100b2f50 con 0x7f68fc003a30 2023-11-28T11:08:59.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.002+0000 7f6916106700 1 -- 172.21.15.28:0/4200085018 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f691006f5b0 con 0x7f68fc003a30 2023-11-28T11:08:59.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.003+0000 7f6916106700 1 -- 172.21.15.28:0/4200085018 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69100c5ee0 con 0x7f68fc003a30 2023-11-28T11:08:59.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.004+0000 7f6907fff700 1 -- 172.21.15.28:0/4200085018 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6900010050 con 0x7f68fc003a30 2023-11-28T11:08:59.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.004+0000 7f6907fff700 1 --2- 172.21.15.28:0/4200085018 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68f00682f0 0x7f68f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.004+0000 7f6907fff700 1 -- 172.21.15.28:0/4200085018 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6900087730 con 0x7f68fc003a30 2023-11-28T11:08:59.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.004+0000 7f690effd700 1 --2- 172.21.15.28:0/4200085018 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68f00682f0 0x7f68f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.004+0000 7f690effd700 1 --2- 172.21.15.28:0/4200085018 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68f00682f0 0x7f68f006a7a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f691015ded0 tx=0x7f68f8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.008+0000 7f6907fff700 1 -- 172.21.15.28:0/4200085018 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6900053ca0 con 0x7f68fc003a30 2023-11-28T11:08:59.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.203+0000 7f6916106700 1 -- 172.21.15.28:0/4200085018 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f69100739f0 con 0x7f68fc003a30 2023-11-28T11:08:59.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.204+0000 7f6907fff700 1 -- 172.21.15.28:0/4200085018 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f690002b020 con 0x7f68fc003a30 2023-11-28T11:08:59.205 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:08:59.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.205+0000 7f6916106700 1 -- 172.21.15.28:0/4200085018 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68f00682f0 msgr2=0x7f68f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.205+0000 7f6916106700 1 --2- 172.21.15.28:0/4200085018 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68f00682f0 0x7f68f006a7a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f691015ded0 tx=0x7f68f8009040 comp rx=0 tx=0).stop 2023-11-28T11:08:59.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.205+0000 7f6916106700 1 -- 172.21.15.28:0/4200085018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68fc003a30 msgr2=0x7f69101ef8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.205+0000 7f6916106700 1 --2- 172.21.15.28:0/4200085018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68fc003a30 0x7f69101ef8a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f690000c460 tx=0x7f6900001110 comp rx=0 tx=0).stop 2023-11-28T11:08:59.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.205+0000 7f6916106700 1 -- 172.21.15.28:0/4200085018 shutdown_connections 2023-11-28T11:08:59.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.205+0000 7f6916106700 1 --2- 172.21.15.28:0/4200085018 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68f00682f0 0x7f68f006a7a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.206+0000 7f6916106700 1 --2- 172.21.15.28:0/4200085018 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68fc003a30 0x7f69101ef8a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.206+0000 7f6916106700 1 --2- 172.21.15.28:0/4200085018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f691015e160 0x7f691015e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.206+0000 7f6916106700 1 --2- 172.21.15.28:0/4200085018 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f69101f2f80 0x7f69100c46f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.206+0000 7f6916106700 1 -- 172.21.15.28:0/4200085018 >> 172.21.15.28:0/4200085018 conn(0x7f69100cd680 msgr2=0x7f6910160160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.206+0000 7f6916106700 1 -- 172.21.15.28:0/4200085018 shutdown_connections 2023-11-28T11:08:59.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.206+0000 7f6916106700 1 -- 172.21.15.28:0/4200085018 wait complete. 2023-11-28T11:08:59.219 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:08:59.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.345+0000 7f97089b6700 1 Processor -- start 2023-11-28T11:08:59.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.346+0000 7f97089b6700 1 -- start start 2023-11-28T11:08:59.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.346+0000 7f97089b6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f970415c950 0x7f970406a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.346+0000 7f97089b6700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97040caab0 con 0x7f97040cae50 2023-11-28T11:08:59.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.346+0000 7f97089b6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f970406fc00 con 0x7f97040ca360 2023-11-28T11:08:59.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.346+0000 7f97089b6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f970415cd20 con 0x7f970415c950 2023-11-28T11:08:59.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f9702d9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f97040cae50 0x7f97040cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34806/0 (socket says 172.21.15.28:34806) 2023-11-28T11:08:59.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f9702d9d700 1 -- 172.21.15.28:0/3012803219 learned_addr learned my addr 172.21.15.28:0/3012803219 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:59.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970259c700 1 --2- 172.21.15.28:0/3012803219 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f970415c950 0x7f970406a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2679959492 0 0) 0x7f97040caab0 con 0x7f97040cae50 2023-11-28T11:08:59.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f96f00035b0 con 0x7f97040cae50 2023-11-28T11:08:59.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3221410276 0 0) 0x7f970406fc00 con 0x7f97040ca360 2023-11-28T11:08:59.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97040caab0 con 0x7f97040ca360 2023-11-28T11:08:59.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3446244055 0 0) 0x7f96f00035b0 con 0x7f97040cae50 2023-11-28T11:08:59.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f970415c950 msgr2=0x7f970406a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:59.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 --2- 172.21.15.28:0/3012803219 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f970415c950 0x7f970406a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 >> v1:172.21.15.59:6789/0 conn(0x7f97040ca360 legacy=0x7f970406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970406e770 con 0x7f97040cae50 2023-11-28T11:08:59.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f96fc0019f0 con 0x7f97040cae50 2023-11-28T11:08:59.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970259c700 1 --2- 172.21.15.28:0/3012803219 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f970415c950 0x7f970406a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:59.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 >> v1:172.21.15.28:6789/0 conn(0x7f97040cae50 legacy=0x7f97040cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 --2- 172.21.15.28:0/3012803219 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f970415c950 0x7f96f0003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 --2- 172.21.15.28:0/3012803219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f0004100 0x7f96f0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 --2- 172.21.15.28:0/3012803219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96f0004d90 0x7f96f0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f970415cd20 con 0x7f96f0004d90 2023-11-28T11:08:59.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96f000bac0 con 0x7f96f0004100 2023-11-28T11:08:59.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96f000bc80 con 0x7f970415c950 2023-11-28T11:08:59.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.347+0000 7f9702d9d700 1 --2- 172.21.15.28:0/3012803219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96f0004d90 0x7f96f0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.348+0000 7f970259c700 1 --2- 172.21.15.28:0/3012803219 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f970415c950 0x7f96f0003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.348+0000 7f9701d9b700 1 --2- 172.21.15.28:0/3012803219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f0004100 0x7f96f0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.348+0000 7f9702d9d700 1 -- 172.21.15.28:0/3012803219 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f970415c950 msgr2=0x7f96f0003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:59.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.348+0000 7f9702d9d700 1 --2- 172.21.15.28:0/3012803219 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f970415c950 0x7f96f0003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.348+0000 7f9702d9d700 1 -- 172.21.15.28:0/3012803219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f0004100 msgr2=0x7f96f0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:59.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.348+0000 7f9702d9d700 1 --2- 172.21.15.28:0/3012803219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f0004100 0x7f96f0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.348+0000 7f9702d9d700 1 -- 172.21.15.28:0/3012803219 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96f40094a0 con 0x7f96f0004d90 2023-11-28T11:08:59.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.348+0000 7f9702d9d700 1 --2- 172.21.15.28:0/3012803219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96f0004d90 0x7f96f0005200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f96fc002230 tx=0x7f96f0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17671fdf091ead1e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.348+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc012600 con 0x7f96f0004d90 2023-11-28T11:08:59.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.348+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96fc012780 con 0x7f96f0004d90 2023-11-28T11:08:59.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.348+0000 7f970159a700 1 -- 172.21.15.28:0/3012803219 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96fc012b30 con 0x7f96f0004d90 2023-11-28T11:08:59.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.349+0000 7f97089b6700 1 -- 172.21.15.28:0/3012803219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96f0004d90 msgr2=0x7f96f0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.349+0000 7f97089b6700 1 --2- 172.21.15.28:0/3012803219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f96f0004d90 0x7f96f0005200 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f96fc002230 tx=0x7f96f0005880 comp rx=0 tx=0).stop 2023-11-28T11:08:59.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.349+0000 7f9702d9d700 1 -- 172.21.15.28:0/3012803219 reap_dead start 2023-11-28T11:08:59.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.349+0000 7f97089b6700 1 -- 172.21.15.28:0/3012803219 shutdown_connections 2023-11-28T11:08:59.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.349+0000 7f97089b6700 1 -- 172.21.15.28:0/3012803219 >> 172.21.15.28:0/3012803219 conn(0x7f97040b0df0 msgr2=0x7f97040b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.349+0000 7f97089b6700 1 -- 172.21.15.28:0/3012803219 shutdown_connections 2023-11-28T11:08:59.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.349+0000 7f97089b6700 1 -- 172.21.15.28:0/3012803219 wait complete. 2023-11-28T11:08:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.350+0000 7f97089b6700 1 Processor -- start 2023-11-28T11:08:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.350+0000 7f97089b6700 1 -- start start 2023-11-28T11:08:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f97089b6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f96f0004d90 0x7f97040c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f97089b6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97040ae200 0x7f97040c8ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f97089b6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97041570c0 0x7f97040b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f97089b6700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97040ac4d0 con 0x7f97040ae200 2023-11-28T11:08:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f97089b6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97040ab8c0 con 0x7f97041570c0 2023-11-28T11:08:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f97089b6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97040abba0 con 0x7f96f0004d90 2023-11-28T11:08:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f9701d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97040ae200 0x7f97040c8ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f9701d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97040ae200 0x7f97040c8ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46758/0 (socket says 172.21.15.28:46758) 2023-11-28T11:08:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f9701d9b700 1 -- 172.21.15.28:0/4204527482 learned_addr learned my addr 172.21.15.28:0/4204527482 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f970259c700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f96f0004d90 0x7f97040c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f9702d9d700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97041570c0 0x7f97040b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f9701d9b700 1 -- 172.21.15.28:0/4204527482 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f96f0004d90 msgr2=0x7f97040c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f9701d9b700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f96f0004d90 0x7f97040c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f9701d9b700 1 -- 172.21.15.28:0/4204527482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97041570c0 msgr2=0x7f97040b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f9701d9b700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97041570c0 0x7f97040b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f9701d9b700 1 -- 172.21.15.28:0/4204527482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f970406ea10 con 0x7f97040ae200 2023-11-28T11:08:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f9702d9d700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97041570c0 0x7f97040b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f970259c700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f96f0004d90 0x7f97040c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.351+0000 7f9701d9b700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97040ae200 0x7f97040c8ae0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00f7e0 tx=0x7f96ec00fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.352+0000 7f96faffd700 1 -- 172.21.15.28:0/4204527482 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec001520 con 0x7f97040ae200 2023-11-28T11:08:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.352+0000 7f96faffd700 1 -- 172.21.15.28:0/4204527482 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96ec0052e0 con 0x7f97040ae200 2023-11-28T11:08:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.352+0000 7f97089b6700 1 -- 172.21.15.28:0/4204527482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97040ca7d0 con 0x7f97040ae200 2023-11-28T11:08:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.352+0000 7f96faffd700 1 -- 172.21.15.28:0/4204527482 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96ec003680 con 0x7f97040ae200 2023-11-28T11:08:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.352+0000 7f97089b6700 1 -- 172.21.15.28:0/4204527482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f970406ecf0 con 0x7f97040ae200 2023-11-28T11:08:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.353+0000 7f97089b6700 1 -- 172.21.15.28:0/4204527482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97040ac640 con 0x7f97040ae200 2023-11-28T11:08:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.353+0000 7f96faffd700 1 -- 172.21.15.28:0/4204527482 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f96ec003800 con 0x7f97040ae200 2023-11-28T11:08:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.353+0000 7f96faffd700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f96e40682f0 0x7f96e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.353+0000 7f96faffd700 1 -- 172.21.15.28:0/4204527482 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f96ec086030 con 0x7f97040ae200 2023-11-28T11:08:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.353+0000 7f970259c700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f96e40682f0 0x7f96e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.354+0000 7f970259c700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f96e40682f0 0x7f96e406a7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97040c9540 tx=0x7f96f4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.358+0000 7f96faffd700 1 -- 172.21.15.28:0/4204527482 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f96ec0525a0 con 0x7f97040ae200 2023-11-28T11:08:59.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.549+0000 7f97089b6700 1 -- 172.21.15.28:0/4204527482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f97040739f0 con 0x7f97040ae200 2023-11-28T11:08:59.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.549+0000 7f96faffd700 1 -- 172.21.15.28:0/4204527482 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f96ec055bc0 con 0x7f97040ae200 2023-11-28T11:08:59.553 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:08:59.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.551+0000 7f97089b6700 1 -- 172.21.15.28:0/4204527482 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f96e40682f0 msgr2=0x7f96e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.551+0000 7f97089b6700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f96e40682f0 0x7f96e406a7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f97040c9540 tx=0x7f96f4008040 comp rx=0 tx=0).stop 2023-11-28T11:08:59.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.551+0000 7f97089b6700 1 -- 172.21.15.28:0/4204527482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97040ae200 msgr2=0x7f97040c8ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.551+0000 7f97089b6700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97040ae200 0x7f97040c8ae0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f96ec00f7e0 tx=0x7f96ec00fba0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.551+0000 7f97089b6700 1 -- 172.21.15.28:0/4204527482 shutdown_connections 2023-11-28T11:08:59.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.551+0000 7f97089b6700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f96e40682f0 0x7f96e406a7a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.551+0000 7f97089b6700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f97041570c0 0x7f97040b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.551+0000 7f97089b6700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f96f0004d90 0x7f97040c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.551+0000 7f97089b6700 1 --2- 172.21.15.28:0/4204527482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f97040ae200 0x7f97040c8ae0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.551+0000 7f97089b6700 1 -- 172.21.15.28:0/4204527482 >> 172.21.15.28:0/4204527482 conn(0x7f97040b0df0 msgr2=0x7f97040b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.551+0000 7f97089b6700 1 -- 172.21.15.28:0/4204527482 shutdown_connections 2023-11-28T11:08:59.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.551+0000 7f97089b6700 1 -- 172.21.15.28:0/4204527482 wait complete. 2023-11-28T11:08:59.565 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:08:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.694+0000 7fc7e20bf700 1 Processor -- start 2023-11-28T11:08:59.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.695+0000 7fc7e20bf700 1 -- start start 2023-11-28T11:08:59.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.695+0000 7fc7e20bf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7dc0b2150 0x7fc7dc06f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.695+0000 7fc7e20bf700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7dc15a860 con 0x7fc7dc15a490 2023-11-28T11:08:59.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.695+0000 7fc7e20bf700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7dc06b510 con 0x7fc7dc06dd40 2023-11-28T11:08:59.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.695+0000 7fc7e20bf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc0c9a80 con 0x7fc7dc0b2150 2023-11-28T11:08:59.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.695+0000 7fc7db7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc7dc15a490 0x7fc7dc06d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34822/0 (socket says 172.21.15.28:34822) 2023-11-28T11:08:59.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7db7fe700 1 -- 172.21.15.28:0/349358174 learned_addr learned my addr 172.21.15.28:0/349358174 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:08:59.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7daffd700 1 --2- 172.21.15.28:0/349358174 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7dc0b2150 0x7fc7dc06f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 593949118 0 0) 0x7fc7dc15a860 con 0x7fc7dc15a490 2023-11-28T11:08:59.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7c80035b0 con 0x7fc7dc15a490 2023-11-28T11:08:59.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3412510681 0 0) 0x7fc7dc06b510 con 0x7fc7dc06dd40 2023-11-28T11:08:59.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7dc15a860 con 0x7fc7dc06dd40 2023-11-28T11:08:59.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2911704694 0 0) 0x7fc7c80035b0 con 0x7fc7dc15a490 2023-11-28T11:08:59.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7dc0b2150 msgr2=0x7fc7dc06f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:59.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 --2- 172.21.15.28:0/349358174 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7dc0b2150 0x7fc7dc06f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 >> v1:172.21.15.59:6789/0 conn(0x7fc7dc06dd40 legacy=0x7fc7dc0ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc06b230 con 0x7fc7dc15a490 2023-11-28T11:08:59.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fc7c4001670 con 0x7fc7dc15a490 2023-11-28T11:08:59.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7daffd700 1 --2- 172.21.15.28:0/349358174 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7dc0b2150 0x7fc7dc06f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:08:59.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 >> v1:172.21.15.28:6789/0 conn(0x7fc7dc15a490 legacy=0x7fc7dc06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 --2- 172.21.15.28:0/349358174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7c8003bc0 0x7fc7c8003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 --2- 172.21.15.28:0/349358174 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7dc0b2150 0x7fc7c8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 --2- 172.21.15.28:0/349358174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7c8004d90 0x7fc7c8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc0c9a80 con 0x7fc7c8003bc0 2023-11-28T11:08:59.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c800bac0 con 0x7fc7c8004d90 2023-11-28T11:08:59.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7c800bc80 con 0x7fc7dc0b2150 2023-11-28T11:08:59.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7db7fe700 1 --2- 172.21.15.28:0/349358174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7c8003bc0 0x7fc7c8003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7dbfff700 1 --2- 172.21.15.28:0/349358174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7c8004d90 0x7fc7c8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.696+0000 7fc7daffd700 1 --2- 172.21.15.28:0/349358174 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7dc0b2150 0x7fc7c8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.697+0000 7fc7db7fe700 1 -- 172.21.15.28:0/349358174 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7dc0b2150 msgr2=0x7fc7c8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:59.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.697+0000 7fc7db7fe700 1 --2- 172.21.15.28:0/349358174 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7dc0b2150 0x7fc7c8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.697+0000 7fc7db7fe700 1 -- 172.21.15.28:0/349358174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7c8004d90 msgr2=0x7fc7c8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:08:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.697+0000 7fc7db7fe700 1 --2- 172.21.15.28:0/349358174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7c8004d90 0x7fc7c8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.697+0000 7fc7db7fe700 1 -- 172.21.15.28:0/349358174 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7cc0094a0 con 0x7fc7c8003bc0 2023-11-28T11:08:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.697+0000 7fc7daffd700 1 --2- 172.21.15.28:0/349358174 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7dc0b2150 0x7fc7c8004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.697+0000 7fc7db7fe700 1 --2- 172.21.15.28:0/349358174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7c8003bc0 0x7fc7c8003fb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400c210 tx=0x7fc7c8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=394aeae865dd9ec6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.697+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c400cdb0 con 0x7fc7c8003bc0 2023-11-28T11:08:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.697+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c4001af0 con 0x7fc7c8003bc0 2023-11-28T11:08:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.697+0000 7fc7da7fc700 1 -- 172.21.15.28:0/349358174 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4005ba0 con 0x7fc7c8003bc0 2023-11-28T11:08:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.697+0000 7fc7e20bf700 1 -- 172.21.15.28:0/349358174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7c8003bc0 msgr2=0x7fc7c8003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.697+0000 7fc7e20bf700 1 --2- 172.21.15.28:0/349358174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7c8003bc0 0x7fc7c8003fb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400c210 tx=0x7fc7c8005880 comp rx=0 tx=0).stop 2023-11-28T11:08:59.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.697+0000 7fc7dbfff700 1 -- 172.21.15.28:0/349358174 reap_dead start 2023-11-28T11:08:59.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.697+0000 7fc7e20bf700 1 -- 172.21.15.28:0/349358174 shutdown_connections 2023-11-28T11:08:59.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.697+0000 7fc7e20bf700 1 -- 172.21.15.28:0/349358174 >> 172.21.15.28:0/349358174 conn(0x7fc7dc0b0df0 msgr2=0x7fc7dc0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.698+0000 7fc7e20bf700 1 -- 172.21.15.28:0/349358174 shutdown_connections 2023-11-28T11:08:59.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.698+0000 7fc7e20bf700 1 -- 172.21.15.28:0/349358174 wait complete. 2023-11-28T11:08:59.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.699+0000 7fc7e20bf700 1 Processor -- start 2023-11-28T11:08:59.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.699+0000 7fc7e20bf700 1 -- start start 2023-11-28T11:08:59.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7e20bf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7c8003bc0 0x7fc7dc0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7e20bf700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7dc0cdf90 0x7fc7dc153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7e20bf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7dc0c6d20 0x7fc7dc0c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7e20bf700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc0abf30 con 0x7fc7dc0cdf90 2023-11-28T11:08:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7e20bf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc0ac210 con 0x7fc7dc0c6d20 2023-11-28T11:08:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7e20bf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7dc0ac4f0 con 0x7fc7c8003bc0 2023-11-28T11:08:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7daffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7dc0cdf90 0x7fc7dc153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7daffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7dc0cdf90 0x7fc7dc153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46776/0 (socket says 172.21.15.28:46776) 2023-11-28T11:08:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7daffd700 1 -- 172.21.15.28:0/1774445519 learned_addr learned my addr 172.21.15.28:0/1774445519 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:08:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7db7fe700 1 --2- 172.21.15.28:0/1774445519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7c8003bc0 0x7fc7dc0ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7dbfff700 1 --2- 172.21.15.28:0/1774445519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7dc0c6d20 0x7fc7dc0c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7daffd700 1 -- 172.21.15.28:0/1774445519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7c8003bc0 msgr2=0x7fc7dc0ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7daffd700 1 --2- 172.21.15.28:0/1774445519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7c8003bc0 0x7fc7dc0ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7daffd700 1 -- 172.21.15.28:0/1774445519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7dc0c6d20 msgr2=0x7fc7dc0c71c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7daffd700 1 --2- 172.21.15.28:0/1774445519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7dc0c6d20 0x7fc7dc0c71c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7daffd700 1 -- 172.21.15.28:0/1774445519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7dc0ca000 con 0x7fc7dc0cdf90 2023-11-28T11:08:59.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7dbfff700 1 --2- 172.21.15.28:0/1774445519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7dc0c6d20 0x7fc7dc0c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:08:59.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.700+0000 7fc7daffd700 1 --2- 172.21.15.28:0/1774445519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7dc0cdf90 0x7fc7dc153980 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc010c30 tx=0x7fc7cc010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.701+0000 7fc7d3fff700 1 -- 172.21.15.28:0/1774445519 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc00e8d0 con 0x7fc7dc0cdf90 2023-11-28T11:08:59.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.701+0000 7fc7d3fff700 1 -- 172.21.15.28:0/1774445519 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7cc00ea30 con 0x7fc7dc0cdf90 2023-11-28T11:08:59.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.701+0000 7fc7e20bf700 1 -- 172.21.15.28:0/1774445519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7dc0c9d20 con 0x7fc7dc0cdf90 2023-11-28T11:08:59.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.701+0000 7fc7d3fff700 1 -- 172.21.15.28:0/1774445519 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7cc00eda0 con 0x7fc7dc0cdf90 2023-11-28T11:08:59.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.701+0000 7fc7e20bf700 1 -- 172.21.15.28:0/1774445519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7dc06ae90 con 0x7fc7dc0cdf90 2023-11-28T11:08:59.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.702+0000 7fc7e20bf700 1 -- 172.21.15.28:0/1774445519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7dc0739f0 con 0x7fc7dc0cdf90 2023-11-28T11:08:59.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.702+0000 7fc7d3fff700 1 -- 172.21.15.28:0/1774445519 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc7cc01c030 con 0x7fc7dc0cdf90 2023-11-28T11:08:59.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.702+0000 7fc7d3fff700 1 --2- 172.21.15.28:0/1774445519 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc7bc0682f0 0x7fc7bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:08:59.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.703+0000 7fc7d3fff700 1 -- 172.21.15.28:0/1774445519 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc7cc0862a0 con 0x7fc7dc0cdf90 2023-11-28T11:08:59.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.703+0000 7fc7db7fe700 1 --2- 172.21.15.28:0/1774445519 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc7bc0682f0 0x7fc7bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:08:59.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.703+0000 7fc7db7fe700 1 --2- 172.21.15.28:0/1774445519 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc7bc0682f0 0x7fc7bc06a7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40018b0 tx=0x7fc7c400a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:08:59.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.707+0000 7fc7d3fff700 1 -- 172.21.15.28:0/1774445519 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc7cc052810 con 0x7fc7dc0cdf90 2023-11-28T11:08:59.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.898+0000 7fc7e20bf700 1 -- 172.21.15.28:0/1774445519 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fc7dc0cba50 con 0x7fc7dc0cdf90 2023-11-28T11:08:59.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.899+0000 7fc7d3fff700 1 -- 172.21.15.28:0/1774445519 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fc7cc055e30 con 0x7fc7dc0cdf90 2023-11-28T11:08:59.900 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:08:59.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.901+0000 7fc7e20bf700 1 -- 172.21.15.28:0/1774445519 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc7bc0682f0 msgr2=0x7fc7bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.901+0000 7fc7e20bf700 1 --2- 172.21.15.28:0/1774445519 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc7bc0682f0 0x7fc7bc06a7a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc7c40018b0 tx=0x7fc7c400a000 comp rx=0 tx=0).stop 2023-11-28T11:08:59.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.901+0000 7fc7e20bf700 1 -- 172.21.15.28:0/1774445519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7dc0cdf90 msgr2=0x7fc7dc153980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:08:59.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.901+0000 7fc7e20bf700 1 --2- 172.21.15.28:0/1774445519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7dc0cdf90 0x7fc7dc153980 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc010c30 tx=0x7fc7cc010f40 comp rx=0 tx=0).stop 2023-11-28T11:08:59.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.901+0000 7fc7e20bf700 1 -- 172.21.15.28:0/1774445519 shutdown_connections 2023-11-28T11:08:59.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.901+0000 7fc7e20bf700 1 --2- 172.21.15.28:0/1774445519 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc7bc0682f0 0x7fc7bc06a7a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.901+0000 7fc7e20bf700 1 --2- 172.21.15.28:0/1774445519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7dc0c6d20 0x7fc7dc0c71c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.901+0000 7fc7e20bf700 1 --2- 172.21.15.28:0/1774445519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc7c8003bc0 0x7fc7dc0ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.901+0000 7fc7e20bf700 1 --2- 172.21.15.28:0/1774445519 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc7dc0cdf90 0x7fc7dc153980 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:08:59.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.901+0000 7fc7e20bf700 1 -- 172.21.15.28:0/1774445519 >> 172.21.15.28:0/1774445519 conn(0x7fc7dc0b0df0 msgr2=0x7fc7dc0b2520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:08:59.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.901+0000 7fc7e20bf700 1 -- 172.21.15.28:0/1774445519 shutdown_connections 2023-11-28T11:08:59.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:08:59.901+0000 7fc7e20bf700 1 -- 172.21.15.28:0/1774445519 wait complete. 2023-11-28T11:08:59.915 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:00.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.040+0000 7f840edec700 1 Processor -- start 2023-11-28T11:09:00.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.040+0000 7f840edec700 1 -- start start 2023-11-28T11:09:00.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.041+0000 7f840edec700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f840806aa70 0x7f84080ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.041+0000 7f840edec700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f840815c680 0x7f840815cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.041+0000 7f840edec700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f840806ed80 con 0x7f8408158a30 2023-11-28T11:09:00.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.041+0000 7f840edec700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f840806f060 con 0x7f840806aa70 2023-11-28T11:09:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.041+0000 7f840edec700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84080cd950 con 0x7f840815c680 2023-11-28T11:09:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.041+0000 7f8407fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f840806aa70 0x7f84080ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.041+0000 7f8407fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f840806aa70 0x7f84080ce860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46782/0 (socket says 172.21.15.28:46782) 2023-11-28T11:09:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.041+0000 7f8407fff700 1 -- 172.21.15.28:0/2300803852 learned_addr learned my addr 172.21.15.28:0/2300803852 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.041+0000 7f840d389700 1 --2- 172.21.15.28:0/2300803852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f840815c680 0x7f840815cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.041+0000 7f8407fff700 1 -- 172.21.15.28:0/2300803852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f840815c680 msgr2=0x7f840815cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.041+0000 7f8407fff700 1 --2- 172.21.15.28:0/2300803852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f840815c680 0x7f840815cab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.041+0000 7f8407fff700 1 -- 172.21.15.28:0/2300803852 >> v1:172.21.15.59:6789/0 conn(0x7f8408158a30 legacy=0x7f84080ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.041+0000 7f8407fff700 1 -- 172.21.15.28:0/2300803852 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84080cdc30 con 0x7f840806aa70 2023-11-28T11:09:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.041+0000 7f8407fff700 1 --2- 172.21.15.28:0/2300803852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f840806aa70 0x7f84080ce860 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f83f40098f0 tx=0x7f83f4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516f11a6e024e967 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.041+0000 7f84077fe700 1 -- 172.21.15.28:0/2300803852 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4004230 con 0x7f840806aa70 2023-11-28T11:09:00.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.042+0000 7f84077fe700 1 -- 172.21.15.28:0/2300803852 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f4004390 con 0x7f840806aa70 2023-11-28T11:09:00.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.042+0000 7f84077fe700 1 -- 172.21.15.28:0/2300803852 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f40036a0 con 0x7f840806aa70 2023-11-28T11:09:00.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.042+0000 7f840edec700 1 -- 172.21.15.28:0/2300803852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f840806aa70 msgr2=0x7f84080ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.042+0000 7f840edec700 1 --2- 172.21.15.28:0/2300803852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f840806aa70 0x7f84080ce860 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f83f40098f0 tx=0x7f83f4009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:00.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.042+0000 7f840edec700 1 -- 172.21.15.28:0/2300803852 shutdown_connections 2023-11-28T11:09:00.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.042+0000 7f840edec700 1 --2- 172.21.15.28:0/2300803852 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f840815c680 0x7f840815cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.042+0000 7f840edec700 1 --2- 172.21.15.28:0/2300803852 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f840806aa70 0x7f84080ce860 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.042+0000 7f840edec700 1 -- 172.21.15.28:0/2300803852 >> 172.21.15.28:0/2300803852 conn(0x7f84080b0dd0 msgr2=0x7f84080b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:00.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.042+0000 7f840edec700 1 -- 172.21.15.28:0/2300803852 shutdown_connections 2023-11-28T11:09:00.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.042+0000 7f840edec700 1 -- 172.21.15.28:0/2300803852 wait complete. 2023-11-28T11:09:00.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.043+0000 7f840edec700 1 Processor -- start 2023-11-28T11:09:00.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.043+0000 7f840edec700 1 -- start start 2023-11-28T11:09:00.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.044+0000 7f840edec700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f840806aa70 0x7f8408150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.044+0000 7f840edec700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8408158a30 0x7f8408151410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.044+0000 7f840edec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f840815c680 0x7f84080c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.044+0000 7f840edec700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8408151040 con 0x7f8408158a30 2023-11-28T11:09:00.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.044+0000 7f840edec700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84080c56a0 con 0x7f840815c680 2023-11-28T11:09:00.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.044+0000 7f840edec700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84080c5980 con 0x7f840806aa70 2023-11-28T11:09:00.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.044+0000 7f8407fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8408158a30 0x7f8408151410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.044+0000 7f8407fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8408158a30 0x7f8408151410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46790/0 (socket says 172.21.15.28:46790) 2023-11-28T11:09:00.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.044+0000 7f8407fff700 1 -- 172.21.15.28:0/3503013084 learned_addr learned my addr 172.21.15.28:0/3503013084 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:00.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.044+0000 7f840d389700 1 --2- 172.21.15.28:0/3503013084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f840815c680 0x7f84080c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.044+0000 7f840cb88700 1 --2- 172.21.15.28:0/3503013084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f840806aa70 0x7f8408150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.045+0000 7f8407fff700 1 -- 172.21.15.28:0/3503013084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f840806aa70 msgr2=0x7f8408150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.045+0000 7f8407fff700 1 --2- 172.21.15.28:0/3503013084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f840806aa70 0x7f8408150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.045+0000 7f8407fff700 1 -- 172.21.15.28:0/3503013084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f840815c680 msgr2=0x7f84080c34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.045+0000 7f8407fff700 1 --2- 172.21.15.28:0/3503013084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f840815c680 0x7f84080c34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.045+0000 7f8407fff700 1 -- 172.21.15.28:0/3503013084 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f4009550 con 0x7f8408158a30 2023-11-28T11:09:00.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.045+0000 7f8407fff700 1 --2- 172.21.15.28:0/3503013084 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8408158a30 0x7f8408151410 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009130 tx=0x7f83f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.045+0000 7f84057fa700 1 -- 172.21.15.28:0/3503013084 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4003c00 con 0x7f8408158a30 2023-11-28T11:09:00.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.045+0000 7f84057fa700 1 -- 172.21.15.28:0/3503013084 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f401f070 con 0x7f8408158a30 2023-11-28T11:09:00.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.045+0000 7f84057fa700 1 -- 172.21.15.28:0/3503013084 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f400b660 con 0x7f8408158a30 2023-11-28T11:09:00.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.045+0000 7f840edec700 1 -- 172.21.15.28:0/3503013084 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840806f340 con 0x7f8408158a30 2023-11-28T11:09:00.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.045+0000 7f840edec700 1 -- 172.21.15.28:0/3503013084 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84080ceda0 con 0x7f8408158a30 2023-11-28T11:09:00.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.046+0000 7f840edec700 1 -- 172.21.15.28:0/3503013084 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f840806e9c0 con 0x7f8408158a30 2023-11-28T11:09:00.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.046+0000 7f84057fa700 1 -- 172.21.15.28:0/3503013084 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f83f402a020 con 0x7f8408158a30 2023-11-28T11:09:00.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.047+0000 7f84057fa700 1 --2- 172.21.15.28:0/3503013084 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83ec0682f0 0x7f83ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.047+0000 7f84057fa700 1 -- 172.21.15.28:0/3503013084 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f83f4087750 con 0x7f8408158a30 2023-11-28T11:09:00.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.047+0000 7f840cb88700 1 --2- 172.21.15.28:0/3503013084 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83ec0682f0 0x7f83ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.047+0000 7f840cb88700 1 --2- 172.21.15.28:0/3503013084 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83ec0682f0 0x7f83ec06a7a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00cb10 tx=0x7f83fc007ce0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.051+0000 7f84057fa700 1 -- 172.21.15.28:0/3503013084 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f83f4053cc0 con 0x7f8408158a30 2023-11-28T11:09:00.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.258+0000 7f840edec700 1 -- 172.21.15.28:0/3503013084 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f840806e9c0 con 0x7f8408158a30 2023-11-28T11:09:00.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.258+0000 7f84057fa700 1 -- 172.21.15.28:0/3503013084 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v25) v1 ==== 80+0+0 (secure 0 0 0) 0x7f83f4010380 con 0x7f8408158a30 2023-11-28T11:09:00.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.260+0000 7f840edec700 1 -- 172.21.15.28:0/3503013084 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83ec0682f0 msgr2=0x7f83ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.260+0000 7f840edec700 1 --2- 172.21.15.28:0/3503013084 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83ec0682f0 0x7f83ec06a7a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f83fc00cb10 tx=0x7f83fc007ce0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.260+0000 7f840edec700 1 -- 172.21.15.28:0/3503013084 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8408158a30 msgr2=0x7f8408151410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.260+0000 7f840edec700 1 --2- 172.21.15.28:0/3503013084 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8408158a30 0x7f8408151410 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009130 tx=0x7f83f4004750 comp rx=0 tx=0).stop 2023-11-28T11:09:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.260+0000 7f840edec700 1 -- 172.21.15.28:0/3503013084 shutdown_connections 2023-11-28T11:09:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.260+0000 7f840edec700 1 --2- 172.21.15.28:0/3503013084 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83ec0682f0 0x7f83ec06a7a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.260+0000 7f840edec700 1 --2- 172.21.15.28:0/3503013084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f840815c680 0x7f84080c34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.260+0000 7f840edec700 1 --2- 172.21.15.28:0/3503013084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f840806aa70 0x7f8408150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.260+0000 7f840edec700 1 --2- 172.21.15.28:0/3503013084 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8408158a30 0x7f8408151410 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.260+0000 7f840edec700 1 -- 172.21.15.28:0/3503013084 >> 172.21.15.28:0/3503013084 conn(0x7f84080b0dd0 msgr2=0x7f84080ca6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.260+0000 7f840edec700 1 -- 172.21.15.28:0/3503013084 shutdown_connections 2023-11-28T11:09:00.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.260+0000 7f840edec700 1 -- 172.21.15.28:0/3503013084 wait complete. 2023-11-28T11:09:00.263 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2023-11-28T11:09:00.273 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi028.front.sepia.ceph.com... 2023-11-28T11:09:00.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:00.274 DEBUG:teuthology.orchestra.run.smithi028:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-11-28T11:09:00.310 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi028.stderr:ceph-fuse[110727]: fuse finished with error 0 and tester_r 0 2023-11-28T11:09:00.312 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:00.363 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:09:00.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:00.364 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-11-28T11:09:00.389 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:00.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.552+0000 7fee5ef86700 1 Processor -- start 2023-11-28T11:09:00.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.552+0000 7fee5ef86700 1 -- start start 2023-11-28T11:09:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.553+0000 7fee5ef86700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fee5806d9d0 0x7fee5815b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.553+0000 7fee5ef86700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5806c3a0 con 0x7fee5806dda0 2023-11-28T11:09:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.553+0000 7fee5ef86700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee5806c0c0 con 0x7fee5806bc30 2023-11-28T11:09:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.553+0000 7fee5ef86700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee5815ab60 con 0x7fee5806d9d0 2023-11-28T11:09:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.553+0000 7fee5d523700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fee5806dda0 0x7fee580c9930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34858/0 (socket says 172.21.15.28:34858) 2023-11-28T11:09:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.553+0000 7fee5d523700 1 -- 172.21.15.28:0/3996665914 learned_addr learned my addr 172.21.15.28:0/3996665914 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.553+0000 7fee5cd22700 1 --2- 172.21.15.28:0/3996665914 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fee5806d9d0 0x7fee5815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.554+0000 7fee577fe700 1 -- 172.21.15.28:0/3996665914 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285717372 0 0) 0x7fee5806c3a0 con 0x7fee5806dda0 2023-11-28T11:09:00.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.554+0000 7fee577fe700 1 -- 172.21.15.28:0/3996665914 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee480035b0 con 0x7fee5806dda0 2023-11-28T11:09:00.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.554+0000 7fee577fe700 1 -- 172.21.15.28:0/3996665914 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 663951959 0 0) 0x7fee5806c0c0 con 0x7fee5806bc30 2023-11-28T11:09:00.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.554+0000 7fee577fe700 1 -- 172.21.15.28:0/3996665914 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee5806c3a0 con 0x7fee5806bc30 2023-11-28T11:09:00.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.554+0000 7fee5cd22700 1 -- 172.21.15.28:0/3996665914 >> v1:172.21.15.59:6789/0 conn(0x7fee5806bc30 legacy=0x7fee58150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.554+0000 7fee5cd22700 1 -- 172.21.15.28:0/3996665914 >> v1:172.21.15.28:6789/0 conn(0x7fee5806dda0 legacy=0x7fee580c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.554+0000 7fee577fe700 1 -- 172.21.15.28:0/3996665914 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1668374888 0 0) 0x7fee480035b0 con 0x7fee5806dda0 2023-11-28T11:09:00.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.554+0000 7fee5cd22700 1 -- 172.21.15.28:0/3996665914 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5815ae40 con 0x7fee5806d9d0 2023-11-28T11:09:00.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.554+0000 7fee5cd22700 1 --2- 172.21.15.28:0/3996665914 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fee5806d9d0 0x7fee5815b640 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009870 tx=0x7fee4c009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96c78adbb1bf5f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.554+0000 7fee577fe700 1 -- 172.21.15.28:0/3996665914 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee500019f0 con 0x7fee5806d9d0 2023-11-28T11:09:00.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.554+0000 7fee577fe700 1 -- 172.21.15.28:0/3996665914 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee4c004030 con 0x7fee5806d9d0 2023-11-28T11:09:00.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.554+0000 7fee577fe700 1 -- 172.21.15.28:0/3996665914 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee4c0043c0 con 0x7fee5806d9d0 2023-11-28T11:09:00.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.555+0000 7fee5ef86700 1 -- 172.21.15.28:0/3996665914 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fee5806d9d0 msgr2=0x7fee5815b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.555+0000 7fee5ef86700 1 --2- 172.21.15.28:0/3996665914 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fee5806d9d0 0x7fee5815b640 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee4c009870 tx=0x7fee4c009b80 comp rx=0 tx=0).stop 2023-11-28T11:09:00.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.555+0000 7fee5ef86700 1 -- 172.21.15.28:0/3996665914 shutdown_connections 2023-11-28T11:09:00.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.555+0000 7fee5ef86700 1 --2- 172.21.15.28:0/3996665914 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fee5806d9d0 0x7fee5815b640 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.555+0000 7fee5ef86700 1 -- 172.21.15.28:0/3996665914 >> 172.21.15.28:0/3996665914 conn(0x7fee580b0dd0 msgr2=0x7fee580b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:00.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.555+0000 7fee5ef86700 1 -- 172.21.15.28:0/3996665914 shutdown_connections 2023-11-28T11:09:00.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.555+0000 7fee5ef86700 1 -- 172.21.15.28:0/3996665914 wait complete. 2023-11-28T11:09:00.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.556+0000 7fee5ef86700 1 Processor -- start 2023-11-28T11:09:00.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.557+0000 7fee5ef86700 1 -- start start 2023-11-28T11:09:00.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5ef86700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fee5806bc30 0x7fee5814d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5ef86700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5806d9d0 0x7fee580b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5ef86700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee5806dda0 0x7fee580ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5ef86700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee58155080 con 0x7fee5806dda0 2023-11-28T11:09:00.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5ef86700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee5814e020 con 0x7fee5806d9d0 2023-11-28T11:09:00.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5ef86700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee581552d0 con 0x7fee5806bc30 2023-11-28T11:09:00.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5d523700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee5806dda0 0x7fee580ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5d523700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee5806dda0 0x7fee580ca7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46806/0 (socket says 172.21.15.28:46806) 2023-11-28T11:09:00.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5d523700 1 -- 172.21.15.28:0/831207163 learned_addr learned my addr 172.21.15.28:0/831207163 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:00.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee57fff700 1 --2- 172.21.15.28:0/831207163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5806d9d0 0x7fee580b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5cd22700 1 --2- 172.21.15.28:0/831207163 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fee5806bc30 0x7fee5814d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5d523700 1 -- 172.21.15.28:0/831207163 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fee5806bc30 msgr2=0x7fee5814d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5d523700 1 --2- 172.21.15.28:0/831207163 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fee5806bc30 0x7fee5814d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5d523700 1 -- 172.21.15.28:0/831207163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5806d9d0 msgr2=0x7fee580b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5d523700 1 --2- 172.21.15.28:0/831207163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5806d9d0 0x7fee580b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5d523700 1 -- 172.21.15.28:0/831207163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee4c0094a0 con 0x7fee5806dda0 2023-11-28T11:09:00.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee5d523700 1 --2- 172.21.15.28:0/831207163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee5806dda0 0x7fee580ca7c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fee5000cc10 tx=0x7fee5000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.558+0000 7fee557fa700 1 -- 172.21.15.28:0/831207163 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5000aaa0 con 0x7fee5806dda0 2023-11-28T11:09:00.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.559+0000 7fee557fa700 1 -- 172.21.15.28:0/831207163 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5000ac00 con 0x7fee5806dda0 2023-11-28T11:09:00.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.559+0000 7fee5ef86700 1 -- 172.21.15.28:0/831207163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee5806fac0 con 0x7fee5806dda0 2023-11-28T11:09:00.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.559+0000 7fee557fa700 1 -- 172.21.15.28:0/831207163 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee500070d0 con 0x7fee5806dda0 2023-11-28T11:09:00.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.559+0000 7fee5ef86700 1 -- 172.21.15.28:0/831207163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee5815b120 con 0x7fee5806dda0 2023-11-28T11:09:00.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.560+0000 7fee5ef86700 1 -- 172.21.15.28:0/831207163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee581515c0 con 0x7fee5806dda0 2023-11-28T11:09:00.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.560+0000 7fee557fa700 1 -- 172.21.15.28:0/831207163 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fee50019030 con 0x7fee5806dda0 2023-11-28T11:09:00.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.560+0000 7fee557fa700 1 --2- 172.21.15.28:0/831207163 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fee3c0682f0 0x7fee3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.561+0000 7fee5cd22700 1 --2- 172.21.15.28:0/831207163 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fee3c0682f0 0x7fee3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.561+0000 7fee557fa700 1 -- 172.21.15.28:0/831207163 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fee50085ce0 con 0x7fee5806dda0 2023-11-28T11:09:00.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.561+0000 7fee5cd22700 1 --2- 172.21.15.28:0/831207163 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fee3c0682f0 0x7fee3c06a7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0097c0 tx=0x7fee4c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.566+0000 7fee557fa700 1 -- 172.21.15.28:0/831207163 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fee50052250 con 0x7fee5806dda0 2023-11-28T11:09:00.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.759+0000 7fee5ef86700 1 -- 172.21.15.28:0/831207163 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fee5815bef0 con 0x7fee5806dda0 2023-11-28T11:09:00.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.760+0000 7fee557fa700 1 -- 172.21.15.28:0/831207163 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19642 (secure 0 0 0) 0x7fee50017020 con 0x7fee5806dda0 2023-11-28T11:09:00.762 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:00.762 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:50.531733+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:00.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.762+0000 7fee5ef86700 1 -- 172.21.15.28:0/831207163 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fee3c0682f0 msgr2=0x7fee3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.762+0000 7fee5ef86700 1 --2- 172.21.15.28:0/831207163 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fee3c0682f0 0x7fee3c06a7a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fee4c0097c0 tx=0x7fee4c008030 comp rx=0 tx=0).stop 2023-11-28T11:09:00.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.762+0000 7fee5ef86700 1 -- 172.21.15.28:0/831207163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee5806dda0 msgr2=0x7fee580ca7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.762+0000 7fee5ef86700 1 --2- 172.21.15.28:0/831207163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee5806dda0 0x7fee580ca7c0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fee5000cc10 tx=0x7fee5000cf20 comp rx=0 tx=0).stop 2023-11-28T11:09:00.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.763+0000 7fee5ef86700 1 -- 172.21.15.28:0/831207163 shutdown_connections 2023-11-28T11:09:00.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.763+0000 7fee5ef86700 1 --2- 172.21.15.28:0/831207163 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fee3c0682f0 0x7fee3c06a7a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.763+0000 7fee5ef86700 1 --2- 172.21.15.28:0/831207163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fee5806dda0 0x7fee580ca7c0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.763+0000 7fee5ef86700 1 --2- 172.21.15.28:0/831207163 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fee5806bc30 0x7fee5814d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.763+0000 7fee5ef86700 1 --2- 172.21.15.28:0/831207163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee5806d9d0 0x7fee580b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.763+0000 7fee5ef86700 1 -- 172.21.15.28:0/831207163 >> 172.21.15.28:0/831207163 conn(0x7fee580b0dd0 msgr2=0x7fee58159740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:00.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.763+0000 7fee5ef86700 1 -- 172.21.15.28:0/831207163 shutdown_connections 2023-11-28T11:09:00.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.763+0000 7fee5ef86700 1 -- 172.21.15.28:0/831207163 wait complete. 2023-11-28T11:09:00.777 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:00.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.899+0000 7fd0e801c700 1 Processor -- start 2023-11-28T11:09:00.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.899+0000 7fd0e801c700 1 -- start start 2023-11-28T11:09:00.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.900+0000 7fd0e801c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd0e0159590 0x7fd0e006e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.900+0000 7fd0e801c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0e00ce890 0x7fd0e00ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.900+0000 7fd0e801c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0e015d120 con 0x7fd0e015db70 2023-11-28T11:09:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.900+0000 7fd0e801c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0e00cf250 con 0x7fd0e0159590 2023-11-28T11:09:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.900+0000 7fd0e801c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e015d6e0 con 0x7fd0e00ce890 2023-11-28T11:09:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.900+0000 7fd0e65b9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0e00ce890 0x7fd0e00ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.900+0000 7fd0e5db8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd0e0159590 0x7fd0e006e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.900+0000 7fd0e55b7700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd0e015db70 0x7fd0e00ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:45654/0 (socket says 172.21.15.28:45654) 2023-11-28T11:09:00.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.900+0000 7fd0e55b7700 1 -- 172.21.15.28:0/2356586153 learned_addr learned my addr 172.21.15.28:0/2356586153 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.901+0000 7fd0e65b9700 1 -- 172.21.15.28:0/2356586153 >> v2:172.21.15.59:3300/0 conn(0x7fd0e0159590 msgr2=0x7fd0e006e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.901+0000 7fd0e65b9700 1 --2- 172.21.15.28:0/2356586153 >> v2:172.21.15.59:3300/0 conn(0x7fd0e0159590 0x7fd0e006e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.901+0000 7fd0e65b9700 1 -- 172.21.15.28:0/2356586153 >> v1:172.21.15.59:6789/0 conn(0x7fd0e015db70 legacy=0x7fd0e00ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.901+0000 7fd0e65b9700 1 -- 172.21.15.28:0/2356586153 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0e00aef60 con 0x7fd0e00ce890 2023-11-28T11:09:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.901+0000 7fd0e65b9700 1 --2- 172.21.15.28:0/2356586153 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0e00ce890 0x7fd0e00ced10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8009d90 tx=0x7fd0d800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1f32833f5dc3d082 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.901+0000 7fd0e4db6700 1 -- 172.21.15.28:0/2356586153 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d800e070 con 0x7fd0e00ce890 2023-11-28T11:09:00.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.901+0000 7fd0e4db6700 1 -- 172.21.15.28:0/2356586153 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0d800bb20 con 0x7fd0e00ce890 2023-11-28T11:09:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.901+0000 7fd0e4db6700 1 -- 172.21.15.28:0/2356586153 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d8012430 con 0x7fd0e00ce890 2023-11-28T11:09:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.902+0000 7fd0e801c700 1 -- 172.21.15.28:0/2356586153 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0e00ce890 msgr2=0x7fd0e00ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.902+0000 7fd0e801c700 1 --2- 172.21.15.28:0/2356586153 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0e00ce890 0x7fd0e00ced10 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8009d90 tx=0x7fd0d800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.902+0000 7fd0e801c700 1 -- 172.21.15.28:0/2356586153 shutdown_connections 2023-11-28T11:09:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.902+0000 7fd0e801c700 1 --2- 172.21.15.28:0/2356586153 >> v2:172.21.15.59:3300/0 conn(0x7fd0e0159590 0x7fd0e006e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.902+0000 7fd0e801c700 1 --2- 172.21.15.28:0/2356586153 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0e00ce890 0x7fd0e00ced10 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.907 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.902+0000 7fd0e801c700 1 -- 172.21.15.28:0/2356586153 >> 172.21.15.28:0/2356586153 conn(0x7fd0e00b1650 msgr2=0x7fd0e00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:00.907 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.902+0000 7fd0e801c700 1 -- 172.21.15.28:0/2356586153 shutdown_connections 2023-11-28T11:09:00.907 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.902+0000 7fd0e801c700 1 -- 172.21.15.28:0/2356586153 wait complete. 2023-11-28T11:09:00.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.903+0000 7fd0e801c700 1 Processor -- start 2023-11-28T11:09:00.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.903+0000 7fd0e801c700 1 -- start start 2023-11-28T11:09:00.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e801c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0e00ce890 0x7fd0e015a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e801c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e0159590 0x7fd0e00b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e801c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0e015db70 0x7fd0e0146260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e801c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e015c330 con 0x7fd0e0159590 2023-11-28T11:09:00.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e801c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e015b290 con 0x7fd0e015db70 2023-11-28T11:09:00.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e801c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e015b570 con 0x7fd0e00ce890 2023-11-28T11:09:00.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e55b7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e0159590 0x7fd0e00b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e55b7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e0159590 0x7fd0e00b2bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46818/0 (socket says 172.21.15.28:46818) 2023-11-28T11:09:00.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e55b7700 1 -- 172.21.15.28:0/2426461464 learned_addr learned my addr 172.21.15.28:0/2426461464 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:00.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e5db8700 1 --2- 172.21.15.28:0/2426461464 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0e00ce890 0x7fd0e015a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e65b9700 1 --2- 172.21.15.28:0/2426461464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0e015db70 0x7fd0e0146260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e55b7700 1 -- 172.21.15.28:0/2426461464 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0e00ce890 msgr2=0x7fd0e015a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e55b7700 1 --2- 172.21.15.28:0/2426461464 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0e00ce890 0x7fd0e015a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e55b7700 1 -- 172.21.15.28:0/2426461464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0e015db70 msgr2=0x7fd0e0146260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:00.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e55b7700 1 --2- 172.21.15.28:0/2426461464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0e015db70 0x7fd0e0146260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:00.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e55b7700 1 -- 172.21.15.28:0/2426461464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0d8009a70 con 0x7fd0e0159590 2023-11-28T11:09:00.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e55b7700 1 --2- 172.21.15.28:0/2426461464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e0159590 0x7fd0e00b2bc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc00c480 tx=0x7fd0dc00c790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0d67fc700 1 -- 172.21.15.28:0/2426461464 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc005d70 con 0x7fd0e0159590 2023-11-28T11:09:00.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0d67fc700 1 -- 172.21.15.28:0/2426461464 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0dc00ce90 con 0x7fd0e0159590 2023-11-28T11:09:00.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e801c700 1 -- 172.21.15.28:0/2426461464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e015d370 con 0x7fd0e0159590 2023-11-28T11:09:00.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0d67fc700 1 -- 172.21.15.28:0/2426461464 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0dc0072d0 con 0x7fd0e0159590 2023-11-28T11:09:00.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e801c700 1 -- 172.21.15.28:0/2426461464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e00af240 con 0x7fd0e0159590 2023-11-28T11:09:00.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.905+0000 7fd0e801c700 1 -- 172.21.15.28:0/2426461464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e015e010 con 0x7fd0e0159590 2023-11-28T11:09:00.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.906+0000 7fd0d67fc700 1 -- 172.21.15.28:0/2426461464 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd0dc019030 con 0x7fd0e0159590 2023-11-28T11:09:00.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.906+0000 7fd0d67fc700 1 --2- 172.21.15.28:0/2426461464 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd0c40682f0 0x7fd0c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:00.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.906+0000 7fd0d67fc700 1 -- 172.21.15.28:0/2426461464 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd0dc085010 con 0x7fd0e0159590 2023-11-28T11:09:00.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.906+0000 7fd0e5db8700 1 --2- 172.21.15.28:0/2426461464 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd0c40682f0 0x7fd0c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:00.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.907+0000 7fd0e5db8700 1 --2- 172.21.15.28:0/2426461464 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd0c40682f0 0x7fd0c406a7a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0009be0 tx=0x7fd0d0009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:00.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:00.911+0000 7fd0d67fc700 1 -- 172.21.15.28:0/2426461464 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd0dc04b5e0 con 0x7fd0e0159590 2023-11-28T11:09:01.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.104+0000 7fd0e801c700 1 -- 172.21.15.28:0/2426461464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0e00675a0 con 0x7fd0e0159590 2023-11-28T11:09:01.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.105+0000 7fd0d67fc700 1 -- 172.21.15.28:0/2426461464 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19642 (secure 0 0 0) 0x7fd0dc017090 con 0x7fd0e0159590 2023-11-28T11:09:01.107 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:01.107 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:50.531733+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:01.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.108+0000 7fd0e801c700 1 -- 172.21.15.28:0/2426461464 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd0c40682f0 msgr2=0x7fd0c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.108+0000 7fd0e801c700 1 --2- 172.21.15.28:0/2426461464 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd0c40682f0 0x7fd0c406a7a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd0d0009be0 tx=0x7fd0d0009210 comp rx=0 tx=0).stop 2023-11-28T11:09:01.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.108+0000 7fd0e801c700 1 -- 172.21.15.28:0/2426461464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e0159590 msgr2=0x7fd0e00b2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.108+0000 7fd0e801c700 1 --2- 172.21.15.28:0/2426461464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e0159590 0x7fd0e00b2bc0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd0dc00c480 tx=0x7fd0dc00c790 comp rx=0 tx=0).stop 2023-11-28T11:09:01.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.108+0000 7fd0e801c700 1 -- 172.21.15.28:0/2426461464 shutdown_connections 2023-11-28T11:09:01.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.108+0000 7fd0e801c700 1 --2- 172.21.15.28:0/2426461464 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd0c40682f0 0x7fd0c406a7a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.108+0000 7fd0e801c700 1 --2- 172.21.15.28:0/2426461464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd0e015db70 0x7fd0e0146260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.108+0000 7fd0e801c700 1 --2- 172.21.15.28:0/2426461464 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd0e00ce890 0x7fd0e015a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.108+0000 7fd0e801c700 1 --2- 172.21.15.28:0/2426461464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd0e0159590 0x7fd0e00b2bc0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.108+0000 7fd0e801c700 1 -- 172.21.15.28:0/2426461464 >> 172.21.15.28:0/2426461464 conn(0x7fd0e00b1650 msgr2=0x7fd0e00afdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:01.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.108+0000 7fd0e801c700 1 -- 172.21.15.28:0/2426461464 shutdown_connections 2023-11-28T11:09:01.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.108+0000 7fd0e801c700 1 -- 172.21.15.28:0/2426461464 wait complete. 2023-11-28T11:09:01.122 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:09:01.250 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.250+0000 7fae1924d700 1 Processor -- start 2023-11-28T11:09:01.251 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.250+0000 7fae1924d700 1 -- start start 2023-11-28T11:09:01.254 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae1924d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fae1415b5e0 0x7fae140b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.254 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae1924d700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae1415b150 con 0x7fae14143fa0 2023-11-28T11:09:01.254 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae1924d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae140cb230 con 0x7fae140b2ba0 2023-11-28T11:09:01.254 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae1924d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae1415abd0 con 0x7fae1415b5e0 2023-11-28T11:09:01.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae12d9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fae14143fa0 0x7fae1415ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34882/0 (socket says 172.21.15.28:34882) 2023-11-28T11:09:01.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae12d9d700 1 -- 172.21.15.28:0/3638499451 learned_addr learned my addr 172.21.15.28:0/3638499451 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:01.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae1259c700 1 --2- 172.21.15.28:0/3638499451 >> v2:172.21.15.59:3300/0 conn(0x7fae1415b5e0 0x7fae140b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 492383047 0 0) 0x7fae1415b150 con 0x7fae14143fa0 2023-11-28T11:09:01.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae000035b0 con 0x7fae14143fa0 2023-11-28T11:09:01.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2674153784 0 0) 0x7fae140cb230 con 0x7fae140b2ba0 2023-11-28T11:09:01.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae1415b150 con 0x7fae140b2ba0 2023-11-28T11:09:01.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 662633872 0 0) 0x7fae000035b0 con 0x7fae14143fa0 2023-11-28T11:09:01.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 >> v2:172.21.15.59:3300/0 conn(0x7fae1415b5e0 msgr2=0x7fae140b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae11d9b700 1 --2- 172.21.15.28:0/3638499451 >> v2:172.21.15.59:3300/0 conn(0x7fae1415b5e0 0x7fae140b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 >> v1:172.21.15.59:6789/0 conn(0x7fae140b2ba0 legacy=0x7fae1415bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.251+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae140cba90 con 0x7fae14143fa0 2023-11-28T11:09:01.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fae0c0019f0 con 0x7fae14143fa0 2023-11-28T11:09:01.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 >> v1:172.21.15.28:6789/0 conn(0x7fae14143fa0 legacy=0x7fae1415ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae11d9b700 1 --2- 172.21.15.28:0/3638499451 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae00003b90 0x7fae00003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae11d9b700 1 --2- 172.21.15.28:0/3638499451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae00004850 0x7fae00004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae11d9b700 1 --2- 172.21.15.28:0/3638499451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae00005200 0x7fae0000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae1415abd0 con 0x7fae00004850 2023-11-28T11:09:01.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae0000c060 con 0x7fae00005200 2023-11-28T11:09:01.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae0000c220 con 0x7fae00003b90 2023-11-28T11:09:01.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae1259c700 1 --2- 172.21.15.28:0/3638499451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae00004850 0x7fae00004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae1359e700 1 --2- 172.21.15.28:0/3638499451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae00005200 0x7fae0000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae12d9d700 1 --2- 172.21.15.28:0/3638499451 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae00003b90 0x7fae00003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae1259c700 1 -- 172.21.15.28:0/3638499451 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae00003b90 msgr2=0x7fae00003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae1259c700 1 --2- 172.21.15.28:0/3638499451 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae00003b90 0x7fae00003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae1259c700 1 -- 172.21.15.28:0/3638499451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae00005200 msgr2=0x7fae0000ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:01.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae1259c700 1 --2- 172.21.15.28:0/3638499451 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae00005200 0x7fae0000ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae1359e700 1 -- 172.21.15.28:0/3638499451 reap_dead start 2023-11-28T11:09:01.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae1259c700 1 -- 172.21.15.28:0/3638499451 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae000035b0 con 0x7fae00004850 2023-11-28T11:09:01.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae12d9d700 1 --2- 172.21.15.28:0/3638499451 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae00003b90 0x7fae00003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:01.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae1259c700 1 --2- 172.21.15.28:0/3638499451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae00004850 0x7fae00004cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009b60 tx=0x7fae00005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ff638e63c506ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc00e6c0 con 0x7fae00004850 2023-11-28T11:09:01.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadfc00e820 con 0x7fae00004850 2023-11-28T11:09:01.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.252+0000 7fae11d9b700 1 -- 172.21.15.28:0/3638499451 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadfc00eb90 con 0x7fae00004850 2023-11-28T11:09:01.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.253+0000 7fae1924d700 1 -- 172.21.15.28:0/3638499451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae00004850 msgr2=0x7fae00004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.253+0000 7fae1924d700 1 --2- 172.21.15.28:0/3638499451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae00004850 0x7fae00004cc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fadfc009b60 tx=0x7fae00005880 comp rx=0 tx=0).stop 2023-11-28T11:09:01.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.253+0000 7fae1924d700 1 -- 172.21.15.28:0/3638499451 shutdown_connections 2023-11-28T11:09:01.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.253+0000 7fae1924d700 1 --2- 172.21.15.28:0/3638499451 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae00004850 0x7fae00004cc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.253+0000 7fae1924d700 1 -- 172.21.15.28:0/3638499451 >> 172.21.15.28:0/3638499451 conn(0x7fae1406e9b0 msgr2=0x7fae140cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:01.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.253+0000 7fae1924d700 1 -- 172.21.15.28:0/3638499451 shutdown_connections 2023-11-28T11:09:01.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.253+0000 7fae1924d700 1 -- 172.21.15.28:0/3638499451 wait complete. 2023-11-28T11:09:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.254+0000 7fae1924d700 1 Processor -- start 2023-11-28T11:09:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.254+0000 7fae1924d700 1 -- start start 2023-11-28T11:09:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.255+0000 7fae1924d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae00004850 0x7fae140cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.255+0000 7fae1924d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae14158cd0 0x7fae141f1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.255+0000 7fae1924d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae140b3a10 0x7fae140b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.255+0000 7fae1924d700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae140b43c0 con 0x7fae00004850 2023-11-28T11:09:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.255+0000 7fae1924d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae140b59c0 con 0x7fae140b3a10 2023-11-28T11:09:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.255+0000 7fae1924d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae141591d0 con 0x7fae14158cd0 2023-11-28T11:09:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.255+0000 7fae12d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae00004850 0x7fae140cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.255+0000 7fae12d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae00004850 0x7fae140cf390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46840/0 (socket says 172.21.15.28:46840) 2023-11-28T11:09:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.255+0000 7fae12d9d700 1 -- 172.21.15.28:0/1910447327 learned_addr learned my addr 172.21.15.28:0/1910447327 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.256+0000 7fae1359e700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae140b3a10 0x7fae140b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.256+0000 7fae1259c700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae14158cd0 0x7fae141f1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.256+0000 7fae12d9d700 1 -- 172.21.15.28:0/1910447327 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae14158cd0 msgr2=0x7fae141f1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.256+0000 7fae12d9d700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae14158cd0 0x7fae141f1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.256+0000 7fae12d9d700 1 -- 172.21.15.28:0/1910447327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae140b3a10 msgr2=0x7fae140b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.256+0000 7fae12d9d700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae140b3a10 0x7fae140b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.256+0000 7fae12d9d700 1 -- 172.21.15.28:0/1910447327 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae140cb4d0 con 0x7fae00004850 2023-11-28T11:09:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.256+0000 7fae1259c700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae14158cd0 0x7fae141f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.256+0000 7fae1359e700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae140b3a10 0x7fae140b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.256+0000 7fae12d9d700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae00004850 0x7fae140cf390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00e470 tx=0x7fae0c007c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.256+0000 7fae0b7fe700 1 -- 172.21.15.28:0/1910447327 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c013420 con 0x7fae00004850 2023-11-28T11:09:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.256+0000 7fae0b7fe700 1 -- 172.21.15.28:0/1910447327 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae0c013580 con 0x7fae00004850 2023-11-28T11:09:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.256+0000 7fae1924d700 1 -- 172.21.15.28:0/1910447327 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae140b2f90 con 0x7fae00004850 2023-11-28T11:09:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.256+0000 7fae0b7fe700 1 -- 172.21.15.28:0/1910447327 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae0c0138f0 con 0x7fae00004850 2023-11-28T11:09:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.256+0000 7fae1924d700 1 -- 172.21.15.28:0/1910447327 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae140cb7b0 con 0x7fae00004850 2023-11-28T11:09:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.257+0000 7fae1924d700 1 -- 172.21.15.28:0/1910447327 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae14066940 con 0x7fae00004850 2023-11-28T11:09:01.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.257+0000 7fae0b7fe700 1 -- 172.21.15.28:0/1910447327 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fae0c013a50 con 0x7fae00004850 2023-11-28T11:09:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.258+0000 7fae0b7fe700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fadf40682f0 0x7fadf406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.258+0000 7fae0b7fe700 1 -- 172.21.15.28:0/1910447327 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fae0c0869f0 con 0x7fae00004850 2023-11-28T11:09:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.258+0000 7fae1259c700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fadf40682f0 0x7fadf406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.258+0000 7fae1259c700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fadf40682f0 0x7fadf406a7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae140b31e0 tx=0x7fadfc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:01.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.262+0000 7fae0b7fe700 1 -- 172.21.15.28:0/1910447327 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fae0c088e60 con 0x7fae00004850 2023-11-28T11:09:01.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:01.500+0000 7fae1924d700 1 -- 172.21.15.28:0/1910447327 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fae140739f0 con 0x7fae00004850 2023-11-28T11:09:02.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.230+0000 7fae0b7fe700 1 -- 172.21.15.28:0/1910447327 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v8) v1 ==== 118+0+0 (secure 0 0 0) 0x7fae0c052f60 con 0x7fae00004850 2023-11-28T11:09:02.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.233+0000 7fae1924d700 1 -- 172.21.15.28:0/1910447327 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fadf40682f0 msgr2=0x7fadf406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.233+0000 7fae1924d700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fadf40682f0 0x7fadf406a7a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fae140b31e0 tx=0x7fadfc008030 comp rx=0 tx=0).stop 2023-11-28T11:09:02.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.233+0000 7fae1924d700 1 -- 172.21.15.28:0/1910447327 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae00004850 msgr2=0x7fae140cf390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.233+0000 7fae1924d700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae00004850 0x7fae140cf390 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fae0c00e470 tx=0x7fae0c007c90 comp rx=0 tx=0).stop 2023-11-28T11:09:02.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.233+0000 7fae1924d700 1 -- 172.21.15.28:0/1910447327 shutdown_connections 2023-11-28T11:09:02.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.233+0000 7fae1924d700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fadf40682f0 0x7fadf406a7a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.233+0000 7fae1924d700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fae00004850 0x7fae140cf390 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.233+0000 7fae1924d700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fae140b3a10 0x7fae140b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.233+0000 7fae1924d700 1 --2- 172.21.15.28:0/1910447327 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fae14158cd0 0x7fae141f1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.233+0000 7fae1924d700 1 -- 172.21.15.28:0/1910447327 >> 172.21.15.28:0/1910447327 conn(0x7fae1406e9b0 msgr2=0x7fae1406eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:02.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.233+0000 7fae1924d700 1 -- 172.21.15.28:0/1910447327 shutdown_connections 2023-11-28T11:09:02.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.233+0000 7fae1924d700 1 -- 172.21.15.28:0/1910447327 wait complete. 2023-11-28T11:09:02.250 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd']} 2023-11-28T11:09:02.250 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:09:02.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.385+0000 7f9a8e0a3700 1 Processor -- start 2023-11-28T11:09:02.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.385+0000 7f9a8e0a3700 1 -- start start 2023-11-28T11:09:02.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.385+0000 7f9a8e0a3700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9a880b1630 0x7f9a880cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.385+0000 7f9a8e0a3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9a880b1a00 0x7f9a880caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.386+0000 7f9a8e0a3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a880ce550 con 0x7f9a880cb3e0 2023-11-28T11:09:02.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.386+0000 7f9a8e0a3700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a880ce830 con 0x7f9a880b1630 2023-11-28T11:09:02.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.386+0000 7f9a8e0a3700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a880ceb10 con 0x7f9a880b1a00 2023-11-28T11:09:02.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.386+0000 7f9a877fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9a880b1630 0x7f9a880cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.386+0000 7f9a877fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9a880b1630 0x7f9a880cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46844/0 (socket says 172.21.15.28:46844) 2023-11-28T11:09:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.386+0000 7f9a877fe700 1 -- 172.21.15.28:0/918372878 learned_addr learned my addr 172.21.15.28:0/918372878 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.386+0000 7f9a86ffd700 1 --2- 172.21.15.28:0/918372878 >> v2:172.21.15.59:3300/0 conn(0x7f9a880b1a00 0x7f9a880caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.386+0000 7f9a877fe700 1 -- 172.21.15.28:0/918372878 >> v2:172.21.15.59:3300/0 conn(0x7f9a880b1a00 msgr2=0x7f9a880caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.386+0000 7f9a877fe700 1 --2- 172.21.15.28:0/918372878 >> v2:172.21.15.59:3300/0 conn(0x7f9a880b1a00 0x7f9a880caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.386+0000 7f9a877fe700 1 -- 172.21.15.28:0/918372878 >> v1:172.21.15.59:6789/0 conn(0x7f9a880cb3e0 legacy=0x7f9a88160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.386+0000 7f9a877fe700 1 -- 172.21.15.28:0/918372878 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a880cedf0 con 0x7f9a880b1630 2023-11-28T11:09:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.386+0000 7f9a877fe700 1 --2- 172.21.15.28:0/918372878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a880b1630 0x7f9a880cf5f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9a780098f0 tx=0x7f9a78009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=978f8a19c32ba086 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.386+0000 7f9a867fc700 1 -- 172.21.15.28:0/918372878 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78004230 con 0x7f9a880b1630 2023-11-28T11:09:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.386+0000 7f9a867fc700 1 -- 172.21.15.28:0/918372878 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a78004390 con 0x7f9a880b1630 2023-11-28T11:09:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.386+0000 7f9a867fc700 1 -- 172.21.15.28:0/918372878 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a780036a0 con 0x7f9a880b1630 2023-11-28T11:09:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.387+0000 7f9a8e0a3700 1 -- 172.21.15.28:0/918372878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a880b1630 msgr2=0x7f9a880cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.387+0000 7f9a8e0a3700 1 --2- 172.21.15.28:0/918372878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a880b1630 0x7f9a880cf5f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9a780098f0 tx=0x7f9a78009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:02.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.387+0000 7f9a8e0a3700 1 -- 172.21.15.28:0/918372878 shutdown_connections 2023-11-28T11:09:02.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.387+0000 7f9a8e0a3700 1 --2- 172.21.15.28:0/918372878 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a880b1630 0x7f9a880cf5f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.387+0000 7f9a8e0a3700 1 --2- 172.21.15.28:0/918372878 >> v2:172.21.15.59:3300/0 conn(0x7f9a880b1a00 0x7f9a880caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.387+0000 7f9a8e0a3700 1 -- 172.21.15.28:0/918372878 >> 172.21.15.28:0/918372878 conn(0x7f9a880cc630 msgr2=0x7f9a880cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:02.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.387+0000 7f9a8e0a3700 1 -- 172.21.15.28:0/918372878 shutdown_connections 2023-11-28T11:09:02.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.387+0000 7f9a8e0a3700 1 -- 172.21.15.28:0/918372878 wait complete. 2023-11-28T11:09:02.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.388+0000 7f9a8e0a3700 1 Processor -- start 2023-11-28T11:09:02.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.388+0000 7f9a8e0a3700 1 -- start start 2023-11-28T11:09:02.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a8e0a3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a880b1630 0x7f9a881f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a8e0a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a880b1a00 0x7f9a881f31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a8e0a3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9a880cb3e0 0x7f9a880b50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a8e0a3700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880b5610 con 0x7f9a880b1630 2023-11-28T11:09:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a8e0a3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a881f38f0 con 0x7f9a880b1a00 2023-11-28T11:09:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a8e0a3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a880b2b10 con 0x7f9a880cb3e0 2023-11-28T11:09:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a877fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a880b1630 0x7f9a881f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a877fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a880b1630 0x7f9a881f3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46850/0 (socket says 172.21.15.28:46850) 2023-11-28T11:09:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a877fe700 1 -- 172.21.15.28:0/3432416172 learned_addr learned my addr 172.21.15.28:0/3432416172 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a86ffd700 1 --2- 172.21.15.28:0/3432416172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a880b1a00 0x7f9a881f31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a87fff700 1 --2- 172.21.15.28:0/3432416172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9a880cb3e0 0x7f9a880b50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a877fe700 1 -- 172.21.15.28:0/3432416172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9a880cb3e0 msgr2=0x7f9a880b50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a877fe700 1 --2- 172.21.15.28:0/3432416172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9a880cb3e0 0x7f9a880b50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a877fe700 1 -- 172.21.15.28:0/3432416172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a880b1a00 msgr2=0x7f9a881f31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a877fe700 1 --2- 172.21.15.28:0/3432416172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a880b1a00 0x7f9a881f31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.389+0000 7f9a877fe700 1 -- 172.21.15.28:0/3432416172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a78009550 con 0x7f9a880b1630 2023-11-28T11:09:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.390+0000 7f9a877fe700 1 --2- 172.21.15.28:0/3432416172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a880b1630 0x7f9a881f3be0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009130 tx=0x7f9a78004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.390+0000 7f9a6ffff700 1 -- 172.21.15.28:0/3432416172 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78011630 con 0x7f9a880b1630 2023-11-28T11:09:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.390+0000 7f9a6ffff700 1 -- 172.21.15.28:0/3432416172 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a78011790 con 0x7f9a880b1630 2023-11-28T11:09:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.390+0000 7f9a6ffff700 1 -- 172.21.15.28:0/3432416172 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78011b40 con 0x7f9a880b1630 2023-11-28T11:09:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.390+0000 7f9a8e0a3700 1 -- 172.21.15.28:0/3432416172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a880b1ea0 con 0x7f9a880b1630 2023-11-28T11:09:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.390+0000 7f9a8e0a3700 1 -- 172.21.15.28:0/3432416172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a880cf0d0 con 0x7f9a880b1630 2023-11-28T11:09:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.391+0000 7f9a8e0a3700 1 -- 172.21.15.28:0/3432416172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a880ce140 con 0x7f9a880b1630 2023-11-28T11:09:02.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.391+0000 7f9a6ffff700 1 -- 172.21.15.28:0/3432416172 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9a78017020 con 0x7f9a880b1630 2023-11-28T11:09:02.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.391+0000 7f9a6ffff700 1 --2- 172.21.15.28:0/3432416172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9a740682f0 0x7f9a7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:02.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.392+0000 7f9a6ffff700 1 -- 172.21.15.28:0/3432416172 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9a780877e0 con 0x7f9a880b1630 2023-11-28T11:09:02.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.392+0000 7f9a86ffd700 1 --2- 172.21.15.28:0/3432416172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9a740682f0 0x7f9a7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:02.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.392+0000 7f9a86ffd700 1 --2- 172.21.15.28:0/3432416172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9a740682f0 0x7f9a7406a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a880c94b0 tx=0x7f9a7c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:02.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.396+0000 7f9a6ffff700 1 -- 172.21.15.28:0/3432416172 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9a78053c70 con 0x7f9a880b1630 2023-11-28T11:09:02.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:02.641+0000 7f9a8e0a3700 1 -- 172.21.15.28:0/3432416172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f9a880739f0 con 0x7f9a880b1630 2023-11-28T11:09:03.244 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.244+0000 7f9a6ffff700 1 -- 172.21.15.28:0/3432416172 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v9) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9a78057290 con 0x7f9a880b1630 2023-11-28T11:09:03.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.246+0000 7f9a8e0a3700 1 -- 172.21.15.28:0/3432416172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9a740682f0 msgr2=0x7f9a7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.246+0000 7f9a8e0a3700 1 --2- 172.21.15.28:0/3432416172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9a740682f0 0x7f9a7406a7a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9a880c94b0 tx=0x7f9a7c008040 comp rx=0 tx=0).stop 2023-11-28T11:09:03.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.246+0000 7f9a8e0a3700 1 -- 172.21.15.28:0/3432416172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a880b1630 msgr2=0x7f9a881f3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.246+0000 7f9a8e0a3700 1 --2- 172.21.15.28:0/3432416172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a880b1630 0x7f9a881f3be0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f9a78009130 tx=0x7f9a78004750 comp rx=0 tx=0).stop 2023-11-28T11:09:03.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.247+0000 7f9a8e0a3700 1 -- 172.21.15.28:0/3432416172 shutdown_connections 2023-11-28T11:09:03.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.247+0000 7f9a8e0a3700 1 --2- 172.21.15.28:0/3432416172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9a740682f0 0x7f9a7406a7a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.247+0000 7f9a8e0a3700 1 --2- 172.21.15.28:0/3432416172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9a880b1630 0x7f9a881f3be0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.247+0000 7f9a8e0a3700 1 --2- 172.21.15.28:0/3432416172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9a880cb3e0 0x7f9a880b50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.247+0000 7f9a8e0a3700 1 --2- 172.21.15.28:0/3432416172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9a880b1a00 0x7f9a881f31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.247+0000 7f9a8e0a3700 1 -- 172.21.15.28:0/3432416172 >> 172.21.15.28:0/3432416172 conn(0x7f9a880cc630 msgr2=0x7f9a8806b950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:03.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.247+0000 7f9a8e0a3700 1 -- 172.21.15.28:0/3432416172 shutdown_connections 2023-11-28T11:09:03.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.247+0000 7f9a8e0a3700 1 -- 172.21.15.28:0/3432416172 wait complete. 2023-11-28T11:09:03.262 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:09:03.263 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:03.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.394+0000 7f41e7497700 1 Processor -- start 2023-11-28T11:09:03.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.395+0000 7f41e7497700 1 -- start start 2023-11-28T11:09:03.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.395+0000 7f41e7497700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f41e0159590 0x7f41e006e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.395+0000 7f41e7497700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f41e015db70 0x7f41e00ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.395+0000 7f41e7497700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41e015d120 con 0x7f41e00ce890 2023-11-28T11:09:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.395+0000 7f41e7497700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41e015d6e0 con 0x7f41e0159590 2023-11-28T11:09:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.395+0000 7f41e7497700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e00aef60 con 0x7f41e015db70 2023-11-28T11:09:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.396+0000 7f41e5a34700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f41e00ce890 0x7f41e00caf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34918/0 (socket says 172.21.15.28:34918) 2023-11-28T11:09:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.396+0000 7f41e5a34700 1 -- 172.21.15.28:0/449493548 learned_addr learned my addr 172.21.15.28:0/449493548 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.396+0000 7f41e4a32700 1 --2- 172.21.15.28:0/449493548 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f41e015db70 0x7f41e00ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.396+0000 7f41e5233700 1 --2- 172.21.15.28:0/449493548 >> v2:172.21.15.59:3300/0 conn(0x7f41e0159590 0x7f41e006e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.396+0000 7f41d3fff700 1 -- 172.21.15.28:0/449493548 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182871292 0 0) 0x7f41e015d120 con 0x7f41e00ce890 2023-11-28T11:09:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.396+0000 7f41d3fff700 1 -- 172.21.15.28:0/449493548 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41cc0035b0 con 0x7f41e00ce890 2023-11-28T11:09:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.396+0000 7f41e4a32700 1 -- 172.21.15.28:0/449493548 >> v2:172.21.15.59:3300/0 conn(0x7f41e0159590 msgr2=0x7f41e006e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:03.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.396+0000 7f41e4a32700 1 --2- 172.21.15.28:0/449493548 >> v2:172.21.15.59:3300/0 conn(0x7f41e0159590 0x7f41e006e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.396+0000 7f41e4a32700 1 -- 172.21.15.28:0/449493548 >> v1:172.21.15.28:6789/0 conn(0x7f41e00ce890 legacy=0x7f41e00caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.396+0000 7f41e4a32700 1 -- 172.21.15.28:0/449493548 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41e00af240 con 0x7f41e015db70 2023-11-28T11:09:03.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.396+0000 7f41e4a32700 1 --2- 172.21.15.28:0/449493548 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f41e015db70 0x7f41e00ae730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009840 tx=0x7f41d4009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d3695f238828141b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.396+0000 7f41d3fff700 1 -- 172.21.15.28:0/449493548 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4011830 con 0x7f41e015db70 2023-11-28T11:09:03.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.396+0000 7f41d3fff700 1 -- 172.21.15.28:0/449493548 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d4011990 con 0x7f41e015db70 2023-11-28T11:09:03.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.396+0000 7f41d3fff700 1 -- 172.21.15.28:0/449493548 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d4011d00 con 0x7f41e015db70 2023-11-28T11:09:03.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.397+0000 7f41e7497700 1 -- 172.21.15.28:0/449493548 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f41e015db70 msgr2=0x7f41e00ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.397+0000 7f41e7497700 1 --2- 172.21.15.28:0/449493548 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f41e015db70 0x7f41e00ae730 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009840 tx=0x7f41d4009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:03.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.397+0000 7f41e7497700 1 -- 172.21.15.28:0/449493548 shutdown_connections 2023-11-28T11:09:03.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.397+0000 7f41e7497700 1 --2- 172.21.15.28:0/449493548 >> v2:172.21.15.59:3300/0 conn(0x7f41e0159590 0x7f41e006e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.397+0000 7f41e7497700 1 --2- 172.21.15.28:0/449493548 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f41e015db70 0x7f41e00ae730 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.397+0000 7f41e7497700 1 -- 172.21.15.28:0/449493548 >> 172.21.15.28:0/449493548 conn(0x7f41e00b1650 msgr2=0x7f41e00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:03.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.397+0000 7f41e7497700 1 -- 172.21.15.28:0/449493548 shutdown_connections 2023-11-28T11:09:03.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.397+0000 7f41e7497700 1 -- 172.21.15.28:0/449493548 wait complete. 2023-11-28T11:09:03.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.398+0000 7f41e7497700 1 Processor -- start 2023-11-28T11:09:03.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.398+0000 7f41e7497700 1 -- start start 2023-11-28T11:09:03.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.399+0000 7f41e7497700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41e00ce890 0x7f41e00afaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.399+0000 7f41e7497700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f41e0159590 0x7f41e00ad9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.399+0000 7f41e7497700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41e015db70 0x7f41e00adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.399+0000 7f41e7497700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e00c7c00 con 0x7f41e00ce890 2023-11-28T11:09:03.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.399+0000 7f41e7497700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41e00c7ee0 con 0x7f41e015db70 2023-11-28T11:09:03.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.399+0000 7f41e7497700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41e00c81c0 con 0x7f41e0159590 2023-11-28T11:09:03.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.399+0000 7f41e5233700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41e00ce890 0x7f41e00afaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.399+0000 7f41e5233700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41e00ce890 0x7f41e00afaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46864/0 (socket says 172.21.15.28:46864) 2023-11-28T11:09:03.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.399+0000 7f41e5233700 1 -- 172.21.15.28:0/1347538907 learned_addr learned my addr 172.21.15.28:0/1347538907 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:03.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.399+0000 7f41e4a32700 1 --2- 172.21.15.28:0/1347538907 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f41e0159590 0x7f41e00ad9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.399+0000 7f41e5a34700 1 --2- 172.21.15.28:0/1347538907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41e015db70 0x7f41e00adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.400+0000 7f41e5233700 1 -- 172.21.15.28:0/1347538907 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f41e0159590 msgr2=0x7f41e00ad9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.400+0000 7f41e5233700 1 --2- 172.21.15.28:0/1347538907 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f41e0159590 0x7f41e00ad9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.400+0000 7f41e5233700 1 -- 172.21.15.28:0/1347538907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41e015db70 msgr2=0x7f41e00adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.400+0000 7f41e5233700 1 --2- 172.21.15.28:0/1347538907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41e015db70 0x7f41e00adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.400+0000 7f41e5233700 1 -- 172.21.15.28:0/1347538907 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d40094a0 con 0x7f41e00ce890 2023-11-28T11:09:03.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.400+0000 7f41e5233700 1 --2- 172.21.15.28:0/1347538907 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41e00ce890 0x7f41e00afaa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00e9b0 tx=0x7f41dc00ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.400+0000 7f41d1ffb700 1 -- 172.21.15.28:0/1347538907 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc009710 con 0x7f41e00ce890 2023-11-28T11:09:03.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.400+0000 7f41d1ffb700 1 -- 172.21.15.28:0/1347538907 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc00ee50 con 0x7f41e00ce890 2023-11-28T11:09:03.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.400+0000 7f41d1ffb700 1 -- 172.21.15.28:0/1347538907 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc00ca50 con 0x7f41e00ce890 2023-11-28T11:09:03.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.400+0000 7f41e7497700 1 -- 172.21.15.28:0/1347538907 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41e00aec70 con 0x7f41e00ce890 2023-11-28T11:09:03.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.400+0000 7f41e7497700 1 -- 172.21.15.28:0/1347538907 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41e015d370 con 0x7f41e00ce890 2023-11-28T11:09:03.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.401+0000 7f41e7497700 1 -- 172.21.15.28:0/1347538907 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41e015dfe0 con 0x7f41e00ce890 2023-11-28T11:09:03.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.402+0000 7f41d1ffb700 1 -- 172.21.15.28:0/1347538907 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f41dc023070 con 0x7f41e00ce890 2023-11-28T11:09:03.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.402+0000 7f41d1ffb700 1 --2- 172.21.15.28:0/1347538907 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f41c40682f0 0x7f41c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.402+0000 7f41d1ffb700 1 -- 172.21.15.28:0/1347538907 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f41dc086220 con 0x7f41e00ce890 2023-11-28T11:09:03.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.402+0000 7f41e4a32700 1 --2- 172.21.15.28:0/1347538907 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f41c40682f0 0x7f41c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.402+0000 7f41e4a32700 1 --2- 172.21.15.28:0/1347538907 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f41c40682f0 0x7f41c406a7a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009fd0 tx=0x7f41d401c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.406+0000 7f41d1ffb700 1 -- 172.21.15.28:0/1347538907 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f41dc0526b0 con 0x7f41e00ce890 2023-11-28T11:09:03.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.644+0000 7f41e7497700 1 -- 172.21.15.28:0/1347538907 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41e01f3830 con 0x7f41e00ce890 2023-11-28T11:09:03.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.645+0000 7f41d1ffb700 1 -- 172.21.15.28:0/1347538907 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4644 (secure 0 0 0) 0x7f41dc055cd0 con 0x7f41e00ce890 2023-11-28T11:09:03.646 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:03.646 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/3112877467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3112877467},{"type":"v1","addr":"172.21.15.59:6834","nonce":3112877467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3448236489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3448236489},{"type":"v1","addr":"172.21.15.28:6836","nonce":3448236489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4232269940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4232269940},{"type":"v1","addr":"172.21.15.28:6837","nonce":4232269940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:45.924816+0000","modified":"2023-11-28T11:09:03.234785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/3293053261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3293053261},{"type":"v1","addr":"172.21.15.59:6835","nonce":3293053261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:09:03.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.647+0000 7f41e7497700 1 -- 172.21.15.28:0/1347538907 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f41c40682f0 msgr2=0x7f41c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.647+0000 7f41e7497700 1 --2- 172.21.15.28:0/1347538907 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f41c40682f0 0x7f41c406a7a0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f41d4009fd0 tx=0x7f41d401c040 comp rx=0 tx=0).stop 2023-11-28T11:09:03.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.647+0000 7f41e7497700 1 -- 172.21.15.28:0/1347538907 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41e00ce890 msgr2=0x7f41e00afaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.647+0000 7f41e7497700 1 --2- 172.21.15.28:0/1347538907 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41e00ce890 0x7f41e00afaa0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f41dc00e9b0 tx=0x7f41dc00ecc0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.647+0000 7f41e7497700 1 -- 172.21.15.28:0/1347538907 shutdown_connections 2023-11-28T11:09:03.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.647+0000 7f41e7497700 1 --2- 172.21.15.28:0/1347538907 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f41c40682f0 0x7f41c406a7a0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.647+0000 7f41e7497700 1 --2- 172.21.15.28:0/1347538907 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41e00ce890 0x7f41e00afaa0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.647+0000 7f41e7497700 1 --2- 172.21.15.28:0/1347538907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41e015db70 0x7f41e00adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.647+0000 7f41e7497700 1 --2- 172.21.15.28:0/1347538907 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f41e0159590 0x7f41e00ad9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.647+0000 7f41e7497700 1 -- 172.21.15.28:0/1347538907 >> 172.21.15.28:0/1347538907 conn(0x7f41e00b1650 msgr2=0x7f41e006b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:03.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.647+0000 7f41e7497700 1 -- 172.21.15.28:0/1347538907 shutdown_connections 2023-11-28T11:09:03.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.647+0000 7f41e7497700 1 -- 172.21.15.28:0/1347538907 wait complete. 2023-11-28T11:09:03.649 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 9 2023-11-28T11:09:03.660 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:03.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.783+0000 7fe4f2a4c700 1 Processor -- start 2023-11-28T11:09:03.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.783+0000 7fe4f2a4c700 1 -- start start 2023-11-28T11:09:03.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.784+0000 7fe4f2a4c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4ec15c990 0x7fe4ec0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.784+0000 7fe4f2a4c700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ec0ce340 con 0x7fe4ec0cc310 2023-11-28T11:09:03.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.784+0000 7fe4f2a4c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4ec0cebe0 con 0x7fe4ec161090 2023-11-28T11:09:03.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.784+0000 7fe4f2a4c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec0cb3f0 con 0x7fe4ec15c990 2023-11-28T11:09:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.784+0000 7fe4f0fe9700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe4ec0cc310 0x7fe4ec0cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34934/0 (socket says 172.21.15.28:34934) 2023-11-28T11:09:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.784+0000 7fe4f0fe9700 1 -- 172.21.15.28:0/313623584 learned_addr learned my addr 172.21.15.28:0/313623584 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.784+0000 7fe4ebfff700 1 --2- 172.21.15.28:0/313623584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4ec15c990 0x7fe4ec0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.784+0000 7fe4eaffd700 1 -- 172.21.15.28:0/313623584 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985286986 0 0) 0x7fe4ec0ce340 con 0x7fe4ec0cc310 2023-11-28T11:09:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.784+0000 7fe4eaffd700 1 -- 172.21.15.28:0/313623584 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4d80035b0 con 0x7fe4ec0cc310 2023-11-28T11:09:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.784+0000 7fe4ebfff700 1 -- 172.21.15.28:0/313623584 >> v1:172.21.15.59:6789/0 conn(0x7fe4ec161090 legacy=0x7fe4ec0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.784+0000 7fe4ebfff700 1 -- 172.21.15.28:0/313623584 >> v1:172.21.15.28:6789/0 conn(0x7fe4ec0cc310 legacy=0x7fe4ec0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.784+0000 7fe4eaffd700 1 -- 172.21.15.28:0/313623584 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2300346627 0 0) 0x7fe4ec0cebe0 con 0x7fe4ec161090 2023-11-28T11:09:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.784+0000 7fe4ebfff700 1 -- 172.21.15.28:0/313623584 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4ec0ce590 con 0x7fe4ec15c990 2023-11-28T11:09:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.785+0000 7fe4ebfff700 1 --2- 172.21.15.28:0/313623584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4ec15c990 0x7fe4ec0cf070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009870 tx=0x7fe4dc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0a156199db5b8d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.785+0000 7fe4eaffd700 1 -- 172.21.15.28:0/313623584 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc011830 con 0x7fe4ec15c990 2023-11-28T11:09:03.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.785+0000 7fe4eaffd700 1 -- 172.21.15.28:0/313623584 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4dc011990 con 0x7fe4ec15c990 2023-11-28T11:09:03.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.785+0000 7fe4eaffd700 1 -- 172.21.15.28:0/313623584 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4dc011d00 con 0x7fe4ec15c990 2023-11-28T11:09:03.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.785+0000 7fe4f2a4c700 1 -- 172.21.15.28:0/313623584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4ec15c990 msgr2=0x7fe4ec0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.785+0000 7fe4f2a4c700 1 --2- 172.21.15.28:0/313623584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4ec15c990 0x7fe4ec0cf070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc009870 tx=0x7fe4dc009b80 comp rx=0 tx=0).stop 2023-11-28T11:09:03.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.785+0000 7fe4f2a4c700 1 -- 172.21.15.28:0/313623584 shutdown_connections 2023-11-28T11:09:03.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.785+0000 7fe4f2a4c700 1 --2- 172.21.15.28:0/313623584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4ec15c990 0x7fe4ec0cf070 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.785+0000 7fe4f2a4c700 1 -- 172.21.15.28:0/313623584 >> 172.21.15.28:0/313623584 conn(0x7fe4ec0b1650 msgr2=0x7fe4ec0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:03.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.785+0000 7fe4f2a4c700 1 -- 172.21.15.28:0/313623584 shutdown_connections 2023-11-28T11:09:03.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.785+0000 7fe4f2a4c700 1 -- 172.21.15.28:0/313623584 wait complete. 2023-11-28T11:09:03.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.787+0000 7fe4f2a4c700 1 Processor -- start 2023-11-28T11:09:03.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.787+0000 7fe4f2a4c700 1 -- start start 2023-11-28T11:09:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.787+0000 7fe4f2a4c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4ec0cc310 0x7fe4ec15b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.787+0000 7fe4f2a4c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4ec15c990 0x7fe4ec0b3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.787+0000 7fe4f2a4c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4ec161090 0x7fe4ec0c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.787+0000 7fe4f2a4c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec0b4050 con 0x7fe4ec15c990 2023-11-28T11:09:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.787+0000 7fe4f2a4c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec0ad3e0 con 0x7fe4ec0cc310 2023-11-28T11:09:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.787+0000 7fe4f2a4c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4ec0b3d80 con 0x7fe4ec161090 2023-11-28T11:09:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4eb7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4ec15c990 0x7fe4ec0b3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4eb7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4ec15c990 0x7fe4ec0b3840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46880/0 (socket says 172.21.15.28:46880) 2023-11-28T11:09:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4ebfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4ec0cc310 0x7fe4ec15b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4eb7fe700 1 -- 172.21.15.28:0/3704560535 learned_addr learned my addr 172.21.15.28:0/3704560535 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4f0fe9700 1 --2- 172.21.15.28:0/3704560535 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4ec161090 0x7fe4ec0c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4eb7fe700 1 -- 172.21.15.28:0/3704560535 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4ec161090 msgr2=0x7fe4ec0c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4eb7fe700 1 --2- 172.21.15.28:0/3704560535 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4ec161090 0x7fe4ec0c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4eb7fe700 1 -- 172.21.15.28:0/3704560535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4ec0cc310 msgr2=0x7fe4ec15b110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4eb7fe700 1 --2- 172.21.15.28:0/3704560535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4ec0cc310 0x7fe4ec15b110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4eb7fe700 1 -- 172.21.15.28:0/3704560535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4dc0094a0 con 0x7fe4ec15c990 2023-11-28T11:09:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4eb7fe700 1 --2- 172.21.15.28:0/3704560535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4ec15c990 0x7fe4ec0b3840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400c520 tx=0x7fe4d400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4e8ff9700 1 -- 172.21.15.28:0/3704560535 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d40054e0 con 0x7fe4ec15c990 2023-11-28T11:09:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4e8ff9700 1 -- 172.21.15.28:0/3704560535 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4d4003680 con 0x7fe4ec15c990 2023-11-28T11:09:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4f2a4c700 1 -- 172.21.15.28:0/3704560535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4ec0cb080 con 0x7fe4ec15c990 2023-11-28T11:09:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4e8ff9700 1 -- 172.21.15.28:0/3704560535 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4d4003a30 con 0x7fe4ec15c990 2023-11-28T11:09:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.788+0000 7fe4f2a4c700 1 -- 172.21.15.28:0/3704560535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4ec0ce870 con 0x7fe4ec15c990 2023-11-28T11:09:03.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.789+0000 7fe4f2a4c700 1 -- 172.21.15.28:0/3704560535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4ec0cf5b0 con 0x7fe4ec15c990 2023-11-28T11:09:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.790+0000 7fe4e8ff9700 1 -- 172.21.15.28:0/3704560535 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe4d4019030 con 0x7fe4ec15c990 2023-11-28T11:09:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.790+0000 7fe4e8ff9700 1 --2- 172.21.15.28:0/3704560535 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe4cc0682f0 0x7fe4cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.790+0000 7fe4e8ff9700 1 -- 172.21.15.28:0/3704560535 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe4d4085250 con 0x7fe4ec15c990 2023-11-28T11:09:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.790+0000 7fe4ebfff700 1 --2- 172.21.15.28:0/3704560535 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe4cc0682f0 0x7fe4cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.790+0000 7fe4ebfff700 1 --2- 172.21.15.28:0/3704560535 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe4cc0682f0 0x7fe4cc06a7a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0090b0 tx=0x7fe4dc01c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:03.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.794+0000 7fe4e8ff9700 1 -- 172.21.15.28:0/3704560535 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe4d40517c0 con 0x7fe4ec15c990 2023-11-28T11:09:03.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.988+0000 7fe4f2a4c700 1 -- 172.21.15.28:0/3704560535 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe4ec0675a0 con 0x7fe4ec15c990 2023-11-28T11:09:03.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.989+0000 7fe4e8ff9700 1 -- 172.21.15.28:0/3704560535 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19642 (secure 0 0 0) 0x7fe4d4017090 con 0x7fe4ec15c990 2023-11-28T11:09:03.990 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:03.990 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:50.531733+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:03.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.991+0000 7fe4f2a4c700 1 -- 172.21.15.28:0/3704560535 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe4cc0682f0 msgr2=0x7fe4cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.991+0000 7fe4f2a4c700 1 --2- 172.21.15.28:0/3704560535 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe4cc0682f0 0x7fe4cc06a7a0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe4dc0090b0 tx=0x7fe4dc01c040 comp rx=0 tx=0).stop 2023-11-28T11:09:03.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.991+0000 7fe4f2a4c700 1 -- 172.21.15.28:0/3704560535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4ec15c990 msgr2=0x7fe4ec0b3840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:03.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.991+0000 7fe4f2a4c700 1 --2- 172.21.15.28:0/3704560535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4ec15c990 0x7fe4ec0b3840 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe4d400c520 tx=0x7fe4d400c830 comp rx=0 tx=0).stop 2023-11-28T11:09:03.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.991+0000 7fe4f2a4c700 1 -- 172.21.15.28:0/3704560535 shutdown_connections 2023-11-28T11:09:03.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.991+0000 7fe4f2a4c700 1 --2- 172.21.15.28:0/3704560535 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe4cc0682f0 0x7fe4cc06a7a0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.991+0000 7fe4f2a4c700 1 --2- 172.21.15.28:0/3704560535 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4ec161090 0x7fe4ec0c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.991+0000 7fe4f2a4c700 1 --2- 172.21.15.28:0/3704560535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4ec0cc310 0x7fe4ec15b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.991+0000 7fe4f2a4c700 1 --2- 172.21.15.28:0/3704560535 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4ec15c990 0x7fe4ec0b3840 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:03.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.991+0000 7fe4f2a4c700 1 -- 172.21.15.28:0/3704560535 >> 172.21.15.28:0/3704560535 conn(0x7fe4ec0b1650 msgr2=0x7fe4ec06b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:03.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.991+0000 7fe4f2a4c700 1 -- 172.21.15.28:0/3704560535 shutdown_connections 2023-11-28T11:09:03.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:03.991+0000 7fe4f2a4c700 1 -- 172.21.15.28:0/3704560535 wait complete. 2023-11-28T11:09:04.004 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:09:04.005 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:09:04.129 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.128+0000 7f82140e4700 1 Processor -- start 2023-11-28T11:09:04.129 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.128+0000 7f82140e4700 1 -- start start 2023-11-28T11:09:04.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.129+0000 7f82140e4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f820c159590 0x7f820c06e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.129+0000 7f82140e4700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820c15d120 con 0x7f820c0ce890 2023-11-28T11:09:04.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.129+0000 7f82140e4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f820c15d6e0 con 0x7f820c15db70 2023-11-28T11:09:04.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.129+0000 7f82140e4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0aef60 con 0x7f820c159590 2023-11-28T11:09:04.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.129+0000 7f8212681700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f820c0ce890 0x7f820c0caf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34948/0 (socket says 172.21.15.28:34948) 2023-11-28T11:09:04.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.129+0000 7f8212681700 1 -- 172.21.15.28:0/3294194846 learned_addr learned my addr 172.21.15.28:0/3294194846 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:04.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.129+0000 7f8211e80700 1 --2- 172.21.15.28:0/3294194846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f820c159590 0x7f820c06e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.130+0000 7f8210e7e700 1 -- 172.21.15.28:0/3294194846 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 456526942 0 0) 0x7f820c15d120 con 0x7f820c0ce890 2023-11-28T11:09:04.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.130+0000 7f8210e7e700 1 -- 172.21.15.28:0/3294194846 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81f80035b0 con 0x7f820c0ce890 2023-11-28T11:09:04.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.130+0000 7f8210e7e700 1 -- 172.21.15.28:0/3294194846 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990463033 0 0) 0x7f820c15d6e0 con 0x7f820c15db70 2023-11-28T11:09:04.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.130+0000 7f8211e80700 1 -- 172.21.15.28:0/3294194846 >> v1:172.21.15.59:6789/0 conn(0x7f820c15db70 legacy=0x7f820c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.130+0000 7f8211e80700 1 -- 172.21.15.28:0/3294194846 >> v1:172.21.15.28:6789/0 conn(0x7f820c0ce890 legacy=0x7f820c0caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.130+0000 7f8211e80700 1 -- 172.21.15.28:0/3294194846 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f820c0af240 con 0x7f820c159590 2023-11-28T11:09:04.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.130+0000 7f8211e80700 1 --2- 172.21.15.28:0/3294194846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f820c159590 0x7f820c06e400 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009870 tx=0x7f81fc009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d06905c3fc77cda server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:04.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.130+0000 7f8210e7e700 1 -- 172.21.15.28:0/3294194846 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc011830 con 0x7f820c159590 2023-11-28T11:09:04.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.130+0000 7f8210e7e700 1 -- 172.21.15.28:0/3294194846 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc011990 con 0x7f820c159590 2023-11-28T11:09:04.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.130+0000 7f8210e7e700 1 -- 172.21.15.28:0/3294194846 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc011d00 con 0x7f820c159590 2023-11-28T11:09:04.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.131+0000 7f82140e4700 1 -- 172.21.15.28:0/3294194846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f820c159590 msgr2=0x7f820c06e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.131+0000 7f82140e4700 1 --2- 172.21.15.28:0/3294194846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f820c159590 0x7f820c06e400 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f81fc009870 tx=0x7f81fc009b80 comp rx=0 tx=0).stop 2023-11-28T11:09:04.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.131+0000 7f82140e4700 1 -- 172.21.15.28:0/3294194846 shutdown_connections 2023-11-28T11:09:04.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.131+0000 7f82140e4700 1 --2- 172.21.15.28:0/3294194846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f820c159590 0x7f820c06e400 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.131+0000 7f82140e4700 1 -- 172.21.15.28:0/3294194846 >> 172.21.15.28:0/3294194846 conn(0x7f820c0b1650 msgr2=0x7f820c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:04.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.131+0000 7f82140e4700 1 -- 172.21.15.28:0/3294194846 shutdown_connections 2023-11-28T11:09:04.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.131+0000 7f82140e4700 1 -- 172.21.15.28:0/3294194846 wait complete. 2023-11-28T11:09:04.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.132+0000 7f82140e4700 1 Processor -- start 2023-11-28T11:09:04.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.132+0000 7f82140e4700 1 -- start start 2023-11-28T11:09:04.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f82140e4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f820c0ce890 0x7f820c156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f82140e4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f820c159590 0x7f820c153690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f82140e4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f820c15db70 0x7f820c0c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f82140e4700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c0c9640 con 0x7f820c0ce890 2023-11-28T11:09:04.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f82140e4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f820c155a50 con 0x7f820c159590 2023-11-28T11:09:04.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f82140e4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f820c155d30 con 0x7f820c15db70 2023-11-28T11:09:04.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f8211e80700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f820c0ce890 0x7f820c156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f8211e80700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f820c0ce890 0x7f820c156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46892/0 (socket says 172.21.15.28:46892) 2023-11-28T11:09:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f8211e80700 1 -- 172.21.15.28:0/3855510677 learned_addr learned my addr 172.21.15.28:0/3855510677 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f821167f700 1 --2- 172.21.15.28:0/3855510677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f820c159590 0x7f820c153690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f8212681700 1 --2- 172.21.15.28:0/3855510677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f820c15db70 0x7f820c0c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f8211e80700 1 -- 172.21.15.28:0/3855510677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f820c15db70 msgr2=0x7f820c0c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f8211e80700 1 --2- 172.21.15.28:0/3855510677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f820c15db70 0x7f820c0c8b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f8211e80700 1 -- 172.21.15.28:0/3855510677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f820c159590 msgr2=0x7f820c153690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f8211e80700 1 --2- 172.21.15.28:0/3855510677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f820c159590 0x7f820c153690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f8211e80700 1 -- 172.21.15.28:0/3855510677 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc0094a0 con 0x7f820c0ce890 2023-11-28T11:09:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f8211e80700 1 --2- 172.21.15.28:0/3855510677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f820c0ce890 0x7f820c156530 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0090b0 tx=0x7f81fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:04.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.133+0000 7f82027fc700 1 -- 172.21.15.28:0/3855510677 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc0036a0 con 0x7f820c0ce890 2023-11-28T11:09:04.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.134+0000 7f82027fc700 1 -- 172.21.15.28:0/3855510677 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81fc01f070 con 0x7f820c0ce890 2023-11-28T11:09:04.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.134+0000 7f82027fc700 1 -- 172.21.15.28:0/3855510677 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81fc003a40 con 0x7f820c0ce890 2023-11-28T11:09:04.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.134+0000 7f82140e4700 1 -- 172.21.15.28:0/3855510677 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f820c15dfe0 con 0x7f820c0ce890 2023-11-28T11:09:04.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.134+0000 7f82140e4700 1 -- 172.21.15.28:0/3855510677 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f820c15d370 con 0x7f820c0ce890 2023-11-28T11:09:04.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.134+0000 7f82140e4700 1 -- 172.21.15.28:0/3855510677 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f820c159960 con 0x7f820c0ce890 2023-11-28T11:09:04.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.135+0000 7f82027fc700 1 -- 172.21.15.28:0/3855510677 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f81fc00f050 con 0x7f820c0ce890 2023-11-28T11:09:04.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.135+0000 7f82027fc700 1 --2- 172.21.15.28:0/3855510677 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f81f00682f0 0x7f81f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.135+0000 7f82027fc700 1 -- 172.21.15.28:0/3855510677 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f81fc021070 con 0x7f820c0ce890 2023-11-28T11:09:04.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.135+0000 7f821167f700 1 --2- 172.21.15.28:0/3855510677 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f81f00682f0 0x7f81f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.136+0000 7f821167f700 1 --2- 172.21.15.28:0/3855510677 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f81f00682f0 0x7f81f006a7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f820c156c00 tx=0x7f8208009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:04.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.140+0000 7f82027fc700 1 -- 172.21.15.28:0/3855510677 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f81fc053ed0 con 0x7f820c0ce890 2023-11-28T11:09:04.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.331+0000 7f82140e4700 1 -- 172.21.15.28:0/3855510677 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f820c0675a0 con 0x7f81f00682f0 2023-11-28T11:09:04.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.342+0000 7f82027fc700 1 -- 172.21.15.28:0/3855510677 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367554 (secure 0 0 0) 0x7f820c0675a0 con 0x7f81f00682f0 2023-11-28T11:09:04.348 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:04.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.346+0000 7f82140e4700 1 -- 172.21.15.28:0/3855510677 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f81f00682f0 msgr2=0x7f81f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.346+0000 7f82140e4700 1 --2- 172.21.15.28:0/3855510677 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f81f00682f0 0x7f81f006a7a0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f820c156c00 tx=0x7f8208009040 comp rx=0 tx=0).stop 2023-11-28T11:09:04.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.346+0000 7f82140e4700 1 -- 172.21.15.28:0/3855510677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f820c0ce890 msgr2=0x7f820c156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.346+0000 7f82140e4700 1 --2- 172.21.15.28:0/3855510677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f820c0ce890 0x7f820c156530 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f81fc0090b0 tx=0x7f81fc004750 comp rx=0 tx=0).stop 2023-11-28T11:09:04.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.346+0000 7f82140e4700 1 -- 172.21.15.28:0/3855510677 shutdown_connections 2023-11-28T11:09:04.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.346+0000 7f82140e4700 1 --2- 172.21.15.28:0/3855510677 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f81f00682f0 0x7f81f006a7a0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.346+0000 7f82140e4700 1 --2- 172.21.15.28:0/3855510677 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f820c0ce890 0x7f820c156530 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.346+0000 7f82140e4700 1 --2- 172.21.15.28:0/3855510677 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f820c15db70 0x7f820c0c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.346+0000 7f82140e4700 1 --2- 172.21.15.28:0/3855510677 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f820c159590 0x7f820c153690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.346+0000 7f82140e4700 1 -- 172.21.15.28:0/3855510677 >> 172.21.15.28:0/3855510677 conn(0x7f820c0b1650 msgr2=0x7f820c06aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:04.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.346+0000 7f82140e4700 1 -- 172.21.15.28:0/3855510677 shutdown_connections 2023-11-28T11:09:04.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.346+0000 7f82140e4700 1 -- 172.21.15.28:0/3855510677 wait complete. 2023-11-28T11:09:04.354 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:09:04.360 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-11-28T11:09:03.165857+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594755,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67236,"kb_used_data":3300,"kb_used_omap":0,"kb_used_meta":63872,"kb_avail":754907484,"statfs":{"total":773094113280,"available":773025263616,"internally_reserved":0,"allocated":3379200,"data_stored":2063757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65404928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2057,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002638"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108863+0000","last_change":"2023-11-28T11:08:45.948753+0000","last_active":"2023-11-28T11:08:50.108863+0000","last_peered":"2023-11-28T11:08:50.108863+0000","last_clean":"2023-11-28T11:08:50.108863+0000","last_became_active":"2023-11-28T11:08:45.948435+0000","last_became_peered":"2023-11-28T11:08:45.948435+0000","last_unstale":"2023-11-28T11:08:50.108863+0000","last_undegraded":"2023-11-28T11:08:50.108863+0000","last_fullsized":"2023-11-28T11:08:50.108863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:28.944484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105146+0000","last_change":"2023-11-28T11:08:45.946452+0000","last_active":"2023-11-28T11:08:50.105146+0000","last_peered":"2023-11-28T11:08:50.105146+0000","last_clean":"2023-11-28T11:08:50.105146+0000","last_became_active":"2023-11-28T11:08:45.946145+0000","last_became_peered":"2023-11-28T11:08:45.946145+0000","last_unstale":"2023-11-28T11:08:50.105146+0000","last_undegraded":"2023-11-28T11:08:50.105146+0000","last_fullsized":"2023-11-28T11:08:50.105146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:30.639887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113411+0000","last_change":"2023-11-28T11:08:45.941140+0000","last_active":"2023-11-28T11:08:50.113411+0000","last_peered":"2023-11-28T11:08:50.113411+0000","last_clean":"2023-11-28T11:08:50.113411+0000","last_became_active":"2023-11-28T11:08:45.939618+0000","last_became_peered":"2023-11-28T11:08:45.939618+0000","last_unstale":"2023-11-28T11:08:50.113411+0000","last_undegraded":"2023-11-28T11:08:50.113411+0000","last_fullsized":"2023-11-28T11:08:50.113411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:44.444264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109455+0000","last_change":"2023-11-28T11:08:45.947665+0000","last_active":"2023-11-28T11:08:50.109455+0000","last_peered":"2023-11-28T11:08:50.109455+0000","last_clean":"2023-11-28T11:08:50.109455+0000","last_became_active":"2023-11-28T11:08:45.947338+0000","last_became_peered":"2023-11-28T11:08:45.947338+0000","last_unstale":"2023-11-28T11:08:50.109455+0000","last_undegraded":"2023-11-28T11:08:50.109455+0000","last_fullsized":"2023-11-28T11:08:50.109455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:23:53.819338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103496+0000","last_change":"2023-11-28T11:08:45.950223+0000","last_active":"2023-11-28T11:08:50.103496+0000","last_peered":"2023-11-28T11:08:50.103496+0000","last_clean":"2023-11-28T11:08:50.103496+0000","last_became_active":"2023-11-28T11:08:45.949811+0000","last_became_peered":"2023-11-28T11:08:45.949811+0000","last_unstale":"2023-11-28T11:08:50.103496+0000","last_undegraded":"2023-11-28T11:08:50.103496+0000","last_fullsized":"2023-11-28T11:08:50.103496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112890+0000","last_change":"2023-11-28T11:08:45.942787+0000","last_active":"2023-11-28T11:08:50.112890+0000","last_peered":"2023-11-28T11:08:50.112890+0000","last_clean":"2023-11-28T11:08:50.112890+0000","last_became_active":"2023-11-28T11:08:45.941820+0000","last_became_peered":"2023-11-28T11:08:45.941820+0000","last_unstale":"2023-11-28T11:08:50.112890+0000","last_undegraded":"2023-11-28T11:08:50.112890+0000","last_fullsized":"2023-11-28T11:08:50.112890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:22.504112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115835+0000","last_change":"2023-11-28T11:08:45.953364+0000","last_active":"2023-11-28T11:08:50.115835+0000","last_peered":"2023-11-28T11:08:50.115835+0000","last_clean":"2023-11-28T11:08:50.115835+0000","last_became_active":"2023-11-28T11:08:45.953033+0000","last_became_peered":"2023-11-28T11:08:45.953033+0000","last_unstale":"2023-11-28T11:08:50.115835+0000","last_undegraded":"2023-11-28T11:08:50.115835+0000","last_fullsized":"2023-11-28T11:08:50.115835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105175+0000","last_change":"2023-11-28T11:08:45.947045+0000","last_active":"2023-11-28T11:08:50.105175+0000","last_peered":"2023-11-28T11:08:50.105175+0000","last_clean":"2023-11-28T11:08:50.105175+0000","last_became_active":"2023-11-28T11:08:45.946756+0000","last_became_peered":"2023-11-28T11:08:45.946756+0000","last_unstale":"2023-11-28T11:08:50.105175+0000","last_undegraded":"2023-11-28T11:08:50.105175+0000","last_fullsized":"2023-11-28T11:08:50.105175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:56.811805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103692+0000","last_change":"2023-11-28T11:08:45.946321+0000","last_active":"2023-11-28T11:08:50.103692+0000","last_peered":"2023-11-28T11:08:50.103692+0000","last_clean":"2023-11-28T11:08:50.103692+0000","last_became_active":"2023-11-28T11:08:45.943687+0000","last_became_peered":"2023-11-28T11:08:45.943687+0000","last_unstale":"2023-11-28T11:08:50.103692+0000","last_undegraded":"2023-11-28T11:08:50.103692+0000","last_fullsized":"2023-11-28T11:08:50.103692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:23.627932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105317+0000","last_change":"2023-11-28T11:08:45.948529+0000","last_active":"2023-11-28T11:08:50.105317+0000","last_peered":"2023-11-28T11:08:50.105317+0000","last_clean":"2023-11-28T11:08:50.105317+0000","last_became_active":"2023-11-28T11:08:45.948249+0000","last_became_peered":"2023-11-28T11:08:45.948249+0000","last_unstale":"2023-11-28T11:08:50.105317+0000","last_undegraded":"2023-11-28T11:08:50.105317+0000","last_fullsized":"2023-11-28T11:08:50.105317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113311+0000","last_change":"2023-11-28T11:08:45.942374+0000","last_active":"2023-11-28T11:08:50.113311+0000","last_peered":"2023-11-28T11:08:50.113311+0000","last_clean":"2023-11-28T11:08:50.113311+0000","last_became_active":"2023-11-28T11:08:45.942083+0000","last_became_peered":"2023-11-28T11:08:45.942083+0000","last_unstale":"2023-11-28T11:08:50.113311+0000","last_undegraded":"2023-11-28T11:08:50.113311+0000","last_fullsized":"2023-11-28T11:08:50.113311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:23.627932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108474+0000","last_change":"2023-11-28T11:08:45.947339+0000","last_active":"2023-11-28T11:08:50.108474+0000","last_peered":"2023-11-28T11:08:50.108474+0000","last_clean":"2023-11-28T11:08:50.108474+0000","last_became_active":"2023-11-28T11:08:45.947066+0000","last_became_peered":"2023-11-28T11:08:45.947066+0000","last_unstale":"2023-11-28T11:08:50.108474+0000","last_undegraded":"2023-11-28T11:08:50.108474+0000","last_fullsized":"2023-11-28T11:08:50.108474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:48.010812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109210+0000","last_change":"2023-11-28T11:08:45.949094+0000","last_active":"2023-11-28T11:08:50.109210+0000","last_peered":"2023-11-28T11:08:50.109210+0000","last_clean":"2023-11-28T11:08:50.109210+0000","last_became_active":"2023-11-28T11:08:45.948523+0000","last_became_peered":"2023-11-28T11:08:45.948523+0000","last_unstale":"2023-11-28T11:08:50.109210+0000","last_undegraded":"2023-11-28T11:08:50.109210+0000","last_fullsized":"2023-11-28T11:08:50.109210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:34.193700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105493+0000","last_change":"2023-11-28T11:08:45.945708+0000","last_active":"2023-11-28T11:08:50.105493+0000","last_peered":"2023-11-28T11:08:50.105493+0000","last_clean":"2023-11-28T11:08:50.105493+0000","last_became_active":"2023-11-28T11:08:45.945408+0000","last_became_peered":"2023-11-28T11:08:45.945408+0000","last_unstale":"2023-11-28T11:08:50.105493+0000","last_undegraded":"2023-11-28T11:08:50.105493+0000","last_fullsized":"2023-11-28T11:08:50.105493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:34.416718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115608+0000","last_change":"2023-11-28T11:08:45.954131+0000","last_active":"2023-11-28T11:08:50.115608+0000","last_peered":"2023-11-28T11:08:50.115608+0000","last_clean":"2023-11-28T11:08:50.115608+0000","last_became_active":"2023-11-28T11:08:45.953824+0000","last_became_peered":"2023-11-28T11:08:45.953824+0000","last_unstale":"2023-11-28T11:08:50.115608+0000","last_undegraded":"2023-11-28T11:08:50.115608+0000","last_fullsized":"2023-11-28T11:08:50.115608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113028+0000","last_change":"2023-11-28T11:08:45.943568+0000","last_active":"2023-11-28T11:08:50.113028+0000","last_peered":"2023-11-28T11:08:50.113028+0000","last_clean":"2023-11-28T11:08:50.113028+0000","last_became_active":"2023-11-28T11:08:45.942637+0000","last_became_peered":"2023-11-28T11:08:45.942637+0000","last_unstale":"2023-11-28T11:08:50.113028+0000","last_undegraded":"2023-11-28T11:08:50.113028+0000","last_fullsized":"2023-11-28T11:08:50.113028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:17.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106326+0000","last_change":"2023-11-28T11:08:45.944177+0000","last_active":"2023-11-28T11:08:50.106326+0000","last_peered":"2023-11-28T11:08:50.106326+0000","last_clean":"2023-11-28T11:08:50.106326+0000","last_became_active":"2023-11-28T11:08:45.943369+0000","last_became_peered":"2023-11-28T11:08:45.943369+0000","last_unstale":"2023-11-28T11:08:50.106326+0000","last_undegraded":"2023-11-28T11:08:50.106326+0000","last_fullsized":"2023-11-28T11:08:50.106326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:56.811805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113581+0000","last_change":"2023-11-28T11:08:45.943949+0000","last_active":"2023-11-28T11:08:50.113581+0000","last_peered":"2023-11-28T11:08:50.113581+0000","last_clean":"2023-11-28T11:08:50.113581+0000","last_became_active":"2023-11-28T11:08:45.942746+0000","last_became_peered":"2023-11-28T11:08:45.942746+0000","last_unstale":"2023-11-28T11:08:50.113581+0000","last_undegraded":"2023-11-28T11:08:50.113581+0000","last_fullsized":"2023-11-28T11:08:50.113581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:49.186302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110612+0000","last_change":"2023-11-28T11:08:45.954401+0000","last_active":"2023-11-28T11:08:50.110612+0000","last_peered":"2023-11-28T11:08:50.110612+0000","last_clean":"2023-11-28T11:08:50.110612+0000","last_became_active":"2023-11-28T11:08:45.954130+0000","last_became_peered":"2023-11-28T11:08:45.954130+0000","last_unstale":"2023-11-28T11:08:50.110612+0000","last_undegraded":"2023-11-28T11:08:50.110612+0000","last_fullsized":"2023-11-28T11:08:50.110612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:55.295125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113075+0000","last_change":"2023-11-28T11:08:45.956559+0000","last_active":"2023-11-28T11:08:50.113075+0000","last_peered":"2023-11-28T11:08:50.113075+0000","last_clean":"2023-11-28T11:08:50.113075+0000","last_became_active":"2023-11-28T11:08:45.956269+0000","last_became_peered":"2023-11-28T11:08:45.956269+0000","last_unstale":"2023-11-28T11:08:50.113075+0000","last_undegraded":"2023-11-28T11:08:50.113075+0000","last_fullsized":"2023-11-28T11:08:50.113075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:17.189127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115796+0000","last_change":"2023-11-28T11:08:45.951043+0000","last_active":"2023-11-28T11:08:50.115796+0000","last_peered":"2023-11-28T11:08:50.115796+0000","last_clean":"2023-11-28T11:08:50.115796+0000","last_became_active":"2023-11-28T11:08:45.950740+0000","last_became_peered":"2023-11-28T11:08:45.950740+0000","last_unstale":"2023-11-28T11:08:50.115796+0000","last_undegraded":"2023-11-28T11:08:50.115796+0000","last_fullsized":"2023-11-28T11:08:50.115796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103425+0000","last_change":"2023-11-28T11:08:45.940737+0000","last_active":"2023-11-28T11:08:50.103425+0000","last_peered":"2023-11-28T11:08:50.103425+0000","last_clean":"2023-11-28T11:08:50.103425+0000","last_became_active":"2023-11-28T11:08:45.940490+0000","last_became_peered":"2023-11-28T11:08:45.940490+0000","last_unstale":"2023-11-28T11:08:50.103425+0000","last_undegraded":"2023-11-28T11:08:50.103425+0000","last_fullsized":"2023-11-28T11:08:50.103425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.107940+0000","last_change":"2023-11-28T11:08:45.945519+0000","last_active":"2023-11-28T11:08:50.107940+0000","last_peered":"2023-11-28T11:08:50.107940+0000","last_clean":"2023-11-28T11:08:50.107940+0000","last_became_active":"2023-11-28T11:08:45.944911+0000","last_became_peered":"2023-11-28T11:08:45.944911+0000","last_unstale":"2023-11-28T11:08:50.107940+0000","last_undegraded":"2023-11-28T11:08:50.107940+0000","last_fullsized":"2023-11-28T11:08:50.107940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:05.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115473+0000","last_change":"2023-11-28T11:08:45.954870+0000","last_active":"2023-11-28T11:08:50.115473+0000","last_peered":"2023-11-28T11:08:50.115473+0000","last_clean":"2023-11-28T11:08:50.115473+0000","last_became_active":"2023-11-28T11:08:45.954442+0000","last_became_peered":"2023-11-28T11:08:45.954442+0000","last_unstale":"2023-11-28T11:08:50.115473+0000","last_undegraded":"2023-11-28T11:08:50.115473+0000","last_fullsized":"2023-11-28T11:08:50.115473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120450+0000","last_change":"2023-11-28T11:08:45.954073+0000","last_active":"2023-11-28T11:08:50.120450+0000","last_peered":"2023-11-28T11:08:50.120450+0000","last_clean":"2023-11-28T11:08:50.120450+0000","last_became_active":"2023-11-28T11:08:45.953886+0000","last_became_peered":"2023-11-28T11:08:45.953886+0000","last_unstale":"2023-11-28T11:08:50.120450+0000","last_undegraded":"2023-11-28T11:08:50.120450+0000","last_fullsized":"2023-11-28T11:08:50.120450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:56.811805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.121555+0000","last_change":"2023-11-28T11:08:45.943121+0000","last_active":"2023-11-28T11:08:50.121555+0000","last_peered":"2023-11-28T11:08:50.121555+0000","last_clean":"2023-11-28T11:08:50.121555+0000","last_became_active":"2023-11-28T11:08:45.942796+0000","last_became_peered":"2023-11-28T11:08:45.942796+0000","last_unstale":"2023-11-28T11:08:50.121555+0000","last_undegraded":"2023-11-28T11:08:50.121555+0000","last_fullsized":"2023-11-28T11:08:50.121555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:55.085644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106066+0000","last_change":"2023-11-28T11:08:45.957422+0000","last_active":"2023-11-28T11:08:50.106066+0000","last_peered":"2023-11-28T11:08:50.106066+0000","last_clean":"2023-11-28T11:08:50.106066+0000","last_became_active":"2023-11-28T11:08:45.956775+0000","last_became_peered":"2023-11-28T11:08:45.956775+0000","last_unstale":"2023-11-28T11:08:50.106066+0000","last_undegraded":"2023-11-28T11:08:50.106066+0000","last_fullsized":"2023-11-28T11:08:50.106066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.536690+0000","last_change":"2023-11-28T11:08:50.132954+0000","last_active":"2023-11-28T11:08:50.536690+0000","last_peered":"2023-11-28T11:08:50.536690+0000","last_clean":"2023-11-28T11:08:50.536690+0000","last_became_active":"2023-11-28T11:08:50.132496+0000","last_became_peered":"2023-11-28T11:08:50.132496+0000","last_unstale":"2023-11-28T11:08:50.536690+0000","last_undegraded":"2023-11-28T11:08:50.536690+0000","last_fullsized":"2023-11-28T11:08:50.536690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:19:52.225499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112687+0000","last_change":"2023-11-28T11:08:45.515522+0000","last_active":"2023-11-28T11:08:50.112687+0000","last_peered":"2023-11-28T11:08:50.112687+0000","last_clean":"2023-11-28T11:08:50.112687+0000","last_became_active":"2023-11-28T11:08:45.515238+0000","last_became_peered":"2023-11-28T11:08:45.515238+0000","last_unstale":"2023-11-28T11:08:50.112687+0000","last_undegraded":"2023-11-28T11:08:50.112687+0000","last_fullsized":"2023-11-28T11:08:50.112687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109971+0000","last_change":"2023-11-28T11:08:45.945792+0000","last_active":"2023-11-28T11:08:50.109971+0000","last_peered":"2023-11-28T11:08:50.109971+0000","last_clean":"2023-11-28T11:08:50.109971+0000","last_became_active":"2023-11-28T11:08:45.945536+0000","last_became_peered":"2023-11-28T11:08:45.945536+0000","last_unstale":"2023-11-28T11:08:50.109971+0000","last_undegraded":"2023-11-28T11:08:50.109971+0000","last_fullsized":"2023-11-28T11:08:50.109971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:49.186302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571306+0000","last_change":"2023-11-28T11:08:50.124166+0000","last_active":"2023-11-28T11:08:51.571306+0000","last_peered":"2023-11-28T11:08:51.571306+0000","last_clean":"2023-11-28T11:08:51.571306+0000","last_became_active":"2023-11-28T11:08:50.123744+0000","last_became_peered":"2023-11-28T11:08:50.123744+0000","last_unstale":"2023-11-28T11:08:51.571306+0000","last_undegraded":"2023-11-28T11:08:51.571306+0000","last_fullsized":"2023-11-28T11:08:51.571306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:58.631960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119490+0000","last_change":"2023-11-28T11:08:45.516923+0000","last_active":"2023-11-28T11:08:50.119490+0000","last_peered":"2023-11-28T11:08:50.119490+0000","last_clean":"2023-11-28T11:08:50.119490+0000","last_became_active":"2023-11-28T11:08:45.516618+0000","last_became_peered":"2023-11-28T11:08:45.516618+0000","last_unstale":"2023-11-28T11:08:50.119490+0000","last_undegraded":"2023-11-28T11:08:50.119490+0000","last_fullsized":"2023-11-28T11:08:50.119490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103765+0000","last_change":"2023-11-28T11:08:45.941787+0000","last_active":"2023-11-28T11:08:50.103765+0000","last_peered":"2023-11-28T11:08:50.103765+0000","last_clean":"2023-11-28T11:08:50.103765+0000","last_became_active":"2023-11-28T11:08:45.941379+0000","last_became_peered":"2023-11-28T11:08:45.941379+0000","last_unstale":"2023-11-28T11:08:50.103765+0000","last_undegraded":"2023-11-28T11:08:50.103765+0000","last_fullsized":"2023-11-28T11:08:50.103765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.437643+0000","last_change":"2023-11-28T11:08:50.136708+0000","last_active":"2023-11-28T11:08:51.437643+0000","last_peered":"2023-11-28T11:08:51.437643+0000","last_clean":"2023-11-28T11:08:51.437643+0000","last_became_active":"2023-11-28T11:08:50.133846+0000","last_became_peered":"2023-11-28T11:08:50.133846+0000","last_unstale":"2023-11-28T11:08:51.437643+0000","last_undegraded":"2023-11-28T11:08:51.437643+0000","last_fullsized":"2023-11-28T11:08:51.437643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:11:43.645799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109280+0000","last_change":"2023-11-28T11:08:45.524714+0000","last_active":"2023-11-28T11:08:50.109280+0000","last_peered":"2023-11-28T11:08:50.109280+0000","last_clean":"2023-11-28T11:08:50.109280+0000","last_became_active":"2023-11-28T11:08:45.524202+0000","last_became_peered":"2023-11-28T11:08:45.524202+0000","last_unstale":"2023-11-28T11:08:50.109280+0000","last_undegraded":"2023-11-28T11:08:50.109280+0000","last_fullsized":"2023-11-28T11:08:50.109280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:49.800409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110311+0000","last_change":"2023-11-28T11:08:45.952452+0000","last_active":"2023-11-28T11:08:50.110311+0000","last_peered":"2023-11-28T11:08:50.110311+0000","last_clean":"2023-11-28T11:08:50.110311+0000","last_became_active":"2023-11-28T11:08:45.952159+0000","last_became_peered":"2023-11-28T11:08:45.952159+0000","last_unstale":"2023-11-28T11:08:50.110311+0000","last_undegraded":"2023-11-28T11:08:50.110311+0000","last_fullsized":"2023-11-28T11:08:50.110311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:58:31.173555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569992+0000","last_change":"2023-11-28T11:08:50.123500+0000","last_active":"2023-11-28T11:08:51.569992+0000","last_peered":"2023-11-28T11:08:51.569992+0000","last_clean":"2023-11-28T11:08:51.569992+0000","last_became_active":"2023-11-28T11:08:50.123102+0000","last_became_peered":"2023-11-28T11:08:50.123102+0000","last_unstale":"2023-11-28T11:08:51.569992+0000","last_undegraded":"2023-11-28T11:08:51.569992+0000","last_fullsized":"2023-11-28T11:08:51.569992+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:36.688715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106385+0000","last_change":"2023-11-28T11:08:45.523570+0000","last_active":"2023-11-28T11:08:50.106385+0000","last_peered":"2023-11-28T11:08:50.106385+0000","last_clean":"2023-11-28T11:08:50.106385+0000","last_became_active":"2023-11-28T11:08:45.523159+0000","last_became_peered":"2023-11-28T11:08:45.523159+0000","last_unstale":"2023-11-28T11:08:50.106385+0000","last_undegraded":"2023-11-28T11:08:50.106385+0000","last_fullsized":"2023-11-28T11:08:50.106385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109190+0000","last_change":"2023-11-28T11:08:45.943601+0000","last_active":"2023-11-28T11:08:50.109190+0000","last_peered":"2023-11-28T11:08:50.109190+0000","last_clean":"2023-11-28T11:08:50.109190+0000","last_became_active":"2023-11-28T11:08:45.943333+0000","last_became_peered":"2023-11-28T11:08:45.943333+0000","last_unstale":"2023-11-28T11:08:50.109190+0000","last_undegraded":"2023-11-28T11:08:50.109190+0000","last_fullsized":"2023-11-28T11:08:50.109190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569794+0000","last_change":"2023-11-28T11:08:50.126326+0000","last_active":"2023-11-28T11:08:51.569794+0000","last_peered":"2023-11-28T11:08:51.569794+0000","last_clean":"2023-11-28T11:08:51.569794+0000","last_became_active":"2023-11-28T11:08:50.125641+0000","last_became_peered":"2023-11-28T11:08:50.125641+0000","last_unstale":"2023-11-28T11:08:51.569794+0000","last_undegraded":"2023-11-28T11:08:51.569794+0000","last_fullsized":"2023-11-28T11:08:51.569794+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:59.602687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105179+0000","last_change":"2023-11-28T11:08:45.518910+0000","last_active":"2023-11-28T11:08:50.105179+0000","last_peered":"2023-11-28T11:08:50.105179+0000","last_clean":"2023-11-28T11:08:50.105179+0000","last_became_active":"2023-11-28T11:08:45.518652+0000","last_became_peered":"2023-11-28T11:08:45.518652+0000","last_unstale":"2023-11-28T11:08:50.105179+0000","last_undegraded":"2023-11-28T11:08:50.105179+0000","last_fullsized":"2023-11-28T11:08:50.105179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:49.800409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.107754+0000","last_change":"2023-11-28T11:08:45.951273+0000","last_active":"2023-11-28T11:08:50.107754+0000","last_peered":"2023-11-28T11:08:50.107754+0000","last_clean":"2023-11-28T11:08:50.107754+0000","last_became_active":"2023-11-28T11:08:45.950973+0000","last_became_peered":"2023-11-28T11:08:45.950973+0000","last_unstale":"2023-11-28T11:08:50.107754+0000","last_undegraded":"2023-11-28T11:08:50.107754+0000","last_fullsized":"2023-11-28T11:08:50.107754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:44.385589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571794+0000","last_change":"2023-11-28T11:08:50.129634+0000","last_active":"2023-11-28T11:08:51.571794+0000","last_peered":"2023-11-28T11:08:51.571794+0000","last_clean":"2023-11-28T11:08:51.571794+0000","last_became_active":"2023-11-28T11:08:50.128894+0000","last_became_peered":"2023-11-28T11:08:50.128894+0000","last_unstale":"2023-11-28T11:08:51.571794+0000","last_undegraded":"2023-11-28T11:08:51.571794+0000","last_fullsized":"2023-11-28T11:08:51.571794+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.162990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115765+0000","last_change":"2023-11-28T11:08:45.520619+0000","last_active":"2023-11-28T11:08:50.115765+0000","last_peered":"2023-11-28T11:08:50.115765+0000","last_clean":"2023-11-28T11:08:50.115765+0000","last_became_active":"2023-11-28T11:08:45.520292+0000","last_became_peered":"2023-11-28T11:08:45.520292+0000","last_unstale":"2023-11-28T11:08:50.115765+0000","last_undegraded":"2023-11-28T11:08:50.115765+0000","last_fullsized":"2023-11-28T11:08:50.115765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:02.026801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119666+0000","last_change":"2023-11-28T11:08:45.950965+0000","last_active":"2023-11-28T11:08:50.119666+0000","last_peered":"2023-11-28T11:08:50.119666+0000","last_clean":"2023-11-28T11:08:50.119666+0000","last_became_active":"2023-11-28T11:08:45.950660+0000","last_became_peered":"2023-11-28T11:08:45.950660+0000","last_unstale":"2023-11-28T11:08:50.119666+0000","last_undegraded":"2023-11-28T11:08:50.119666+0000","last_fullsized":"2023-11-28T11:08:50.119666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:34.416718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570510+0000","last_change":"2023-11-28T11:08:50.125734+0000","last_active":"2023-11-28T11:08:51.570510+0000","last_peered":"2023-11-28T11:08:51.570510+0000","last_clean":"2023-11-28T11:08:51.570510+0000","last_became_active":"2023-11-28T11:08:50.125332+0000","last_became_peered":"2023-11-28T11:08:50.125332+0000","last_unstale":"2023-11-28T11:08:51.570510+0000","last_undegraded":"2023-11-28T11:08:51.570510+0000","last_fullsized":"2023-11-28T11:08:51.570510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:54.778391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"25'8","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:54.575921+0000","last_change":"2023-11-28T11:08:45.519282+0000","last_active":"2023-11-28T11:08:54.575921+0000","last_peered":"2023-11-28T11:08:54.575921+0000","last_clean":"2023-11-28T11:08:54.575921+0000","last_became_active":"2023-11-28T11:08:45.518976+0000","last_became_peered":"2023-11-28T11:08:45.518976+0000","last_unstale":"2023-11-28T11:08:54.575921+0000","last_undegraded":"2023-11-28T11:08:54.575921+0000","last_fullsized":"2023-11-28T11:08:54.575921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:28.586127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3605,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113107+0000","last_change":"2023-11-28T11:08:45.951387+0000","last_active":"2023-11-28T11:08:50.113107+0000","last_peered":"2023-11-28T11:08:50.113107+0000","last_clean":"2023-11-28T11:08:50.113107+0000","last_became_active":"2023-11-28T11:08:45.951104+0000","last_became_peered":"2023-11-28T11:08:45.951104+0000","last_unstale":"2023-11-28T11:08:50.113107+0000","last_undegraded":"2023-11-28T11:08:50.113107+0000","last_fullsized":"2023-11-28T11:08:50.113107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:19:48.658152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537972+0000","last_change":"2023-11-28T11:08:50.124789+0000","last_active":"2023-11-28T11:08:50.537972+0000","last_peered":"2023-11-28T11:08:50.537972+0000","last_clean":"2023-11-28T11:08:50.537972+0000","last_became_active":"2023-11-28T11:08:50.123561+0000","last_became_peered":"2023-11-28T11:08:50.123561+0000","last_unstale":"2023-11-28T11:08:50.537972+0000","last_undegraded":"2023-11-28T11:08:50.537972+0000","last_fullsized":"2023-11-28T11:08:50.537972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:29.317492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115695+0000","last_change":"2023-11-28T11:08:45.518953+0000","last_active":"2023-11-28T11:08:50.115695+0000","last_peered":"2023-11-28T11:08:50.115695+0000","last_clean":"2023-11-28T11:08:50.115695+0000","last_became_active":"2023-11-28T11:08:45.518679+0000","last_became_peered":"2023-11-28T11:08:45.518679+0000","last_unstale":"2023-11-28T11:08:50.115695+0000","last_undegraded":"2023-11-28T11:08:50.115695+0000","last_fullsized":"2023-11-28T11:08:50.115695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:54.080163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108292+0000","last_change":"2023-11-28T11:08:45.945377+0000","last_active":"2023-11-28T11:08:50.108292+0000","last_peered":"2023-11-28T11:08:50.108292+0000","last_clean":"2023-11-28T11:08:50.108292+0000","last_became_active":"2023-11-28T11:08:45.945104+0000","last_became_peered":"2023-11-28T11:08:45.945104+0000","last_unstale":"2023-11-28T11:08:50.108292+0000","last_undegraded":"2023-11-28T11:08:50.108292+0000","last_fullsized":"2023-11-28T11:08:50.108292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.572169+0000","last_change":"2023-11-28T11:08:50.130966+0000","last_active":"2023-11-28T11:08:51.572169+0000","last_peered":"2023-11-28T11:08:51.572169+0000","last_clean":"2023-11-28T11:08:51.572169+0000","last_became_active":"2023-11-28T11:08:50.130573+0000","last_became_peered":"2023-11-28T11:08:50.130573+0000","last_unstale":"2023-11-28T11:08:51.572169+0000","last_undegraded":"2023-11-28T11:08:51.572169+0000","last_fullsized":"2023-11-28T11:08:51.572169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:59.602687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103540+0000","last_change":"2023-11-28T11:08:45.521275+0000","last_active":"2023-11-28T11:08:50.103540+0000","last_peered":"2023-11-28T11:08:50.103540+0000","last_clean":"2023-11-28T11:08:50.103540+0000","last_became_active":"2023-11-28T11:08:45.520908+0000","last_became_peered":"2023-11-28T11:08:45.520908+0000","last_unstale":"2023-11-28T11:08:50.103540+0000","last_undegraded":"2023-11-28T11:08:50.103540+0000","last_fullsized":"2023-11-28T11:08:50.103540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:33.188220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105967+0000","last_change":"2023-11-28T11:08:45.957604+0000","last_active":"2023-11-28T11:08:50.105967+0000","last_peered":"2023-11-28T11:08:50.105967+0000","last_clean":"2023-11-28T11:08:50.105967+0000","last_became_active":"2023-11-28T11:08:45.957135+0000","last_became_peered":"2023-11-28T11:08:45.957135+0000","last_unstale":"2023-11-28T11:08:50.105967+0000","last_undegraded":"2023-11-28T11:08:50.105967+0000","last_fullsized":"2023-11-28T11:08:50.105967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.572546+0000","last_change":"2023-11-28T11:08:50.129359+0000","last_active":"2023-11-28T11:08:51.572546+0000","last_peered":"2023-11-28T11:08:51.572546+0000","last_clean":"2023-11-28T11:08:51.572546+0000","last_became_active":"2023-11-28T11:08:50.128928+0000","last_became_peered":"2023-11-28T11:08:50.128928+0000","last_unstale":"2023-11-28T11:08:51.572546+0000","last_undegraded":"2023-11-28T11:08:51.572546+0000","last_fullsized":"2023-11-28T11:08:51.572546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:24.842089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115717+0000","last_change":"2023-11-28T11:08:45.517068+0000","last_active":"2023-11-28T11:08:50.115717+0000","last_peered":"2023-11-28T11:08:50.115717+0000","last_clean":"2023-11-28T11:08:50.115717+0000","last_became_active":"2023-11-28T11:08:45.516742+0000","last_became_peered":"2023-11-28T11:08:45.516742+0000","last_unstale":"2023-11-28T11:08:50.115717+0000","last_undegraded":"2023-11-28T11:08:50.115717+0000","last_fullsized":"2023-11-28T11:08:50.115717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:00.442293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.104172+0000","last_change":"2023-11-28T11:08:45.941624+0000","last_active":"2023-11-28T11:08:50.104172+0000","last_peered":"2023-11-28T11:08:50.104172+0000","last_clean":"2023-11-28T11:08:50.104172+0000","last_became_active":"2023-11-28T11:08:45.941042+0000","last_became_peered":"2023-11-28T11:08:45.941042+0000","last_unstale":"2023-11-28T11:08:50.104172+0000","last_undegraded":"2023-11-28T11:08:50.104172+0000","last_fullsized":"2023-11-28T11:08:50.104172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:39.158725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537699+0000","last_change":"2023-11-28T11:08:50.121180+0000","last_active":"2023-11-28T11:08:50.537699+0000","last_peered":"2023-11-28T11:08:50.537699+0000","last_clean":"2023-11-28T11:08:50.537699+0000","last_became_active":"2023-11-28T11:08:50.120757+0000","last_became_peered":"2023-11-28T11:08:50.120757+0000","last_unstale":"2023-11-28T11:08:50.537699+0000","last_undegraded":"2023-11-28T11:08:50.537699+0000","last_fullsized":"2023-11-28T11:08:50.537699+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:20.989038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108439+0000","last_change":"2023-11-28T11:08:45.516801+0000","last_active":"2023-11-28T11:08:50.108439+0000","last_peered":"2023-11-28T11:08:50.108439+0000","last_clean":"2023-11-28T11:08:50.108439+0000","last_became_active":"2023-11-28T11:08:45.516457+0000","last_became_peered":"2023-11-28T11:08:45.516457+0000","last_unstale":"2023-11-28T11:08:50.108439+0000","last_undegraded":"2023-11-28T11:08:50.108439+0000","last_fullsized":"2023-11-28T11:08:50.108439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.111359+0000","last_change":"2023-11-28T11:08:45.956905+0000","last_active":"2023-11-28T11:08:50.111359+0000","last_peered":"2023-11-28T11:08:50.111359+0000","last_clean":"2023-11-28T11:08:50.111359+0000","last_became_active":"2023-11-28T11:08:45.956698+0000","last_became_peered":"2023-11-28T11:08:45.956698+0000","last_unstale":"2023-11-28T11:08:50.111359+0000","last_undegraded":"2023-11-28T11:08:50.111359+0000","last_fullsized":"2023-11-28T11:08:50.111359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:23.627932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570657+0000","last_change":"2023-11-28T11:08:50.130733+0000","last_active":"2023-11-28T11:08:51.570657+0000","last_peered":"2023-11-28T11:08:51.570657+0000","last_clean":"2023-11-28T11:08:51.570657+0000","last_became_active":"2023-11-28T11:08:50.130213+0000","last_became_peered":"2023-11-28T11:08:50.130213+0000","last_unstale":"2023-11-28T11:08:51.570657+0000","last_undegraded":"2023-11-28T11:08:51.570657+0000","last_fullsized":"2023-11-28T11:08:51.570657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:36.656033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.116413+0000","last_change":"2023-11-28T11:08:45.516890+0000","last_active":"2023-11-28T11:08:50.116413+0000","last_peered":"2023-11-28T11:08:50.116413+0000","last_clean":"2023-11-28T11:08:50.116413+0000","last_became_active":"2023-11-28T11:08:45.516595+0000","last_became_peered":"2023-11-28T11:08:45.516595+0000","last_unstale":"2023-11-28T11:08:50.116413+0000","last_undegraded":"2023-11-28T11:08:50.116413+0000","last_fullsized":"2023-11-28T11:08:50.116413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:58:30.168075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105047+0000","last_change":"2023-11-28T11:08:45.956070+0000","last_active":"2023-11-28T11:08:50.105047+0000","last_peered":"2023-11-28T11:08:50.105047+0000","last_clean":"2023-11-28T11:08:50.105047+0000","last_became_active":"2023-11-28T11:08:45.955774+0000","last_became_peered":"2023-11-28T11:08:45.955774+0000","last_unstale":"2023-11-28T11:08:50.105047+0000","last_undegraded":"2023-11-28T11:08:50.105047+0000","last_fullsized":"2023-11-28T11:08:50.105047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571154+0000","last_change":"2023-11-28T11:08:50.132292+0000","last_active":"2023-11-28T11:08:51.571154+0000","last_peered":"2023-11-28T11:08:51.571154+0000","last_clean":"2023-11-28T11:08:51.571154+0000","last_became_active":"2023-11-28T11:08:50.131887+0000","last_became_peered":"2023-11-28T11:08:50.131887+0000","last_unstale":"2023-11-28T11:08:51.571154+0000","last_undegraded":"2023-11-28T11:08:51.571154+0000","last_fullsized":"2023-11-28T11:08:51.571154+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:08.804864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115394+0000","last_change":"2023-11-28T11:08:45.516041+0000","last_active":"2023-11-28T11:08:50.115394+0000","last_peered":"2023-11-28T11:08:50.115394+0000","last_clean":"2023-11-28T11:08:50.115394+0000","last_became_active":"2023-11-28T11:08:45.515742+0000","last_became_peered":"2023-11-28T11:08:45.515742+0000","last_unstale":"2023-11-28T11:08:50.115394+0000","last_undegraded":"2023-11-28T11:08:50.115394+0000","last_fullsized":"2023-11-28T11:08:50.115394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:14.983939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113365+0000","last_change":"2023-11-28T11:08:45.945858+0000","last_active":"2023-11-28T11:08:50.113365+0000","last_peered":"2023-11-28T11:08:50.113365+0000","last_clean":"2023-11-28T11:08:50.113365+0000","last_became_active":"2023-11-28T11:08:45.945552+0000","last_became_peered":"2023-11-28T11:08:45.945552+0000","last_unstale":"2023-11-28T11:08:50.113365+0000","last_undegraded":"2023-11-28T11:08:50.113365+0000","last_fullsized":"2023-11-28T11:08:50.113365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571022+0000","last_change":"2023-11-28T11:08:50.123713+0000","last_active":"2023-11-28T11:08:51.571022+0000","last_peered":"2023-11-28T11:08:51.571022+0000","last_clean":"2023-11-28T11:08:51.571022+0000","last_became_active":"2023-11-28T11:08:50.123264+0000","last_became_peered":"2023-11-28T11:08:50.123264+0000","last_unstale":"2023-11-28T11:08:51.571022+0000","last_undegraded":"2023-11-28T11:08:51.571022+0000","last_fullsized":"2023-11-28T11:08:51.571022+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:04.199211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115481+0000","last_change":"2023-11-28T11:08:45.515894+0000","last_active":"2023-11-28T11:08:50.115481+0000","last_peered":"2023-11-28T11:08:50.115481+0000","last_clean":"2023-11-28T11:08:50.115481+0000","last_became_active":"2023-11-28T11:08:45.515593+0000","last_became_peered":"2023-11-28T11:08:45.515593+0000","last_unstale":"2023-11-28T11:08:50.115481+0000","last_undegraded":"2023-11-28T11:08:50.115481+0000","last_fullsized":"2023-11-28T11:08:50.115481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:31.747218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109629+0000","last_change":"2023-11-28T11:08:45.949548+0000","last_active":"2023-11-28T11:08:50.109629+0000","last_peered":"2023-11-28T11:08:50.109629+0000","last_clean":"2023-11-28T11:08:50.109629+0000","last_became_active":"2023-11-28T11:08:45.949217+0000","last_became_peered":"2023-11-28T11:08:45.949217+0000","last_unstale":"2023-11-28T11:08:50.109629+0000","last_undegraded":"2023-11-28T11:08:50.109629+0000","last_fullsized":"2023-11-28T11:08:50.109629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:01.447773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571373+0000","last_change":"2023-11-28T11:08:50.127125+0000","last_active":"2023-11-28T11:08:51.571373+0000","last_peered":"2023-11-28T11:08:51.571373+0000","last_clean":"2023-11-28T11:08:51.571373+0000","last_became_active":"2023-11-28T11:08:50.124511+0000","last_became_peered":"2023-11-28T11:08:50.124511+0000","last_unstale":"2023-11-28T11:08:51.571373+0000","last_undegraded":"2023-11-28T11:08:51.571373+0000","last_fullsized":"2023-11-28T11:08:51.571373+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:47.985900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105609+0000","last_change":"2023-11-28T11:08:45.513992+0000","last_active":"2023-11-28T11:08:50.105609+0000","last_peered":"2023-11-28T11:08:50.105609+0000","last_clean":"2023-11-28T11:08:50.105609+0000","last_became_active":"2023-11-28T11:08:45.513697+0000","last_became_peered":"2023-11-28T11:08:45.513697+0000","last_unstale":"2023-11-28T11:08:50.105609+0000","last_undegraded":"2023-11-28T11:08:50.105609+0000","last_fullsized":"2023-11-28T11:08:50.105609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:28.586127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103999+0000","last_change":"2023-11-28T11:08:45.940813+0000","last_active":"2023-11-28T11:08:50.103999+0000","last_peered":"2023-11-28T11:08:50.103999+0000","last_clean":"2023-11-28T11:08:50.103999+0000","last_became_active":"2023-11-28T11:08:45.940552+0000","last_became_peered":"2023-11-28T11:08:45.940552+0000","last_unstale":"2023-11-28T11:08:50.103999+0000","last_undegraded":"2023-11-28T11:08:50.103999+0000","last_fullsized":"2023-11-28T11:08:50.103999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:17.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571495+0000","last_change":"2023-11-28T11:08:50.121770+0000","last_active":"2023-11-28T11:08:51.571495+0000","last_peered":"2023-11-28T11:08:51.571495+0000","last_clean":"2023-11-28T11:08:51.571495+0000","last_became_active":"2023-11-28T11:08:50.121277+0000","last_became_peered":"2023-11-28T11:08:50.121277+0000","last_unstale":"2023-11-28T11:08:51.571495+0000","last_undegraded":"2023-11-28T11:08:51.571495+0000","last_fullsized":"2023-11-28T11:08:51.571495+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:42.759036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109255+0000","last_change":"2023-11-28T11:08:45.519824+0000","last_active":"2023-11-28T11:08:50.109255+0000","last_peered":"2023-11-28T11:08:50.109255+0000","last_clean":"2023-11-28T11:08:50.109255+0000","last_became_active":"2023-11-28T11:08:45.519241+0000","last_became_peered":"2023-11-28T11:08:45.519241+0000","last_unstale":"2023-11-28T11:08:50.109255+0000","last_undegraded":"2023-11-28T11:08:50.109255+0000","last_fullsized":"2023-11-28T11:08:50.109255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:31.747218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112858+0000","last_change":"2023-11-28T11:08:45.946401+0000","last_active":"2023-11-28T11:08:50.112858+0000","last_peered":"2023-11-28T11:08:50.112858+0000","last_clean":"2023-11-28T11:08:50.112858+0000","last_became_active":"2023-11-28T11:08:45.946070+0000","last_became_peered":"2023-11-28T11:08:45.946070+0000","last_unstale":"2023-11-28T11:08:50.112858+0000","last_undegraded":"2023-11-28T11:08:50.112858+0000","last_fullsized":"2023-11-28T11:08:50.112858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:05.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570768+0000","last_change":"2023-11-28T11:08:50.131808+0000","last_active":"2023-11-28T11:08:51.570768+0000","last_peered":"2023-11-28T11:08:51.570768+0000","last_clean":"2023-11-28T11:08:51.570768+0000","last_became_active":"2023-11-28T11:08:50.131311+0000","last_became_peered":"2023-11-28T11:08:50.131311+0000","last_unstale":"2023-11-28T11:08:51.570768+0000","last_undegraded":"2023-11-28T11:08:51.570768+0000","last_fullsized":"2023-11-28T11:08:51.570768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:06.157479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109818+0000","last_change":"2023-11-28T11:08:45.517239+0000","last_active":"2023-11-28T11:08:50.109818+0000","last_peered":"2023-11-28T11:08:50.109818+0000","last_clean":"2023-11-28T11:08:50.109818+0000","last_became_active":"2023-11-28T11:08:45.516694+0000","last_became_peered":"2023-11-28T11:08:45.516694+0000","last_unstale":"2023-11-28T11:08:50.109818+0000","last_undegraded":"2023-11-28T11:08:50.109818+0000","last_fullsized":"2023-11-28T11:08:50.109818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113094+0000","last_change":"2023-11-28T11:08:45.947974+0000","last_active":"2023-11-28T11:08:50.113094+0000","last_peered":"2023-11-28T11:08:50.113094+0000","last_clean":"2023-11-28T11:08:50.113094+0000","last_became_active":"2023-11-28T11:08:45.947674+0000","last_became_peered":"2023-11-28T11:08:45.947674+0000","last_unstale":"2023-11-28T11:08:50.113094+0000","last_undegraded":"2023-11-28T11:08:50.113094+0000","last_fullsized":"2023-11-28T11:08:50.113094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:44.385589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.538066+0000","last_change":"2023-11-28T11:08:50.126205+0000","last_active":"2023-11-28T11:08:50.538066+0000","last_peered":"2023-11-28T11:08:50.538066+0000","last_clean":"2023-11-28T11:08:50.538066+0000","last_became_active":"2023-11-28T11:08:50.125446+0000","last_became_peered":"2023-11-28T11:08:50.125446+0000","last_unstale":"2023-11-28T11:08:50.538066+0000","last_undegraded":"2023-11-28T11:08:50.538066+0000","last_fullsized":"2023-11-28T11:08:50.538066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:59.602687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113146+0000","last_change":"2023-11-28T11:08:45.512742+0000","last_active":"2023-11-28T11:08:50.113146+0000","last_peered":"2023-11-28T11:08:50.113146+0000","last_clean":"2023-11-28T11:08:50.113146+0000","last_became_active":"2023-11-28T11:08:45.512446+0000","last_became_peered":"2023-11-28T11:08:45.512446+0000","last_unstale":"2023-11-28T11:08:50.113146+0000","last_undegraded":"2023-11-28T11:08:50.113146+0000","last_fullsized":"2023-11-28T11:08:50.113146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113682+0000","last_change":"2023-11-28T11:08:45.955541+0000","last_active":"2023-11-28T11:08:50.113682+0000","last_peered":"2023-11-28T11:08:50.113682+0000","last_clean":"2023-11-28T11:08:50.113682+0000","last_became_active":"2023-11-28T11:08:45.955223+0000","last_became_peered":"2023-11-28T11:08:45.955223+0000","last_unstale":"2023-11-28T11:08:50.113682+0000","last_undegraded":"2023-11-28T11:08:50.113682+0000","last_fullsized":"2023-11-28T11:08:50.113682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:24:04.835870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.536215+0000","last_change":"2023-11-28T11:08:50.134010+0000","last_active":"2023-11-28T11:08:50.536215+0000","last_peered":"2023-11-28T11:08:50.536215+0000","last_clean":"2023-11-28T11:08:50.536215+0000","last_became_active":"2023-11-28T11:08:50.133519+0000","last_became_peered":"2023-11-28T11:08:50.133519+0000","last_unstale":"2023-11-28T11:08:50.536215+0000","last_undegraded":"2023-11-28T11:08:50.536215+0000","last_fullsized":"2023-11-28T11:08:50.536215+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:47.676934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109102+0000","last_change":"2023-11-28T11:08:45.515039+0000","last_active":"2023-11-28T11:08:50.109102+0000","last_peered":"2023-11-28T11:08:50.109102+0000","last_clean":"2023-11-28T11:08:50.109102+0000","last_became_active":"2023-11-28T11:08:45.514380+0000","last_became_peered":"2023-11-28T11:08:45.514380+0000","last_unstale":"2023-11-28T11:08:50.109102+0000","last_undegraded":"2023-11-28T11:08:50.109102+0000","last_fullsized":"2023-11-28T11:08:50.109102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:28.586127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120542+0000","last_change":"2023-11-28T11:08:45.945761+0000","last_active":"2023-11-28T11:08:50.120542+0000","last_peered":"2023-11-28T11:08:50.120542+0000","last_clean":"2023-11-28T11:08:50.120542+0000","last_became_active":"2023-11-28T11:08:45.945501+0000","last_became_peered":"2023-11-28T11:08:45.945501+0000","last_unstale":"2023-11-28T11:08:50.120542+0000","last_undegraded":"2023-11-28T11:08:50.120542+0000","last_fullsized":"2023-11-28T11:08:50.120542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:30.639887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.535821+0000","last_change":"2023-11-28T11:08:50.129233+0000","last_active":"2023-11-28T11:08:50.535821+0000","last_peered":"2023-11-28T11:08:50.535821+0000","last_clean":"2023-11-28T11:08:50.535821+0000","last_became_active":"2023-11-28T11:08:50.128785+0000","last_became_peered":"2023-11-28T11:08:50.128785+0000","last_unstale":"2023-11-28T11:08:50.535821+0000","last_undegraded":"2023-11-28T11:08:50.535821+0000","last_fullsized":"2023-11-28T11:08:50.535821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:14.088471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.111193+0000","last_change":"2023-11-28T11:08:45.519399+0000","last_active":"2023-11-28T11:08:50.111193+0000","last_peered":"2023-11-28T11:08:50.111193+0000","last_clean":"2023-11-28T11:08:50.111193+0000","last_became_active":"2023-11-28T11:08:45.519141+0000","last_became_peered":"2023-11-28T11:08:45.519141+0000","last_unstale":"2023-11-28T11:08:50.111193+0000","last_undegraded":"2023-11-28T11:08:50.111193+0000","last_fullsized":"2023-11-28T11:08:50.111193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:24.093818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113185+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:08:50.113185+0000","last_peered":"2023-11-28T11:08:50.113185+0000","last_clean":"2023-11-28T11:08:50.113185+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:08:50.113185+0000","last_undegraded":"2023-11-28T11:08:50.113185+0000","last_fullsized":"2023-11-28T11:08:50.113185+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109007+0000","last_change":"2023-11-28T11:08:45.955853+0000","last_active":"2023-11-28T11:08:50.109007+0000","last_peered":"2023-11-28T11:08:50.109007+0000","last_clean":"2023-11-28T11:08:50.109007+0000","last_became_active":"2023-11-28T11:08:45.955563+0000","last_became_peered":"2023-11-28T11:08:45.955563+0000","last_unstale":"2023-11-28T11:08:50.109007+0000","last_undegraded":"2023-11-28T11:08:50.109007+0000","last_fullsized":"2023-11-28T11:08:50.109007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:00.598900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569792+0000","last_change":"2023-11-28T11:08:50.129450+0000","last_active":"2023-11-28T11:08:51.569792+0000","last_peered":"2023-11-28T11:08:51.569792+0000","last_clean":"2023-11-28T11:08:51.569792+0000","last_became_active":"2023-11-28T11:08:50.128981+0000","last_became_peered":"2023-11-28T11:08:50.128981+0000","last_unstale":"2023-11-28T11:08:51.569792+0000","last_undegraded":"2023-11-28T11:08:51.569792+0000","last_fullsized":"2023-11-28T11:08:51.569792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:14.466494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108203+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:08:50.108203+0000","last_peered":"2023-11-28T11:08:50.108203+0000","last_clean":"2023-11-28T11:08:50.108203+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:08:50.108203+0000","last_undegraded":"2023-11-28T11:08:50.108203+0000","last_fullsized":"2023-11-28T11:08:50.108203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.513023+0000","last_change":"2023-11-28T11:08:45.522154+0000","last_active":"2023-11-28T11:08:50.513023+0000","last_peered":"2023-11-28T11:08:50.513023+0000","last_clean":"2023-11-28T11:08:50.513023+0000","last_became_active":"2023-11-28T11:08:45.518652+0000","last_became_peered":"2023-11-28T11:08:45.518652+0000","last_unstale":"2023-11-28T11:08:50.513023+0000","last_undegraded":"2023-11-28T11:08:50.513023+0000","last_fullsized":"2023-11-28T11:08:50.513023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:49.800409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113254+0000","last_change":"2023-11-28T11:08:45.946786+0000","last_active":"2023-11-28T11:08:50.113254+0000","last_peered":"2023-11-28T11:08:50.113254+0000","last_clean":"2023-11-28T11:08:50.113254+0000","last_became_active":"2023-11-28T11:08:45.946446+0000","last_became_peered":"2023-11-28T11:08:45.946446+0000","last_unstale":"2023-11-28T11:08:50.113254+0000","last_undegraded":"2023-11-28T11:08:50.113254+0000","last_fullsized":"2023-11-28T11:08:50.113254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:39.158725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537975+0000","last_change":"2023-11-28T11:08:50.123220+0000","last_active":"2023-11-28T11:08:50.537975+0000","last_peered":"2023-11-28T11:08:50.537975+0000","last_clean":"2023-11-28T11:08:50.537975+0000","last_became_active":"2023-11-28T11:08:50.122776+0000","last_became_peered":"2023-11-28T11:08:50.122776+0000","last_unstale":"2023-11-28T11:08:50.537975+0000","last_undegraded":"2023-11-28T11:08:50.537975+0000","last_fullsized":"2023-11-28T11:08:50.537975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:35.176626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119767+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:08:50.119767+0000","last_peered":"2023-11-28T11:08:50.119767+0000","last_clean":"2023-11-28T11:08:50.119767+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:08:50.119767+0000","last_undegraded":"2023-11-28T11:08:50.119767+0000","last_fullsized":"2023-11-28T11:08:50.119767+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":167,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115593+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:08:50.115593+0000","last_peered":"2023-11-28T11:08:50.115593+0000","last_clean":"2023-11-28T11:08:50.115593+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:08:50.115593+0000","last_undegraded":"2023-11-28T11:08:50.115593+0000","last_fullsized":"2023-11-28T11:08:50.115593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103405+0000","last_change":"2023-11-28T11:08:45.521876+0000","last_active":"2023-11-28T11:08:50.103405+0000","last_peered":"2023-11-28T11:08:50.103405+0000","last_clean":"2023-11-28T11:08:50.103405+0000","last_became_active":"2023-11-28T11:08:45.521580+0000","last_became_peered":"2023-11-28T11:08:45.521580+0000","last_unstale":"2023-11-28T11:08:50.103405+0000","last_undegraded":"2023-11-28T11:08:50.103405+0000","last_fullsized":"2023-11-28T11:08:50.103405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.104071+0000","last_change":"2023-11-28T11:08:45.948040+0000","last_active":"2023-11-28T11:08:50.104071+0000","last_peered":"2023-11-28T11:08:50.104071+0000","last_clean":"2023-11-28T11:08:50.104071+0000","last_became_active":"2023-11-28T11:08:45.947704+0000","last_became_peered":"2023-11-28T11:08:45.947704+0000","last_unstale":"2023-11-28T11:08:50.104071+0000","last_undegraded":"2023-11-28T11:08:50.104071+0000","last_fullsized":"2023-11-28T11:08:50.104071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571808+0000","last_change":"2023-11-28T11:08:50.124074+0000","last_active":"2023-11-28T11:08:51.571808+0000","last_peered":"2023-11-28T11:08:51.571808+0000","last_clean":"2023-11-28T11:08:51.571808+0000","last_became_active":"2023-11-28T11:08:50.123630+0000","last_became_peered":"2023-11-28T11:08:50.123630+0000","last_unstale":"2023-11-28T11:08:51.571808+0000","last_undegraded":"2023-11-28T11:08:51.571808+0000","last_fullsized":"2023-11-28T11:08:51.571808+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:52.786613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115788+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:08:50.115788+0000","last_peered":"2023-11-28T11:08:50.115788+0000","last_clean":"2023-11-28T11:08:50.115788+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:08:50.115788+0000","last_undegraded":"2023-11-28T11:08:50.115788+0000","last_fullsized":"2023-11-28T11:08:50.115788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119464+0000","last_change":"2023-11-28T11:08:45.510984+0000","last_active":"2023-11-28T11:08:50.119464+0000","last_peered":"2023-11-28T11:08:50.119464+0000","last_clean":"2023-11-28T11:08:50.119464+0000","last_became_active":"2023-11-28T11:08:45.510734+0000","last_became_peered":"2023-11-28T11:08:45.510734+0000","last_unstale":"2023-11-28T11:08:50.119464+0000","last_undegraded":"2023-11-28T11:08:50.119464+0000","last_fullsized":"2023-11-28T11:08:50.119464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:33.188220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120521+0000","last_change":"2023-11-28T11:08:45.946753+0000","last_active":"2023-11-28T11:08:50.120521+0000","last_peered":"2023-11-28T11:08:50.120521+0000","last_clean":"2023-11-28T11:08:50.120521+0000","last_became_active":"2023-11-28T11:08:45.946460+0000","last_became_peered":"2023-11-28T11:08:45.946460+0000","last_unstale":"2023-11-28T11:08:50.120521+0000","last_undegraded":"2023-11-28T11:08:50.120521+0000","last_fullsized":"2023-11-28T11:08:50.120521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:20:10.730266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570567+0000","last_change":"2023-11-28T11:08:50.132450+0000","last_active":"2023-11-28T11:08:51.570567+0000","last_peered":"2023-11-28T11:08:51.570567+0000","last_clean":"2023-11-28T11:08:51.570567+0000","last_became_active":"2023-11-28T11:08:50.132043+0000","last_became_peered":"2023-11-28T11:08:50.132043+0000","last_unstale":"2023-11-28T11:08:51.570567+0000","last_undegraded":"2023-11-28T11:08:51.570567+0000","last_fullsized":"2023-11-28T11:08:51.570567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:20.989038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115361+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:08:50.115361+0000","last_peered":"2023-11-28T11:08:50.115361+0000","last_clean":"2023-11-28T11:08:50.115361+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:08:50.115361+0000","last_undegraded":"2023-11-28T11:08:50.115361+0000","last_fullsized":"2023-11-28T11:08:50.115361+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108211+0000","last_change":"2023-11-28T11:08:45.513181+0000","last_active":"2023-11-28T11:08:50.108211+0000","last_peered":"2023-11-28T11:08:50.108211+0000","last_clean":"2023-11-28T11:08:50.108211+0000","last_became_active":"2023-11-28T11:08:45.512903+0000","last_became_peered":"2023-11-28T11:08:45.512903+0000","last_unstale":"2023-11-28T11:08:50.108211+0000","last_undegraded":"2023-11-28T11:08:50.108211+0000","last_fullsized":"2023-11-28T11:08:50.108211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109406+0000","last_change":"2023-11-28T11:08:45.947505+0000","last_active":"2023-11-28T11:08:50.109406+0000","last_peered":"2023-11-28T11:08:50.109406+0000","last_clean":"2023-11-28T11:08:50.109406+0000","last_became_active":"2023-11-28T11:08:45.947199+0000","last_became_peered":"2023-11-28T11:08:45.947199+0000","last_unstale":"2023-11-28T11:08:50.109406+0000","last_undegraded":"2023-11-28T11:08:50.109406+0000","last_fullsized":"2023-11-28T11:08:50.109406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:55.085644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.437473+0000","last_change":"2023-11-28T11:08:50.126736+0000","last_active":"2023-11-28T11:08:51.437473+0000","last_peered":"2023-11-28T11:08:51.437473+0000","last_clean":"2023-11-28T11:08:51.437473+0000","last_became_active":"2023-11-28T11:08:50.126282+0000","last_became_peered":"2023-11-28T11:08:50.126282+0000","last_unstale":"2023-11-28T11:08:51.437473+0000","last_undegraded":"2023-11-28T11:08:51.437473+0000","last_fullsized":"2023-11-28T11:08:51.437473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:36.656033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.104943+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:08:50.104943+0000","last_peered":"2023-11-28T11:08:50.104943+0000","last_clean":"2023-11-28T11:08:50.104943+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:08:50.104943+0000","last_undegraded":"2023-11-28T11:08:50.104943+0000","last_fullsized":"2023-11-28T11:08:50.104943+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112806+0000","last_change":"2023-11-28T11:08:45.524531+0000","last_active":"2023-11-28T11:08:50.112806+0000","last_peered":"2023-11-28T11:08:50.112806+0000","last_clean":"2023-11-28T11:08:50.112806+0000","last_became_active":"2023-11-28T11:08:45.524334+0000","last_became_peered":"2023-11-28T11:08:45.524334+0000","last_unstale":"2023-11-28T11:08:50.112806+0000","last_undegraded":"2023-11-28T11:08:50.112806+0000","last_fullsized":"2023-11-28T11:08:50.112806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119778+0000","last_change":"2023-11-28T11:08:45.944621+0000","last_active":"2023-11-28T11:08:50.119778+0000","last_peered":"2023-11-28T11:08:50.119778+0000","last_clean":"2023-11-28T11:08:50.119778+0000","last_became_active":"2023-11-28T11:08:45.944368+0000","last_became_peered":"2023-11-28T11:08:45.944368+0000","last_unstale":"2023-11-28T11:08:50.119778+0000","last_undegraded":"2023-11-28T11:08:50.119778+0000","last_fullsized":"2023-11-28T11:08:50.119778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:01.447773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570969+0000","last_change":"2023-11-28T11:08:50.131902+0000","last_active":"2023-11-28T11:08:51.570969+0000","last_peered":"2023-11-28T11:08:51.570969+0000","last_clean":"2023-11-28T11:08:51.570969+0000","last_became_active":"2023-11-28T11:08:50.131485+0000","last_became_peered":"2023-11-28T11:08:50.131485+0000","last_unstale":"2023-11-28T11:08:51.570969+0000","last_undegraded":"2023-11-28T11:08:51.570969+0000","last_fullsized":"2023-11-28T11:08:51.570969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:08.804864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110005+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:08:50.110005+0000","last_peered":"2023-11-28T11:08:50.110005+0000","last_clean":"2023-11-28T11:08:50.110005+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:08:50.110005+0000","last_undegraded":"2023-11-28T11:08:50.110005+0000","last_fullsized":"2023-11-28T11:08:50.110005+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120247+0000","last_change":"2023-11-28T11:08:45.514598+0000","last_active":"2023-11-28T11:08:50.120247+0000","last_peered":"2023-11-28T11:08:50.120247+0000","last_clean":"2023-11-28T11:08:50.120247+0000","last_became_active":"2023-11-28T11:08:45.514348+0000","last_became_peered":"2023-11-28T11:08:45.514348+0000","last_unstale":"2023-11-28T11:08:50.120247+0000","last_undegraded":"2023-11-28T11:08:50.120247+0000","last_fullsized":"2023-11-28T11:08:50.120247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103691+0000","last_change":"2023-11-28T11:08:45.950031+0000","last_active":"2023-11-28T11:08:50.103691+0000","last_peered":"2023-11-28T11:08:50.103691+0000","last_clean":"2023-11-28T11:08:50.103691+0000","last_became_active":"2023-11-28T11:08:45.949457+0000","last_became_peered":"2023-11-28T11:08:45.949457+0000","last_unstale":"2023-11-28T11:08:50.103691+0000","last_undegraded":"2023-11-28T11:08:50.103691+0000","last_fullsized":"2023-11-28T11:08:50.103691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:34.416718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.437351+0000","last_change":"2023-11-28T11:08:50.124074+0000","last_active":"2023-11-28T11:08:51.437351+0000","last_peered":"2023-11-28T11:08:51.437351+0000","last_clean":"2023-11-28T11:08:51.437351+0000","last_became_active":"2023-11-28T11:08:50.123710+0000","last_became_peered":"2023-11-28T11:08:50.123710+0000","last_unstale":"2023-11-28T11:08:51.437351+0000","last_undegraded":"2023-11-28T11:08:51.437351+0000","last_fullsized":"2023-11-28T11:08:51.437351+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:42:35.306388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110239+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:08:50.110239+0000","last_peered":"2023-11-28T11:08:50.110239+0000","last_clean":"2023-11-28T11:08:50.110239+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:08:50.110239+0000","last_undegraded":"2023-11-28T11:08:50.110239+0000","last_fullsized":"2023-11-28T11:08:50.110239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105883+0000","last_change":"2023-11-28T11:08:45.514165+0000","last_active":"2023-11-28T11:08:50.105883+0000","last_peered":"2023-11-28T11:08:50.105883+0000","last_clean":"2023-11-28T11:08:50.105883+0000","last_became_active":"2023-11-28T11:08:45.513651+0000","last_became_peered":"2023-11-28T11:08:45.513651+0000","last_unstale":"2023-11-28T11:08:50.105883+0000","last_undegraded":"2023-11-28T11:08:50.105883+0000","last_fullsized":"2023-11-28T11:08:50.105883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119668+0000","last_change":"2023-11-28T11:08:45.949610+0000","last_active":"2023-11-28T11:08:50.119668+0000","last_peered":"2023-11-28T11:08:50.119668+0000","last_clean":"2023-11-28T11:08:50.119668+0000","last_became_active":"2023-11-28T11:08:45.949348+0000","last_became_peered":"2023-11-28T11:08:45.949348+0000","last_unstale":"2023-11-28T11:08:50.119668+0000","last_undegraded":"2023-11-28T11:08:50.119668+0000","last_fullsized":"2023-11-28T11:08:50.119668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:23:53.819338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570113+0000","last_change":"2023-11-28T11:08:50.135779+0000","last_active":"2023-11-28T11:08:51.570113+0000","last_peered":"2023-11-28T11:08:51.570113+0000","last_clean":"2023-11-28T11:08:51.570113+0000","last_became_active":"2023-11-28T11:08:50.131379+0000","last_became_peered":"2023-11-28T11:08:50.131379+0000","last_unstale":"2023-11-28T11:08:51.570113+0000","last_undegraded":"2023-11-28T11:08:51.570113+0000","last_fullsized":"2023-11-28T11:08:51.570113+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:42:35.306388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103804+0000","last_change":"2023-11-28T11:08:45.523090+0000","last_active":"2023-11-28T11:08:50.103804+0000","last_peered":"2023-11-28T11:08:50.103804+0000","last_clean":"2023-11-28T11:08:50.103804+0000","last_became_active":"2023-11-28T11:08:45.522445+0000","last_became_peered":"2023-11-28T11:08:45.522445+0000","last_unstale":"2023-11-28T11:08:50.103804+0000","last_undegraded":"2023-11-28T11:08:50.103804+0000","last_fullsized":"2023-11-28T11:08:50.103804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108643+0000","last_change":"2023-11-28T11:08:45.943849+0000","last_active":"2023-11-28T11:08:50.108643+0000","last_peered":"2023-11-28T11:08:50.108643+0000","last_clean":"2023-11-28T11:08:50.108643+0000","last_became_active":"2023-11-28T11:08:45.943426+0000","last_became_peered":"2023-11-28T11:08:45.943426+0000","last_unstale":"2023-11-28T11:08:50.108643+0000","last_undegraded":"2023-11-28T11:08:50.108643+0000","last_fullsized":"2023-11-28T11:08:50.108643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:29.591607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570960+0000","last_change":"2023-11-28T11:08:50.129403+0000","last_active":"2023-11-28T11:08:51.570960+0000","last_peered":"2023-11-28T11:08:51.570960+0000","last_clean":"2023-11-28T11:08:51.570960+0000","last_became_active":"2023-11-28T11:08:50.128984+0000","last_became_peered":"2023-11-28T11:08:50.128984+0000","last_unstale":"2023-11-28T11:08:51.570960+0000","last_undegraded":"2023-11-28T11:08:51.570960+0000","last_fullsized":"2023-11-28T11:08:51.570960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:54.778391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105418+0000","last_change":"2023-11-28T11:08:45.514673+0000","last_active":"2023-11-28T11:08:50.105418+0000","last_peered":"2023-11-28T11:08:50.105418+0000","last_clean":"2023-11-28T11:08:50.105418+0000","last_became_active":"2023-11-28T11:08:45.513328+0000","last_became_peered":"2023-11-28T11:08:45.513328+0000","last_unstale":"2023-11-28T11:08:50.105418+0000","last_undegraded":"2023-11-28T11:08:50.105418+0000","last_fullsized":"2023-11-28T11:08:50.105418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:55.864512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112696+0000","last_change":"2023-11-28T11:08:45.948904+0000","last_active":"2023-11-28T11:08:50.112696+0000","last_peered":"2023-11-28T11:08:50.112696+0000","last_clean":"2023-11-28T11:08:50.112696+0000","last_became_active":"2023-11-28T11:08:45.948629+0000","last_became_peered":"2023-11-28T11:08:45.948629+0000","last_unstale":"2023-11-28T11:08:50.112696+0000","last_undegraded":"2023-11-28T11:08:50.112696+0000","last_fullsized":"2023-11-28T11:08:50.112696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:55.031649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537746+0000","last_change":"2023-11-28T11:08:50.124117+0000","last_active":"2023-11-28T11:08:50.537746+0000","last_peered":"2023-11-28T11:08:50.537746+0000","last_clean":"2023-11-28T11:08:50.537746+0000","last_became_active":"2023-11-28T11:08:50.123257+0000","last_became_peered":"2023-11-28T11:08:50.123257+0000","last_unstale":"2023-11-28T11:08:50.537746+0000","last_undegraded":"2023-11-28T11:08:50.537746+0000","last_fullsized":"2023-11-28T11:08:50.537746+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:24.842089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105770+0000","last_change":"2023-11-28T11:08:45.511721+0000","last_active":"2023-11-28T11:08:50.105770+0000","last_peered":"2023-11-28T11:08:50.105770+0000","last_clean":"2023-11-28T11:08:50.105770+0000","last_became_active":"2023-11-28T11:08:45.511413+0000","last_became_peered":"2023-11-28T11:08:45.511413+0000","last_unstale":"2023-11-28T11:08:50.105770+0000","last_undegraded":"2023-11-28T11:08:50.105770+0000","last_fullsized":"2023-11-28T11:08:50.105770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:24.093818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.111702+0000","last_change":"2023-11-28T11:08:45.945561+0000","last_active":"2023-11-28T11:08:50.111702+0000","last_peered":"2023-11-28T11:08:50.111702+0000","last_clean":"2023-11-28T11:08:50.111702+0000","last_became_active":"2023-11-28T11:08:45.945101+0000","last_became_peered":"2023-11-28T11:08:45.945101+0000","last_unstale":"2023-11-28T11:08:50.111702+0000","last_undegraded":"2023-11-28T11:08:50.111702+0000","last_fullsized":"2023-11-28T11:08:50.111702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:00.598900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.535418+0000","last_change":"2023-11-28T11:08:50.132859+0000","last_active":"2023-11-28T11:08:50.535418+0000","last_peered":"2023-11-28T11:08:50.535418+0000","last_clean":"2023-11-28T11:08:50.535418+0000","last_became_active":"2023-11-28T11:08:50.132397+0000","last_became_peered":"2023-11-28T11:08:50.132397+0000","last_unstale":"2023-11-28T11:08:50.535418+0000","last_undegraded":"2023-11-28T11:08:50.535418+0000","last_fullsized":"2023-11-28T11:08:50.535418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:07.333534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112935+0000","last_change":"2023-11-28T11:08:45.511864+0000","last_active":"2023-11-28T11:08:50.112935+0000","last_peered":"2023-11-28T11:08:50.112935+0000","last_clean":"2023-11-28T11:08:50.112935+0000","last_became_active":"2023-11-28T11:08:45.511500+0000","last_became_peered":"2023-11-28T11:08:45.511500+0000","last_unstale":"2023-11-28T11:08:50.112935+0000","last_undegraded":"2023-11-28T11:08:50.112935+0000","last_fullsized":"2023-11-28T11:08:50.112935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:55.864512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108100+0000","last_change":"2023-11-28T11:08:45.945158+0000","last_active":"2023-11-28T11:08:50.108100+0000","last_peered":"2023-11-28T11:08:50.108100+0000","last_clean":"2023-11-28T11:08:50.108100+0000","last_became_active":"2023-11-28T11:08:45.944341+0000","last_became_peered":"2023-11-28T11:08:45.944341+0000","last_unstale":"2023-11-28T11:08:50.108100+0000","last_undegraded":"2023-11-28T11:08:50.108100+0000","last_fullsized":"2023-11-28T11:08:50.108100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569813+0000","last_change":"2023-11-28T11:08:50.136091+0000","last_active":"2023-11-28T11:08:51.569813+0000","last_peered":"2023-11-28T11:08:51.569813+0000","last_clean":"2023-11-28T11:08:51.569813+0000","last_became_active":"2023-11-28T11:08:50.135640+0000","last_became_peered":"2023-11-28T11:08:50.135640+0000","last_unstale":"2023-11-28T11:08:51.569813+0000","last_undegraded":"2023-11-28T11:08:51.569813+0000","last_fullsized":"2023-11-28T11:08:51.569813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:19:52.258463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106438+0000","last_change":"2023-11-28T11:08:45.522945+0000","last_active":"2023-11-28T11:08:50.106438+0000","last_peered":"2023-11-28T11:08:50.106438+0000","last_clean":"2023-11-28T11:08:50.106438+0000","last_became_active":"2023-11-28T11:08:45.522100+0000","last_became_peered":"2023-11-28T11:08:45.522100+0000","last_unstale":"2023-11-28T11:08:50.106438+0000","last_undegraded":"2023-11-28T11:08:50.106438+0000","last_fullsized":"2023-11-28T11:08:50.106438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:24.093818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.116376+0000","last_change":"2023-11-28T11:08:45.955608+0000","last_active":"2023-11-28T11:08:50.116376+0000","last_peered":"2023-11-28T11:08:50.116376+0000","last_clean":"2023-11-28T11:08:50.116376+0000","last_became_active":"2023-11-28T11:08:45.955293+0000","last_became_peered":"2023-11-28T11:08:45.955293+0000","last_unstale":"2023-11-28T11:08:50.116376+0000","last_undegraded":"2023-11-28T11:08:50.116376+0000","last_fullsized":"2023-11-28T11:08:50.116376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:33.088404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108532+0000","last_change":"2023-11-28T11:08:45.953710+0000","last_active":"2023-11-28T11:08:50.108532+0000","last_peered":"2023-11-28T11:08:50.108532+0000","last_clean":"2023-11-28T11:08:50.108532+0000","last_became_active":"2023-11-28T11:08:45.953349+0000","last_became_peered":"2023-11-28T11:08:45.953349+0000","last_unstale":"2023-11-28T11:08:50.108532+0000","last_undegraded":"2023-11-28T11:08:50.108532+0000","last_fullsized":"2023-11-28T11:08:50.108532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:03.032281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.107845+0000","last_change":"2023-11-28T11:08:45.945358+0000","last_active":"2023-11-28T11:08:50.107845+0000","last_peered":"2023-11-28T11:08:50.107845+0000","last_clean":"2023-11-28T11:08:50.107845+0000","last_became_active":"2023-11-28T11:08:45.944619+0000","last_became_peered":"2023-11-28T11:08:45.944619+0000","last_unstale":"2023-11-28T11:08:50.107845+0000","last_undegraded":"2023-11-28T11:08:50.107845+0000","last_fullsized":"2023-11-28T11:08:50.107845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:17.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112795+0000","last_change":"2023-11-28T11:08:45.951315+0000","last_active":"2023-11-28T11:08:50.112795+0000","last_peered":"2023-11-28T11:08:50.112795+0000","last_clean":"2023-11-28T11:08:50.112795+0000","last_became_active":"2023-11-28T11:08:45.950992+0000","last_became_peered":"2023-11-28T11:08:45.950992+0000","last_unstale":"2023-11-28T11:08:50.112795+0000","last_undegraded":"2023-11-28T11:08:50.112795+0000","last_fullsized":"2023-11-28T11:08:50.112795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:51.178080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109781+0000","last_change":"2023-11-28T11:08:45.954451+0000","last_active":"2023-11-28T11:08:50.109781+0000","last_peered":"2023-11-28T11:08:50.109781+0000","last_clean":"2023-11-28T11:08:50.109781+0000","last_became_active":"2023-11-28T11:08:45.954162+0000","last_became_peered":"2023-11-28T11:08:45.954162+0000","last_unstale":"2023-11-28T11:08:50.109781+0000","last_undegraded":"2023-11-28T11:08:50.109781+0000","last_fullsized":"2023-11-28T11:08:50.109781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:39.158725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119893+0000","last_change":"2023-11-28T11:08:45.944860+0000","last_active":"2023-11-28T11:08:50.119893+0000","last_peered":"2023-11-28T11:08:50.119893+0000","last_clean":"2023-11-28T11:08:50.119893+0000","last_became_active":"2023-11-28T11:08:45.944591+0000","last_became_peered":"2023-11-28T11:08:45.944591+0000","last_unstale":"2023-11-28T11:08:50.119893+0000","last_undegraded":"2023-11-28T11:08:50.119893+0000","last_fullsized":"2023-11-28T11:08:50.119893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4387,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":703590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9232,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362608,"statfs":{"total":96636764160,"available":96627310592,"internally_reserved":0,"allocated":860160,"data_stored":699769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.249}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":108440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8224,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363616,"statfs":{"total":96636764160,"available":96628342784,"internally_reserved":0,"allocated":286720,"data_stored":113062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8084,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363756,"statfs":{"total":96636764160,"available":96628486144,"internally_reserved":0,"allocated":274432,"data_stored":110037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:09:04.367 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:09:04.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.525+0000 7f8a1928f700 1 Processor -- start 2023-11-28T11:09:04.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.525+0000 7f8a1928f700 1 -- start start 2023-11-28T11:09:04.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.526+0000 7f8a1928f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a1415a6c0 0x7f8a140cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.526+0000 7f8a1928f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8a140d0020 0x7f8a140ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.526+0000 7f8a1928f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a140cfb90 con 0x7f8a140b2660 2023-11-28T11:09:04.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.526+0000 7f8a1928f700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a140cb930 con 0x7f8a140d0020 2023-11-28T11:09:04.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.526+0000 7f8a1928f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a140cf260 con 0x7f8a1415a6c0 2023-11-28T11:09:04.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.526+0000 7f8a127fc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8a140d0020 0x7f8a140ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.526+0000 7f8a127fc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8a140d0020 0x7f8a140ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46900/0 (socket says 172.21.15.28:46900) 2023-11-28T11:09:04.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.526+0000 7f8a127fc700 1 -- 172.21.15.28:0/3059167082 learned_addr learned my addr 172.21.15.28:0/3059167082 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.526+0000 7f8a12ffd700 1 --2- 172.21.15.28:0/3059167082 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a1415a6c0 0x7f8a140cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.526+0000 7f8a127fc700 1 -- 172.21.15.28:0/3059167082 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a1415a6c0 msgr2=0x7f8a140cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.526+0000 7f8a127fc700 1 --2- 172.21.15.28:0/3059167082 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a1415a6c0 0x7f8a140cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.526+0000 7f8a127fc700 1 -- 172.21.15.28:0/3059167082 >> v1:172.21.15.59:6789/0 conn(0x7f8a140b2660 legacy=0x7f8a140b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.526+0000 7f8a127fc700 1 -- 172.21.15.28:0/3059167082 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a140cf540 con 0x7f8a140d0020 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.526+0000 7f8a127fc700 1 --2- 172.21.15.28:0/3059167082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a140d0020 0x7f8a140ccea0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0098f0 tx=0x7f89fc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee31dd8a5ecbb000 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.527+0000 7f8a11ffb700 1 -- 172.21.15.28:0/3059167082 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc004230 con 0x7f8a140d0020 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.527+0000 7f8a11ffb700 1 -- 172.21.15.28:0/3059167082 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc004390 con 0x7f8a140d0020 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.527+0000 7f8a11ffb700 1 -- 172.21.15.28:0/3059167082 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc0036a0 con 0x7f8a140d0020 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.527+0000 7f8a1928f700 1 -- 172.21.15.28:0/3059167082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a140d0020 msgr2=0x7f8a140ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.527+0000 7f8a1928f700 1 --2- 172.21.15.28:0/3059167082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a140d0020 0x7f8a140ccea0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f89fc0098f0 tx=0x7f89fc009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:04.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.527+0000 7f8a1928f700 1 -- 172.21.15.28:0/3059167082 shutdown_connections 2023-11-28T11:09:04.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.527+0000 7f8a1928f700 1 --2- 172.21.15.28:0/3059167082 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a1415a6c0 0x7f8a140cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.527+0000 7f8a1928f700 1 --2- 172.21.15.28:0/3059167082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a140d0020 0x7f8a140ccea0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.527+0000 7f8a1928f700 1 -- 172.21.15.28:0/3059167082 >> 172.21.15.28:0/3059167082 conn(0x7f8a140cd660 msgr2=0x7f8a140cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:04.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.527+0000 7f8a1928f700 1 -- 172.21.15.28:0/3059167082 shutdown_connections 2023-11-28T11:09:04.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.527+0000 7f8a1928f700 1 -- 172.21.15.28:0/3059167082 wait complete. 2023-11-28T11:09:04.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.529+0000 7f8a1928f700 1 Processor -- start 2023-11-28T11:09:04.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.529+0000 7f8a1928f700 1 -- start start 2023-11-28T11:09:04.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a1928f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a140b2660 0x7f8a140c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a1928f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a140d0020 0x7f8a140b51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a1928f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a1415a6c0 0x7f8a140b5710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a1928f700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a141e6270 con 0x7f8a140d0020 2023-11-28T11:09:04.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a1928f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a14148db0 con 0x7f8a140b2660 2023-11-28T11:09:04.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a1928f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a140c7a70 con 0x7f8a1415a6c0 2023-11-28T11:09:04.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a127fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a140d0020 0x7f8a140b51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a127fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a140d0020 0x7f8a140b51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46908/0 (socket says 172.21.15.28:46908) 2023-11-28T11:09:04.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a127fc700 1 -- 172.21.15.28:0/4281765583 learned_addr learned my addr 172.21.15.28:0/4281765583 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:04.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a12ffd700 1 --2- 172.21.15.28:0/4281765583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a140b2660 0x7f8a140c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a137fe700 1 --2- 172.21.15.28:0/4281765583 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a1415a6c0 0x7f8a140b5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a127fc700 1 -- 172.21.15.28:0/4281765583 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a1415a6c0 msgr2=0x7f8a140b5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a127fc700 1 --2- 172.21.15.28:0/4281765583 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a1415a6c0 0x7f8a140b5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a127fc700 1 -- 172.21.15.28:0/4281765583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a140b2660 msgr2=0x7f8a140c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a127fc700 1 --2- 172.21.15.28:0/4281765583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a140b2660 0x7f8a140c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a127fc700 1 -- 172.21.15.28:0/4281765583 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89fc009550 con 0x7f8a140d0020 2023-11-28T11:09:04.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.530+0000 7f8a127fc700 1 --2- 172.21.15.28:0/4281765583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a140d0020 0x7f8a140b51d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009130 tx=0x7f89fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:04.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.531+0000 7f8a0b7fe700 1 -- 172.21.15.28:0/4281765583 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc003cd0 con 0x7f8a140d0020 2023-11-28T11:09:04.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.531+0000 7f8a0b7fe700 1 -- 172.21.15.28:0/4281765583 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89fc01f070 con 0x7f8a140d0020 2023-11-28T11:09:04.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.531+0000 7f8a1928f700 1 -- 172.21.15.28:0/4281765583 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a140cd3e0 con 0x7f8a140d0020 2023-11-28T11:09:04.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.531+0000 7f8a0b7fe700 1 -- 172.21.15.28:0/4281765583 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89fc00b8a0 con 0x7f8a140d0020 2023-11-28T11:09:04.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.531+0000 7f8a1928f700 1 -- 172.21.15.28:0/4281765583 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a140cf820 con 0x7f8a140d0020 2023-11-28T11:09:04.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.532+0000 7f8a1928f700 1 -- 172.21.15.28:0/4281765583 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a140d0470 con 0x7f8a140d0020 2023-11-28T11:09:04.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.532+0000 7f8a0b7fe700 1 -- 172.21.15.28:0/4281765583 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f89fc02a020 con 0x7f8a140d0020 2023-11-28T11:09:04.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.532+0000 7f8a0b7fe700 1 --2- 172.21.15.28:0/4281765583 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f89f40682f0 0x7f89f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.532+0000 7f8a0b7fe700 1 -- 172.21.15.28:0/4281765583 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f89fc087680 con 0x7f8a140d0020 2023-11-28T11:09:04.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.532+0000 7f8a12ffd700 1 --2- 172.21.15.28:0/4281765583 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f89f40682f0 0x7f89f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.533+0000 7f8a12ffd700 1 --2- 172.21.15.28:0/4281765583 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f89f40682f0 0x7f89f406a7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a04006fd0 tx=0x7f8a04008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:04.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.537+0000 7f8a0b7fe700 1 -- 172.21.15.28:0/4281765583 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f89fc053bf0 con 0x7f8a140d0020 2023-11-28T11:09:04.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.729+0000 7f8a1928f700 1 -- 172.21.15.28:0/4281765583 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8a140d0470 con 0x7f89f40682f0 2023-11-28T11:09:04.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.739+0000 7f8a0b7fe700 1 -- 172.21.15.28:0/4281765583 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367554 (secure 0 0 0) 0x7f8a140d0470 con 0x7f89f40682f0 2023-11-28T11:09:04.744 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:04.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.743+0000 7f8a1928f700 1 -- 172.21.15.28:0/4281765583 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f89f40682f0 msgr2=0x7f89f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.743+0000 7f8a1928f700 1 --2- 172.21.15.28:0/4281765583 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f89f40682f0 0x7f89f406a7a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8a04006fd0 tx=0x7f8a04008040 comp rx=0 tx=0).stop 2023-11-28T11:09:04.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.743+0000 7f8a1928f700 1 -- 172.21.15.28:0/4281765583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a140d0020 msgr2=0x7f8a140b51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.743+0000 7f8a1928f700 1 --2- 172.21.15.28:0/4281765583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a140d0020 0x7f8a140b51d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f89fc009130 tx=0x7f89fc004750 comp rx=0 tx=0).stop 2023-11-28T11:09:04.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.743+0000 7f8a1928f700 1 -- 172.21.15.28:0/4281765583 shutdown_connections 2023-11-28T11:09:04.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.743+0000 7f8a1928f700 1 --2- 172.21.15.28:0/4281765583 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f89f40682f0 0x7f89f406a7a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.743+0000 7f8a1928f700 1 --2- 172.21.15.28:0/4281765583 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8a1415a6c0 0x7f8a140b5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.743+0000 7f8a1928f700 1 --2- 172.21.15.28:0/4281765583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8a140b2660 0x7f8a140c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.743+0000 7f8a1928f700 1 --2- 172.21.15.28:0/4281765583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8a140d0020 0x7f8a140b51d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.743+0000 7f8a1928f700 1 -- 172.21.15.28:0/4281765583 >> 172.21.15.28:0/4281765583 conn(0x7f8a140cd660 msgr2=0x7f8a140b1050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:04.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.743+0000 7f8a1928f700 1 -- 172.21.15.28:0/4281765583 shutdown_connections 2023-11-28T11:09:04.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.743+0000 7f8a1928f700 1 -- 172.21.15.28:0/4281765583 wait complete. 2023-11-28T11:09:04.749 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:09:04.756 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-11-28T11:09:03.165857+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594755,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":206,"num_write_kb":2433,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":158,"ondisk_log_size":158,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67236,"kb_used_data":3300,"kb_used_omap":0,"kb_used_meta":63872,"kb_avail":754907484,"statfs":{"total":773094113280,"available":773025263616,"internally_reserved":0,"allocated":3379200,"data_stored":2063757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65404928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2057,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002638"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108863+0000","last_change":"2023-11-28T11:08:45.948753+0000","last_active":"2023-11-28T11:08:50.108863+0000","last_peered":"2023-11-28T11:08:50.108863+0000","last_clean":"2023-11-28T11:08:50.108863+0000","last_became_active":"2023-11-28T11:08:45.948435+0000","last_became_peered":"2023-11-28T11:08:45.948435+0000","last_unstale":"2023-11-28T11:08:50.108863+0000","last_undegraded":"2023-11-28T11:08:50.108863+0000","last_fullsized":"2023-11-28T11:08:50.108863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:28.944484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105146+0000","last_change":"2023-11-28T11:08:45.946452+0000","last_active":"2023-11-28T11:08:50.105146+0000","last_peered":"2023-11-28T11:08:50.105146+0000","last_clean":"2023-11-28T11:08:50.105146+0000","last_became_active":"2023-11-28T11:08:45.946145+0000","last_became_peered":"2023-11-28T11:08:45.946145+0000","last_unstale":"2023-11-28T11:08:50.105146+0000","last_undegraded":"2023-11-28T11:08:50.105146+0000","last_fullsized":"2023-11-28T11:08:50.105146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:30.639887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113411+0000","last_change":"2023-11-28T11:08:45.941140+0000","last_active":"2023-11-28T11:08:50.113411+0000","last_peered":"2023-11-28T11:08:50.113411+0000","last_clean":"2023-11-28T11:08:50.113411+0000","last_became_active":"2023-11-28T11:08:45.939618+0000","last_became_peered":"2023-11-28T11:08:45.939618+0000","last_unstale":"2023-11-28T11:08:50.113411+0000","last_undegraded":"2023-11-28T11:08:50.113411+0000","last_fullsized":"2023-11-28T11:08:50.113411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:44.444264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109455+0000","last_change":"2023-11-28T11:08:45.947665+0000","last_active":"2023-11-28T11:08:50.109455+0000","last_peered":"2023-11-28T11:08:50.109455+0000","last_clean":"2023-11-28T11:08:50.109455+0000","last_became_active":"2023-11-28T11:08:45.947338+0000","last_became_peered":"2023-11-28T11:08:45.947338+0000","last_unstale":"2023-11-28T11:08:50.109455+0000","last_undegraded":"2023-11-28T11:08:50.109455+0000","last_fullsized":"2023-11-28T11:08:50.109455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:23:53.819338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103496+0000","last_change":"2023-11-28T11:08:45.950223+0000","last_active":"2023-11-28T11:08:50.103496+0000","last_peered":"2023-11-28T11:08:50.103496+0000","last_clean":"2023-11-28T11:08:50.103496+0000","last_became_active":"2023-11-28T11:08:45.949811+0000","last_became_peered":"2023-11-28T11:08:45.949811+0000","last_unstale":"2023-11-28T11:08:50.103496+0000","last_undegraded":"2023-11-28T11:08:50.103496+0000","last_fullsized":"2023-11-28T11:08:50.103496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112890+0000","last_change":"2023-11-28T11:08:45.942787+0000","last_active":"2023-11-28T11:08:50.112890+0000","last_peered":"2023-11-28T11:08:50.112890+0000","last_clean":"2023-11-28T11:08:50.112890+0000","last_became_active":"2023-11-28T11:08:45.941820+0000","last_became_peered":"2023-11-28T11:08:45.941820+0000","last_unstale":"2023-11-28T11:08:50.112890+0000","last_undegraded":"2023-11-28T11:08:50.112890+0000","last_fullsized":"2023-11-28T11:08:50.112890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:22.504112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115835+0000","last_change":"2023-11-28T11:08:45.953364+0000","last_active":"2023-11-28T11:08:50.115835+0000","last_peered":"2023-11-28T11:08:50.115835+0000","last_clean":"2023-11-28T11:08:50.115835+0000","last_became_active":"2023-11-28T11:08:45.953033+0000","last_became_peered":"2023-11-28T11:08:45.953033+0000","last_unstale":"2023-11-28T11:08:50.115835+0000","last_undegraded":"2023-11-28T11:08:50.115835+0000","last_fullsized":"2023-11-28T11:08:50.115835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105175+0000","last_change":"2023-11-28T11:08:45.947045+0000","last_active":"2023-11-28T11:08:50.105175+0000","last_peered":"2023-11-28T11:08:50.105175+0000","last_clean":"2023-11-28T11:08:50.105175+0000","last_became_active":"2023-11-28T11:08:45.946756+0000","last_became_peered":"2023-11-28T11:08:45.946756+0000","last_unstale":"2023-11-28T11:08:50.105175+0000","last_undegraded":"2023-11-28T11:08:50.105175+0000","last_fullsized":"2023-11-28T11:08:50.105175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:56.811805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103692+0000","last_change":"2023-11-28T11:08:45.946321+0000","last_active":"2023-11-28T11:08:50.103692+0000","last_peered":"2023-11-28T11:08:50.103692+0000","last_clean":"2023-11-28T11:08:50.103692+0000","last_became_active":"2023-11-28T11:08:45.943687+0000","last_became_peered":"2023-11-28T11:08:45.943687+0000","last_unstale":"2023-11-28T11:08:50.103692+0000","last_undegraded":"2023-11-28T11:08:50.103692+0000","last_fullsized":"2023-11-28T11:08:50.103692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:23.627932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105317+0000","last_change":"2023-11-28T11:08:45.948529+0000","last_active":"2023-11-28T11:08:50.105317+0000","last_peered":"2023-11-28T11:08:50.105317+0000","last_clean":"2023-11-28T11:08:50.105317+0000","last_became_active":"2023-11-28T11:08:45.948249+0000","last_became_peered":"2023-11-28T11:08:45.948249+0000","last_unstale":"2023-11-28T11:08:50.105317+0000","last_undegraded":"2023-11-28T11:08:50.105317+0000","last_fullsized":"2023-11-28T11:08:50.105317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113311+0000","last_change":"2023-11-28T11:08:45.942374+0000","last_active":"2023-11-28T11:08:50.113311+0000","last_peered":"2023-11-28T11:08:50.113311+0000","last_clean":"2023-11-28T11:08:50.113311+0000","last_became_active":"2023-11-28T11:08:45.942083+0000","last_became_peered":"2023-11-28T11:08:45.942083+0000","last_unstale":"2023-11-28T11:08:50.113311+0000","last_undegraded":"2023-11-28T11:08:50.113311+0000","last_fullsized":"2023-11-28T11:08:50.113311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:23.627932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108474+0000","last_change":"2023-11-28T11:08:45.947339+0000","last_active":"2023-11-28T11:08:50.108474+0000","last_peered":"2023-11-28T11:08:50.108474+0000","last_clean":"2023-11-28T11:08:50.108474+0000","last_became_active":"2023-11-28T11:08:45.947066+0000","last_became_peered":"2023-11-28T11:08:45.947066+0000","last_unstale":"2023-11-28T11:08:50.108474+0000","last_undegraded":"2023-11-28T11:08:50.108474+0000","last_fullsized":"2023-11-28T11:08:50.108474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:48.010812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109210+0000","last_change":"2023-11-28T11:08:45.949094+0000","last_active":"2023-11-28T11:08:50.109210+0000","last_peered":"2023-11-28T11:08:50.109210+0000","last_clean":"2023-11-28T11:08:50.109210+0000","last_became_active":"2023-11-28T11:08:45.948523+0000","last_became_peered":"2023-11-28T11:08:45.948523+0000","last_unstale":"2023-11-28T11:08:50.109210+0000","last_undegraded":"2023-11-28T11:08:50.109210+0000","last_fullsized":"2023-11-28T11:08:50.109210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:34.193700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105493+0000","last_change":"2023-11-28T11:08:45.945708+0000","last_active":"2023-11-28T11:08:50.105493+0000","last_peered":"2023-11-28T11:08:50.105493+0000","last_clean":"2023-11-28T11:08:50.105493+0000","last_became_active":"2023-11-28T11:08:45.945408+0000","last_became_peered":"2023-11-28T11:08:45.945408+0000","last_unstale":"2023-11-28T11:08:50.105493+0000","last_undegraded":"2023-11-28T11:08:50.105493+0000","last_fullsized":"2023-11-28T11:08:50.105493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:34.416718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115608+0000","last_change":"2023-11-28T11:08:45.954131+0000","last_active":"2023-11-28T11:08:50.115608+0000","last_peered":"2023-11-28T11:08:50.115608+0000","last_clean":"2023-11-28T11:08:50.115608+0000","last_became_active":"2023-11-28T11:08:45.953824+0000","last_became_peered":"2023-11-28T11:08:45.953824+0000","last_unstale":"2023-11-28T11:08:50.115608+0000","last_undegraded":"2023-11-28T11:08:50.115608+0000","last_fullsized":"2023-11-28T11:08:50.115608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113028+0000","last_change":"2023-11-28T11:08:45.943568+0000","last_active":"2023-11-28T11:08:50.113028+0000","last_peered":"2023-11-28T11:08:50.113028+0000","last_clean":"2023-11-28T11:08:50.113028+0000","last_became_active":"2023-11-28T11:08:45.942637+0000","last_became_peered":"2023-11-28T11:08:45.942637+0000","last_unstale":"2023-11-28T11:08:50.113028+0000","last_undegraded":"2023-11-28T11:08:50.113028+0000","last_fullsized":"2023-11-28T11:08:50.113028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:17.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106326+0000","last_change":"2023-11-28T11:08:45.944177+0000","last_active":"2023-11-28T11:08:50.106326+0000","last_peered":"2023-11-28T11:08:50.106326+0000","last_clean":"2023-11-28T11:08:50.106326+0000","last_became_active":"2023-11-28T11:08:45.943369+0000","last_became_peered":"2023-11-28T11:08:45.943369+0000","last_unstale":"2023-11-28T11:08:50.106326+0000","last_undegraded":"2023-11-28T11:08:50.106326+0000","last_fullsized":"2023-11-28T11:08:50.106326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:56.811805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113581+0000","last_change":"2023-11-28T11:08:45.943949+0000","last_active":"2023-11-28T11:08:50.113581+0000","last_peered":"2023-11-28T11:08:50.113581+0000","last_clean":"2023-11-28T11:08:50.113581+0000","last_became_active":"2023-11-28T11:08:45.942746+0000","last_became_peered":"2023-11-28T11:08:45.942746+0000","last_unstale":"2023-11-28T11:08:50.113581+0000","last_undegraded":"2023-11-28T11:08:50.113581+0000","last_fullsized":"2023-11-28T11:08:50.113581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:49.186302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110612+0000","last_change":"2023-11-28T11:08:45.954401+0000","last_active":"2023-11-28T11:08:50.110612+0000","last_peered":"2023-11-28T11:08:50.110612+0000","last_clean":"2023-11-28T11:08:50.110612+0000","last_became_active":"2023-11-28T11:08:45.954130+0000","last_became_peered":"2023-11-28T11:08:45.954130+0000","last_unstale":"2023-11-28T11:08:50.110612+0000","last_undegraded":"2023-11-28T11:08:50.110612+0000","last_fullsized":"2023-11-28T11:08:50.110612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:55.295125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113075+0000","last_change":"2023-11-28T11:08:45.956559+0000","last_active":"2023-11-28T11:08:50.113075+0000","last_peered":"2023-11-28T11:08:50.113075+0000","last_clean":"2023-11-28T11:08:50.113075+0000","last_became_active":"2023-11-28T11:08:45.956269+0000","last_became_peered":"2023-11-28T11:08:45.956269+0000","last_unstale":"2023-11-28T11:08:50.113075+0000","last_undegraded":"2023-11-28T11:08:50.113075+0000","last_fullsized":"2023-11-28T11:08:50.113075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:17.189127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115796+0000","last_change":"2023-11-28T11:08:45.951043+0000","last_active":"2023-11-28T11:08:50.115796+0000","last_peered":"2023-11-28T11:08:50.115796+0000","last_clean":"2023-11-28T11:08:50.115796+0000","last_became_active":"2023-11-28T11:08:45.950740+0000","last_became_peered":"2023-11-28T11:08:45.950740+0000","last_unstale":"2023-11-28T11:08:50.115796+0000","last_undegraded":"2023-11-28T11:08:50.115796+0000","last_fullsized":"2023-11-28T11:08:50.115796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103425+0000","last_change":"2023-11-28T11:08:45.940737+0000","last_active":"2023-11-28T11:08:50.103425+0000","last_peered":"2023-11-28T11:08:50.103425+0000","last_clean":"2023-11-28T11:08:50.103425+0000","last_became_active":"2023-11-28T11:08:45.940490+0000","last_became_peered":"2023-11-28T11:08:45.940490+0000","last_unstale":"2023-11-28T11:08:50.103425+0000","last_undegraded":"2023-11-28T11:08:50.103425+0000","last_fullsized":"2023-11-28T11:08:50.103425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.107940+0000","last_change":"2023-11-28T11:08:45.945519+0000","last_active":"2023-11-28T11:08:50.107940+0000","last_peered":"2023-11-28T11:08:50.107940+0000","last_clean":"2023-11-28T11:08:50.107940+0000","last_became_active":"2023-11-28T11:08:45.944911+0000","last_became_peered":"2023-11-28T11:08:45.944911+0000","last_unstale":"2023-11-28T11:08:50.107940+0000","last_undegraded":"2023-11-28T11:08:50.107940+0000","last_fullsized":"2023-11-28T11:08:50.107940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:05.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115473+0000","last_change":"2023-11-28T11:08:45.954870+0000","last_active":"2023-11-28T11:08:50.115473+0000","last_peered":"2023-11-28T11:08:50.115473+0000","last_clean":"2023-11-28T11:08:50.115473+0000","last_became_active":"2023-11-28T11:08:45.954442+0000","last_became_peered":"2023-11-28T11:08:45.954442+0000","last_unstale":"2023-11-28T11:08:50.115473+0000","last_undegraded":"2023-11-28T11:08:50.115473+0000","last_fullsized":"2023-11-28T11:08:50.115473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120450+0000","last_change":"2023-11-28T11:08:45.954073+0000","last_active":"2023-11-28T11:08:50.120450+0000","last_peered":"2023-11-28T11:08:50.120450+0000","last_clean":"2023-11-28T11:08:50.120450+0000","last_became_active":"2023-11-28T11:08:45.953886+0000","last_became_peered":"2023-11-28T11:08:45.953886+0000","last_unstale":"2023-11-28T11:08:50.120450+0000","last_undegraded":"2023-11-28T11:08:50.120450+0000","last_fullsized":"2023-11-28T11:08:50.120450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:56.811805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.121555+0000","last_change":"2023-11-28T11:08:45.943121+0000","last_active":"2023-11-28T11:08:50.121555+0000","last_peered":"2023-11-28T11:08:50.121555+0000","last_clean":"2023-11-28T11:08:50.121555+0000","last_became_active":"2023-11-28T11:08:45.942796+0000","last_became_peered":"2023-11-28T11:08:45.942796+0000","last_unstale":"2023-11-28T11:08:50.121555+0000","last_undegraded":"2023-11-28T11:08:50.121555+0000","last_fullsized":"2023-11-28T11:08:50.121555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:55.085644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106066+0000","last_change":"2023-11-28T11:08:45.957422+0000","last_active":"2023-11-28T11:08:50.106066+0000","last_peered":"2023-11-28T11:08:50.106066+0000","last_clean":"2023-11-28T11:08:50.106066+0000","last_became_active":"2023-11-28T11:08:45.956775+0000","last_became_peered":"2023-11-28T11:08:45.956775+0000","last_unstale":"2023-11-28T11:08:50.106066+0000","last_undegraded":"2023-11-28T11:08:50.106066+0000","last_fullsized":"2023-11-28T11:08:50.106066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.536690+0000","last_change":"2023-11-28T11:08:50.132954+0000","last_active":"2023-11-28T11:08:50.536690+0000","last_peered":"2023-11-28T11:08:50.536690+0000","last_clean":"2023-11-28T11:08:50.536690+0000","last_became_active":"2023-11-28T11:08:50.132496+0000","last_became_peered":"2023-11-28T11:08:50.132496+0000","last_unstale":"2023-11-28T11:08:50.536690+0000","last_undegraded":"2023-11-28T11:08:50.536690+0000","last_fullsized":"2023-11-28T11:08:50.536690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:19:52.225499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112687+0000","last_change":"2023-11-28T11:08:45.515522+0000","last_active":"2023-11-28T11:08:50.112687+0000","last_peered":"2023-11-28T11:08:50.112687+0000","last_clean":"2023-11-28T11:08:50.112687+0000","last_became_active":"2023-11-28T11:08:45.515238+0000","last_became_peered":"2023-11-28T11:08:45.515238+0000","last_unstale":"2023-11-28T11:08:50.112687+0000","last_undegraded":"2023-11-28T11:08:50.112687+0000","last_fullsized":"2023-11-28T11:08:50.112687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109971+0000","last_change":"2023-11-28T11:08:45.945792+0000","last_active":"2023-11-28T11:08:50.109971+0000","last_peered":"2023-11-28T11:08:50.109971+0000","last_clean":"2023-11-28T11:08:50.109971+0000","last_became_active":"2023-11-28T11:08:45.945536+0000","last_became_peered":"2023-11-28T11:08:45.945536+0000","last_unstale":"2023-11-28T11:08:50.109971+0000","last_undegraded":"2023-11-28T11:08:50.109971+0000","last_fullsized":"2023-11-28T11:08:50.109971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:49.186302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571306+0000","last_change":"2023-11-28T11:08:50.124166+0000","last_active":"2023-11-28T11:08:51.571306+0000","last_peered":"2023-11-28T11:08:51.571306+0000","last_clean":"2023-11-28T11:08:51.571306+0000","last_became_active":"2023-11-28T11:08:50.123744+0000","last_became_peered":"2023-11-28T11:08:50.123744+0000","last_unstale":"2023-11-28T11:08:51.571306+0000","last_undegraded":"2023-11-28T11:08:51.571306+0000","last_fullsized":"2023-11-28T11:08:51.571306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:58.631960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119490+0000","last_change":"2023-11-28T11:08:45.516923+0000","last_active":"2023-11-28T11:08:50.119490+0000","last_peered":"2023-11-28T11:08:50.119490+0000","last_clean":"2023-11-28T11:08:50.119490+0000","last_became_active":"2023-11-28T11:08:45.516618+0000","last_became_peered":"2023-11-28T11:08:45.516618+0000","last_unstale":"2023-11-28T11:08:50.119490+0000","last_undegraded":"2023-11-28T11:08:50.119490+0000","last_fullsized":"2023-11-28T11:08:50.119490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103765+0000","last_change":"2023-11-28T11:08:45.941787+0000","last_active":"2023-11-28T11:08:50.103765+0000","last_peered":"2023-11-28T11:08:50.103765+0000","last_clean":"2023-11-28T11:08:50.103765+0000","last_became_active":"2023-11-28T11:08:45.941379+0000","last_became_peered":"2023-11-28T11:08:45.941379+0000","last_unstale":"2023-11-28T11:08:50.103765+0000","last_undegraded":"2023-11-28T11:08:50.103765+0000","last_fullsized":"2023-11-28T11:08:50.103765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.437643+0000","last_change":"2023-11-28T11:08:50.136708+0000","last_active":"2023-11-28T11:08:51.437643+0000","last_peered":"2023-11-28T11:08:51.437643+0000","last_clean":"2023-11-28T11:08:51.437643+0000","last_became_active":"2023-11-28T11:08:50.133846+0000","last_became_peered":"2023-11-28T11:08:50.133846+0000","last_unstale":"2023-11-28T11:08:51.437643+0000","last_undegraded":"2023-11-28T11:08:51.437643+0000","last_fullsized":"2023-11-28T11:08:51.437643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:11:43.645799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109280+0000","last_change":"2023-11-28T11:08:45.524714+0000","last_active":"2023-11-28T11:08:50.109280+0000","last_peered":"2023-11-28T11:08:50.109280+0000","last_clean":"2023-11-28T11:08:50.109280+0000","last_became_active":"2023-11-28T11:08:45.524202+0000","last_became_peered":"2023-11-28T11:08:45.524202+0000","last_unstale":"2023-11-28T11:08:50.109280+0000","last_undegraded":"2023-11-28T11:08:50.109280+0000","last_fullsized":"2023-11-28T11:08:50.109280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:49.800409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110311+0000","last_change":"2023-11-28T11:08:45.952452+0000","last_active":"2023-11-28T11:08:50.110311+0000","last_peered":"2023-11-28T11:08:50.110311+0000","last_clean":"2023-11-28T11:08:50.110311+0000","last_became_active":"2023-11-28T11:08:45.952159+0000","last_became_peered":"2023-11-28T11:08:45.952159+0000","last_unstale":"2023-11-28T11:08:50.110311+0000","last_undegraded":"2023-11-28T11:08:50.110311+0000","last_fullsized":"2023-11-28T11:08:50.110311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:58:31.173555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569992+0000","last_change":"2023-11-28T11:08:50.123500+0000","last_active":"2023-11-28T11:08:51.569992+0000","last_peered":"2023-11-28T11:08:51.569992+0000","last_clean":"2023-11-28T11:08:51.569992+0000","last_became_active":"2023-11-28T11:08:50.123102+0000","last_became_peered":"2023-11-28T11:08:50.123102+0000","last_unstale":"2023-11-28T11:08:51.569992+0000","last_undegraded":"2023-11-28T11:08:51.569992+0000","last_fullsized":"2023-11-28T11:08:51.569992+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:36.688715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106385+0000","last_change":"2023-11-28T11:08:45.523570+0000","last_active":"2023-11-28T11:08:50.106385+0000","last_peered":"2023-11-28T11:08:50.106385+0000","last_clean":"2023-11-28T11:08:50.106385+0000","last_became_active":"2023-11-28T11:08:45.523159+0000","last_became_peered":"2023-11-28T11:08:45.523159+0000","last_unstale":"2023-11-28T11:08:50.106385+0000","last_undegraded":"2023-11-28T11:08:50.106385+0000","last_fullsized":"2023-11-28T11:08:50.106385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109190+0000","last_change":"2023-11-28T11:08:45.943601+0000","last_active":"2023-11-28T11:08:50.109190+0000","last_peered":"2023-11-28T11:08:50.109190+0000","last_clean":"2023-11-28T11:08:50.109190+0000","last_became_active":"2023-11-28T11:08:45.943333+0000","last_became_peered":"2023-11-28T11:08:45.943333+0000","last_unstale":"2023-11-28T11:08:50.109190+0000","last_undegraded":"2023-11-28T11:08:50.109190+0000","last_fullsized":"2023-11-28T11:08:50.109190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569794+0000","last_change":"2023-11-28T11:08:50.126326+0000","last_active":"2023-11-28T11:08:51.569794+0000","last_peered":"2023-11-28T11:08:51.569794+0000","last_clean":"2023-11-28T11:08:51.569794+0000","last_became_active":"2023-11-28T11:08:50.125641+0000","last_became_peered":"2023-11-28T11:08:50.125641+0000","last_unstale":"2023-11-28T11:08:51.569794+0000","last_undegraded":"2023-11-28T11:08:51.569794+0000","last_fullsized":"2023-11-28T11:08:51.569794+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:59.602687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105179+0000","last_change":"2023-11-28T11:08:45.518910+0000","last_active":"2023-11-28T11:08:50.105179+0000","last_peered":"2023-11-28T11:08:50.105179+0000","last_clean":"2023-11-28T11:08:50.105179+0000","last_became_active":"2023-11-28T11:08:45.518652+0000","last_became_peered":"2023-11-28T11:08:45.518652+0000","last_unstale":"2023-11-28T11:08:50.105179+0000","last_undegraded":"2023-11-28T11:08:50.105179+0000","last_fullsized":"2023-11-28T11:08:50.105179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:49.800409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.107754+0000","last_change":"2023-11-28T11:08:45.951273+0000","last_active":"2023-11-28T11:08:50.107754+0000","last_peered":"2023-11-28T11:08:50.107754+0000","last_clean":"2023-11-28T11:08:50.107754+0000","last_became_active":"2023-11-28T11:08:45.950973+0000","last_became_peered":"2023-11-28T11:08:45.950973+0000","last_unstale":"2023-11-28T11:08:50.107754+0000","last_undegraded":"2023-11-28T11:08:50.107754+0000","last_fullsized":"2023-11-28T11:08:50.107754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:44.385589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571794+0000","last_change":"2023-11-28T11:08:50.129634+0000","last_active":"2023-11-28T11:08:51.571794+0000","last_peered":"2023-11-28T11:08:51.571794+0000","last_clean":"2023-11-28T11:08:51.571794+0000","last_became_active":"2023-11-28T11:08:50.128894+0000","last_became_peered":"2023-11-28T11:08:50.128894+0000","last_unstale":"2023-11-28T11:08:51.571794+0000","last_undegraded":"2023-11-28T11:08:51.571794+0000","last_fullsized":"2023-11-28T11:08:51.571794+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.162990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115765+0000","last_change":"2023-11-28T11:08:45.520619+0000","last_active":"2023-11-28T11:08:50.115765+0000","last_peered":"2023-11-28T11:08:50.115765+0000","last_clean":"2023-11-28T11:08:50.115765+0000","last_became_active":"2023-11-28T11:08:45.520292+0000","last_became_peered":"2023-11-28T11:08:45.520292+0000","last_unstale":"2023-11-28T11:08:50.115765+0000","last_undegraded":"2023-11-28T11:08:50.115765+0000","last_fullsized":"2023-11-28T11:08:50.115765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:02.026801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119666+0000","last_change":"2023-11-28T11:08:45.950965+0000","last_active":"2023-11-28T11:08:50.119666+0000","last_peered":"2023-11-28T11:08:50.119666+0000","last_clean":"2023-11-28T11:08:50.119666+0000","last_became_active":"2023-11-28T11:08:45.950660+0000","last_became_peered":"2023-11-28T11:08:45.950660+0000","last_unstale":"2023-11-28T11:08:50.119666+0000","last_undegraded":"2023-11-28T11:08:50.119666+0000","last_fullsized":"2023-11-28T11:08:50.119666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:34.416718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570510+0000","last_change":"2023-11-28T11:08:50.125734+0000","last_active":"2023-11-28T11:08:51.570510+0000","last_peered":"2023-11-28T11:08:51.570510+0000","last_clean":"2023-11-28T11:08:51.570510+0000","last_became_active":"2023-11-28T11:08:50.125332+0000","last_became_peered":"2023-11-28T11:08:50.125332+0000","last_unstale":"2023-11-28T11:08:51.570510+0000","last_undegraded":"2023-11-28T11:08:51.570510+0000","last_fullsized":"2023-11-28T11:08:51.570510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:54.778391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"25'8","reported_seq":22,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:54.575921+0000","last_change":"2023-11-28T11:08:45.519282+0000","last_active":"2023-11-28T11:08:54.575921+0000","last_peered":"2023-11-28T11:08:54.575921+0000","last_clean":"2023-11-28T11:08:54.575921+0000","last_became_active":"2023-11-28T11:08:45.518976+0000","last_became_peered":"2023-11-28T11:08:45.518976+0000","last_unstale":"2023-11-28T11:08:54.575921+0000","last_undegraded":"2023-11-28T11:08:54.575921+0000","last_fullsized":"2023-11-28T11:08:54.575921+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:28.586127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3605,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113107+0000","last_change":"2023-11-28T11:08:45.951387+0000","last_active":"2023-11-28T11:08:50.113107+0000","last_peered":"2023-11-28T11:08:50.113107+0000","last_clean":"2023-11-28T11:08:50.113107+0000","last_became_active":"2023-11-28T11:08:45.951104+0000","last_became_peered":"2023-11-28T11:08:45.951104+0000","last_unstale":"2023-11-28T11:08:50.113107+0000","last_undegraded":"2023-11-28T11:08:50.113107+0000","last_fullsized":"2023-11-28T11:08:50.113107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:19:48.658152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537972+0000","last_change":"2023-11-28T11:08:50.124789+0000","last_active":"2023-11-28T11:08:50.537972+0000","last_peered":"2023-11-28T11:08:50.537972+0000","last_clean":"2023-11-28T11:08:50.537972+0000","last_became_active":"2023-11-28T11:08:50.123561+0000","last_became_peered":"2023-11-28T11:08:50.123561+0000","last_unstale":"2023-11-28T11:08:50.537972+0000","last_undegraded":"2023-11-28T11:08:50.537972+0000","last_fullsized":"2023-11-28T11:08:50.537972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:29.317492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115695+0000","last_change":"2023-11-28T11:08:45.518953+0000","last_active":"2023-11-28T11:08:50.115695+0000","last_peered":"2023-11-28T11:08:50.115695+0000","last_clean":"2023-11-28T11:08:50.115695+0000","last_became_active":"2023-11-28T11:08:45.518679+0000","last_became_peered":"2023-11-28T11:08:45.518679+0000","last_unstale":"2023-11-28T11:08:50.115695+0000","last_undegraded":"2023-11-28T11:08:50.115695+0000","last_fullsized":"2023-11-28T11:08:50.115695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:54.080163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108292+0000","last_change":"2023-11-28T11:08:45.945377+0000","last_active":"2023-11-28T11:08:50.108292+0000","last_peered":"2023-11-28T11:08:50.108292+0000","last_clean":"2023-11-28T11:08:50.108292+0000","last_became_active":"2023-11-28T11:08:45.945104+0000","last_became_peered":"2023-11-28T11:08:45.945104+0000","last_unstale":"2023-11-28T11:08:50.108292+0000","last_undegraded":"2023-11-28T11:08:50.108292+0000","last_fullsized":"2023-11-28T11:08:50.108292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.572169+0000","last_change":"2023-11-28T11:08:50.130966+0000","last_active":"2023-11-28T11:08:51.572169+0000","last_peered":"2023-11-28T11:08:51.572169+0000","last_clean":"2023-11-28T11:08:51.572169+0000","last_became_active":"2023-11-28T11:08:50.130573+0000","last_became_peered":"2023-11-28T11:08:50.130573+0000","last_unstale":"2023-11-28T11:08:51.572169+0000","last_undegraded":"2023-11-28T11:08:51.572169+0000","last_fullsized":"2023-11-28T11:08:51.572169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:59.602687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103540+0000","last_change":"2023-11-28T11:08:45.521275+0000","last_active":"2023-11-28T11:08:50.103540+0000","last_peered":"2023-11-28T11:08:50.103540+0000","last_clean":"2023-11-28T11:08:50.103540+0000","last_became_active":"2023-11-28T11:08:45.520908+0000","last_became_peered":"2023-11-28T11:08:45.520908+0000","last_unstale":"2023-11-28T11:08:50.103540+0000","last_undegraded":"2023-11-28T11:08:50.103540+0000","last_fullsized":"2023-11-28T11:08:50.103540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:33.188220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105967+0000","last_change":"2023-11-28T11:08:45.957604+0000","last_active":"2023-11-28T11:08:50.105967+0000","last_peered":"2023-11-28T11:08:50.105967+0000","last_clean":"2023-11-28T11:08:50.105967+0000","last_became_active":"2023-11-28T11:08:45.957135+0000","last_became_peered":"2023-11-28T11:08:45.957135+0000","last_unstale":"2023-11-28T11:08:50.105967+0000","last_undegraded":"2023-11-28T11:08:50.105967+0000","last_fullsized":"2023-11-28T11:08:50.105967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.572546+0000","last_change":"2023-11-28T11:08:50.129359+0000","last_active":"2023-11-28T11:08:51.572546+0000","last_peered":"2023-11-28T11:08:51.572546+0000","last_clean":"2023-11-28T11:08:51.572546+0000","last_became_active":"2023-11-28T11:08:50.128928+0000","last_became_peered":"2023-11-28T11:08:50.128928+0000","last_unstale":"2023-11-28T11:08:51.572546+0000","last_undegraded":"2023-11-28T11:08:51.572546+0000","last_fullsized":"2023-11-28T11:08:51.572546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:24.842089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115717+0000","last_change":"2023-11-28T11:08:45.517068+0000","last_active":"2023-11-28T11:08:50.115717+0000","last_peered":"2023-11-28T11:08:50.115717+0000","last_clean":"2023-11-28T11:08:50.115717+0000","last_became_active":"2023-11-28T11:08:45.516742+0000","last_became_peered":"2023-11-28T11:08:45.516742+0000","last_unstale":"2023-11-28T11:08:50.115717+0000","last_undegraded":"2023-11-28T11:08:50.115717+0000","last_fullsized":"2023-11-28T11:08:50.115717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:00.442293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.104172+0000","last_change":"2023-11-28T11:08:45.941624+0000","last_active":"2023-11-28T11:08:50.104172+0000","last_peered":"2023-11-28T11:08:50.104172+0000","last_clean":"2023-11-28T11:08:50.104172+0000","last_became_active":"2023-11-28T11:08:45.941042+0000","last_became_peered":"2023-11-28T11:08:45.941042+0000","last_unstale":"2023-11-28T11:08:50.104172+0000","last_undegraded":"2023-11-28T11:08:50.104172+0000","last_fullsized":"2023-11-28T11:08:50.104172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:39.158725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537699+0000","last_change":"2023-11-28T11:08:50.121180+0000","last_active":"2023-11-28T11:08:50.537699+0000","last_peered":"2023-11-28T11:08:50.537699+0000","last_clean":"2023-11-28T11:08:50.537699+0000","last_became_active":"2023-11-28T11:08:50.120757+0000","last_became_peered":"2023-11-28T11:08:50.120757+0000","last_unstale":"2023-11-28T11:08:50.537699+0000","last_undegraded":"2023-11-28T11:08:50.537699+0000","last_fullsized":"2023-11-28T11:08:50.537699+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:20.989038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108439+0000","last_change":"2023-11-28T11:08:45.516801+0000","last_active":"2023-11-28T11:08:50.108439+0000","last_peered":"2023-11-28T11:08:50.108439+0000","last_clean":"2023-11-28T11:08:50.108439+0000","last_became_active":"2023-11-28T11:08:45.516457+0000","last_became_peered":"2023-11-28T11:08:45.516457+0000","last_unstale":"2023-11-28T11:08:50.108439+0000","last_undegraded":"2023-11-28T11:08:50.108439+0000","last_fullsized":"2023-11-28T11:08:50.108439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.111359+0000","last_change":"2023-11-28T11:08:45.956905+0000","last_active":"2023-11-28T11:08:50.111359+0000","last_peered":"2023-11-28T11:08:50.111359+0000","last_clean":"2023-11-28T11:08:50.111359+0000","last_became_active":"2023-11-28T11:08:45.956698+0000","last_became_peered":"2023-11-28T11:08:45.956698+0000","last_unstale":"2023-11-28T11:08:50.111359+0000","last_undegraded":"2023-11-28T11:08:50.111359+0000","last_fullsized":"2023-11-28T11:08:50.111359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:23.627932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570657+0000","last_change":"2023-11-28T11:08:50.130733+0000","last_active":"2023-11-28T11:08:51.570657+0000","last_peered":"2023-11-28T11:08:51.570657+0000","last_clean":"2023-11-28T11:08:51.570657+0000","last_became_active":"2023-11-28T11:08:50.130213+0000","last_became_peered":"2023-11-28T11:08:50.130213+0000","last_unstale":"2023-11-28T11:08:51.570657+0000","last_undegraded":"2023-11-28T11:08:51.570657+0000","last_fullsized":"2023-11-28T11:08:51.570657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:36.656033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.116413+0000","last_change":"2023-11-28T11:08:45.516890+0000","last_active":"2023-11-28T11:08:50.116413+0000","last_peered":"2023-11-28T11:08:50.116413+0000","last_clean":"2023-11-28T11:08:50.116413+0000","last_became_active":"2023-11-28T11:08:45.516595+0000","last_became_peered":"2023-11-28T11:08:45.516595+0000","last_unstale":"2023-11-28T11:08:50.116413+0000","last_undegraded":"2023-11-28T11:08:50.116413+0000","last_fullsized":"2023-11-28T11:08:50.116413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:58:30.168075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105047+0000","last_change":"2023-11-28T11:08:45.956070+0000","last_active":"2023-11-28T11:08:50.105047+0000","last_peered":"2023-11-28T11:08:50.105047+0000","last_clean":"2023-11-28T11:08:50.105047+0000","last_became_active":"2023-11-28T11:08:45.955774+0000","last_became_peered":"2023-11-28T11:08:45.955774+0000","last_unstale":"2023-11-28T11:08:50.105047+0000","last_undegraded":"2023-11-28T11:08:50.105047+0000","last_fullsized":"2023-11-28T11:08:50.105047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571154+0000","last_change":"2023-11-28T11:08:50.132292+0000","last_active":"2023-11-28T11:08:51.571154+0000","last_peered":"2023-11-28T11:08:51.571154+0000","last_clean":"2023-11-28T11:08:51.571154+0000","last_became_active":"2023-11-28T11:08:50.131887+0000","last_became_peered":"2023-11-28T11:08:50.131887+0000","last_unstale":"2023-11-28T11:08:51.571154+0000","last_undegraded":"2023-11-28T11:08:51.571154+0000","last_fullsized":"2023-11-28T11:08:51.571154+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:08.804864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115394+0000","last_change":"2023-11-28T11:08:45.516041+0000","last_active":"2023-11-28T11:08:50.115394+0000","last_peered":"2023-11-28T11:08:50.115394+0000","last_clean":"2023-11-28T11:08:50.115394+0000","last_became_active":"2023-11-28T11:08:45.515742+0000","last_became_peered":"2023-11-28T11:08:45.515742+0000","last_unstale":"2023-11-28T11:08:50.115394+0000","last_undegraded":"2023-11-28T11:08:50.115394+0000","last_fullsized":"2023-11-28T11:08:50.115394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:14.983939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113365+0000","last_change":"2023-11-28T11:08:45.945858+0000","last_active":"2023-11-28T11:08:50.113365+0000","last_peered":"2023-11-28T11:08:50.113365+0000","last_clean":"2023-11-28T11:08:50.113365+0000","last_became_active":"2023-11-28T11:08:45.945552+0000","last_became_peered":"2023-11-28T11:08:45.945552+0000","last_unstale":"2023-11-28T11:08:50.113365+0000","last_undegraded":"2023-11-28T11:08:50.113365+0000","last_fullsized":"2023-11-28T11:08:50.113365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571022+0000","last_change":"2023-11-28T11:08:50.123713+0000","last_active":"2023-11-28T11:08:51.571022+0000","last_peered":"2023-11-28T11:08:51.571022+0000","last_clean":"2023-11-28T11:08:51.571022+0000","last_became_active":"2023-11-28T11:08:50.123264+0000","last_became_peered":"2023-11-28T11:08:50.123264+0000","last_unstale":"2023-11-28T11:08:51.571022+0000","last_undegraded":"2023-11-28T11:08:51.571022+0000","last_fullsized":"2023-11-28T11:08:51.571022+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:04.199211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115481+0000","last_change":"2023-11-28T11:08:45.515894+0000","last_active":"2023-11-28T11:08:50.115481+0000","last_peered":"2023-11-28T11:08:50.115481+0000","last_clean":"2023-11-28T11:08:50.115481+0000","last_became_active":"2023-11-28T11:08:45.515593+0000","last_became_peered":"2023-11-28T11:08:45.515593+0000","last_unstale":"2023-11-28T11:08:50.115481+0000","last_undegraded":"2023-11-28T11:08:50.115481+0000","last_fullsized":"2023-11-28T11:08:50.115481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:31.747218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109629+0000","last_change":"2023-11-28T11:08:45.949548+0000","last_active":"2023-11-28T11:08:50.109629+0000","last_peered":"2023-11-28T11:08:50.109629+0000","last_clean":"2023-11-28T11:08:50.109629+0000","last_became_active":"2023-11-28T11:08:45.949217+0000","last_became_peered":"2023-11-28T11:08:45.949217+0000","last_unstale":"2023-11-28T11:08:50.109629+0000","last_undegraded":"2023-11-28T11:08:50.109629+0000","last_fullsized":"2023-11-28T11:08:50.109629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:01.447773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571373+0000","last_change":"2023-11-28T11:08:50.127125+0000","last_active":"2023-11-28T11:08:51.571373+0000","last_peered":"2023-11-28T11:08:51.571373+0000","last_clean":"2023-11-28T11:08:51.571373+0000","last_became_active":"2023-11-28T11:08:50.124511+0000","last_became_peered":"2023-11-28T11:08:50.124511+0000","last_unstale":"2023-11-28T11:08:51.571373+0000","last_undegraded":"2023-11-28T11:08:51.571373+0000","last_fullsized":"2023-11-28T11:08:51.571373+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:47.985900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105609+0000","last_change":"2023-11-28T11:08:45.513992+0000","last_active":"2023-11-28T11:08:50.105609+0000","last_peered":"2023-11-28T11:08:50.105609+0000","last_clean":"2023-11-28T11:08:50.105609+0000","last_became_active":"2023-11-28T11:08:45.513697+0000","last_became_peered":"2023-11-28T11:08:45.513697+0000","last_unstale":"2023-11-28T11:08:50.105609+0000","last_undegraded":"2023-11-28T11:08:50.105609+0000","last_fullsized":"2023-11-28T11:08:50.105609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:28.586127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103999+0000","last_change":"2023-11-28T11:08:45.940813+0000","last_active":"2023-11-28T11:08:50.103999+0000","last_peered":"2023-11-28T11:08:50.103999+0000","last_clean":"2023-11-28T11:08:50.103999+0000","last_became_active":"2023-11-28T11:08:45.940552+0000","last_became_peered":"2023-11-28T11:08:45.940552+0000","last_unstale":"2023-11-28T11:08:50.103999+0000","last_undegraded":"2023-11-28T11:08:50.103999+0000","last_fullsized":"2023-11-28T11:08:50.103999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:17.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571495+0000","last_change":"2023-11-28T11:08:50.121770+0000","last_active":"2023-11-28T11:08:51.571495+0000","last_peered":"2023-11-28T11:08:51.571495+0000","last_clean":"2023-11-28T11:08:51.571495+0000","last_became_active":"2023-11-28T11:08:50.121277+0000","last_became_peered":"2023-11-28T11:08:50.121277+0000","last_unstale":"2023-11-28T11:08:51.571495+0000","last_undegraded":"2023-11-28T11:08:51.571495+0000","last_fullsized":"2023-11-28T11:08:51.571495+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:42.759036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109255+0000","last_change":"2023-11-28T11:08:45.519824+0000","last_active":"2023-11-28T11:08:50.109255+0000","last_peered":"2023-11-28T11:08:50.109255+0000","last_clean":"2023-11-28T11:08:50.109255+0000","last_became_active":"2023-11-28T11:08:45.519241+0000","last_became_peered":"2023-11-28T11:08:45.519241+0000","last_unstale":"2023-11-28T11:08:50.109255+0000","last_undegraded":"2023-11-28T11:08:50.109255+0000","last_fullsized":"2023-11-28T11:08:50.109255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:31.747218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112858+0000","last_change":"2023-11-28T11:08:45.946401+0000","last_active":"2023-11-28T11:08:50.112858+0000","last_peered":"2023-11-28T11:08:50.112858+0000","last_clean":"2023-11-28T11:08:50.112858+0000","last_became_active":"2023-11-28T11:08:45.946070+0000","last_became_peered":"2023-11-28T11:08:45.946070+0000","last_unstale":"2023-11-28T11:08:50.112858+0000","last_undegraded":"2023-11-28T11:08:50.112858+0000","last_fullsized":"2023-11-28T11:08:50.112858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:05.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570768+0000","last_change":"2023-11-28T11:08:50.131808+0000","last_active":"2023-11-28T11:08:51.570768+0000","last_peered":"2023-11-28T11:08:51.570768+0000","last_clean":"2023-11-28T11:08:51.570768+0000","last_became_active":"2023-11-28T11:08:50.131311+0000","last_became_peered":"2023-11-28T11:08:50.131311+0000","last_unstale":"2023-11-28T11:08:51.570768+0000","last_undegraded":"2023-11-28T11:08:51.570768+0000","last_fullsized":"2023-11-28T11:08:51.570768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:06.157479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109818+0000","last_change":"2023-11-28T11:08:45.517239+0000","last_active":"2023-11-28T11:08:50.109818+0000","last_peered":"2023-11-28T11:08:50.109818+0000","last_clean":"2023-11-28T11:08:50.109818+0000","last_became_active":"2023-11-28T11:08:45.516694+0000","last_became_peered":"2023-11-28T11:08:45.516694+0000","last_unstale":"2023-11-28T11:08:50.109818+0000","last_undegraded":"2023-11-28T11:08:50.109818+0000","last_fullsized":"2023-11-28T11:08:50.109818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113094+0000","last_change":"2023-11-28T11:08:45.947974+0000","last_active":"2023-11-28T11:08:50.113094+0000","last_peered":"2023-11-28T11:08:50.113094+0000","last_clean":"2023-11-28T11:08:50.113094+0000","last_became_active":"2023-11-28T11:08:45.947674+0000","last_became_peered":"2023-11-28T11:08:45.947674+0000","last_unstale":"2023-11-28T11:08:50.113094+0000","last_undegraded":"2023-11-28T11:08:50.113094+0000","last_fullsized":"2023-11-28T11:08:50.113094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:44.385589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.538066+0000","last_change":"2023-11-28T11:08:50.126205+0000","last_active":"2023-11-28T11:08:50.538066+0000","last_peered":"2023-11-28T11:08:50.538066+0000","last_clean":"2023-11-28T11:08:50.538066+0000","last_became_active":"2023-11-28T11:08:50.125446+0000","last_became_peered":"2023-11-28T11:08:50.125446+0000","last_unstale":"2023-11-28T11:08:50.538066+0000","last_undegraded":"2023-11-28T11:08:50.538066+0000","last_fullsized":"2023-11-28T11:08:50.538066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:59.602687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113146+0000","last_change":"2023-11-28T11:08:45.512742+0000","last_active":"2023-11-28T11:08:50.113146+0000","last_peered":"2023-11-28T11:08:50.113146+0000","last_clean":"2023-11-28T11:08:50.113146+0000","last_became_active":"2023-11-28T11:08:45.512446+0000","last_became_peered":"2023-11-28T11:08:45.512446+0000","last_unstale":"2023-11-28T11:08:50.113146+0000","last_undegraded":"2023-11-28T11:08:50.113146+0000","last_fullsized":"2023-11-28T11:08:50.113146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113682+0000","last_change":"2023-11-28T11:08:45.955541+0000","last_active":"2023-11-28T11:08:50.113682+0000","last_peered":"2023-11-28T11:08:50.113682+0000","last_clean":"2023-11-28T11:08:50.113682+0000","last_became_active":"2023-11-28T11:08:45.955223+0000","last_became_peered":"2023-11-28T11:08:45.955223+0000","last_unstale":"2023-11-28T11:08:50.113682+0000","last_undegraded":"2023-11-28T11:08:50.113682+0000","last_fullsized":"2023-11-28T11:08:50.113682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:24:04.835870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.536215+0000","last_change":"2023-11-28T11:08:50.134010+0000","last_active":"2023-11-28T11:08:50.536215+0000","last_peered":"2023-11-28T11:08:50.536215+0000","last_clean":"2023-11-28T11:08:50.536215+0000","last_became_active":"2023-11-28T11:08:50.133519+0000","last_became_peered":"2023-11-28T11:08:50.133519+0000","last_unstale":"2023-11-28T11:08:50.536215+0000","last_undegraded":"2023-11-28T11:08:50.536215+0000","last_fullsized":"2023-11-28T11:08:50.536215+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:47.676934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109102+0000","last_change":"2023-11-28T11:08:45.515039+0000","last_active":"2023-11-28T11:08:50.109102+0000","last_peered":"2023-11-28T11:08:50.109102+0000","last_clean":"2023-11-28T11:08:50.109102+0000","last_became_active":"2023-11-28T11:08:45.514380+0000","last_became_peered":"2023-11-28T11:08:45.514380+0000","last_unstale":"2023-11-28T11:08:50.109102+0000","last_undegraded":"2023-11-28T11:08:50.109102+0000","last_fullsized":"2023-11-28T11:08:50.109102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:28.586127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120542+0000","last_change":"2023-11-28T11:08:45.945761+0000","last_active":"2023-11-28T11:08:50.120542+0000","last_peered":"2023-11-28T11:08:50.120542+0000","last_clean":"2023-11-28T11:08:50.120542+0000","last_became_active":"2023-11-28T11:08:45.945501+0000","last_became_peered":"2023-11-28T11:08:45.945501+0000","last_unstale":"2023-11-28T11:08:50.120542+0000","last_undegraded":"2023-11-28T11:08:50.120542+0000","last_fullsized":"2023-11-28T11:08:50.120542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:30.639887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.535821+0000","last_change":"2023-11-28T11:08:50.129233+0000","last_active":"2023-11-28T11:08:50.535821+0000","last_peered":"2023-11-28T11:08:50.535821+0000","last_clean":"2023-11-28T11:08:50.535821+0000","last_became_active":"2023-11-28T11:08:50.128785+0000","last_became_peered":"2023-11-28T11:08:50.128785+0000","last_unstale":"2023-11-28T11:08:50.535821+0000","last_undegraded":"2023-11-28T11:08:50.535821+0000","last_fullsized":"2023-11-28T11:08:50.535821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:14.088471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.111193+0000","last_change":"2023-11-28T11:08:45.519399+0000","last_active":"2023-11-28T11:08:50.111193+0000","last_peered":"2023-11-28T11:08:50.111193+0000","last_clean":"2023-11-28T11:08:50.111193+0000","last_became_active":"2023-11-28T11:08:45.519141+0000","last_became_peered":"2023-11-28T11:08:45.519141+0000","last_unstale":"2023-11-28T11:08:50.111193+0000","last_undegraded":"2023-11-28T11:08:50.111193+0000","last_fullsized":"2023-11-28T11:08:50.111193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:24.093818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113185+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:08:50.113185+0000","last_peered":"2023-11-28T11:08:50.113185+0000","last_clean":"2023-11-28T11:08:50.113185+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:08:50.113185+0000","last_undegraded":"2023-11-28T11:08:50.113185+0000","last_fullsized":"2023-11-28T11:08:50.113185+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109007+0000","last_change":"2023-11-28T11:08:45.955853+0000","last_active":"2023-11-28T11:08:50.109007+0000","last_peered":"2023-11-28T11:08:50.109007+0000","last_clean":"2023-11-28T11:08:50.109007+0000","last_became_active":"2023-11-28T11:08:45.955563+0000","last_became_peered":"2023-11-28T11:08:45.955563+0000","last_unstale":"2023-11-28T11:08:50.109007+0000","last_undegraded":"2023-11-28T11:08:50.109007+0000","last_fullsized":"2023-11-28T11:08:50.109007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:00.598900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569792+0000","last_change":"2023-11-28T11:08:50.129450+0000","last_active":"2023-11-28T11:08:51.569792+0000","last_peered":"2023-11-28T11:08:51.569792+0000","last_clean":"2023-11-28T11:08:51.569792+0000","last_became_active":"2023-11-28T11:08:50.128981+0000","last_became_peered":"2023-11-28T11:08:50.128981+0000","last_unstale":"2023-11-28T11:08:51.569792+0000","last_undegraded":"2023-11-28T11:08:51.569792+0000","last_fullsized":"2023-11-28T11:08:51.569792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:14.466494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108203+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:08:50.108203+0000","last_peered":"2023-11-28T11:08:50.108203+0000","last_clean":"2023-11-28T11:08:50.108203+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:08:50.108203+0000","last_undegraded":"2023-11-28T11:08:50.108203+0000","last_fullsized":"2023-11-28T11:08:50.108203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.513023+0000","last_change":"2023-11-28T11:08:45.522154+0000","last_active":"2023-11-28T11:08:50.513023+0000","last_peered":"2023-11-28T11:08:50.513023+0000","last_clean":"2023-11-28T11:08:50.513023+0000","last_became_active":"2023-11-28T11:08:45.518652+0000","last_became_peered":"2023-11-28T11:08:45.518652+0000","last_unstale":"2023-11-28T11:08:50.513023+0000","last_undegraded":"2023-11-28T11:08:50.513023+0000","last_fullsized":"2023-11-28T11:08:50.513023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:49.800409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113254+0000","last_change":"2023-11-28T11:08:45.946786+0000","last_active":"2023-11-28T11:08:50.113254+0000","last_peered":"2023-11-28T11:08:50.113254+0000","last_clean":"2023-11-28T11:08:50.113254+0000","last_became_active":"2023-11-28T11:08:45.946446+0000","last_became_peered":"2023-11-28T11:08:45.946446+0000","last_unstale":"2023-11-28T11:08:50.113254+0000","last_undegraded":"2023-11-28T11:08:50.113254+0000","last_fullsized":"2023-11-28T11:08:50.113254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:39.158725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537975+0000","last_change":"2023-11-28T11:08:50.123220+0000","last_active":"2023-11-28T11:08:50.537975+0000","last_peered":"2023-11-28T11:08:50.537975+0000","last_clean":"2023-11-28T11:08:50.537975+0000","last_became_active":"2023-11-28T11:08:50.122776+0000","last_became_peered":"2023-11-28T11:08:50.122776+0000","last_unstale":"2023-11-28T11:08:50.537975+0000","last_undegraded":"2023-11-28T11:08:50.537975+0000","last_fullsized":"2023-11-28T11:08:50.537975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:35.176626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119767+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:08:50.119767+0000","last_peered":"2023-11-28T11:08:50.119767+0000","last_clean":"2023-11-28T11:08:50.119767+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:08:50.119767+0000","last_undegraded":"2023-11-28T11:08:50.119767+0000","last_fullsized":"2023-11-28T11:08:50.119767+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":167,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115593+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:08:50.115593+0000","last_peered":"2023-11-28T11:08:50.115593+0000","last_clean":"2023-11-28T11:08:50.115593+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:08:50.115593+0000","last_undegraded":"2023-11-28T11:08:50.115593+0000","last_fullsized":"2023-11-28T11:08:50.115593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103405+0000","last_change":"2023-11-28T11:08:45.521876+0000","last_active":"2023-11-28T11:08:50.103405+0000","last_peered":"2023-11-28T11:08:50.103405+0000","last_clean":"2023-11-28T11:08:50.103405+0000","last_became_active":"2023-11-28T11:08:45.521580+0000","last_became_peered":"2023-11-28T11:08:45.521580+0000","last_unstale":"2023-11-28T11:08:50.103405+0000","last_undegraded":"2023-11-28T11:08:50.103405+0000","last_fullsized":"2023-11-28T11:08:50.103405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.104071+0000","last_change":"2023-11-28T11:08:45.948040+0000","last_active":"2023-11-28T11:08:50.104071+0000","last_peered":"2023-11-28T11:08:50.104071+0000","last_clean":"2023-11-28T11:08:50.104071+0000","last_became_active":"2023-11-28T11:08:45.947704+0000","last_became_peered":"2023-11-28T11:08:45.947704+0000","last_unstale":"2023-11-28T11:08:50.104071+0000","last_undegraded":"2023-11-28T11:08:50.104071+0000","last_fullsized":"2023-11-28T11:08:50.104071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571808+0000","last_change":"2023-11-28T11:08:50.124074+0000","last_active":"2023-11-28T11:08:51.571808+0000","last_peered":"2023-11-28T11:08:51.571808+0000","last_clean":"2023-11-28T11:08:51.571808+0000","last_became_active":"2023-11-28T11:08:50.123630+0000","last_became_peered":"2023-11-28T11:08:50.123630+0000","last_unstale":"2023-11-28T11:08:51.571808+0000","last_undegraded":"2023-11-28T11:08:51.571808+0000","last_fullsized":"2023-11-28T11:08:51.571808+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:52.786613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115788+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:08:50.115788+0000","last_peered":"2023-11-28T11:08:50.115788+0000","last_clean":"2023-11-28T11:08:50.115788+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:08:50.115788+0000","last_undegraded":"2023-11-28T11:08:50.115788+0000","last_fullsized":"2023-11-28T11:08:50.115788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119464+0000","last_change":"2023-11-28T11:08:45.510984+0000","last_active":"2023-11-28T11:08:50.119464+0000","last_peered":"2023-11-28T11:08:50.119464+0000","last_clean":"2023-11-28T11:08:50.119464+0000","last_became_active":"2023-11-28T11:08:45.510734+0000","last_became_peered":"2023-11-28T11:08:45.510734+0000","last_unstale":"2023-11-28T11:08:50.119464+0000","last_undegraded":"2023-11-28T11:08:50.119464+0000","last_fullsized":"2023-11-28T11:08:50.119464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:33.188220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120521+0000","last_change":"2023-11-28T11:08:45.946753+0000","last_active":"2023-11-28T11:08:50.120521+0000","last_peered":"2023-11-28T11:08:50.120521+0000","last_clean":"2023-11-28T11:08:50.120521+0000","last_became_active":"2023-11-28T11:08:45.946460+0000","last_became_peered":"2023-11-28T11:08:45.946460+0000","last_unstale":"2023-11-28T11:08:50.120521+0000","last_undegraded":"2023-11-28T11:08:50.120521+0000","last_fullsized":"2023-11-28T11:08:50.120521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:20:10.730266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570567+0000","last_change":"2023-11-28T11:08:50.132450+0000","last_active":"2023-11-28T11:08:51.570567+0000","last_peered":"2023-11-28T11:08:51.570567+0000","last_clean":"2023-11-28T11:08:51.570567+0000","last_became_active":"2023-11-28T11:08:50.132043+0000","last_became_peered":"2023-11-28T11:08:50.132043+0000","last_unstale":"2023-11-28T11:08:51.570567+0000","last_undegraded":"2023-11-28T11:08:51.570567+0000","last_fullsized":"2023-11-28T11:08:51.570567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:20.989038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115361+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:08:50.115361+0000","last_peered":"2023-11-28T11:08:50.115361+0000","last_clean":"2023-11-28T11:08:50.115361+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:08:50.115361+0000","last_undegraded":"2023-11-28T11:08:50.115361+0000","last_fullsized":"2023-11-28T11:08:50.115361+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108211+0000","last_change":"2023-11-28T11:08:45.513181+0000","last_active":"2023-11-28T11:08:50.108211+0000","last_peered":"2023-11-28T11:08:50.108211+0000","last_clean":"2023-11-28T11:08:50.108211+0000","last_became_active":"2023-11-28T11:08:45.512903+0000","last_became_peered":"2023-11-28T11:08:45.512903+0000","last_unstale":"2023-11-28T11:08:50.108211+0000","last_undegraded":"2023-11-28T11:08:50.108211+0000","last_fullsized":"2023-11-28T11:08:50.108211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109406+0000","last_change":"2023-11-28T11:08:45.947505+0000","last_active":"2023-11-28T11:08:50.109406+0000","last_peered":"2023-11-28T11:08:50.109406+0000","last_clean":"2023-11-28T11:08:50.109406+0000","last_became_active":"2023-11-28T11:08:45.947199+0000","last_became_peered":"2023-11-28T11:08:45.947199+0000","last_unstale":"2023-11-28T11:08:50.109406+0000","last_undegraded":"2023-11-28T11:08:50.109406+0000","last_fullsized":"2023-11-28T11:08:50.109406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:55.085644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.437473+0000","last_change":"2023-11-28T11:08:50.126736+0000","last_active":"2023-11-28T11:08:51.437473+0000","last_peered":"2023-11-28T11:08:51.437473+0000","last_clean":"2023-11-28T11:08:51.437473+0000","last_became_active":"2023-11-28T11:08:50.126282+0000","last_became_peered":"2023-11-28T11:08:50.126282+0000","last_unstale":"2023-11-28T11:08:51.437473+0000","last_undegraded":"2023-11-28T11:08:51.437473+0000","last_fullsized":"2023-11-28T11:08:51.437473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:36.656033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.104943+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:08:50.104943+0000","last_peered":"2023-11-28T11:08:50.104943+0000","last_clean":"2023-11-28T11:08:50.104943+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:08:50.104943+0000","last_undegraded":"2023-11-28T11:08:50.104943+0000","last_fullsized":"2023-11-28T11:08:50.104943+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112806+0000","last_change":"2023-11-28T11:08:45.524531+0000","last_active":"2023-11-28T11:08:50.112806+0000","last_peered":"2023-11-28T11:08:50.112806+0000","last_clean":"2023-11-28T11:08:50.112806+0000","last_became_active":"2023-11-28T11:08:45.524334+0000","last_became_peered":"2023-11-28T11:08:45.524334+0000","last_unstale":"2023-11-28T11:08:50.112806+0000","last_undegraded":"2023-11-28T11:08:50.112806+0000","last_fullsized":"2023-11-28T11:08:50.112806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119778+0000","last_change":"2023-11-28T11:08:45.944621+0000","last_active":"2023-11-28T11:08:50.119778+0000","last_peered":"2023-11-28T11:08:50.119778+0000","last_clean":"2023-11-28T11:08:50.119778+0000","last_became_active":"2023-11-28T11:08:45.944368+0000","last_became_peered":"2023-11-28T11:08:45.944368+0000","last_unstale":"2023-11-28T11:08:50.119778+0000","last_undegraded":"2023-11-28T11:08:50.119778+0000","last_fullsized":"2023-11-28T11:08:50.119778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:01.447773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570969+0000","last_change":"2023-11-28T11:08:50.131902+0000","last_active":"2023-11-28T11:08:51.570969+0000","last_peered":"2023-11-28T11:08:51.570969+0000","last_clean":"2023-11-28T11:08:51.570969+0000","last_became_active":"2023-11-28T11:08:50.131485+0000","last_became_peered":"2023-11-28T11:08:50.131485+0000","last_unstale":"2023-11-28T11:08:51.570969+0000","last_undegraded":"2023-11-28T11:08:51.570969+0000","last_fullsized":"2023-11-28T11:08:51.570969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:08.804864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110005+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:08:50.110005+0000","last_peered":"2023-11-28T11:08:50.110005+0000","last_clean":"2023-11-28T11:08:50.110005+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:08:50.110005+0000","last_undegraded":"2023-11-28T11:08:50.110005+0000","last_fullsized":"2023-11-28T11:08:50.110005+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120247+0000","last_change":"2023-11-28T11:08:45.514598+0000","last_active":"2023-11-28T11:08:50.120247+0000","last_peered":"2023-11-28T11:08:50.120247+0000","last_clean":"2023-11-28T11:08:50.120247+0000","last_became_active":"2023-11-28T11:08:45.514348+0000","last_became_peered":"2023-11-28T11:08:45.514348+0000","last_unstale":"2023-11-28T11:08:50.120247+0000","last_undegraded":"2023-11-28T11:08:50.120247+0000","last_fullsized":"2023-11-28T11:08:50.120247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103691+0000","last_change":"2023-11-28T11:08:45.950031+0000","last_active":"2023-11-28T11:08:50.103691+0000","last_peered":"2023-11-28T11:08:50.103691+0000","last_clean":"2023-11-28T11:08:50.103691+0000","last_became_active":"2023-11-28T11:08:45.949457+0000","last_became_peered":"2023-11-28T11:08:45.949457+0000","last_unstale":"2023-11-28T11:08:50.103691+0000","last_undegraded":"2023-11-28T11:08:50.103691+0000","last_fullsized":"2023-11-28T11:08:50.103691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:34.416718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.437351+0000","last_change":"2023-11-28T11:08:50.124074+0000","last_active":"2023-11-28T11:08:51.437351+0000","last_peered":"2023-11-28T11:08:51.437351+0000","last_clean":"2023-11-28T11:08:51.437351+0000","last_became_active":"2023-11-28T11:08:50.123710+0000","last_became_peered":"2023-11-28T11:08:50.123710+0000","last_unstale":"2023-11-28T11:08:51.437351+0000","last_undegraded":"2023-11-28T11:08:51.437351+0000","last_fullsized":"2023-11-28T11:08:51.437351+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:42:35.306388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110239+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:08:50.110239+0000","last_peered":"2023-11-28T11:08:50.110239+0000","last_clean":"2023-11-28T11:08:50.110239+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:08:50.110239+0000","last_undegraded":"2023-11-28T11:08:50.110239+0000","last_fullsized":"2023-11-28T11:08:50.110239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105883+0000","last_change":"2023-11-28T11:08:45.514165+0000","last_active":"2023-11-28T11:08:50.105883+0000","last_peered":"2023-11-28T11:08:50.105883+0000","last_clean":"2023-11-28T11:08:50.105883+0000","last_became_active":"2023-11-28T11:08:45.513651+0000","last_became_peered":"2023-11-28T11:08:45.513651+0000","last_unstale":"2023-11-28T11:08:50.105883+0000","last_undegraded":"2023-11-28T11:08:50.105883+0000","last_fullsized":"2023-11-28T11:08:50.105883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119668+0000","last_change":"2023-11-28T11:08:45.949610+0000","last_active":"2023-11-28T11:08:50.119668+0000","last_peered":"2023-11-28T11:08:50.119668+0000","last_clean":"2023-11-28T11:08:50.119668+0000","last_became_active":"2023-11-28T11:08:45.949348+0000","last_became_peered":"2023-11-28T11:08:45.949348+0000","last_unstale":"2023-11-28T11:08:50.119668+0000","last_undegraded":"2023-11-28T11:08:50.119668+0000","last_fullsized":"2023-11-28T11:08:50.119668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:23:53.819338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570113+0000","last_change":"2023-11-28T11:08:50.135779+0000","last_active":"2023-11-28T11:08:51.570113+0000","last_peered":"2023-11-28T11:08:51.570113+0000","last_clean":"2023-11-28T11:08:51.570113+0000","last_became_active":"2023-11-28T11:08:50.131379+0000","last_became_peered":"2023-11-28T11:08:50.131379+0000","last_unstale":"2023-11-28T11:08:51.570113+0000","last_undegraded":"2023-11-28T11:08:51.570113+0000","last_fullsized":"2023-11-28T11:08:51.570113+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:42:35.306388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103804+0000","last_change":"2023-11-28T11:08:45.523090+0000","last_active":"2023-11-28T11:08:50.103804+0000","last_peered":"2023-11-28T11:08:50.103804+0000","last_clean":"2023-11-28T11:08:50.103804+0000","last_became_active":"2023-11-28T11:08:45.522445+0000","last_became_peered":"2023-11-28T11:08:45.522445+0000","last_unstale":"2023-11-28T11:08:50.103804+0000","last_undegraded":"2023-11-28T11:08:50.103804+0000","last_fullsized":"2023-11-28T11:08:50.103804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108643+0000","last_change":"2023-11-28T11:08:45.943849+0000","last_active":"2023-11-28T11:08:50.108643+0000","last_peered":"2023-11-28T11:08:50.108643+0000","last_clean":"2023-11-28T11:08:50.108643+0000","last_became_active":"2023-11-28T11:08:45.943426+0000","last_became_peered":"2023-11-28T11:08:45.943426+0000","last_unstale":"2023-11-28T11:08:50.108643+0000","last_undegraded":"2023-11-28T11:08:50.108643+0000","last_fullsized":"2023-11-28T11:08:50.108643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:29.591607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570960+0000","last_change":"2023-11-28T11:08:50.129403+0000","last_active":"2023-11-28T11:08:51.570960+0000","last_peered":"2023-11-28T11:08:51.570960+0000","last_clean":"2023-11-28T11:08:51.570960+0000","last_became_active":"2023-11-28T11:08:50.128984+0000","last_became_peered":"2023-11-28T11:08:50.128984+0000","last_unstale":"2023-11-28T11:08:51.570960+0000","last_undegraded":"2023-11-28T11:08:51.570960+0000","last_fullsized":"2023-11-28T11:08:51.570960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:54.778391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105418+0000","last_change":"2023-11-28T11:08:45.514673+0000","last_active":"2023-11-28T11:08:50.105418+0000","last_peered":"2023-11-28T11:08:50.105418+0000","last_clean":"2023-11-28T11:08:50.105418+0000","last_became_active":"2023-11-28T11:08:45.513328+0000","last_became_peered":"2023-11-28T11:08:45.513328+0000","last_unstale":"2023-11-28T11:08:50.105418+0000","last_undegraded":"2023-11-28T11:08:50.105418+0000","last_fullsized":"2023-11-28T11:08:50.105418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:55.864512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112696+0000","last_change":"2023-11-28T11:08:45.948904+0000","last_active":"2023-11-28T11:08:50.112696+0000","last_peered":"2023-11-28T11:08:50.112696+0000","last_clean":"2023-11-28T11:08:50.112696+0000","last_became_active":"2023-11-28T11:08:45.948629+0000","last_became_peered":"2023-11-28T11:08:45.948629+0000","last_unstale":"2023-11-28T11:08:50.112696+0000","last_undegraded":"2023-11-28T11:08:50.112696+0000","last_fullsized":"2023-11-28T11:08:50.112696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:55.031649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537746+0000","last_change":"2023-11-28T11:08:50.124117+0000","last_active":"2023-11-28T11:08:50.537746+0000","last_peered":"2023-11-28T11:08:50.537746+0000","last_clean":"2023-11-28T11:08:50.537746+0000","last_became_active":"2023-11-28T11:08:50.123257+0000","last_became_peered":"2023-11-28T11:08:50.123257+0000","last_unstale":"2023-11-28T11:08:50.537746+0000","last_undegraded":"2023-11-28T11:08:50.537746+0000","last_fullsized":"2023-11-28T11:08:50.537746+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:24.842089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105770+0000","last_change":"2023-11-28T11:08:45.511721+0000","last_active":"2023-11-28T11:08:50.105770+0000","last_peered":"2023-11-28T11:08:50.105770+0000","last_clean":"2023-11-28T11:08:50.105770+0000","last_became_active":"2023-11-28T11:08:45.511413+0000","last_became_peered":"2023-11-28T11:08:45.511413+0000","last_unstale":"2023-11-28T11:08:50.105770+0000","last_undegraded":"2023-11-28T11:08:50.105770+0000","last_fullsized":"2023-11-28T11:08:50.105770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:24.093818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.111702+0000","last_change":"2023-11-28T11:08:45.945561+0000","last_active":"2023-11-28T11:08:50.111702+0000","last_peered":"2023-11-28T11:08:50.111702+0000","last_clean":"2023-11-28T11:08:50.111702+0000","last_became_active":"2023-11-28T11:08:45.945101+0000","last_became_peered":"2023-11-28T11:08:45.945101+0000","last_unstale":"2023-11-28T11:08:50.111702+0000","last_undegraded":"2023-11-28T11:08:50.111702+0000","last_fullsized":"2023-11-28T11:08:50.111702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:00.598900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.535418+0000","last_change":"2023-11-28T11:08:50.132859+0000","last_active":"2023-11-28T11:08:50.535418+0000","last_peered":"2023-11-28T11:08:50.535418+0000","last_clean":"2023-11-28T11:08:50.535418+0000","last_became_active":"2023-11-28T11:08:50.132397+0000","last_became_peered":"2023-11-28T11:08:50.132397+0000","last_unstale":"2023-11-28T11:08:50.535418+0000","last_undegraded":"2023-11-28T11:08:50.535418+0000","last_fullsized":"2023-11-28T11:08:50.535418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:07.333534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112935+0000","last_change":"2023-11-28T11:08:45.511864+0000","last_active":"2023-11-28T11:08:50.112935+0000","last_peered":"2023-11-28T11:08:50.112935+0000","last_clean":"2023-11-28T11:08:50.112935+0000","last_became_active":"2023-11-28T11:08:45.511500+0000","last_became_peered":"2023-11-28T11:08:45.511500+0000","last_unstale":"2023-11-28T11:08:50.112935+0000","last_undegraded":"2023-11-28T11:08:50.112935+0000","last_fullsized":"2023-11-28T11:08:50.112935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:55.864512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108100+0000","last_change":"2023-11-28T11:08:45.945158+0000","last_active":"2023-11-28T11:08:50.108100+0000","last_peered":"2023-11-28T11:08:50.108100+0000","last_clean":"2023-11-28T11:08:50.108100+0000","last_became_active":"2023-11-28T11:08:45.944341+0000","last_became_peered":"2023-11-28T11:08:45.944341+0000","last_unstale":"2023-11-28T11:08:50.108100+0000","last_undegraded":"2023-11-28T11:08:50.108100+0000","last_fullsized":"2023-11-28T11:08:50.108100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569813+0000","last_change":"2023-11-28T11:08:50.136091+0000","last_active":"2023-11-28T11:08:51.569813+0000","last_peered":"2023-11-28T11:08:51.569813+0000","last_clean":"2023-11-28T11:08:51.569813+0000","last_became_active":"2023-11-28T11:08:50.135640+0000","last_became_peered":"2023-11-28T11:08:50.135640+0000","last_unstale":"2023-11-28T11:08:51.569813+0000","last_undegraded":"2023-11-28T11:08:51.569813+0000","last_fullsized":"2023-11-28T11:08:51.569813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:19:52.258463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106438+0000","last_change":"2023-11-28T11:08:45.522945+0000","last_active":"2023-11-28T11:08:50.106438+0000","last_peered":"2023-11-28T11:08:50.106438+0000","last_clean":"2023-11-28T11:08:50.106438+0000","last_became_active":"2023-11-28T11:08:45.522100+0000","last_became_peered":"2023-11-28T11:08:45.522100+0000","last_unstale":"2023-11-28T11:08:50.106438+0000","last_undegraded":"2023-11-28T11:08:50.106438+0000","last_fullsized":"2023-11-28T11:08:50.106438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:24.093818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.116376+0000","last_change":"2023-11-28T11:08:45.955608+0000","last_active":"2023-11-28T11:08:50.116376+0000","last_peered":"2023-11-28T11:08:50.116376+0000","last_clean":"2023-11-28T11:08:50.116376+0000","last_became_active":"2023-11-28T11:08:45.955293+0000","last_became_peered":"2023-11-28T11:08:45.955293+0000","last_unstale":"2023-11-28T11:08:50.116376+0000","last_undegraded":"2023-11-28T11:08:50.116376+0000","last_fullsized":"2023-11-28T11:08:50.116376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:33.088404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108532+0000","last_change":"2023-11-28T11:08:45.953710+0000","last_active":"2023-11-28T11:08:50.108532+0000","last_peered":"2023-11-28T11:08:50.108532+0000","last_clean":"2023-11-28T11:08:50.108532+0000","last_became_active":"2023-11-28T11:08:45.953349+0000","last_became_peered":"2023-11-28T11:08:45.953349+0000","last_unstale":"2023-11-28T11:08:50.108532+0000","last_undegraded":"2023-11-28T11:08:50.108532+0000","last_fullsized":"2023-11-28T11:08:50.108532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:03.032281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.107845+0000","last_change":"2023-11-28T11:08:45.945358+0000","last_active":"2023-11-28T11:08:50.107845+0000","last_peered":"2023-11-28T11:08:50.107845+0000","last_clean":"2023-11-28T11:08:50.107845+0000","last_became_active":"2023-11-28T11:08:45.944619+0000","last_became_peered":"2023-11-28T11:08:45.944619+0000","last_unstale":"2023-11-28T11:08:50.107845+0000","last_undegraded":"2023-11-28T11:08:50.107845+0000","last_fullsized":"2023-11-28T11:08:50.107845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:17.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112795+0000","last_change":"2023-11-28T11:08:45.951315+0000","last_active":"2023-11-28T11:08:50.112795+0000","last_peered":"2023-11-28T11:08:50.112795+0000","last_clean":"2023-11-28T11:08:50.112795+0000","last_became_active":"2023-11-28T11:08:45.950992+0000","last_became_peered":"2023-11-28T11:08:45.950992+0000","last_unstale":"2023-11-28T11:08:50.112795+0000","last_undegraded":"2023-11-28T11:08:50.112795+0000","last_fullsized":"2023-11-28T11:08:50.112795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:51.178080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109781+0000","last_change":"2023-11-28T11:08:45.954451+0000","last_active":"2023-11-28T11:08:50.109781+0000","last_peered":"2023-11-28T11:08:50.109781+0000","last_clean":"2023-11-28T11:08:50.109781+0000","last_became_active":"2023-11-28T11:08:45.954162+0000","last_became_peered":"2023-11-28T11:08:45.954162+0000","last_unstale":"2023-11-28T11:08:50.109781+0000","last_undegraded":"2023-11-28T11:08:50.109781+0000","last_fullsized":"2023-11-28T11:08:50.109781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:39.158725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119893+0000","last_change":"2023-11-28T11:08:45.944860+0000","last_active":"2023-11-28T11:08:50.119893+0000","last_peered":"2023-11-28T11:08:50.119893+0000","last_clean":"2023-11-28T11:08:50.119893+0000","last_became_active":"2023-11-28T11:08:45.944591+0000","last_became_peered":"2023-11-28T11:08:45.944591+0000","last_unstale":"2023-11-28T11:08:50.119893+0000","last_undegraded":"2023-11-28T11:08:50.119893+0000","last_fullsized":"2023-11-28T11:08:50.119893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4387,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":17,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":8774,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574855,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":703590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65700000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9232,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362608,"statfs":{"total":96636764160,"available":96627310592,"internally_reserved":0,"allocated":860160,"data_stored":699769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26400000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59999999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.249}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73399999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":108440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.627}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59599999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8224,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363616,"statfs":{"total":96636764160,"available":96628342784,"internally_reserved":0,"allocated":286720,"data_stored":113062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64600000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8084,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363756,"statfs":{"total":96636764160,"available":96628486144,"internally_reserved":0,"allocated":274432,"data_stored":110037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54600000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3821,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:09:04.762 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:09:04.762 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-28T11:09:04.762 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-28T11:09:04.763 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-28T11:09:04.763 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-28T11:09:04.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.920+0000 7f3e33bab700 1 Processor -- start 2023-11-28T11:09:04.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.921+0000 7f3e33bab700 1 -- start start 2023-11-28T11:09:04.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.921+0000 7f3e33bab700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e2c15a6c0 0x7f3e2c0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.921+0000 7f3e33bab700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3e2c0b2660 0x7f3e2c0b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.921+0000 7f3e33bab700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e2c0cfb90 con 0x7f3e2c0d0020 2023-11-28T11:09:04.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.921+0000 7f3e33bab700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e2c0b3020 con 0x7f3e2c0b2660 2023-11-28T11:09:04.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.921+0000 7f3e33bab700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c0cb930 con 0x7f3e2c15a6c0 2023-11-28T11:09:04.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.922+0000 7f3e32148700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3e2c0b2660 0x7f3e2c0b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.922+0000 7f3e31146700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3e2c0d0020 0x7f3e2c0ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34972/0 (socket says 172.21.15.28:34972) 2023-11-28T11:09:04.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.922+0000 7f3e31146700 1 -- 172.21.15.28:0/2249370317 learned_addr learned my addr 172.21.15.28:0/2249370317 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:04.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.922+0000 7f3e31947700 1 --2- 172.21.15.28:0/2249370317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e2c15a6c0 0x7f3e2c0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.922+0000 7f3e32148700 1 -- 172.21.15.28:0/2249370317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e2c15a6c0 msgr2=0x7f3e2c0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:04.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.922+0000 7f3e32148700 1 --2- 172.21.15.28:0/2249370317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e2c15a6c0 0x7f3e2c0cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.922+0000 7f3e32148700 1 -- 172.21.15.28:0/2249370317 >> v1:172.21.15.28:6789/0 conn(0x7f3e2c0d0020 legacy=0x7f3e2c0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.922+0000 7f3e30945700 1 -- 172.21.15.28:0/2249370317 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3950944412 0 0) 0x7f3e2c0cfb90 con 0x7f3e2c0d0020 2023-11-28T11:09:04.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.922+0000 7f3e32148700 1 -- 172.21.15.28:0/2249370317 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e2c0cf260 con 0x7f3e2c0b2660 2023-11-28T11:09:04.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.922+0000 7f3e32148700 1 --2- 172.21.15.28:0/2249370317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c0b2660 0x7f3e2c0b2ae0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009d90 tx=0x7f3e2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bafbdee7efe1f40d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:04.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.922+0000 7f3e30945700 1 -- 172.21.15.28:0/2249370317 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2400e070 con 0x7f3e2c0b2660 2023-11-28T11:09:04.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.922+0000 7f3e30945700 1 -- 172.21.15.28:0/2249370317 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2400bb20 con 0x7f3e2c0b2660 2023-11-28T11:09:04.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.922+0000 7f3e30945700 1 -- 172.21.15.28:0/2249370317 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e24012430 con 0x7f3e2c0b2660 2023-11-28T11:09:04.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.923+0000 7f3e33bab700 1 -- 172.21.15.28:0/2249370317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c0b2660 msgr2=0x7f3e2c0b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.923+0000 7f3e33bab700 1 --2- 172.21.15.28:0/2249370317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c0b2660 0x7f3e2c0b2ae0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f3e24009d90 tx=0x7f3e2400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:04.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.923+0000 7f3e33bab700 1 -- 172.21.15.28:0/2249370317 shutdown_connections 2023-11-28T11:09:04.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.923+0000 7f3e33bab700 1 --2- 172.21.15.28:0/2249370317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c0b2660 0x7f3e2c0b2ae0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.923+0000 7f3e33bab700 1 --2- 172.21.15.28:0/2249370317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e2c15a6c0 0x7f3e2c0cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.923+0000 7f3e33bab700 1 -- 172.21.15.28:0/2249370317 >> 172.21.15.28:0/2249370317 conn(0x7f3e2c0cd660 msgr2=0x7f3e2c0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:04.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.923+0000 7f3e33bab700 1 -- 172.21.15.28:0/2249370317 shutdown_connections 2023-11-28T11:09:04.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.923+0000 7f3e33bab700 1 -- 172.21.15.28:0/2249370317 wait complete. 2023-11-28T11:09:04.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.925+0000 7f3e33bab700 1 Processor -- start 2023-11-28T11:09:04.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.925+0000 7f3e33bab700 1 -- start start 2023-11-28T11:09:04.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.925+0000 7f3e33bab700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e2c0b2660 0x7f3e2c15be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e33bab700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c0d0020 0x7f3e2c151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e33bab700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e2c15a6c0 0x7f3e2c15d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e33bab700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c15c790 con 0x7f3e2c0d0020 2023-11-28T11:09:04.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e33bab700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c0cab50 con 0x7f3e2c0b2660 2023-11-28T11:09:04.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e33bab700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2c0cae30 con 0x7f3e2c15a6c0 2023-11-28T11:09:04.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e31146700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c0d0020 0x7f3e2c151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e31146700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c0d0020 0x7f3e2c151200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46920/0 (socket says 172.21.15.28:46920) 2023-11-28T11:09:04.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e31146700 1 -- 172.21.15.28:0/1243601542 learned_addr learned my addr 172.21.15.28:0/1243601542 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:04.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e31947700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e2c0b2660 0x7f3e2c15be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e32148700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e2c15a6c0 0x7f3e2c15d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e31146700 1 -- 172.21.15.28:0/1243601542 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e2c15a6c0 msgr2=0x7f3e2c15d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e31146700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e2c15a6c0 0x7f3e2c15d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e31146700 1 -- 172.21.15.28:0/1243601542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e2c0b2660 msgr2=0x7f3e2c15be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:04.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e31146700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e2c0b2660 0x7f3e2c15be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:04.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e31146700 1 -- 172.21.15.28:0/1243601542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e24009a70 con 0x7f3e2c0d0020 2023-11-28T11:09:04.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e32148700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e2c15a6c0 0x7f3e2c15d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:04.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e31947700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e2c0b2660 0x7f3e2c15be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:04.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e31146700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c0d0020 0x7f3e2c151200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e200011e0 tx=0x7f3e2000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:04.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e1e7fc700 1 -- 172.21.15.28:0/1243601542 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20004c80 con 0x7f3e2c0d0020 2023-11-28T11:09:04.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e1e7fc700 1 -- 172.21.15.28:0/1243601542 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e20004e00 con 0x7f3e2c0d0020 2023-11-28T11:09:04.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e1e7fc700 1 -- 172.21.15.28:0/1243601542 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e20007310 con 0x7f3e2c0d0020 2023-11-28T11:09:04.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e33bab700 1 -- 172.21.15.28:0/1243601542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2c0cf820 con 0x7f3e2c0d0020 2023-11-28T11:09:04.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.926+0000 7f3e33bab700 1 -- 172.21.15.28:0/1243601542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2c0cf540 con 0x7f3e2c0d0020 2023-11-28T11:09:04.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.927+0000 7f3e33bab700 1 -- 172.21.15.28:0/1243601542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e2c0d0470 con 0x7f3e2c0d0020 2023-11-28T11:09:04.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.928+0000 7f3e1e7fc700 1 -- 172.21.15.28:0/1243601542 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3e2001a020 con 0x7f3e2c0d0020 2023-11-28T11:09:04.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.928+0000 7f3e1e7fc700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3e100682f0 0x7f3e1006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:04.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.928+0000 7f3e1e7fc700 1 -- 172.21.15.28:0/1243601542 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3e20085bc0 con 0x7f3e2c0d0020 2023-11-28T11:09:04.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.928+0000 7f3e31947700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3e100682f0 0x7f3e1006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:04.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.929+0000 7f3e31947700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3e100682f0 0x7f3e1006a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009c30 tx=0x7f3e28008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:04.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:04.933+0000 7f3e1e7fc700 1 -- 172.21.15.28:0/1243601542 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3e200521e0 con 0x7f3e2c0d0020 2023-11-28T11:09:05.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.180+0000 7f3e33bab700 1 -- 172.21.15.28:0/1243601542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3e2c0675a0 con 0x7f3e2c0d0020 2023-11-28T11:09:05.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.183+0000 7f3e1e7fc700 1 -- 172.21.15.28:0/1243601542 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+166378 (secure 0 0 0) 0x7f3e20055800 con 0x7f3e2c0d0020 2023-11-28T11:09:05.188 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:05.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.186+0000 7f3e33bab700 1 -- 172.21.15.28:0/1243601542 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3e100682f0 msgr2=0x7f3e1006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.186+0000 7f3e33bab700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3e100682f0 0x7f3e1006a7a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f3e28009c30 tx=0x7f3e28008040 comp rx=0 tx=0).stop 2023-11-28T11:09:05.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.186+0000 7f3e33bab700 1 -- 172.21.15.28:0/1243601542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c0d0020 msgr2=0x7f3e2c151200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.186+0000 7f3e33bab700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c0d0020 0x7f3e2c151200 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f3e200011e0 tx=0x7f3e2000cae0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.186+0000 7f3e33bab700 1 -- 172.21.15.28:0/1243601542 shutdown_connections 2023-11-28T11:09:05.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.186+0000 7f3e33bab700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3e100682f0 0x7f3e1006a7a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.186+0000 7f3e33bab700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e2c15a6c0 0x7f3e2c15d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.186+0000 7f3e33bab700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e2c0b2660 0x7f3e2c15be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.186+0000 7f3e33bab700 1 --2- 172.21.15.28:0/1243601542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e2c0d0020 0x7f3e2c151200 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.186+0000 7f3e33bab700 1 -- 172.21.15.28:0/1243601542 >> 172.21.15.28:0/1243601542 conn(0x7f3e2c0cd660 msgr2=0x7f3e2c0cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:05.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.186+0000 7f3e33bab700 1 -- 172.21.15.28:0/1243601542 shutdown_connections 2023-11-28T11:09:05.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.186+0000 7f3e33bab700 1 -- 172.21.15.28:0/1243601542 wait complete. 2023-11-28T11:09:05.199 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":5,"active_gid":4098,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6832","nonce":103239},{"type":"v1","addr":"172.21.15.28:6833","nonce":103239}]},"active_addr":"172.21.15.28:6833/103239","active_change":"2023-11-28T11:08:33.154422+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":3220807973}]},{"addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":1483797844}]},{"addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":876222447}]},{"addrvec":[{"type":"v2","addr":"172.21.15.28:0","nonce":1757863723}]}]} 2023-11-28T11:09:05.202 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-28T11:09:05.202 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-28T11:09:05.202 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:05.329 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.329+0000 7fbcd2862700 1 Processor -- start 2023-11-28T11:09:05.329 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.329+0000 7fbcd2862700 1 -- start start 2023-11-28T11:09:05.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.329+0000 7fbcd2862700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbccc15c640 0x7fbccc0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.329+0000 7fbcd2862700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbccc0d2820 0x7fbccc0b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.329+0000 7fbcd2862700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbccc0d1470 con 0x7fbccc06bf30 2023-11-28T11:09:05.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.329+0000 7fbcd2862700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbccc15ca10 con 0x7fbccc0d2820 2023-11-28T11:09:05.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.329+0000 7fbcd2862700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0cd4e0 con 0x7fbccc15c640 2023-11-28T11:09:05.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbcd0dff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbccc06bf30 0x7fbccc06c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:34988/0 (socket says 172.21.15.28:34988) 2023-11-28T11:09:05.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbcd0dff700 1 -- 172.21.15.28:0/742227609 learned_addr learned my addr 172.21.15.28:0/742227609 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:05.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccbfff700 1 --2- 172.21.15.28:0/742227609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbccc15c640 0x7fbccc0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccb7fe700 1 --2- 172.21.15.28:0/742227609 >> v2:172.21.15.59:3300/0 conn(0x7fbccc0d2820 0x7fbccc0b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 -- 172.21.15.28:0/742227609 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181877471 0 0) 0x7fbccc0d1470 con 0x7fbccc06bf30 2023-11-28T11:09:05.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 -- 172.21.15.28:0/742227609 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbcb40035b0 con 0x7fbccc06bf30 2023-11-28T11:09:05.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 -- 172.21.15.28:0/742227609 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3049614105 0 0) 0x7fbcb40035b0 con 0x7fbccc06bf30 2023-11-28T11:09:05.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 -- 172.21.15.28:0/742227609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbccc15c640 msgr2=0x7fbccc0cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:05.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 --2- 172.21.15.28:0/742227609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbccc15c640 0x7fbccc0cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 -- 172.21.15.28:0/742227609 >> v2:172.21.15.59:3300/0 conn(0x7fbccc0d2820 msgr2=0x7fbccc0b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:05.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 --2- 172.21.15.28:0/742227609 >> v2:172.21.15.59:3300/0 conn(0x7fbccc0d2820 0x7fbccc0b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 -- 172.21.15.28:0/742227609 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc0cd7c0 con 0x7fbccc06bf30 2023-11-28T11:09:05.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 -- 172.21.15.28:0/742227609 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fbcc40019f0 con 0x7fbccc06bf30 2023-11-28T11:09:05.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 -- 172.21.15.28:0/742227609 >> v1:172.21.15.28:6789/0 conn(0x7fbccc06bf30 legacy=0x7fbccc06c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 --2- 172.21.15.28:0/742227609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcb4003a00 0x7fbcb4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 --2- 172.21.15.28:0/742227609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbccc15c640 0x7fbcb4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 --2- 172.21.15.28:0/742227609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcb4004d90 0x7fbcb4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 -- 172.21.15.28:0/742227609 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc15ca10 con 0x7fbcb4004d90 2023-11-28T11:09:05.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 -- 172.21.15.28:0/742227609 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0cd4e0 con 0x7fbcb4003a00 2023-11-28T11:09:05.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccaffd700 1 -- 172.21.15.28:0/742227609 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcb400ba10 con 0x7fbccc15c640 2023-11-28T11:09:05.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbccb7fe700 1 --2- 172.21.15.28:0/742227609 >> v2:172.21.15.59:3300/0 conn(0x7fbccc0d2820 0x7fbccc0b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:05.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.330+0000 7fbcd0dff700 1 --2- 172.21.15.28:0/742227609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcb4004d90 0x7fbcb4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.331+0000 7fbccbfff700 1 --2- 172.21.15.28:0/742227609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcb4003a00 0x7fbcb4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.331+0000 7fbccb7fe700 1 --2- 172.21.15.28:0/742227609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbccc15c640 0x7fbcb4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.331+0000 7fbcd0dff700 1 -- 172.21.15.28:0/742227609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbccc15c640 msgr2=0x7fbcb4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:05.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.331+0000 7fbcd0dff700 1 --2- 172.21.15.28:0/742227609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbccc15c640 0x7fbcb4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.331+0000 7fbcd0dff700 1 -- 172.21.15.28:0/742227609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcb4003a00 msgr2=0x7fbcb4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:05.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.331+0000 7fbcd0dff700 1 --2- 172.21.15.28:0/742227609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbcb4003a00 0x7fbcb4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.331+0000 7fbcd0dff700 1 -- 172.21.15.28:0/742227609 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb40035b0 con 0x7fbcb4004d90 2023-11-28T11:09:05.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.331+0000 7fbcd0dff700 1 --2- 172.21.15.28:0/742227609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcb4004d90 0x7fbcb4005200 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400a860 tx=0x7fbcb4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=686ae644e2bdd40 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:05.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.331+0000 7fbccaffd700 1 -- 172.21.15.28:0/742227609 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4012600 con 0x7fbcb4004d90 2023-11-28T11:09:05.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.331+0000 7fbccaffd700 1 -- 172.21.15.28:0/742227609 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc4012780 con 0x7fbcb4004d90 2023-11-28T11:09:05.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.331+0000 7fbccaffd700 1 -- 172.21.15.28:0/742227609 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4012b30 con 0x7fbcb4004d90 2023-11-28T11:09:05.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.332+0000 7fbcd2862700 1 -- 172.21.15.28:0/742227609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcb4004d90 msgr2=0x7fbcb4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.332+0000 7fbcd2862700 1 --2- 172.21.15.28:0/742227609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbcb4004d90 0x7fbcb4005200 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400a860 tx=0x7fbcb4005880 comp rx=0 tx=0).stop 2023-11-28T11:09:05.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.332+0000 7fbcd0dff700 1 -- 172.21.15.28:0/742227609 reap_dead start 2023-11-28T11:09:05.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.332+0000 7fbcd2862700 1 -- 172.21.15.28:0/742227609 shutdown_connections 2023-11-28T11:09:05.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.332+0000 7fbcd2862700 1 -- 172.21.15.28:0/742227609 >> 172.21.15.28:0/742227609 conn(0x7fbccc0d2df0 msgr2=0x7fbccc0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:05.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.332+0000 7fbcd2862700 1 -- 172.21.15.28:0/742227609 shutdown_connections 2023-11-28T11:09:05.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.332+0000 7fbcd2862700 1 -- 172.21.15.28:0/742227609 wait complete. 2023-11-28T11:09:05.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.333+0000 7fbcd2862700 1 Processor -- start 2023-11-28T11:09:05.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.333+0000 7fbcd2862700 1 -- start start 2023-11-28T11:09:05.340 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.334+0000 7fbcd2862700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbccc14c220 0x7fbccc1e18a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.340 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.334+0000 7fbcd2862700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbccc0c7fd0 0x7fbccc0cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.340 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.334+0000 7fbcd2862700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbccc0c7620 0x7fbccc0cc410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.340 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.334+0000 7fbcd2862700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0c68f0 con 0x7fbccc14c220 2023-11-28T11:09:05.340 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.334+0000 7fbcd2862700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0c7190 con 0x7fbccc0c7620 2023-11-28T11:09:05.340 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.334+0000 7fbcd2862700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbccc0c8490 con 0x7fbccc0c7fd0 2023-11-28T11:09:05.340 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.334+0000 7fbccb7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbccc0c7fd0 0x7fbccc0cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.341 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.334+0000 7fbccbfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbccc14c220 0x7fbccc1e18a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.341 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.334+0000 7fbcd0dff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbccc0c7620 0x7fbccc0cc410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.341 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.334+0000 7fbccb7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbccc0c7fd0 0x7fbccc0cbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:52698/0 (socket says 172.21.15.28:52698) 2023-11-28T11:09:05.341 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.334+0000 7fbccbfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbccc14c220 0x7fbccc1e18a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46938/0 (socket says 172.21.15.28:46938) 2023-11-28T11:09:05.341 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.334+0000 7fbccb7fe700 1 -- 172.21.15.28:0/4047767936 learned_addr learned my addr 172.21.15.28:0/4047767936 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:05.341 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.334+0000 7fbcd0dff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbccc0c7620 0x7fbccc0cc410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:34232/0 (socket says 172.21.15.28:34232) 2023-11-28T11:09:05.341 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.335+0000 7fbccbfff700 1 -- 172.21.15.28:0/4047767936 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbccc0c7fd0 msgr2=0x7fbccc0cbed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.341 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.335+0000 7fbccbfff700 1 --2- 172.21.15.28:0/4047767936 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbccc0c7fd0 0x7fbccc0cbed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.341 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.335+0000 7fbccbfff700 1 -- 172.21.15.28:0/4047767936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbccc0c7620 msgr2=0x7fbccc0cc410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.341 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.335+0000 7fbccbfff700 1 --2- 172.21.15.28:0/4047767936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbccc0c7620 0x7fbccc0cc410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.341 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.335+0000 7fbccbfff700 1 -- 172.21.15.28:0/4047767936 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc0cdaa0 con 0x7fbccc14c220 2023-11-28T11:09:05.341 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.335+0000 7fbccbfff700 1 --2- 172.21.15.28:0/4047767936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbccc14c220 0x7fbccc1e18a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc011990 tx=0x7fbcbc011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:05.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.335+0000 7fbcc8ff9700 1 -- 172.21.15.28:0/4047767936 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc00f6d0 con 0x7fbccc14c220 2023-11-28T11:09:05.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.335+0000 7fbcc8ff9700 1 -- 172.21.15.28:0/4047767936 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcbc00f850 con 0x7fbccc14c220 2023-11-28T11:09:05.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.335+0000 7fbcd2862700 1 -- 172.21.15.28:0/4047767936 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbccc0d1710 con 0x7fbccc14c220 2023-11-28T11:09:05.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.335+0000 7fbcc8ff9700 1 -- 172.21.15.28:0/4047767936 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcbc00fc00 con 0x7fbccc14c220 2023-11-28T11:09:05.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.335+0000 7fbcd2862700 1 -- 172.21.15.28:0/4047767936 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbccc0cdc30 con 0x7fbccc14c220 2023-11-28T11:09:05.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.336+0000 7fbcd2862700 1 -- 172.21.15.28:0/4047767936 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbccc068570 con 0x7fbccc14c220 2023-11-28T11:09:05.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.336+0000 7fbcc8ff9700 1 -- 172.21.15.28:0/4047767936 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbcbc00fd60 con 0x7fbccc14c220 2023-11-28T11:09:05.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.337+0000 7fbcc8ff9700 1 --2- 172.21.15.28:0/4047767936 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbcb80682f0 0x7fbcb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.337+0000 7fbcc8ff9700 1 -- 172.21.15.28:0/4047767936 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbcbc086690 con 0x7fbccc14c220 2023-11-28T11:09:05.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.337+0000 7fbccb7fe700 1 --2- 172.21.15.28:0/4047767936 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbcb80682f0 0x7fbcb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.337+0000 7fbccb7fe700 1 --2- 172.21.15.28:0/4047767936 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbcb80682f0 0x7fbcb806a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0c83c0 tx=0x7fbcc0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:05.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.341+0000 7fbcc8ff9700 1 -- 172.21.15.28:0/4047767936 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbcbc052c00 con 0x7fbccc14c220 2023-11-28T11:09:05.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.537+0000 7fbcd2862700 1 -- 172.21.15.28:0/4047767936 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbccc0c9c10 con 0x7fbccc14c220 2023-11-28T11:09:05.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.539+0000 7fbcc8ff9700 1 -- 172.21.15.28:0/4047767936 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19642 (secure 0 0 0) 0x7fbcbc056220 con 0x7fbccc14c220 2023-11-28T11:09:05.540 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:05.540 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:50.531733+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:05.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.541+0000 7fbcd2862700 1 -- 172.21.15.28:0/4047767936 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbcb80682f0 msgr2=0x7fbcb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.541+0000 7fbcd2862700 1 --2- 172.21.15.28:0/4047767936 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbcb80682f0 0x7fbcb806a7a0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbccc0c83c0 tx=0x7fbcc0008040 comp rx=0 tx=0).stop 2023-11-28T11:09:05.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.541+0000 7fbcd2862700 1 -- 172.21.15.28:0/4047767936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbccc14c220 msgr2=0x7fbccc1e18a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.541+0000 7fbcd2862700 1 --2- 172.21.15.28:0/4047767936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbccc14c220 0x7fbccc1e18a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbcbc011990 tx=0x7fbcbc011ca0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.541+0000 7fbcd2862700 1 -- 172.21.15.28:0/4047767936 shutdown_connections 2023-11-28T11:09:05.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.541+0000 7fbcd2862700 1 --2- 172.21.15.28:0/4047767936 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbcb80682f0 0x7fbcb806a7a0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.541+0000 7fbcd2862700 1 --2- 172.21.15.28:0/4047767936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbccc14c220 0x7fbccc1e18a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.541+0000 7fbcd2862700 1 --2- 172.21.15.28:0/4047767936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbccc0c7620 0x7fbccc0cc410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.541+0000 7fbcd2862700 1 --2- 172.21.15.28:0/4047767936 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbccc0c7fd0 0x7fbccc0cbed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.541+0000 7fbcd2862700 1 -- 172.21.15.28:0/4047767936 >> 172.21.15.28:0/4047767936 conn(0x7fbccc0d2df0 msgr2=0x7fbccc0d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:05.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.541+0000 7fbcd2862700 1 -- 172.21.15.28:0/4047767936 shutdown_connections 2023-11-28T11:09:05.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.541+0000 7fbcd2862700 1 -- 172.21.15.28:0/4047767936 wait complete. 2023-11-28T11:09:05.554 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-28T11:09:05.554 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:05.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.678+0000 7fd6d5421700 1 Processor -- start 2023-11-28T11:09:05.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.678+0000 7fd6d5421700 1 -- start start 2023-11-28T11:09:05.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.679+0000 7fd6d5421700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6d00d2820 0x7fd6d00b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.679+0000 7fd6d5421700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd6d006bf30 0x7fd6d006c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.679+0000 7fd6d5421700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6d006c890 con 0x7fd6d015c640 2023-11-28T11:09:05.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.679+0000 7fd6d5421700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6d015ca10 con 0x7fd6d006bf30 2023-11-28T11:09:05.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.679+0000 7fd6d5421700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d00d1470 con 0x7fd6d00d2820 2023-11-28T11:09:05.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.679+0000 7fd6ce7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6d00d2820 0x7fd6d00b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.679+0000 7fd6cf7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd6d006bf30 0x7fd6d006c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.679+0000 7fd6ce7fc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6d00d2820 0x7fd6d00b5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:52704/0 (socket says 172.21.15.28:52704) 2023-11-28T11:09:05.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.679+0000 7fd6ce7fc700 1 -- 172.21.15.28:0/1212281535 learned_addr learned my addr 172.21.15.28:0/1212281535 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:05.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.679+0000 7fd6ceffd700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fd6d015c640 0x7fd6d00cf040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:45790/0 (socket says 172.21.15.28:45790) 2023-11-28T11:09:05.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.679+0000 7fd6cf7fe700 1 -- 172.21.15.28:0/1212281535 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6d00d2820 msgr2=0x7fd6d00b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:05.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.679+0000 7fd6cf7fe700 1 --2- 172.21.15.28:0/1212281535 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6d00d2820 0x7fd6d00b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.679+0000 7fd6cf7fe700 1 -- 172.21.15.28:0/1212281535 >> v1:172.21.15.59:6789/0 conn(0x7fd6d015c640 legacy=0x7fd6d00cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.679+0000 7fd6cf7fe700 1 -- 172.21.15.28:0/1212281535 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d00cd4e0 con 0x7fd6d006bf30 2023-11-28T11:09:05.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.680+0000 7fd6cf7fe700 1 --2- 172.21.15.28:0/1212281535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d006bf30 0x7fd6d006c350 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009d90 tx=0x7fd6c800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e22307e9cb308017 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:05.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.680+0000 7fd6cdffb700 1 -- 172.21.15.28:0/1212281535 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c800e070 con 0x7fd6d006bf30 2023-11-28T11:09:05.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.680+0000 7fd6cdffb700 1 -- 172.21.15.28:0/1212281535 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c800bb20 con 0x7fd6d006bf30 2023-11-28T11:09:05.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.680+0000 7fd6cdffb700 1 -- 172.21.15.28:0/1212281535 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c8012430 con 0x7fd6d006bf30 2023-11-28T11:09:05.684 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.680+0000 7fd6d5421700 1 -- 172.21.15.28:0/1212281535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d006bf30 msgr2=0x7fd6d006c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.684 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.680+0000 7fd6d5421700 1 --2- 172.21.15.28:0/1212281535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d006bf30 0x7fd6d006c350 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd6c8009d90 tx=0x7fd6c800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:05.684 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.680+0000 7fd6d5421700 1 -- 172.21.15.28:0/1212281535 shutdown_connections 2023-11-28T11:09:05.684 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.680+0000 7fd6d5421700 1 --2- 172.21.15.28:0/1212281535 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d006bf30 0x7fd6d006c350 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.684 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.680+0000 7fd6d5421700 1 --2- 172.21.15.28:0/1212281535 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6d00d2820 0x7fd6d00b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.684 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.680+0000 7fd6d5421700 1 -- 172.21.15.28:0/1212281535 >> 172.21.15.28:0/1212281535 conn(0x7fd6d00d2df0 msgr2=0x7fd6d00d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:05.684 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.680+0000 7fd6d5421700 1 -- 172.21.15.28:0/1212281535 shutdown_connections 2023-11-28T11:09:05.684 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.680+0000 7fd6d5421700 1 -- 172.21.15.28:0/1212281535 wait complete. 2023-11-28T11:09:05.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.682+0000 7fd6d5421700 1 Processor -- start 2023-11-28T11:09:05.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.682+0000 7fd6d5421700 1 -- start start 2023-11-28T11:09:05.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.682+0000 7fd6d5421700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d006bf30 0x7fd6d00c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.682+0000 7fd6d5421700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d00d2820 0x7fd6d0158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.682+0000 7fd6d5421700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6d015c640 0x7fd6d00c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.682+0000 7fd6d5421700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d00c8270 con 0x7fd6d00d2820 2023-11-28T11:09:05.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.682+0000 7fd6d5421700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d00c8550 con 0x7fd6d006bf30 2023-11-28T11:09:05.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.682+0000 7fd6d5421700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d00c8830 con 0x7fd6d015c640 2023-11-28T11:09:05.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6ce7fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d00d2820 0x7fd6d0158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6ce7fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d00d2820 0x7fd6d0158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46954/0 (socket says 172.21.15.28:46954) 2023-11-28T11:09:05.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6ce7fc700 1 -- 172.21.15.28:0/2863785005 learned_addr learned my addr 172.21.15.28:0/2863785005 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:05.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6ceffd700 1 --2- 172.21.15.28:0/2863785005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d006bf30 0x7fd6d00c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6cf7fe700 1 --2- 172.21.15.28:0/2863785005 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6d015c640 0x7fd6d00c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6ce7fc700 1 -- 172.21.15.28:0/2863785005 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6d015c640 msgr2=0x7fd6d00c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6ce7fc700 1 --2- 172.21.15.28:0/2863785005 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6d015c640 0x7fd6d00c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6ce7fc700 1 -- 172.21.15.28:0/2863785005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d006bf30 msgr2=0x7fd6d00c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6ce7fc700 1 --2- 172.21.15.28:0/2863785005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d006bf30 0x7fd6d00c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6ce7fc700 1 -- 172.21.15.28:0/2863785005 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c8009a70 con 0x7fd6d00d2820 2023-11-28T11:09:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6ceffd700 1 --2- 172.21.15.28:0/2863785005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d006bf30 0x7fd6d00c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6ce7fc700 1 --2- 172.21.15.28:0/2863785005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d00d2820 0x7fd6d0158690 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000e910 tx=0x7fd6c000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6b77fe700 1 -- 172.21.15.28:0/2863785005 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c00096a0 con 0x7fd6d00d2820 2023-11-28T11:09:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6b77fe700 1 -- 172.21.15.28:0/2863785005 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c0013680 con 0x7fd6d00d2820 2023-11-28T11:09:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6d5421700 1 -- 172.21.15.28:0/2863785005 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d00d1710 con 0x7fd6d00d2820 2023-11-28T11:09:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6b77fe700 1 -- 172.21.15.28:0/2863785005 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c0013a30 con 0x7fd6d00d2820 2023-11-28T11:09:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.683+0000 7fd6d5421700 1 -- 172.21.15.28:0/2863785005 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d00cd7c0 con 0x7fd6d00d2820 2023-11-28T11:09:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.684+0000 7fd6d5421700 1 -- 172.21.15.28:0/2863785005 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d00cdaa0 con 0x7fd6d00d2820 2023-11-28T11:09:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.685+0000 7fd6b77fe700 1 -- 172.21.15.28:0/2863785005 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd6c0019030 con 0x7fd6d00d2820 2023-11-28T11:09:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.685+0000 7fd6b77fe700 1 --2- 172.21.15.28:0/2863785005 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd6bc0682f0 0x7fd6bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.685+0000 7fd6b77fe700 1 -- 172.21.15.28:0/2863785005 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd6c0085d20 con 0x7fd6d00d2820 2023-11-28T11:09:05.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.685+0000 7fd6ceffd700 1 --2- 172.21.15.28:0/2863785005 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd6bc0682f0 0x7fd6bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:05.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.686+0000 7fd6ceffd700 1 --2- 172.21.15.28:0/2863785005 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd6bc0682f0 0x7fd6bc06a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4000f60 tx=0x7fd6c4007ed0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:05.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.690+0000 7fd6b77fe700 1 -- 172.21.15.28:0/2863785005 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd6c0052290 con 0x7fd6d00d2820 2023-11-28T11:09:05.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.883+0000 7fd6d5421700 1 -- 172.21.15.28:0/2863785005 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6d00ce3e0 con 0x7fd6d00d2820 2023-11-28T11:09:05.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.885+0000 7fd6b77fe700 1 -- 172.21.15.28:0/2863785005 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19642 (secure 0 0 0) 0x7fd6c00558b0 con 0x7fd6d00d2820 2023-11-28T11:09:05.886 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:05.886 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:50.531733+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:05.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.887+0000 7fd6d5421700 1 -- 172.21.15.28:0/2863785005 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd6bc0682f0 msgr2=0x7fd6bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.887+0000 7fd6d5421700 1 --2- 172.21.15.28:0/2863785005 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd6bc0682f0 0x7fd6bc06a7a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd6c4000f60 tx=0x7fd6c4007ed0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.887+0000 7fd6d5421700 1 -- 172.21.15.28:0/2863785005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d00d2820 msgr2=0x7fd6d0158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:05.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.887+0000 7fd6d5421700 1 --2- 172.21.15.28:0/2863785005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d00d2820 0x7fd6d0158690 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000e910 tx=0x7fd6c000ec20 comp rx=0 tx=0).stop 2023-11-28T11:09:05.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.887+0000 7fd6d5421700 1 -- 172.21.15.28:0/2863785005 shutdown_connections 2023-11-28T11:09:05.891 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.887+0000 7fd6d5421700 1 --2- 172.21.15.28:0/2863785005 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd6bc0682f0 0x7fd6bc06a7a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.891 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.887+0000 7fd6d5421700 1 --2- 172.21.15.28:0/2863785005 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd6d015c640 0x7fd6d00c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.891 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.887+0000 7fd6d5421700 1 --2- 172.21.15.28:0/2863785005 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd6d006bf30 0x7fd6d00c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.892 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.887+0000 7fd6d5421700 1 --2- 172.21.15.28:0/2863785005 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd6d00d2820 0x7fd6d0158690 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:05.892 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.887+0000 7fd6d5421700 1 -- 172.21.15.28:0/2863785005 >> 172.21.15.28:0/2863785005 conn(0x7fd6d00d2df0 msgr2=0x7fd6d00b2ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:05.892 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.887+0000 7fd6d5421700 1 -- 172.21.15.28:0/2863785005 shutdown_connections 2023-11-28T11:09:05.892 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:05.887+0000 7fd6d5421700 1 -- 172.21.15.28:0/2863785005 wait complete. 2023-11-28T11:09:05.900 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-28T11:09:05.901 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-28T11:09:05.901 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-28T11:09:05.901 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-28T11:09:05.902 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-28T11:09:05.902 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-28T11:09:05.903 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-28T11:09:05.903 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-28T11:09:06.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.120+0000 7f10d7adf700 1 Processor -- start 2023-11-28T11:09:06.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.122+0000 7f0efec1a700 1 Processor -- start 2023-11-28T11:09:06.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.122+0000 7f45665ea700 1 Processor -- start 2023-11-28T11:09:06.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.122+0000 7f9850021700 1 Processor -- start 2023-11-28T11:09:06.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.121+0000 7f10d7adf700 1 -- start start 2023-11-28T11:09:06.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.122+0000 7f10d7adf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10d00c2630 0x7f10d00ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.122+0000 7f10d7adf700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f10d00ce230 0x7f10d006a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.122+0000 7f10d7adf700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10d00cceb0 con 0x7f10d00ce6a0 2023-11-28T11:09:06.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.122+0000 7f10d7adf700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10d006b180 con 0x7f10d00ce230 2023-11-28T11:09:06.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.122+0000 7f10d7adf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00cd190 con 0x7f10d00c2630 2023-11-28T11:09:06.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.122+0000 7f0efec1a700 1 -- start start 2023-11-28T11:09:06.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.123+0000 7f0efec1a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0ef80cd120 0x7f0ef806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.123+0000 7f0efec1a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ef806b860 0x7f0ef80c9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.123+0000 7f0efec1a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ef806a280 con 0x7f0ef81588a0 2023-11-28T11:09:06.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.123+0000 7f0efec1a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ef80ce720 con 0x7f0ef80cd120 2023-11-28T11:09:06.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.123+0000 7f0efec1a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef806a560 con 0x7f0ef806b860 2023-11-28T11:09:06.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f0ef7fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0ef80cd120 0x7f0ef806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f0ef7fff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0ef80cd120 0x7f0ef806b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:34254/0 (socket says 172.21.15.28:34254) 2023-11-28T11:09:06.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f0ef7fff700 1 -- 172.21.15.28:0/3109754004 learned_addr learned my addr 172.21.15.28:0/3109754004 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f0efd1b7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ef806b860 0x7f0ef80c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f0efd1b7700 1 -- 172.21.15.28:0/3109754004 >> v2:172.21.15.59:3300/0 conn(0x7f0ef80cd120 msgr2=0x7f0ef806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f0efd1b7700 1 --2- 172.21.15.28:0/3109754004 >> v2:172.21.15.59:3300/0 conn(0x7f0ef80cd120 0x7f0ef806b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f0efd1b7700 1 -- 172.21.15.28:0/3109754004 >> v1:172.21.15.59:6789/0 conn(0x7f0ef81588a0 legacy=0x7f0ef80ce010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:09:06.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f0efd1b7700 1 -- 172.21.15.28:0/3109754004 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ef806a840 con 0x7f0ef806b860 2023-11-28T11:09:06.129 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.122+0000 7f45665ea700 1 -- start start 2023-11-28T11:09:06.129 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.123+0000 7f45665ea700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f45601588a0 0x7f45600ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.129 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.123+0000 7f45665ea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45600c9920 0x7f45600c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.129 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.123+0000 7f45665ea700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456006a280 con 0x7f45600cd120 2023-11-28T11:09:06.130 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.123+0000 7f45665ea700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45600ca2e0 con 0x7f45601588a0 2023-11-28T11:09:06.130 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.123+0000 7f45665ea700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456006a560 con 0x7f45600c9920 2023-11-28T11:09:06.130 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f455f7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f45600cd120 0x7f456006b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35024/0 (socket says 172.21.15.28:35024) 2023-11-28T11:09:06.130 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f455f7fe700 1 -- 172.21.15.28:0/27914202 learned_addr learned my addr 172.21.15.28:0/27914202 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:06.130 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.125+0000 7f455ffff700 1 --2- 172.21.15.28:0/27914202 >> v2:172.21.15.59:3300/0 conn(0x7f45601588a0 0x7f45600ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.130 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.125+0000 7f455ffff700 1 -- 172.21.15.28:0/27914202 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45600c9920 msgr2=0x7f45600c9da0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:09:06.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.125+0000 7f455ffff700 1 --2- 172.21.15.28:0/27914202 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45600c9920 0x7f45600c9da0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.125+0000 7f455ffff700 1 -- 172.21.15.28:0/27914202 >> v1:172.21.15.28:6789/0 conn(0x7f45600cd120 legacy=0x7f456006b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.125+0000 7f455ffff700 1 -- 172.21.15.28:0/27914202 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456006a840 con 0x7f45601588a0 2023-11-28T11:09:06.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.122+0000 7f9850021700 1 -- start start 2023-11-28T11:09:06.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f9850021700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9848160c70 0x7f98480adb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f9850021700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9848164b30 con 0x7f9848071070 2023-11-28T11:09:06.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f9850021700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9848164e10 con 0x7f98481646c0 2023-11-28T11:09:06.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f9850021700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9848161040 con 0x7f9848160c70 2023-11-28T11:09:06.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f984e5be700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9848071070 0x7f984806cfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35034/0 (socket says 172.21.15.28:35034) 2023-11-28T11:09:06.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f984e5be700 1 -- 172.21.15.28:0/884828714 learned_addr learned my addr 172.21.15.28:0/884828714 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:06.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.125+0000 7f984ddbd700 1 --2- 172.21.15.28:0/884828714 >> v2:172.21.15.28:3300/0 conn(0x7f9848160c70 0x7f98480adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.125+0000 7f984cdbb700 1 -- 172.21.15.28:0/884828714 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2881229767 0 0) 0x7f9848164b30 con 0x7f9848071070 2023-11-28T11:09:06.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.125+0000 7f984cdbb700 1 -- 172.21.15.28:0/884828714 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98340035b0 con 0x7f9848071070 2023-11-28T11:09:06.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.126+0000 7f984ddbd700 1 -- 172.21.15.28:0/884828714 >> v1:172.21.15.59:6789/0 conn(0x7f98481646c0 legacy=0x7f9848070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.126+0000 7f984ddbd700 1 -- 172.21.15.28:0/884828714 >> v1:172.21.15.28:6789/0 conn(0x7f9848071070 legacy=0x7f984806cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.126+0000 7f984ddbd700 1 -- 172.21.15.28:0/884828714 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98481650f0 con 0x7f9848160c70 2023-11-28T11:09:06.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.124+0000 7f10d587b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10d00c2630 0x7f10d00ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.125+0000 7f10d587b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10d00c2630 0x7f10d00ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:52728/0 (socket says 172.21.15.28:52728) 2023-11-28T11:09:06.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.125+0000 7f10d587b700 1 -- 172.21.15.28:0/3427962624 learned_addr learned my addr 172.21.15.28:0/3427962624 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.125+0000 7f10d507a700 1 --2- 172.21.15.28:0/3427962624 >> v2:172.21.15.28:3300/0 conn(0x7f10d00ce230 0x7f10d006a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.125+0000 7f10d587b700 1 -- 172.21.15.28:0/3427962624 >> v2:172.21.15.28:3300/0 conn(0x7f10d00ce230 msgr2=0x7f10d006a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.125+0000 7f10d587b700 1 --2- 172.21.15.28:0/3427962624 >> v2:172.21.15.28:3300/0 conn(0x7f10d00ce230 0x7f10d006a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.125+0000 7f10d587b700 1 -- 172.21.15.28:0/3427962624 >> v1:172.21.15.28:6789/0 conn(0x7f10d00ce6a0 legacy=0x7f10d006d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.125+0000 7f10d587b700 1 -- 172.21.15.28:0/3427962624 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d00cd470 con 0x7f10d00c2630 2023-11-28T11:09:06.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.126+0000 7f10d587b700 1 --2- 172.21.15.28:0/3427962624 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10d00c2630 0x7f10d00ca820 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0098f0 tx=0x7f10cc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2c7b52f9f71c9b31 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.126+0000 7f10d4879700 1 -- 172.21.15.28:0/3427962624 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc00b8a0 con 0x7f10d00c2630 2023-11-28T11:09:06.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.126+0000 7f10d4879700 1 -- 172.21.15.28:0/3427962624 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10cc00ba00 con 0x7f10d00c2630 2023-11-28T11:09:06.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.126+0000 7f10d4879700 1 -- 172.21.15.28:0/3427962624 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc00bd90 con 0x7f10d00c2630 2023-11-28T11:09:06.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.126+0000 7f455ffff700 1 --2- 172.21.15.28:0/27914202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45601588a0 0x7f45600ce010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4558009d90 tx=0x7f455800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=65c2d7369e74c9e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.127+0000 7f984ddbd700 1 --2- 172.21.15.28:0/884828714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848160c70 0x7f98480adb00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9838009840 tx=0x7f9838009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3ef6ad5c6e3cdbe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.127+0000 7f455effd700 1 -- 172.21.15.28:0/27914202 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455800bd90 con 0x7f45601588a0 2023-11-28T11:09:06.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.127+0000 7f455effd700 1 -- 172.21.15.28:0/27914202 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4558009e80 con 0x7f45601588a0 2023-11-28T11:09:06.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.127+0000 7f455effd700 1 -- 172.21.15.28:0/27914202 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558003c60 con 0x7f45601588a0 2023-11-28T11:09:06.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.128+0000 7f45665ea700 1 -- 172.21.15.28:0/27914202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45601588a0 msgr2=0x7f45600ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.128+0000 7f45665ea700 1 --2- 172.21.15.28:0/27914202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45601588a0 0x7f45600ce010 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4558009d90 tx=0x7f455800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:06.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.127+0000 7f10d7adf700 1 -- 172.21.15.28:0/3427962624 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10d00c2630 msgr2=0x7f10d00ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.127+0000 7f10d7adf700 1 --2- 172.21.15.28:0/3427962624 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10d00c2630 0x7f10d00ca820 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f10cc0098f0 tx=0x7f10cc009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:06.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.128+0000 7f10d7adf700 1 -- 172.21.15.28:0/3427962624 shutdown_connections 2023-11-28T11:09:06.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.128+0000 7f10d7adf700 1 --2- 172.21.15.28:0/3427962624 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10d00c2630 0x7f10d00ca820 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.128+0000 7f10d7adf700 1 --2- 172.21.15.28:0/3427962624 >> v2:172.21.15.28:3300/0 conn(0x7f10d00ce230 0x7f10d006a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.128+0000 7f10d7adf700 1 -- 172.21.15.28:0/3427962624 >> 172.21.15.28:0/3427962624 conn(0x7f10d00b0dd0 msgr2=0x7f10d00b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.128+0000 7f10d7adf700 1 -- 172.21.15.28:0/3427962624 shutdown_connections 2023-11-28T11:09:06.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.128+0000 7f10d7adf700 1 -- 172.21.15.28:0/3427962624 wait complete. 2023-11-28T11:09:06.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.128+0000 7f984cdbb700 1 -- 172.21.15.28:0/884828714 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838004230 con 0x7f9848160c70 2023-11-28T11:09:06.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.128+0000 7f984cdbb700 1 -- 172.21.15.28:0/884828714 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9838004390 con 0x7f9848160c70 2023-11-28T11:09:06.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.128+0000 7f984cdbb700 1 -- 172.21.15.28:0/884828714 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98380036c0 con 0x7f9848160c70 2023-11-28T11:09:06.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.129+0000 7f9850021700 1 -- 172.21.15.28:0/884828714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848160c70 msgr2=0x7f98480adb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.129+0000 7f9850021700 1 --2- 172.21.15.28:0/884828714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848160c70 0x7f98480adb00 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f9838009840 tx=0x7f9838009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:06.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.129+0000 7f0efd1b7700 1 --2- 172.21.15.28:0/3109754004 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ef806b860 0x7f0ef80c9920 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009840 tx=0x7f0ee8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bc4f086160251f08 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.129+0000 7f9850021700 1 -- 172.21.15.28:0/884828714 shutdown_connections 2023-11-28T11:09:06.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.129+0000 7f9850021700 1 --2- 172.21.15.28:0/884828714 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848160c70 0x7f98480adb00 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.129+0000 7f9850021700 1 -- 172.21.15.28:0/884828714 >> 172.21.15.28:0/884828714 conn(0x7f98480ce9e0 msgr2=0x7f98480cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.130+0000 7f10d7adf700 1 Processor -- start 2023-11-28T11:09:06.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.130+0000 7f45665ea700 1 -- 172.21.15.28:0/27914202 shutdown_connections 2023-11-28T11:09:06.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.130+0000 7f45665ea700 1 --2- 172.21.15.28:0/27914202 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45600c9920 0x7f45600c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.130+0000 7f45665ea700 1 --2- 172.21.15.28:0/27914202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45601588a0 0x7f45600ce010 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.130+0000 7f45665ea700 1 -- 172.21.15.28:0/27914202 >> 172.21.15.28:0/27914202 conn(0x7f45600cf670 msgr2=0x7f45600cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.130+0000 7f9850021700 1 -- 172.21.15.28:0/884828714 shutdown_connections 2023-11-28T11:09:06.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.130+0000 7f10d7adf700 1 -- start start 2023-11-28T11:09:06.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.131+0000 7f10d7adf700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10d00c2630 0x7f10d00c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.131+0000 7f10d7adf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10d00ce230 0x7f10d00b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.131+0000 7f10d7adf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10d00ce6a0 0x7f10d0161520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.131+0000 7f10d7adf700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00b4530 con 0x7f10d00c2630 2023-11-28T11:09:06.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.131+0000 7f10d7adf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00c63b0 con 0x7f10d00ce230 2023-11-28T11:09:06.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.131+0000 7f10d7adf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10d00b47e0 con 0x7f10d00ce6a0 2023-11-28T11:09:06.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.131+0000 7f10d587b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10d00c2630 0x7f10d00c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.131+0000 7f10d587b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10d00c2630 0x7f10d00c8e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46982/0 (socket says 172.21.15.28:46982) 2023-11-28T11:09:06.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.131+0000 7f10d587b700 1 -- 172.21.15.28:0/1531661739 learned_addr learned my addr 172.21.15.28:0/1531661739 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.131+0000 7f10d507a700 1 --2- 172.21.15.28:0/1531661739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10d00ce230 0x7f10d00b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.131+0000 7f10d607c700 1 --2- 172.21.15.28:0/1531661739 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10d00ce6a0 0x7f10d0161520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.130+0000 7f9850021700 1 -- 172.21.15.28:0/884828714 wait complete. 2023-11-28T11:09:06.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.132+0000 7f9850021700 1 Processor -- start 2023-11-28T11:09:06.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.129+0000 7fc41720c700 1 Processor -- start 2023-11-28T11:09:06.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.129+0000 7fc41720c700 1 -- start start 2023-11-28T11:09:06.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.130+0000 7fc41720c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc410160be0 0x7fc4100c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.130+0000 7fc41720c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc4100adec0 0x7fc41016a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.130+0000 7fc41720c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4100cdb60 con 0x7fc4100c9df0 2023-11-28T11:09:06.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.130+0000 7fc41720c700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4100cde40 con 0x7fc410160be0 2023-11-28T11:09:06.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.130+0000 7fc41720c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4100ce120 con 0x7fc4100adec0 2023-11-28T11:09:06.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.132+0000 7f10d587b700 1 -- 172.21.15.28:0/1531661739 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10d00ce6a0 msgr2=0x7f10d0161520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.132+0000 7f10d587b700 1 --2- 172.21.15.28:0/1531661739 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10d00ce6a0 0x7f10d0161520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.132+0000 7f10d587b700 1 -- 172.21.15.28:0/1531661739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10d00ce230 msgr2=0x7f10d00b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.132+0000 7f10d587b700 1 --2- 172.21.15.28:0/1531661739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10d00ce230 0x7f10d00b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.132+0000 7f10d587b700 1 -- 172.21.15.28:0/1531661739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10cc009550 con 0x7f10d00c2630 2023-11-28T11:09:06.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.132+0000 7fc414fa8700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc410160be0 0x7fc4100c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.132+0000 7fc414fa8700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc410160be0 0x7fc4100c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:46988/0 (socket says 172.21.15.28:46988) 2023-11-28T11:09:06.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.132+0000 7fc414fa8700 1 -- 172.21.15.28:0/352731195 learned_addr learned my addr 172.21.15.28:0/352731195 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.132+0000 7f10d587b700 1 --2- 172.21.15.28:0/1531661739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10d00c2630 0x7f10d00c8e80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009c40 tx=0x7f10cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.133+0000 7fc414fa8700 1 -- 172.21.15.28:0/352731195 >> v2:172.21.15.59:3300/0 conn(0x7fc4100adec0 msgr2=0x7fc41016a580 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:09:06.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.133+0000 7fc414fa8700 1 --2- 172.21.15.28:0/352731195 >> v2:172.21.15.59:3300/0 conn(0x7fc4100adec0 0x7fc41016a580 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.133+0000 7fc414fa8700 1 -- 172.21.15.28:0/352731195 >> v1:172.21.15.59:6789/0 conn(0x7fc4100c9df0 legacy=0x7fc4100ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.133+0000 7fc414fa8700 1 -- 172.21.15.28:0/352731195 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41006a1e0 con 0x7fc410160be0 2023-11-28T11:09:06.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.133+0000 7f10c67fc700 1 -- 172.21.15.28:0/1531661739 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc004450 con 0x7f10d00c2630 2023-11-28T11:09:06.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.133+0000 7f10c67fc700 1 -- 172.21.15.28:0/1531661739 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10cc0045b0 con 0x7f10d00c2630 2023-11-28T11:09:06.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.133+0000 7f10c67fc700 1 -- 172.21.15.28:0/1531661739 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10cc0038e0 con 0x7f10d00c2630 2023-11-28T11:09:06.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.133+0000 7f10d7adf700 1 -- 172.21.15.28:0/1531661739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10d00cda30 con 0x7f10d00c2630 2023-11-28T11:09:06.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.134+0000 7f0ef77fe700 1 -- 172.21.15.28:0/3109754004 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8004230 con 0x7f0ef806b860 2023-11-28T11:09:06.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.134+0000 7f0ef77fe700 1 -- 172.21.15.28:0/3109754004 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ee8004390 con 0x7f0ef806b860 2023-11-28T11:09:06.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.134+0000 7f0ef77fe700 1 -- 172.21.15.28:0/3109754004 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ee80036a0 con 0x7f0ef806b860 2023-11-28T11:09:06.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.134+0000 7f45665ea700 1 -- 172.21.15.28:0/27914202 shutdown_connections 2023-11-28T11:09:06.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.135+0000 7f0efec1a700 1 -- 172.21.15.28:0/3109754004 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ef806b860 msgr2=0x7f0ef80c9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.135+0000 7f0efec1a700 1 --2- 172.21.15.28:0/3109754004 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ef806b860 0x7f0ef80c9920 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0ee8009840 tx=0x7f0ee8009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:06.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.134+0000 7f45665ea700 1 -- 172.21.15.28:0/27914202 wait complete. 2023-11-28T11:09:06.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.134+0000 7f9850021700 1 -- start start 2023-11-28T11:09:06.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.135+0000 7f9850021700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9848071070 0x7f98481f6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.135+0000 7f9850021700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848160c70 0x7f98480b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.135+0000 7f9850021700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98481646c0 0x7f98481f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.135+0000 7f9850021700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98480b1df0 con 0x7f9848160c70 2023-11-28T11:09:06.162 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.135+0000 7f9850021700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98480b2090 con 0x7f98481646c0 2023-11-28T11:09:06.162 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.135+0000 7f9850021700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98480b2370 con 0x7f9848071070 2023-11-28T11:09:06.162 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.135+0000 7fc414fa8700 1 --2- 172.21.15.28:0/352731195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc410160be0 0x7fc4100c98b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc4100707b0 tx=0x7fc40800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b9f416d9a85a8ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.162 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.133+0000 7f10d7adf700 1 -- 172.21.15.28:0/1531661739 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10d00cd750 con 0x7f10d00c2630 2023-11-28T11:09:06.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.135+0000 7fc40f7fe700 1 -- 172.21.15.28:0/352731195 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc40800e070 con 0x7fc410160be0 2023-11-28T11:09:06.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.135+0000 7fc40f7fe700 1 -- 172.21.15.28:0/352731195 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc408003a20 con 0x7fc410160be0 2023-11-28T11:09:06.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.135+0000 7fc40f7fe700 1 -- 172.21.15.28:0/352731195 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc408003dd0 con 0x7fc410160be0 2023-11-28T11:09:06.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7f45665ea700 1 Processor -- start 2023-11-28T11:09:06.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7f984e5be700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98481646c0 0x7f98481f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7f984ddbd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9848071070 0x7f98481f6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7f984e5be700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98481646c0 0x7f98481f5c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:34284/0 (socket says 172.21.15.28:34284) 2023-11-28T11:09:06.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7f984e5be700 1 -- 172.21.15.28:0/4285851174 learned_addr learned my addr 172.21.15.28:0/4285851174 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7f984ddbd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9848071070 0x7f98481f6ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:52754/0 (socket says 172.21.15.28:52754) 2023-11-28T11:09:06.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7fc41720c700 1 -- 172.21.15.28:0/352731195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc410160be0 msgr2=0x7fc4100c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7fc41720c700 1 --2- 172.21.15.28:0/352731195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc410160be0 0x7fc4100c98b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc4100707b0 tx=0x7fc40800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:06.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7f984d5bc700 1 --2- 172.21.15.28:0/4285851174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848160c70 0x7f98480b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7fc41720c700 1 -- 172.21.15.28:0/352731195 shutdown_connections 2023-11-28T11:09:06.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7fc41720c700 1 --2- 172.21.15.28:0/352731195 >> v2:172.21.15.59:3300/0 conn(0x7fc4100adec0 0x7fc41016a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7fc41720c700 1 --2- 172.21.15.28:0/352731195 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc410160be0 0x7fc4100c98b0 secure :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fc4100707b0 tx=0x7fc40800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:06.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7fc41720c700 1 -- 172.21.15.28:0/352731195 >> 172.21.15.28:0/352731195 conn(0x7fc4100cf670 msgr2=0x7fc4100cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7f984ddbd700 1 -- 172.21.15.28:0/4285851174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98481646c0 msgr2=0x7f98481f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7f984ddbd700 1 --2- 172.21.15.28:0/4285851174 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98481646c0 0x7f98481f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7f984ddbd700 1 -- 172.21.15.28:0/4285851174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848160c70 msgr2=0x7f98480b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7f984ddbd700 1 --2- 172.21.15.28:0/4285851174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848160c70 0x7f98480b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7f984ddbd700 1 -- 172.21.15.28:0/4285851174 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98380094a0 con 0x7f9848071070 2023-11-28T11:09:06.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.137+0000 7f984d5bc700 1 --2- 172.21.15.28:0/4285851174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9848160c70 0x7f98480b18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:06.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.137+0000 7f10c67fc700 1 -- 172.21.15.28:0/1531661739 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f10cc02a030 con 0x7f10d00c2630 2023-11-28T11:09:06.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.137+0000 7f984ddbd700 1 --2- 172.21.15.28:0/4285851174 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9848071070 0x7f98481f6ff0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9838013000 tx=0x7f9838004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.136+0000 7f45665ea700 1 -- start start 2023-11-28T11:09:06.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.137+0000 7f983e7fc700 1 -- 172.21.15.28:0/4285851174 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838011630 con 0x7f9848071070 2023-11-28T11:09:06.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.137+0000 7f983e7fc700 1 -- 172.21.15.28:0/4285851174 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9838011790 con 0x7f9848071070 2023-11-28T11:09:06.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.137+0000 7f983e7fc700 1 -- 172.21.15.28:0/4285851174 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9838011b40 con 0x7f9848071070 2023-11-28T11:09:06.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.137+0000 7f9850021700 1 -- 172.21.15.28:0/4285851174 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98480ae040 con 0x7f9848071070 2023-11-28T11:09:06.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.137+0000 7f9850021700 1 -- 172.21.15.28:0/4285851174 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98481653d0 con 0x7f9848071070 2023-11-28T11:09:06.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.137+0000 7f10c67fc700 1 --2- 172.21.15.28:0/1531661739 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f10b40683c0 0x7f10b406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.137+0000 7f10c67fc700 1 -- 172.21.15.28:0/1531661739 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f10cc086f40 con 0x7f10d00c2630 2023-11-28T11:09:06.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.137+0000 7f45665ea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45600c9920 0x7f45600b4390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.137+0000 7f45665ea700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45600cd120 0x7f456016ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f45665ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45601588a0 0x7f456006e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f45665ea700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456016b180 con 0x7f45600cd120 2023-11-28T11:09:06.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f45665ea700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45600b35d0 con 0x7f45601588a0 2023-11-28T11:09:06.172 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f45665ea700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45600b38b0 con 0x7f45600c9920 2023-11-28T11:09:06.172 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f10d7adf700 1 --2- 172.21.15.28:0/1531661739 >> [v2:172.21.15.59:6824/103734,v1:172.21.15.59:6825/103734] conn(0x7f10b80015e0 0x7f10b8003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.172 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f10d7adf700 1 -- 172.21.15.28:0/1531661739 --> [v2:172.21.15.59:6824/103734,v1:172.21.15.59:6825/103734] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f10b8006bc0 con 0x7f10b80015e0 2023-11-28T11:09:06.172 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f455ffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45600c9920 0x7f45600b4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.173 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f4564b87700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45601588a0 0x7f456006e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.173 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f455ffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45600c9920 0x7f45600b4390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:52760/0 (socket says 172.21.15.28:52760) 2023-11-28T11:09:06.173 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f455ffff700 1 -- 172.21.15.28:0/2233327209 learned_addr learned my addr 172.21.15.28:0/2233327209 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.173 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f10d607c700 1 --2- 172.21.15.28:0/1531661739 >> [v2:172.21.15.59:6824/103734,v1:172.21.15.59:6825/103734] conn(0x7f10b80015e0 0x7f10b8003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f10d507a700 1 --2- 172.21.15.28:0/1531661739 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f10b40683c0 0x7f10b406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f0efec1a700 1 -- 172.21.15.28:0/3109754004 shutdown_connections 2023-11-28T11:09:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f0efec1a700 1 --2- 172.21.15.28:0/3109754004 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ef806b860 0x7f0ef80c9920 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f0efec1a700 1 --2- 172.21.15.28:0/3109754004 >> v2:172.21.15.59:3300/0 conn(0x7f0ef80cd120 0x7f0ef806b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f0efec1a700 1 -- 172.21.15.28:0/3109754004 >> 172.21.15.28:0/3109754004 conn(0x7f0ef80cf670 msgr2=0x7f0ef80cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7fc41720c700 1 -- 172.21.15.28:0/352731195 shutdown_connections 2023-11-28T11:09:06.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f0efec1a700 1 -- 172.21.15.28:0/3109754004 shutdown_connections 2023-11-28T11:09:06.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f10d607c700 1 --2- 172.21.15.28:0/1531661739 >> [v2:172.21.15.59:6824/103734,v1:172.21.15.59:6825/103734] conn(0x7f10b80015e0 0x7f10b8003a90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.139+0000 7f9850021700 1 -- 172.21.15.28:0/4285851174 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9830000fc0 con 0x7f9848071070 2023-11-28T11:09:06.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f455ffff700 1 -- 172.21.15.28:0/2233327209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45601588a0 msgr2=0x7f456006e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f455ffff700 1 --2- 172.21.15.28:0/2233327209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45601588a0 0x7f456006e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f455ffff700 1 -- 172.21.15.28:0/2233327209 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45600cd120 msgr2=0x7f456016ac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f455ffff700 1 --2- 172.21.15.28:0/2233327209 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f45600cd120 0x7f456016ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f455ffff700 1 -- 172.21.15.28:0/2233327209 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4558009a70 con 0x7f45600c9920 2023-11-28T11:09:06.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.139+0000 7f455ffff700 1 --2- 172.21.15.28:0/2233327209 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45600c9920 0x7f45600b4390 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4558009d60 tx=0x7f4558004ad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.139+0000 7f455cff9700 1 -- 172.21.15.28:0/2233327209 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455801e070 con 0x7f45600c9920 2023-11-28T11:09:06.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.139+0000 7f455cff9700 1 -- 172.21.15.28:0/2233327209 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4558004710 con 0x7f45600c9920 2023-11-28T11:09:06.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.139+0000 7f455cff9700 1 -- 172.21.15.28:0/2233327209 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4558010480 con 0x7f45600c9920 2023-11-28T11:09:06.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.139+0000 7f45665ea700 1 -- 172.21.15.28:0/2233327209 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456006ae00 con 0x7f45600c9920 2023-11-28T11:09:06.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.139+0000 7f983e7fc700 1 -- 172.21.15.28:0/4285851174 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f983802d070 con 0x7f9848071070 2023-11-28T11:09:06.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.139+0000 7f45665ea700 1 -- 172.21.15.28:0/2233327209 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456006ab20 con 0x7f45600c9920 2023-11-28T11:09:06.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.140+0000 7f983e7fc700 1 --2- 172.21.15.28:0/4285851174 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f982c0682f0 0x7f982c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.138+0000 7f0efec1a700 1 -- 172.21.15.28:0/3109754004 wait complete. 2023-11-28T11:09:06.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.140+0000 7f0efec1a700 1 Processor -- start 2023-11-28T11:09:06.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.140+0000 7f983e7fc700 1 -- 172.21.15.28:0/4285851174 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f9838086bf0 con 0x7f9848071070 2023-11-28T11:09:06.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.140+0000 7f984d5bc700 1 --2- 172.21.15.28:0/4285851174 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f982c0682f0 0x7f982c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.140+0000 7f983e7fc700 1 --2- 172.21.15.28:0/4285851174 >> [v2:172.21.15.28:6816/103512,v1:172.21.15.28:6817/103512] conn(0x7f982c071f40 0x7f982c074370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.140+0000 7f983e7fc700 1 -- 172.21.15.28:0/4285851174 --> [v2:172.21.15.28:6816/103512,v1:172.21.15.28:6817/103512] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f982c074a40 con 0x7f982c071f40 2023-11-28T11:09:06.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.140+0000 7f45665ea700 1 -- 172.21.15.28:0/2233327209 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f456006ba30 con 0x7f45600c9920 2023-11-28T11:09:06.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.140+0000 7f983e7fc700 1 -- 172.21.15.28:0/4285851174 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f983808c2d0 con 0x7f9848071070 2023-11-28T11:09:06.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.140+0000 7f984e5be700 1 --2- 172.21.15.28:0/4285851174 >> [v2:172.21.15.28:6816/103512,v1:172.21.15.28:6817/103512] conn(0x7f982c071f40 0x7f982c074370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.140+0000 7f10c67fc700 1 -- 172.21.15.28:0/1531661739 <== osd.7 v2:172.21.15.59:6824/103734 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f10b8006bc0 con 0x7f10b80015e0 2023-11-28T11:09:06.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.141+0000 7f984e5be700 1 --2- 172.21.15.28:0/4285851174 >> [v2:172.21.15.28:6816/103512,v1:172.21.15.28:6817/103512] conn(0x7f982c071f40 0x7f982c074370 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.141+0000 7f984d5bc700 1 --2- 172.21.15.28:0/4285851174 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f982c0682f0 0x7f982c06a7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f98480cd2d0 tx=0x7f9844008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.141+0000 7f455cff9700 1 -- 172.21.15.28:0/2233327209 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f455802b3f0 con 0x7f45600c9920 2023-11-28T11:09:06.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f983e7fc700 1 -- 172.21.15.28:0/4285851174 <== osd.2 v2:172.21.15.28:6816/103512 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f982c074a40 con 0x7f982c071f40 2023-11-28T11:09:06.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.140+0000 7f0efec1a700 1 -- start start 2023-11-28T11:09:06.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0efec1a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ef806b860 0x7f0ef81e6990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0efec1a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ef80cd120 0x7f0ef80b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0efec1a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ef81588a0 0x7f0ef81e4310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0efec1a700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef81e6570 con 0x7f0ef81588a0 2023-11-28T11:09:06.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0efec1a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef81e58f0 con 0x7f0ef806b860 2023-11-28T11:09:06.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0efec1a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ef81e5bd0 con 0x7f0ef80cd120 2023-11-28T11:09:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0ef7fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ef80cd120 0x7f0ef80b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0efd1b7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ef81588a0 0x7f0ef81e4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0ef7fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ef80cd120 0x7f0ef80b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:52772/0 (socket says 172.21.15.28:52772) 2023-11-28T11:09:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0ef7fff700 1 -- 172.21.15.28:0/3449036368 learned_addr learned my addr 172.21.15.28:0/3449036368 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0ef7fff700 1 -- 172.21.15.28:0/3449036368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ef806b860 msgr2=0x7f0ef81e6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0ef7fff700 1 --2- 172.21.15.28:0/3449036368 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ef806b860 0x7f0ef81e6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0ef7fff700 1 -- 172.21.15.28:0/3449036368 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ef81588a0 msgr2=0x7f0ef81e4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0ef7fff700 1 --2- 172.21.15.28:0/3449036368 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ef81588a0 0x7f0ef81e4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0ef7fff700 1 -- 172.21.15.28:0/3449036368 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ee80094a0 con 0x7f0ef80cd120 2023-11-28T11:09:06.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f0efd1b7700 1 --2- 172.21.15.28:0/3449036368 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ef81588a0 0x7f0ef81e4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:06.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f455cff9700 1 --2- 172.21.15.28:0/2233327209 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f45480684f0 0x7f454806a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f10d507a700 1 --2- 172.21.15.28:0/1531661739 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f10b40683c0 0x7f10b406a870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f10d015f040 tx=0x7f10c0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.143+0000 7f0ef7fff700 1 --2- 172.21.15.28:0/3449036368 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ef80cd120 0x7f0ef80b3ec0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0eec008db0 tx=0x7f0eec00a4d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f455cff9700 1 -- 172.21.15.28:0/2233327209 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f455800e030 con 0x7f45600c9920 2023-11-28T11:09:06.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f455cff9700 1 --2- 172.21.15.28:0/2233327209 >> [v2:172.21.15.28:6824/103511,v1:172.21.15.28:6825/103511] conn(0x7f45480721c0 0x7f45480745f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f455cff9700 1 -- 172.21.15.28:0/2233327209 --> [v2:172.21.15.28:6824/103511,v1:172.21.15.28:6825/103511] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4548074cc0 con 0x7f45480721c0 2023-11-28T11:09:06.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f455cff9700 1 -- 172.21.15.28:0/2233327209 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f45580878e0 con 0x7f45600c9920 2023-11-28T11:09:06.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7f4564b87700 1 --2- 172.21.15.28:0/2233327209 >> [v2:172.21.15.28:6824/103511,v1:172.21.15.28:6825/103511] conn(0x7f45480721c0 0x7f45480745f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.142+0000 7fe1a97ed700 1 Processor -- start 2023-11-28T11:09:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.143+0000 7f4ef2dba700 1 Processor -- start 2023-11-28T11:09:06.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.143+0000 7fe1a97ed700 1 -- start start 2023-11-28T11:09:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.143+0000 7f4ef2dba700 1 -- start start 2023-11-28T11:09:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.144+0000 7f4ef2dba700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4eec1646c0 0x7f4eec070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.144+0000 7f4ef2dba700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eec071300 con 0x7f4eec160c70 2023-11-28T11:09:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.144+0000 7f4ef2dba700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4eec0ae210 con 0x7f4eec070ea0 2023-11-28T11:09:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.144+0000 7f4ef2dba700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4eec161040 con 0x7f4eec1646c0 2023-11-28T11:09:06.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.144+0000 7fe1a97ed700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe1a41592b0 0x7fe1a40cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.144+0000 7fe1a97ed700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe1a40caf90 0x7fe1a4070de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.144+0000 7fe1a97ed700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1a415a4a0 con 0x7fe1a4070960 2023-11-28T11:09:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.144+0000 7fe1a97ed700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a4071320 con 0x7fe1a41592b0 2023-11-28T11:09:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.144+0000 7fe1a97ed700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1a415a780 con 0x7fe1a40caf90 2023-11-28T11:09:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.144+0000 7f4ef0b56700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4eec160c70 0x7f4eec0adb00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35082/0 (socket says 172.21.15.28:35082) 2023-11-28T11:09:06.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.144+0000 7f4ef0b56700 1 -- 172.21.15.28:0/633590652 learned_addr learned my addr 172.21.15.28:0/633590652 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7f4564b87700 1 --2- 172.21.15.28:0/2233327209 >> [v2:172.21.15.28:6824/103511,v1:172.21.15.28:6825/103511] conn(0x7f45480721c0 0x7f45480745f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7f4eeb7fe700 1 -- 172.21.15.28:0/633590652 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769632524 0 0) 0x7f4eec071300 con 0x7f4eec160c70 2023-11-28T11:09:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7f4eeb7fe700 1 -- 172.21.15.28:0/633590652 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ed80035b0 con 0x7f4eec160c70 2023-11-28T11:09:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7f4eeb7fe700 1 -- 172.21.15.28:0/633590652 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 300099927 0 0) 0x7f4ed80035b0 con 0x7f4eec160c70 2023-11-28T11:09:06.189 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7f4eeb7fe700 1 -- 172.21.15.28:0/633590652 >> v2:172.21.15.59:3300/0 conn(0x7f4eec1646c0 msgr2=0x7f4eec070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7f4eeb7fe700 1 --2- 172.21.15.28:0/633590652 >> v2:172.21.15.59:3300/0 conn(0x7f4eec1646c0 0x7f4eec070960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7f4eeb7fe700 1 -- 172.21.15.28:0/633590652 >> v1:172.21.15.59:6789/0 conn(0x7f4eec070ea0 legacy=0x7f4eec06cfc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:09:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7f4eeb7fe700 1 -- 172.21.15.28:0/633590652 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec164b30 con 0x7f4eec160c70 2023-11-28T11:09:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7f4eeb7fe700 1 -- 172.21.15.28:0/633590652 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f4ed4001670 con 0x7f4eec160c70 2023-11-28T11:09:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7f4eeb7fe700 1 -- 172.21.15.28:0/633590652 >> v1:172.21.15.28:6789/0 conn(0x7f4eec160c70 legacy=0x7f4eec0adb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7f4eeb7fe700 1 --2- 172.21.15.28:0/633590652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed8003a00 0x7f4ed8003df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7fe1a27fc700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe1a4070960 0x7fe1a40ca880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:45876/0 (socket says 172.21.15.28:45876) 2023-11-28T11:09:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7fe1a27fc700 1 -- 172.21.15.28:0/327882935 learned_addr learned my addr 172.21.15.28:0/327882935 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:06.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7fe1a2ffd700 1 --2- 172.21.15.28:0/327882935 >> v2:172.21.15.28:3300/0 conn(0x7fe1a41592b0 0x7fe1a40cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7fe1a1ffb700 1 -- 172.21.15.28:0/327882935 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2461721398 0 0) 0x7fe1a415a4a0 con 0x7fe1a4070960 2023-11-28T11:09:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7fe1a1ffb700 1 -- 172.21.15.28:0/327882935 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe18c0035b0 con 0x7fe1a4070960 2023-11-28T11:09:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7fe1a2ffd700 1 -- 172.21.15.28:0/327882935 >> v2:172.21.15.59:3300/0 conn(0x7fe1a40caf90 msgr2=0x7fe1a4070de0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:09:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7fe1a2ffd700 1 --2- 172.21.15.28:0/327882935 >> v2:172.21.15.59:3300/0 conn(0x7fe1a40caf90 0x7fe1a4070de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7fe1a2ffd700 1 -- 172.21.15.28:0/327882935 >> v1:172.21.15.59:6789/0 conn(0x7fe1a4070960 legacy=0x7fe1a40ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7fe1a2ffd700 1 -- 172.21.15.28:0/327882935 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a40ca080 con 0x7fe1a41592b0 2023-11-28T11:09:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7fe1a2ffd700 1 --2- 172.21.15.28:0/327882935 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1a41592b0 0x7fe1a40cc5b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe1a406bab0 tx=0x7fe194009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b3eef5b74e63e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.145+0000 7f4eeb7fe700 1 --2- 172.21.15.28:0/633590652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4ed8004850 0x7f4ed8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7f4eeb7fe700 1 --2- 172.21.15.28:0/633590652 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4ed8005320 0x7f4ed800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7f4eeb7fe700 1 -- 172.21.15.28:0/633590652 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec161040 con 0x7f4ed8004850 2023-11-28T11:09:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7f4eeb7fe700 1 -- 172.21.15.28:0/633590652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed800c1e0 con 0x7f4ed8003a00 2023-11-28T11:09:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7f4eeb7fe700 1 -- 172.21.15.28:0/633590652 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed800c3a0 con 0x7f4ed8005320 2023-11-28T11:09:06.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7fe1a1ffb700 1 -- 172.21.15.28:0/327882935 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194011830 con 0x7fe1a41592b0 2023-11-28T11:09:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7f4ef0b56700 1 --2- 172.21.15.28:0/633590652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed8003a00 0x7f4ed8003df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7fe1a1ffb700 1 -- 172.21.15.28:0/327882935 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe194011990 con 0x7fe1a41592b0 2023-11-28T11:09:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7fe1a1ffb700 1 -- 172.21.15.28:0/327882935 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe194011d00 con 0x7fe1a41592b0 2023-11-28T11:09:06.193 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7f455cff9700 1 -- 172.21.15.28:0/2233327209 <== osd.1 v2:172.21.15.28:6824/103511 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4548074cc0 con 0x7f45480721c0 2023-11-28T11:09:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.144+0000 7fc41720c700 1 -- 172.21.15.28:0/352731195 wait complete. 2023-11-28T11:09:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7fc41720c700 1 Processor -- start 2023-11-28T11:09:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7f455f7fe700 1 --2- 172.21.15.28:0/2233327209 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f45480684f0 0x7f454806a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7f4ef0b56700 1 -- 172.21.15.28:0/633590652 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4ed8005320 msgr2=0x7f4ed800ba10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:09:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7f4ef0b56700 1 --2- 172.21.15.28:0/633590652 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4ed8005320 0x7f4ed800ba10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7f4ef0b56700 1 -- 172.21.15.28:0/633590652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4ed8004850 msgr2=0x7f4ed8004cc0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:09:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7f4ef0b56700 1 --2- 172.21.15.28:0/633590652 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4ed8004850 0x7f4ed8004cc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7f4ef0b56700 1 -- 172.21.15.28:0/633590652 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed80035b0 con 0x7f4ed8003a00 2023-11-28T11:09:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.146+0000 7fc41720c700 1 -- start start 2023-11-28T11:09:06.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.147+0000 7f455f7fe700 1 --2- 172.21.15.28:0/2233327209 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f45480684f0 0x7f454806a9a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f45600c2080 tx=0x7f4550005cf0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.147+0000 7fe1a97ed700 1 -- 172.21.15.28:0/327882935 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1a41592b0 msgr2=0x7fe1a40cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.147+0000 7fe1a97ed700 1 --2- 172.21.15.28:0/327882935 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1a41592b0 0x7fe1a40cc5b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe1a406bab0 tx=0x7fe194009b20 comp rx=0 tx=0).stop 2023-11-28T11:09:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.147+0000 7fe1a97ed700 1 -- 172.21.15.28:0/327882935 shutdown_connections 2023-11-28T11:09:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.147+0000 7fe1a97ed700 1 --2- 172.21.15.28:0/327882935 >> v2:172.21.15.59:3300/0 conn(0x7fe1a40caf90 0x7fe1a4070de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.147+0000 7fe1a97ed700 1 --2- 172.21.15.28:0/327882935 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1a41592b0 0x7fe1a40cc5b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.147+0000 7fe1a97ed700 1 -- 172.21.15.28:0/327882935 >> 172.21.15.28:0/327882935 conn(0x7fe1a40ce9e0 msgr2=0x7fe1a40cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.147+0000 7f0ef57fa700 1 -- 172.21.15.28:0/3449036368 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec010070 con 0x7f0ef80cd120 2023-11-28T11:09:06.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.147+0000 7f0ef57fa700 1 -- 172.21.15.28:0/3449036368 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0eec00aae0 con 0x7f0ef80cd120 2023-11-28T11:09:06.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.147+0000 7f0ef57fa700 1 -- 172.21.15.28:0/3449036368 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0eec008ea0 con 0x7f0ef80cd120 2023-11-28T11:09:06.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.147+0000 7fe1a97ed700 1 -- 172.21.15.28:0/327882935 shutdown_connections 2023-11-28T11:09:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.147+0000 7f0efec1a700 1 -- 172.21.15.28:0/3449036368 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ef806ae00 con 0x7f0ef80cd120 2023-11-28T11:09:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.147+0000 7f0efec1a700 1 -- 172.21.15.28:0/3449036368 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ef806ab20 con 0x7f0ef80cd120 2023-11-28T11:09:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.148+0000 7f4ef0b56700 1 --2- 172.21.15.28:0/633590652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed8003a00 0x7f4ed8003df0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400c2f0 tx=0x7f4ed8005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cf51f8e2e0117fc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.148+0000 7fc41720c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4100adec0 0x7fc4100b37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.148+0000 7fc41720c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4100c9df0 0x7fc41014bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.148+0000 7fc41720c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc410161bb0 0x7fc41014e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.148+0000 7fc41720c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41014c2c0 con 0x7fc410161bb0 2023-11-28T11:09:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.148+0000 7fc41720c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41014d670 con 0x7fc4100adec0 2023-11-28T11:09:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.148+0000 7fc41720c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41014c5d0 con 0x7fc4100c9df0 2023-11-28T11:09:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.148+0000 7fc4157a9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc410161bb0 0x7fc41014e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.148+0000 7fc4157a9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc410161bb0 0x7fc41014e8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47042/0 (socket says 172.21.15.28:47042) 2023-11-28T11:09:06.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.148+0000 7fc4157a9700 1 -- 172.21.15.28:0/3725891536 learned_addr learned my addr 172.21.15.28:0/3725891536 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.148+0000 7f4eeb7fe700 1 -- 172.21.15.28:0/633590652 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400ce30 con 0x7f4ed8003a00 2023-11-28T11:09:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.148+0000 7f4eeb7fe700 1 -- 172.21.15.28:0/633590652 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed4005960 con 0x7f4ed8003a00 2023-11-28T11:09:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.148+0000 7f4eeb7fe700 1 -- 172.21.15.28:0/633590652 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed4005cf0 con 0x7f4ed8003a00 2023-11-28T11:09:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.148+0000 7f4ef1357700 1 -- 172.21.15.28:0/633590652 reap_dead start 2023-11-28T11:09:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7fc4157a9700 1 -- 172.21.15.28:0/3725891536 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4100c9df0 msgr2=0x7fc41014bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7fc4157a9700 1 --2- 172.21.15.28:0/3725891536 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4100c9df0 0x7fc41014bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7fc4157a9700 1 -- 172.21.15.28:0/3725891536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4100adec0 msgr2=0x7fc4100b37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7fc414fa8700 1 --2- 172.21.15.28:0/3725891536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4100adec0 0x7fc4100b37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7fc4157a9700 1 --2- 172.21.15.28:0/3725891536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4100adec0 0x7fc4100b37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7fc4157a9700 1 -- 172.21.15.28:0/3725891536 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc408009b30 con 0x7fc410161bb0 2023-11-28T11:09:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7f0efec1a700 1 -- 172.21.15.28:0/3449036368 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ef8071970 con 0x7f0ef80cd120 2023-11-28T11:09:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7fc414fa8700 1 --2- 172.21.15.28:0/3725891536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4100adec0 0x7fc4100b37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:06.201 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7fc4157a9700 1 --2- 172.21.15.28:0/3725891536 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc410161bb0 0x7fc41014e8b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc40400b450 tx=0x7fc40400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7f5abcf78700 1 Processor -- start 2023-11-28T11:09:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7f5abcf78700 1 -- start start 2023-11-28T11:09:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.147+0000 7fe1a97ed700 1 -- 172.21.15.28:0/327882935 wait complete. 2023-11-28T11:09:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7fe1a97ed700 1 Processor -- start 2023-11-28T11:09:06.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7f4ef2dba700 1 -- 172.21.15.28:0/633590652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed8003a00 msgr2=0x7f4ed8003df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7f4ef2dba700 1 --2- 172.21.15.28:0/633590652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed8003a00 0x7f4ed8003df0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4ed400c2f0 tx=0x7f4ed8005880 comp rx=0 tx=0).stop 2023-11-28T11:09:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7fc40d7fa700 1 -- 172.21.15.28:0/3725891536 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404011830 con 0x7fc410161bb0 2023-11-28T11:09:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.150+0000 7fc40d7fa700 1 -- 172.21.15.28:0/3725891536 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4040119b0 con 0x7fc410161bb0 2023-11-28T11:09:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.150+0000 7fc41720c700 1 -- 172.21.15.28:0/3725891536 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41006cd50 con 0x7fc410161bb0 2023-11-28T11:09:06.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.150+0000 7fc41720c700 1 -- 172.21.15.28:0/3725891536 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc41006a4c0 con 0x7fc410161bb0 2023-11-28T11:09:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.150+0000 7f5abcf78700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ab80ad7b0 0x7f5ab80adc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.150+0000 7f5abcf78700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5ab8168570 0x7f5ab816a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.150+0000 7f5abcf78700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ab80cdb60 con 0x7f5ab8160be0 2023-11-28T11:09:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.150+0000 7f5abcf78700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ab80ae160 con 0x7f5ab8168570 2023-11-28T11:09:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.150+0000 7f5abcf78700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab80cde40 con 0x7f5ab80ad7b0 2023-11-28T11:09:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7f5ab6d9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5ab8168570 0x7f5ab816a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7f5ab6d9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5ab8168570 0x7f5ab816a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:34338/0 (socket says 172.21.15.28:34338) 2023-11-28T11:09:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7f5ab6d9d700 1 -- 172.21.15.28:0/240758303 learned_addr learned my addr 172.21.15.28:0/240758303 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7f5ab5d9b700 1 --2- 172.21.15.28:0/240758303 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ab80ad7b0 0x7f5ab80adc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.149+0000 7fe1a97ed700 1 -- start start 2023-11-28T11:09:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7fe1a97ed700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1a4070960 0x7fe1a40c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7fe1a97ed700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1a40caf90 0x7fe1a40c1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7fe1a97ed700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1a41592b0 0x7fe1a415c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7fe1a97ed700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a40c1ca0 con 0x7fe1a41592b0 2023-11-28T11:09:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7fe1a97ed700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a40b0320 con 0x7fe1a40caf90 2023-11-28T11:09:06.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7fe1a97ed700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1a40b2120 con 0x7fe1a4070960 2023-11-28T11:09:06.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7f5ab6d9d700 1 -- 172.21.15.28:0/240758303 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ab80ad7b0 msgr2=0x7f5ab80adc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7f5ab6d9d700 1 --2- 172.21.15.28:0/240758303 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ab80ad7b0 0x7f5ab80adc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7f5ab6d9d700 1 -- 172.21.15.28:0/240758303 >> v1:172.21.15.59:6789/0 conn(0x7f5ab8160be0 legacy=0x7f5ab80c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7f5ab6d9d700 1 -- 172.21.15.28:0/240758303 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab80ce120 con 0x7f5ab8168570 2023-11-28T11:09:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7f4ef2dba700 1 -- 172.21.15.28:0/633590652 shutdown_connections 2023-11-28T11:09:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7f4ef2dba700 1 --2- 172.21.15.28:0/633590652 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed8003a00 0x7f4ed8003df0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7f4ef2dba700 1 -- 172.21.15.28:0/633590652 >> 172.21.15.28:0/633590652 conn(0x7f4eec0ce9e0 msgr2=0x7f4eec0cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7f4ef2dba700 1 -- 172.21.15.28:0/633590652 shutdown_connections 2023-11-28T11:09:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7f4ef2dba700 1 -- 172.21.15.28:0/633590652 wait complete. 2023-11-28T11:09:06.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7f5ab6d9d700 1 --2- 172.21.15.28:0/240758303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ab8168570 0x7f5ab816a950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009e50 tx=0x7f5ab000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fb859dd4946772b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.151+0000 7f5ab559a700 1 -- 172.21.15.28:0/240758303 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab000e070 con 0x7f5ab8168570 2023-11-28T11:09:06.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.152+0000 7f5ab559a700 1 -- 172.21.15.28:0/240758303 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ab000b910 con 0x7f5ab8168570 2023-11-28T11:09:06.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.152+0000 7f5ab559a700 1 -- 172.21.15.28:0/240758303 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ab000bcc0 con 0x7f5ab8168570 2023-11-28T11:09:06.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.150+0000 7fc40d7fa700 1 -- 172.21.15.28:0/3725891536 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404011d60 con 0x7fc410161bb0 2023-11-28T11:09:06.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.152+0000 7fe1a37fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1a41592b0 0x7fe1a415c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.152+0000 7fe1a37fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1a41592b0 0x7fe1a415c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47054/0 (socket says 172.21.15.28:47054) 2023-11-28T11:09:06.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.152+0000 7fe1a37fe700 1 -- 172.21.15.28:0/2500645130 learned_addr learned my addr 172.21.15.28:0/2500645130 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.152+0000 7fe1a37fe700 1 -- 172.21.15.28:0/2500645130 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1a4070960 msgr2=0x7fe1a40c4f80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:09:06.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.152+0000 7fe1a37fe700 1 --2- 172.21.15.28:0/2500645130 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1a4070960 0x7fe1a40c4f80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.152+0000 7fe1a37fe700 1 -- 172.21.15.28:0/2500645130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1a40caf90 msgr2=0x7fe1a40c1760 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:09:06.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.152+0000 7fe1a37fe700 1 --2- 172.21.15.28:0/2500645130 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1a40caf90 0x7fe1a40c1760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.152+0000 7fe1a37fe700 1 -- 172.21.15.28:0/2500645130 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1940094a0 con 0x7fe1a41592b0 2023-11-28T11:09:06.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.153+0000 7fc40d7fa700 1 -- 172.21.15.28:0/3725891536 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc404021070 con 0x7fc410161bb0 2023-11-28T11:09:06.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.153+0000 7f4ef2dba700 1 Processor -- start 2023-11-28T11:09:06.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.153+0000 7fe1a37fe700 1 --2- 172.21.15.28:0/2500645130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1a41592b0 0x7fe1a415c5b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe19800b4a0 tx=0x7fe19800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7fe18b7fe700 1 -- 172.21.15.28:0/2500645130 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198011830 con 0x7fe1a41592b0 2023-11-28T11:09:06.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7fe18b7fe700 1 -- 172.21.15.28:0/2500645130 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1980119b0 con 0x7fe1a41592b0 2023-11-28T11:09:06.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7fe18b7fe700 1 -- 172.21.15.28:0/2500645130 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe198011d60 con 0x7fe1a41592b0 2023-11-28T11:09:06.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7f5abcf78700 1 -- 172.21.15.28:0/240758303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ab8168570 msgr2=0x7f5ab816a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7f5abcf78700 1 --2- 172.21.15.28:0/240758303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ab8168570 0x7f5ab816a950 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f5ab0009e50 tx=0x7f5ab000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:06.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.153+0000 7f4ef2dba700 1 -- start start 2023-11-28T11:09:06.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7f5abcf78700 1 -- 172.21.15.28:0/240758303 shutdown_connections 2023-11-28T11:09:06.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7f5abcf78700 1 --2- 172.21.15.28:0/240758303 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ab8168570 0x7f5ab816a950 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7f5abcf78700 1 --2- 172.21.15.28:0/240758303 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ab80ad7b0 0x7f5ab80adc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7f5abcf78700 1 -- 172.21.15.28:0/240758303 >> 172.21.15.28:0/240758303 conn(0x7f5ab80cf670 msgr2=0x7f5ab80cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7f5abcf78700 1 -- 172.21.15.28:0/240758303 shutdown_connections 2023-11-28T11:09:06.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7f5abcf78700 1 -- 172.21.15.28:0/240758303 wait complete. 2023-11-28T11:09:06.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f4ef2dba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed8003a00 0x7f4eec1f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f4ef2dba700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4eec0c41d0 0x7f4eec0c4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f4ef2dba700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4eec0c2630 0x7f4eec0c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f4ef2dba700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec0c4b80 con 0x7f4eec0c41d0 2023-11-28T11:09:06.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f4ef2dba700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec0c3390 con 0x7f4ed8003a00 2023-11-28T11:09:06.213 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f4ef2dba700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec0c3670 con 0x7f4eec0c2630 2023-11-28T11:09:06.213 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.153+0000 7fc40d7fa700 1 --2- 172.21.15.28:0/3725891536 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3fc0682f0 0x7fc3fc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.213 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7fc41720c700 1 -- 172.21.15.28:0/3725891536 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc3f0000fc0 con 0x7fc410161bb0 2023-11-28T11:09:06.213 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7fc40d7fa700 1 -- 172.21.15.28:0/3725891536 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc404085e40 con 0x7fc410161bb0 2023-11-28T11:09:06.213 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7fc40d7fa700 1 --2- 172.21.15.28:0/3725891536 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x7fc3fc071ec0 0x7fc3fc0742f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.213 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7fc40d7fa700 1 -- 172.21.15.28:0/3725891536 --> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc3fc0749c0 con 0x7fc3fc071ec0 2023-11-28T11:09:06.213 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.154+0000 7fc40d7fa700 1 -- 172.21.15.28:0/3725891536 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc4040523b0 con 0x7fc410161bb0 2023-11-28T11:09:06.214 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f4eebfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4eec0c41d0 0x7f4eec0c4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.214 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f4eebfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4eec0c41d0 0x7f4eec0c4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47056/0 (socket says 172.21.15.28:47056) 2023-11-28T11:09:06.214 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f4eebfff700 1 -- 172.21.15.28:0/2075192494 learned_addr learned my addr 172.21.15.28:0/2075192494 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.214 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f0ef57fa700 1 -- 172.21.15.28:0/3449036368 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0eec01d030 con 0x7f0ef80cd120 2023-11-28T11:09:06.214 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f0ef57fa700 1 --2- 172.21.15.28:0/3449036368 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0ee40683c0 0x7f0ee406a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.214 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f0ef57fa700 1 -- 172.21.15.28:0/3449036368 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0eec085760 con 0x7f0ef80cd120 2023-11-28T11:09:06.214 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f0ef57fa700 1 --2- 172.21.15.28:0/3449036368 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x7f0ee4072090 0x7f0ee40744c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.214 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f0ef57fa700 1 -- 172.21.15.28:0/3449036368 --> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0ee4074b90 con 0x7f0ee4072090 2023-11-28T11:09:06.214 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f0ef57fa700 1 -- 172.21.15.28:0/3449036368 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0eec087d20 con 0x7f0ef80cd120 2023-11-28T11:09:06.215 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f4eebfff700 1 -- 172.21.15.28:0/2075192494 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4eec0c2630 msgr2=0x7f4eec0c2ad0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:09:06.215 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f4eebfff700 1 --2- 172.21.15.28:0/2075192494 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4eec0c2630 0x7f4eec0c2ad0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.215 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f4eebfff700 1 -- 172.21.15.28:0/2075192494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed8003a00 msgr2=0x7f4eec1f5c30 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:09:06.215 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f4eebfff700 1 --2- 172.21.15.28:0/2075192494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4ed8003a00 0x7f4eec1f5c30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.215 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7f4eebfff700 1 -- 172.21.15.28:0/2075192494 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4eec1650f0 con 0x7f4eec0c41d0 2023-11-28T11:09:06.215 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.156+0000 7f4eebfff700 1 --2- 172.21.15.28:0/2075192494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4eec0c41d0 0x7f4eec0c4640 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400b520 tx=0x7f4ee400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7fe1a97ed700 1 -- 172.21.15.28:0/2500645130 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1a40c9d90 con 0x7fe1a41592b0 2023-11-28T11:09:06.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.155+0000 7fe1a97ed700 1 -- 172.21.15.28:0/2500645130 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1a40ca360 con 0x7fe1a41592b0 2023-11-28T11:09:06.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.156+0000 7f4ee97fa700 1 -- 172.21.15.28:0/2075192494 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4011830 con 0x7f4eec0c41d0 2023-11-28T11:09:06.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.156+0000 7f4ee97fa700 1 -- 172.21.15.28:0/2075192494 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ee4011990 con 0x7f4eec0c41d0 2023-11-28T11:09:06.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.156+0000 7f4ee97fa700 1 -- 172.21.15.28:0/2075192494 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ee4011d40 con 0x7f4eec0c41d0 2023-11-28T11:09:06.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.157+0000 7f0efd1b7700 1 --2- 172.21.15.28:0/3449036368 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x7f0ee4072090 0x7f0ee40744c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.157+0000 7f0efc9b6700 1 --2- 172.21.15.28:0/3449036368 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0ee40683c0 0x7f0ee406a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.157+0000 7f0efc9b6700 1 --2- 172.21.15.28:0/3449036368 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0ee40683c0 0x7f0ee406a870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8152b60 tx=0x7f0ef0009250 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.157+0000 7f0efd1b7700 1 --2- 172.21.15.28:0/3449036368 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x7f0ee4072090 0x7f0ee40744c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.156+0000 7f4ef2dba700 1 -- 172.21.15.28:0/2075192494 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eec164e10 con 0x7f4eec0c41d0 2023-11-28T11:09:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.157+0000 7f4ef2dba700 1 -- 172.21.15.28:0/2075192494 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec1653d0 con 0x7f4eec0c41d0 2023-11-28T11:09:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.156+0000 7f5abcf78700 1 Processor -- start 2023-11-28T11:09:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.157+0000 7f5abcf78700 1 -- start start 2023-11-28T11:09:06.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.156+0000 7fe1a97ed700 1 -- 172.21.15.28:0/2500645130 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe1a415d1d0 con 0x7fe1a41592b0 2023-11-28T11:09:06.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.159+0000 7f0ef57fa700 1 -- 172.21.15.28:0/3449036368 <== osd.0 v2:172.21.15.28:6800/103508 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f0ee4074b90 con 0x7f0ee4072090 2023-11-28T11:09:06.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.159+0000 7fe18b7fe700 1 -- 172.21.15.28:0/2500645130 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe198021070 con 0x7fe1a41592b0 2023-11-28T11:09:06.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.159+0000 7fe18b7fe700 1 --2- 172.21.15.28:0/2500645130 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe190068340 0x7fe19006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.160+0000 7fe18b7fe700 1 -- 172.21.15.28:0/2500645130 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe198085e20 con 0x7fe1a41592b0 2023-11-28T11:09:06.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.160+0000 7fe18b7fe700 1 --2- 172.21.15.28:0/2500645130 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x7fe190071f90 0x7fe1900743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.160+0000 7fe18b7fe700 1 -- 172.21.15.28:0/2500645130 --> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe190074a90 con 0x7fe190071f90 2023-11-28T11:09:06.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.160+0000 7fe18b7fe700 1 -- 172.21.15.28:0/2500645130 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe198052430 con 0x7fe1a41592b0 2023-11-28T11:09:06.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.160+0000 7f4ee97fa700 1 -- 172.21.15.28:0/2075192494 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4ee4021070 con 0x7f4eec0c41d0 2023-11-28T11:09:06.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.158+0000 7f5abcf78700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ab80ad7b0 0x7f5ab80b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.158+0000 7f5abcf78700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ab8160be0 0x7f5ab8159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.160+0000 7fe1a2ffd700 1 --2- 172.21.15.28:0/2500645130 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe190068340 0x7fe19006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.158+0000 7f5abcf78700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ab8168570 0x7f5ab815bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.158+0000 7f5abcf78700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab80b3410 con 0x7f5ab8160be0 2023-11-28T11:09:06.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.158+0000 7f5abcf78700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab815a3e0 con 0x7f5ab8168570 2023-11-28T11:09:06.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.158+0000 7f5abcf78700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ab80b4140 con 0x7f5ab80ad7b0 2023-11-28T11:09:06.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.158+0000 7f5ab5d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ab8160be0 0x7f5ab8159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.158+0000 7f5ab5d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ab8160be0 0x7f5ab8159ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47068/0 (socket says 172.21.15.28:47068) 2023-11-28T11:09:06.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.158+0000 7f5ab5d9b700 1 -- 172.21.15.28:0/2906631907 learned_addr learned my addr 172.21.15.28:0/2906631907 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.158+0000 7f5ab659c700 1 --2- 172.21.15.28:0/2906631907 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ab80ad7b0 0x7f5ab80b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.159+0000 7f5ab6d9d700 1 --2- 172.21.15.28:0/2906631907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ab8168570 0x7f5ab815bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.161+0000 7f5ab5d9b700 1 -- 172.21.15.28:0/2906631907 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ab80ad7b0 msgr2=0x7f5ab80b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.161+0000 7f5ab5d9b700 1 --2- 172.21.15.28:0/2906631907 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ab80ad7b0 0x7f5ab80b2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.161+0000 7f5ab5d9b700 1 -- 172.21.15.28:0/2906631907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ab8168570 msgr2=0x7f5ab815bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.161+0000 7f5ab5d9b700 1 --2- 172.21.15.28:0/2906631907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ab8168570 0x7f5ab815bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.161+0000 7f5ab5d9b700 1 -- 172.21.15.28:0/2906631907 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab0009b30 con 0x7f5ab8160be0 2023-11-28T11:09:06.222 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.161+0000 7f4ee97fa700 1 --2- 172.21.15.28:0/2075192494 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4ecc068340 0x7f4ecc06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.222 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.161+0000 7f4ee97fa700 1 -- 172.21.15.28:0/2075192494 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f4ee4085de0 con 0x7f4eec0c41d0 2023-11-28T11:09:06.222 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.161+0000 7f5ab5d9b700 1 --2- 172.21.15.28:0/2906631907 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ab8160be0 0x7f5ab8159ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00e870 tx=0x7f5aac00eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.222 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.161+0000 7fe1a2ffd700 1 --2- 172.21.15.28:0/2500645130 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe190068340 0x7fe19006a7f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40cc720 tx=0x7fe194008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.161+0000 7f4ef0b56700 1 --2- 172.21.15.28:0/2075192494 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4ecc068340 0x7f4ecc06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.161+0000 7f5aa6ffd700 1 -- 172.21.15.28:0/2906631907 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac013680 con 0x7f5ab8160be0 2023-11-28T11:09:06.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.161+0000 7f5aa6ffd700 1 -- 172.21.15.28:0/2906631907 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac0137e0 con 0x7f5ab8160be0 2023-11-28T11:09:06.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.161+0000 7f5abcf78700 1 -- 172.21.15.28:0/2906631907 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ab806a4c0 con 0x7f5ab8160be0 2023-11-28T11:09:06.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.161+0000 7f5aa6ffd700 1 -- 172.21.15.28:0/2906631907 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac013b50 con 0x7f5ab8160be0 2023-11-28T11:09:06.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.161+0000 7f5abcf78700 1 -- 172.21.15.28:0/2906631907 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ab806a1e0 con 0x7f5ab8160be0 2023-11-28T11:09:06.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.162+0000 7f4ef0b56700 1 --2- 172.21.15.28:0/2075192494 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4ecc068340 0x7f4ecc06a7f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4000c00 tx=0x7f4ed4007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.162+0000 7f5abcf78700 1 -- 172.21.15.28:0/2906631907 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5a9c000fc0 con 0x7f5ab8160be0 2023-11-28T11:09:06.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.164+0000 7f5aa6ffd700 1 -- 172.21.15.28:0/2906631907 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5aac019030 con 0x7f5ab8160be0 2023-11-28T11:09:06.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.164+0000 7f5aa6ffd700 1 --2- 172.21.15.28:0/2906631907 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5a98068340 0x7f5a9806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.164+0000 7f5aa6ffd700 1 -- 172.21.15.28:0/2906631907 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5aac086040 con 0x7f5ab8160be0 2023-11-28T11:09:06.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.164+0000 7f5ab659c700 1 --2- 172.21.15.28:0/2906631907 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5a98068340 0x7f5a9806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.164+0000 7f5aa6ffd700 1 --2- 172.21.15.28:0/2906631907 >> [v2:172.21.15.59:6801/103695,v1:172.21.15.59:6803/103695] conn(0x7f5a98072010 0x7f5a98074440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.164+0000 7f5aa6ffd700 1 -- 172.21.15.28:0/2906631907 --> [v2:172.21.15.59:6801/103695,v1:172.21.15.59:6803/103695] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5a98074b10 con 0x7f5a98072010 2023-11-28T11:09:06.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.164+0000 7f5aa6ffd700 1 -- 172.21.15.28:0/2906631907 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5aac0864c0 con 0x7f5ab8160be0 2023-11-28T11:09:06.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.165+0000 7f5ab659c700 1 --2- 172.21.15.28:0/2906631907 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5a98068340 0x7f5a9806a7f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8000f60 tx=0x7f5aa8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.165+0000 7fe1a27fc700 1 --2- 172.21.15.28:0/2500645130 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x7fe190071f90 0x7fe1900743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.165+0000 7fe1a27fc700 1 --2- 172.21.15.28:0/2500645130 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x7fe190071f90 0x7fe1900743c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.166+0000 7fc414fa8700 1 --2- 172.21.15.28:0/3725891536 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3fc0682f0 0x7fc3fc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.166+0000 7f4ef2dba700 1 --2- 172.21.15.28:0/2075192494 >> [v2:172.21.15.28:6808/103510,v1:172.21.15.28:6809/103510] conn(0x7f4ed00015e0 0x7f4ed0003a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.166+0000 7f4ef2dba700 1 -- 172.21.15.28:0/2075192494 --> [v2:172.21.15.28:6808/103510,v1:172.21.15.28:6809/103510] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4ed0006bc0 con 0x7f4ed00015e0 2023-11-28T11:09:06.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.166+0000 7f4ef1357700 1 --2- 172.21.15.28:0/2075192494 >> [v2:172.21.15.28:6808/103510,v1:172.21.15.28:6809/103510] conn(0x7f4ed00015e0 0x7f4ed0003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.166+0000 7fc414fa8700 1 --2- 172.21.15.28:0/3725891536 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3fc0682f0 0x7fc3fc06a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc41006ffc0 tx=0x7fc408009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.166+0000 7f4ef1357700 1 --2- 172.21.15.28:0/2075192494 >> [v2:172.21.15.28:6808/103510,v1:172.21.15.28:6809/103510] conn(0x7f4ed00015e0 0x7f4ed0003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.167+0000 7fe18b7fe700 1 -- 172.21.15.28:0/2500645130 <== osd.6 v2:172.21.15.59:6816/103697 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fe190074a90 con 0x7fe190071f90 2023-11-28T11:09:06.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.168+0000 7f4ee97fa700 1 -- 172.21.15.28:0/2075192494 <== osd.3 v2:172.21.15.28:6808/103510 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4ed0006bc0 con 0x7f4ed00015e0 2023-11-28T11:09:06.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.168+0000 7f5ab6d9d700 1 --2- 172.21.15.28:0/2906631907 >> [v2:172.21.15.59:6801/103695,v1:172.21.15.59:6803/103695] conn(0x7f5a98072010 0x7f5a98074440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.168+0000 7f5ab6d9d700 1 --2- 172.21.15.28:0/2906631907 >> [v2:172.21.15.59:6801/103695,v1:172.21.15.59:6803/103695] conn(0x7f5a98072010 0x7f5a98074440 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.169+0000 7f5aa6ffd700 1 -- 172.21.15.28:0/2906631907 <== osd.4 v2:172.21.15.59:6801/103695 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5a98074b10 con 0x7f5a98072010 2023-11-28T11:09:06.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.170+0000 7f9850021700 1 -- 172.21.15.28:0/4285851174 --> [v2:172.21.15.28:6816/103512,v1:172.21.15.28:6817/103512] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9830002d40 con 0x7f982c071f40 2023-11-28T11:09:06.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.171+0000 7f983e7fc700 1 -- 172.21.15.28:0/4285851174 <== osd.2 v2:172.21.15.28:6816/103512 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9830002d40 con 0x7f982c071f40 2023-11-28T11:09:06.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.171+0000 7fc40ffff700 1 --2- 172.21.15.28:0/3725891536 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x7fc3fc071ec0 0x7fc3fc0742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.171+0000 7fc40ffff700 1 --2- 172.21.15.28:0/3725891536 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x7fc3fc071ec0 0x7fc3fc0742f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.172+0000 7f9850021700 1 -- 172.21.15.28:0/4285851174 >> [v2:172.21.15.28:6816/103512,v1:172.21.15.28:6817/103512] conn(0x7f982c071f40 msgr2=0x7f982c074370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.172+0000 7f9850021700 1 --2- 172.21.15.28:0/4285851174 >> [v2:172.21.15.28:6816/103512,v1:172.21.15.28:6817/103512] conn(0x7f982c071f40 0x7f982c074370 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.172+0000 7fc40d7fa700 1 -- 172.21.15.28:0/3725891536 <== osd.5 v2:172.21.15.59:6800/103696 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc3fc0749c0 con 0x7fc3fc071ec0 2023-11-28T11:09:06.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.172+0000 7f10d7adf700 1 -- 172.21.15.28:0/1531661739 --> [v2:172.21.15.59:6824/103734,v1:172.21.15.59:6825/103734] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f10b8005ca0 con 0x7f10b80015e0 2023-11-28T11:09:06.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.172+0000 7f9850021700 1 -- 172.21.15.28:0/4285851174 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f982c0682f0 msgr2=0x7f982c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.172+0000 7f9850021700 1 --2- 172.21.15.28:0/4285851174 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f982c0682f0 0x7f982c06a7a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f98480cd2d0 tx=0x7f9844008040 comp rx=0 tx=0).stop 2023-11-28T11:09:06.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.172+0000 7f9850021700 1 -- 172.21.15.28:0/4285851174 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9848071070 msgr2=0x7f98481f6ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.172+0000 7f9850021700 1 --2- 172.21.15.28:0/4285851174 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9848071070 0x7f98481f6ff0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9838013000 tx=0x7f9838004750 comp rx=0 tx=0).stop 2023-11-28T11:09:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.172+0000 7f984e5be700 1 -- 172.21.15.28:0/4285851174 reap_dead start 2023-11-28T11:09:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.172+0000 7f9850021700 1 -- 172.21.15.28:0/4285851174 shutdown_connections 2023-11-28T11:09:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.172+0000 7f9850021700 1 -- 172.21.15.28:0/4285851174 >> 172.21.15.28:0/4285851174 conn(0x7f98480ce9e0 msgr2=0x7f98480cabe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.172+0000 7f10c67fc700 1 -- 172.21.15.28:0/1531661739 <== osd.7 v2:172.21.15.59:6824/103734 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f10b8005ca0 con 0x7f10b80015e0 2023-11-28T11:09:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.173+0000 7f10d7adf700 1 -- 172.21.15.28:0/1531661739 >> [v2:172.21.15.59:6824/103734,v1:172.21.15.59:6825/103734] conn(0x7f10b80015e0 msgr2=0x7f10b8003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.173+0000 7f10d7adf700 1 --2- 172.21.15.28:0/1531661739 >> [v2:172.21.15.59:6824/103734,v1:172.21.15.59:6825/103734] conn(0x7f10b80015e0 0x7f10b8003a90 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.173+0000 7f10d7adf700 1 -- 172.21.15.28:0/1531661739 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f10b40683c0 msgr2=0x7f10b406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.173+0000 7f10d7adf700 1 --2- 172.21.15.28:0/1531661739 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f10b40683c0 0x7f10b406a870 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f10d015f040 tx=0x7f10c0008040 comp rx=0 tx=0).stop 2023-11-28T11:09:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.173+0000 7f10d7adf700 1 -- 172.21.15.28:0/1531661739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10d00c2630 msgr2=0x7f10d00c8e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.173+0000 7f10d7adf700 1 --2- 172.21.15.28:0/1531661739 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10d00c2630 0x7f10d00c8e80 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f10cc009c40 tx=0x7f10cc004750 comp rx=0 tx=0).stop 2023-11-28T11:09:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.173+0000 7f10d607c700 1 -- 172.21.15.28:0/1531661739 reap_dead start 2023-11-28T11:09:06.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.173+0000 7f10d7adf700 1 -- 172.21.15.28:0/1531661739 shutdown_connections 2023-11-28T11:09:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.173+0000 7f10d7adf700 1 -- 172.21.15.28:0/1531661739 >> 172.21.15.28:0/1531661739 conn(0x7f10d00b0dd0 msgr2=0x7f10d00caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.174+0000 7f10d7adf700 1 -- 172.21.15.28:0/1531661739 shutdown_connections 2023-11-28T11:09:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.174+0000 7f10d7adf700 1 -- 172.21.15.28:0/1531661739 wait complete. 2023-11-28T11:09:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.174+0000 7f45665ea700 1 -- 172.21.15.28:0/2233327209 --> [v2:172.21.15.28:6824/103511,v1:172.21.15.28:6825/103511] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f45600ae500 con 0x7f45480721c0 2023-11-28T11:09:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.175+0000 7f455cff9700 1 -- 172.21.15.28:0/2233327209 <== osd.1 v2:172.21.15.28:6824/103511 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f45600ae500 con 0x7f45480721c0 2023-11-28T11:09:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.175+0000 7f45665ea700 1 -- 172.21.15.28:0/2233327209 >> [v2:172.21.15.28:6824/103511,v1:172.21.15.28:6825/103511] conn(0x7f45480721c0 msgr2=0x7f45480745f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.175+0000 7f45665ea700 1 --2- 172.21.15.28:0/2233327209 >> [v2:172.21.15.28:6824/103511,v1:172.21.15.28:6825/103511] conn(0x7f45480721c0 0x7f45480745f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.175+0000 7f45665ea700 1 -- 172.21.15.28:0/2233327209 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f45480684f0 msgr2=0x7f454806a9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.175+0000 7f45665ea700 1 --2- 172.21.15.28:0/2233327209 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f45480684f0 0x7f454806a9a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f45600c2080 tx=0x7f4550005cf0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.175+0000 7f45665ea700 1 -- 172.21.15.28:0/2233327209 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45600c9920 msgr2=0x7f45600b4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.175+0000 7f45665ea700 1 --2- 172.21.15.28:0/2233327209 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f45600c9920 0x7f45600b4390 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4558009d60 tx=0x7f4558004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.175+0000 7f4564b87700 1 -- 172.21.15.28:0/2233327209 reap_dead start 2023-11-28T11:09:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.175+0000 7f45665ea700 1 -- 172.21.15.28:0/2233327209 shutdown_connections 2023-11-28T11:09:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.175+0000 7f45665ea700 1 -- 172.21.15.28:0/2233327209 >> 172.21.15.28:0/2233327209 conn(0x7f45600cf670 msgr2=0x7f456016b510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.175+0000 7f45665ea700 1 -- 172.21.15.28:0/2233327209 shutdown_connections 2023-11-28T11:09:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.176+0000 7f45665ea700 1 -- 172.21.15.28:0/2233327209 wait complete. 2023-11-28T11:09:06.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.182+0000 7f9850021700 1 -- 172.21.15.28:0/4285851174 shutdown_connections 2023-11-28T11:09:06.236 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.191+0000 7fc41720c700 1 -- 172.21.15.28:0/3725891536 --> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc3f0002d40 con 0x7fc3fc071ec0 2023-11-28T11:09:06.236 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.185+0000 7f9850021700 1 -- 172.21.15.28:0/4285851174 wait complete. 2023-11-28T11:09:06.236 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.192+0000 7f0efec1a700 1 -- 172.21.15.28:0/3449036368 --> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ef81e66b0 con 0x7f0ee4072090 2023-11-28T11:09:06.237 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.192+0000 7fc40d7fa700 1 -- 172.21.15.28:0/3725891536 <== osd.5 v2:172.21.15.59:6800/103696 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc3f0002d40 con 0x7fc3fc071ec0 2023-11-28T11:09:06.237 INFO:teuthology.orchestra.run.smithi028.stdout:55834574857 2023-11-28T11:09:06.238 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T11:09:06.239 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.194+0000 7fc41720c700 1 -- 172.21.15.28:0/3725891536 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x7fc3fc071ec0 msgr2=0x7fc3fc0742f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.239 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.194+0000 7fc41720c700 1 --2- 172.21.15.28:0/3725891536 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x7fc3fc071ec0 0x7fc3fc0742f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.239 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.195+0000 7fc41720c700 1 -- 172.21.15.28:0/3725891536 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3fc0682f0 msgr2=0x7fc3fc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.239 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.195+0000 7fc41720c700 1 --2- 172.21.15.28:0/3725891536 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3fc0682f0 0x7fc3fc06a7a0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fc41006ffc0 tx=0x7fc408009000 comp rx=0 tx=0).stop 2023-11-28T11:09:06.239 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.195+0000 7fc41720c700 1 -- 172.21.15.28:0/3725891536 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc410161bb0 msgr2=0x7fc41014e8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.239 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.195+0000 7fc41720c700 1 --2- 172.21.15.28:0/3725891536 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc410161bb0 0x7fc41014e8b0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fc40400b450 tx=0x7fc40400b810 comp rx=0 tx=0).stop 2023-11-28T11:09:06.239 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.195+0000 7fc4157a9700 1 -- 172.21.15.28:0/3725891536 reap_dead start 2023-11-28T11:09:06.239 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.195+0000 7fc41720c700 1 -- 172.21.15.28:0/3725891536 shutdown_connections 2023-11-28T11:09:06.239 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.195+0000 7fc41720c700 1 -- 172.21.15.28:0/3725891536 >> 172.21.15.28:0/3725891536 conn(0x7fc4100cf670 msgr2=0x7fc4100b3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.239 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.195+0000 7fc41720c700 1 -- 172.21.15.28:0/3725891536 shutdown_connections 2023-11-28T11:09:06.240 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.195+0000 7fc41720c700 1 -- 172.21.15.28:0/3725891536 wait complete. 2023-11-28T11:09:06.240 INFO:teuthology.orchestra.run.smithi028.stdout:55834574856 2023-11-28T11:09:06.241 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T11:09:06.241 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.199+0000 7fe1a97ed700 1 -- 172.21.15.28:0/2500645130 --> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe1a40cadb0 con 0x7fe190071f90 2023-11-28T11:09:06.241 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.200+0000 7f0ef57fa700 1 -- 172.21.15.28:0/3449036368 <== osd.0 v2:172.21.15.28:6800/103508 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0ef81e66b0 con 0x7f0ee4072090 2023-11-28T11:09:06.241 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.200+0000 7fe18b7fe700 1 -- 172.21.15.28:0/2500645130 <== osd.6 v2:172.21.15.59:6816/103697 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe1a40cadb0 con 0x7fe190071f90 2023-11-28T11:09:06.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.200+0000 7f4ef2dba700 1 -- 172.21.15.28:0/2075192494 --> [v2:172.21.15.28:6808/103510,v1:172.21.15.28:6809/103510] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4ed0005c40 con 0x7f4ed00015e0 2023-11-28T11:09:06.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.200+0000 7fe1a97ed700 1 -- 172.21.15.28:0/2500645130 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x7fe190071f90 msgr2=0x7fe1900743c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.200+0000 7fe1a97ed700 1 --2- 172.21.15.28:0/2500645130 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x7fe190071f90 0x7fe1900743c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.200+0000 7fe1a97ed700 1 -- 172.21.15.28:0/2500645130 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe190068340 msgr2=0x7fe19006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.200+0000 7fe1a97ed700 1 --2- 172.21.15.28:0/2500645130 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe190068340 0x7fe19006a7f0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe1a40cc720 tx=0x7fe194008030 comp rx=0 tx=0).stop 2023-11-28T11:09:06.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.200+0000 7fe1a97ed700 1 -- 172.21.15.28:0/2500645130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1a41592b0 msgr2=0x7fe1a415c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.200+0000 7fe1a97ed700 1 --2- 172.21.15.28:0/2500645130 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1a41592b0 0x7fe1a415c5b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe19800b4a0 tx=0x7fe19800b860 comp rx=0 tx=0).stop 2023-11-28T11:09:06.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.200+0000 7fe1a37fe700 1 -- 172.21.15.28:0/2500645130 reap_dead start 2023-11-28T11:09:06.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.201+0000 7fe1a97ed700 1 -- 172.21.15.28:0/2500645130 shutdown_connections 2023-11-28T11:09:06.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.201+0000 7fe1a97ed700 1 -- 172.21.15.28:0/2500645130 >> 172.21.15.28:0/2500645130 conn(0x7fe1a40ce9e0 msgr2=0x7fe1a40cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.201+0000 7fe1a97ed700 1 -- 172.21.15.28:0/2500645130 shutdown_connections 2023-11-28T11:09:06.244 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.201+0000 7fe1a97ed700 1 -- 172.21.15.28:0/2500645130 wait complete. 2023-11-28T11:09:06.244 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.201+0000 7f4ee97fa700 1 -- 172.21.15.28:0/2075192494 <== osd.3 v2:172.21.15.28:6808/103510 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4ed0005c40 con 0x7f4ed00015e0 2023-11-28T11:09:06.244 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.204+0000 7f5abcf78700 1 -- 172.21.15.28:0/2906631907 --> [v2:172.21.15.59:6801/103695,v1:172.21.15.59:6803/103695] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5a9c002d40 con 0x7f5a98072010 2023-11-28T11:09:06.244 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.205+0000 7f5aa6ffd700 1 -- 172.21.15.28:0/2906631907 <== osd.4 v2:172.21.15.59:6801/103695 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5a9c002d40 con 0x7f5a98072010 2023-11-28T11:09:06.245 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.205+0000 7f0efec1a700 1 -- 172.21.15.28:0/3449036368 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x7f0ee4072090 msgr2=0x7f0ee40744c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.245 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.205+0000 7f0efec1a700 1 --2- 172.21.15.28:0/3449036368 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x7f0ee4072090 0x7f0ee40744c0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.245 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.205+0000 7f0efec1a700 1 -- 172.21.15.28:0/3449036368 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0ee40683c0 msgr2=0x7f0ee406a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.245 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.205+0000 7f0efec1a700 1 --2- 172.21.15.28:0/3449036368 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0ee40683c0 0x7f0ee406a870 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8152b60 tx=0x7f0ef0009250 comp rx=0 tx=0).stop 2023-11-28T11:09:06.245 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.205+0000 7f0efec1a700 1 -- 172.21.15.28:0/3449036368 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ef80cd120 msgr2=0x7f0ef80b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.245 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.205+0000 7f0efec1a700 1 --2- 172.21.15.28:0/3449036368 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ef80cd120 0x7f0ef80b3ec0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0eec008db0 tx=0x7f0eec00a4d0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.245 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.205+0000 7f0efd1b7700 1 -- 172.21.15.28:0/3449036368 reap_dead start 2023-11-28T11:09:06.245 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.205+0000 7f0efec1a700 1 -- 172.21.15.28:0/3449036368 shutdown_connections 2023-11-28T11:09:06.246 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.205+0000 7f0efec1a700 1 -- 172.21.15.28:0/3449036368 >> 172.21.15.28:0/3449036368 conn(0x7f0ef80cf670 msgr2=0x7f0ef806d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.246 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.205+0000 7f0efec1a700 1 -- 172.21.15.28:0/3449036368 shutdown_connections 2023-11-28T11:09:06.246 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.205+0000 7f0efec1a700 1 -- 172.21.15.28:0/3449036368 wait complete. 2023-11-28T11:09:06.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.207+0000 7f4ef2dba700 1 -- 172.21.15.28:0/2075192494 >> [v2:172.21.15.28:6808/103510,v1:172.21.15.28:6809/103510] conn(0x7f4ed00015e0 msgr2=0x7f4ed0003a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.207+0000 7f4ef2dba700 1 --2- 172.21.15.28:0/2075192494 >> [v2:172.21.15.28:6808/103510,v1:172.21.15.28:6809/103510] conn(0x7f4ed00015e0 0x7f4ed0003a90 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.207+0000 7f4ef2dba700 1 -- 172.21.15.28:0/2075192494 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4ecc068340 msgr2=0x7f4ecc06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.207+0000 7f4ef2dba700 1 --2- 172.21.15.28:0/2075192494 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4ecc068340 0x7f4ecc06a7f0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4000c00 tx=0x7f4ed4007000 comp rx=0 tx=0).stop 2023-11-28T11:09:06.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.207+0000 7f4ef2dba700 1 -- 172.21.15.28:0/2075192494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4eec0c41d0 msgr2=0x7f4eec0c4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.207+0000 7f4ef2dba700 1 --2- 172.21.15.28:0/2075192494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4eec0c41d0 0x7f4eec0c4640 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4ee400b520 tx=0x7f4ee400b8e0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.207+0000 7f4ef1357700 1 -- 172.21.15.28:0/2075192494 reap_dead start 2023-11-28T11:09:06.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.207+0000 7f4ef2dba700 1 -- 172.21.15.28:0/2075192494 shutdown_connections 2023-11-28T11:09:06.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.207+0000 7f4ef2dba700 1 -- 172.21.15.28:0/2075192494 >> 172.21.15.28:0/2075192494 conn(0x7f4eec0ce9e0 msgr2=0x7f4eec0af990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.207+0000 7f4ef2dba700 1 -- 172.21.15.28:0/2075192494 shutdown_connections 2023-11-28T11:09:06.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.207+0000 7f4ef2dba700 1 -- 172.21.15.28:0/2075192494 wait complete. 2023-11-28T11:09:06.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.209+0000 7f5abcf78700 1 -- 172.21.15.28:0/2906631907 >> [v2:172.21.15.59:6801/103695,v1:172.21.15.59:6803/103695] conn(0x7f5a98072010 msgr2=0x7f5a98074440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.209+0000 7f5abcf78700 1 --2- 172.21.15.28:0/2906631907 >> [v2:172.21.15.59:6801/103695,v1:172.21.15.59:6803/103695] conn(0x7f5a98072010 0x7f5a98074440 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.209+0000 7f5abcf78700 1 -- 172.21.15.28:0/2906631907 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5a98068340 msgr2=0x7f5a9806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.209+0000 7f5abcf78700 1 --2- 172.21.15.28:0/2906631907 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5a98068340 0x7f5a9806a7f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5aa8000f60 tx=0x7f5aa8009040 comp rx=0 tx=0).stop 2023-11-28T11:09:06.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.209+0000 7f5abcf78700 1 -- 172.21.15.28:0/2906631907 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ab8160be0 msgr2=0x7f5ab8159ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.249 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.209+0000 7f5abcf78700 1 --2- 172.21.15.28:0/2906631907 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ab8160be0 0x7f5ab8159ea0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5aac00e870 tx=0x7f5aac00eb80 comp rx=0 tx=0).stop 2023-11-28T11:09:06.249 INFO:teuthology.orchestra.run.smithi028.stdout:55834574857 2023-11-28T11:09:06.250 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T11:09:06.250 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.211+0000 7f5ab6d9d700 1 -- 172.21.15.28:0/2906631907 reap_dead start 2023-11-28T11:09:06.250 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.211+0000 7f5abcf78700 1 -- 172.21.15.28:0/2906631907 shutdown_connections 2023-11-28T11:09:06.250 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.211+0000 7f5abcf78700 1 -- 172.21.15.28:0/2906631907 >> 172.21.15.28:0/2906631907 conn(0x7f5ab80cf670 msgr2=0x7f5ab80afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.251 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.211+0000 7f5abcf78700 1 -- 172.21.15.28:0/2906631907 shutdown_connections 2023-11-28T11:09:06.251 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.211+0000 7f5abcf78700 1 -- 172.21.15.28:0/2906631907 wait complete. 2023-11-28T11:09:06.251 INFO:teuthology.orchestra.run.smithi028.stdout:55834574857 2023-11-28T11:09:06.251 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T11:09:06.252 INFO:teuthology.orchestra.run.smithi028.stdout:55834574856 2023-11-28T11:09:06.252 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T11:09:06.253 INFO:teuthology.orchestra.run.smithi028.stdout:55834574857 2023-11-28T11:09:06.254 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T11:09:06.254 INFO:teuthology.orchestra.run.smithi028.stdout:55834574856 2023-11-28T11:09:06.255 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T11:09:06.256 INFO:teuthology.orchestra.run.smithi028.stdout:55834574857 2023-11-28T11:09:06.256 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T11:09:06.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.502+0000 7f1934ddd700 1 Processor -- start 2023-11-28T11:09:06.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.502+0000 7f1934ddd700 1 -- start start 2023-11-28T11:09:06.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.502+0000 7f01e8b30700 1 Processor -- start 2023-11-28T11:09:06.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.503+0000 7f01e8b30700 1 -- start start 2023-11-28T11:09:06.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.503+0000 7ff8e648e700 1 Processor -- start 2023-11-28T11:09:06.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.503+0000 7ff8e648e700 1 -- start start 2023-11-28T11:09:06.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f1934ddd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f193015a380 0x7f19300c2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f1934ddd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f193006d560 0x7f19300ca4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f1934ddd700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19300c9440 con 0x7f19300caa20 2023-11-28T11:09:06.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f1934ddd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19300c9720 con 0x7f193015a380 2023-11-28T11:09:06.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f1934ddd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19300c9a00 con 0x7f193006d560 2023-11-28T11:09:06.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f192ed9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f19300caa20 0x7f193006a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35146/0 (socket says 172.21.15.28:35146) 2023-11-28T11:09:06.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f192ed9d700 1 -- 172.21.15.28:0/876057981 learned_addr learned my addr 172.21.15.28:0/876057981 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:06.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f192e59c700 1 --2- 172.21.15.28:0/876057981 >> v2:172.21.15.59:3300/0 conn(0x7f193015a380 0x7f19300c2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f01e8b30700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f01e40c2530 0x7f01e406d560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f01e8b30700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01e40cabf0 0x7f01e415dfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f01e8b30700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e40c9440 con 0x7f01e415a380 2023-11-28T11:09:06.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f01e8b30700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e406daa0 con 0x7f01e40c2530 2023-11-28T11:09:06.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f01e8b30700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e40c29b0 con 0x7f01e40cabf0 2023-11-28T11:09:06.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f8aacf25700 1 Processor -- start 2023-11-28T11:09:06.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f8aacf25700 1 -- start start 2023-11-28T11:09:06.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7ff8e648e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e006a1e0 0x7ff8e00ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7ff8e648e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff8e00cce20 0x7ff8e006a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7ff8e648e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8e00cd2b0 con 0x7ff8e00c9a40 2023-11-28T11:09:06.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7ff8e648e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8e015cb10 con 0x7ff8e00cce20 2023-11-28T11:09:06.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7ff8e648e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e00ce890 con 0x7ff8e006a1e0 2023-11-28T11:09:06.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.504+0000 7f88d8470700 1 Processor -- start 2023-11-28T11:09:06.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7ff8dffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e006a1e0 0x7ff8e00ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7ff8dffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e006a1e0 0x7ff8e00ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:52858/0 (socket says 172.21.15.28:52858) 2023-11-28T11:09:06.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7ff8dffff700 1 -- 172.21.15.28:0/3835698348 learned_addr learned my addr 172.21.15.28:0/3835698348 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7f01e1d9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f01e40c2530 0x7f01e406d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7f01e1d9b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f01e40c2530 0x7f01e406d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:34388/0 (socket says 172.21.15.28:34388) 2023-11-28T11:09:06.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7f01e1d9b700 1 -- 172.21.15.28:0/1180045477 learned_addr learned my addr 172.21.15.28:0/1180045477 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7ff8df7fe700 1 --2- 172.21.15.28:0/3835698348 >> v2:172.21.15.59:3300/0 conn(0x7ff8e00cce20 0x7ff8e006a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7ff8dffff700 1 -- 172.21.15.28:0/3835698348 >> v2:172.21.15.59:3300/0 conn(0x7ff8e00cce20 msgr2=0x7ff8e006a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7ff8dffff700 1 --2- 172.21.15.28:0/3835698348 >> v2:172.21.15.59:3300/0 conn(0x7ff8e00cce20 0x7ff8e006a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7f192dd9b700 1 --2- 172.21.15.28:0/876057981 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f193006d560 0x7f19300ca4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7f192d59a700 1 -- 172.21.15.28:0/876057981 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1004538946 0 0) 0x7f19300c9440 con 0x7f19300caa20 2023-11-28T11:09:06.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7f192d59a700 1 -- 172.21.15.28:0/876057981 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19180035b0 con 0x7f19300caa20 2023-11-28T11:09:06.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7f192e59c700 1 -- 172.21.15.28:0/876057981 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f193006d560 msgr2=0x7f19300ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7f192e59c700 1 --2- 172.21.15.28:0/876057981 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f193006d560 0x7f19300ca4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7f192e59c700 1 -- 172.21.15.28:0/876057981 >> v1:172.21.15.28:6789/0 conn(0x7f19300caa20 legacy=0x7f193006a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7f192e59c700 1 -- 172.21.15.28:0/876057981 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19300c9ce0 con 0x7f193015a380 2023-11-28T11:09:06.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7ff8deffd700 1 -- 172.21.15.28:0/3835698348 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 351469210 0 0) 0x7ff8e00cd2b0 con 0x7ff8e00c9a40 2023-11-28T11:09:06.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7ff8dffff700 1 -- 172.21.15.28:0/3835698348 >> v1:172.21.15.59:6789/0 conn(0x7ff8e00c9a40 legacy=0x7ff8e006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7ff8dffff700 1 -- 172.21.15.28:0/3835698348 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8e00cd590 con 0x7ff8e006a1e0 2023-11-28T11:09:06.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7f192e59c700 1 --2- 172.21.15.28:0/876057981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193015a380 0x7f19300c2530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1920009840 tx=0x7f1920009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2a3cf7bf6893246 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f192d59a700 1 -- 172.21.15.28:0/876057981 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920011830 con 0x7f193015a380 2023-11-28T11:09:06.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7ff8dffff700 1 --2- 172.21.15.28:0/3835698348 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e006a1e0 0x7ff8e00ce350 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00098f0 tx=0x7ff8d0009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9547e8d272ef6904 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f192d59a700 1 -- 172.21.15.28:0/876057981 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1920011990 con 0x7f193015a380 2023-11-28T11:09:06.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f192d59a700 1 -- 172.21.15.28:0/876057981 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920011d20 con 0x7f193015a380 2023-11-28T11:09:06.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7ff8deffd700 1 -- 172.21.15.28:0/3835698348 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d000b8a0 con 0x7ff8e006a1e0 2023-11-28T11:09:06.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f1934ddd700 1 -- 172.21.15.28:0/876057981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193015a380 msgr2=0x7f19300c2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f1934ddd700 1 --2- 172.21.15.28:0/876057981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193015a380 0x7f19300c2530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1920009840 tx=0x7f1920009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:06.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7fb69a4d4700 1 Processor -- start 2023-11-28T11:09:06.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7fb69a4d4700 1 -- start start 2023-11-28T11:09:06.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f8aacf25700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8aa8160c90 0x7f8aa80ce320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f8aacf25700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8aa80cca20 0x7f8aa80c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f8aacf25700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aa80ca3a0 con 0x7f8aa80c2b60 2023-11-28T11:09:06.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f8aacf25700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa8161410 con 0x7f8aa80cca20 2023-11-28T11:09:06.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f8aacf25700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa81616f0 con 0x7f8aa8160c90 2023-11-28T11:09:06.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f8aa659c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8aa8160c90 0x7f8aa80ce320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f8aa659c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8aa8160c90 0x7f8aa80ce320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:52864/0 (socket says 172.21.15.28:52864) 2023-11-28T11:09:06.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f8aa659c700 1 -- 172.21.15.28:0/1384571472 learned_addr learned my addr 172.21.15.28:0/1384571472 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f8aa659c700 1 -- 172.21.15.28:0/1384571472 >> v2:172.21.15.28:3300/0 conn(0x7f8aa80cca20 msgr2=0x7f8aa80c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f8aa659c700 1 --2- 172.21.15.28:0/1384571472 >> v2:172.21.15.28:3300/0 conn(0x7f8aa80cca20 0x7f8aa80c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f8aa659c700 1 -- 172.21.15.28:0/1384571472 >> v1:172.21.15.59:6789/0 conn(0x7f8aa80c2b60 legacy=0x7f8aa80cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f8aa659c700 1 -- 172.21.15.28:0/1384571472 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa81619d0 con 0x7f8aa8160c90 2023-11-28T11:09:06.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f01e2d9d700 1 --2- 172.21.15.28:0/1180045477 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01e40cabf0 0x7f01e415dfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7ff8deffd700 1 -- 172.21.15.28:0/3835698348 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d000ba00 con 0x7ff8e006a1e0 2023-11-28T11:09:06.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7ff8deffd700 1 -- 172.21.15.28:0/3835698348 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d000bd90 con 0x7ff8e006a1e0 2023-11-28T11:09:06.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7ff8e648e700 1 -- 172.21.15.28:0/3835698348 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e006a1e0 msgr2=0x7ff8e00ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7ff8e648e700 1 --2- 172.21.15.28:0/3835698348 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e006a1e0 0x7ff8e00ce350 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7ff8d00098f0 tx=0x7ff8d0009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:06.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.505+0000 7f88d8470700 1 -- start start 2023-11-28T11:09:06.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f88d8470700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d0159590 0x7f88d006e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f88d8470700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f88d015db70 0x7f88d00ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f88d8470700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d015d120 con 0x7f88d00ce890 2023-11-28T11:09:06.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f88d8470700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d015d6e0 con 0x7f88d015db70 2023-11-28T11:09:06.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.506+0000 7f88d8470700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00aef60 con 0x7f88d0159590 2023-11-28T11:09:06.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f88d620c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d0159590 0x7f88d006e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f88d620c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d0159590 0x7f88d006e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:52872/0 (socket says 172.21.15.28:52872) 2023-11-28T11:09:06.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f88d620c700 1 -- 172.21.15.28:0/2019993879 learned_addr learned my addr 172.21.15.28:0/2019993879 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f01e1d9b700 1 -- 172.21.15.28:0/1180045477 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01e40cabf0 msgr2=0x7f01e415dfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f01e1d9b700 1 --2- 172.21.15.28:0/1180045477 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01e40cabf0 0x7f01e415dfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f01e1d9b700 1 -- 172.21.15.28:0/1180045477 >> v1:172.21.15.59:6789/0 conn(0x7f01e415a380 legacy=0x7f01e40ca4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f01e1d9b700 1 -- 172.21.15.28:0/1180045477 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e40c9720 con 0x7f01e40c2530 2023-11-28T11:09:06.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7fe4a072f700 1 Processor -- start 2023-11-28T11:09:06.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f1934ddd700 1 -- 172.21.15.28:0/876057981 shutdown_connections 2023-11-28T11:09:06.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f1934ddd700 1 --2- 172.21.15.28:0/876057981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193015a380 0x7f19300c2530 secure :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1920009840 tx=0x7f1920009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:06.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f1934ddd700 1 --2- 172.21.15.28:0/876057981 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f193006d560 0x7f19300ca4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f1934ddd700 1 -- 172.21.15.28:0/876057981 >> 172.21.15.28:0/876057981 conn(0x7f19300cf670 msgr2=0x7f19300cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7ff8e648e700 1 -- 172.21.15.28:0/3835698348 shutdown_connections 2023-11-28T11:09:06.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7ff8e648e700 1 --2- 172.21.15.28:0/3835698348 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e006a1e0 0x7ff8e00ce350 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7ff8e648e700 1 --2- 172.21.15.28:0/3835698348 >> v2:172.21.15.59:3300/0 conn(0x7ff8e00cce20 0x7ff8e006a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7ff8e648e700 1 -- 172.21.15.28:0/3835698348 >> 172.21.15.28:0/3835698348 conn(0x7ff8e00cf670 msgr2=0x7ff8e00cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f8aa659c700 1 --2- 172.21.15.28:0/1384571472 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8aa8160c90 0x7f8aa80ce320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0098f0 tx=0x7f8a9c009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a4e7bc74c06ecd84 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f01e1d9b700 1 --2- 172.21.15.28:0/1180045477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e40c2530 0x7f01e406d560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f01d80098f0 tx=0x7f01d8009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3a5a14e2f6e5002 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7ff8e648e700 1 -- 172.21.15.28:0/3835698348 shutdown_connections 2023-11-28T11:09:06.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f88d620c700 1 -- 172.21.15.28:0/2019993879 >> v2:172.21.15.28:3300/0 conn(0x7f88d015db70 msgr2=0x7f88d00ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f88d620c700 1 --2- 172.21.15.28:0/2019993879 >> v2:172.21.15.28:3300/0 conn(0x7f88d015db70 0x7f88d00ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f88d620c700 1 -- 172.21.15.28:0/2019993879 >> v1:172.21.15.59:6789/0 conn(0x7f88d00ce890 legacy=0x7f88d00caf50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:09:06.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f88d620c700 1 -- 172.21.15.28:0/2019993879 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d00af240 con 0x7f88d0159590 2023-11-28T11:09:06.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f01e159a700 1 -- 172.21.15.28:0/1180045477 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8004230 con 0x7f01e40c2530 2023-11-28T11:09:06.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f88d620c700 1 --2- 172.21.15.28:0/2019993879 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d0159590 0x7f88d006e400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f88c000bb90 tx=0x7f88c000bea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eede4051c72c54dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f01e159a700 1 -- 172.21.15.28:0/1180045477 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d8004390 con 0x7f01e40c2530 2023-11-28T11:09:06.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f01e159a700 1 -- 172.21.15.28:0/1180045477 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d80036a0 con 0x7f01e40c2530 2023-11-28T11:09:06.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7f88d520a700 1 -- 172.21.15.28:0/2019993879 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c000dd10 con 0x7f88d0159590 2023-11-28T11:09:06.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7ff8e648e700 1 -- 172.21.15.28:0/3835698348 wait complete. 2023-11-28T11:09:06.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7fb69a4d4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb69415df40 0x7fb6940adf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7fb69a4d4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb694071070 0x7fb6940cc990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7fb69a4d4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb69415a7e0 con 0x7fb69415a410 2023-11-28T11:09:06.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7fb69a4d4700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb69415e3c0 con 0x7fb694071070 2023-11-28T11:09:06.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7fb69a4d4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69415e6a0 con 0x7fb69415df40 2023-11-28T11:09:06.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7fb6937fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb69415df40 0x7fb6940adf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7fb6937fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb69415df40 0x7fb6940adf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:52878/0 (socket says 172.21.15.28:52878) 2023-11-28T11:09:06.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7fb6937fe700 1 -- 172.21.15.28:0/847018377 learned_addr learned my addr 172.21.15.28:0/847018377 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7fb698a71700 1 --2- 172.21.15.28:0/847018377 >> v2:172.21.15.59:3300/0 conn(0x7fb694071070 0x7fb6940cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7f01e8b30700 1 -- 172.21.15.28:0/1180045477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e40c2530 msgr2=0x7f01e406d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7f01e8b30700 1 --2- 172.21.15.28:0/1180045477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e40c2530 0x7f01e406d560 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f01d80098f0 tx=0x7f01d8009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:06.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7f88d520a700 1 -- 172.21.15.28:0/2019993879 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c000de70 con 0x7f88d0159590 2023-11-28T11:09:06.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7f88d520a700 1 -- 172.21.15.28:0/2019993879 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c0004230 con 0x7f88d0159590 2023-11-28T11:09:06.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f88d8470700 1 -- 172.21.15.28:0/2019993879 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d0159590 msgr2=0x7f88d006e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f88d8470700 1 --2- 172.21.15.28:0/2019993879 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d0159590 0x7f88d006e400 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f88c000bb90 tx=0x7f88c000bea0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7fe4a072f700 1 -- start start 2023-11-28T11:09:06.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7fe4a072f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4981588a0 0x7fe4980ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7fe4a072f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe4980c9920 0x7fe4980c9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fe4a072f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe49806a280 con 0x7fe4980cd120 2023-11-28T11:09:06.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fe4a072f700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4980ca2e0 con 0x7fe4980c9920 2023-11-28T11:09:06.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fe4a072f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49806a560 con 0x7fe4981588a0 2023-11-28T11:09:06.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fe49dcca700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fe4980cd120 0x7fe49806b320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:45968/0 (socket says 172.21.15.28:45968) 2023-11-28T11:09:06.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fe49dcca700 1 -- 172.21.15.28:0/189056047 learned_addr learned my addr 172.21.15.28:0/189056047 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:06.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fb6937fe700 1 -- 172.21.15.28:0/847018377 >> v2:172.21.15.59:3300/0 conn(0x7fb694071070 msgr2=0x7fb6940cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fb6937fe700 1 --2- 172.21.15.28:0/847018377 >> v2:172.21.15.59:3300/0 conn(0x7fb694071070 0x7fb6940cc990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fb6937fe700 1 -- 172.21.15.28:0/847018377 >> v1:172.21.15.59:6789/0 conn(0x7fb69415a410 legacy=0x7fb694070960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:09:06.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fb6937fe700 1 -- 172.21.15.28:0/847018377 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb69415e980 con 0x7fb69415df40 2023-11-28T11:09:06.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7ff8e648e700 1 Processor -- start 2023-11-28T11:09:06.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f88d8470700 1 -- 172.21.15.28:0/2019993879 shutdown_connections 2023-11-28T11:09:06.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f88d8470700 1 --2- 172.21.15.28:0/2019993879 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d0159590 0x7f88d006e400 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f88d8470700 1 --2- 172.21.15.28:0/2019993879 >> v2:172.21.15.28:3300/0 conn(0x7f88d015db70 0x7f88d00ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f88d8470700 1 -- 172.21.15.28:0/2019993879 >> 172.21.15.28:0/2019993879 conn(0x7f88d00b1650 msgr2=0x7f88d00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fe49e4cb700 1 --2- 172.21.15.28:0/189056047 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4981588a0 0x7fe4980ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fe49eccc700 1 --2- 172.21.15.28:0/189056047 >> v2:172.21.15.28:3300/0 conn(0x7fe4980c9920 0x7fe4980c9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f8aa559a700 1 -- 172.21.15.28:0/1384571472 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c00b8a0 con 0x7f8aa8160c90 2023-11-28T11:09:06.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f8aa559a700 1 -- 172.21.15.28:0/1384571472 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9c00ba00 con 0x7f8aa8160c90 2023-11-28T11:09:06.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f8aa559a700 1 -- 172.21.15.28:0/1384571472 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c00bd90 con 0x7f8aa8160c90 2023-11-28T11:09:06.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f1934ddd700 1 -- 172.21.15.28:0/876057981 shutdown_connections 2023-11-28T11:09:06.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f1934ddd700 1 -- 172.21.15.28:0/876057981 wait complete. 2023-11-28T11:09:06.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7ff8e648e700 1 -- start start 2023-11-28T11:09:06.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fe49e4cb700 1 -- 172.21.15.28:0/189056047 >> v2:172.21.15.28:3300/0 conn(0x7fe4980c9920 msgr2=0x7fe4980c9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fe49e4cb700 1 --2- 172.21.15.28:0/189056047 >> v2:172.21.15.28:3300/0 conn(0x7fe4980c9920 0x7fe4980c9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fe49e4cb700 1 -- 172.21.15.28:0/189056047 >> v1:172.21.15.59:6789/0 conn(0x7fe4980cd120 legacy=0x7fe49806b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fe49e4cb700 1 -- 172.21.15.28:0/189056047 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49806a840 con 0x7fe4981588a0 2023-11-28T11:09:06.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f88d8470700 1 -- 172.21.15.28:0/2019993879 shutdown_connections 2023-11-28T11:09:06.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7fe49e4cb700 1 --2- 172.21.15.28:0/189056047 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4981588a0 0x7fe4980ce010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe494009870 tx=0x7fe494009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41a319ec6f6a8c24 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7f01e8b30700 1 -- 172.21.15.28:0/1180045477 shutdown_connections 2023-11-28T11:09:06.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7f01e8b30700 1 --2- 172.21.15.28:0/1180045477 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01e40cabf0 0x7f01e415dfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7f01e8b30700 1 --2- 172.21.15.28:0/1180045477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e40c2530 0x7f01e406d560 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7f01e8b30700 1 -- 172.21.15.28:0/1180045477 >> 172.21.15.28:0/1180045477 conn(0x7f01e40cf670 msgr2=0x7f01e40cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7fb6937fe700 1 --2- 172.21.15.28:0/847018377 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb69415df40 0x7fb6940adf90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009e50 tx=0x7fb68c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f2a017220e37b39 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7fb692ffd700 1 -- 172.21.15.28:0/847018377 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c00e070 con 0x7fb69415df40 2023-11-28T11:09:06.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7fb692ffd700 1 -- 172.21.15.28:0/847018377 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb68c003a20 con 0x7fb69415df40 2023-11-28T11:09:06.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7fb692ffd700 1 -- 172.21.15.28:0/847018377 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb68c003dd0 con 0x7fb69415df40 2023-11-28T11:09:06.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7f8aacf25700 1 -- 172.21.15.28:0/1384571472 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8aa8160c90 msgr2=0x7f8aa80ce320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7f8aacf25700 1 --2- 172.21.15.28:0/1384571472 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8aa8160c90 0x7f8aa80ce320 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0098f0 tx=0x7f8a9c009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:06.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7f01e8b30700 1 -- 172.21.15.28:0/1180045477 shutdown_connections 2023-11-28T11:09:06.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7f01e8b30700 1 -- 172.21.15.28:0/1180045477 wait complete. 2023-11-28T11:09:06.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7ff8e648e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e006a1e0 0x7ff8e00c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7ff8e648e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8e00c9a40 0x7ff8e01f6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7ff8e648e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e00cce20 0x7ff8e00c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7ff8e648e700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e00c3dd0 con 0x7ff8e00c9a40 2023-11-28T11:09:06.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7ff8e648e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e01f6890 con 0x7ff8e00cce20 2023-11-28T11:09:06.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7ff8e648e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8e01f6580 con 0x7ff8e006a1e0 2023-11-28T11:09:06.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7ff8df7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8e00c9a40 0x7ff8e01f6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7ff8df7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8e00c9a40 0x7ff8e01f6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47140/0 (socket says 172.21.15.28:47140) 2023-11-28T11:09:06.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7ff8df7fe700 1 -- 172.21.15.28:0/1705659144 learned_addr learned my addr 172.21.15.28:0/1705659144 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7ff8dffff700 1 --2- 172.21.15.28:0/1705659144 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e006a1e0 0x7ff8e00c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7ff8df7fe700 1 -- 172.21.15.28:0/1705659144 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e006a1e0 msgr2=0x7ff8e00c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7ff8df7fe700 1 --2- 172.21.15.28:0/1705659144 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e006a1e0 0x7ff8e00c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7ff8df7fe700 1 -- 172.21.15.28:0/1705659144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e00cce20 msgr2=0x7ff8e00c75f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:09:06.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7ff8df7fe700 1 --2- 172.21.15.28:0/1705659144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e00cce20 0x7ff8e00c75f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7ff8df7fe700 1 -- 172.21.15.28:0/1705659144 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d0009550 con 0x7ff8e00c9a40 2023-11-28T11:09:06.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7fb69a4d4700 1 -- 172.21.15.28:0/847018377 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb69415df40 msgr2=0x7fb6940adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7fb69a4d4700 1 --2- 172.21.15.28:0/847018377 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb69415df40 0x7fb6940adf90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fb68c009e50 tx=0x7fb68c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:06.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7fb69a4d4700 1 -- 172.21.15.28:0/847018377 shutdown_connections 2023-11-28T11:09:06.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7fb69a4d4700 1 --2- 172.21.15.28:0/847018377 >> v2:172.21.15.59:3300/0 conn(0x7fb694071070 0x7fb6940cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7fb69a4d4700 1 --2- 172.21.15.28:0/847018377 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb69415df40 0x7fb6940adf90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7fb69a4d4700 1 -- 172.21.15.28:0/847018377 >> 172.21.15.28:0/847018377 conn(0x7fb6940ce9e0 msgr2=0x7fb6940cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7fb69a4d4700 1 -- 172.21.15.28:0/847018377 shutdown_connections 2023-11-28T11:09:06.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7fb69a4d4700 1 -- 172.21.15.28:0/847018377 wait complete. 2023-11-28T11:09:06.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7f1934ddd700 1 Processor -- start 2023-11-28T11:09:06.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.511+0000 7ff8df7fe700 1 --2- 172.21.15.28:0/1705659144 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8e00c9a40 0x7ff8e01f6d20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400d830 tx=0x7ff8d400db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.512+0000 7ff8dcff9700 1 -- 172.21.15.28:0/1705659144 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d40125f0 con 0x7ff8e00c9a40 2023-11-28T11:09:06.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.512+0000 7ff8e648e700 1 -- 172.21.15.28:0/1705659144 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8e00cdb50 con 0x7ff8e00c9a40 2023-11-28T11:09:06.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.512+0000 7f1934ddd700 1 -- start start 2023-11-28T11:09:06.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7f1934ddd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193006d560 0x7f19300c4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7f1934ddd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19300caa20 0x7f193015bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7f1934ddd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f193015aa30 0x7f19300c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7f1934ddd700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193015aec0 con 0x7f19300caa20 2023-11-28T11:09:06.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7f1934ddd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f193015b1a0 con 0x7f193006d560 2023-11-28T11:09:06.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7f1934ddd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f193015b480 con 0x7f193015aa30 2023-11-28T11:09:06.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7f192ed9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f193015aa30 0x7f19300c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7f192dd9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19300caa20 0x7f193015bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7f192dd9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19300caa20 0x7f193015bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47144/0 (socket says 172.21.15.28:47144) 2023-11-28T11:09:06.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7f192dd9b700 1 -- 172.21.15.28:0/3042041135 learned_addr learned my addr 172.21.15.28:0/3042041135 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f192e59c700 1 --2- 172.21.15.28:0/3042041135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193006d560 0x7f19300c4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f192ed9d700 1 -- 172.21.15.28:0/3042041135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193006d560 msgr2=0x7f19300c4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f192ed9d700 1 --2- 172.21.15.28:0/3042041135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193006d560 0x7f19300c4d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f192ed9d700 1 -- 172.21.15.28:0/3042041135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19300caa20 msgr2=0x7f193015bf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f192ed9d700 1 --2- 172.21.15.28:0/3042041135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19300caa20 0x7f193015bf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f192ed9d700 1 -- 172.21.15.28:0/3042041135 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19200094a0 con 0x7f193015aa30 2023-11-28T11:09:06.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7fb69a4d4700 1 Processor -- start 2023-11-28T11:09:06.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7fb69a4d4700 1 -- start start 2023-11-28T11:09:06.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7f8aacf25700 1 -- 172.21.15.28:0/1384571472 shutdown_connections 2023-11-28T11:09:06.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7f8aacf25700 1 --2- 172.21.15.28:0/1384571472 >> v2:172.21.15.28:3300/0 conn(0x7f8aa80cca20 0x7f8aa80c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7f8aacf25700 1 --2- 172.21.15.28:0/1384571472 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8aa8160c90 0x7f8aa80ce320 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7f8aacf25700 1 -- 172.21.15.28:0/1384571472 >> 172.21.15.28:0/1384571472 conn(0x7f8aa80b1650 msgr2=0x7f8aa80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.512+0000 7f01e8b30700 1 Processor -- start 2023-11-28T11:09:06.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.512+0000 7f01e8b30700 1 -- start start 2023-11-28T11:09:06.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e8b30700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01e40c2530 0x7f01e4156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e8b30700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01e40cabf0 0x7f01e40b3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e8b30700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e415a380 0x7f01e40b36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e8b30700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e40c61d0 con 0x7f01e40c2530 2023-11-28T11:09:06.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e8b30700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e40c64b0 con 0x7f01e415a380 2023-11-28T11:09:06.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e8b30700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e40c6790 con 0x7f01e40cabf0 2023-11-28T11:09:06.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e1d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01e40cabf0 0x7f01e40b3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e415a380 0x7f01e40b36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e1d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01e40cabf0 0x7f01e40b3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:52910/0 (socket says 172.21.15.28:52910) 2023-11-28T11:09:06.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e1d9b700 1 -- 172.21.15.28:0/3407113172 learned_addr learned my addr 172.21.15.28:0/3407113172 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e259c700 1 --2- 172.21.15.28:0/3407113172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01e40c2530 0x7f01e4156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.512+0000 7ff8e648e700 1 -- 172.21.15.28:0/1705659144 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8e00cd870 con 0x7ff8e00c9a40 2023-11-28T11:09:06.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.512+0000 7ff8dcff9700 1 -- 172.21.15.28:0/1705659144 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8d4012750 con 0x7ff8e00c9a40 2023-11-28T11:09:06.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.512+0000 7ff8dcff9700 1 -- 172.21.15.28:0/1705659144 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8d4012b00 con 0x7ff8e00c9a40 2023-11-28T11:09:06.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7ff8e648e700 1 -- 172.21.15.28:0/1705659144 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8c00052f0 con 0x7ff8e00c9a40 2023-11-28T11:09:06.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.507+0000 7f27e7915700 1 Processor -- start 2023-11-28T11:09:06.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.508+0000 7f27e7915700 1 -- start start 2023-11-28T11:09:06.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f27e7915700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27e006a1e0 0x7f27e00c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f27e7915700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f27e006b5c0 0x7f27e006d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f27e7915700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27e0169750 con 0x7f27e0089580 2023-11-28T11:09:06.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f27e7915700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27e00ca0d0 con 0x7f27e006b5c0 2023-11-28T11:09:06.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f27e7915700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e00cd9d0 con 0x7f27e006a1e0 2023-11-28T11:09:06.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f27e56b1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27e006a1e0 0x7f27e00c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f27e56b1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27e006a1e0 0x7f27e00c9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:52888/0 (socket says 172.21.15.28:52888) 2023-11-28T11:09:06.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f27e56b1700 1 -- 172.21.15.28:0/1213135084 learned_addr learned my addr 172.21.15.28:0/1213135084 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f27e56b1700 1 -- 172.21.15.28:0/1213135084 >> v2:172.21.15.28:3300/0 conn(0x7f27e006b5c0 msgr2=0x7f27e006d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.509+0000 7f27e5eb2700 1 --2- 172.21.15.28:0/1213135084 >> v2:172.21.15.28:3300/0 conn(0x7f27e006b5c0 0x7f27e006d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f27e56b1700 1 --2- 172.21.15.28:0/1213135084 >> v2:172.21.15.28:3300/0 conn(0x7f27e006b5c0 0x7f27e006d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f27e56b1700 1 -- 172.21.15.28:0/1213135084 >> v1:172.21.15.28:6789/0 conn(0x7f27e0089580 legacy=0x7f27e006cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:06.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f27e56b1700 1 -- 172.21.15.28:0/1213135084 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e00cdcb0 con 0x7f27e006a1e0 2023-11-28T11:09:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f27e5eb2700 1 --2- 172.21.15.28:0/1213135084 >> v2:172.21.15.28:3300/0 conn(0x7f27e006b5c0 0x7f27e006d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f27e56b1700 1 --2- 172.21.15.28:0/1213135084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27e006a1e0 0x7f27e00c9a40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0078e0 tx=0x7f27dc007bf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=190e32f98233d7d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.510+0000 7f88d8470700 1 -- 172.21.15.28:0/2019993879 wait complete. 2023-11-28T11:09:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.512+0000 7f88d8470700 1 Processor -- start 2023-11-28T11:09:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7f88d8470700 1 -- start start 2023-11-28T11:09:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f88d8470700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d00ce890 0x7f88d00cc320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f88d8470700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88d0159590 0x7f88d0158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f88d8470700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d015db70 0x7f88d01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f88d8470700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d0152800 con 0x7f88d0159590 2023-11-28T11:09:06.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f88d8470700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00b3770 con 0x7f88d00ce890 2023-11-28T11:09:06.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f88d8470700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d00b2af0 con 0x7f88d015db70 2023-11-28T11:09:06.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.512+0000 7fe49d4c9700 1 -- 172.21.15.28:0/189056047 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49400c880 con 0x7fe4981588a0 2023-11-28T11:09:06.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.512+0000 7fe49d4c9700 1 -- 172.21.15.28:0/189056047 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49400c9e0 con 0x7fe4981588a0 2023-11-28T11:09:06.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.512+0000 7fe49d4c9700 1 -- 172.21.15.28:0/189056047 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe49400cd50 con 0x7fe4981588a0 2023-11-28T11:09:06.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7fe4a072f700 1 -- 172.21.15.28:0/189056047 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4981588a0 msgr2=0x7fe4980ce010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.513+0000 7fe4a072f700 1 --2- 172.21.15.28:0/189056047 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4981588a0 0x7fe4980ce010 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe494009870 tx=0x7fe494009b80 comp rx=0 tx=0).stop 2023-11-28T11:09:06.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7fe4a072f700 1 -- 172.21.15.28:0/189056047 shutdown_connections 2023-11-28T11:09:06.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7fe4a072f700 1 --2- 172.21.15.28:0/189056047 >> v2:172.21.15.28:3300/0 conn(0x7fe4980c9920 0x7fe4980c9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7fe4a072f700 1 --2- 172.21.15.28:0/189056047 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4981588a0 0x7fe4980ce010 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7fe4a072f700 1 -- 172.21.15.28:0/189056047 >> 172.21.15.28:0/189056047 conn(0x7fe4980cf670 msgr2=0x7fe4980cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7fb69a4d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb694071070 0x7fb69414a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f8aacf25700 1 -- 172.21.15.28:0/1384571472 shutdown_connections 2023-11-28T11:09:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7fb69a4d4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb69415a410 0x7fb69414ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7fb69a4d4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb69415a9a0 0x7fb69415bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7fb69a4d4700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69415ba40 con 0x7fb69415a9a0 2023-11-28T11:09:06.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7fb69a4d4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6940b3560 con 0x7fb694071070 2023-11-28T11:09:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e2d9d700 1 -- 172.21.15.28:0/3407113172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01e40cabf0 msgr2=0x7f01e40b3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e2d9d700 1 --2- 172.21.15.28:0/3407113172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01e40cabf0 0x7f01e40b3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e2d9d700 1 -- 172.21.15.28:0/3407113172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01e40c2530 msgr2=0x7f01e4156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e2d9d700 1 --2- 172.21.15.28:0/3407113172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01e40c2530 0x7f01e4156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7fb69a4d4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb69414c8e0 con 0x7fb69415a410 2023-11-28T11:09:06.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f01e2d9d700 1 -- 172.21.15.28:0/3407113172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d8009550 con 0x7f01e415a380 2023-11-28T11:09:06.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7f192ed9d700 1 --2- 172.21.15.28:0/3042041135 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f193015aa30 0x7f19300c3ec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f192800caa0 tx=0x7f192800cdb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7f27d7fff700 1 -- 172.21.15.28:0/1213135084 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc00f070 con 0x7f27e006a1e0 2023-11-28T11:09:06.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7f191effd700 1 -- 172.21.15.28:0/3042041135 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192800a9e0 con 0x7f193015aa30 2023-11-28T11:09:06.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7f191effd700 1 -- 172.21.15.28:0/3042041135 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192800ab40 con 0x7f193015aa30 2023-11-28T11:09:06.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7f191effd700 1 -- 172.21.15.28:0/3042041135 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1928012400 con 0x7f193015aa30 2023-11-28T11:09:06.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.514+0000 7fb693fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb694071070 0x7fb69414a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7fb693fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb694071070 0x7fb69414a6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:34450/0 (socket says 172.21.15.28:34450) 2023-11-28T11:09:06.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7fb693fff700 1 -- 172.21.15.28:0/2336017823 learned_addr learned my addr 172.21.15.28:0/2336017823 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7f8aacf25700 1 -- 172.21.15.28:0/1384571472 wait complete. 2023-11-28T11:09:06.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7f27d7fff700 1 -- 172.21.15.28:0/1213135084 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27dc004030 con 0x7f27e006a1e0 2023-11-28T11:09:06.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7f27d7fff700 1 -- 172.21.15.28:0/1213135084 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27dc0043c0 con 0x7f27e006a1e0 2023-11-28T11:09:06.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7fe4a072f700 1 -- 172.21.15.28:0/189056047 shutdown_connections 2023-11-28T11:09:06.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7fb693fff700 1 -- 172.21.15.28:0/2336017823 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb69415a410 msgr2=0x7fb69414ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7fb693fff700 1 --2- 172.21.15.28:0/2336017823 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb69415a410 0x7fb69414ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7fb693fff700 1 -- 172.21.15.28:0/2336017823 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb69415a9a0 msgr2=0x7fb69415bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7fb693fff700 1 --2- 172.21.15.28:0/2336017823 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb69415a9a0 0x7fb69415bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7fb693fff700 1 -- 172.21.15.28:0/2336017823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb68c009b30 con 0x7fb694071070 2023-11-28T11:09:06.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7fe4a072f700 1 -- 172.21.15.28:0/189056047 wait complete. 2023-11-28T11:09:06.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7fb693fff700 1 --2- 172.21.15.28:0/2336017823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb694071070 0x7fb69414a6c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb68400b550 tx=0x7fb68400b910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7fb690ff9700 1 -- 172.21.15.28:0/2336017823 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684011830 con 0x7fb694071070 2023-11-28T11:09:06.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7fb690ff9700 1 -- 172.21.15.28:0/2336017823 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6840119b0 con 0x7fb694071070 2023-11-28T11:09:06.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.515+0000 7fb690ff9700 1 -- 172.21.15.28:0/2336017823 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb684011d60 con 0x7fb694071070 2023-11-28T11:09:06.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f27e7915700 1 -- 172.21.15.28:0/1213135084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27e006a1e0 msgr2=0x7f27e00c9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f27e7915700 1 --2- 172.21.15.28:0/1213135084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27e006a1e0 0x7f27e00c9a40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f27dc0078e0 tx=0x7f27dc007bf0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f27e7915700 1 -- 172.21.15.28:0/1213135084 shutdown_connections 2023-11-28T11:09:06.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f27e7915700 1 --2- 172.21.15.28:0/1213135084 >> v2:172.21.15.28:3300/0 conn(0x7f27e006b5c0 0x7f27e006d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f27e7915700 1 --2- 172.21.15.28:0/1213135084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27e006a1e0 0x7f27e00c9a40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f27e7915700 1 -- 172.21.15.28:0/1213135084 >> 172.21.15.28:0/1213135084 conn(0x7f27e00cf670 msgr2=0x7f27e00cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f27e7915700 1 -- 172.21.15.28:0/1213135084 shutdown_connections 2023-11-28T11:09:06.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f27e7915700 1 -- 172.21.15.28:0/1213135084 wait complete. 2023-11-28T11:09:06.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f01e2d9d700 1 --2- 172.21.15.28:0/3407113172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e415a380 0x7f01e40b36a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00efc0 tx=0x7f01dc00c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7fb69a4d4700 1 -- 172.21.15.28:0/2336017823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6940adbd0 con 0x7fb694071070 2023-11-28T11:09:06.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7fb69a4d4700 1 -- 172.21.15.28:0/2336017823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69415ec60 con 0x7fb694071070 2023-11-28T11:09:06.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f8aacf25700 1 Processor -- start 2023-11-28T11:09:06.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f01d2ffd700 1 -- 172.21.15.28:0/3407113172 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc00cc90 con 0x7f01e415a380 2023-11-28T11:09:06.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f01d2ffd700 1 -- 172.21.15.28:0/3407113172 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01dc00f040 con 0x7f01e415a380 2023-11-28T11:09:06.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f01d2ffd700 1 -- 172.21.15.28:0/3407113172 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01dc0146c0 con 0x7f01e415a380 2023-11-28T11:09:06.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f8aacf25700 1 -- start start 2023-11-28T11:09:06.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.517+0000 7fe4a072f700 1 Processor -- start 2023-11-28T11:09:06.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f88d6a0d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d015db70 0x7f88d01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f88d6a0d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d015db70 0x7f88d01587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:52914/0 (socket says 172.21.15.28:52914) 2023-11-28T11:09:06.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f88d6a0d700 1 -- 172.21.15.28:0/992419668 learned_addr learned my addr 172.21.15.28:0/992419668 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.516+0000 7f88d5a0b700 1 --2- 172.21.15.28:0/992419668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88d0159590 0x7f88d0158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.517+0000 7f88d6a0d700 1 -- 172.21.15.28:0/992419668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d00ce890 msgr2=0x7f88d00cc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.517+0000 7f88d6a0d700 1 --2- 172.21.15.28:0/992419668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d00ce890 0x7f88d00cc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.517+0000 7f88d6a0d700 1 -- 172.21.15.28:0/992419668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88d0159590 msgr2=0x7f88d0158290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.517+0000 7f88d6a0d700 1 --2- 172.21.15.28:0/992419668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88d0159590 0x7f88d0158290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.517+0000 7f88d6a0d700 1 -- 172.21.15.28:0/992419668 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88c000b7f0 con 0x7f88d015db70 2023-11-28T11:09:06.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.517+0000 7f8aacf25700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aa80c2b60 0x7f8aa80b5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.518+0000 7f8aacf25700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8aa80cca20 0x7f8aa80b4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.518+0000 7f8aacf25700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa8160c90 0x7f8aa814ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.518+0000 7f8aacf25700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa80b3360 con 0x7f8aa8160c90 2023-11-28T11:09:06.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.518+0000 7f8aacf25700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa80b3640 con 0x7f8aa80c2b60 2023-11-28T11:09:06.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.518+0000 7f8aacf25700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa80b3920 con 0x7f8aa80cca20 2023-11-28T11:09:06.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.517+0000 7f88d6a0d700 1 --2- 172.21.15.28:0/992419668 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d015db70 0x7f88d01587d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f88c800b780 tx=0x7f88c800bb40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.517+0000 7f01e8b30700 1 -- 172.21.15.28:0/3407113172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e40c9ce0 con 0x7f01e415a380 2023-11-28T11:09:06.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.517+0000 7f01e8b30700 1 -- 172.21.15.28:0/3407113172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e40c9a00 con 0x7f01e415a380 2023-11-28T11:09:06.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.518+0000 7f8aa5d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8aa80cca20 0x7f8aa80b4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.518+0000 7f8aa5d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8aa80cca20 0x7f8aa80b4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:52924/0 (socket says 172.21.15.28:52924) 2023-11-28T11:09:06.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.518+0000 7f8aa5d9b700 1 -- 172.21.15.28:0/3107421958 learned_addr learned my addr 172.21.15.28:0/3107421958 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.518+0000 7f8aa659c700 1 --2- 172.21.15.28:0/3107421958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aa80c2b60 0x7f8aa80b5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.518+0000 7f88c6ffd700 1 -- 172.21.15.28:0/992419668 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c800f840 con 0x7f88d015db70 2023-11-28T11:09:06.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.518+0000 7f88c6ffd700 1 -- 172.21.15.28:0/992419668 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c800f9a0 con 0x7f88d015db70 2023-11-28T11:09:06.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.518+0000 7f88c6ffd700 1 -- 172.21.15.28:0/992419668 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c800fd50 con 0x7f88d015db70 2023-11-28T11:09:06.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.518+0000 7f27e7915700 1 Processor -- start 2023-11-28T11:09:06.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.518+0000 7f27e7915700 1 -- start start 2023-11-28T11:09:06.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.519+0000 7f8aa6d9d700 1 --2- 172.21.15.28:0/3107421958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa8160c90 0x7f8aa814ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.519+0000 7f1934ddd700 1 -- 172.21.15.28:0/3042041135 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f193006d930 con 0x7f193015aa30 2023-11-28T11:09:06.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.519+0000 7f1934ddd700 1 -- 172.21.15.28:0/3042041135 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19300c9fc0 con 0x7f193015aa30 2023-11-28T11:09:06.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.519+0000 7fb690ff9700 1 -- 172.21.15.28:0/2336017823 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb684021070 con 0x7fb694071070 2023-11-28T11:09:06.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.519+0000 7f8aa6d9d700 1 -- 172.21.15.28:0/3107421958 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8aa80cca20 msgr2=0x7f8aa80b4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.519+0000 7f8aa6d9d700 1 --2- 172.21.15.28:0/3107421958 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8aa80cca20 0x7f8aa80b4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.519+0000 7f8aa6d9d700 1 -- 172.21.15.28:0/3107421958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aa80c2b60 msgr2=0x7f8aa80b5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.519+0000 7f8aa6d9d700 1 --2- 172.21.15.28:0/3107421958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aa80c2b60 0x7f8aa80b5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.519+0000 7f8aa6d9d700 1 -- 172.21.15.28:0/3107421958 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a9c009550 con 0x7f8aa8160c90 2023-11-28T11:09:06.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7f8aa6d9d700 1 --2- 172.21.15.28:0/3107421958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa8160c90 0x7f8aa814ced0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800b3f0 tx=0x7f8a9800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.519+0000 7f27e7915700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27e006a1e0 0x7f27e020a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.519+0000 7f27e7915700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27e006b5c0 0x7f27e00c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.519+0000 7f27e7915700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27e0089580 0x7f27e00c6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.519+0000 7f27e7915700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e020ae00 con 0x7f27e0089580 2023-11-28T11:09:06.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.519+0000 7f27e7915700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27e00c3560 con 0x7f27e006b5c0 2023-11-28T11:09:06.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.519+0000 7f27e7915700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27e00c37b0 con 0x7f27e006a1e0 2023-11-28T11:09:06.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7f27e5eb2700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27e0089580 0x7f27e00c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7f27e5eb2700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27e0089580 0x7f27e00c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47172/0 (socket says 172.21.15.28:47172) 2023-11-28T11:09:06.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7f27e5eb2700 1 -- 172.21.15.28:0/502279446 learned_addr learned my addr 172.21.15.28:0/502279446 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7f27e5eb2700 1 -- 172.21.15.28:0/502279446 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27e006a1e0 msgr2=0x7f27e020a8c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:09:06.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7f27e5eb2700 1 --2- 172.21.15.28:0/502279446 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27e006a1e0 0x7f27e020a8c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.575 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7f27e5eb2700 1 -- 172.21.15.28:0/502279446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27e006b5c0 msgr2=0x7f27e00c4830 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:09:06.575 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7f27e5eb2700 1 --2- 172.21.15.28:0/502279446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27e006b5c0 0x7f27e00c4830 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.575 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7f27e5eb2700 1 -- 172.21.15.28:0/502279446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27dc0075c0 con 0x7f27e0089580 2023-11-28T11:09:06.575 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7f27e5eb2700 1 --2- 172.21.15.28:0/502279446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27e0089580 0x7f27e00c6470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f27d000c7f0 tx=0x7f27d000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.575 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7fb690ff9700 1 --2- 172.21.15.28:0/2336017823 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb67c068460 0x7fb67c06a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.575 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7fb690ff9700 1 -- 172.21.15.28:0/2336017823 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb684085f60 con 0x7fb694071070 2023-11-28T11:09:06.575 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7fb69a4d4700 1 -- 172.21.15.28:0/2336017823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6740052f0 con 0x7fb694071070 2023-11-28T11:09:06.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7ff8dcff9700 1 -- 172.21.15.28:0/1705659144 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff8d4012c60 con 0x7ff8e00c9a40 2023-11-28T11:09:06.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.521+0000 7fe4a072f700 1 -- start start 2023-11-28T11:09:06.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7f01d2ffd700 1 -- 172.21.15.28:0/3407113172 <== mon.1 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f01dc014820 con 0x7f01e415a380 2023-11-28T11:09:06.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.520+0000 7f01d2ffd700 1 --2- 172.21.15.28:0/3407113172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01c40682f0 0x7f01c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.521+0000 7f01d2ffd700 1 -- 172.21.15.28:0/3407113172 <== mon.1 v2:172.21.15.59:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f01dc086c00 con 0x7f01e415a380 2023-11-28T11:09:06.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.521+0000 7f01e259c700 1 --2- 172.21.15.28:0/3407113172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01c40682f0 0x7f01c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.521+0000 7ff8dcff9700 1 --2- 172.21.15.28:0/1705659144 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff8cc0682f0 0x7ff8cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.521+0000 7ff8dcff9700 1 -- 172.21.15.28:0/1705659144 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff8d4086e20 con 0x7ff8e00c9a40 2023-11-28T11:09:06.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.521+0000 7fb6937fe700 1 --2- 172.21.15.28:0/2336017823 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb67c068460 0x7fb67c06a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.521+0000 7f88d8470700 1 -- 172.21.15.28:0/992419668 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88d00aec70 con 0x7f88d015db70 2023-11-28T11:09:06.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.521+0000 7f88d8470700 1 -- 172.21.15.28:0/992419668 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d015d370 con 0x7f88d015db70 2023-11-28T11:09:06.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7f191effd700 1 -- 172.21.15.28:0/3042041135 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1928019030 con 0x7f193015aa30 2023-11-28T11:09:06.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7ff8dffff700 1 --2- 172.21.15.28:0/1705659144 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff8cc0682f0 0x7ff8cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7f191effd700 1 --2- 172.21.15.28:0/3042041135 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f19100683f0 0x7f191006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe4a072f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4980c9920 0x7fe49814a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe4a072f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4980cd120 0x7fe49814d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe4a072f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4981588a0 0x7fe49814d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe4a072f700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4980ad2c0 con 0x7fe4980cd120 2023-11-28T11:09:06.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe4a072f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4980ac6b0 con 0x7fe4981588a0 2023-11-28T11:09:06.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe4a072f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4980ac990 con 0x7fe4980c9920 2023-11-28T11:09:06.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe49e4cb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4980c9920 0x7fe49814a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe49e4cb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4980c9920 0x7fe49814a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:52942/0 (socket says 172.21.15.28:52942) 2023-11-28T11:09:06.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe49e4cb700 1 -- 172.21.15.28:0/1178072698 learned_addr learned my addr 172.21.15.28:0/1178072698 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:06.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe49dcca700 1 --2- 172.21.15.28:0/1178072698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4980cd120 0x7fe49814d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe49eccc700 1 --2- 172.21.15.28:0/1178072698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4981588a0 0x7fe49814d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7f191effd700 1 -- 172.21.15.28:0/3042041135 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1928085920 con 0x7f193015aa30 2023-11-28T11:09:06.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe49e4cb700 1 -- 172.21.15.28:0/1178072698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4981588a0 msgr2=0x7fe49814d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe49e4cb700 1 --2- 172.21.15.28:0/1178072698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4981588a0 0x7fe49814d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe49e4cb700 1 -- 172.21.15.28:0/1178072698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4980cd120 msgr2=0x7fe49814d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe49e4cb700 1 --2- 172.21.15.28:0/1178072698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4980cd120 0x7fe49814d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fe49e4cb700 1 -- 172.21.15.28:0/1178072698 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4940094a0 con 0x7fe4980c9920 2023-11-28T11:09:06.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.522+0000 7fb6937fe700 1 --2- 172.21.15.28:0/2336017823 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb67c068460 0x7fb67c06a910 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb694070c90 tx=0x7fb68c010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.523+0000 7ff8dffff700 1 --2- 172.21.15.28:0/1705659144 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff8cc0682f0 0x7ff8cc06a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009c40 tx=0x7ff8d001d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.523+0000 7fe49e4cb700 1 --2- 172.21.15.28:0/1178072698 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4980c9920 0x7fe49814a9f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe4980ca420 tx=0x7fe494004750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.523+0000 7f01e8b30700 1 -- 172.21.15.28:0/3407113172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01c80052f0 con 0x7f01e415a380 2023-11-28T11:09:06.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.523+0000 7f192e59c700 1 --2- 172.21.15.28:0/3042041135 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f19100683f0 0x7f191006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.523+0000 7f01e259c700 1 --2- 172.21.15.28:0/3407113172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01c40682f0 0x7f01c406a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f01d4007fd0 tx=0x7f01d4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.523+0000 7f88d8470700 1 -- 172.21.15.28:0/992419668 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d015dfe0 con 0x7f88d015db70 2023-11-28T11:09:06.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.523+0000 7fe48effd700 1 -- 172.21.15.28:0/1178072698 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe494004440 con 0x7fe4980c9920 2023-11-28T11:09:06.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.523+0000 7fe48effd700 1 -- 172.21.15.28:0/1178072698 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe49401d040 con 0x7fe4980c9920 2023-11-28T11:09:06.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.523+0000 7fe4a072f700 1 -- 172.21.15.28:0/1178072698 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49806ae00 con 0x7fe4980c9920 2023-11-28T11:09:06.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.523+0000 7f1934ddd700 1 -- 172.21.15.28:0/3042041135 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f193015a750 con 0x7f193015aa30 2023-11-28T11:09:06.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.523+0000 7fe4a072f700 1 -- 172.21.15.28:0/1178072698 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49806ab20 con 0x7fe4980c9920 2023-11-28T11:09:06.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.523+0000 7fe48effd700 1 -- 172.21.15.28:0/1178072698 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4940189b0 con 0x7fe4980c9920 2023-11-28T11:09:06.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.524+0000 7f192e59c700 1 --2- 172.21.15.28:0/3042041135 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f19100683f0 0x7f191006a8a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19300ce900 tx=0x7f1920008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.524+0000 7f27d5ffb700 1 -- 172.21.15.28:0/502279446 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0011800 con 0x7f27e0089580 2023-11-28T11:09:06.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.524+0000 7f27d5ffb700 1 -- 172.21.15.28:0/502279446 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27d0011960 con 0x7f27e0089580 2023-11-28T11:09:06.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.524+0000 7f27d5ffb700 1 -- 172.21.15.28:0/502279446 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27d0011cd0 con 0x7f27e0089580 2023-11-28T11:09:06.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.524+0000 7f8a96ffd700 1 -- 172.21.15.28:0/3107421958 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98011830 con 0x7f8aa8160c90 2023-11-28T11:09:06.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.524+0000 7f8a96ffd700 1 -- 172.21.15.28:0/3107421958 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a98011990 con 0x7f8aa8160c90 2023-11-28T11:09:06.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.524+0000 7f8a96ffd700 1 -- 172.21.15.28:0/3107421958 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a98011d40 con 0x7f8aa8160c90 2023-11-28T11:09:06.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.524+0000 7ff8dcff9700 1 -- 172.21.15.28:0/1705659144 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff8d40532a0 con 0x7ff8e00c9a40 2023-11-28T11:09:06.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.524+0000 7f27e7915700 1 -- 172.21.15.28:0/502279446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27e00ce270 con 0x7f27e0089580 2023-11-28T11:09:06.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.524+0000 7fe4a072f700 1 -- 172.21.15.28:0/1178072698 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4800052f0 con 0x7fe4980c9920 2023-11-28T11:09:06.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.524+0000 7f27e7915700 1 -- 172.21.15.28:0/502279446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27e00cdf90 con 0x7f27e0089580 2023-11-28T11:09:06.588 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.524+0000 7f8aacf25700 1 -- 172.21.15.28:0/3107421958 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa8161f90 con 0x7f8aa8160c90 2023-11-28T11:09:06.588 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.524+0000 7f8aacf25700 1 -- 172.21.15.28:0/3107421958 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa8161cb0 con 0x7f8aa8160c90 2023-11-28T11:09:06.588 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.525+0000 7f88c6ffd700 1 -- 172.21.15.28:0/992419668 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f88c80213f0 con 0x7f88d015db70 2023-11-28T11:09:06.589 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.525+0000 7f8aacf25700 1 -- 172.21.15.28:0/3107421958 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8c0052f0 con 0x7f8aa8160c90 2023-11-28T11:09:06.589 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.526+0000 7f27e7915700 1 -- 172.21.15.28:0/502279446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e00ce790 con 0x7f27e0089580 2023-11-28T11:09:06.589 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.526+0000 7f88c6ffd700 1 --2- 172.21.15.28:0/992419668 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f88b40682f0 0x7f88b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.589 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.526+0000 7f88c6ffd700 1 -- 172.21.15.28:0/992419668 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f88c80869c0 con 0x7f88d015db70 2023-11-28T11:09:06.590 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.526+0000 7f88d620c700 1 --2- 172.21.15.28:0/992419668 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f88b40682f0 0x7f88b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.590 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.527+0000 7f88d620c700 1 --2- 172.21.15.28:0/992419668 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f88b40682f0 0x7f88b406a7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f88c0000c00 tx=0x7f88c0012000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.590 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.527+0000 7f27d5ffb700 1 -- 172.21.15.28:0/502279446 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f27d0021070 con 0x7f27e0089580 2023-11-28T11:09:06.590 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.527+0000 7f8a96ffd700 1 -- 172.21.15.28:0/3107421958 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8a98021070 con 0x7f8aa8160c90 2023-11-28T11:09:06.591 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.527+0000 7fb690ff9700 1 -- 172.21.15.28:0/2336017823 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb684015080 con 0x7fb694071070 2023-11-28T11:09:06.591 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.527+0000 7f27d5ffb700 1 --2- 172.21.15.28:0/502279446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f27c40682f0 0x7f27c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.591 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.528+0000 7f8a96ffd700 1 --2- 172.21.15.28:0/3107421958 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8a88068340 0x7f8a8806a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.528+0000 7f8a96ffd700 1 -- 172.21.15.28:0/3107421958 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8a98086200 con 0x7f8aa8160c90 2023-11-28T11:09:06.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.528+0000 7f8aa659c700 1 --2- 172.21.15.28:0/3107421958 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8a88068340 0x7f8a8806a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.528+0000 7f8aa659c700 1 --2- 172.21.15.28:0/3107421958 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8a88068340 0x7f8a8806a7f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c014040 tx=0x7f8a9c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.529+0000 7f88c6ffd700 1 -- 172.21.15.28:0/992419668 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f88c80569a0 con 0x7f88d015db70 2023-11-28T11:09:06.593 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.530+0000 7fe48effd700 1 -- 172.21.15.28:0/1178072698 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe494028030 con 0x7fe4980c9920 2023-11-28T11:09:06.593 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.530+0000 7fe48effd700 1 --2- 172.21.15.28:0/1178072698 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe47c068340 0x7fe47c06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:06.593 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.530+0000 7fe49dcca700 1 --2- 172.21.15.28:0/1178072698 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe47c068340 0x7fe47c06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.593 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.531+0000 7fe49dcca700 1 --2- 172.21.15.28:0/1178072698 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe47c068340 0x7fe47c06a7f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe498156ef0 tx=0x7fe488005f40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.593 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.532+0000 7f8a96ffd700 1 -- 172.21.15.28:0/3107421958 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8a98052810 con 0x7f8aa8160c90 2023-11-28T11:09:06.593 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.532+0000 7f01d2ffd700 1 -- 172.21.15.28:0/3407113172 <== mon.1 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f01dc056ab0 con 0x7f01e415a380 2023-11-28T11:09:06.594 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.533+0000 7fe48effd700 1 -- 172.21.15.28:0/1178072698 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe49405f750 con 0x7fe4980c9920 2023-11-28T11:09:06.594 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.533+0000 7f191effd700 1 -- 172.21.15.28:0/3042041135 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f192804b5e0 con 0x7f193015aa30 2023-11-28T11:09:06.594 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.538+0000 7fe48effd700 1 -- 172.21.15.28:0/1178072698 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe494053ab0 con 0x7fe4980c9920 2023-11-28T11:09:06.595 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.528+0000 7f27d5ffb700 1 -- 172.21.15.28:0/502279446 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f27d00862e0 con 0x7f27e0089580 2023-11-28T11:09:06.595 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.528+0000 7f27e56b1700 1 --2- 172.21.15.28:0/502279446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f27c40682f0 0x7f27c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:06.595 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.528+0000 7f27e56b1700 1 --2- 172.21.15.28:0/502279446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f27c40682f0 0x7f27c406a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f27e00c52b0 tx=0x7f27dc00a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:06.595 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.539+0000 7f27d5ffb700 1 -- 172.21.15.28:0/502279446 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f27d00527f0 con 0x7f27e0089580 2023-11-28T11:09:06.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.919+0000 7f1934ddd700 1 -- 172.21.15.28:0/3042041135 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f1930068570 con 0x7f193015aa30 2023-11-28T11:09:06.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.919+0000 7f191effd700 1 -- 172.21.15.28:0/3042041135 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1928017070 con 0x7f193015aa30 2023-11-28T11:09:06.922 INFO:teuthology.orchestra.run.smithi028.stdout:55834574855 2023-11-28T11:09:06.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.923+0000 7f1934ddd700 1 -- 172.21.15.28:0/3042041135 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f19100683f0 msgr2=0x7f191006a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.923+0000 7f1934ddd700 1 --2- 172.21.15.28:0/3042041135 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f19100683f0 0x7f191006a8a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19300ce900 tx=0x7f1920008030 comp rx=0 tx=0).stop 2023-11-28T11:09:06.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.923+0000 7f1934ddd700 1 -- 172.21.15.28:0/3042041135 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f193015aa30 msgr2=0x7f19300c3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.923+0000 7f1934ddd700 1 --2- 172.21.15.28:0/3042041135 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f193015aa30 0x7f19300c3ec0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f192800caa0 tx=0x7f192800cdb0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.923+0000 7f1934ddd700 1 -- 172.21.15.28:0/3042041135 shutdown_connections 2023-11-28T11:09:06.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.923+0000 7f1934ddd700 1 --2- 172.21.15.28:0/3042041135 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f19100683f0 0x7f191006a8a0 secure :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f19300ce900 tx=0x7f1920008030 comp rx=0 tx=0).stop 2023-11-28T11:09:06.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.923+0000 7f1934ddd700 1 --2- 172.21.15.28:0/3042041135 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f193015aa30 0x7f19300c3ec0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.923+0000 7f1934ddd700 1 --2- 172.21.15.28:0/3042041135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f193006d560 0x7f19300c4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.923+0000 7f1934ddd700 1 --2- 172.21.15.28:0/3042041135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19300caa20 0x7f193015bf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.923+0000 7f1934ddd700 1 -- 172.21.15.28:0/3042041135 >> 172.21.15.28:0/3042041135 conn(0x7f19300cf670 msgr2=0x7f1930153710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.923+0000 7f1934ddd700 1 -- 172.21.15.28:0/3042041135 shutdown_connections 2023-11-28T11:09:06.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.923+0000 7f1934ddd700 1 -- 172.21.15.28:0/3042041135 wait complete. 2023-11-28T11:09:06.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.931+0000 7fb69a4d4700 1 -- 172.21.15.28:0/2336017823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fb6740055a0 con 0x7fb694071070 2023-11-28T11:09:06.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.934+0000 7f8aacf25700 1 -- 172.21.15.28:0/3107421958 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f8a8c005870 con 0x7f8aa8160c90 2023-11-28T11:09:06.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.929+0000 7ff8e648e700 1 -- 172.21.15.28:0/1705659144 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff8c0005870 con 0x7ff8e00c9a40 2023-11-28T11:09:06.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.936+0000 7f01e8b30700 1 -- 172.21.15.28:0/3407113172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f01c80055a0 con 0x7f01e415a380 2023-11-28T11:09:06.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.936+0000 7ff8dcff9700 1 -- 172.21.15.28:0/1705659144 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff8d40568c0 con 0x7ff8e00c9a40 2023-11-28T11:09:06.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.936+0000 7fb690ff9700 1 -- 172.21.15.28:0/2336017823 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb6840524d0 con 0x7fb694071070 2023-11-28T11:09:06.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.936+0000 7f8a96ffd700 1 -- 172.21.15.28:0/3107421958 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8a98009200 con 0x7f8aa8160c90 2023-11-28T11:09:06.937 INFO:teuthology.orchestra.run.smithi028.stdout:55834574856 2023-11-28T11:09:06.937 INFO:teuthology.orchestra.run.smithi028.stdout:55834574855 2023-11-28T11:09:06.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.937+0000 7f01d2ffd700 1 -- 172.21.15.28:0/3407113172 <== mon.1 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f01dc018020 con 0x7f01e415a380 2023-11-28T11:09:06.938 INFO:teuthology.orchestra.run.smithi028.stdout:55834574856 2023-11-28T11:09:06.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.939+0000 7f8aacf25700 1 -- 172.21.15.28:0/3107421958 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8a88068340 msgr2=0x7f8a8806a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.939+0000 7f8aacf25700 1 --2- 172.21.15.28:0/3107421958 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8a88068340 0x7f8a8806a7f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c014040 tx=0x7f8a9c008030 comp rx=0 tx=0).stop 2023-11-28T11:09:06.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.939+0000 7f8aacf25700 1 -- 172.21.15.28:0/3107421958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa8160c90 msgr2=0x7f8aa814ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.939+0000 7f8aacf25700 1 --2- 172.21.15.28:0/3107421958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa8160c90 0x7f8aa814ced0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8a9800b3f0 tx=0x7f8a9800b700 comp rx=0 tx=0).stop 2023-11-28T11:09:06.941 INFO:teuthology.orchestra.run.smithi028.stdout:55834574856 2023-11-28T11:09:06.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7f01e8b30700 1 -- 172.21.15.28:0/3407113172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01c40682f0 msgr2=0x7f01c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7f01e8b30700 1 --2- 172.21.15.28:0/3407113172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01c40682f0 0x7f01c406a7a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f01d4007fd0 tx=0x7f01d4009040 comp rx=0 tx=0).stop 2023-11-28T11:09:06.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7f01e8b30700 1 -- 172.21.15.28:0/3407113172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e415a380 msgr2=0x7f01e40b36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7f01e8b30700 1 --2- 172.21.15.28:0/3407113172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e415a380 0x7f01e40b36a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f01dc00efc0 tx=0x7f01dc00c3a0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7f01e8b30700 1 -- 172.21.15.28:0/3407113172 shutdown_connections 2023-11-28T11:09:06.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7f01e8b30700 1 --2- 172.21.15.28:0/3407113172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01c40682f0 0x7f01c406a7a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7f01e8b30700 1 --2- 172.21.15.28:0/3407113172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01e40c2530 0x7f01e4156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7f01e8b30700 1 --2- 172.21.15.28:0/3407113172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f01e415a380 0x7f01e40b36a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7f01e8b30700 1 --2- 172.21.15.28:0/3407113172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01e40cabf0 0x7f01e40b3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7f01e8b30700 1 -- 172.21.15.28:0/3407113172 >> 172.21.15.28:0/3407113172 conn(0x7f01e40cf670 msgr2=0x7f01e41584f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7f01e8b30700 1 -- 172.21.15.28:0/3407113172 shutdown_connections 2023-11-28T11:09:06.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7f01e8b30700 1 -- 172.21.15.28:0/3407113172 wait complete. 2023-11-28T11:09:06.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7ff8e648e700 1 -- 172.21.15.28:0/1705659144 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff8cc0682f0 msgr2=0x7ff8cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7ff8e648e700 1 --2- 172.21.15.28:0/1705659144 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff8cc0682f0 0x7ff8cc06a7a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff8d0009c40 tx=0x7ff8d001d040 comp rx=0 tx=0).stop 2023-11-28T11:09:06.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7ff8e648e700 1 -- 172.21.15.28:0/1705659144 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8e00c9a40 msgr2=0x7ff8e01f6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.940+0000 7ff8e648e700 1 --2- 172.21.15.28:0/1705659144 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8e00c9a40 0x7ff8e01f6d20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff8d400d830 tx=0x7ff8d400db40 comp rx=0 tx=0).stop 2023-11-28T11:09:06.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7f8aacf25700 1 -- 172.21.15.28:0/3107421958 shutdown_connections 2023-11-28T11:09:06.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7f8aacf25700 1 --2- 172.21.15.28:0/3107421958 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8a88068340 0x7f8a8806a7f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7f8aacf25700 1 --2- 172.21.15.28:0/3107421958 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8aa8160c90 0x7f8aa814ced0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7f8aacf25700 1 --2- 172.21.15.28:0/3107421958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8aa80c2b60 0x7f8aa80b5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7f8aacf25700 1 --2- 172.21.15.28:0/3107421958 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8aa80cca20 0x7f8aa80b4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7f8aacf25700 1 -- 172.21.15.28:0/3107421958 >> 172.21.15.28:0/3107421958 conn(0x7f8aa80b1650 msgr2=0x7f8aa8162360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7ff8e648e700 1 -- 172.21.15.28:0/1705659144 shutdown_connections 2023-11-28T11:09:06.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7ff8e648e700 1 --2- 172.21.15.28:0/1705659144 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff8cc0682f0 0x7ff8cc06a7a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7ff8e648e700 1 --2- 172.21.15.28:0/1705659144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff8e00cce20 0x7ff8e00c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7ff8e648e700 1 --2- 172.21.15.28:0/1705659144 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8e006a1e0 0x7ff8e00c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7ff8e648e700 1 --2- 172.21.15.28:0/1705659144 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff8e00c9a40 0x7ff8e01f6d20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7ff8e648e700 1 -- 172.21.15.28:0/1705659144 >> 172.21.15.28:0/1705659144 conn(0x7ff8e00cf670 msgr2=0x7ff8e00af7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7ff8e648e700 1 -- 172.21.15.28:0/1705659144 shutdown_connections 2023-11-28T11:09:06.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7f8aacf25700 1 -- 172.21.15.28:0/3107421958 shutdown_connections 2023-11-28T11:09:06.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7f8aacf25700 1 -- 172.21.15.28:0/3107421958 wait complete. 2023-11-28T11:09:06.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.941+0000 7ff8e648e700 1 -- 172.21.15.28:0/1705659144 wait complete. 2023-11-28T11:09:06.947 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.1 2023-11-28T11:09:06.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.943+0000 7fb69a4d4700 1 -- 172.21.15.28:0/2336017823 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb67c068460 msgr2=0x7fb67c06a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.943+0000 7fb69a4d4700 1 --2- 172.21.15.28:0/2336017823 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb67c068460 0x7fb67c06a910 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fb694070c90 tx=0x7fb68c010000 comp rx=0 tx=0).stop 2023-11-28T11:09:06.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.943+0000 7fb69a4d4700 1 -- 172.21.15.28:0/2336017823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb694071070 msgr2=0x7fb69414a6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.943+0000 7fb69a4d4700 1 --2- 172.21.15.28:0/2336017823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb694071070 0x7fb69414a6c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fb68400b550 tx=0x7fb68400b910 comp rx=0 tx=0).stop 2023-11-28T11:09:06.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.943+0000 7fb69a4d4700 1 -- 172.21.15.28:0/2336017823 shutdown_connections 2023-11-28T11:09:06.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.943+0000 7fb69a4d4700 1 --2- 172.21.15.28:0/2336017823 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb67c068460 0x7fb67c06a910 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.943+0000 7fb69a4d4700 1 --2- 172.21.15.28:0/2336017823 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb69415a9a0 0x7fb69415bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.943+0000 7fb69a4d4700 1 --2- 172.21.15.28:0/2336017823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb694071070 0x7fb69414a6c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.943+0000 7fb69a4d4700 1 --2- 172.21.15.28:0/2336017823 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb69415a410 0x7fb69414ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.943+0000 7fb69a4d4700 1 -- 172.21.15.28:0/2336017823 >> 172.21.15.28:0/2336017823 conn(0x7fb6940ce9e0 msgr2=0x7fb6940cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.946+0000 7f88d8470700 1 -- 172.21.15.28:0/992419668 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f88d00b38b0 con 0x7f88d015db70 2023-11-28T11:09:06.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.946+0000 7f88c6ffd700 1 -- 172.21.15.28:0/992419668 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f88d00b38b0 con 0x7f88d015db70 2023-11-28T11:09:06.948 INFO:teuthology.orchestra.run.smithi028.stdout:55834574855 2023-11-28T11:09:06.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.950+0000 7f27e7915700 1 -- 172.21.15.28:0/502279446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f27e00c46a0 con 0x7f27e0089580 2023-11-28T11:09:06.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.950+0000 7f27d5ffb700 1 -- 172.21.15.28:0/502279446 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f27d0055e10 con 0x7f27e0089580 2023-11-28T11:09:06.951 INFO:teuthology.orchestra.run.smithi028.stdout:55834574856 2023-11-28T11:09:06.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.943+0000 7fb69a4d4700 1 -- 172.21.15.28:0/2336017823 shutdown_connections 2023-11-28T11:09:06.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.952+0000 7fb69a4d4700 1 -- 172.21.15.28:0/2336017823 wait complete. 2023-11-28T11:09:06.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.952+0000 7f88d8470700 1 -- 172.21.15.28:0/992419668 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f88b40682f0 msgr2=0x7f88b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.952+0000 7f88d8470700 1 --2- 172.21.15.28:0/992419668 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f88b40682f0 0x7f88b406a7a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f88c0000c00 tx=0x7f88c0012000 comp rx=0 tx=0).stop 2023-11-28T11:09:06.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.952+0000 7f88d8470700 1 -- 172.21.15.28:0/992419668 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d015db70 msgr2=0x7f88d01587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.952+0000 7f88d8470700 1 --2- 172.21.15.28:0/992419668 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d015db70 0x7f88d01587d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f88c800b780 tx=0x7f88c800bb40 comp rx=0 tx=0).stop 2023-11-28T11:09:06.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.952+0000 7f88d8470700 1 -- 172.21.15.28:0/992419668 shutdown_connections 2023-11-28T11:09:06.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.952+0000 7f88d8470700 1 --2- 172.21.15.28:0/992419668 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f88b40682f0 0x7f88b406a7a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.952+0000 7f88d8470700 1 --2- 172.21.15.28:0/992419668 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88d015db70 0x7f88d01587d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.952+0000 7f88d8470700 1 --2- 172.21.15.28:0/992419668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d00ce890 0x7f88d00cc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.952+0000 7f88d8470700 1 --2- 172.21.15.28:0/992419668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88d0159590 0x7f88d0158290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.952+0000 7f88d8470700 1 -- 172.21.15.28:0/992419668 >> 172.21.15.28:0/992419668 conn(0x7f88d00b1650 msgr2=0x7f88d006aa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.952+0000 7fe4a072f700 1 -- 172.21.15.28:0/1178072698 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fe480005870 con 0x7fe4980c9920 2023-11-28T11:09:06.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.952+0000 7f88d8470700 1 -- 172.21.15.28:0/992419668 shutdown_connections 2023-11-28T11:09:06.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.952+0000 7f88d8470700 1 -- 172.21.15.28:0/992419668 wait complete. 2023-11-28T11:09:06.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.953+0000 7fe48effd700 1 -- 172.21.15.28:0/1178072698 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe494026020 con 0x7fe4980c9920 2023-11-28T11:09:06.955 INFO:teuthology.orchestra.run.smithi028.stdout:55834574856 2023-11-28T11:09:06.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.956+0000 7f27e7915700 1 -- 172.21.15.28:0/502279446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f27c40682f0 msgr2=0x7f27c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.956+0000 7f27e7915700 1 --2- 172.21.15.28:0/502279446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f27c40682f0 0x7f27c406a7a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f27e00c52b0 tx=0x7f27dc00a000 comp rx=0 tx=0).stop 2023-11-28T11:09:06.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.956+0000 7f27e7915700 1 -- 172.21.15.28:0/502279446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27e0089580 msgr2=0x7f27e00c6470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.956+0000 7f27e7915700 1 --2- 172.21.15.28:0/502279446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27e0089580 0x7f27e00c6470 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f27d000c7f0 tx=0x7f27d000cbb0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.957+0000 7fe4a072f700 1 -- 172.21.15.28:0/1178072698 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe47c068340 msgr2=0x7fe47c06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.957+0000 7fe4a072f700 1 --2- 172.21.15.28:0/1178072698 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe47c068340 0x7fe47c06a7f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe498156ef0 tx=0x7fe488005f40 comp rx=0 tx=0).stop 2023-11-28T11:09:06.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.957+0000 7fe4a072f700 1 -- 172.21.15.28:0/1178072698 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4980c9920 msgr2=0x7fe49814a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:06.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.957+0000 7fe4a072f700 1 --2- 172.21.15.28:0/1178072698 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4980c9920 0x7fe49814a9f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe4980ca420 tx=0x7fe494004750 comp rx=0 tx=0).stop 2023-11-28T11:09:06.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.959+0000 7f27e7915700 1 -- 172.21.15.28:0/502279446 shutdown_connections 2023-11-28T11:09:06.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.959+0000 7f27e7915700 1 --2- 172.21.15.28:0/502279446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f27c40682f0 0x7f27c406a7a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.959+0000 7f27e7915700 1 --2- 172.21.15.28:0/502279446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f27e0089580 0x7f27e00c6470 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.959+0000 7f27e7915700 1 --2- 172.21.15.28:0/502279446 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27e006a1e0 0x7f27e020a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.959+0000 7f27e7915700 1 --2- 172.21.15.28:0/502279446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27e006b5c0 0x7f27e00c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.959+0000 7f27e7915700 1 -- 172.21.15.28:0/502279446 >> 172.21.15.28:0/502279446 conn(0x7f27e00cf670 msgr2=0x7f27e00cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.962+0000 7f27e7915700 1 -- 172.21.15.28:0/502279446 shutdown_connections 2023-11-28T11:09:06.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.962+0000 7fe4a072f700 1 -- 172.21.15.28:0/1178072698 shutdown_connections 2023-11-28T11:09:06.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.962+0000 7fe4a072f700 1 --2- 172.21.15.28:0/1178072698 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe47c068340 0x7fe47c06a7f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.962+0000 7fe4a072f700 1 --2- 172.21.15.28:0/1178072698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4981588a0 0x7fe49814d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.962+0000 7fe4a072f700 1 --2- 172.21.15.28:0/1178072698 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4980c9920 0x7fe49814a9f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.962+0000 7fe4a072f700 1 --2- 172.21.15.28:0/1178072698 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4980cd120 0x7fe49814d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:06.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.962+0000 7fe4a072f700 1 -- 172.21.15.28:0/1178072698 >> 172.21.15.28:0/1178072698 conn(0x7fe4980cf670 msgr2=0x7fe49806d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:06.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.962+0000 7f27e7915700 1 -- 172.21.15.28:0/502279446 wait complete. 2023-11-28T11:09:06.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.962+0000 7fe4a072f700 1 -- 172.21.15.28:0/1178072698 shutdown_connections 2023-11-28T11:09:06.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:06.962+0000 7fe4a072f700 1 -- 172.21.15.28:0/1178072698 wait complete. 2023-11-28T11:09:06.968 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.5 2023-11-28T11:09:06.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.2 2023-11-28T11:09:06.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.6 2023-11-28T11:09:06.974 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.7 2023-11-28T11:09:06.975 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.3 2023-11-28T11:09:06.980 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.4 2023-11-28T11:09:06.982 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574856 for osd.0 2023-11-28T11:09:07.948 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T11:09:07.969 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T11:09:07.970 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T11:09:07.970 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T11:09:07.976 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T11:09:07.976 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T11:09:07.981 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T11:09:07.982 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T11:09:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.161+0000 7f5973ab1700 1 Processor -- start 2023-11-28T11:09:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.161+0000 7f5973ab1700 1 -- start start 2023-11-28T11:09:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.162+0000 7f5973ab1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f596c160be0 0x7f596c0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.162+0000 7f5973ab1700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596c0cdb60 con 0x7f596c0c9df0 2023-11-28T11:09:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.162+0000 7f5973ab1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f596c0cde40 con 0x7f596c0adec0 2023-11-28T11:09:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.162+0000 7f5973ab1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c0ce120 con 0x7f596c160be0 2023-11-28T11:09:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.163+0000 7f597204e700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f596c0adec0 0x7f596c16a580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:46044/0 (socket says 172.21.15.28:46044) 2023-11-28T11:09:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.163+0000 7f597204e700 1 -- 172.21.15.28:0/818981598 learned_addr learned my addr 172.21.15.28:0/818981598 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.165+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890944177 0 0) 0x7f596c0cde40 con 0x7f596c0adec0 2023-11-28T11:09:08.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.165+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59580035b0 con 0x7f596c0adec0 2023-11-28T11:09:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.165+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 822019693 0 0) 0x7f596c0cdb60 con 0x7f596c0c9df0 2023-11-28T11:09:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.165+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f596c0cde40 con 0x7f596c0c9df0 2023-11-28T11:09:08.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.165+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3348741997 0 0) 0x7f596c0cde40 con 0x7f596c0c9df0 2023-11-28T11:09:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.165+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f596c160be0 msgr2=0x7f596c0c98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.165+0000 7f597084b700 1 --2- 172.21.15.28:0/818981598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f596c160be0 0x7f596c0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.165+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 >> v1:172.21.15.59:6789/0 conn(0x7f596c0adec0 legacy=0x7f596c16a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.165+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c06a1e0 con 0x7f596c0c9df0 2023-11-28T11:09:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.165+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f5968001520 con 0x7f596c0c9df0 2023-11-28T11:09:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.165+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 >> v1:172.21.15.28:6789/0 conn(0x7f596c0c9df0 legacy=0x7f596c0ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.165+0000 7f597084b700 1 --2- 172.21.15.28:0/818981598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5958004c20 0x7f5958005010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.165+0000 7f597084b700 1 --2- 172.21.15.28:0/818981598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59580037f0 0x7f5958003c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.166+0000 7f597084b700 1 --2- 172.21.15.28:0/818981598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59580042c0 0x7f595800ba90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.166+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c0ce120 con 0x7f59580042c0 2023-11-28T11:09:08.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.166+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595800c230 con 0x7f5958004c20 2023-11-28T11:09:08.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.166+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595800c3f0 con 0x7f59580037f0 2023-11-28T11:09:08.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.166+0000 7f597204e700 1 --2- 172.21.15.28:0/818981598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59580042c0 0x7f595800ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.166+0000 7f597104c700 1 --2- 172.21.15.28:0/818981598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59580037f0 0x7f5958003c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.167+0000 7f597104c700 1 -- 172.21.15.28:0/818981598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5958004c20 msgr2=0x7f5958005010 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:09:08.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.167+0000 7f597104c700 1 --2- 172.21.15.28:0/818981598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5958004c20 0x7f5958005010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.167+0000 7f597104c700 1 -- 172.21.15.28:0/818981598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59580042c0 msgr2=0x7f595800ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.167+0000 7f597104c700 1 --2- 172.21.15.28:0/818981598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59580042c0 0x7f595800ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.167+0000 7f597104c700 1 -- 172.21.15.28:0/818981598 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c06a1e0 con 0x7f59580037f0 2023-11-28T11:09:08.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.167+0000 7f597104c700 1 --2- 172.21.15.28:0/818981598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59580037f0 0x7f5958003c60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5968007f70 tx=0x7f5958005880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=191ef31a99593094 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.168+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f596800cb70 con 0x7f59580037f0 2023-11-28T11:09:08.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.168+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f596800ccd0 con 0x7f59580037f0 2023-11-28T11:09:08.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.168+0000 7f597084b700 1 -- 172.21.15.28:0/818981598 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968012620 con 0x7f59580037f0 2023-11-28T11:09:08.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.169+0000 7f5973ab1700 1 -- 172.21.15.28:0/818981598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59580037f0 msgr2=0x7f5958003c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.169+0000 7f5973ab1700 1 --2- 172.21.15.28:0/818981598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59580037f0 0x7f5958003c60 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5968007f70 tx=0x7f5958005880 comp rx=0 tx=0).stop 2023-11-28T11:09:08.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.169+0000 7f597204e700 1 -- 172.21.15.28:0/818981598 reap_dead start 2023-11-28T11:09:08.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.174+0000 7f5973ab1700 1 -- 172.21.15.28:0/818981598 shutdown_connections 2023-11-28T11:09:08.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.174+0000 7f5973ab1700 1 -- 172.21.15.28:0/818981598 >> 172.21.15.28:0/818981598 conn(0x7f596c0cf670 msgr2=0x7f596c0cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.174+0000 7f5973ab1700 1 -- 172.21.15.28:0/818981598 shutdown_connections 2023-11-28T11:09:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.174+0000 7f5973ab1700 1 -- 172.21.15.28:0/818981598 wait complete. 2023-11-28T11:09:08.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.176+0000 7f5973ab1700 1 Processor -- start 2023-11-28T11:09:08.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.177+0000 7f5973ab1700 1 -- start start 2023-11-28T11:09:08.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.178+0000 7f5973ab1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59580037f0 0x7f596c150f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.178+0000 7f5973ab1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f596c151480 0x7f596c162710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.178+0000 7f5973ab1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f596c162c50 0x7f596c1434e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.178+0000 7f5973ab1700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c1518f0 con 0x7f596c151480 2023-11-28T11:09:08.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.178+0000 7f5973ab1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c1640e0 con 0x7f59580037f0 2023-11-28T11:09:08.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.178+0000 7f5973ab1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f596c1634d0 con 0x7f596c162c50 2023-11-28T11:09:08.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.179+0000 7f597104c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f596c151480 0x7f596c162710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.179+0000 7f597104c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f596c151480 0x7f596c162710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47212/0 (socket says 172.21.15.28:47212) 2023-11-28T11:09:08.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.179+0000 7f597104c700 1 -- 172.21.15.28:0/3571176403 learned_addr learned my addr 172.21.15.28:0/3571176403 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:08.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.180+0000 7f597104c700 1 -- 172.21.15.28:0/3571176403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f596c162c50 msgr2=0x7f596c1434e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.180+0000 7f597104c700 1 --2- 172.21.15.28:0/3571176403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f596c162c50 0x7f596c1434e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.180+0000 7f597104c700 1 -- 172.21.15.28:0/3571176403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59580037f0 msgr2=0x7f596c150f40 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T11:09:08.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.180+0000 7f597104c700 1 --2- 172.21.15.28:0/3571176403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59580037f0 0x7f596c150f40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.180+0000 7f597104c700 1 -- 172.21.15.28:0/3571176403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f596c06cd50 con 0x7f596c151480 2023-11-28T11:09:08.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.180+0000 7f597104c700 1 --2- 172.21.15.28:0/3571176403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f596c151480 0x7f596c162710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f595800c580 tx=0x7f59680017e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.181+0000 7f59627fc700 1 -- 172.21.15.28:0/3571176403 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968001ab0 con 0x7f596c151480 2023-11-28T11:09:08.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.181+0000 7f59627fc700 1 -- 172.21.15.28:0/3571176403 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59680052d0 con 0x7f596c151480 2023-11-28T11:09:08.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.181+0000 7f59627fc700 1 -- 172.21.15.28:0/3571176403 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5968003680 con 0x7f596c151480 2023-11-28T11:09:08.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.181+0000 7f5973ab1700 1 -- 172.21.15.28:0/3571176403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596c06a4c0 con 0x7f596c151480 2023-11-28T11:09:08.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.181+0000 7f5973ab1700 1 -- 172.21.15.28:0/3571176403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596c0cd830 con 0x7f596c151480 2023-11-28T11:09:08.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.184+0000 7f59627fc700 1 -- 172.21.15.28:0/3571176403 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5968010050 con 0x7f596c151480 2023-11-28T11:09:08.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.184+0000 7f59627fc700 1 --2- 172.21.15.28:0/3571176403 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5958069f70 0x7f595806c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.184+0000 7f59627fc700 1 -- 172.21.15.28:0/3571176403 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5968087780 con 0x7f596c151480 2023-11-28T11:09:08.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.184+0000 7f597184d700 1 --2- 172.21.15.28:0/3571176403 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5958069f70 0x7f595806c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.193+0000 7f5973ab1700 1 -- 172.21.15.28:0/3571176403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5950005320 con 0x7f596c151480 2023-11-28T11:09:08.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.194+0000 7f597184d700 1 --2- 172.21.15.28:0/3571176403 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5958069f70 0x7f595806c420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f596c071cb0 tx=0x7f595c006cb0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.198+0000 7f59627fc700 1 -- 172.21.15.28:0/3571176403 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5968053c90 con 0x7f596c151480 2023-11-28T11:09:08.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.222+0000 7f88edd6f700 1 Processor -- start 2023-11-28T11:09:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.223+0000 7f88edd6f700 1 -- start start 2023-11-28T11:09:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.224+0000 7f88edd6f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88e81588a0 0x7f88e80ce010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.224+0000 7f88edd6f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f88e80cd120 0x7f88e806b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.224+0000 7f88edd6f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f88e806b860 0x7f88e80c9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.224+0000 7f88edd6f700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e806a280 con 0x7f88e806b860 2023-11-28T11:09:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.224+0000 7f88edd6f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88e806a560 con 0x7f88e80cd120 2023-11-28T11:09:08.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.224+0000 7f88edd6f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e806a840 con 0x7f88e81588a0 2023-11-28T11:09:08.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.225+0000 7f88e7fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f88e806b860 0x7f88e80c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.225+0000 7f88e7fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f88e806b860 0x7f88e80c9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47220/0 (socket says 172.21.15.28:47220) 2023-11-28T11:09:08.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.225+0000 7f88e7fff700 1 -- 172.21.15.28:0/2904212084 learned_addr learned my addr 172.21.15.28:0/2904212084 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:08.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.225+0000 7f88e6ffd700 1 --2- 172.21.15.28:0/2904212084 >> v2:172.21.15.59:3300/0 conn(0x7f88e80cd120 0x7f88e806b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.225+0000 7f88e77fe700 1 --2- 172.21.15.28:0/2904212084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88e81588a0 0x7f88e80ce010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.226+0000 7f21a99a8700 1 Processor -- start 2023-11-28T11:09:08.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.225+0000 7f88e6ffd700 1 -- 172.21.15.28:0/2904212084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88e81588a0 msgr2=0x7f88e80ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.225+0000 7f88e6ffd700 1 --2- 172.21.15.28:0/2904212084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88e81588a0 0x7f88e80ce010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.225+0000 7f88e6ffd700 1 -- 172.21.15.28:0/2904212084 >> v2:172.21.15.28:3300/0 conn(0x7f88e806b860 msgr2=0x7f88e80c9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.225+0000 7f88e6ffd700 1 --2- 172.21.15.28:0/2904212084 >> v2:172.21.15.28:3300/0 conn(0x7f88e806b860 0x7f88e80c9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.225+0000 7f88e6ffd700 1 -- 172.21.15.28:0/2904212084 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e806ab20 con 0x7f88e80cd120 2023-11-28T11:09:08.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.225+0000 7f88e7fff700 1 --2- 172.21.15.28:0/2904212084 >> v2:172.21.15.28:3300/0 conn(0x7f88e806b860 0x7f88e80c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:08.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.225+0000 7f88e6ffd700 1 --2- 172.21.15.28:0/2904212084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e80cd120 0x7f88e806b320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f88dc008840 tx=0x7f88dc00d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dae0c7c24b35b6ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.226+0000 7f21a99a8700 1 -- start start 2023-11-28T11:09:08.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.226+0000 7f5f990f0700 1 Processor -- start 2023-11-28T11:09:08.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.226+0000 7f5f990f0700 1 -- start start 2023-11-28T11:09:08.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f5f990f0700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5f940cd120 0x7f5f94158630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f5f990f0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f9406b320 0x7f5f9406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f5f990f0700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f9406a560 con 0x7f5f94071970 2023-11-28T11:09:08.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f5f990f0700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f940ce720 con 0x7f5f940cd120 2023-11-28T11:09:08.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f5f990f0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f94158b70 con 0x7f5f9406b320 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.227+0000 7f21a99a8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f21a4071970 0x7f21a4158630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.227+0000 7f21a99a8700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f21a406b320 0x7f21a406b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.227+0000 7f21a99a8700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21a406a560 con 0x7f21a40cd120 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.227+0000 7f21a99a8700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a40ce720 con 0x7f21a406b320 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.227+0000 7f21a99a8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21a4158b70 con 0x7f21a4071970 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f21a27fc700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f21a40cd120 0x7f21a40ce010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35282/0 (socket says 172.21.15.28:35282) 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f21a27fc700 1 -- 172.21.15.28:0/3265254465 learned_addr learned my addr 172.21.15.28:0/3265254465 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:08.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.226+0000 7f3032790700 1 Processor -- start 2023-11-28T11:09:08.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.227+0000 7f3032790700 1 -- start start 2023-11-28T11:09:08.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f3032790700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f302c158a30 0x7f302c06aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f3032790700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f302c0ce150 0x7f302c0ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f3032790700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f302c06ed80 con 0x7f302c0ceb10 2023-11-28T11:09:08.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f3032790700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f302c06f060 con 0x7f302c158a30 2023-11-28T11:09:08.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f3032790700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0cd950 con 0x7f302c0ce150 2023-11-28T11:09:08.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f21a2ffd700 1 --2- 172.21.15.28:0/3265254465 >> v2:172.21.15.59:3300/0 conn(0x7f21a4071970 0x7f21a4158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f21a37fe700 1 --2- 172.21.15.28:0/3265254465 >> v2:172.21.15.28:3300/0 conn(0x7f21a406b320 0x7f21a406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f5f92d9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5f94071970 0x7f5f940ce010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35284/0 (socket says 172.21.15.28:35284) 2023-11-28T11:09:08.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.228+0000 7f5f92d9d700 1 -- 172.21.15.28:0/2255897095 learned_addr learned my addr 172.21.15.28:0/2255897095 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:08.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.229+0000 7f5f9359e700 1 --2- 172.21.15.28:0/2255897095 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f9406b320 0x7f5f9406b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.229+0000 7f302bfff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f302c158a30 0x7f302c06aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.229+0000 7f302bfff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f302c158a30 0x7f302c06aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47234/0 (socket says 172.21.15.28:47234) 2023-11-28T11:09:08.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.229+0000 7f302bfff700 1 -- 172.21.15.28:0/572043263 learned_addr learned my addr 172.21.15.28:0/572043263 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:08.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.229+0000 7f5f9359e700 1 -- 172.21.15.28:0/2255897095 >> v2:172.21.15.28:3300/0 conn(0x7f5f940cd120 msgr2=0x7f5f94158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.229+0000 7f5f9359e700 1 --2- 172.21.15.28:0/2255897095 >> v2:172.21.15.28:3300/0 conn(0x7f5f940cd120 0x7f5f94158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.229+0000 7f5f9359e700 1 -- 172.21.15.28:0/2255897095 >> v1:172.21.15.28:6789/0 conn(0x7f5f94071970 legacy=0x7f5f940ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.229+0000 7f5f9359e700 1 -- 172.21.15.28:0/2255897095 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f9406a840 con 0x7f5f9406b320 2023-11-28T11:09:08.236 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.229+0000 7f5f9359e700 1 --2- 172.21.15.28:0/2255897095 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f9406b320 0x7f5f9406b7a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009d90 tx=0x7f5f8c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9f5b7b16e070c8a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.236 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.229+0000 7f302b7fe700 1 -- 172.21.15.28:0/572043263 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943374103 0 0) 0x7f302c06ed80 con 0x7f302c0ceb10 2023-11-28T11:09:08.236 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.229+0000 7f302b7fe700 1 -- 172.21.15.28:0/572043263 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30180057c0 con 0x7f302c0ceb10 2023-11-28T11:09:08.241 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f30237fe700 1 --2- 172.21.15.28:0/572043263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f302c0ce150 0x7f302c0ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.241 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f302b7fe700 1 -- 172.21.15.28:0/572043263 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4203977560 0 0) 0x7f30180057c0 con 0x7f302c0ceb10 2023-11-28T11:09:08.241 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f302b7fe700 1 -- 172.21.15.28:0/572043263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f302c0ce150 msgr2=0x7f302c0ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f302b7fe700 1 --2- 172.21.15.28:0/572043263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f302c0ce150 0x7f302c0ce5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f302b7fe700 1 -- 172.21.15.28:0/572043263 >> v2:172.21.15.28:3300/0 conn(0x7f302c158a30 msgr2=0x7f302c06aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f302b7fe700 1 --2- 172.21.15.28:0/572043263 >> v2:172.21.15.28:3300/0 conn(0x7f302c158a30 0x7f302c06aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f302b7fe700 1 -- 172.21.15.28:0/572043263 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c0cdc30 con 0x7f302c0ceb10 2023-11-28T11:09:08.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f302b7fe700 1 -- 172.21.15.28:0/572043263 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f30240019f0 con 0x7f302c0ceb10 2023-11-28T11:09:08.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f302b7fe700 1 -- 172.21.15.28:0/572043263 >> v1:172.21.15.28:6789/0 conn(0x7f302c0ceb10 legacy=0x7f302c0ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f302b7fe700 1 --2- 172.21.15.28:0/572043263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3018005c10 0x7f3018006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f302b7fe700 1 --2- 172.21.15.28:0/572043263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3018006640 0x7f3018006b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f302b7fe700 1 --2- 172.21.15.28:0/572043263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30180072c0 0x7f301800dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f302b7fe700 1 -- 172.21.15.28:0/572043263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c06f060 con 0x7f3018006640 2023-11-28T11:09:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f302b7fe700 1 -- 172.21.15.28:0/572043263 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0cd950 con 0x7f3018005c10 2023-11-28T11:09:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f302b7fe700 1 -- 172.21.15.28:0/572043263 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f301800e210 con 0x7f30180072c0 2023-11-28T11:09:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f21a1ffb700 1 -- 172.21.15.28:0/3265254465 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4150958208 0 0) 0x7f21a406a560 con 0x7f21a40cd120 2023-11-28T11:09:08.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f21a1ffb700 1 -- 172.21.15.28:0/3265254465 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f218c0035b0 con 0x7f21a40cd120 2023-11-28T11:09:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f302bfff700 1 --2- 172.21.15.28:0/572043263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3018005c10 0x7f3018006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f3030d2d700 1 --2- 172.21.15.28:0/572043263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30180072c0 0x7f301800dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f5f91d9b700 1 -- 172.21.15.28:0/2255897095 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c00e070 con 0x7f5f9406b320 2023-11-28T11:09:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f5f91d9b700 1 -- 172.21.15.28:0/2255897095 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f8c00bbd0 con 0x7f5f9406b320 2023-11-28T11:09:08.244 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f5f91d9b700 1 -- 172.21.15.28:0/2255897095 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f8c012430 con 0x7f5f9406b320 2023-11-28T11:09:08.245 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f21a37fe700 1 -- 172.21.15.28:0/3265254465 >> v2:172.21.15.59:3300/0 conn(0x7f21a4071970 msgr2=0x7f21a4158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.245 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f21a37fe700 1 --2- 172.21.15.28:0/3265254465 >> v2:172.21.15.59:3300/0 conn(0x7f21a4071970 0x7f21a4158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.245 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f21a37fe700 1 -- 172.21.15.28:0/3265254465 >> v1:172.21.15.28:6789/0 conn(0x7f21a40cd120 legacy=0x7f21a40ce010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.245 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.230+0000 7f21a37fe700 1 -- 172.21.15.28:0/3265254465 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21a406a840 con 0x7f21a406b320 2023-11-28T11:09:08.245 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f3030d2d700 1 -- 172.21.15.28:0/572043263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3018005c10 msgr2=0x7f3018006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f3030d2d700 1 --2- 172.21.15.28:0/572043263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3018005c10 0x7f3018006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f3030d2d700 1 -- 172.21.15.28:0/572043263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3018006640 msgr2=0x7f3018006b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f3030d2d700 1 --2- 172.21.15.28:0/572043263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3018006640 0x7f3018006b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f3030d2d700 1 -- 172.21.15.28:0/572043263 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30180057c0 con 0x7f30180072c0 2023-11-28T11:09:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f21a37fe700 1 --2- 172.21.15.28:0/3265254465 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21a406b320 0x7f21a406b7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f21a40cc120 tx=0x7f2194009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23fdc8b1450f3e61 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.246 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f3030d2d700 1 --2- 172.21.15.28:0/572043263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30180072c0 0x7f301800dc20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f302400a890 tx=0x7f3018007a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d51ae6145dab999 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f5f990f0700 1 -- 172.21.15.28:0/2255897095 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f9406b320 msgr2=0x7f5f9406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f5f990f0700 1 --2- 172.21.15.28:0/2255897095 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f9406b320 0x7f5f9406b7a0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5f8c009d90 tx=0x7f5f8c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:08.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f302b7fe700 1 -- 172.21.15.28:0/572043263 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024012620 con 0x7f30180072c0 2023-11-28T11:09:08.247 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f302b7fe700 1 -- 172.21.15.28:0/572043263 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30240127a0 con 0x7f30180072c0 2023-11-28T11:09:08.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f302b7fe700 1 -- 172.21.15.28:0/572043263 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3024012b50 con 0x7f30180072c0 2023-11-28T11:09:08.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f21a1ffb700 1 -- 172.21.15.28:0/3265254465 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194011830 con 0x7f21a406b320 2023-11-28T11:09:08.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f21a1ffb700 1 -- 172.21.15.28:0/3265254465 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2194011990 con 0x7f21a406b320 2023-11-28T11:09:08.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f21a1ffb700 1 -- 172.21.15.28:0/3265254465 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2194011d00 con 0x7f21a406b320 2023-11-28T11:09:08.249 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f5f990f0700 1 -- 172.21.15.28:0/2255897095 shutdown_connections 2023-11-28T11:09:08.249 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f5f990f0700 1 --2- 172.21.15.28:0/2255897095 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f9406b320 0x7f5f9406b7a0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.249 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f5f990f0700 1 --2- 172.21.15.28:0/2255897095 >> v2:172.21.15.28:3300/0 conn(0x7f5f940cd120 0x7f5f94158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.249 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f5f990f0700 1 -- 172.21.15.28:0/2255897095 >> 172.21.15.28:0/2255897095 conn(0x7f5f940cf670 msgr2=0x7f5f940cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.249 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.231+0000 7f5f990f0700 1 -- 172.21.15.28:0/2255897095 shutdown_connections 2023-11-28T11:09:08.249 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f88e67fc700 1 -- 172.21.15.28:0/2904212084 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc010070 con 0x7f88e80cd120 2023-11-28T11:09:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f88e67fc700 1 -- 172.21.15.28:0/2904212084 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88dc00d8d0 con 0x7f88e80cd120 2023-11-28T11:09:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f88e67fc700 1 -- 172.21.15.28:0/2904212084 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88dc00dc40 con 0x7f88e80cd120 2023-11-28T11:09:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f21a99a8700 1 -- 172.21.15.28:0/3265254465 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21a406b320 msgr2=0x7f21a406b7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f21a99a8700 1 --2- 172.21.15.28:0/3265254465 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21a406b320 0x7f21a406b7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f21a40cc120 tx=0x7f2194009b20 comp rx=0 tx=0).stop 2023-11-28T11:09:08.250 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f3032790700 1 -- 172.21.15.28:0/572043263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30180072c0 msgr2=0x7f301800dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.251 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f3032790700 1 --2- 172.21.15.28:0/572043263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30180072c0 0x7f301800dc20 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f302400a890 tx=0x7f3018007a90 comp rx=0 tx=0).stop 2023-11-28T11:09:08.251 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f3030d2d700 1 -- 172.21.15.28:0/572043263 reap_dead start 2023-11-28T11:09:08.251 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f3032790700 1 -- 172.21.15.28:0/572043263 shutdown_connections 2023-11-28T11:09:08.251 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f3032790700 1 -- 172.21.15.28:0/572043263 >> 172.21.15.28:0/572043263 conn(0x7f302c0b0dd0 msgr2=0x7f302c0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.251 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f21a99a8700 1 -- 172.21.15.28:0/3265254465 shutdown_connections 2023-11-28T11:09:08.251 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f21a99a8700 1 --2- 172.21.15.28:0/3265254465 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21a406b320 0x7f21a406b7a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.251 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f21a99a8700 1 --2- 172.21.15.28:0/3265254465 >> v2:172.21.15.59:3300/0 conn(0x7f21a4071970 0x7f21a4158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.251 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f21a99a8700 1 -- 172.21.15.28:0/3265254465 >> 172.21.15.28:0/3265254465 conn(0x7f21a40cf670 msgr2=0x7f21a40cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.252 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f3032790700 1 -- 172.21.15.28:0/572043263 shutdown_connections 2023-11-28T11:09:08.252 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f5f990f0700 1 -- 172.21.15.28:0/2255897095 wait complete. 2023-11-28T11:09:08.252 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.232+0000 7f3032790700 1 -- 172.21.15.28:0/572043263 wait complete. 2023-11-28T11:09:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.233+0000 7f88edd6f700 1 -- 172.21.15.28:0/2904212084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e80cd120 msgr2=0x7f88e806b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.233+0000 7f88edd6f700 1 --2- 172.21.15.28:0/2904212084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e80cd120 0x7f88e806b320 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f88dc008840 tx=0x7f88dc00d330 comp rx=0 tx=0).stop 2023-11-28T11:09:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.233+0000 7f88edd6f700 1 -- 172.21.15.28:0/2904212084 shutdown_connections 2023-11-28T11:09:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.233+0000 7f88edd6f700 1 --2- 172.21.15.28:0/2904212084 >> v2:172.21.15.28:3300/0 conn(0x7f88e806b860 0x7f88e80c9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.233+0000 7f88edd6f700 1 --2- 172.21.15.28:0/2904212084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88e81588a0 0x7f88e80ce010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.233+0000 7f88edd6f700 1 --2- 172.21.15.28:0/2904212084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e80cd120 0x7f88e806b320 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.233+0000 7f88edd6f700 1 -- 172.21.15.28:0/2904212084 >> 172.21.15.28:0/2904212084 conn(0x7f88e80cf670 msgr2=0x7f88e80cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.253 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.233+0000 7f88edd6f700 1 -- 172.21.15.28:0/2904212084 shutdown_connections 2023-11-28T11:09:08.254 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.234+0000 7f3032790700 1 Processor -- start 2023-11-28T11:09:08.254 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.234+0000 7f5f990f0700 1 Processor -- start 2023-11-28T11:09:08.254 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.234+0000 7f5f990f0700 1 -- start start 2023-11-28T11:09:08.254 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.235+0000 7f5f990f0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f9406b320 0x7f5f940b1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.254 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.235+0000 7f5f990f0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f94071970 0x7f5f940c4c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.254 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.235+0000 7f5f990f0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f940cd120 0x7f5f940c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.235+0000 7f5f990f0700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f94146e90 con 0x7f5f94071970 2023-11-28T11:09:08.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.235+0000 7f5f990f0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f940b2440 con 0x7f5f940cd120 2023-11-28T11:09:08.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.235+0000 7f5f990f0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f940b3e10 con 0x7f5f9406b320 2023-11-28T11:09:08.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.233+0000 7f21a99a8700 1 -- 172.21.15.28:0/3265254465 shutdown_connections 2023-11-28T11:09:08.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.233+0000 7f88edd6f700 1 -- 172.21.15.28:0/2904212084 wait complete. 2023-11-28T11:09:08.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.235+0000 7f88edd6f700 1 Processor -- start 2023-11-28T11:09:08.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.235+0000 7f5f9259c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f94071970 0x7f5f940c4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.235+0000 7f5f9259c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f94071970 0x7f5f940c4c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47252/0 (socket says 172.21.15.28:47252) 2023-11-28T11:09:08.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.235+0000 7f5f9259c700 1 -- 172.21.15.28:0/84822416 learned_addr learned my addr 172.21.15.28:0/84822416 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:08.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.235+0000 7f5f9359e700 1 --2- 172.21.15.28:0/84822416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f940cd120 0x7f5f940c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.234+0000 7f21a99a8700 1 -- 172.21.15.28:0/3265254465 wait complete. 2023-11-28T11:09:08.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f21a99a8700 1 Processor -- start 2023-11-28T11:09:08.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.235+0000 7f3032790700 1 -- start start 2023-11-28T11:09:08.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f3032790700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f302c14c850 0x7f302c0c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f5f9259c700 1 -- 172.21.15.28:0/84822416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f9406b320 msgr2=0x7f5f940b1f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f5f9259c700 1 --2- 172.21.15.28:0/84822416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f9406b320 0x7f5f940b1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f5f9259c700 1 -- 172.21.15.28:0/84822416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f940cd120 msgr2=0x7f5f940c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f5f9259c700 1 --2- 172.21.15.28:0/84822416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f940cd120 0x7f5f940c3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f5f9259c700 1 -- 172.21.15.28:0/84822416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f8c009a70 con 0x7f5f94071970 2023-11-28T11:09:08.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f3032790700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f302c0c8530 0x7f302c0c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f3032790700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f302c0c49c0 0x7f302c0c4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f3032790700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0c37b0 con 0x7f302c0c8530 2023-11-28T11:09:08.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f3032790700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0c5370 con 0x7f302c0c49c0 2023-11-28T11:09:08.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f3032790700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0c68f0 con 0x7f302c14c850 2023-11-28T11:09:08.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f21a99a8700 1 -- start start 2023-11-28T11:09:08.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f302bfff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f302c14c850 0x7f302c0c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f302bfff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f302c14c850 0x7f302c0c63b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:53012/0 (socket says 172.21.15.28:53012) 2023-11-28T11:09:08.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f302bfff700 1 -- 172.21.15.28:0/484781435 learned_addr learned my addr 172.21.15.28:0/484781435 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:08.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f30237fe700 1 --2- 172.21.15.28:0/484781435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f302c0c8530 0x7f302c0c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f3030d2d700 1 --2- 172.21.15.28:0/484781435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f302c0c49c0 0x7f302c0c4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f5f9259c700 1 --2- 172.21.15.28:0/84822416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f94071970 0x7f5f940c4c50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800b4a0 tx=0x7f5f8800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f30237fe700 1 -- 172.21.15.28:0/484781435 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f302c14c850 msgr2=0x7f302c0c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f30237fe700 1 --2- 172.21.15.28:0/484781435 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f302c14c850 0x7f302c0c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f30237fe700 1 -- 172.21.15.28:0/484781435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f302c0c49c0 msgr2=0x7f302c0c4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f30237fe700 1 --2- 172.21.15.28:0/484781435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f302c0c49c0 0x7f302c0c4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f30237fe700 1 -- 172.21.15.28:0/484781435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c06e9c0 con 0x7f302c0c8530 2023-11-28T11:09:08.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f302bfff700 1 --2- 172.21.15.28:0/484781435 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f302c14c850 0x7f302c0c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:08.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f30237fe700 1 --2- 172.21.15.28:0/484781435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f302c0c8530 0x7f302c0c89a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f301400e5c0 tx=0x7f301400e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f21a99a8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21a406b320 0x7f21a414b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f21a99a8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21a40cd120 0x7f21a4146e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f21a99a8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a4149450 0x7f21a4147340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f21a99a8700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a41482a0 con 0x7f21a40cd120 2023-11-28T11:09:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f21a99a8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4147970 con 0x7f21a4149450 2023-11-28T11:09:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f21a99a8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21a4147c50 con 0x7f21a406b320 2023-11-28T11:09:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.236+0000 7f88edd6f700 1 -- start start 2023-11-28T11:09:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f88edd6f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88e806b860 0x7f88e8201af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f88edd6f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e80cd120 0x7f88e81f3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f88edd6f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88e81588a0 0x7f88e81e2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f88edd6f700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e81f45c0 con 0x7f88e806b860 2023-11-28T11:09:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f88edd6f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88e81fec40 con 0x7f88e80cd120 2023-11-28T11:09:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f88edd6f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88e81feee0 con 0x7f88e81588a0 2023-11-28T11:09:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f21a2ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21a406b320 0x7f21a414b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f21a2ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21a406b320 0x7f21a414b6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:53020/0 (socket says 172.21.15.28:53020) 2023-11-28T11:09:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f21a2ffd700 1 -- 172.21.15.28:0/2116153355 learned_addr learned my addr 172.21.15.28:0/2116153355 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:08.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f21a37fe700 1 --2- 172.21.15.28:0/2116153355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a4149450 0x7f21a4147340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f88e7fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88e81588a0 0x7f88e81e2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f88e7fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88e81588a0 0x7f88e81e2ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:53026/0 (socket says 172.21.15.28:53026) 2023-11-28T11:09:08.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f88e7fff700 1 -- 172.21.15.28:0/2403998586 learned_addr learned my addr 172.21.15.28:0/2403998586 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:08.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f30297fa700 1 -- 172.21.15.28:0/484781435 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f301400c070 con 0x7f302c0c8530 2023-11-28T11:09:08.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f30297fa700 1 -- 172.21.15.28:0/484781435 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f301400ee90 con 0x7f302c0c8530 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f88e6ffd700 1 --2- 172.21.15.28:0/2403998586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e80cd120 0x7f88e81f3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f21a27fc700 1 --2- 172.21.15.28:0/2116153355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21a40cd120 0x7f21a4146e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f21a2ffd700 1 -- 172.21.15.28:0/2116153355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a4149450 msgr2=0x7f21a4147340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f21a2ffd700 1 --2- 172.21.15.28:0/2116153355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a4149450 0x7f21a4147340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f21a2ffd700 1 -- 172.21.15.28:0/2116153355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21a40cd120 msgr2=0x7f21a4146e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f21a2ffd700 1 --2- 172.21.15.28:0/2116153355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21a40cd120 0x7f21a4146e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f21a2ffd700 1 -- 172.21.15.28:0/2116153355 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21940094a0 con 0x7f21a406b320 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f21a27fc700 1 --2- 172.21.15.28:0/2116153355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21a40cd120 0x7f21a4146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f88e77fe700 1 --2- 172.21.15.28:0/2403998586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88e806b860 0x7f88e8201af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f3032790700 1 -- 172.21.15.28:0/484781435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302c06f340 con 0x7f302c0c8530 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f3032790700 1 -- 172.21.15.28:0/484781435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302c06eb50 con 0x7f302c0c8530 2023-11-28T11:09:08.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f30297fa700 1 -- 172.21.15.28:0/484781435 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3014004260 con 0x7f302c0c8530 2023-11-28T11:09:08.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f88e7fff700 1 -- 172.21.15.28:0/2403998586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e80cd120 msgr2=0x7f88e81f3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f88e7fff700 1 --2- 172.21.15.28:0/2403998586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e80cd120 0x7f88e81f3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f88e7fff700 1 -- 172.21.15.28:0/2403998586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88e806b860 msgr2=0x7f88e8201af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f88e7fff700 1 --2- 172.21.15.28:0/2403998586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88e806b860 0x7f88e8201af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.238+0000 7f88e7fff700 1 -- 172.21.15.28:0/2403998586 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88e0009b30 con 0x7f88e81588a0 2023-11-28T11:09:08.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.239+0000 7f88e7fff700 1 --2- 172.21.15.28:0/2403998586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88e81588a0 0x7f88e81e2ec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f88e000eed0 tx=0x7f88e000b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f5f7b7fe700 1 -- 172.21.15.28:0/84822416 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f88011830 con 0x7f5f94071970 2023-11-28T11:09:08.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f5f7b7fe700 1 -- 172.21.15.28:0/84822416 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f88011990 con 0x7f5f94071970 2023-11-28T11:09:08.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f5f7b7fe700 1 -- 172.21.15.28:0/84822416 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f88011d40 con 0x7f5f94071970 2023-11-28T11:09:08.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f5f990f0700 1 -- 172.21.15.28:0/84822416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f9406ae00 con 0x7f5f94071970 2023-11-28T11:09:08.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.237+0000 7f5f990f0700 1 -- 172.21.15.28:0/84822416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f9406ab20 con 0x7f5f94071970 2023-11-28T11:09:08.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.239+0000 7f5f990f0700 1 -- 172.21.15.28:0/84822416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f740052f0 con 0x7f5f94071970 2023-11-28T11:09:08.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.239+0000 7f21a2ffd700 1 --2- 172.21.15.28:0/2116153355 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21a406b320 0x7f21a414b6f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f219800e810 tx=0x7f219800eb20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.240+0000 7f30297fa700 1 -- 172.21.15.28:0/484781435 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f30140043c0 con 0x7f302c0c8530 2023-11-28T11:09:08.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.241+0000 7f30297fa700 1 --2- 172.21.15.28:0/484781435 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f300c0683b0 0x7f300c06a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.241+0000 7f30297fa700 1 -- 172.21.15.28:0/484781435 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3014085890 con 0x7f302c0c8530 2023-11-28T11:09:08.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7f3032790700 1 -- 172.21.15.28:0/484781435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30100052f0 con 0x7f302c0c8530 2023-11-28T11:09:08.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.241+0000 7f5f7b7fe700 1 -- 172.21.15.28:0/84822416 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5f88021070 con 0x7f5f94071970 2023-11-28T11:09:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.240+0000 7f218bfff700 1 -- 172.21.15.28:0/2116153355 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f219800c070 con 0x7f21a406b320 2023-11-28T11:09:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.240+0000 7f218bfff700 1 -- 172.21.15.28:0/2116153355 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2198013610 con 0x7f21a406b320 2023-11-28T11:09:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.240+0000 7f218bfff700 1 -- 172.21.15.28:0/2116153355 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2198013980 con 0x7f21a406b320 2023-11-28T11:09:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.240+0000 7f21a99a8700 1 -- 172.21.15.28:0/2116153355 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21a406ae00 con 0x7f21a406b320 2023-11-28T11:09:08.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.240+0000 7f21a99a8700 1 -- 172.21.15.28:0/2116153355 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21a406ab20 con 0x7f21a406b320 2023-11-28T11:09:08.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.241+0000 7f88cffff700 1 -- 172.21.15.28:0/2403998586 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e0018070 con 0x7f88e81588a0 2023-11-28T11:09:08.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.241+0000 7f88cffff700 1 -- 172.21.15.28:0/2403998586 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88e0003a20 con 0x7f88e81588a0 2023-11-28T11:09:08.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.241+0000 7f88cffff700 1 -- 172.21.15.28:0/2403998586 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88e0010040 con 0x7f88e81588a0 2023-11-28T11:09:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.240+0000 7ff682127700 1 Processor -- start 2023-11-28T11:09:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.240+0000 7ff682127700 1 -- start start 2023-11-28T11:09:08.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7ff682127700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff67c160be0 0x7ff67c0c98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7ff682127700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff67c0adec0 0x7ff67c16a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7ff682127700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff67c0cdb60 con 0x7ff67c0c9df0 2023-11-28T11:09:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7ff682127700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67c0cde40 con 0x7ff67c0adec0 2023-11-28T11:09:08.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7ff682127700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67c0ce120 con 0x7ff67c160be0 2023-11-28T11:09:08.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7ff67b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff67c160be0 0x7ff67c0c98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7ff67b7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff67c160be0 0x7ff67c0c98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:34562/0 (socket says 172.21.15.28:34562) 2023-11-28T11:09:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7ff67b7fe700 1 -- 172.21.15.28:0/4049630619 learned_addr learned my addr 172.21.15.28:0/4049630619 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:08.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7ff67b7fe700 1 -- 172.21.15.28:0/4049630619 >> v2:172.21.15.28:3300/0 conn(0x7ff67c0adec0 msgr2=0x7ff67c16a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7ff67b7fe700 1 --2- 172.21.15.28:0/4049630619 >> v2:172.21.15.28:3300/0 conn(0x7ff67c0adec0 0x7ff67c16a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7ff67b7fe700 1 -- 172.21.15.28:0/4049630619 >> v1:172.21.15.59:6789/0 conn(0x7ff67c0c9df0 legacy=0x7ff67c0ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7ff67b7fe700 1 -- 172.21.15.28:0/4049630619 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67c06a1e0 con 0x7ff67c160be0 2023-11-28T11:09:08.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7ff67b7fe700 1 --2- 172.21.15.28:0/4049630619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff67c160be0 0x7ff67c0c98b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0673f0 tx=0x7ff66c009bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c187487c2b35bed8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.240+0000 7f2a6ae2d700 1 Processor -- start 2023-11-28T11:09:08.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7f5f7b7fe700 1 --2- 172.21.15.28:0/84822416 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5f800683c0 0x7f5f8006a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7f5f7b7fe700 1 -- 172.21.15.28:0/84822416 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5f880864e0 con 0x7f5f94071970 2023-11-28T11:09:08.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7f5f92d9d700 1 --2- 172.21.15.28:0/84822416 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5f800683c0 0x7f5f8006a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.242+0000 7f2a6ae2d700 1 -- start start 2023-11-28T11:09:08.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.243+0000 7f302bfff700 1 --2- 172.21.15.28:0/484781435 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f300c0683b0 0x7f300c06a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.243+0000 7f2a6ae2d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2a64160c70 0x7f2a640adb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.243+0000 7f2a6ae2d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2a641646c0 0x7f2a64070960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.243+0000 7f2a6ae2d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a64070ea0 0x7f2a6406a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.243+0000 7f2a6ae2d700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a64071320 con 0x7f2a64160c70 2023-11-28T11:09:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.243+0000 7f2a6ae2d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a64161040 con 0x7f2a641646c0 2023-11-28T11:09:08.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.243+0000 7f2a6ae2d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a64164b30 con 0x7f2a64070ea0 2023-11-28T11:09:08.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.243+0000 7f2a68bc9700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2a64160c70 0x7f2a640adb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7f2a68bc9700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2a64160c70 0x7f2a640adb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47286/0 (socket says 172.21.15.28:47286) 2023-11-28T11:09:08.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7f2a68bc9700 1 -- 172.21.15.28:0/1688359259 learned_addr learned my addr 172.21.15.28:0/1688359259 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:08.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7f5f92d9d700 1 --2- 172.21.15.28:0/84822416 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5f800683c0 0x7f5f8006a870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5f84001670 tx=0x7f5f84007ad0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7f302bfff700 1 --2- 172.21.15.28:0/484781435 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f300c0683b0 0x7f300c06a860 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f301c00a6c0 tx=0x7f301c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7f2a68bc9700 1 -- 172.21.15.28:0/1688359259 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a64070ea0 msgr2=0x7f2a6406a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7f2a68bc9700 1 --2- 172.21.15.28:0/1688359259 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a64070ea0 0x7f2a6406a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7f2a68bc9700 1 -- 172.21.15.28:0/1688359259 >> v2:172.21.15.59:3300/0 conn(0x7f2a641646c0 msgr2=0x7f2a64070960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T11:09:08.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7f2a68bc9700 1 --2- 172.21.15.28:0/1688359259 >> v2:172.21.15.59:3300/0 conn(0x7f2a641646c0 0x7f2a64070960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7f2a68bc9700 1 -- 172.21.15.28:0/1688359259 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a64164e10 con 0x7f2a64160c70 2023-11-28T11:09:08.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7ff67a7fc700 1 -- 172.21.15.28:0/4049630619 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c004230 con 0x7ff67c160be0 2023-11-28T11:09:08.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7ff67a7fc700 1 -- 172.21.15.28:0/4049630619 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c004390 con 0x7ff67c160be0 2023-11-28T11:09:08.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7ff67a7fc700 1 -- 172.21.15.28:0/4049630619 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c0036a0 con 0x7ff67c160be0 2023-11-28T11:09:08.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7f2a68bc9700 1 --2- 172.21.15.28:0/1688359259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a64160c70 0x7f2a640adb00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2a640b4b70 tx=0x7f2a54009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e342013e79480b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.243+0000 7f218bfff700 1 -- 172.21.15.28:0/2116153355 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2198023070 con 0x7f21a406b320 2023-11-28T11:09:08.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7f218bfff700 1 --2- 172.21.15.28:0/2116153355 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f218c068860 0x7f218c06ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7f218bfff700 1 -- 172.21.15.28:0/2116153355 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2198085c70 con 0x7f21a406b320 2023-11-28T11:09:08.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7f21a27fc700 1 --2- 172.21.15.28:0/2116153355 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f218c068860 0x7f218c06ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.244+0000 7f88edd6f700 1 -- 172.21.15.28:0/2403998586 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88dc008520 con 0x7f88e81588a0 2023-11-28T11:09:08.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.245+0000 7f88edd6f700 1 -- 172.21.15.28:0/2403998586 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88e80ce550 con 0x7f88e81588a0 2023-11-28T11:09:08.288 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.245+0000 7f21a27fc700 1 --2- 172.21.15.28:0/2116153355 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f218c068860 0x7f218c06ad10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f21a40c8580 tx=0x7f219c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.288 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.245+0000 7f21a99a8700 1 -- 172.21.15.28:0/2116153355 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21840052f0 con 0x7f21a406b320 2023-11-28T11:09:08.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.245+0000 7ff682127700 1 -- 172.21.15.28:0/4049630619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff67c160be0 msgr2=0x7ff67c0c98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.245+0000 7ff682127700 1 --2- 172.21.15.28:0/4049630619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff67c160be0 0x7ff67c0c98b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0673f0 tx=0x7ff66c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.245+0000 7ff682127700 1 -- 172.21.15.28:0/4049630619 shutdown_connections 2023-11-28T11:09:08.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.245+0000 7ff682127700 1 --2- 172.21.15.28:0/4049630619 >> v2:172.21.15.28:3300/0 conn(0x7ff67c0adec0 0x7ff67c16a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.245+0000 7ff682127700 1 --2- 172.21.15.28:0/4049630619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff67c160be0 0x7ff67c0c98b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.245+0000 7ff682127700 1 -- 172.21.15.28:0/4049630619 >> 172.21.15.28:0/4049630619 conn(0x7ff67c0cf670 msgr2=0x7ff67c0cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.245+0000 7ff682127700 1 -- 172.21.15.28:0/4049630619 shutdown_connections 2023-11-28T11:09:08.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.245+0000 7fd7ba2d3700 1 Processor -- start 2023-11-28T11:09:08.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.245+0000 7fd7ba2d3700 1 -- start start 2023-11-28T11:09:08.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.245+0000 7ff682127700 1 -- 172.21.15.28:0/4049630619 wait complete. 2023-11-28T11:09:08.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.246+0000 7ff682127700 1 Processor -- start 2023-11-28T11:09:08.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7fd7ba2d3700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd7b406a1e0 0x7fd7b40ce350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7fd7ba2d3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b40cce20 0x7fd7b406a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.246+0000 7f88edd6f700 1 -- 172.21.15.28:0/2403998586 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88e806ae00 con 0x7f88e81588a0 2023-11-28T11:09:08.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7f88cffff700 1 -- 172.21.15.28:0/2403998586 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f88e000b920 con 0x7f88e81588a0 2023-11-28T11:09:08.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7fd7ba2d3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd7b40c9a40 0x7fd7b406aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7fd7ba2d3700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b415cb10 con 0x7fd7b406a1e0 2023-11-28T11:09:08.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7fd7ba2d3700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7b40ce890 con 0x7fd7b40c9a40 2023-11-28T11:09:08.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7fd7ba2d3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b406b030 con 0x7fd7b40cce20 2023-11-28T11:09:08.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7fd7b3fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd7b406a1e0 0x7fd7b40ce350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7fd7b3fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd7b406a1e0 0x7fd7b40ce350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47290/0 (socket says 172.21.15.28:47290) 2023-11-28T11:09:08.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7fd7b3fff700 1 -- 172.21.15.28:0/3226237102 learned_addr learned my addr 172.21.15.28:0/3226237102 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:08.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7fd7b37fe700 1 --2- 172.21.15.28:0/3226237102 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b40cce20 0x7fd7b406a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7f88cffff700 1 --2- 172.21.15.28:0/2403998586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f88d4068340 0x7f88d406a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7f88cffff700 1 -- 172.21.15.28:0/2403998586 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f88e0085fb0 con 0x7f88e81588a0 2023-11-28T11:09:08.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7f88e77fe700 1 --2- 172.21.15.28:0/2403998586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f88d4068340 0x7f88d406a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7fd7b3fff700 1 -- 172.21.15.28:0/3226237102 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b40cce20 msgr2=0x7fd7b406a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7fd7b3fff700 1 --2- 172.21.15.28:0/3226237102 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b40cce20 0x7fd7b406a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7fd7b3fff700 1 -- 172.21.15.28:0/3226237102 >> v2:172.21.15.59:3300/0 conn(0x7fd7b40c9a40 msgr2=0x7fd7b406aaf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T11:09:08.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7fd7b3fff700 1 --2- 172.21.15.28:0/3226237102 >> v2:172.21.15.59:3300/0 conn(0x7fd7b40c9a40 0x7fd7b406aaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7fd7b3fff700 1 -- 172.21.15.28:0/3226237102 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b40cd2b0 con 0x7fd7b406a1e0 2023-11-28T11:09:08.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7fd7b3fff700 1 --2- 172.21.15.28:0/3226237102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b406a1e0 0x7fd7b40ce350 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40cc4b0 tx=0x7fd7ac00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=def678d693186858 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7f88e77fe700 1 --2- 172.21.15.28:0/2403998586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f88d4068340 0x7f88d406a7f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f88e81e3c40 tx=0x7f88d8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.247+0000 7ff682127700 1 -- start start 2023-11-28T11:09:08.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7f5f7b7fe700 1 -- 172.21.15.28:0/84822416 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5f880529e0 con 0x7f5f94071970 2023-11-28T11:09:08.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7ff682127700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff67c0adec0 0x7ff67c14e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7ff682127700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff67c0c9df0 0x7ff67c162c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7ff682127700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff67c160be0 0x7ff67c06d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7ff682127700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c14f2a0 con 0x7ff67c0adec0 2023-11-28T11:09:08.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7ff682127700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c163170 con 0x7ff67c0c9df0 2023-11-28T11:09:08.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7ff682127700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67c14e720 con 0x7ff67c160be0 2023-11-28T11:09:08.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7ff67b7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff67c0adec0 0x7ff67c14e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7ff67b7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff67c0adec0 0x7ff67c14e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47296/0 (socket says 172.21.15.28:47296) 2023-11-28T11:09:08.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7ff67b7fe700 1 -- 172.21.15.28:0/3330063590 learned_addr learned my addr 172.21.15.28:0/3330063590 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:08.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7ff67affd700 1 --2- 172.21.15.28:0/3330063590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff67c0c9df0 0x7ff67c162c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7ff67bfff700 1 --2- 172.21.15.28:0/3330063590 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff67c160be0 0x7ff67c06d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7ff67b7fe700 1 -- 172.21.15.28:0/3330063590 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff67c160be0 msgr2=0x7ff67c06d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7ff67b7fe700 1 --2- 172.21.15.28:0/3330063590 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff67c160be0 0x7ff67c06d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.248+0000 7ff67b7fe700 1 -- 172.21.15.28:0/3330063590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff67c0c9df0 msgr2=0x7ff67c162c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.249+0000 7ff67b7fe700 1 --2- 172.21.15.28:0/3330063590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff67c0c9df0 0x7ff67c162c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.249+0000 7ff67b7fe700 1 -- 172.21.15.28:0/3330063590 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff66c009550 con 0x7ff67c0adec0 2023-11-28T11:09:08.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.249+0000 7ff67affd700 1 --2- 172.21.15.28:0/3330063590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff67c0c9df0 0x7ff67c162c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:08.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.249+0000 7ff67b7fe700 1 --2- 172.21.15.28:0/3330063590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff67c0adec0 0x7ff67c14e090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0cc4e0 tx=0x7ff66c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.251+0000 7f218bfff700 1 -- 172.21.15.28:0/2116153355 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2198055ba0 con 0x7f21a406b320 2023-11-28T11:09:08.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.253+0000 7f30297fa700 1 -- 172.21.15.28:0/484781435 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3014051d90 con 0x7f302c0c8530 2023-11-28T11:09:08.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.253+0000 7fd7b2ffd700 1 -- 172.21.15.28:0/3226237102 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac00e070 con 0x7fd7b406a1e0 2023-11-28T11:09:08.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.253+0000 7fd7b2ffd700 1 -- 172.21.15.28:0/3226237102 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7ac00b8d0 con 0x7fd7b406a1e0 2023-11-28T11:09:08.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.253+0000 7fd7b2ffd700 1 -- 172.21.15.28:0/3226237102 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7ac00bc80 con 0x7fd7b406a1e0 2023-11-28T11:09:08.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.254+0000 7fd7ba2d3700 1 -- 172.21.15.28:0/3226237102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b406a1e0 msgr2=0x7fd7b40ce350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.254+0000 7fd7ba2d3700 1 --2- 172.21.15.28:0/3226237102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b406a1e0 0x7fd7b40ce350 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40cc4b0 tx=0x7fd7ac00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:08.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.253+0000 7ff663fff700 1 -- 172.21.15.28:0/3330063590 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c003e60 con 0x7ff67c0adec0 2023-11-28T11:09:08.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.253+0000 7ff663fff700 1 -- 172.21.15.28:0/3330063590 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66c011630 con 0x7ff67c0adec0 2023-11-28T11:09:08.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.253+0000 7ff663fff700 1 -- 172.21.15.28:0/3330063590 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66c0119e0 con 0x7ff67c0adec0 2023-11-28T11:09:08.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.253+0000 7ff682127700 1 -- 172.21.15.28:0/3330063590 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff67c06cd50 con 0x7ff67c0adec0 2023-11-28T11:09:08.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.253+0000 7ff682127700 1 -- 172.21.15.28:0/3330063590 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff67c06a4c0 con 0x7ff67c0adec0 2023-11-28T11:09:08.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.255+0000 7ff663fff700 1 -- 172.21.15.28:0/3330063590 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff66c017020 con 0x7ff67c0adec0 2023-11-28T11:09:08.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.256+0000 7ff663fff700 1 --2- 172.21.15.28:0/3330063590 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff6680682f0 0x7ff66806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.256+0000 7ff663fff700 1 -- 172.21.15.28:0/3330063590 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ff66c0874b0 con 0x7ff67c0adec0 2023-11-28T11:09:08.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.253+0000 7f2a637fe700 1 -- 172.21.15.28:0/1688359259 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400b870 con 0x7f2a64160c70 2023-11-28T11:09:08.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.255+0000 7f2a637fe700 1 -- 172.21.15.28:0/1688359259 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5400b9d0 con 0x7f2a64160c70 2023-11-28T11:09:08.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.255+0000 7f2a637fe700 1 -- 172.21.15.28:0/1688359259 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5400bd40 con 0x7f2a64160c70 2023-11-28T11:09:08.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.256+0000 7f2a6ae2d700 1 -- 172.21.15.28:0/1688359259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a64160c70 msgr2=0x7f2a640adb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.256+0000 7f2a6ae2d700 1 --2- 172.21.15.28:0/1688359259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a64160c70 0x7f2a640adb00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2a640b4b70 tx=0x7f2a54009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.257+0000 7f2a6ae2d700 1 -- 172.21.15.28:0/1688359259 shutdown_connections 2023-11-28T11:09:08.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.257+0000 7f2a6ae2d700 1 --2- 172.21.15.28:0/1688359259 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a64070ea0 0x7f2a6406a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.257+0000 7f2a6ae2d700 1 --2- 172.21.15.28:0/1688359259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a64160c70 0x7f2a640adb00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.257+0000 7f2a6ae2d700 1 --2- 172.21.15.28:0/1688359259 >> v2:172.21.15.59:3300/0 conn(0x7f2a641646c0 0x7f2a64070960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.257+0000 7f2a6ae2d700 1 -- 172.21.15.28:0/1688359259 >> 172.21.15.28:0/1688359259 conn(0x7f2a640ce9e0 msgr2=0x7f2a640cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.257+0000 7ff682127700 1 -- 172.21.15.28:0/3330063590 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67c0cd830 con 0x7ff67c0adec0 2023-11-28T11:09:08.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.259+0000 7f88cffff700 1 -- 172.21.15.28:0/2403998586 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f88e00524b0 con 0x7f88e81588a0 2023-11-28T11:09:08.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.259+0000 7fd7ba2d3700 1 -- 172.21.15.28:0/3226237102 shutdown_connections 2023-11-28T11:09:08.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.259+0000 7fd7ba2d3700 1 --2- 172.21.15.28:0/3226237102 >> v2:172.21.15.59:3300/0 conn(0x7fd7b40c9a40 0x7fd7b406aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.259+0000 7fd7ba2d3700 1 --2- 172.21.15.28:0/3226237102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b406a1e0 0x7fd7b40ce350 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.259+0000 7fd7ba2d3700 1 --2- 172.21.15.28:0/3226237102 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b40cce20 0x7fd7b406a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.259+0000 7fd7ba2d3700 1 -- 172.21.15.28:0/3226237102 >> 172.21.15.28:0/3226237102 conn(0x7fd7b40cf670 msgr2=0x7fd7b40cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.261+0000 7f2a6ae2d700 1 -- 172.21.15.28:0/1688359259 shutdown_connections 2023-11-28T11:09:08.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.261+0000 7f2a6ae2d700 1 -- 172.21.15.28:0/1688359259 wait complete. 2023-11-28T11:09:08.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.263+0000 7f2a6ae2d700 1 Processor -- start 2023-11-28T11:09:08.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.263+0000 7f2a6ae2d700 1 -- start start 2023-11-28T11:09:08.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.264+0000 7f2a6ae2d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a64070ea0 0x7f2a64162ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.264+0000 7f2a6ae2d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a64160c70 0x7f2a640c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.264+0000 7f2a6ae2d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a641646c0 0x7f2a6414c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.264+0000 7f2a6ae2d700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a64161730 con 0x7f2a641646c0 2023-11-28T11:09:08.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.264+0000 7f2a6ae2d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a64161a10 con 0x7f2a64070ea0 2023-11-28T11:09:08.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.264+0000 7f2a6ae2d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a64161cf0 con 0x7f2a64160c70 2023-11-28T11:09:08.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.264+0000 7f2a693ca700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a641646c0 0x7f2a6414c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.264+0000 7f2a693ca700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a641646c0 0x7f2a6414c670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47306/0 (socket says 172.21.15.28:47306) 2023-11-28T11:09:08.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.264+0000 7f2a693ca700 1 -- 172.21.15.28:0/2934433135 learned_addr learned my addr 172.21.15.28:0/2934433135 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:08.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.265+0000 7f2a63fff700 1 --2- 172.21.15.28:0/2934433135 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a64160c70 0x7f2a640c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.265+0000 7f2a693ca700 1 -- 172.21.15.28:0/2934433135 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a64160c70 msgr2=0x7f2a640c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.265+0000 7f2a693ca700 1 --2- 172.21.15.28:0/2934433135 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a64160c70 0x7f2a640c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.265+0000 7f2a693ca700 1 -- 172.21.15.28:0/2934433135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a64070ea0 msgr2=0x7f2a64162ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.265+0000 7f2a693ca700 1 --2- 172.21.15.28:0/2934433135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a64070ea0 0x7f2a64162ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.265+0000 7f2a693ca700 1 -- 172.21.15.28:0/2934433135 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a54009550 con 0x7f2a641646c0 2023-11-28T11:09:08.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.265+0000 7f2a63fff700 1 --2- 172.21.15.28:0/2934433135 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a64160c70 0x7f2a640c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:08.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.265+0000 7f2a693ca700 1 --2- 172.21.15.28:0/2934433135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a641646c0 0x7f2a6414c670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00b970 tx=0x7f2a5c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.266+0000 7fd7ba2d3700 1 -- 172.21.15.28:0/3226237102 shutdown_connections 2023-11-28T11:09:08.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.266+0000 7fd7ba2d3700 1 -- 172.21.15.28:0/3226237102 wait complete. 2023-11-28T11:09:08.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.268+0000 7fd7ba2d3700 1 Processor -- start 2023-11-28T11:09:08.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.268+0000 7fd7ba2d3700 1 -- start start 2023-11-28T11:09:08.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.269+0000 7fd7ba2d3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b406a1e0 0x7fd7b414ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.269+0000 7fd7ba2d3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b40c9a40 0x7fd7b40c23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.269+0000 7fd7ba2d3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b40cce20 0x7fd7b41f5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.270+0000 7fd7ba2d3700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b414d3a0 con 0x7fd7b406a1e0 2023-11-28T11:09:08.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.270+0000 7fd7ba2d3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b41f6b20 con 0x7fd7b40c9a40 2023-11-28T11:09:08.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.270+0000 7fd7ba2d3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7b41f6560 con 0x7fd7b40cce20 2023-11-28T11:09:08.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.268+0000 7ff663fff700 1 -- 172.21.15.28:0/3330063590 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff66c053a20 con 0x7ff67c0adec0 2023-11-28T11:09:08.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.268+0000 7f2a617fa700 1 -- 172.21.15.28:0/2934433135 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c00c930 con 0x7f2a641646c0 2023-11-28T11:09:08.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.268+0000 7f2a617fa700 1 -- 172.21.15.28:0/2934433135 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a5c00cab0 con 0x7f2a641646c0 2023-11-28T11:09:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.268+0000 7f2a617fa700 1 -- 172.21.15.28:0/2934433135 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5c00ce60 con 0x7f2a641646c0 2023-11-28T11:09:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.270+0000 7fd7b8870700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b40cce20 0x7fd7b41f5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.270+0000 7fd7b8870700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b40cce20 0x7fd7b41f5bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:53070/0 (socket says 172.21.15.28:53070) 2023-11-28T11:09:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.270+0000 7fd7b8870700 1 -- 172.21.15.28:0/3484553650 learned_addr learned my addr 172.21.15.28:0/3484553650 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.270+0000 7fd7b8870700 1 -- 172.21.15.28:0/3484553650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b40c9a40 msgr2=0x7fd7b40c23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.270+0000 7fd7b8870700 1 --2- 172.21.15.28:0/3484553650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b40c9a40 0x7fd7b40c23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.270+0000 7fd7b8870700 1 -- 172.21.15.28:0/3484553650 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b406a1e0 msgr2=0x7fd7b414ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.270+0000 7fd7b8870700 1 --2- 172.21.15.28:0/3484553650 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b406a1e0 0x7fd7b414ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.270+0000 7fd7b8870700 1 -- 172.21.15.28:0/3484553650 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ac009b30 con 0x7fd7b40cce20 2023-11-28T11:09:08.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.270+0000 7f2a6ae2d700 1 -- 172.21.15.28:0/2934433135 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a641653d0 con 0x7f2a641646c0 2023-11-28T11:09:08.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.270+0000 7f2a6ae2d700 1 -- 172.21.15.28:0/2934433135 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a641650f0 con 0x7f2a641646c0 2023-11-28T11:09:08.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.272+0000 7f2a617fa700 1 -- 172.21.15.28:0/2934433135 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2a5c01b030 con 0x7f2a641646c0 2023-11-28T11:09:08.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.273+0000 7f2a617fa700 1 --2- 172.21.15.28:0/2934433135 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2a50068320 0x7f2a5006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.273+0000 7f2a617fa700 1 -- 172.21.15.28:0/2934433135 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2a5c006e40 con 0x7f2a641646c0 2023-11-28T11:09:08.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.273+0000 7f2a68bc9700 1 --2- 172.21.15.28:0/2934433135 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2a50068320 0x7f2a5006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.274+0000 7f2a68bc9700 1 --2- 172.21.15.28:0/2934433135 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2a50068320 0x7f2a5006a7d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009c10 tx=0x7f2a54010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.277+0000 7ff67affd700 1 --2- 172.21.15.28:0/3330063590 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff6680682f0 0x7ff66806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.277+0000 7fd7b8870700 1 --2- 172.21.15.28:0/3484553650 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b40cce20 0x7fd7b41f5bf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400b4a0 tx=0x7fd7a400b860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.279+0000 7ff67affd700 1 --2- 172.21.15.28:0/3330063590 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff6680682f0 0x7ff66806a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff67c06e190 tx=0x7ff670007ec0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.281+0000 7fd7b0ff9700 1 -- 172.21.15.28:0/3484553650 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a4014e30 con 0x7fd7b40cce20 2023-11-28T11:09:08.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.281+0000 7fd7b0ff9700 1 -- 172.21.15.28:0/3484553650 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7a401c440 con 0x7fd7b40cce20 2023-11-28T11:09:08.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.281+0000 7fd7b0ff9700 1 -- 172.21.15.28:0/3484553650 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7a401c7f0 con 0x7fd7b40cce20 2023-11-28T11:09:08.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.283+0000 7fd7ba2d3700 1 -- 172.21.15.28:0/3484553650 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7b40cd870 con 0x7fd7b40cce20 2023-11-28T11:09:08.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.283+0000 7fd7ba2d3700 1 -- 172.21.15.28:0/3484553650 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7b40cd590 con 0x7fd7b40cce20 2023-11-28T11:09:08.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.285+0000 7fd7b0ff9700 1 -- 172.21.15.28:0/3484553650 <== mon.2 v2:172.21.15.59:3301/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd7a4023030 con 0x7fd7b40cce20 2023-11-28T11:09:08.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.285+0000 7fd7ba2d3700 1 -- 172.21.15.28:0/3484553650 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7b40cdb50 con 0x7fd7b40cce20 2023-11-28T11:09:08.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.285+0000 7fd7b0ff9700 1 --2- 172.21.15.28:0/3484553650 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7a0068340 0x7fd7a006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.286+0000 7fd7b0ff9700 1 -- 172.21.15.28:0/3484553650 <== mon.2 v2:172.21.15.59:3301/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fd7a408dbf0 con 0x7fd7b40cce20 2023-11-28T11:09:08.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.287+0000 7f2a6ae2d700 1 -- 172.21.15.28:0/2934433135 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a640ae040 con 0x7f2a641646c0 2023-11-28T11:09:08.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.294+0000 7fd7b0ff9700 1 -- 172.21.15.28:0/3484553650 <== mon.2 v2:172.21.15.59:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd7a405a200 con 0x7fd7b40cce20 2023-11-28T11:09:08.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.295+0000 7fd7b3fff700 1 --2- 172.21.15.28:0/3484553650 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7a0068340 0x7fd7a006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.296+0000 7fd7b3fff700 1 --2- 172.21.15.28:0/3484553650 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7a0068340 0x7fd7a006a7f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40c4070 tx=0x7fd7ac010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.296+0000 7f2a617fa700 1 -- 172.21.15.28:0/2934433135 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2a5c04d5e0 con 0x7f2a641646c0 2023-11-28T11:09:08.593 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.593+0000 7f5973ab1700 1 -- 172.21.15.28:0/3571176403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f59500055d0 con 0x7f596c151480 2023-11-28T11:09:08.594 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.593+0000 7f59627fc700 1 -- 172.21.15.28:0/3571176403 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f59680572b0 con 0x7f596c151480 2023-11-28T11:09:08.596 INFO:teuthology.orchestra.run.smithi028.stdout:55834574856 2023-11-28T11:09:08.597 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.597+0000 7f5973ab1700 1 -- 172.21.15.28:0/3571176403 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5958069f70 msgr2=0x7f595806c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.597+0000 7f5973ab1700 1 --2- 172.21.15.28:0/3571176403 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5958069f70 0x7f595806c420 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f596c071cb0 tx=0x7f595c006cb0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.597+0000 7f5973ab1700 1 -- 172.21.15.28:0/3571176403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f596c151480 msgr2=0x7f596c162710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.597+0000 7f5973ab1700 1 --2- 172.21.15.28:0/3571176403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f596c151480 0x7f596c162710 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f595800c580 tx=0x7f59680017e0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.597+0000 7f5973ab1700 1 -- 172.21.15.28:0/3571176403 shutdown_connections 2023-11-28T11:09:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.597+0000 7f5973ab1700 1 --2- 172.21.15.28:0/3571176403 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5958069f70 0x7f595806c420 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.597+0000 7f5973ab1700 1 --2- 172.21.15.28:0/3571176403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f596c162c50 0x7f596c1434e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.597+0000 7f5973ab1700 1 --2- 172.21.15.28:0/3571176403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f59580037f0 0x7f596c150f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.597+0000 7f5973ab1700 1 --2- 172.21.15.28:0/3571176403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f596c151480 0x7f596c162710 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.598 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.597+0000 7f5973ab1700 1 -- 172.21.15.28:0/3571176403 >> 172.21.15.28:0/3571176403 conn(0x7f596c0cf670 msgr2=0x7f596c06e6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.597+0000 7f5973ab1700 1 -- 172.21.15.28:0/3571176403 shutdown_connections 2023-11-28T11:09:08.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.597+0000 7f5973ab1700 1 -- 172.21.15.28:0/3571176403 wait complete. 2023-11-28T11:09:08.618 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2023-11-28T11:09:08.619 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:08.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.630+0000 7f5f990f0700 1 -- 172.21.15.28:0/84822416 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f5f74005870 con 0x7f5f94071970 2023-11-28T11:09:08.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.631+0000 7f5f7b7fe700 1 -- 172.21.15.28:0/84822416 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5f88056000 con 0x7f5f94071970 2023-11-28T11:09:08.632 INFO:teuthology.orchestra.run.smithi028.stdout:55834574857 2023-11-28T11:09:08.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.634+0000 7f21a99a8700 1 -- 172.21.15.28:0/2116153355 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f21840055a0 con 0x7f21a406b320 2023-11-28T11:09:08.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.635+0000 7f218bfff700 1 -- 172.21.15.28:0/2116153355 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2198055790 con 0x7f21a406b320 2023-11-28T11:09:08.635 INFO:teuthology.orchestra.run.smithi028.stdout:55834574856 2023-11-28T11:09:08.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.638+0000 7f21a99a8700 1 -- 172.21.15.28:0/2116153355 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f218c068860 msgr2=0x7f218c06ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.638+0000 7f21a99a8700 1 --2- 172.21.15.28:0/2116153355 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f218c068860 0x7f218c06ad10 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f21a40c8580 tx=0x7f219c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:08.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.638+0000 7f21a99a8700 1 -- 172.21.15.28:0/2116153355 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21a406b320 msgr2=0x7f21a414b6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.638+0000 7f21a99a8700 1 --2- 172.21.15.28:0/2116153355 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21a406b320 0x7f21a414b6f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f219800e810 tx=0x7f219800eb20 comp rx=0 tx=0).stop 2023-11-28T11:09:08.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.638+0000 7f5f990f0700 1 -- 172.21.15.28:0/84822416 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5f800683c0 msgr2=0x7f5f8006a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.638+0000 7f5f990f0700 1 --2- 172.21.15.28:0/84822416 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5f800683c0 0x7f5f8006a870 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5f84001670 tx=0x7f5f84007ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.638+0000 7f5f990f0700 1 -- 172.21.15.28:0/84822416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f94071970 msgr2=0x7f5f940c4c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.638+0000 7f5f990f0700 1 --2- 172.21.15.28:0/84822416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f94071970 0x7f5f940c4c50 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5f8800b4a0 tx=0x7f5f8800b860 comp rx=0 tx=0).stop 2023-11-28T11:09:08.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.638+0000 7f21a99a8700 1 -- 172.21.15.28:0/2116153355 shutdown_connections 2023-11-28T11:09:08.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.638+0000 7f21a99a8700 1 --2- 172.21.15.28:0/2116153355 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f218c068860 0x7f218c06ad10 secure :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f21a40c8580 tx=0x7f219c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:08.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.638+0000 7f21a99a8700 1 --2- 172.21.15.28:0/2116153355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a4149450 0x7f21a4147340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.638+0000 7f21a99a8700 1 --2- 172.21.15.28:0/2116153355 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21a406b320 0x7f21a414b6f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.638+0000 7f21a99a8700 1 --2- 172.21.15.28:0/2116153355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21a40cd120 0x7f21a4146e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.638+0000 7f21a99a8700 1 -- 172.21.15.28:0/2116153355 >> 172.21.15.28:0/2116153355 conn(0x7f21a40cf670 msgr2=0x7f21a40b38c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.639+0000 7f5f990f0700 1 -- 172.21.15.28:0/84822416 shutdown_connections 2023-11-28T11:09:08.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.639+0000 7f5f990f0700 1 --2- 172.21.15.28:0/84822416 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5f800683c0 0x7f5f8006a870 secure :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f5f84001670 tx=0x7f5f84007ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.639+0000 7f5f990f0700 1 --2- 172.21.15.28:0/84822416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f940cd120 0x7f5f940c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.639+0000 7f5f990f0700 1 --2- 172.21.15.28:0/84822416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f9406b320 0x7f5f940b1f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.639+0000 7f5f990f0700 1 --2- 172.21.15.28:0/84822416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f94071970 0x7f5f940c4c50 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.639+0000 7f5f990f0700 1 -- 172.21.15.28:0/84822416 >> 172.21.15.28:0/84822416 conn(0x7f5f940cf670 msgr2=0x7f5f940c9d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.641+0000 7f5f990f0700 1 -- 172.21.15.28:0/84822416 shutdown_connections 2023-11-28T11:09:08.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.644+0000 7f5f990f0700 1 -- 172.21.15.28:0/84822416 wait complete. 2023-11-28T11:09:08.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.644+0000 7f21a99a8700 1 -- 172.21.15.28:0/2116153355 shutdown_connections 2023-11-28T11:09:08.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.644+0000 7f21a99a8700 1 -- 172.21.15.28:0/2116153355 wait complete. 2023-11-28T11:09:08.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.647+0000 7f3032790700 1 -- 172.21.15.28:0/484781435 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f30100055a0 con 0x7f302c0c8530 2023-11-28T11:09:08.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.650+0000 7f30297fa700 1 -- 172.21.15.28:0/484781435 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f30140553b0 con 0x7f302c0c8530 2023-11-28T11:09:08.651 INFO:teuthology.orchestra.run.smithi028.stdout:55834574857 2023-11-28T11:09:08.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.650+0000 7f88edd6f700 1 -- 172.21.15.28:0/2403998586 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f88e80ce6e0 con 0x7f88e81588a0 2023-11-28T11:09:08.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.651+0000 7f88cffff700 1 -- 172.21.15.28:0/2403998586 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f88e001b320 con 0x7f88e81588a0 2023-11-28T11:09:08.653 INFO:teuthology.orchestra.run.smithi028.stdout:55834574857 2023-11-28T11:09:08.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.654+0000 7f3032790700 1 -- 172.21.15.28:0/484781435 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f300c0683b0 msgr2=0x7f300c06a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.654+0000 7f3032790700 1 --2- 172.21.15.28:0/484781435 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f300c0683b0 0x7f300c06a860 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f301c00a6c0 tx=0x7f301c008040 comp rx=0 tx=0).stop 2023-11-28T11:09:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.654+0000 7f3032790700 1 -- 172.21.15.28:0/484781435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f302c0c8530 msgr2=0x7f302c0c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.654+0000 7f3032790700 1 --2- 172.21.15.28:0/484781435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f302c0c8530 0x7f302c0c89a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f301400e5c0 tx=0x7f301400e8d0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.654+0000 7f3032790700 1 -- 172.21.15.28:0/484781435 shutdown_connections 2023-11-28T11:09:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.654+0000 7f3032790700 1 --2- 172.21.15.28:0/484781435 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f300c0683b0 0x7f300c06a860 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.654+0000 7f3032790700 1 --2- 172.21.15.28:0/484781435 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f302c0c49c0 0x7f302c0c4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.654+0000 7f3032790700 1 --2- 172.21.15.28:0/484781435 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f302c14c850 0x7f302c0c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.654+0000 7f3032790700 1 --2- 172.21.15.28:0/484781435 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f302c0c8530 0x7f302c0c89a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.654+0000 7f3032790700 1 -- 172.21.15.28:0/484781435 >> 172.21.15.28:0/484781435 conn(0x7f302c0b0dd0 msgr2=0x7f302c06ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.654+0000 7f3032790700 1 -- 172.21.15.28:0/484781435 shutdown_connections 2023-11-28T11:09:08.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.654+0000 7f3032790700 1 -- 172.21.15.28:0/484781435 wait complete. 2023-11-28T11:09:08.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.657+0000 7f88edd6f700 1 -- 172.21.15.28:0/2403998586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f88d4068340 msgr2=0x7f88d406a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.657+0000 7f88edd6f700 1 --2- 172.21.15.28:0/2403998586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f88d4068340 0x7f88d406a7f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f88e81e3c40 tx=0x7f88d8008040 comp rx=0 tx=0).stop 2023-11-28T11:09:08.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.657+0000 7f88edd6f700 1 -- 172.21.15.28:0/2403998586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88e81588a0 msgr2=0x7f88e81e2ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.657+0000 7f88edd6f700 1 --2- 172.21.15.28:0/2403998586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88e81588a0 0x7f88e81e2ec0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f88e000eed0 tx=0x7f88e000b390 comp rx=0 tx=0).stop 2023-11-28T11:09:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.658+0000 7f88edd6f700 1 -- 172.21.15.28:0/2403998586 shutdown_connections 2023-11-28T11:09:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.658+0000 7f88edd6f700 1 --2- 172.21.15.28:0/2403998586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f88d4068340 0x7f88d406a7f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.658+0000 7f88edd6f700 1 --2- 172.21.15.28:0/2403998586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f88e806b860 0x7f88e8201af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.658+0000 7f88edd6f700 1 --2- 172.21.15.28:0/2403998586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f88e81588a0 0x7f88e81e2ec0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.658+0000 7f88edd6f700 1 --2- 172.21.15.28:0/2403998586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88e80cd120 0x7f88e81f3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.658+0000 7f88edd6f700 1 -- 172.21.15.28:0/2403998586 >> 172.21.15.28:0/2403998586 conn(0x7f88e80cf670 msgr2=0x7f88e80ae9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.660+0000 7f88edd6f700 1 -- 172.21.15.28:0/2403998586 shutdown_connections 2023-11-28T11:09:08.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.660+0000 7f88edd6f700 1 -- 172.21.15.28:0/2403998586 wait complete. 2023-11-28T11:09:08.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.665+0000 7ff682127700 1 -- 172.21.15.28:0/3330063590 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7ff67c068570 con 0x7ff67c0adec0 2023-11-28T11:09:08.667 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.6 2023-11-28T11:09:08.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.666+0000 7ff663fff700 1 -- 172.21.15.28:0/3330063590 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff66c028030 con 0x7ff67c0adec0 2023-11-28T11:09:08.667 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:08.668 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.2 2023-11-28T11:09:08.668 INFO:teuthology.orchestra.run.smithi028.stdout:55834574857 2023-11-28T11:09:08.668 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:08.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.668+0000 7ff682127700 1 -- 172.21.15.28:0/3330063590 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff6680682f0 msgr2=0x7ff66806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.668+0000 7ff682127700 1 --2- 172.21.15.28:0/3330063590 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff6680682f0 0x7ff66806a7a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ff67c06e190 tx=0x7ff670007ec0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.668+0000 7ff682127700 1 -- 172.21.15.28:0/3330063590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff67c0adec0 msgr2=0x7ff67c14e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.668+0000 7ff682127700 1 --2- 172.21.15.28:0/3330063590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff67c0adec0 0x7ff67c14e090 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff67c0cc4e0 tx=0x7ff66c004750 comp rx=0 tx=0).stop 2023-11-28T11:09:08.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.669+0000 7ff682127700 1 -- 172.21.15.28:0/3330063590 shutdown_connections 2023-11-28T11:09:08.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.669+0000 7ff682127700 1 --2- 172.21.15.28:0/3330063590 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff6680682f0 0x7ff66806a7a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.669+0000 7ff682127700 1 --2- 172.21.15.28:0/3330063590 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff67c0adec0 0x7ff67c14e090 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.669+0000 7ff682127700 1 --2- 172.21.15.28:0/3330063590 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff67c160be0 0x7ff67c06d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.669+0000 7ff682127700 1 --2- 172.21.15.28:0/3330063590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff67c0c9df0 0x7ff67c162c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.669+0000 7ff682127700 1 -- 172.21.15.28:0/3330063590 >> 172.21.15.28:0/3330063590 conn(0x7ff67c0cf670 msgr2=0x7ff67c0ad7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.669+0000 7ff682127700 1 -- 172.21.15.28:0/3330063590 shutdown_connections 2023-11-28T11:09:08.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.669+0000 7ff682127700 1 -- 172.21.15.28:0/3330063590 wait complete. 2023-11-28T11:09:08.678 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.5 2023-11-28T11:09:08.678 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:08.683 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.7 2023-11-28T11:09:08.684 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:08.684 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2023-11-28T11:09:08.684 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:08.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.686+0000 7fd7ba2d3700 1 -- 172.21.15.28:0/3484553650 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fd7b4068570 con 0x7fd7b40cce20 2023-11-28T11:09:08.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.686+0000 7f2a6ae2d700 1 -- 172.21.15.28:0/2934433135 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2a64161eb0 con 0x7f2a641646c0 2023-11-28T11:09:08.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.687+0000 7f2a617fa700 1 -- 172.21.15.28:0/2934433135 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2a5c019030 con 0x7f2a641646c0 2023-11-28T11:09:08.688 INFO:teuthology.orchestra.run.smithi028.stdout:55834574857 2023-11-28T11:09:08.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.687+0000 7fd7b0ff9700 1 -- 172.21.15.28:0/3484553650 <== mon.2 v2:172.21.15.59:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd7a405d820 con 0x7fd7b40cce20 2023-11-28T11:09:08.688 INFO:teuthology.orchestra.run.smithi028.stdout:55834574856 2023-11-28T11:09:08.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7f2a6ae2d700 1 -- 172.21.15.28:0/2934433135 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2a50068320 msgr2=0x7f2a5006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7f2a6ae2d700 1 --2- 172.21.15.28:0/2934433135 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2a50068320 0x7f2a5006a7d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f2a54009c10 tx=0x7f2a54010000 comp rx=0 tx=0).stop 2023-11-28T11:09:08.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7f2a6ae2d700 1 -- 172.21.15.28:0/2934433135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a641646c0 msgr2=0x7f2a6414c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7f2a6ae2d700 1 --2- 172.21.15.28:0/2934433135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a641646c0 0x7f2a6414c670 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c00b970 tx=0x7f2a5c00bc80 comp rx=0 tx=0).stop 2023-11-28T11:09:08.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7f2a6ae2d700 1 -- 172.21.15.28:0/2934433135 shutdown_connections 2023-11-28T11:09:08.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7f2a6ae2d700 1 --2- 172.21.15.28:0/2934433135 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2a50068320 0x7f2a5006a7d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7f2a6ae2d700 1 --2- 172.21.15.28:0/2934433135 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a641646c0 0x7f2a6414c670 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7f2a6ae2d700 1 --2- 172.21.15.28:0/2934433135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a64070ea0 0x7f2a64162ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7f2a6ae2d700 1 --2- 172.21.15.28:0/2934433135 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a64160c70 0x7f2a640c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7f2a6ae2d700 1 -- 172.21.15.28:0/2934433135 >> 172.21.15.28:0/2934433135 conn(0x7f2a640ce9e0 msgr2=0x7f2a6406a930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.691 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7f2a6ae2d700 1 -- 172.21.15.28:0/2934433135 shutdown_connections 2023-11-28T11:09:08.691 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7fd7ba2d3700 1 -- 172.21.15.28:0/3484553650 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7a0068340 msgr2=0x7fd7a006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.691 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7fd7ba2d3700 1 --2- 172.21.15.28:0/3484553650 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7a0068340 0x7fd7a006a7f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fd7b40c4070 tx=0x7fd7ac010000 comp rx=0 tx=0).stop 2023-11-28T11:09:08.691 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7fd7ba2d3700 1 -- 172.21.15.28:0/3484553650 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b40cce20 msgr2=0x7fd7b41f5bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.691 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7fd7ba2d3700 1 --2- 172.21.15.28:0/3484553650 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b40cce20 0x7fd7b41f5bf0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd7a400b4a0 tx=0x7fd7a400b860 comp rx=0 tx=0).stop 2023-11-28T11:09:08.691 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7f2a6ae2d700 1 -- 172.21.15.28:0/2934433135 wait complete. 2023-11-28T11:09:08.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7fd7ba2d3700 1 -- 172.21.15.28:0/3484553650 shutdown_connections 2023-11-28T11:09:08.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7fd7ba2d3700 1 --2- 172.21.15.28:0/3484553650 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7a0068340 0x7fd7a006a7f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7fd7ba2d3700 1 --2- 172.21.15.28:0/3484553650 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7b406a1e0 0x7fd7b414ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7fd7ba2d3700 1 --2- 172.21.15.28:0/3484553650 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7b40cce20 0x7fd7b41f5bf0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7fd7ba2d3700 1 --2- 172.21.15.28:0/3484553650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7b40c9a40 0x7fd7b40c23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7fd7ba2d3700 1 -- 172.21.15.28:0/3484553650 >> 172.21.15.28:0/3484553650 conn(0x7fd7b40cf670 msgr2=0x7fd7b40acbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7fd7ba2d3700 1 -- 172.21.15.28:0/3484553650 shutdown_connections 2023-11-28T11:09:08.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.689+0000 7fd7ba2d3700 1 -- 172.21.15.28:0/3484553650 wait complete. 2023-11-28T11:09:08.708 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-11-28T11:09:08.709 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:08.709 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.0 2023-11-28T11:09:08.709 DEBUG:teuthology.parallel:result is None 2023-11-28T11:09:08.709 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T11:09:08.709 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:09:08.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.872+0000 7f9429224700 1 Processor -- start 2023-11-28T11:09:08.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.872+0000 7f9429224700 1 -- start start 2023-11-28T11:09:08.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.873+0000 7f9429224700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f942406bc30 0x7f942415b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.873+0000 7f9429224700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f942406dda0 0x7f942415e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.873+0000 7f9429224700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f942406c3a0 con 0x7f942406d9d0 2023-11-28T11:09:08.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.873+0000 7f9429224700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942415ee90 con 0x7f942406dda0 2023-11-28T11:09:08.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.873+0000 7f9429224700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f942406c0c0 con 0x7f942406bc30 2023-11-28T11:09:08.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.873+0000 7f942359e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f942406dda0 0x7f942415e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.873+0000 7f942359e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f942406dda0 0x7f942415e950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47324/0 (socket says 172.21.15.28:47324) 2023-11-28T11:09:08.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.873+0000 7f942359e700 1 -- 172.21.15.28:0/2876845924 learned_addr learned my addr 172.21.15.28:0/2876845924 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:08.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.873+0000 7f942259c700 1 --2- 172.21.15.28:0/2876845924 >> v2:172.21.15.59:3300/0 conn(0x7f942406bc30 0x7f942415b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.873+0000 7f942359e700 1 -- 172.21.15.28:0/2876845924 >> v2:172.21.15.59:3300/0 conn(0x7f942406bc30 msgr2=0x7f942415b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.873+0000 7f942359e700 1 --2- 172.21.15.28:0/2876845924 >> v2:172.21.15.59:3300/0 conn(0x7f942406bc30 0x7f942415b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.873+0000 7f942359e700 1 -- 172.21.15.28:0/2876845924 >> v1:172.21.15.59:6789/0 conn(0x7f942406d9d0 legacy=0x7f9424150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.873+0000 7f942359e700 1 -- 172.21.15.28:0/2876845924 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942415ab60 con 0x7f942406dda0 2023-11-28T11:09:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.873+0000 7f942359e700 1 --2- 172.21.15.28:0/2876845924 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942406dda0 0x7f942415e950 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f94240b5420 tx=0x7f941c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c29c18d921d991d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.874+0000 7f9421d9b700 1 -- 172.21.15.28:0/2876845924 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c00e070 con 0x7f942406dda0 2023-11-28T11:09:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.874+0000 7f9421d9b700 1 -- 172.21.15.28:0/2876845924 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c003a20 con 0x7f942406dda0 2023-11-28T11:09:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.874+0000 7f9421d9b700 1 -- 172.21.15.28:0/2876845924 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c003dd0 con 0x7f942406dda0 2023-11-28T11:09:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.874+0000 7f9429224700 1 -- 172.21.15.28:0/2876845924 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942406dda0 msgr2=0x7f942415e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.874+0000 7f9429224700 1 --2- 172.21.15.28:0/2876845924 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942406dda0 0x7f942415e950 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f94240b5420 tx=0x7f941c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.874+0000 7f9429224700 1 -- 172.21.15.28:0/2876845924 shutdown_connections 2023-11-28T11:09:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.874+0000 7f9429224700 1 --2- 172.21.15.28:0/2876845924 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942406dda0 0x7f942415e950 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.874+0000 7f9429224700 1 --2- 172.21.15.28:0/2876845924 >> v2:172.21.15.59:3300/0 conn(0x7f942406bc30 0x7f942415b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.874+0000 7f9429224700 1 -- 172.21.15.28:0/2876845924 >> 172.21.15.28:0/2876845924 conn(0x7f94240b0dd0 msgr2=0x7f94240b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.874+0000 7f9429224700 1 -- 172.21.15.28:0/2876845924 shutdown_connections 2023-11-28T11:09:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.874+0000 7f9429224700 1 -- 172.21.15.28:0/2876845924 wait complete. 2023-11-28T11:09:08.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.876+0000 7f9429224700 1 Processor -- start 2023-11-28T11:09:08.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.876+0000 7f9429224700 1 -- start start 2023-11-28T11:09:08.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.876+0000 7f9429224700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f942406bc30 0x7f9424150890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f9429224700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f942406d9d0 0x7f9424151b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f9429224700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942406dda0 0x7f94240ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f9429224700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94240ccfe0 con 0x7f942406dda0 2023-11-28T11:09:08.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f9429224700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94241523a0 con 0x7f942406bc30 2023-11-28T11:09:08.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f9429224700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9424152680 con 0x7f942406d9d0 2023-11-28T11:09:08.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f942359e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942406dda0 0x7f94240ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f942359e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942406dda0 0x7f94240ca7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47330/0 (socket says 172.21.15.28:47330) 2023-11-28T11:09:08.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f942359e700 1 -- 172.21.15.28:0/2190971948 learned_addr learned my addr 172.21.15.28:0/2190971948 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:08.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f9422d9d700 1 --2- 172.21.15.28:0/2190971948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f942406bc30 0x7f9424150890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f942259c700 1 --2- 172.21.15.28:0/2190971948 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f942406d9d0 0x7f9424151b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f942359e700 1 -- 172.21.15.28:0/2190971948 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f942406d9d0 msgr2=0x7f9424151b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f942359e700 1 --2- 172.21.15.28:0/2190971948 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f942406d9d0 0x7f9424151b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f942359e700 1 -- 172.21.15.28:0/2190971948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f942406bc30 msgr2=0x7f9424150890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:08.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f942359e700 1 --2- 172.21.15.28:0/2190971948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f942406bc30 0x7f9424150890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:08.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f942359e700 1 -- 172.21.15.28:0/2190971948 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f941c009b30 con 0x7f942406dda0 2023-11-28T11:09:08.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f942359e700 1 --2- 172.21.15.28:0/2190971948 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942406dda0 0x7f94240ca7f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f941c015040 tx=0x7f941c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f940b7fe700 1 -- 172.21.15.28:0/2190971948 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c0094d0 con 0x7f942406dda0 2023-11-28T11:09:08.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f940b7fe700 1 -- 172.21.15.28:0/2190971948 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f941c004240 con 0x7f942406dda0 2023-11-28T11:09:08.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f9429224700 1 -- 172.21.15.28:0/2190971948 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f942415b120 con 0x7f942406dda0 2023-11-28T11:09:08.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.877+0000 7f940b7fe700 1 -- 172.21.15.28:0/2190971948 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f941c004630 con 0x7f942406dda0 2023-11-28T11:09:08.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.878+0000 7f9429224700 1 -- 172.21.15.28:0/2190971948 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f942415ae40 con 0x7f942406dda0 2023-11-28T11:09:08.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.879+0000 7f9429224700 1 -- 172.21.15.28:0/2190971948 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f942406fac0 con 0x7f942406dda0 2023-11-28T11:09:08.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.879+0000 7f940b7fe700 1 -- 172.21.15.28:0/2190971948 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f941c004790 con 0x7f942406dda0 2023-11-28T11:09:08.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.879+0000 7f940b7fe700 1 --2- 172.21.15.28:0/2190971948 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94100682f0 0x7f941006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:08.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.879+0000 7f940b7fe700 1 -- 172.21.15.28:0/2190971948 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f941c087160 con 0x7f942406dda0 2023-11-28T11:09:08.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.879+0000 7f9422d9d700 1 --2- 172.21.15.28:0/2190971948 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94100682f0 0x7f941006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:08.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.880+0000 7f9422d9d700 1 --2- 172.21.15.28:0/2190971948 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94100682f0 0x7f941006a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9418007fd0 tx=0x7f9418007ec0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:08.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:08.884+0000 7f940b7fe700 1 -- 172.21.15.28:0/2190971948 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f941c0536d0 con 0x7f942406dda0 2023-11-28T11:09:09.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.074+0000 7f9429224700 1 -- 172.21.15.28:0/2190971948 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f94240675a0 con 0x7f94100682f0 2023-11-28T11:09:09.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.084+0000 7f940b7fe700 1 -- 172.21.15.28:0/2190971948 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367498 (secure 0 0 0) 0x7f94240675a0 con 0x7f94100682f0 2023-11-28T11:09:09.089 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:09.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.087+0000 7f9429224700 1 -- 172.21.15.28:0/2190971948 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94100682f0 msgr2=0x7f941006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.087+0000 7f9429224700 1 --2- 172.21.15.28:0/2190971948 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94100682f0 0x7f941006a7a0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f9418007fd0 tx=0x7f9418007ec0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.087+0000 7f9429224700 1 -- 172.21.15.28:0/2190971948 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942406dda0 msgr2=0x7f94240ca7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.087+0000 7f9429224700 1 --2- 172.21.15.28:0/2190971948 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942406dda0 0x7f94240ca7f0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f941c015040 tx=0x7f941c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.088+0000 7f9429224700 1 -- 172.21.15.28:0/2190971948 shutdown_connections 2023-11-28T11:09:09.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.088+0000 7f9429224700 1 --2- 172.21.15.28:0/2190971948 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94100682f0 0x7f941006a7a0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.088+0000 7f9429224700 1 --2- 172.21.15.28:0/2190971948 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f942406dda0 0x7f94240ca7f0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.088+0000 7f9429224700 1 --2- 172.21.15.28:0/2190971948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f942406bc30 0x7f9424150890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.088+0000 7f9429224700 1 --2- 172.21.15.28:0/2190971948 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f942406d9d0 0x7f9424151b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.088+0000 7f9429224700 1 -- 172.21.15.28:0/2190971948 >> 172.21.15.28:0/2190971948 conn(0x7f94240b0dd0 msgr2=0x7f94240afdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:09.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.088+0000 7f9429224700 1 -- 172.21.15.28:0/2190971948 shutdown_connections 2023-11-28T11:09:09.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.088+0000 7f9429224700 1 -- 172.21.15.28:0/2190971948 wait complete. 2023-11-28T11:09:09.100 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:09:09.101 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":33,"stamp":"2023-11-28T11:09:07.166936+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594896,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67236,"kb_used_data":3300,"kb_used_omap":0,"kb_used_meta":63872,"kb_avail":754907484,"statfs":{"total":773094113280,"available":773025263616,"internally_reserved":0,"allocated":3379200,"data_stored":2064039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65404928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2198,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002914"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108863+0000","last_change":"2023-11-28T11:08:45.948753+0000","last_active":"2023-11-28T11:08:50.108863+0000","last_peered":"2023-11-28T11:08:50.108863+0000","last_clean":"2023-11-28T11:08:50.108863+0000","last_became_active":"2023-11-28T11:08:45.948435+0000","last_became_peered":"2023-11-28T11:08:45.948435+0000","last_unstale":"2023-11-28T11:08:50.108863+0000","last_undegraded":"2023-11-28T11:08:50.108863+0000","last_fullsized":"2023-11-28T11:08:50.108863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:28.944484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105146+0000","last_change":"2023-11-28T11:08:45.946452+0000","last_active":"2023-11-28T11:08:50.105146+0000","last_peered":"2023-11-28T11:08:50.105146+0000","last_clean":"2023-11-28T11:08:50.105146+0000","last_became_active":"2023-11-28T11:08:45.946145+0000","last_became_peered":"2023-11-28T11:08:45.946145+0000","last_unstale":"2023-11-28T11:08:50.105146+0000","last_undegraded":"2023-11-28T11:08:50.105146+0000","last_fullsized":"2023-11-28T11:08:50.105146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:30.639887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113411+0000","last_change":"2023-11-28T11:08:45.941140+0000","last_active":"2023-11-28T11:08:50.113411+0000","last_peered":"2023-11-28T11:08:50.113411+0000","last_clean":"2023-11-28T11:08:50.113411+0000","last_became_active":"2023-11-28T11:08:45.939618+0000","last_became_peered":"2023-11-28T11:08:45.939618+0000","last_unstale":"2023-11-28T11:08:50.113411+0000","last_undegraded":"2023-11-28T11:08:50.113411+0000","last_fullsized":"2023-11-28T11:08:50.113411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:44.444264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109455+0000","last_change":"2023-11-28T11:08:45.947665+0000","last_active":"2023-11-28T11:08:50.109455+0000","last_peered":"2023-11-28T11:08:50.109455+0000","last_clean":"2023-11-28T11:08:50.109455+0000","last_became_active":"2023-11-28T11:08:45.947338+0000","last_became_peered":"2023-11-28T11:08:45.947338+0000","last_unstale":"2023-11-28T11:08:50.109455+0000","last_undegraded":"2023-11-28T11:08:50.109455+0000","last_fullsized":"2023-11-28T11:08:50.109455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:23:53.819338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103496+0000","last_change":"2023-11-28T11:08:45.950223+0000","last_active":"2023-11-28T11:08:50.103496+0000","last_peered":"2023-11-28T11:08:50.103496+0000","last_clean":"2023-11-28T11:08:50.103496+0000","last_became_active":"2023-11-28T11:08:45.949811+0000","last_became_peered":"2023-11-28T11:08:45.949811+0000","last_unstale":"2023-11-28T11:08:50.103496+0000","last_undegraded":"2023-11-28T11:08:50.103496+0000","last_fullsized":"2023-11-28T11:08:50.103496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112890+0000","last_change":"2023-11-28T11:08:45.942787+0000","last_active":"2023-11-28T11:08:50.112890+0000","last_peered":"2023-11-28T11:08:50.112890+0000","last_clean":"2023-11-28T11:08:50.112890+0000","last_became_active":"2023-11-28T11:08:45.941820+0000","last_became_peered":"2023-11-28T11:08:45.941820+0000","last_unstale":"2023-11-28T11:08:50.112890+0000","last_undegraded":"2023-11-28T11:08:50.112890+0000","last_fullsized":"2023-11-28T11:08:50.112890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:22.504112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115835+0000","last_change":"2023-11-28T11:08:45.953364+0000","last_active":"2023-11-28T11:08:50.115835+0000","last_peered":"2023-11-28T11:08:50.115835+0000","last_clean":"2023-11-28T11:08:50.115835+0000","last_became_active":"2023-11-28T11:08:45.953033+0000","last_became_peered":"2023-11-28T11:08:45.953033+0000","last_unstale":"2023-11-28T11:08:50.115835+0000","last_undegraded":"2023-11-28T11:08:50.115835+0000","last_fullsized":"2023-11-28T11:08:50.115835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105175+0000","last_change":"2023-11-28T11:08:45.947045+0000","last_active":"2023-11-28T11:08:50.105175+0000","last_peered":"2023-11-28T11:08:50.105175+0000","last_clean":"2023-11-28T11:08:50.105175+0000","last_became_active":"2023-11-28T11:08:45.946756+0000","last_became_peered":"2023-11-28T11:08:45.946756+0000","last_unstale":"2023-11-28T11:08:50.105175+0000","last_undegraded":"2023-11-28T11:08:50.105175+0000","last_fullsized":"2023-11-28T11:08:50.105175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:56.811805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103692+0000","last_change":"2023-11-28T11:08:45.946321+0000","last_active":"2023-11-28T11:08:50.103692+0000","last_peered":"2023-11-28T11:08:50.103692+0000","last_clean":"2023-11-28T11:08:50.103692+0000","last_became_active":"2023-11-28T11:08:45.943687+0000","last_became_peered":"2023-11-28T11:08:45.943687+0000","last_unstale":"2023-11-28T11:08:50.103692+0000","last_undegraded":"2023-11-28T11:08:50.103692+0000","last_fullsized":"2023-11-28T11:08:50.103692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:23.627932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105317+0000","last_change":"2023-11-28T11:08:45.948529+0000","last_active":"2023-11-28T11:08:50.105317+0000","last_peered":"2023-11-28T11:08:50.105317+0000","last_clean":"2023-11-28T11:08:50.105317+0000","last_became_active":"2023-11-28T11:08:45.948249+0000","last_became_peered":"2023-11-28T11:08:45.948249+0000","last_unstale":"2023-11-28T11:08:50.105317+0000","last_undegraded":"2023-11-28T11:08:50.105317+0000","last_fullsized":"2023-11-28T11:08:50.105317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113311+0000","last_change":"2023-11-28T11:08:45.942374+0000","last_active":"2023-11-28T11:08:50.113311+0000","last_peered":"2023-11-28T11:08:50.113311+0000","last_clean":"2023-11-28T11:08:50.113311+0000","last_became_active":"2023-11-28T11:08:45.942083+0000","last_became_peered":"2023-11-28T11:08:45.942083+0000","last_unstale":"2023-11-28T11:08:50.113311+0000","last_undegraded":"2023-11-28T11:08:50.113311+0000","last_fullsized":"2023-11-28T11:08:50.113311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:23.627932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108474+0000","last_change":"2023-11-28T11:08:45.947339+0000","last_active":"2023-11-28T11:08:50.108474+0000","last_peered":"2023-11-28T11:08:50.108474+0000","last_clean":"2023-11-28T11:08:50.108474+0000","last_became_active":"2023-11-28T11:08:45.947066+0000","last_became_peered":"2023-11-28T11:08:45.947066+0000","last_unstale":"2023-11-28T11:08:50.108474+0000","last_undegraded":"2023-11-28T11:08:50.108474+0000","last_fullsized":"2023-11-28T11:08:50.108474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:48.010812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109210+0000","last_change":"2023-11-28T11:08:45.949094+0000","last_active":"2023-11-28T11:08:50.109210+0000","last_peered":"2023-11-28T11:08:50.109210+0000","last_clean":"2023-11-28T11:08:50.109210+0000","last_became_active":"2023-11-28T11:08:45.948523+0000","last_became_peered":"2023-11-28T11:08:45.948523+0000","last_unstale":"2023-11-28T11:08:50.109210+0000","last_undegraded":"2023-11-28T11:08:50.109210+0000","last_fullsized":"2023-11-28T11:08:50.109210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:34.193700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105493+0000","last_change":"2023-11-28T11:08:45.945708+0000","last_active":"2023-11-28T11:08:50.105493+0000","last_peered":"2023-11-28T11:08:50.105493+0000","last_clean":"2023-11-28T11:08:50.105493+0000","last_became_active":"2023-11-28T11:08:45.945408+0000","last_became_peered":"2023-11-28T11:08:45.945408+0000","last_unstale":"2023-11-28T11:08:50.105493+0000","last_undegraded":"2023-11-28T11:08:50.105493+0000","last_fullsized":"2023-11-28T11:08:50.105493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:34.416718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115608+0000","last_change":"2023-11-28T11:08:45.954131+0000","last_active":"2023-11-28T11:08:50.115608+0000","last_peered":"2023-11-28T11:08:50.115608+0000","last_clean":"2023-11-28T11:08:50.115608+0000","last_became_active":"2023-11-28T11:08:45.953824+0000","last_became_peered":"2023-11-28T11:08:45.953824+0000","last_unstale":"2023-11-28T11:08:50.115608+0000","last_undegraded":"2023-11-28T11:08:50.115608+0000","last_fullsized":"2023-11-28T11:08:50.115608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113028+0000","last_change":"2023-11-28T11:08:45.943568+0000","last_active":"2023-11-28T11:08:50.113028+0000","last_peered":"2023-11-28T11:08:50.113028+0000","last_clean":"2023-11-28T11:08:50.113028+0000","last_became_active":"2023-11-28T11:08:45.942637+0000","last_became_peered":"2023-11-28T11:08:45.942637+0000","last_unstale":"2023-11-28T11:08:50.113028+0000","last_undegraded":"2023-11-28T11:08:50.113028+0000","last_fullsized":"2023-11-28T11:08:50.113028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:17.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106326+0000","last_change":"2023-11-28T11:08:45.944177+0000","last_active":"2023-11-28T11:08:50.106326+0000","last_peered":"2023-11-28T11:08:50.106326+0000","last_clean":"2023-11-28T11:08:50.106326+0000","last_became_active":"2023-11-28T11:08:45.943369+0000","last_became_peered":"2023-11-28T11:08:45.943369+0000","last_unstale":"2023-11-28T11:08:50.106326+0000","last_undegraded":"2023-11-28T11:08:50.106326+0000","last_fullsized":"2023-11-28T11:08:50.106326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:56.811805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113581+0000","last_change":"2023-11-28T11:08:45.943949+0000","last_active":"2023-11-28T11:08:50.113581+0000","last_peered":"2023-11-28T11:08:50.113581+0000","last_clean":"2023-11-28T11:08:50.113581+0000","last_became_active":"2023-11-28T11:08:45.942746+0000","last_became_peered":"2023-11-28T11:08:45.942746+0000","last_unstale":"2023-11-28T11:08:50.113581+0000","last_undegraded":"2023-11-28T11:08:50.113581+0000","last_fullsized":"2023-11-28T11:08:50.113581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:49.186302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110612+0000","last_change":"2023-11-28T11:08:45.954401+0000","last_active":"2023-11-28T11:08:50.110612+0000","last_peered":"2023-11-28T11:08:50.110612+0000","last_clean":"2023-11-28T11:08:50.110612+0000","last_became_active":"2023-11-28T11:08:45.954130+0000","last_became_peered":"2023-11-28T11:08:45.954130+0000","last_unstale":"2023-11-28T11:08:50.110612+0000","last_undegraded":"2023-11-28T11:08:50.110612+0000","last_fullsized":"2023-11-28T11:08:50.110612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:55.295125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113075+0000","last_change":"2023-11-28T11:08:45.956559+0000","last_active":"2023-11-28T11:08:50.113075+0000","last_peered":"2023-11-28T11:08:50.113075+0000","last_clean":"2023-11-28T11:08:50.113075+0000","last_became_active":"2023-11-28T11:08:45.956269+0000","last_became_peered":"2023-11-28T11:08:45.956269+0000","last_unstale":"2023-11-28T11:08:50.113075+0000","last_undegraded":"2023-11-28T11:08:50.113075+0000","last_fullsized":"2023-11-28T11:08:50.113075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:17.189127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115796+0000","last_change":"2023-11-28T11:08:45.951043+0000","last_active":"2023-11-28T11:08:50.115796+0000","last_peered":"2023-11-28T11:08:50.115796+0000","last_clean":"2023-11-28T11:08:50.115796+0000","last_became_active":"2023-11-28T11:08:45.950740+0000","last_became_peered":"2023-11-28T11:08:45.950740+0000","last_unstale":"2023-11-28T11:08:50.115796+0000","last_undegraded":"2023-11-28T11:08:50.115796+0000","last_fullsized":"2023-11-28T11:08:50.115796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103425+0000","last_change":"2023-11-28T11:08:45.940737+0000","last_active":"2023-11-28T11:08:50.103425+0000","last_peered":"2023-11-28T11:08:50.103425+0000","last_clean":"2023-11-28T11:08:50.103425+0000","last_became_active":"2023-11-28T11:08:45.940490+0000","last_became_peered":"2023-11-28T11:08:45.940490+0000","last_unstale":"2023-11-28T11:08:50.103425+0000","last_undegraded":"2023-11-28T11:08:50.103425+0000","last_fullsized":"2023-11-28T11:08:50.103425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.107940+0000","last_change":"2023-11-28T11:08:45.945519+0000","last_active":"2023-11-28T11:08:50.107940+0000","last_peered":"2023-11-28T11:08:50.107940+0000","last_clean":"2023-11-28T11:08:50.107940+0000","last_became_active":"2023-11-28T11:08:45.944911+0000","last_became_peered":"2023-11-28T11:08:45.944911+0000","last_unstale":"2023-11-28T11:08:50.107940+0000","last_undegraded":"2023-11-28T11:08:50.107940+0000","last_fullsized":"2023-11-28T11:08:50.107940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:05.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115473+0000","last_change":"2023-11-28T11:08:45.954870+0000","last_active":"2023-11-28T11:08:50.115473+0000","last_peered":"2023-11-28T11:08:50.115473+0000","last_clean":"2023-11-28T11:08:50.115473+0000","last_became_active":"2023-11-28T11:08:45.954442+0000","last_became_peered":"2023-11-28T11:08:45.954442+0000","last_unstale":"2023-11-28T11:08:50.115473+0000","last_undegraded":"2023-11-28T11:08:50.115473+0000","last_fullsized":"2023-11-28T11:08:50.115473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120450+0000","last_change":"2023-11-28T11:08:45.954073+0000","last_active":"2023-11-28T11:08:50.120450+0000","last_peered":"2023-11-28T11:08:50.120450+0000","last_clean":"2023-11-28T11:08:50.120450+0000","last_became_active":"2023-11-28T11:08:45.953886+0000","last_became_peered":"2023-11-28T11:08:45.953886+0000","last_unstale":"2023-11-28T11:08:50.120450+0000","last_undegraded":"2023-11-28T11:08:50.120450+0000","last_fullsized":"2023-11-28T11:08:50.120450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:56.811805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.121555+0000","last_change":"2023-11-28T11:08:45.943121+0000","last_active":"2023-11-28T11:08:50.121555+0000","last_peered":"2023-11-28T11:08:50.121555+0000","last_clean":"2023-11-28T11:08:50.121555+0000","last_became_active":"2023-11-28T11:08:45.942796+0000","last_became_peered":"2023-11-28T11:08:45.942796+0000","last_unstale":"2023-11-28T11:08:50.121555+0000","last_undegraded":"2023-11-28T11:08:50.121555+0000","last_fullsized":"2023-11-28T11:08:50.121555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:55.085644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106066+0000","last_change":"2023-11-28T11:08:45.957422+0000","last_active":"2023-11-28T11:08:50.106066+0000","last_peered":"2023-11-28T11:08:50.106066+0000","last_clean":"2023-11-28T11:08:50.106066+0000","last_became_active":"2023-11-28T11:08:45.956775+0000","last_became_peered":"2023-11-28T11:08:45.956775+0000","last_unstale":"2023-11-28T11:08:50.106066+0000","last_undegraded":"2023-11-28T11:08:50.106066+0000","last_fullsized":"2023-11-28T11:08:50.106066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.536690+0000","last_change":"2023-11-28T11:08:50.132954+0000","last_active":"2023-11-28T11:08:50.536690+0000","last_peered":"2023-11-28T11:08:50.536690+0000","last_clean":"2023-11-28T11:08:50.536690+0000","last_became_active":"2023-11-28T11:08:50.132496+0000","last_became_peered":"2023-11-28T11:08:50.132496+0000","last_unstale":"2023-11-28T11:08:50.536690+0000","last_undegraded":"2023-11-28T11:08:50.536690+0000","last_fullsized":"2023-11-28T11:08:50.536690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:19:52.225499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112687+0000","last_change":"2023-11-28T11:08:45.515522+0000","last_active":"2023-11-28T11:08:50.112687+0000","last_peered":"2023-11-28T11:08:50.112687+0000","last_clean":"2023-11-28T11:08:50.112687+0000","last_became_active":"2023-11-28T11:08:45.515238+0000","last_became_peered":"2023-11-28T11:08:45.515238+0000","last_unstale":"2023-11-28T11:08:50.112687+0000","last_undegraded":"2023-11-28T11:08:50.112687+0000","last_fullsized":"2023-11-28T11:08:50.112687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109971+0000","last_change":"2023-11-28T11:08:45.945792+0000","last_active":"2023-11-28T11:08:50.109971+0000","last_peered":"2023-11-28T11:08:50.109971+0000","last_clean":"2023-11-28T11:08:50.109971+0000","last_became_active":"2023-11-28T11:08:45.945536+0000","last_became_peered":"2023-11-28T11:08:45.945536+0000","last_unstale":"2023-11-28T11:08:50.109971+0000","last_undegraded":"2023-11-28T11:08:50.109971+0000","last_fullsized":"2023-11-28T11:08:50.109971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:49.186302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571306+0000","last_change":"2023-11-28T11:08:50.124166+0000","last_active":"2023-11-28T11:08:51.571306+0000","last_peered":"2023-11-28T11:08:51.571306+0000","last_clean":"2023-11-28T11:08:51.571306+0000","last_became_active":"2023-11-28T11:08:50.123744+0000","last_became_peered":"2023-11-28T11:08:50.123744+0000","last_unstale":"2023-11-28T11:08:51.571306+0000","last_undegraded":"2023-11-28T11:08:51.571306+0000","last_fullsized":"2023-11-28T11:08:51.571306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:58.631960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119490+0000","last_change":"2023-11-28T11:08:45.516923+0000","last_active":"2023-11-28T11:08:50.119490+0000","last_peered":"2023-11-28T11:08:50.119490+0000","last_clean":"2023-11-28T11:08:50.119490+0000","last_became_active":"2023-11-28T11:08:45.516618+0000","last_became_peered":"2023-11-28T11:08:45.516618+0000","last_unstale":"2023-11-28T11:08:50.119490+0000","last_undegraded":"2023-11-28T11:08:50.119490+0000","last_fullsized":"2023-11-28T11:08:50.119490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103765+0000","last_change":"2023-11-28T11:08:45.941787+0000","last_active":"2023-11-28T11:08:50.103765+0000","last_peered":"2023-11-28T11:08:50.103765+0000","last_clean":"2023-11-28T11:08:50.103765+0000","last_became_active":"2023-11-28T11:08:45.941379+0000","last_became_peered":"2023-11-28T11:08:45.941379+0000","last_unstale":"2023-11-28T11:08:50.103765+0000","last_undegraded":"2023-11-28T11:08:50.103765+0000","last_fullsized":"2023-11-28T11:08:50.103765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.437643+0000","last_change":"2023-11-28T11:08:50.136708+0000","last_active":"2023-11-28T11:08:51.437643+0000","last_peered":"2023-11-28T11:08:51.437643+0000","last_clean":"2023-11-28T11:08:51.437643+0000","last_became_active":"2023-11-28T11:08:50.133846+0000","last_became_peered":"2023-11-28T11:08:50.133846+0000","last_unstale":"2023-11-28T11:08:51.437643+0000","last_undegraded":"2023-11-28T11:08:51.437643+0000","last_fullsized":"2023-11-28T11:08:51.437643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:11:43.645799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109280+0000","last_change":"2023-11-28T11:08:45.524714+0000","last_active":"2023-11-28T11:08:50.109280+0000","last_peered":"2023-11-28T11:08:50.109280+0000","last_clean":"2023-11-28T11:08:50.109280+0000","last_became_active":"2023-11-28T11:08:45.524202+0000","last_became_peered":"2023-11-28T11:08:45.524202+0000","last_unstale":"2023-11-28T11:08:50.109280+0000","last_undegraded":"2023-11-28T11:08:50.109280+0000","last_fullsized":"2023-11-28T11:08:50.109280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:49.800409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110311+0000","last_change":"2023-11-28T11:08:45.952452+0000","last_active":"2023-11-28T11:08:50.110311+0000","last_peered":"2023-11-28T11:08:50.110311+0000","last_clean":"2023-11-28T11:08:50.110311+0000","last_became_active":"2023-11-28T11:08:45.952159+0000","last_became_peered":"2023-11-28T11:08:45.952159+0000","last_unstale":"2023-11-28T11:08:50.110311+0000","last_undegraded":"2023-11-28T11:08:50.110311+0000","last_fullsized":"2023-11-28T11:08:50.110311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:58:31.173555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569992+0000","last_change":"2023-11-28T11:08:50.123500+0000","last_active":"2023-11-28T11:08:51.569992+0000","last_peered":"2023-11-28T11:08:51.569992+0000","last_clean":"2023-11-28T11:08:51.569992+0000","last_became_active":"2023-11-28T11:08:50.123102+0000","last_became_peered":"2023-11-28T11:08:50.123102+0000","last_unstale":"2023-11-28T11:08:51.569992+0000","last_undegraded":"2023-11-28T11:08:51.569992+0000","last_fullsized":"2023-11-28T11:08:51.569992+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:36.688715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106385+0000","last_change":"2023-11-28T11:08:45.523570+0000","last_active":"2023-11-28T11:08:50.106385+0000","last_peered":"2023-11-28T11:08:50.106385+0000","last_clean":"2023-11-28T11:08:50.106385+0000","last_became_active":"2023-11-28T11:08:45.523159+0000","last_became_peered":"2023-11-28T11:08:45.523159+0000","last_unstale":"2023-11-28T11:08:50.106385+0000","last_undegraded":"2023-11-28T11:08:50.106385+0000","last_fullsized":"2023-11-28T11:08:50.106385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109190+0000","last_change":"2023-11-28T11:08:45.943601+0000","last_active":"2023-11-28T11:08:50.109190+0000","last_peered":"2023-11-28T11:08:50.109190+0000","last_clean":"2023-11-28T11:08:50.109190+0000","last_became_active":"2023-11-28T11:08:45.943333+0000","last_became_peered":"2023-11-28T11:08:45.943333+0000","last_unstale":"2023-11-28T11:08:50.109190+0000","last_undegraded":"2023-11-28T11:08:50.109190+0000","last_fullsized":"2023-11-28T11:08:50.109190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569794+0000","last_change":"2023-11-28T11:08:50.126326+0000","last_active":"2023-11-28T11:08:51.569794+0000","last_peered":"2023-11-28T11:08:51.569794+0000","last_clean":"2023-11-28T11:08:51.569794+0000","last_became_active":"2023-11-28T11:08:50.125641+0000","last_became_peered":"2023-11-28T11:08:50.125641+0000","last_unstale":"2023-11-28T11:08:51.569794+0000","last_undegraded":"2023-11-28T11:08:51.569794+0000","last_fullsized":"2023-11-28T11:08:51.569794+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:59.602687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105179+0000","last_change":"2023-11-28T11:08:45.518910+0000","last_active":"2023-11-28T11:08:50.105179+0000","last_peered":"2023-11-28T11:08:50.105179+0000","last_clean":"2023-11-28T11:08:50.105179+0000","last_became_active":"2023-11-28T11:08:45.518652+0000","last_became_peered":"2023-11-28T11:08:45.518652+0000","last_unstale":"2023-11-28T11:08:50.105179+0000","last_undegraded":"2023-11-28T11:08:50.105179+0000","last_fullsized":"2023-11-28T11:08:50.105179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:49.800409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.107754+0000","last_change":"2023-11-28T11:08:45.951273+0000","last_active":"2023-11-28T11:08:50.107754+0000","last_peered":"2023-11-28T11:08:50.107754+0000","last_clean":"2023-11-28T11:08:50.107754+0000","last_became_active":"2023-11-28T11:08:45.950973+0000","last_became_peered":"2023-11-28T11:08:45.950973+0000","last_unstale":"2023-11-28T11:08:50.107754+0000","last_undegraded":"2023-11-28T11:08:50.107754+0000","last_fullsized":"2023-11-28T11:08:50.107754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:44.385589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571794+0000","last_change":"2023-11-28T11:08:50.129634+0000","last_active":"2023-11-28T11:08:51.571794+0000","last_peered":"2023-11-28T11:08:51.571794+0000","last_clean":"2023-11-28T11:08:51.571794+0000","last_became_active":"2023-11-28T11:08:50.128894+0000","last_became_peered":"2023-11-28T11:08:50.128894+0000","last_unstale":"2023-11-28T11:08:51.571794+0000","last_undegraded":"2023-11-28T11:08:51.571794+0000","last_fullsized":"2023-11-28T11:08:51.571794+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.162990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115765+0000","last_change":"2023-11-28T11:08:45.520619+0000","last_active":"2023-11-28T11:08:50.115765+0000","last_peered":"2023-11-28T11:08:50.115765+0000","last_clean":"2023-11-28T11:08:50.115765+0000","last_became_active":"2023-11-28T11:08:45.520292+0000","last_became_peered":"2023-11-28T11:08:45.520292+0000","last_unstale":"2023-11-28T11:08:50.115765+0000","last_undegraded":"2023-11-28T11:08:50.115765+0000","last_fullsized":"2023-11-28T11:08:50.115765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:02.026801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119666+0000","last_change":"2023-11-28T11:08:45.950965+0000","last_active":"2023-11-28T11:08:50.119666+0000","last_peered":"2023-11-28T11:08:50.119666+0000","last_clean":"2023-11-28T11:08:50.119666+0000","last_became_active":"2023-11-28T11:08:45.950660+0000","last_became_peered":"2023-11-28T11:08:45.950660+0000","last_unstale":"2023-11-28T11:08:50.119666+0000","last_undegraded":"2023-11-28T11:08:50.119666+0000","last_fullsized":"2023-11-28T11:08:50.119666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:34.416718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570510+0000","last_change":"2023-11-28T11:08:50.125734+0000","last_active":"2023-11-28T11:08:51.570510+0000","last_peered":"2023-11-28T11:08:51.570510+0000","last_clean":"2023-11-28T11:08:51.570510+0000","last_became_active":"2023-11-28T11:08:50.125332+0000","last_became_peered":"2023-11-28T11:08:50.125332+0000","last_unstale":"2023-11-28T11:08:51.570510+0000","last_undegraded":"2023-11-28T11:08:51.570510+0000","last_fullsized":"2023-11-28T11:08:51.570510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:54.778391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"25'9","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:55.258405+0000","last_change":"2023-11-28T11:08:45.519282+0000","last_active":"2023-11-28T11:08:55.258405+0000","last_peered":"2023-11-28T11:08:55.258405+0000","last_clean":"2023-11-28T11:08:55.258405+0000","last_became_active":"2023-11-28T11:08:45.518976+0000","last_became_peered":"2023-11-28T11:08:45.518976+0000","last_unstale":"2023-11-28T11:08:55.258405+0000","last_undegraded":"2023-11-28T11:08:55.258405+0000","last_fullsized":"2023-11-28T11:08:55.258405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:28.586127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3746,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113107+0000","last_change":"2023-11-28T11:08:45.951387+0000","last_active":"2023-11-28T11:08:50.113107+0000","last_peered":"2023-11-28T11:08:50.113107+0000","last_clean":"2023-11-28T11:08:50.113107+0000","last_became_active":"2023-11-28T11:08:45.951104+0000","last_became_peered":"2023-11-28T11:08:45.951104+0000","last_unstale":"2023-11-28T11:08:50.113107+0000","last_undegraded":"2023-11-28T11:08:50.113107+0000","last_fullsized":"2023-11-28T11:08:50.113107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:19:48.658152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537972+0000","last_change":"2023-11-28T11:08:50.124789+0000","last_active":"2023-11-28T11:08:50.537972+0000","last_peered":"2023-11-28T11:08:50.537972+0000","last_clean":"2023-11-28T11:08:50.537972+0000","last_became_active":"2023-11-28T11:08:50.123561+0000","last_became_peered":"2023-11-28T11:08:50.123561+0000","last_unstale":"2023-11-28T11:08:50.537972+0000","last_undegraded":"2023-11-28T11:08:50.537972+0000","last_fullsized":"2023-11-28T11:08:50.537972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:29.317492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115695+0000","last_change":"2023-11-28T11:08:45.518953+0000","last_active":"2023-11-28T11:08:50.115695+0000","last_peered":"2023-11-28T11:08:50.115695+0000","last_clean":"2023-11-28T11:08:50.115695+0000","last_became_active":"2023-11-28T11:08:45.518679+0000","last_became_peered":"2023-11-28T11:08:45.518679+0000","last_unstale":"2023-11-28T11:08:50.115695+0000","last_undegraded":"2023-11-28T11:08:50.115695+0000","last_fullsized":"2023-11-28T11:08:50.115695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:54.080163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108292+0000","last_change":"2023-11-28T11:08:45.945377+0000","last_active":"2023-11-28T11:08:50.108292+0000","last_peered":"2023-11-28T11:08:50.108292+0000","last_clean":"2023-11-28T11:08:50.108292+0000","last_became_active":"2023-11-28T11:08:45.945104+0000","last_became_peered":"2023-11-28T11:08:45.945104+0000","last_unstale":"2023-11-28T11:08:50.108292+0000","last_undegraded":"2023-11-28T11:08:50.108292+0000","last_fullsized":"2023-11-28T11:08:50.108292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.572169+0000","last_change":"2023-11-28T11:08:50.130966+0000","last_active":"2023-11-28T11:08:51.572169+0000","last_peered":"2023-11-28T11:08:51.572169+0000","last_clean":"2023-11-28T11:08:51.572169+0000","last_became_active":"2023-11-28T11:08:50.130573+0000","last_became_peered":"2023-11-28T11:08:50.130573+0000","last_unstale":"2023-11-28T11:08:51.572169+0000","last_undegraded":"2023-11-28T11:08:51.572169+0000","last_fullsized":"2023-11-28T11:08:51.572169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:59.602687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103540+0000","last_change":"2023-11-28T11:08:45.521275+0000","last_active":"2023-11-28T11:08:50.103540+0000","last_peered":"2023-11-28T11:08:50.103540+0000","last_clean":"2023-11-28T11:08:50.103540+0000","last_became_active":"2023-11-28T11:08:45.520908+0000","last_became_peered":"2023-11-28T11:08:45.520908+0000","last_unstale":"2023-11-28T11:08:50.103540+0000","last_undegraded":"2023-11-28T11:08:50.103540+0000","last_fullsized":"2023-11-28T11:08:50.103540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:33.188220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105967+0000","last_change":"2023-11-28T11:08:45.957604+0000","last_active":"2023-11-28T11:08:50.105967+0000","last_peered":"2023-11-28T11:08:50.105967+0000","last_clean":"2023-11-28T11:08:50.105967+0000","last_became_active":"2023-11-28T11:08:45.957135+0000","last_became_peered":"2023-11-28T11:08:45.957135+0000","last_unstale":"2023-11-28T11:08:50.105967+0000","last_undegraded":"2023-11-28T11:08:50.105967+0000","last_fullsized":"2023-11-28T11:08:50.105967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.572546+0000","last_change":"2023-11-28T11:08:50.129359+0000","last_active":"2023-11-28T11:08:51.572546+0000","last_peered":"2023-11-28T11:08:51.572546+0000","last_clean":"2023-11-28T11:08:51.572546+0000","last_became_active":"2023-11-28T11:08:50.128928+0000","last_became_peered":"2023-11-28T11:08:50.128928+0000","last_unstale":"2023-11-28T11:08:51.572546+0000","last_undegraded":"2023-11-28T11:08:51.572546+0000","last_fullsized":"2023-11-28T11:08:51.572546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:24.842089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115717+0000","last_change":"2023-11-28T11:08:45.517068+0000","last_active":"2023-11-28T11:08:50.115717+0000","last_peered":"2023-11-28T11:08:50.115717+0000","last_clean":"2023-11-28T11:08:50.115717+0000","last_became_active":"2023-11-28T11:08:45.516742+0000","last_became_peered":"2023-11-28T11:08:45.516742+0000","last_unstale":"2023-11-28T11:08:50.115717+0000","last_undegraded":"2023-11-28T11:08:50.115717+0000","last_fullsized":"2023-11-28T11:08:50.115717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:00.442293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.104172+0000","last_change":"2023-11-28T11:08:45.941624+0000","last_active":"2023-11-28T11:08:50.104172+0000","last_peered":"2023-11-28T11:08:50.104172+0000","last_clean":"2023-11-28T11:08:50.104172+0000","last_became_active":"2023-11-28T11:08:45.941042+0000","last_became_peered":"2023-11-28T11:08:45.941042+0000","last_unstale":"2023-11-28T11:08:50.104172+0000","last_undegraded":"2023-11-28T11:08:50.104172+0000","last_fullsized":"2023-11-28T11:08:50.104172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:39.158725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537699+0000","last_change":"2023-11-28T11:08:50.121180+0000","last_active":"2023-11-28T11:08:50.537699+0000","last_peered":"2023-11-28T11:08:50.537699+0000","last_clean":"2023-11-28T11:08:50.537699+0000","last_became_active":"2023-11-28T11:08:50.120757+0000","last_became_peered":"2023-11-28T11:08:50.120757+0000","last_unstale":"2023-11-28T11:08:50.537699+0000","last_undegraded":"2023-11-28T11:08:50.537699+0000","last_fullsized":"2023-11-28T11:08:50.537699+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:20.989038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108439+0000","last_change":"2023-11-28T11:08:45.516801+0000","last_active":"2023-11-28T11:08:50.108439+0000","last_peered":"2023-11-28T11:08:50.108439+0000","last_clean":"2023-11-28T11:08:50.108439+0000","last_became_active":"2023-11-28T11:08:45.516457+0000","last_became_peered":"2023-11-28T11:08:45.516457+0000","last_unstale":"2023-11-28T11:08:50.108439+0000","last_undegraded":"2023-11-28T11:08:50.108439+0000","last_fullsized":"2023-11-28T11:08:50.108439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.111359+0000","last_change":"2023-11-28T11:08:45.956905+0000","last_active":"2023-11-28T11:08:50.111359+0000","last_peered":"2023-11-28T11:08:50.111359+0000","last_clean":"2023-11-28T11:08:50.111359+0000","last_became_active":"2023-11-28T11:08:45.956698+0000","last_became_peered":"2023-11-28T11:08:45.956698+0000","last_unstale":"2023-11-28T11:08:50.111359+0000","last_undegraded":"2023-11-28T11:08:50.111359+0000","last_fullsized":"2023-11-28T11:08:50.111359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:23.627932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570657+0000","last_change":"2023-11-28T11:08:50.130733+0000","last_active":"2023-11-28T11:08:51.570657+0000","last_peered":"2023-11-28T11:08:51.570657+0000","last_clean":"2023-11-28T11:08:51.570657+0000","last_became_active":"2023-11-28T11:08:50.130213+0000","last_became_peered":"2023-11-28T11:08:50.130213+0000","last_unstale":"2023-11-28T11:08:51.570657+0000","last_undegraded":"2023-11-28T11:08:51.570657+0000","last_fullsized":"2023-11-28T11:08:51.570657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:36.656033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.116413+0000","last_change":"2023-11-28T11:08:45.516890+0000","last_active":"2023-11-28T11:08:50.116413+0000","last_peered":"2023-11-28T11:08:50.116413+0000","last_clean":"2023-11-28T11:08:50.116413+0000","last_became_active":"2023-11-28T11:08:45.516595+0000","last_became_peered":"2023-11-28T11:08:45.516595+0000","last_unstale":"2023-11-28T11:08:50.116413+0000","last_undegraded":"2023-11-28T11:08:50.116413+0000","last_fullsized":"2023-11-28T11:08:50.116413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:58:30.168075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105047+0000","last_change":"2023-11-28T11:08:45.956070+0000","last_active":"2023-11-28T11:08:50.105047+0000","last_peered":"2023-11-28T11:08:50.105047+0000","last_clean":"2023-11-28T11:08:50.105047+0000","last_became_active":"2023-11-28T11:08:45.955774+0000","last_became_peered":"2023-11-28T11:08:45.955774+0000","last_unstale":"2023-11-28T11:08:50.105047+0000","last_undegraded":"2023-11-28T11:08:50.105047+0000","last_fullsized":"2023-11-28T11:08:50.105047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571154+0000","last_change":"2023-11-28T11:08:50.132292+0000","last_active":"2023-11-28T11:08:51.571154+0000","last_peered":"2023-11-28T11:08:51.571154+0000","last_clean":"2023-11-28T11:08:51.571154+0000","last_became_active":"2023-11-28T11:08:50.131887+0000","last_became_peered":"2023-11-28T11:08:50.131887+0000","last_unstale":"2023-11-28T11:08:51.571154+0000","last_undegraded":"2023-11-28T11:08:51.571154+0000","last_fullsized":"2023-11-28T11:08:51.571154+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:08.804864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115394+0000","last_change":"2023-11-28T11:08:45.516041+0000","last_active":"2023-11-28T11:08:50.115394+0000","last_peered":"2023-11-28T11:08:50.115394+0000","last_clean":"2023-11-28T11:08:50.115394+0000","last_became_active":"2023-11-28T11:08:45.515742+0000","last_became_peered":"2023-11-28T11:08:45.515742+0000","last_unstale":"2023-11-28T11:08:50.115394+0000","last_undegraded":"2023-11-28T11:08:50.115394+0000","last_fullsized":"2023-11-28T11:08:50.115394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:14.983939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113365+0000","last_change":"2023-11-28T11:08:45.945858+0000","last_active":"2023-11-28T11:08:50.113365+0000","last_peered":"2023-11-28T11:08:50.113365+0000","last_clean":"2023-11-28T11:08:50.113365+0000","last_became_active":"2023-11-28T11:08:45.945552+0000","last_became_peered":"2023-11-28T11:08:45.945552+0000","last_unstale":"2023-11-28T11:08:50.113365+0000","last_undegraded":"2023-11-28T11:08:50.113365+0000","last_fullsized":"2023-11-28T11:08:50.113365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571022+0000","last_change":"2023-11-28T11:08:50.123713+0000","last_active":"2023-11-28T11:08:51.571022+0000","last_peered":"2023-11-28T11:08:51.571022+0000","last_clean":"2023-11-28T11:08:51.571022+0000","last_became_active":"2023-11-28T11:08:50.123264+0000","last_became_peered":"2023-11-28T11:08:50.123264+0000","last_unstale":"2023-11-28T11:08:51.571022+0000","last_undegraded":"2023-11-28T11:08:51.571022+0000","last_fullsized":"2023-11-28T11:08:51.571022+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:04.199211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115481+0000","last_change":"2023-11-28T11:08:45.515894+0000","last_active":"2023-11-28T11:08:50.115481+0000","last_peered":"2023-11-28T11:08:50.115481+0000","last_clean":"2023-11-28T11:08:50.115481+0000","last_became_active":"2023-11-28T11:08:45.515593+0000","last_became_peered":"2023-11-28T11:08:45.515593+0000","last_unstale":"2023-11-28T11:08:50.115481+0000","last_undegraded":"2023-11-28T11:08:50.115481+0000","last_fullsized":"2023-11-28T11:08:50.115481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:31.747218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109629+0000","last_change":"2023-11-28T11:08:45.949548+0000","last_active":"2023-11-28T11:08:50.109629+0000","last_peered":"2023-11-28T11:08:50.109629+0000","last_clean":"2023-11-28T11:08:50.109629+0000","last_became_active":"2023-11-28T11:08:45.949217+0000","last_became_peered":"2023-11-28T11:08:45.949217+0000","last_unstale":"2023-11-28T11:08:50.109629+0000","last_undegraded":"2023-11-28T11:08:50.109629+0000","last_fullsized":"2023-11-28T11:08:50.109629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:01.447773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571373+0000","last_change":"2023-11-28T11:08:50.127125+0000","last_active":"2023-11-28T11:08:51.571373+0000","last_peered":"2023-11-28T11:08:51.571373+0000","last_clean":"2023-11-28T11:08:51.571373+0000","last_became_active":"2023-11-28T11:08:50.124511+0000","last_became_peered":"2023-11-28T11:08:50.124511+0000","last_unstale":"2023-11-28T11:08:51.571373+0000","last_undegraded":"2023-11-28T11:08:51.571373+0000","last_fullsized":"2023-11-28T11:08:51.571373+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:47.985900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105609+0000","last_change":"2023-11-28T11:08:45.513992+0000","last_active":"2023-11-28T11:08:50.105609+0000","last_peered":"2023-11-28T11:08:50.105609+0000","last_clean":"2023-11-28T11:08:50.105609+0000","last_became_active":"2023-11-28T11:08:45.513697+0000","last_became_peered":"2023-11-28T11:08:45.513697+0000","last_unstale":"2023-11-28T11:08:50.105609+0000","last_undegraded":"2023-11-28T11:08:50.105609+0000","last_fullsized":"2023-11-28T11:08:50.105609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:28.586127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103999+0000","last_change":"2023-11-28T11:08:45.940813+0000","last_active":"2023-11-28T11:08:50.103999+0000","last_peered":"2023-11-28T11:08:50.103999+0000","last_clean":"2023-11-28T11:08:50.103999+0000","last_became_active":"2023-11-28T11:08:45.940552+0000","last_became_peered":"2023-11-28T11:08:45.940552+0000","last_unstale":"2023-11-28T11:08:50.103999+0000","last_undegraded":"2023-11-28T11:08:50.103999+0000","last_fullsized":"2023-11-28T11:08:50.103999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:17.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571495+0000","last_change":"2023-11-28T11:08:50.121770+0000","last_active":"2023-11-28T11:08:51.571495+0000","last_peered":"2023-11-28T11:08:51.571495+0000","last_clean":"2023-11-28T11:08:51.571495+0000","last_became_active":"2023-11-28T11:08:50.121277+0000","last_became_peered":"2023-11-28T11:08:50.121277+0000","last_unstale":"2023-11-28T11:08:51.571495+0000","last_undegraded":"2023-11-28T11:08:51.571495+0000","last_fullsized":"2023-11-28T11:08:51.571495+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:42.759036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109255+0000","last_change":"2023-11-28T11:08:45.519824+0000","last_active":"2023-11-28T11:08:50.109255+0000","last_peered":"2023-11-28T11:08:50.109255+0000","last_clean":"2023-11-28T11:08:50.109255+0000","last_became_active":"2023-11-28T11:08:45.519241+0000","last_became_peered":"2023-11-28T11:08:45.519241+0000","last_unstale":"2023-11-28T11:08:50.109255+0000","last_undegraded":"2023-11-28T11:08:50.109255+0000","last_fullsized":"2023-11-28T11:08:50.109255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:31.747218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112858+0000","last_change":"2023-11-28T11:08:45.946401+0000","last_active":"2023-11-28T11:08:50.112858+0000","last_peered":"2023-11-28T11:08:50.112858+0000","last_clean":"2023-11-28T11:08:50.112858+0000","last_became_active":"2023-11-28T11:08:45.946070+0000","last_became_peered":"2023-11-28T11:08:45.946070+0000","last_unstale":"2023-11-28T11:08:50.112858+0000","last_undegraded":"2023-11-28T11:08:50.112858+0000","last_fullsized":"2023-11-28T11:08:50.112858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:05.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570768+0000","last_change":"2023-11-28T11:08:50.131808+0000","last_active":"2023-11-28T11:08:51.570768+0000","last_peered":"2023-11-28T11:08:51.570768+0000","last_clean":"2023-11-28T11:08:51.570768+0000","last_became_active":"2023-11-28T11:08:50.131311+0000","last_became_peered":"2023-11-28T11:08:50.131311+0000","last_unstale":"2023-11-28T11:08:51.570768+0000","last_undegraded":"2023-11-28T11:08:51.570768+0000","last_fullsized":"2023-11-28T11:08:51.570768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:06.157479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109818+0000","last_change":"2023-11-28T11:08:45.517239+0000","last_active":"2023-11-28T11:08:50.109818+0000","last_peered":"2023-11-28T11:08:50.109818+0000","last_clean":"2023-11-28T11:08:50.109818+0000","last_became_active":"2023-11-28T11:08:45.516694+0000","last_became_peered":"2023-11-28T11:08:45.516694+0000","last_unstale":"2023-11-28T11:08:50.109818+0000","last_undegraded":"2023-11-28T11:08:50.109818+0000","last_fullsized":"2023-11-28T11:08:50.109818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113094+0000","last_change":"2023-11-28T11:08:45.947974+0000","last_active":"2023-11-28T11:08:50.113094+0000","last_peered":"2023-11-28T11:08:50.113094+0000","last_clean":"2023-11-28T11:08:50.113094+0000","last_became_active":"2023-11-28T11:08:45.947674+0000","last_became_peered":"2023-11-28T11:08:45.947674+0000","last_unstale":"2023-11-28T11:08:50.113094+0000","last_undegraded":"2023-11-28T11:08:50.113094+0000","last_fullsized":"2023-11-28T11:08:50.113094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:44.385589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.538066+0000","last_change":"2023-11-28T11:08:50.126205+0000","last_active":"2023-11-28T11:08:50.538066+0000","last_peered":"2023-11-28T11:08:50.538066+0000","last_clean":"2023-11-28T11:08:50.538066+0000","last_became_active":"2023-11-28T11:08:50.125446+0000","last_became_peered":"2023-11-28T11:08:50.125446+0000","last_unstale":"2023-11-28T11:08:50.538066+0000","last_undegraded":"2023-11-28T11:08:50.538066+0000","last_fullsized":"2023-11-28T11:08:50.538066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:59.602687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113146+0000","last_change":"2023-11-28T11:08:45.512742+0000","last_active":"2023-11-28T11:08:50.113146+0000","last_peered":"2023-11-28T11:08:50.113146+0000","last_clean":"2023-11-28T11:08:50.113146+0000","last_became_active":"2023-11-28T11:08:45.512446+0000","last_became_peered":"2023-11-28T11:08:45.512446+0000","last_unstale":"2023-11-28T11:08:50.113146+0000","last_undegraded":"2023-11-28T11:08:50.113146+0000","last_fullsized":"2023-11-28T11:08:50.113146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113682+0000","last_change":"2023-11-28T11:08:45.955541+0000","last_active":"2023-11-28T11:08:50.113682+0000","last_peered":"2023-11-28T11:08:50.113682+0000","last_clean":"2023-11-28T11:08:50.113682+0000","last_became_active":"2023-11-28T11:08:45.955223+0000","last_became_peered":"2023-11-28T11:08:45.955223+0000","last_unstale":"2023-11-28T11:08:50.113682+0000","last_undegraded":"2023-11-28T11:08:50.113682+0000","last_fullsized":"2023-11-28T11:08:50.113682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:24:04.835870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.536215+0000","last_change":"2023-11-28T11:08:50.134010+0000","last_active":"2023-11-28T11:08:50.536215+0000","last_peered":"2023-11-28T11:08:50.536215+0000","last_clean":"2023-11-28T11:08:50.536215+0000","last_became_active":"2023-11-28T11:08:50.133519+0000","last_became_peered":"2023-11-28T11:08:50.133519+0000","last_unstale":"2023-11-28T11:08:50.536215+0000","last_undegraded":"2023-11-28T11:08:50.536215+0000","last_fullsized":"2023-11-28T11:08:50.536215+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:47.676934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109102+0000","last_change":"2023-11-28T11:08:45.515039+0000","last_active":"2023-11-28T11:08:50.109102+0000","last_peered":"2023-11-28T11:08:50.109102+0000","last_clean":"2023-11-28T11:08:50.109102+0000","last_became_active":"2023-11-28T11:08:45.514380+0000","last_became_peered":"2023-11-28T11:08:45.514380+0000","last_unstale":"2023-11-28T11:08:50.109102+0000","last_undegraded":"2023-11-28T11:08:50.109102+0000","last_fullsized":"2023-11-28T11:08:50.109102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:28.586127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120542+0000","last_change":"2023-11-28T11:08:45.945761+0000","last_active":"2023-11-28T11:08:50.120542+0000","last_peered":"2023-11-28T11:08:50.120542+0000","last_clean":"2023-11-28T11:08:50.120542+0000","last_became_active":"2023-11-28T11:08:45.945501+0000","last_became_peered":"2023-11-28T11:08:45.945501+0000","last_unstale":"2023-11-28T11:08:50.120542+0000","last_undegraded":"2023-11-28T11:08:50.120542+0000","last_fullsized":"2023-11-28T11:08:50.120542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:30.639887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.535821+0000","last_change":"2023-11-28T11:08:50.129233+0000","last_active":"2023-11-28T11:08:50.535821+0000","last_peered":"2023-11-28T11:08:50.535821+0000","last_clean":"2023-11-28T11:08:50.535821+0000","last_became_active":"2023-11-28T11:08:50.128785+0000","last_became_peered":"2023-11-28T11:08:50.128785+0000","last_unstale":"2023-11-28T11:08:50.535821+0000","last_undegraded":"2023-11-28T11:08:50.535821+0000","last_fullsized":"2023-11-28T11:08:50.535821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:14.088471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.111193+0000","last_change":"2023-11-28T11:08:45.519399+0000","last_active":"2023-11-28T11:08:50.111193+0000","last_peered":"2023-11-28T11:08:50.111193+0000","last_clean":"2023-11-28T11:08:50.111193+0000","last_became_active":"2023-11-28T11:08:45.519141+0000","last_became_peered":"2023-11-28T11:08:45.519141+0000","last_unstale":"2023-11-28T11:08:50.111193+0000","last_undegraded":"2023-11-28T11:08:50.111193+0000","last_fullsized":"2023-11-28T11:08:50.111193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:24.093818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113185+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:08:50.113185+0000","last_peered":"2023-11-28T11:08:50.113185+0000","last_clean":"2023-11-28T11:08:50.113185+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:08:50.113185+0000","last_undegraded":"2023-11-28T11:08:50.113185+0000","last_fullsized":"2023-11-28T11:08:50.113185+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109007+0000","last_change":"2023-11-28T11:08:45.955853+0000","last_active":"2023-11-28T11:08:50.109007+0000","last_peered":"2023-11-28T11:08:50.109007+0000","last_clean":"2023-11-28T11:08:50.109007+0000","last_became_active":"2023-11-28T11:08:45.955563+0000","last_became_peered":"2023-11-28T11:08:45.955563+0000","last_unstale":"2023-11-28T11:08:50.109007+0000","last_undegraded":"2023-11-28T11:08:50.109007+0000","last_fullsized":"2023-11-28T11:08:50.109007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:00.598900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569792+0000","last_change":"2023-11-28T11:08:50.129450+0000","last_active":"2023-11-28T11:08:51.569792+0000","last_peered":"2023-11-28T11:08:51.569792+0000","last_clean":"2023-11-28T11:08:51.569792+0000","last_became_active":"2023-11-28T11:08:50.128981+0000","last_became_peered":"2023-11-28T11:08:50.128981+0000","last_unstale":"2023-11-28T11:08:51.569792+0000","last_undegraded":"2023-11-28T11:08:51.569792+0000","last_fullsized":"2023-11-28T11:08:51.569792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:14.466494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108203+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:08:50.108203+0000","last_peered":"2023-11-28T11:08:50.108203+0000","last_clean":"2023-11-28T11:08:50.108203+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:08:50.108203+0000","last_undegraded":"2023-11-28T11:08:50.108203+0000","last_fullsized":"2023-11-28T11:08:50.108203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.513023+0000","last_change":"2023-11-28T11:08:45.522154+0000","last_active":"2023-11-28T11:08:50.513023+0000","last_peered":"2023-11-28T11:08:50.513023+0000","last_clean":"2023-11-28T11:08:50.513023+0000","last_became_active":"2023-11-28T11:08:45.518652+0000","last_became_peered":"2023-11-28T11:08:45.518652+0000","last_unstale":"2023-11-28T11:08:50.513023+0000","last_undegraded":"2023-11-28T11:08:50.513023+0000","last_fullsized":"2023-11-28T11:08:50.513023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:49.800409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113254+0000","last_change":"2023-11-28T11:08:45.946786+0000","last_active":"2023-11-28T11:08:50.113254+0000","last_peered":"2023-11-28T11:08:50.113254+0000","last_clean":"2023-11-28T11:08:50.113254+0000","last_became_active":"2023-11-28T11:08:45.946446+0000","last_became_peered":"2023-11-28T11:08:45.946446+0000","last_unstale":"2023-11-28T11:08:50.113254+0000","last_undegraded":"2023-11-28T11:08:50.113254+0000","last_fullsized":"2023-11-28T11:08:50.113254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:39.158725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537975+0000","last_change":"2023-11-28T11:08:50.123220+0000","last_active":"2023-11-28T11:08:50.537975+0000","last_peered":"2023-11-28T11:08:50.537975+0000","last_clean":"2023-11-28T11:08:50.537975+0000","last_became_active":"2023-11-28T11:08:50.122776+0000","last_became_peered":"2023-11-28T11:08:50.122776+0000","last_unstale":"2023-11-28T11:08:50.537975+0000","last_undegraded":"2023-11-28T11:08:50.537975+0000","last_fullsized":"2023-11-28T11:08:50.537975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:35.176626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119767+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:08:50.119767+0000","last_peered":"2023-11-28T11:08:50.119767+0000","last_clean":"2023-11-28T11:08:50.119767+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:08:50.119767+0000","last_undegraded":"2023-11-28T11:08:50.119767+0000","last_fullsized":"2023-11-28T11:08:50.119767+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":167,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115593+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:08:50.115593+0000","last_peered":"2023-11-28T11:08:50.115593+0000","last_clean":"2023-11-28T11:08:50.115593+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:08:50.115593+0000","last_undegraded":"2023-11-28T11:08:50.115593+0000","last_fullsized":"2023-11-28T11:08:50.115593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103405+0000","last_change":"2023-11-28T11:08:45.521876+0000","last_active":"2023-11-28T11:08:50.103405+0000","last_peered":"2023-11-28T11:08:50.103405+0000","last_clean":"2023-11-28T11:08:50.103405+0000","last_became_active":"2023-11-28T11:08:45.521580+0000","last_became_peered":"2023-11-28T11:08:45.521580+0000","last_unstale":"2023-11-28T11:08:50.103405+0000","last_undegraded":"2023-11-28T11:08:50.103405+0000","last_fullsized":"2023-11-28T11:08:50.103405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.104071+0000","last_change":"2023-11-28T11:08:45.948040+0000","last_active":"2023-11-28T11:08:50.104071+0000","last_peered":"2023-11-28T11:08:50.104071+0000","last_clean":"2023-11-28T11:08:50.104071+0000","last_became_active":"2023-11-28T11:08:45.947704+0000","last_became_peered":"2023-11-28T11:08:45.947704+0000","last_unstale":"2023-11-28T11:08:50.104071+0000","last_undegraded":"2023-11-28T11:08:50.104071+0000","last_fullsized":"2023-11-28T11:08:50.104071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571808+0000","last_change":"2023-11-28T11:08:50.124074+0000","last_active":"2023-11-28T11:08:51.571808+0000","last_peered":"2023-11-28T11:08:51.571808+0000","last_clean":"2023-11-28T11:08:51.571808+0000","last_became_active":"2023-11-28T11:08:50.123630+0000","last_became_peered":"2023-11-28T11:08:50.123630+0000","last_unstale":"2023-11-28T11:08:51.571808+0000","last_undegraded":"2023-11-28T11:08:51.571808+0000","last_fullsized":"2023-11-28T11:08:51.571808+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:52.786613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115788+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:08:50.115788+0000","last_peered":"2023-11-28T11:08:50.115788+0000","last_clean":"2023-11-28T11:08:50.115788+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:08:50.115788+0000","last_undegraded":"2023-11-28T11:08:50.115788+0000","last_fullsized":"2023-11-28T11:08:50.115788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119464+0000","last_change":"2023-11-28T11:08:45.510984+0000","last_active":"2023-11-28T11:08:50.119464+0000","last_peered":"2023-11-28T11:08:50.119464+0000","last_clean":"2023-11-28T11:08:50.119464+0000","last_became_active":"2023-11-28T11:08:45.510734+0000","last_became_peered":"2023-11-28T11:08:45.510734+0000","last_unstale":"2023-11-28T11:08:50.119464+0000","last_undegraded":"2023-11-28T11:08:50.119464+0000","last_fullsized":"2023-11-28T11:08:50.119464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:33.188220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120521+0000","last_change":"2023-11-28T11:08:45.946753+0000","last_active":"2023-11-28T11:08:50.120521+0000","last_peered":"2023-11-28T11:08:50.120521+0000","last_clean":"2023-11-28T11:08:50.120521+0000","last_became_active":"2023-11-28T11:08:45.946460+0000","last_became_peered":"2023-11-28T11:08:45.946460+0000","last_unstale":"2023-11-28T11:08:50.120521+0000","last_undegraded":"2023-11-28T11:08:50.120521+0000","last_fullsized":"2023-11-28T11:08:50.120521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:20:10.730266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570567+0000","last_change":"2023-11-28T11:08:50.132450+0000","last_active":"2023-11-28T11:08:51.570567+0000","last_peered":"2023-11-28T11:08:51.570567+0000","last_clean":"2023-11-28T11:08:51.570567+0000","last_became_active":"2023-11-28T11:08:50.132043+0000","last_became_peered":"2023-11-28T11:08:50.132043+0000","last_unstale":"2023-11-28T11:08:51.570567+0000","last_undegraded":"2023-11-28T11:08:51.570567+0000","last_fullsized":"2023-11-28T11:08:51.570567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:20.989038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115361+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:08:50.115361+0000","last_peered":"2023-11-28T11:08:50.115361+0000","last_clean":"2023-11-28T11:08:50.115361+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:08:50.115361+0000","last_undegraded":"2023-11-28T11:08:50.115361+0000","last_fullsized":"2023-11-28T11:08:50.115361+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108211+0000","last_change":"2023-11-28T11:08:45.513181+0000","last_active":"2023-11-28T11:08:50.108211+0000","last_peered":"2023-11-28T11:08:50.108211+0000","last_clean":"2023-11-28T11:08:50.108211+0000","last_became_active":"2023-11-28T11:08:45.512903+0000","last_became_peered":"2023-11-28T11:08:45.512903+0000","last_unstale":"2023-11-28T11:08:50.108211+0000","last_undegraded":"2023-11-28T11:08:50.108211+0000","last_fullsized":"2023-11-28T11:08:50.108211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109406+0000","last_change":"2023-11-28T11:08:45.947505+0000","last_active":"2023-11-28T11:08:50.109406+0000","last_peered":"2023-11-28T11:08:50.109406+0000","last_clean":"2023-11-28T11:08:50.109406+0000","last_became_active":"2023-11-28T11:08:45.947199+0000","last_became_peered":"2023-11-28T11:08:45.947199+0000","last_unstale":"2023-11-28T11:08:50.109406+0000","last_undegraded":"2023-11-28T11:08:50.109406+0000","last_fullsized":"2023-11-28T11:08:50.109406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:55.085644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.437473+0000","last_change":"2023-11-28T11:08:50.126736+0000","last_active":"2023-11-28T11:08:51.437473+0000","last_peered":"2023-11-28T11:08:51.437473+0000","last_clean":"2023-11-28T11:08:51.437473+0000","last_became_active":"2023-11-28T11:08:50.126282+0000","last_became_peered":"2023-11-28T11:08:50.126282+0000","last_unstale":"2023-11-28T11:08:51.437473+0000","last_undegraded":"2023-11-28T11:08:51.437473+0000","last_fullsized":"2023-11-28T11:08:51.437473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:36.656033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.104943+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:08:50.104943+0000","last_peered":"2023-11-28T11:08:50.104943+0000","last_clean":"2023-11-28T11:08:50.104943+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:08:50.104943+0000","last_undegraded":"2023-11-28T11:08:50.104943+0000","last_fullsized":"2023-11-28T11:08:50.104943+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112806+0000","last_change":"2023-11-28T11:08:45.524531+0000","last_active":"2023-11-28T11:08:50.112806+0000","last_peered":"2023-11-28T11:08:50.112806+0000","last_clean":"2023-11-28T11:08:50.112806+0000","last_became_active":"2023-11-28T11:08:45.524334+0000","last_became_peered":"2023-11-28T11:08:45.524334+0000","last_unstale":"2023-11-28T11:08:50.112806+0000","last_undegraded":"2023-11-28T11:08:50.112806+0000","last_fullsized":"2023-11-28T11:08:50.112806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119778+0000","last_change":"2023-11-28T11:08:45.944621+0000","last_active":"2023-11-28T11:08:50.119778+0000","last_peered":"2023-11-28T11:08:50.119778+0000","last_clean":"2023-11-28T11:08:50.119778+0000","last_became_active":"2023-11-28T11:08:45.944368+0000","last_became_peered":"2023-11-28T11:08:45.944368+0000","last_unstale":"2023-11-28T11:08:50.119778+0000","last_undegraded":"2023-11-28T11:08:50.119778+0000","last_fullsized":"2023-11-28T11:08:50.119778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:01.447773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570969+0000","last_change":"2023-11-28T11:08:50.131902+0000","last_active":"2023-11-28T11:08:51.570969+0000","last_peered":"2023-11-28T11:08:51.570969+0000","last_clean":"2023-11-28T11:08:51.570969+0000","last_became_active":"2023-11-28T11:08:50.131485+0000","last_became_peered":"2023-11-28T11:08:50.131485+0000","last_unstale":"2023-11-28T11:08:51.570969+0000","last_undegraded":"2023-11-28T11:08:51.570969+0000","last_fullsized":"2023-11-28T11:08:51.570969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:08.804864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110005+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:08:50.110005+0000","last_peered":"2023-11-28T11:08:50.110005+0000","last_clean":"2023-11-28T11:08:50.110005+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:08:50.110005+0000","last_undegraded":"2023-11-28T11:08:50.110005+0000","last_fullsized":"2023-11-28T11:08:50.110005+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120247+0000","last_change":"2023-11-28T11:08:45.514598+0000","last_active":"2023-11-28T11:08:50.120247+0000","last_peered":"2023-11-28T11:08:50.120247+0000","last_clean":"2023-11-28T11:08:50.120247+0000","last_became_active":"2023-11-28T11:08:45.514348+0000","last_became_peered":"2023-11-28T11:08:45.514348+0000","last_unstale":"2023-11-28T11:08:50.120247+0000","last_undegraded":"2023-11-28T11:08:50.120247+0000","last_fullsized":"2023-11-28T11:08:50.120247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103691+0000","last_change":"2023-11-28T11:08:45.950031+0000","last_active":"2023-11-28T11:08:50.103691+0000","last_peered":"2023-11-28T11:08:50.103691+0000","last_clean":"2023-11-28T11:08:50.103691+0000","last_became_active":"2023-11-28T11:08:45.949457+0000","last_became_peered":"2023-11-28T11:08:45.949457+0000","last_unstale":"2023-11-28T11:08:50.103691+0000","last_undegraded":"2023-11-28T11:08:50.103691+0000","last_fullsized":"2023-11-28T11:08:50.103691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:34.416718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.437351+0000","last_change":"2023-11-28T11:08:50.124074+0000","last_active":"2023-11-28T11:08:51.437351+0000","last_peered":"2023-11-28T11:08:51.437351+0000","last_clean":"2023-11-28T11:08:51.437351+0000","last_became_active":"2023-11-28T11:08:50.123710+0000","last_became_peered":"2023-11-28T11:08:50.123710+0000","last_unstale":"2023-11-28T11:08:51.437351+0000","last_undegraded":"2023-11-28T11:08:51.437351+0000","last_fullsized":"2023-11-28T11:08:51.437351+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:42:35.306388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110239+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:08:50.110239+0000","last_peered":"2023-11-28T11:08:50.110239+0000","last_clean":"2023-11-28T11:08:50.110239+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:08:50.110239+0000","last_undegraded":"2023-11-28T11:08:50.110239+0000","last_fullsized":"2023-11-28T11:08:50.110239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105883+0000","last_change":"2023-11-28T11:08:45.514165+0000","last_active":"2023-11-28T11:08:50.105883+0000","last_peered":"2023-11-28T11:08:50.105883+0000","last_clean":"2023-11-28T11:08:50.105883+0000","last_became_active":"2023-11-28T11:08:45.513651+0000","last_became_peered":"2023-11-28T11:08:45.513651+0000","last_unstale":"2023-11-28T11:08:50.105883+0000","last_undegraded":"2023-11-28T11:08:50.105883+0000","last_fullsized":"2023-11-28T11:08:50.105883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119668+0000","last_change":"2023-11-28T11:08:45.949610+0000","last_active":"2023-11-28T11:08:50.119668+0000","last_peered":"2023-11-28T11:08:50.119668+0000","last_clean":"2023-11-28T11:08:50.119668+0000","last_became_active":"2023-11-28T11:08:45.949348+0000","last_became_peered":"2023-11-28T11:08:45.949348+0000","last_unstale":"2023-11-28T11:08:50.119668+0000","last_undegraded":"2023-11-28T11:08:50.119668+0000","last_fullsized":"2023-11-28T11:08:50.119668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:23:53.819338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570113+0000","last_change":"2023-11-28T11:08:50.135779+0000","last_active":"2023-11-28T11:08:51.570113+0000","last_peered":"2023-11-28T11:08:51.570113+0000","last_clean":"2023-11-28T11:08:51.570113+0000","last_became_active":"2023-11-28T11:08:50.131379+0000","last_became_peered":"2023-11-28T11:08:50.131379+0000","last_unstale":"2023-11-28T11:08:51.570113+0000","last_undegraded":"2023-11-28T11:08:51.570113+0000","last_fullsized":"2023-11-28T11:08:51.570113+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:42:35.306388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103804+0000","last_change":"2023-11-28T11:08:45.523090+0000","last_active":"2023-11-28T11:08:50.103804+0000","last_peered":"2023-11-28T11:08:50.103804+0000","last_clean":"2023-11-28T11:08:50.103804+0000","last_became_active":"2023-11-28T11:08:45.522445+0000","last_became_peered":"2023-11-28T11:08:45.522445+0000","last_unstale":"2023-11-28T11:08:50.103804+0000","last_undegraded":"2023-11-28T11:08:50.103804+0000","last_fullsized":"2023-11-28T11:08:50.103804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108643+0000","last_change":"2023-11-28T11:08:45.943849+0000","last_active":"2023-11-28T11:08:50.108643+0000","last_peered":"2023-11-28T11:08:50.108643+0000","last_clean":"2023-11-28T11:08:50.108643+0000","last_became_active":"2023-11-28T11:08:45.943426+0000","last_became_peered":"2023-11-28T11:08:45.943426+0000","last_unstale":"2023-11-28T11:08:50.108643+0000","last_undegraded":"2023-11-28T11:08:50.108643+0000","last_fullsized":"2023-11-28T11:08:50.108643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:29.591607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570960+0000","last_change":"2023-11-28T11:08:50.129403+0000","last_active":"2023-11-28T11:08:51.570960+0000","last_peered":"2023-11-28T11:08:51.570960+0000","last_clean":"2023-11-28T11:08:51.570960+0000","last_became_active":"2023-11-28T11:08:50.128984+0000","last_became_peered":"2023-11-28T11:08:50.128984+0000","last_unstale":"2023-11-28T11:08:51.570960+0000","last_undegraded":"2023-11-28T11:08:51.570960+0000","last_fullsized":"2023-11-28T11:08:51.570960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:54.778391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105418+0000","last_change":"2023-11-28T11:08:45.514673+0000","last_active":"2023-11-28T11:08:50.105418+0000","last_peered":"2023-11-28T11:08:50.105418+0000","last_clean":"2023-11-28T11:08:50.105418+0000","last_became_active":"2023-11-28T11:08:45.513328+0000","last_became_peered":"2023-11-28T11:08:45.513328+0000","last_unstale":"2023-11-28T11:08:50.105418+0000","last_undegraded":"2023-11-28T11:08:50.105418+0000","last_fullsized":"2023-11-28T11:08:50.105418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:55.864512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112696+0000","last_change":"2023-11-28T11:08:45.948904+0000","last_active":"2023-11-28T11:08:50.112696+0000","last_peered":"2023-11-28T11:08:50.112696+0000","last_clean":"2023-11-28T11:08:50.112696+0000","last_became_active":"2023-11-28T11:08:45.948629+0000","last_became_peered":"2023-11-28T11:08:45.948629+0000","last_unstale":"2023-11-28T11:08:50.112696+0000","last_undegraded":"2023-11-28T11:08:50.112696+0000","last_fullsized":"2023-11-28T11:08:50.112696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:55.031649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537746+0000","last_change":"2023-11-28T11:08:50.124117+0000","last_active":"2023-11-28T11:08:50.537746+0000","last_peered":"2023-11-28T11:08:50.537746+0000","last_clean":"2023-11-28T11:08:50.537746+0000","last_became_active":"2023-11-28T11:08:50.123257+0000","last_became_peered":"2023-11-28T11:08:50.123257+0000","last_unstale":"2023-11-28T11:08:50.537746+0000","last_undegraded":"2023-11-28T11:08:50.537746+0000","last_fullsized":"2023-11-28T11:08:50.537746+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:24.842089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105770+0000","last_change":"2023-11-28T11:08:45.511721+0000","last_active":"2023-11-28T11:08:50.105770+0000","last_peered":"2023-11-28T11:08:50.105770+0000","last_clean":"2023-11-28T11:08:50.105770+0000","last_became_active":"2023-11-28T11:08:45.511413+0000","last_became_peered":"2023-11-28T11:08:45.511413+0000","last_unstale":"2023-11-28T11:08:50.105770+0000","last_undegraded":"2023-11-28T11:08:50.105770+0000","last_fullsized":"2023-11-28T11:08:50.105770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:24.093818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.111702+0000","last_change":"2023-11-28T11:08:45.945561+0000","last_active":"2023-11-28T11:08:50.111702+0000","last_peered":"2023-11-28T11:08:50.111702+0000","last_clean":"2023-11-28T11:08:50.111702+0000","last_became_active":"2023-11-28T11:08:45.945101+0000","last_became_peered":"2023-11-28T11:08:45.945101+0000","last_unstale":"2023-11-28T11:08:50.111702+0000","last_undegraded":"2023-11-28T11:08:50.111702+0000","last_fullsized":"2023-11-28T11:08:50.111702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:00.598900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.535418+0000","last_change":"2023-11-28T11:08:50.132859+0000","last_active":"2023-11-28T11:08:50.535418+0000","last_peered":"2023-11-28T11:08:50.535418+0000","last_clean":"2023-11-28T11:08:50.535418+0000","last_became_active":"2023-11-28T11:08:50.132397+0000","last_became_peered":"2023-11-28T11:08:50.132397+0000","last_unstale":"2023-11-28T11:08:50.535418+0000","last_undegraded":"2023-11-28T11:08:50.535418+0000","last_fullsized":"2023-11-28T11:08:50.535418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:07.333534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112935+0000","last_change":"2023-11-28T11:08:45.511864+0000","last_active":"2023-11-28T11:08:50.112935+0000","last_peered":"2023-11-28T11:08:50.112935+0000","last_clean":"2023-11-28T11:08:50.112935+0000","last_became_active":"2023-11-28T11:08:45.511500+0000","last_became_peered":"2023-11-28T11:08:45.511500+0000","last_unstale":"2023-11-28T11:08:50.112935+0000","last_undegraded":"2023-11-28T11:08:50.112935+0000","last_fullsized":"2023-11-28T11:08:50.112935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:55.864512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108100+0000","last_change":"2023-11-28T11:08:45.945158+0000","last_active":"2023-11-28T11:08:50.108100+0000","last_peered":"2023-11-28T11:08:50.108100+0000","last_clean":"2023-11-28T11:08:50.108100+0000","last_became_active":"2023-11-28T11:08:45.944341+0000","last_became_peered":"2023-11-28T11:08:45.944341+0000","last_unstale":"2023-11-28T11:08:50.108100+0000","last_undegraded":"2023-11-28T11:08:50.108100+0000","last_fullsized":"2023-11-28T11:08:50.108100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569813+0000","last_change":"2023-11-28T11:08:50.136091+0000","last_active":"2023-11-28T11:08:51.569813+0000","last_peered":"2023-11-28T11:08:51.569813+0000","last_clean":"2023-11-28T11:08:51.569813+0000","last_became_active":"2023-11-28T11:08:50.135640+0000","last_became_peered":"2023-11-28T11:08:50.135640+0000","last_unstale":"2023-11-28T11:08:51.569813+0000","last_undegraded":"2023-11-28T11:08:51.569813+0000","last_fullsized":"2023-11-28T11:08:51.569813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:19:52.258463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106438+0000","last_change":"2023-11-28T11:08:45.522945+0000","last_active":"2023-11-28T11:08:50.106438+0000","last_peered":"2023-11-28T11:08:50.106438+0000","last_clean":"2023-11-28T11:08:50.106438+0000","last_became_active":"2023-11-28T11:08:45.522100+0000","last_became_peered":"2023-11-28T11:08:45.522100+0000","last_unstale":"2023-11-28T11:08:50.106438+0000","last_undegraded":"2023-11-28T11:08:50.106438+0000","last_fullsized":"2023-11-28T11:08:50.106438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:24.093818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.116376+0000","last_change":"2023-11-28T11:08:45.955608+0000","last_active":"2023-11-28T11:08:50.116376+0000","last_peered":"2023-11-28T11:08:50.116376+0000","last_clean":"2023-11-28T11:08:50.116376+0000","last_became_active":"2023-11-28T11:08:45.955293+0000","last_became_peered":"2023-11-28T11:08:45.955293+0000","last_unstale":"2023-11-28T11:08:50.116376+0000","last_undegraded":"2023-11-28T11:08:50.116376+0000","last_fullsized":"2023-11-28T11:08:50.116376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:33.088404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108532+0000","last_change":"2023-11-28T11:08:45.953710+0000","last_active":"2023-11-28T11:08:50.108532+0000","last_peered":"2023-11-28T11:08:50.108532+0000","last_clean":"2023-11-28T11:08:50.108532+0000","last_became_active":"2023-11-28T11:08:45.953349+0000","last_became_peered":"2023-11-28T11:08:45.953349+0000","last_unstale":"2023-11-28T11:08:50.108532+0000","last_undegraded":"2023-11-28T11:08:50.108532+0000","last_fullsized":"2023-11-28T11:08:50.108532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:03.032281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.107845+0000","last_change":"2023-11-28T11:08:45.945358+0000","last_active":"2023-11-28T11:08:50.107845+0000","last_peered":"2023-11-28T11:08:50.107845+0000","last_clean":"2023-11-28T11:08:50.107845+0000","last_became_active":"2023-11-28T11:08:45.944619+0000","last_became_peered":"2023-11-28T11:08:45.944619+0000","last_unstale":"2023-11-28T11:08:50.107845+0000","last_undegraded":"2023-11-28T11:08:50.107845+0000","last_fullsized":"2023-11-28T11:08:50.107845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:17.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112795+0000","last_change":"2023-11-28T11:08:45.951315+0000","last_active":"2023-11-28T11:08:50.112795+0000","last_peered":"2023-11-28T11:08:50.112795+0000","last_clean":"2023-11-28T11:08:50.112795+0000","last_became_active":"2023-11-28T11:08:45.950992+0000","last_became_peered":"2023-11-28T11:08:45.950992+0000","last_unstale":"2023-11-28T11:08:50.112795+0000","last_undegraded":"2023-11-28T11:08:50.112795+0000","last_fullsized":"2023-11-28T11:08:50.112795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:51.178080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109781+0000","last_change":"2023-11-28T11:08:45.954451+0000","last_active":"2023-11-28T11:08:50.109781+0000","last_peered":"2023-11-28T11:08:50.109781+0000","last_clean":"2023-11-28T11:08:50.109781+0000","last_became_active":"2023-11-28T11:08:45.954162+0000","last_became_peered":"2023-11-28T11:08:45.954162+0000","last_unstale":"2023-11-28T11:08:50.109781+0000","last_undegraded":"2023-11-28T11:08:50.109781+0000","last_fullsized":"2023-11-28T11:08:50.109781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:39.158725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119893+0000","last_change":"2023-11-28T11:08:45.944860+0000","last_active":"2023-11-28T11:08:50.119893+0000","last_peered":"2023-11-28T11:08:50.119893+0000","last_clean":"2023-11-28T11:08:50.119893+0000","last_became_active":"2023-11-28T11:08:45.944591+0000","last_became_peered":"2023-11-28T11:08:45.944591+0000","last_unstale":"2023-11-28T11:08:50.119893+0000","last_undegraded":"2023-11-28T11:08:50.119893+0000","last_fullsized":"2023-11-28T11:08:50.119893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4528,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574857,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":703731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39900000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54500000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574856,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70699999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53900000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9232,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362608,"statfs":{"total":96636764160,"available":96627310592,"internally_reserved":0,"allocated":860160,"data_stored":699769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574857,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":108440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32900000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.753}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8224,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363616,"statfs":{"total":96636764160,"available":96628342784,"internally_reserved":0,"allocated":286720,"data_stored":113203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8084,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363756,"statfs":{"total":96636764160,"available":96628486144,"internally_reserved":0,"allocated":274432,"data_stored":110037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:09:09.110 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:09:09.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.266+0000 7f607aba4700 1 Processor -- start 2023-11-28T11:09:09.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.266+0000 7f607aba4700 1 -- start start 2023-11-28T11:09:09.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f607aba4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f607415a5a0 0x7f607406b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f607aba4700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60740cb570 con 0x7f60740ca280 2023-11-28T11:09:09.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f607aba4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60740ca6f0 con 0x7f60740cc4b0 2023-11-28T11:09:09.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f607aba4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60740cc110 con 0x7f607415a5a0 2023-11-28T11:09:09.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f6073fff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f60740ca280 0x7f60740cba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35394/0 (socket says 172.21.15.28:35394) 2023-11-28T11:09:09.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f6073fff700 1 -- 172.21.15.28:0/2911478657 learned_addr learned my addr 172.21.15.28:0/2911478657 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:09.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f6078940700 1 --2- 172.21.15.28:0/2911478657 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f607415a5a0 0x7f607406b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f60737fe700 1 -- 172.21.15.28:0/2911478657 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 292672719 0 0) 0x7f60740cb570 con 0x7f60740ca280 2023-11-28T11:09:09.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f60737fe700 1 -- 172.21.15.28:0/2911478657 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60640035b0 con 0x7f60740ca280 2023-11-28T11:09:09.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f60737fe700 1 -- 172.21.15.28:0/2911478657 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3516203736 0 0) 0x7f60740ca6f0 con 0x7f60740cc4b0 2023-11-28T11:09:09.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f60737fe700 1 -- 172.21.15.28:0/2911478657 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60740cb570 con 0x7f60740cc4b0 2023-11-28T11:09:09.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f60737fe700 1 -- 172.21.15.28:0/2911478657 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2249953026 0 0) 0x7f60640035b0 con 0x7f60740ca280 2023-11-28T11:09:09.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f6078940700 1 -- 172.21.15.28:0/2911478657 >> v1:172.21.15.59:6789/0 conn(0x7f60740cc4b0 legacy=0x7f60740c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f6078940700 1 -- 172.21.15.28:0/2911478657 >> v1:172.21.15.28:6789/0 conn(0x7f60740ca280 legacy=0x7f60740cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f6078940700 1 -- 172.21.15.28:0/2911478657 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60740ca960 con 0x7f607415a5a0 2023-11-28T11:09:09.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.267+0000 7f6078940700 1 --2- 172.21.15.28:0/2911478657 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f607415a5a0 0x7f607406b930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6068009870 tx=0x7f6068009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca49fcf6d5d8a2bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:09.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.268+0000 7f60737fe700 1 -- 172.21.15.28:0/2911478657 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6060001670 con 0x7f607415a5a0 2023-11-28T11:09:09.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.268+0000 7f60737fe700 1 -- 172.21.15.28:0/2911478657 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6068004030 con 0x7f607415a5a0 2023-11-28T11:09:09.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.268+0000 7f60737fe700 1 -- 172.21.15.28:0/2911478657 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60680043a0 con 0x7f607415a5a0 2023-11-28T11:09:09.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.268+0000 7f607aba4700 1 -- 172.21.15.28:0/2911478657 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f607415a5a0 msgr2=0x7f607406b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.268+0000 7f607aba4700 1 --2- 172.21.15.28:0/2911478657 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f607415a5a0 0x7f607406b930 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6068009870 tx=0x7f6068009b80 comp rx=0 tx=0).stop 2023-11-28T11:09:09.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.268+0000 7f607aba4700 1 -- 172.21.15.28:0/2911478657 shutdown_connections 2023-11-28T11:09:09.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.268+0000 7f607aba4700 1 --2- 172.21.15.28:0/2911478657 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f607415a5a0 0x7f607406b930 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.268+0000 7f607aba4700 1 -- 172.21.15.28:0/2911478657 >> 172.21.15.28:0/2911478657 conn(0x7f60740b1650 msgr2=0x7f60740b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:09.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.268+0000 7f607aba4700 1 -- 172.21.15.28:0/2911478657 shutdown_connections 2023-11-28T11:09:09.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.268+0000 7f607aba4700 1 -- 172.21.15.28:0/2911478657 wait complete. 2023-11-28T11:09:09.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.270+0000 7f607aba4700 1 Processor -- start 2023-11-28T11:09:09.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.270+0000 7f607aba4700 1 -- start start 2023-11-28T11:09:09.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f607aba4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60740ca280 0x7f607414dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f607aba4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60740cc4b0 0x7f60740c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f607aba4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f607415a5a0 0x7f607415bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f607aba4700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60740c7bf0 con 0x7f60740ca280 2023-11-28T11:09:09.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f607aba4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60740c6fe0 con 0x7f60740cc4b0 2023-11-28T11:09:09.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f607aba4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60740c72c0 con 0x7f607415a5a0 2023-11-28T11:09:09.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f6078940700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60740ca280 0x7f607414dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f6078940700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60740ca280 0x7f607414dad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47340/0 (socket says 172.21.15.28:47340) 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f6078940700 1 -- 172.21.15.28:0/2177425993 learned_addr learned my addr 172.21.15.28:0/2177425993 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f6073fff700 1 --2- 172.21.15.28:0/2177425993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60740cc4b0 0x7f60740c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f6079141700 1 --2- 172.21.15.28:0/2177425993 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f607415a5a0 0x7f607415bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f6078940700 1 -- 172.21.15.28:0/2177425993 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f607415a5a0 msgr2=0x7f607415bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f6078940700 1 --2- 172.21.15.28:0/2177425993 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f607415a5a0 0x7f607415bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f6078940700 1 -- 172.21.15.28:0/2177425993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60740cc4b0 msgr2=0x7f60740c8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f6078940700 1 --2- 172.21.15.28:0/2177425993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60740cc4b0 0x7f60740c8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f6078940700 1 -- 172.21.15.28:0/2177425993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60680094a0 con 0x7f60740ca280 2023-11-28T11:09:09.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f6078940700 1 --2- 172.21.15.28:0/2177425993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60740ca280 0x7f607414dad0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f60680090b0 tx=0x7f6068004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f60717fa700 1 -- 172.21.15.28:0/2177425993 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6068003c50 con 0x7f60740ca280 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f60717fa700 1 -- 172.21.15.28:0/2177425993 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f606801f070 con 0x7f60740ca280 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f607aba4700 1 -- 172.21.15.28:0/2177425993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60740caf20 con 0x7f60740ca280 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f60717fa700 1 -- 172.21.15.28:0/2177425993 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60680045b0 con 0x7f60740ca280 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.271+0000 7f607aba4700 1 -- 172.21.15.28:0/2177425993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60740cac40 con 0x7f60740ca280 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.272+0000 7f607aba4700 1 -- 172.21.15.28:0/2177425993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60740cb200 con 0x7f60740ca280 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.273+0000 7f60717fa700 1 -- 172.21.15.28:0/2177425993 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f606802a020 con 0x7f60740ca280 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.273+0000 7f60717fa700 1 --2- 172.21.15.28:0/2177425993 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60580682f0 0x7f605806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.273+0000 7f60717fa700 1 -- 172.21.15.28:0/2177425993 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6068087310 con 0x7f60740ca280 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.273+0000 7f6073fff700 1 --2- 172.21.15.28:0/2177425993 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60580682f0 0x7f605806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.274+0000 7f6073fff700 1 --2- 172.21.15.28:0/2177425993 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60580682f0 0x7f605806a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f60740c6740 tx=0x7f6060001d80 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:09.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.277+0000 7f60717fa700 1 -- 172.21.15.28:0/2177425993 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6068053880 con 0x7f60740ca280 2023-11-28T11:09:09.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.468+0000 7f607aba4700 1 -- 172.21.15.28:0/2177425993 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f60740c7ed0 con 0x7f60580682f0 2023-11-28T11:09:09.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.477+0000 7f60717fa700 1 -- 172.21.15.28:0/2177425993 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+367434 (secure 0 0 0) 0x7f60740c7ed0 con 0x7f60580682f0 2023-11-28T11:09:09.483 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:09.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.481+0000 7f607aba4700 1 -- 172.21.15.28:0/2177425993 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60580682f0 msgr2=0x7f605806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.481+0000 7f607aba4700 1 --2- 172.21.15.28:0/2177425993 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60580682f0 0x7f605806a7a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f60740c6740 tx=0x7f6060001d80 comp rx=0 tx=0).stop 2023-11-28T11:09:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.481+0000 7f607aba4700 1 -- 172.21.15.28:0/2177425993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60740ca280 msgr2=0x7f607414dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.481+0000 7f607aba4700 1 --2- 172.21.15.28:0/2177425993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60740ca280 0x7f607414dad0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f60680090b0 tx=0x7f6068004750 comp rx=0 tx=0).stop 2023-11-28T11:09:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.481+0000 7f607aba4700 1 -- 172.21.15.28:0/2177425993 shutdown_connections 2023-11-28T11:09:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.481+0000 7f607aba4700 1 --2- 172.21.15.28:0/2177425993 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60580682f0 0x7f605806a7a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.481+0000 7f607aba4700 1 --2- 172.21.15.28:0/2177425993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60740ca280 0x7f607414dad0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.481+0000 7f607aba4700 1 --2- 172.21.15.28:0/2177425993 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f607415a5a0 0x7f607415bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.481+0000 7f607aba4700 1 --2- 172.21.15.28:0/2177425993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f60740cc4b0 0x7f60740c8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.481+0000 7f607aba4700 1 -- 172.21.15.28:0/2177425993 >> 172.21.15.28:0/2177425993 conn(0x7f60740b1650 msgr2=0x7f60741572f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:09.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.481+0000 7f607aba4700 1 -- 172.21.15.28:0/2177425993 shutdown_connections 2023-11-28T11:09:09.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.481+0000 7f607aba4700 1 -- 172.21.15.28:0/2177425993 wait complete. 2023-11-28T11:09:09.488 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:09:09.494 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":34,"stamp":"2023-11-28T11:09:09.167587+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":594896,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":207,"num_write_kb":2434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":338,"acting":338,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":338,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":67236,"kb_used_data":3300,"kb_used_omap":0,"kb_used_meta":63872,"kb_avail":754907484,"statfs":{"total":773094113280,"available":773025263616,"internally_reserved":0,"allocated":3379200,"data_stored":2064039,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":65404928},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2198,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003046"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108863+0000","last_change":"2023-11-28T11:08:45.948753+0000","last_active":"2023-11-28T11:08:50.108863+0000","last_peered":"2023-11-28T11:08:50.108863+0000","last_clean":"2023-11-28T11:08:50.108863+0000","last_became_active":"2023-11-28T11:08:45.948435+0000","last_became_peered":"2023-11-28T11:08:45.948435+0000","last_unstale":"2023-11-28T11:08:50.108863+0000","last_undegraded":"2023-11-28T11:08:50.108863+0000","last_fullsized":"2023-11-28T11:08:50.108863+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:28.944484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105146+0000","last_change":"2023-11-28T11:08:45.946452+0000","last_active":"2023-11-28T11:08:50.105146+0000","last_peered":"2023-11-28T11:08:50.105146+0000","last_clean":"2023-11-28T11:08:50.105146+0000","last_became_active":"2023-11-28T11:08:45.946145+0000","last_became_peered":"2023-11-28T11:08:45.946145+0000","last_unstale":"2023-11-28T11:08:50.105146+0000","last_undegraded":"2023-11-28T11:08:50.105146+0000","last_fullsized":"2023-11-28T11:08:50.105146+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:30.639887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113411+0000","last_change":"2023-11-28T11:08:45.941140+0000","last_active":"2023-11-28T11:08:50.113411+0000","last_peered":"2023-11-28T11:08:50.113411+0000","last_clean":"2023-11-28T11:08:50.113411+0000","last_became_active":"2023-11-28T11:08:45.939618+0000","last_became_peered":"2023-11-28T11:08:45.939618+0000","last_unstale":"2023-11-28T11:08:50.113411+0000","last_undegraded":"2023-11-28T11:08:50.113411+0000","last_fullsized":"2023-11-28T11:08:50.113411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:22:44.444264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109455+0000","last_change":"2023-11-28T11:08:45.947665+0000","last_active":"2023-11-28T11:08:50.109455+0000","last_peered":"2023-11-28T11:08:50.109455+0000","last_clean":"2023-11-28T11:08:50.109455+0000","last_became_active":"2023-11-28T11:08:45.947338+0000","last_became_peered":"2023-11-28T11:08:45.947338+0000","last_unstale":"2023-11-28T11:08:50.109455+0000","last_undegraded":"2023-11-28T11:08:50.109455+0000","last_fullsized":"2023-11-28T11:08:50.109455+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:23:53.819338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103496+0000","last_change":"2023-11-28T11:08:45.950223+0000","last_active":"2023-11-28T11:08:50.103496+0000","last_peered":"2023-11-28T11:08:50.103496+0000","last_clean":"2023-11-28T11:08:50.103496+0000","last_became_active":"2023-11-28T11:08:45.949811+0000","last_became_peered":"2023-11-28T11:08:45.949811+0000","last_unstale":"2023-11-28T11:08:50.103496+0000","last_undegraded":"2023-11-28T11:08:50.103496+0000","last_fullsized":"2023-11-28T11:08:50.103496+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112890+0000","last_change":"2023-11-28T11:08:45.942787+0000","last_active":"2023-11-28T11:08:50.112890+0000","last_peered":"2023-11-28T11:08:50.112890+0000","last_clean":"2023-11-28T11:08:50.112890+0000","last_became_active":"2023-11-28T11:08:45.941820+0000","last_became_peered":"2023-11-28T11:08:45.941820+0000","last_unstale":"2023-11-28T11:08:50.112890+0000","last_undegraded":"2023-11-28T11:08:50.112890+0000","last_fullsized":"2023-11-28T11:08:50.112890+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:52:22.504112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115835+0000","last_change":"2023-11-28T11:08:45.953364+0000","last_active":"2023-11-28T11:08:50.115835+0000","last_peered":"2023-11-28T11:08:50.115835+0000","last_clean":"2023-11-28T11:08:50.115835+0000","last_became_active":"2023-11-28T11:08:45.953033+0000","last_became_peered":"2023-11-28T11:08:45.953033+0000","last_unstale":"2023-11-28T11:08:50.115835+0000","last_undegraded":"2023-11-28T11:08:50.115835+0000","last_fullsized":"2023-11-28T11:08:50.115835+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105175+0000","last_change":"2023-11-28T11:08:45.947045+0000","last_active":"2023-11-28T11:08:50.105175+0000","last_peered":"2023-11-28T11:08:50.105175+0000","last_clean":"2023-11-28T11:08:50.105175+0000","last_became_active":"2023-11-28T11:08:45.946756+0000","last_became_peered":"2023-11-28T11:08:45.946756+0000","last_unstale":"2023-11-28T11:08:50.105175+0000","last_undegraded":"2023-11-28T11:08:50.105175+0000","last_fullsized":"2023-11-28T11:08:50.105175+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:56.811805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103692+0000","last_change":"2023-11-28T11:08:45.946321+0000","last_active":"2023-11-28T11:08:50.103692+0000","last_peered":"2023-11-28T11:08:50.103692+0000","last_clean":"2023-11-28T11:08:50.103692+0000","last_became_active":"2023-11-28T11:08:45.943687+0000","last_became_peered":"2023-11-28T11:08:45.943687+0000","last_unstale":"2023-11-28T11:08:50.103692+0000","last_undegraded":"2023-11-28T11:08:50.103692+0000","last_fullsized":"2023-11-28T11:08:50.103692+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:23.627932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105317+0000","last_change":"2023-11-28T11:08:45.948529+0000","last_active":"2023-11-28T11:08:50.105317+0000","last_peered":"2023-11-28T11:08:50.105317+0000","last_clean":"2023-11-28T11:08:50.105317+0000","last_became_active":"2023-11-28T11:08:45.948249+0000","last_became_peered":"2023-11-28T11:08:45.948249+0000","last_unstale":"2023-11-28T11:08:50.105317+0000","last_undegraded":"2023-11-28T11:08:50.105317+0000","last_fullsized":"2023-11-28T11:08:50.105317+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113311+0000","last_change":"2023-11-28T11:08:45.942374+0000","last_active":"2023-11-28T11:08:50.113311+0000","last_peered":"2023-11-28T11:08:50.113311+0000","last_clean":"2023-11-28T11:08:50.113311+0000","last_became_active":"2023-11-28T11:08:45.942083+0000","last_became_peered":"2023-11-28T11:08:45.942083+0000","last_unstale":"2023-11-28T11:08:50.113311+0000","last_undegraded":"2023-11-28T11:08:50.113311+0000","last_fullsized":"2023-11-28T11:08:50.113311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:23.627932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108474+0000","last_change":"2023-11-28T11:08:45.947339+0000","last_active":"2023-11-28T11:08:50.108474+0000","last_peered":"2023-11-28T11:08:50.108474+0000","last_clean":"2023-11-28T11:08:50.108474+0000","last_became_active":"2023-11-28T11:08:45.947066+0000","last_became_peered":"2023-11-28T11:08:45.947066+0000","last_unstale":"2023-11-28T11:08:50.108474+0000","last_undegraded":"2023-11-28T11:08:50.108474+0000","last_fullsized":"2023-11-28T11:08:50.108474+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:48.010812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109210+0000","last_change":"2023-11-28T11:08:45.949094+0000","last_active":"2023-11-28T11:08:50.109210+0000","last_peered":"2023-11-28T11:08:50.109210+0000","last_clean":"2023-11-28T11:08:50.109210+0000","last_became_active":"2023-11-28T11:08:45.948523+0000","last_became_peered":"2023-11-28T11:08:45.948523+0000","last_unstale":"2023-11-28T11:08:50.109210+0000","last_undegraded":"2023-11-28T11:08:50.109210+0000","last_fullsized":"2023-11-28T11:08:50.109210+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:34.193700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105493+0000","last_change":"2023-11-28T11:08:45.945708+0000","last_active":"2023-11-28T11:08:50.105493+0000","last_peered":"2023-11-28T11:08:50.105493+0000","last_clean":"2023-11-28T11:08:50.105493+0000","last_became_active":"2023-11-28T11:08:45.945408+0000","last_became_peered":"2023-11-28T11:08:45.945408+0000","last_unstale":"2023-11-28T11:08:50.105493+0000","last_undegraded":"2023-11-28T11:08:50.105493+0000","last_fullsized":"2023-11-28T11:08:50.105493+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:34.416718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115608+0000","last_change":"2023-11-28T11:08:45.954131+0000","last_active":"2023-11-28T11:08:50.115608+0000","last_peered":"2023-11-28T11:08:50.115608+0000","last_clean":"2023-11-28T11:08:50.115608+0000","last_became_active":"2023-11-28T11:08:45.953824+0000","last_became_peered":"2023-11-28T11:08:45.953824+0000","last_unstale":"2023-11-28T11:08:50.115608+0000","last_undegraded":"2023-11-28T11:08:50.115608+0000","last_fullsized":"2023-11-28T11:08:50.115608+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113028+0000","last_change":"2023-11-28T11:08:45.943568+0000","last_active":"2023-11-28T11:08:50.113028+0000","last_peered":"2023-11-28T11:08:50.113028+0000","last_clean":"2023-11-28T11:08:50.113028+0000","last_became_active":"2023-11-28T11:08:45.942637+0000","last_became_peered":"2023-11-28T11:08:45.942637+0000","last_unstale":"2023-11-28T11:08:50.113028+0000","last_undegraded":"2023-11-28T11:08:50.113028+0000","last_fullsized":"2023-11-28T11:08:50.113028+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:17.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106326+0000","last_change":"2023-11-28T11:08:45.944177+0000","last_active":"2023-11-28T11:08:50.106326+0000","last_peered":"2023-11-28T11:08:50.106326+0000","last_clean":"2023-11-28T11:08:50.106326+0000","last_became_active":"2023-11-28T11:08:45.943369+0000","last_became_peered":"2023-11-28T11:08:45.943369+0000","last_unstale":"2023-11-28T11:08:50.106326+0000","last_undegraded":"2023-11-28T11:08:50.106326+0000","last_fullsized":"2023-11-28T11:08:50.106326+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:56.811805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113581+0000","last_change":"2023-11-28T11:08:45.943949+0000","last_active":"2023-11-28T11:08:50.113581+0000","last_peered":"2023-11-28T11:08:50.113581+0000","last_clean":"2023-11-28T11:08:50.113581+0000","last_became_active":"2023-11-28T11:08:45.942746+0000","last_became_peered":"2023-11-28T11:08:45.942746+0000","last_unstale":"2023-11-28T11:08:50.113581+0000","last_undegraded":"2023-11-28T11:08:50.113581+0000","last_fullsized":"2023-11-28T11:08:50.113581+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:49.186302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110612+0000","last_change":"2023-11-28T11:08:45.954401+0000","last_active":"2023-11-28T11:08:50.110612+0000","last_peered":"2023-11-28T11:08:50.110612+0000","last_clean":"2023-11-28T11:08:50.110612+0000","last_became_active":"2023-11-28T11:08:45.954130+0000","last_became_peered":"2023-11-28T11:08:45.954130+0000","last_unstale":"2023-11-28T11:08:50.110612+0000","last_undegraded":"2023-11-28T11:08:50.110612+0000","last_fullsized":"2023-11-28T11:08:50.110612+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:55.295125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113075+0000","last_change":"2023-11-28T11:08:45.956559+0000","last_active":"2023-11-28T11:08:50.113075+0000","last_peered":"2023-11-28T11:08:50.113075+0000","last_clean":"2023-11-28T11:08:50.113075+0000","last_became_active":"2023-11-28T11:08:45.956269+0000","last_became_peered":"2023-11-28T11:08:45.956269+0000","last_unstale":"2023-11-28T11:08:50.113075+0000","last_undegraded":"2023-11-28T11:08:50.113075+0000","last_fullsized":"2023-11-28T11:08:50.113075+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:34:17.189127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115796+0000","last_change":"2023-11-28T11:08:45.951043+0000","last_active":"2023-11-28T11:08:50.115796+0000","last_peered":"2023-11-28T11:08:50.115796+0000","last_clean":"2023-11-28T11:08:50.115796+0000","last_became_active":"2023-11-28T11:08:45.950740+0000","last_became_peered":"2023-11-28T11:08:45.950740+0000","last_unstale":"2023-11-28T11:08:50.115796+0000","last_undegraded":"2023-11-28T11:08:50.115796+0000","last_fullsized":"2023-11-28T11:08:50.115796+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103425+0000","last_change":"2023-11-28T11:08:45.940737+0000","last_active":"2023-11-28T11:08:50.103425+0000","last_peered":"2023-11-28T11:08:50.103425+0000","last_clean":"2023-11-28T11:08:50.103425+0000","last_became_active":"2023-11-28T11:08:45.940490+0000","last_became_peered":"2023-11-28T11:08:45.940490+0000","last_unstale":"2023-11-28T11:08:50.103425+0000","last_undegraded":"2023-11-28T11:08:50.103425+0000","last_fullsized":"2023-11-28T11:08:50.103425+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.107940+0000","last_change":"2023-11-28T11:08:45.945519+0000","last_active":"2023-11-28T11:08:50.107940+0000","last_peered":"2023-11-28T11:08:50.107940+0000","last_clean":"2023-11-28T11:08:50.107940+0000","last_became_active":"2023-11-28T11:08:45.944911+0000","last_became_peered":"2023-11-28T11:08:45.944911+0000","last_unstale":"2023-11-28T11:08:50.107940+0000","last_undegraded":"2023-11-28T11:08:50.107940+0000","last_fullsized":"2023-11-28T11:08:50.107940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:05.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115473+0000","last_change":"2023-11-28T11:08:45.954870+0000","last_active":"2023-11-28T11:08:50.115473+0000","last_peered":"2023-11-28T11:08:50.115473+0000","last_clean":"2023-11-28T11:08:50.115473+0000","last_became_active":"2023-11-28T11:08:45.954442+0000","last_became_peered":"2023-11-28T11:08:45.954442+0000","last_unstale":"2023-11-28T11:08:50.115473+0000","last_undegraded":"2023-11-28T11:08:50.115473+0000","last_fullsized":"2023-11-28T11:08:50.115473+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120450+0000","last_change":"2023-11-28T11:08:45.954073+0000","last_active":"2023-11-28T11:08:50.120450+0000","last_peered":"2023-11-28T11:08:50.120450+0000","last_clean":"2023-11-28T11:08:50.120450+0000","last_became_active":"2023-11-28T11:08:45.953886+0000","last_became_peered":"2023-11-28T11:08:45.953886+0000","last_unstale":"2023-11-28T11:08:50.120450+0000","last_undegraded":"2023-11-28T11:08:50.120450+0000","last_fullsized":"2023-11-28T11:08:50.120450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:39:56.811805+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.121555+0000","last_change":"2023-11-28T11:08:45.943121+0000","last_active":"2023-11-28T11:08:50.121555+0000","last_peered":"2023-11-28T11:08:50.121555+0000","last_clean":"2023-11-28T11:08:50.121555+0000","last_became_active":"2023-11-28T11:08:45.942796+0000","last_became_peered":"2023-11-28T11:08:45.942796+0000","last_unstale":"2023-11-28T11:08:50.121555+0000","last_undegraded":"2023-11-28T11:08:50.121555+0000","last_fullsized":"2023-11-28T11:08:50.121555+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:55.085644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106066+0000","last_change":"2023-11-28T11:08:45.957422+0000","last_active":"2023-11-28T11:08:50.106066+0000","last_peered":"2023-11-28T11:08:50.106066+0000","last_clean":"2023-11-28T11:08:50.106066+0000","last_became_active":"2023-11-28T11:08:45.956775+0000","last_became_peered":"2023-11-28T11:08:45.956775+0000","last_unstale":"2023-11-28T11:08:50.106066+0000","last_undegraded":"2023-11-28T11:08:50.106066+0000","last_fullsized":"2023-11-28T11:08:50.106066+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.536690+0000","last_change":"2023-11-28T11:08:50.132954+0000","last_active":"2023-11-28T11:08:50.536690+0000","last_peered":"2023-11-28T11:08:50.536690+0000","last_clean":"2023-11-28T11:08:50.536690+0000","last_became_active":"2023-11-28T11:08:50.132496+0000","last_became_peered":"2023-11-28T11:08:50.132496+0000","last_unstale":"2023-11-28T11:08:50.536690+0000","last_undegraded":"2023-11-28T11:08:50.536690+0000","last_fullsized":"2023-11-28T11:08:50.536690+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:19:52.225499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2,0],"acting":[4,3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1a","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112687+0000","last_change":"2023-11-28T11:08:45.515522+0000","last_active":"2023-11-28T11:08:50.112687+0000","last_peered":"2023-11-28T11:08:50.112687+0000","last_clean":"2023-11-28T11:08:50.112687+0000","last_became_active":"2023-11-28T11:08:45.515238+0000","last_became_peered":"2023-11-28T11:08:45.515238+0000","last_unstale":"2023-11-28T11:08:50.112687+0000","last_undegraded":"2023-11-28T11:08:50.112687+0000","last_fullsized":"2023-11-28T11:08:50.112687+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109971+0000","last_change":"2023-11-28T11:08:45.945792+0000","last_active":"2023-11-28T11:08:50.109971+0000","last_peered":"2023-11-28T11:08:50.109971+0000","last_clean":"2023-11-28T11:08:50.109971+0000","last_became_active":"2023-11-28T11:08:45.945536+0000","last_became_peered":"2023-11-28T11:08:45.945536+0000","last_unstale":"2023-11-28T11:08:50.109971+0000","last_undegraded":"2023-11-28T11:08:50.109971+0000","last_fullsized":"2023-11-28T11:08:50.109971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:49.186302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571306+0000","last_change":"2023-11-28T11:08:50.124166+0000","last_active":"2023-11-28T11:08:51.571306+0000","last_peered":"2023-11-28T11:08:51.571306+0000","last_clean":"2023-11-28T11:08:51.571306+0000","last_became_active":"2023-11-28T11:08:50.123744+0000","last_became_peered":"2023-11-28T11:08:50.123744+0000","last_unstale":"2023-11-28T11:08:51.571306+0000","last_undegraded":"2023-11-28T11:08:51.571306+0000","last_fullsized":"2023-11-28T11:08:51.571306+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:58.631960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,3,7],"acting":[6,4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119490+0000","last_change":"2023-11-28T11:08:45.516923+0000","last_active":"2023-11-28T11:08:50.119490+0000","last_peered":"2023-11-28T11:08:50.119490+0000","last_clean":"2023-11-28T11:08:50.119490+0000","last_became_active":"2023-11-28T11:08:45.516618+0000","last_became_peered":"2023-11-28T11:08:45.516618+0000","last_unstale":"2023-11-28T11:08:50.119490+0000","last_undegraded":"2023-11-28T11:08:50.119490+0000","last_fullsized":"2023-11-28T11:08:50.119490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103765+0000","last_change":"2023-11-28T11:08:45.941787+0000","last_active":"2023-11-28T11:08:50.103765+0000","last_peered":"2023-11-28T11:08:50.103765+0000","last_clean":"2023-11-28T11:08:50.103765+0000","last_became_active":"2023-11-28T11:08:45.941379+0000","last_became_peered":"2023-11-28T11:08:45.941379+0000","last_unstale":"2023-11-28T11:08:50.103765+0000","last_undegraded":"2023-11-28T11:08:50.103765+0000","last_fullsized":"2023-11-28T11:08:50.103765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.437643+0000","last_change":"2023-11-28T11:08:50.136708+0000","last_active":"2023-11-28T11:08:51.437643+0000","last_peered":"2023-11-28T11:08:51.437643+0000","last_clean":"2023-11-28T11:08:51.437643+0000","last_became_active":"2023-11-28T11:08:50.133846+0000","last_became_peered":"2023-11-28T11:08:50.133846+0000","last_unstale":"2023-11-28T11:08:51.437643+0000","last_undegraded":"2023-11-28T11:08:51.437643+0000","last_fullsized":"2023-11-28T11:08:51.437643+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:11:43.645799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,1,2],"acting":[0,7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109280+0000","last_change":"2023-11-28T11:08:45.524714+0000","last_active":"2023-11-28T11:08:50.109280+0000","last_peered":"2023-11-28T11:08:50.109280+0000","last_clean":"2023-11-28T11:08:50.109280+0000","last_became_active":"2023-11-28T11:08:45.524202+0000","last_became_peered":"2023-11-28T11:08:45.524202+0000","last_unstale":"2023-11-28T11:08:50.109280+0000","last_undegraded":"2023-11-28T11:08:50.109280+0000","last_fullsized":"2023-11-28T11:08:50.109280+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:49.800409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110311+0000","last_change":"2023-11-28T11:08:45.952452+0000","last_active":"2023-11-28T11:08:50.110311+0000","last_peered":"2023-11-28T11:08:50.110311+0000","last_clean":"2023-11-28T11:08:50.110311+0000","last_became_active":"2023-11-28T11:08:45.952159+0000","last_became_peered":"2023-11-28T11:08:45.952159+0000","last_unstale":"2023-11-28T11:08:50.110311+0000","last_undegraded":"2023-11-28T11:08:50.110311+0000","last_fullsized":"2023-11-28T11:08:50.110311+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:58:31.173555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569992+0000","last_change":"2023-11-28T11:08:50.123500+0000","last_active":"2023-11-28T11:08:51.569992+0000","last_peered":"2023-11-28T11:08:51.569992+0000","last_clean":"2023-11-28T11:08:51.569992+0000","last_became_active":"2023-11-28T11:08:50.123102+0000","last_became_peered":"2023-11-28T11:08:50.123102+0000","last_unstale":"2023-11-28T11:08:51.569992+0000","last_undegraded":"2023-11-28T11:08:51.569992+0000","last_fullsized":"2023-11-28T11:08:51.569992+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:36.688715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7,1],"acting":[6,4,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106385+0000","last_change":"2023-11-28T11:08:45.523570+0000","last_active":"2023-11-28T11:08:50.106385+0000","last_peered":"2023-11-28T11:08:50.106385+0000","last_clean":"2023-11-28T11:08:50.106385+0000","last_became_active":"2023-11-28T11:08:45.523159+0000","last_became_peered":"2023-11-28T11:08:45.523159+0000","last_unstale":"2023-11-28T11:08:50.106385+0000","last_undegraded":"2023-11-28T11:08:50.106385+0000","last_fullsized":"2023-11-28T11:08:50.106385+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109190+0000","last_change":"2023-11-28T11:08:45.943601+0000","last_active":"2023-11-28T11:08:50.109190+0000","last_peered":"2023-11-28T11:08:50.109190+0000","last_clean":"2023-11-28T11:08:50.109190+0000","last_became_active":"2023-11-28T11:08:45.943333+0000","last_became_peered":"2023-11-28T11:08:45.943333+0000","last_unstale":"2023-11-28T11:08:50.109190+0000","last_undegraded":"2023-11-28T11:08:50.109190+0000","last_fullsized":"2023-11-28T11:08:50.109190+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569794+0000","last_change":"2023-11-28T11:08:50.126326+0000","last_active":"2023-11-28T11:08:51.569794+0000","last_peered":"2023-11-28T11:08:51.569794+0000","last_clean":"2023-11-28T11:08:51.569794+0000","last_became_active":"2023-11-28T11:08:50.125641+0000","last_became_peered":"2023-11-28T11:08:50.125641+0000","last_unstale":"2023-11-28T11:08:51.569794+0000","last_undegraded":"2023-11-28T11:08:51.569794+0000","last_fullsized":"2023-11-28T11:08:51.569794+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:59.602687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6,0],"acting":[7,4,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105179+0000","last_change":"2023-11-28T11:08:45.518910+0000","last_active":"2023-11-28T11:08:50.105179+0000","last_peered":"2023-11-28T11:08:50.105179+0000","last_clean":"2023-11-28T11:08:50.105179+0000","last_became_active":"2023-11-28T11:08:45.518652+0000","last_became_peered":"2023-11-28T11:08:45.518652+0000","last_unstale":"2023-11-28T11:08:50.105179+0000","last_undegraded":"2023-11-28T11:08:50.105179+0000","last_fullsized":"2023-11-28T11:08:50.105179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:49.800409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.107754+0000","last_change":"2023-11-28T11:08:45.951273+0000","last_active":"2023-11-28T11:08:50.107754+0000","last_peered":"2023-11-28T11:08:50.107754+0000","last_clean":"2023-11-28T11:08:50.107754+0000","last_became_active":"2023-11-28T11:08:45.950973+0000","last_became_peered":"2023-11-28T11:08:45.950973+0000","last_unstale":"2023-11-28T11:08:50.107754+0000","last_undegraded":"2023-11-28T11:08:50.107754+0000","last_fullsized":"2023-11-28T11:08:50.107754+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:44.385589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571794+0000","last_change":"2023-11-28T11:08:50.129634+0000","last_active":"2023-11-28T11:08:51.571794+0000","last_peered":"2023-11-28T11:08:51.571794+0000","last_clean":"2023-11-28T11:08:51.571794+0000","last_became_active":"2023-11-28T11:08:50.128894+0000","last_became_peered":"2023-11-28T11:08:50.128894+0000","last_unstale":"2023-11-28T11:08:51.571794+0000","last_undegraded":"2023-11-28T11:08:51.571794+0000","last_fullsized":"2023-11-28T11:08:51.571794+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:23:36.162990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1,5],"acting":[3,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"20'3","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115765+0000","last_change":"2023-11-28T11:08:45.520619+0000","last_active":"2023-11-28T11:08:50.115765+0000","last_peered":"2023-11-28T11:08:50.115765+0000","last_clean":"2023-11-28T11:08:50.115765+0000","last_became_active":"2023-11-28T11:08:45.520292+0000","last_became_peered":"2023-11-28T11:08:45.520292+0000","last_unstale":"2023-11-28T11:08:50.115765+0000","last_undegraded":"2023-11-28T11:08:50.115765+0000","last_fullsized":"2023-11-28T11:08:50.115765+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:02.026801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119666+0000","last_change":"2023-11-28T11:08:45.950965+0000","last_active":"2023-11-28T11:08:50.119666+0000","last_peered":"2023-11-28T11:08:50.119666+0000","last_clean":"2023-11-28T11:08:50.119666+0000","last_became_active":"2023-11-28T11:08:45.950660+0000","last_became_peered":"2023-11-28T11:08:45.950660+0000","last_unstale":"2023-11-28T11:08:50.119666+0000","last_undegraded":"2023-11-28T11:08:50.119666+0000","last_fullsized":"2023-11-28T11:08:50.119666+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:34.416718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570510+0000","last_change":"2023-11-28T11:08:50.125734+0000","last_active":"2023-11-28T11:08:51.570510+0000","last_peered":"2023-11-28T11:08:51.570510+0000","last_clean":"2023-11-28T11:08:51.570510+0000","last_became_active":"2023-11-28T11:08:50.125332+0000","last_became_peered":"2023-11-28T11:08:50.125332+0000","last_unstale":"2023-11-28T11:08:51.570510+0000","last_undegraded":"2023-11-28T11:08:51.570510+0000","last_fullsized":"2023-11-28T11:08:51.570510+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:54.778391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3,7],"acting":[1,5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"25'9","reported_seq":23,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:55.258405+0000","last_change":"2023-11-28T11:08:45.519282+0000","last_active":"2023-11-28T11:08:55.258405+0000","last_peered":"2023-11-28T11:08:55.258405+0000","last_clean":"2023-11-28T11:08:55.258405+0000","last_became_active":"2023-11-28T11:08:45.518976+0000","last_became_peered":"2023-11-28T11:08:45.518976+0000","last_unstale":"2023-11-28T11:08:55.258405+0000","last_undegraded":"2023-11-28T11:08:55.258405+0000","last_fullsized":"2023-11-28T11:08:55.258405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:28.586127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3746,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113107+0000","last_change":"2023-11-28T11:08:45.951387+0000","last_active":"2023-11-28T11:08:50.113107+0000","last_peered":"2023-11-28T11:08:50.113107+0000","last_clean":"2023-11-28T11:08:50.113107+0000","last_became_active":"2023-11-28T11:08:45.951104+0000","last_became_peered":"2023-11-28T11:08:45.951104+0000","last_unstale":"2023-11-28T11:08:50.113107+0000","last_undegraded":"2023-11-28T11:08:50.113107+0000","last_fullsized":"2023-11-28T11:08:50.113107+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:19:48.658152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537972+0000","last_change":"2023-11-28T11:08:50.124789+0000","last_active":"2023-11-28T11:08:50.537972+0000","last_peered":"2023-11-28T11:08:50.537972+0000","last_clean":"2023-11-28T11:08:50.537972+0000","last_became_active":"2023-11-28T11:08:50.123561+0000","last_became_peered":"2023-11-28T11:08:50.123561+0000","last_unstale":"2023-11-28T11:08:50.537972+0000","last_undegraded":"2023-11-28T11:08:50.537972+0000","last_fullsized":"2023-11-28T11:08:50.537972+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:29.317492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3,0],"acting":[5,1,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115695+0000","last_change":"2023-11-28T11:08:45.518953+0000","last_active":"2023-11-28T11:08:50.115695+0000","last_peered":"2023-11-28T11:08:50.115695+0000","last_clean":"2023-11-28T11:08:50.115695+0000","last_became_active":"2023-11-28T11:08:45.518679+0000","last_became_peered":"2023-11-28T11:08:45.518679+0000","last_unstale":"2023-11-28T11:08:50.115695+0000","last_undegraded":"2023-11-28T11:08:50.115695+0000","last_fullsized":"2023-11-28T11:08:50.115695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:54.080163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108292+0000","last_change":"2023-11-28T11:08:45.945377+0000","last_active":"2023-11-28T11:08:50.108292+0000","last_peered":"2023-11-28T11:08:50.108292+0000","last_clean":"2023-11-28T11:08:50.108292+0000","last_became_active":"2023-11-28T11:08:45.945104+0000","last_became_peered":"2023-11-28T11:08:45.945104+0000","last_unstale":"2023-11-28T11:08:50.108292+0000","last_undegraded":"2023-11-28T11:08:50.108292+0000","last_fullsized":"2023-11-28T11:08:50.108292+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:38:21.481269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.572169+0000","last_change":"2023-11-28T11:08:50.130966+0000","last_active":"2023-11-28T11:08:51.572169+0000","last_peered":"2023-11-28T11:08:51.572169+0000","last_clean":"2023-11-28T11:08:51.572169+0000","last_became_active":"2023-11-28T11:08:50.130573+0000","last_became_peered":"2023-11-28T11:08:50.130573+0000","last_unstale":"2023-11-28T11:08:51.572169+0000","last_undegraded":"2023-11-28T11:08:51.572169+0000","last_fullsized":"2023-11-28T11:08:51.572169+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:59.602687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2,1],"acting":[3,7,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103540+0000","last_change":"2023-11-28T11:08:45.521275+0000","last_active":"2023-11-28T11:08:50.103540+0000","last_peered":"2023-11-28T11:08:50.103540+0000","last_clean":"2023-11-28T11:08:50.103540+0000","last_became_active":"2023-11-28T11:08:45.520908+0000","last_became_peered":"2023-11-28T11:08:45.520908+0000","last_unstale":"2023-11-28T11:08:50.103540+0000","last_undegraded":"2023-11-28T11:08:50.103540+0000","last_fullsized":"2023-11-28T11:08:50.103540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:33.188220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105967+0000","last_change":"2023-11-28T11:08:45.957604+0000","last_active":"2023-11-28T11:08:50.105967+0000","last_peered":"2023-11-28T11:08:50.105967+0000","last_clean":"2023-11-28T11:08:50.105967+0000","last_became_active":"2023-11-28T11:08:45.957135+0000","last_became_peered":"2023-11-28T11:08:45.957135+0000","last_unstale":"2023-11-28T11:08:50.105967+0000","last_undegraded":"2023-11-28T11:08:50.105967+0000","last_fullsized":"2023-11-28T11:08:50.105967+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.572546+0000","last_change":"2023-11-28T11:08:50.129359+0000","last_active":"2023-11-28T11:08:51.572546+0000","last_peered":"2023-11-28T11:08:51.572546+0000","last_clean":"2023-11-28T11:08:51.572546+0000","last_became_active":"2023-11-28T11:08:50.128928+0000","last_became_peered":"2023-11-28T11:08:50.128928+0000","last_unstale":"2023-11-28T11:08:51.572546+0000","last_undegraded":"2023-11-28T11:08:51.572546+0000","last_fullsized":"2023-11-28T11:08:51.572546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:24.842089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7,0],"acting":[3,1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115717+0000","last_change":"2023-11-28T11:08:45.517068+0000","last_active":"2023-11-28T11:08:50.115717+0000","last_peered":"2023-11-28T11:08:50.115717+0000","last_clean":"2023-11-28T11:08:50.115717+0000","last_became_active":"2023-11-28T11:08:45.516742+0000","last_became_peered":"2023-11-28T11:08:45.516742+0000","last_unstale":"2023-11-28T11:08:50.115717+0000","last_undegraded":"2023-11-28T11:08:50.115717+0000","last_fullsized":"2023-11-28T11:08:50.115717+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:00.442293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.104172+0000","last_change":"2023-11-28T11:08:45.941624+0000","last_active":"2023-11-28T11:08:50.104172+0000","last_peered":"2023-11-28T11:08:50.104172+0000","last_clean":"2023-11-28T11:08:50.104172+0000","last_became_active":"2023-11-28T11:08:45.941042+0000","last_became_peered":"2023-11-28T11:08:45.941042+0000","last_unstale":"2023-11-28T11:08:50.104172+0000","last_undegraded":"2023-11-28T11:08:50.104172+0000","last_fullsized":"2023-11-28T11:08:50.104172+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:39.158725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537699+0000","last_change":"2023-11-28T11:08:50.121180+0000","last_active":"2023-11-28T11:08:50.537699+0000","last_peered":"2023-11-28T11:08:50.537699+0000","last_clean":"2023-11-28T11:08:50.537699+0000","last_became_active":"2023-11-28T11:08:50.120757+0000","last_became_peered":"2023-11-28T11:08:50.120757+0000","last_unstale":"2023-11-28T11:08:50.537699+0000","last_undegraded":"2023-11-28T11:08:50.537699+0000","last_fullsized":"2023-11-28T11:08:50.537699+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:20.989038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1,6],"acting":[5,3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.10","version":"20'8","reported_seq":22,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108439+0000","last_change":"2023-11-28T11:08:45.516801+0000","last_active":"2023-11-28T11:08:50.108439+0000","last_peered":"2023-11-28T11:08:50.108439+0000","last_clean":"2023-11-28T11:08:50.108439+0000","last_became_active":"2023-11-28T11:08:45.516457+0000","last_became_peered":"2023-11-28T11:08:45.516457+0000","last_unstale":"2023-11-28T11:08:50.108439+0000","last_undegraded":"2023-11-28T11:08:50.108439+0000","last_fullsized":"2023-11-28T11:08:50.108439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.111359+0000","last_change":"2023-11-28T11:08:45.956905+0000","last_active":"2023-11-28T11:08:50.111359+0000","last_peered":"2023-11-28T11:08:50.111359+0000","last_clean":"2023-11-28T11:08:50.111359+0000","last_became_active":"2023-11-28T11:08:45.956698+0000","last_became_peered":"2023-11-28T11:08:45.956698+0000","last_unstale":"2023-11-28T11:08:50.111359+0000","last_undegraded":"2023-11-28T11:08:50.111359+0000","last_fullsized":"2023-11-28T11:08:50.111359+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:43:23.627932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570657+0000","last_change":"2023-11-28T11:08:50.130733+0000","last_active":"2023-11-28T11:08:51.570657+0000","last_peered":"2023-11-28T11:08:51.570657+0000","last_clean":"2023-11-28T11:08:51.570657+0000","last_became_active":"2023-11-28T11:08:50.130213+0000","last_became_peered":"2023-11-28T11:08:50.130213+0000","last_unstale":"2023-11-28T11:08:51.570657+0000","last_undegraded":"2023-11-28T11:08:51.570657+0000","last_fullsized":"2023-11-28T11:08:51.570657+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:36.656033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3,4],"acting":[7,6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.f","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.116413+0000","last_change":"2023-11-28T11:08:45.516890+0000","last_active":"2023-11-28T11:08:50.116413+0000","last_peered":"2023-11-28T11:08:50.116413+0000","last_clean":"2023-11-28T11:08:50.116413+0000","last_became_active":"2023-11-28T11:08:45.516595+0000","last_became_peered":"2023-11-28T11:08:45.516595+0000","last_unstale":"2023-11-28T11:08:50.116413+0000","last_undegraded":"2023-11-28T11:08:50.116413+0000","last_fullsized":"2023-11-28T11:08:50.116413+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:58:30.168075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105047+0000","last_change":"2023-11-28T11:08:45.956070+0000","last_active":"2023-11-28T11:08:50.105047+0000","last_peered":"2023-11-28T11:08:50.105047+0000","last_clean":"2023-11-28T11:08:50.105047+0000","last_became_active":"2023-11-28T11:08:45.955774+0000","last_became_peered":"2023-11-28T11:08:45.955774+0000","last_unstale":"2023-11-28T11:08:50.105047+0000","last_undegraded":"2023-11-28T11:08:50.105047+0000","last_fullsized":"2023-11-28T11:08:50.105047+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571154+0000","last_change":"2023-11-28T11:08:50.132292+0000","last_active":"2023-11-28T11:08:51.571154+0000","last_peered":"2023-11-28T11:08:51.571154+0000","last_clean":"2023-11-28T11:08:51.571154+0000","last_became_active":"2023-11-28T11:08:50.131887+0000","last_became_peered":"2023-11-28T11:08:50.131887+0000","last_unstale":"2023-11-28T11:08:51.571154+0000","last_undegraded":"2023-11-28T11:08:51.571154+0000","last_fullsized":"2023-11-28T11:08:51.571154+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:08.804864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1,5],"acting":[2,0,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.e","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115394+0000","last_change":"2023-11-28T11:08:45.516041+0000","last_active":"2023-11-28T11:08:50.115394+0000","last_peered":"2023-11-28T11:08:50.115394+0000","last_clean":"2023-11-28T11:08:50.115394+0000","last_became_active":"2023-11-28T11:08:45.515742+0000","last_became_peered":"2023-11-28T11:08:45.515742+0000","last_unstale":"2023-11-28T11:08:50.115394+0000","last_undegraded":"2023-11-28T11:08:50.115394+0000","last_fullsized":"2023-11-28T11:08:50.115394+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:14.983939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113365+0000","last_change":"2023-11-28T11:08:45.945858+0000","last_active":"2023-11-28T11:08:50.113365+0000","last_peered":"2023-11-28T11:08:50.113365+0000","last_clean":"2023-11-28T11:08:50.113365+0000","last_became_active":"2023-11-28T11:08:45.945552+0000","last_became_peered":"2023-11-28T11:08:45.945552+0000","last_unstale":"2023-11-28T11:08:50.113365+0000","last_undegraded":"2023-11-28T11:08:50.113365+0000","last_fullsized":"2023-11-28T11:08:50.113365+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571022+0000","last_change":"2023-11-28T11:08:50.123713+0000","last_active":"2023-11-28T11:08:51.571022+0000","last_peered":"2023-11-28T11:08:51.571022+0000","last_clean":"2023-11-28T11:08:51.571022+0000","last_became_active":"2023-11-28T11:08:50.123264+0000","last_became_peered":"2023-11-28T11:08:50.123264+0000","last_unstale":"2023-11-28T11:08:51.571022+0000","last_undegraded":"2023-11-28T11:08:51.571022+0000","last_fullsized":"2023-11-28T11:08:51.571022+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:04.199211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5,6],"acting":[2,0,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115481+0000","last_change":"2023-11-28T11:08:45.515894+0000","last_active":"2023-11-28T11:08:50.115481+0000","last_peered":"2023-11-28T11:08:50.115481+0000","last_clean":"2023-11-28T11:08:50.115481+0000","last_became_active":"2023-11-28T11:08:45.515593+0000","last_became_peered":"2023-11-28T11:08:45.515593+0000","last_unstale":"2023-11-28T11:08:50.115481+0000","last_undegraded":"2023-11-28T11:08:50.115481+0000","last_fullsized":"2023-11-28T11:08:50.115481+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:31.747218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109629+0000","last_change":"2023-11-28T11:08:45.949548+0000","last_active":"2023-11-28T11:08:50.109629+0000","last_peered":"2023-11-28T11:08:50.109629+0000","last_clean":"2023-11-28T11:08:50.109629+0000","last_became_active":"2023-11-28T11:08:45.949217+0000","last_became_peered":"2023-11-28T11:08:45.949217+0000","last_unstale":"2023-11-28T11:08:50.109629+0000","last_undegraded":"2023-11-28T11:08:50.109629+0000","last_fullsized":"2023-11-28T11:08:50.109629+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:01.447773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571373+0000","last_change":"2023-11-28T11:08:50.127125+0000","last_active":"2023-11-28T11:08:51.571373+0000","last_peered":"2023-11-28T11:08:51.571373+0000","last_clean":"2023-11-28T11:08:51.571373+0000","last_became_active":"2023-11-28T11:08:50.124511+0000","last_became_peered":"2023-11-28T11:08:50.124511+0000","last_unstale":"2023-11-28T11:08:51.571373+0000","last_undegraded":"2023-11-28T11:08:51.571373+0000","last_fullsized":"2023-11-28T11:08:51.571373+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:47.985900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3,6],"acting":[2,4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105609+0000","last_change":"2023-11-28T11:08:45.513992+0000","last_active":"2023-11-28T11:08:50.105609+0000","last_peered":"2023-11-28T11:08:50.105609+0000","last_clean":"2023-11-28T11:08:50.105609+0000","last_became_active":"2023-11-28T11:08:45.513697+0000","last_became_peered":"2023-11-28T11:08:45.513697+0000","last_unstale":"2023-11-28T11:08:50.105609+0000","last_undegraded":"2023-11-28T11:08:50.105609+0000","last_fullsized":"2023-11-28T11:08:50.105609+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:28.586127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103999+0000","last_change":"2023-11-28T11:08:45.940813+0000","last_active":"2023-11-28T11:08:50.103999+0000","last_peered":"2023-11-28T11:08:50.103999+0000","last_clean":"2023-11-28T11:08:50.103999+0000","last_became_active":"2023-11-28T11:08:45.940552+0000","last_became_peered":"2023-11-28T11:08:45.940552+0000","last_unstale":"2023-11-28T11:08:50.103999+0000","last_undegraded":"2023-11-28T11:08:50.103999+0000","last_fullsized":"2023-11-28T11:08:50.103999+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:17.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571495+0000","last_change":"2023-11-28T11:08:50.121770+0000","last_active":"2023-11-28T11:08:51.571495+0000","last_peered":"2023-11-28T11:08:51.571495+0000","last_clean":"2023-11-28T11:08:51.571495+0000","last_became_active":"2023-11-28T11:08:50.121277+0000","last_became_peered":"2023-11-28T11:08:50.121277+0000","last_unstale":"2023-11-28T11:08:51.571495+0000","last_undegraded":"2023-11-28T11:08:51.571495+0000","last_fullsized":"2023-11-28T11:08:51.571495+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:42.759036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5,6],"acting":[2,7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109255+0000","last_change":"2023-11-28T11:08:45.519824+0000","last_active":"2023-11-28T11:08:50.109255+0000","last_peered":"2023-11-28T11:08:50.109255+0000","last_clean":"2023-11-28T11:08:50.109255+0000","last_became_active":"2023-11-28T11:08:45.519241+0000","last_became_peered":"2023-11-28T11:08:45.519241+0000","last_unstale":"2023-11-28T11:08:50.109255+0000","last_undegraded":"2023-11-28T11:08:50.109255+0000","last_fullsized":"2023-11-28T11:08:50.109255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:28:31.747218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112858+0000","last_change":"2023-11-28T11:08:45.946401+0000","last_active":"2023-11-28T11:08:50.112858+0000","last_peered":"2023-11-28T11:08:50.112858+0000","last_clean":"2023-11-28T11:08:50.112858+0000","last_became_active":"2023-11-28T11:08:45.946070+0000","last_became_peered":"2023-11-28T11:08:45.946070+0000","last_unstale":"2023-11-28T11:08:50.112858+0000","last_undegraded":"2023-11-28T11:08:50.112858+0000","last_fullsized":"2023-11-28T11:08:50.112858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:05.204553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570768+0000","last_change":"2023-11-28T11:08:50.131808+0000","last_active":"2023-11-28T11:08:51.570768+0000","last_peered":"2023-11-28T11:08:51.570768+0000","last_clean":"2023-11-28T11:08:51.570768+0000","last_became_active":"2023-11-28T11:08:50.131311+0000","last_became_peered":"2023-11-28T11:08:50.131311+0000","last_unstale":"2023-11-28T11:08:51.570768+0000","last_undegraded":"2023-11-28T11:08:51.570768+0000","last_fullsized":"2023-11-28T11:08:51.570768+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:47:06.157479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0,6],"acting":[1,4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109818+0000","last_change":"2023-11-28T11:08:45.517239+0000","last_active":"2023-11-28T11:08:50.109818+0000","last_peered":"2023-11-28T11:08:50.109818+0000","last_clean":"2023-11-28T11:08:50.109818+0000","last_became_active":"2023-11-28T11:08:45.516694+0000","last_became_peered":"2023-11-28T11:08:45.516694+0000","last_unstale":"2023-11-28T11:08:50.109818+0000","last_undegraded":"2023-11-28T11:08:50.109818+0000","last_fullsized":"2023-11-28T11:08:50.109818+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113094+0000","last_change":"2023-11-28T11:08:45.947974+0000","last_active":"2023-11-28T11:08:50.113094+0000","last_peered":"2023-11-28T11:08:50.113094+0000","last_clean":"2023-11-28T11:08:50.113094+0000","last_became_active":"2023-11-28T11:08:45.947674+0000","last_became_peered":"2023-11-28T11:08:45.947674+0000","last_unstale":"2023-11-28T11:08:50.113094+0000","last_undegraded":"2023-11-28T11:08:50.113094+0000","last_fullsized":"2023-11-28T11:08:50.113094+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:56:44.385589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.538066+0000","last_change":"2023-11-28T11:08:50.126205+0000","last_active":"2023-11-28T11:08:50.538066+0000","last_peered":"2023-11-28T11:08:50.538066+0000","last_clean":"2023-11-28T11:08:50.538066+0000","last_became_active":"2023-11-28T11:08:50.125446+0000","last_became_peered":"2023-11-28T11:08:50.125446+0000","last_unstale":"2023-11-28T11:08:50.538066+0000","last_undegraded":"2023-11-28T11:08:50.538066+0000","last_fullsized":"2023-11-28T11:08:50.538066+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:40:59.602687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6,2],"acting":[5,4,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113146+0000","last_change":"2023-11-28T11:08:45.512742+0000","last_active":"2023-11-28T11:08:50.113146+0000","last_peered":"2023-11-28T11:08:50.113146+0000","last_clean":"2023-11-28T11:08:50.113146+0000","last_became_active":"2023-11-28T11:08:45.512446+0000","last_became_peered":"2023-11-28T11:08:45.512446+0000","last_unstale":"2023-11-28T11:08:50.113146+0000","last_undegraded":"2023-11-28T11:08:50.113146+0000","last_fullsized":"2023-11-28T11:08:50.113146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113682+0000","last_change":"2023-11-28T11:08:45.955541+0000","last_active":"2023-11-28T11:08:50.113682+0000","last_peered":"2023-11-28T11:08:50.113682+0000","last_clean":"2023-11-28T11:08:50.113682+0000","last_became_active":"2023-11-28T11:08:45.955223+0000","last_became_peered":"2023-11-28T11:08:45.955223+0000","last_unstale":"2023-11-28T11:08:50.113682+0000","last_undegraded":"2023-11-28T11:08:50.113682+0000","last_fullsized":"2023-11-28T11:08:50.113682+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:24:04.835870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.536215+0000","last_change":"2023-11-28T11:08:50.134010+0000","last_active":"2023-11-28T11:08:50.536215+0000","last_peered":"2023-11-28T11:08:50.536215+0000","last_clean":"2023-11-28T11:08:50.536215+0000","last_became_active":"2023-11-28T11:08:50.133519+0000","last_became_peered":"2023-11-28T11:08:50.133519+0000","last_unstale":"2023-11-28T11:08:50.536215+0000","last_undegraded":"2023-11-28T11:08:50.536215+0000","last_fullsized":"2023-11-28T11:08:50.536215+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:41:47.676934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0,2],"acting":[4,5,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109102+0000","last_change":"2023-11-28T11:08:45.515039+0000","last_active":"2023-11-28T11:08:50.109102+0000","last_peered":"2023-11-28T11:08:50.109102+0000","last_clean":"2023-11-28T11:08:50.109102+0000","last_became_active":"2023-11-28T11:08:45.514380+0000","last_became_peered":"2023-11-28T11:08:45.514380+0000","last_unstale":"2023-11-28T11:08:50.109102+0000","last_undegraded":"2023-11-28T11:08:50.109102+0000","last_fullsized":"2023-11-28T11:08:50.109102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:28.586127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120542+0000","last_change":"2023-11-28T11:08:45.945761+0000","last_active":"2023-11-28T11:08:50.120542+0000","last_peered":"2023-11-28T11:08:50.120542+0000","last_clean":"2023-11-28T11:08:50.120542+0000","last_became_active":"2023-11-28T11:08:45.945501+0000","last_became_peered":"2023-11-28T11:08:45.945501+0000","last_unstale":"2023-11-28T11:08:50.120542+0000","last_undegraded":"2023-11-28T11:08:50.120542+0000","last_fullsized":"2023-11-28T11:08:50.120542+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:36:30.639887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.535821+0000","last_change":"2023-11-28T11:08:50.129233+0000","last_active":"2023-11-28T11:08:50.535821+0000","last_peered":"2023-11-28T11:08:50.535821+0000","last_clean":"2023-11-28T11:08:50.535821+0000","last_became_active":"2023-11-28T11:08:50.128785+0000","last_became_peered":"2023-11-28T11:08:50.128785+0000","last_unstale":"2023-11-28T11:08:50.535821+0000","last_undegraded":"2023-11-28T11:08:50.535821+0000","last_fullsized":"2023-11-28T11:08:50.535821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:45:14.088471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7,0],"acting":[4,3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"20'2","reported_seq":16,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.111193+0000","last_change":"2023-11-28T11:08:45.519399+0000","last_active":"2023-11-28T11:08:50.111193+0000","last_peered":"2023-11-28T11:08:50.111193+0000","last_clean":"2023-11-28T11:08:50.111193+0000","last_became_active":"2023-11-28T11:08:45.519141+0000","last_became_peered":"2023-11-28T11:08:45.519141+0000","last_unstale":"2023-11-28T11:08:50.111193+0000","last_undegraded":"2023-11-28T11:08:50.111193+0000","last_fullsized":"2023-11-28T11:08:50.111193+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:24.093818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113185+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:08:50.113185+0000","last_peered":"2023-11-28T11:08:50.113185+0000","last_clean":"2023-11-28T11:08:50.113185+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:08:50.113185+0000","last_undegraded":"2023-11-28T11:08:50.113185+0000","last_fullsized":"2023-11-28T11:08:50.113185+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109007+0000","last_change":"2023-11-28T11:08:45.955853+0000","last_active":"2023-11-28T11:08:50.109007+0000","last_peered":"2023-11-28T11:08:50.109007+0000","last_clean":"2023-11-28T11:08:50.109007+0000","last_became_active":"2023-11-28T11:08:45.955563+0000","last_became_peered":"2023-11-28T11:08:45.955563+0000","last_unstale":"2023-11-28T11:08:50.109007+0000","last_undegraded":"2023-11-28T11:08:50.109007+0000","last_fullsized":"2023-11-28T11:08:50.109007+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:00.598900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569792+0000","last_change":"2023-11-28T11:08:50.129450+0000","last_active":"2023-11-28T11:08:51.569792+0000","last_peered":"2023-11-28T11:08:51.569792+0000","last_clean":"2023-11-28T11:08:51.569792+0000","last_became_active":"2023-11-28T11:08:50.128981+0000","last_became_peered":"2023-11-28T11:08:50.128981+0000","last_unstale":"2023-11-28T11:08:51.569792+0000","last_undegraded":"2023-11-28T11:08:51.569792+0000","last_fullsized":"2023-11-28T11:08:51.569792+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:15:14.466494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4,0],"acting":[3,1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108203+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:08:50.108203+0000","last_peered":"2023-11-28T11:08:50.108203+0000","last_clean":"2023-11-28T11:08:50.108203+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:08:50.108203+0000","last_undegraded":"2023-11-28T11:08:50.108203+0000","last_fullsized":"2023-11-28T11:08:50.108203+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"24'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.513023+0000","last_change":"2023-11-28T11:08:45.522154+0000","last_active":"2023-11-28T11:08:50.513023+0000","last_peered":"2023-11-28T11:08:50.513023+0000","last_clean":"2023-11-28T11:08:50.513023+0000","last_became_active":"2023-11-28T11:08:45.518652+0000","last_became_peered":"2023-11-28T11:08:45.518652+0000","last_unstale":"2023-11-28T11:08:50.513023+0000","last_undegraded":"2023-11-28T11:08:50.513023+0000","last_fullsized":"2023-11-28T11:08:50.513023+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:49.800409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.113254+0000","last_change":"2023-11-28T11:08:45.946786+0000","last_active":"2023-11-28T11:08:50.113254+0000","last_peered":"2023-11-28T11:08:50.113254+0000","last_clean":"2023-11-28T11:08:50.113254+0000","last_became_active":"2023-11-28T11:08:45.946446+0000","last_became_peered":"2023-11-28T11:08:45.946446+0000","last_unstale":"2023-11-28T11:08:50.113254+0000","last_undegraded":"2023-11-28T11:08:50.113254+0000","last_fullsized":"2023-11-28T11:08:50.113254+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:39.158725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537975+0000","last_change":"2023-11-28T11:08:50.123220+0000","last_active":"2023-11-28T11:08:50.537975+0000","last_peered":"2023-11-28T11:08:50.537975+0000","last_clean":"2023-11-28T11:08:50.537975+0000","last_became_active":"2023-11-28T11:08:50.122776+0000","last_became_peered":"2023-11-28T11:08:50.122776+0000","last_unstale":"2023-11-28T11:08:50.537975+0000","last_undegraded":"2023-11-28T11:08:50.537975+0000","last_fullsized":"2023-11-28T11:08:50.537975+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:35.176626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0,4],"acting":[5,1,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119767+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:08:50.119767+0000","last_peered":"2023-11-28T11:08:50.119767+0000","last_clean":"2023-11-28T11:08:50.119767+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:08:50.119767+0000","last_undegraded":"2023-11-28T11:08:50.119767+0000","last_fullsized":"2023-11-28T11:08:50.119767+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":167,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115593+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:08:50.115593+0000","last_peered":"2023-11-28T11:08:50.115593+0000","last_clean":"2023-11-28T11:08:50.115593+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:08:50.115593+0000","last_undegraded":"2023-11-28T11:08:50.115593+0000","last_fullsized":"2023-11-28T11:08:50.115593+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103405+0000","last_change":"2023-11-28T11:08:45.521876+0000","last_active":"2023-11-28T11:08:50.103405+0000","last_peered":"2023-11-28T11:08:50.103405+0000","last_clean":"2023-11-28T11:08:50.103405+0000","last_became_active":"2023-11-28T11:08:45.521580+0000","last_became_peered":"2023-11-28T11:08:45.521580+0000","last_unstale":"2023-11-28T11:08:50.103405+0000","last_undegraded":"2023-11-28T11:08:50.103405+0000","last_fullsized":"2023-11-28T11:08:50.103405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.104071+0000","last_change":"2023-11-28T11:08:45.948040+0000","last_active":"2023-11-28T11:08:50.104071+0000","last_peered":"2023-11-28T11:08:50.104071+0000","last_clean":"2023-11-28T11:08:50.104071+0000","last_became_active":"2023-11-28T11:08:45.947704+0000","last_became_peered":"2023-11-28T11:08:45.947704+0000","last_unstale":"2023-11-28T11:08:50.104071+0000","last_undegraded":"2023-11-28T11:08:50.104071+0000","last_fullsized":"2023-11-28T11:08:50.104071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:01:35.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.571808+0000","last_change":"2023-11-28T11:08:50.124074+0000","last_active":"2023-11-28T11:08:51.571808+0000","last_peered":"2023-11-28T11:08:51.571808+0000","last_clean":"2023-11-28T11:08:51.571808+0000","last_became_active":"2023-11-28T11:08:50.123630+0000","last_became_peered":"2023-11-28T11:08:50.123630+0000","last_unstale":"2023-11-28T11:08:51.571808+0000","last_undegraded":"2023-11-28T11:08:51.571808+0000","last_fullsized":"2023-11-28T11:08:51.571808+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:52.786613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,1,7],"acting":[2,5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115788+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:08:50.115788+0000","last_peered":"2023-11-28T11:08:50.115788+0000","last_clean":"2023-11-28T11:08:50.115788+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:08:50.115788+0000","last_undegraded":"2023-11-28T11:08:50.115788+0000","last_fullsized":"2023-11-28T11:08:50.115788+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119464+0000","last_change":"2023-11-28T11:08:45.510984+0000","last_active":"2023-11-28T11:08:50.119464+0000","last_peered":"2023-11-28T11:08:50.119464+0000","last_clean":"2023-11-28T11:08:50.119464+0000","last_became_active":"2023-11-28T11:08:45.510734+0000","last_became_peered":"2023-11-28T11:08:45.510734+0000","last_unstale":"2023-11-28T11:08:50.119464+0000","last_undegraded":"2023-11-28T11:08:50.119464+0000","last_fullsized":"2023-11-28T11:08:50.119464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:33.188220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120521+0000","last_change":"2023-11-28T11:08:45.946753+0000","last_active":"2023-11-28T11:08:50.120521+0000","last_peered":"2023-11-28T11:08:50.120521+0000","last_clean":"2023-11-28T11:08:50.120521+0000","last_became_active":"2023-11-28T11:08:45.946460+0000","last_became_peered":"2023-11-28T11:08:45.946460+0000","last_unstale":"2023-11-28T11:08:50.120521+0000","last_undegraded":"2023-11-28T11:08:50.120521+0000","last_fullsized":"2023-11-28T11:08:50.120521+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:20:10.730266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570567+0000","last_change":"2023-11-28T11:08:50.132450+0000","last_active":"2023-11-28T11:08:51.570567+0000","last_peered":"2023-11-28T11:08:51.570567+0000","last_clean":"2023-11-28T11:08:51.570567+0000","last_became_active":"2023-11-28T11:08:50.132043+0000","last_became_peered":"2023-11-28T11:08:50.132043+0000","last_unstale":"2023-11-28T11:08:51.570567+0000","last_undegraded":"2023-11-28T11:08:51.570567+0000","last_fullsized":"2023-11-28T11:08:51.570567+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:32:20.989038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5,3],"acting":[7,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.115361+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:08:50.115361+0000","last_peered":"2023-11-28T11:08:50.115361+0000","last_clean":"2023-11-28T11:08:50.115361+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:08:50.115361+0000","last_undegraded":"2023-11-28T11:08:50.115361+0000","last_fullsized":"2023-11-28T11:08:50.115361+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108211+0000","last_change":"2023-11-28T11:08:45.513181+0000","last_active":"2023-11-28T11:08:50.108211+0000","last_peered":"2023-11-28T11:08:50.108211+0000","last_clean":"2023-11-28T11:08:50.108211+0000","last_became_active":"2023-11-28T11:08:45.512903+0000","last_became_peered":"2023-11-28T11:08:45.512903+0000","last_unstale":"2023-11-28T11:08:50.108211+0000","last_undegraded":"2023-11-28T11:08:50.108211+0000","last_fullsized":"2023-11-28T11:08:50.108211+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109406+0000","last_change":"2023-11-28T11:08:45.947505+0000","last_active":"2023-11-28T11:08:50.109406+0000","last_peered":"2023-11-28T11:08:50.109406+0000","last_clean":"2023-11-28T11:08:50.109406+0000","last_became_active":"2023-11-28T11:08:45.947199+0000","last_became_peered":"2023-11-28T11:08:45.947199+0000","last_unstale":"2023-11-28T11:08:50.109406+0000","last_undegraded":"2023-11-28T11:08:50.109406+0000","last_fullsized":"2023-11-28T11:08:50.109406+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:55.085644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.437473+0000","last_change":"2023-11-28T11:08:50.126736+0000","last_active":"2023-11-28T11:08:51.437473+0000","last_peered":"2023-11-28T11:08:51.437473+0000","last_clean":"2023-11-28T11:08:51.437473+0000","last_became_active":"2023-11-28T11:08:50.126282+0000","last_became_peered":"2023-11-28T11:08:50.126282+0000","last_unstale":"2023-11-28T11:08:51.437473+0000","last_undegraded":"2023-11-28T11:08:51.437473+0000","last_fullsized":"2023-11-28T11:08:51.437473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:10:36.656033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4,6],"acting":[0,1,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.104943+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:08:50.104943+0000","last_peered":"2023-11-28T11:08:50.104943+0000","last_clean":"2023-11-28T11:08:50.104943+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:08:50.104943+0000","last_undegraded":"2023-11-28T11:08:50.104943+0000","last_fullsized":"2023-11-28T11:08:50.104943+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112806+0000","last_change":"2023-11-28T11:08:45.524531+0000","last_active":"2023-11-28T11:08:50.112806+0000","last_peered":"2023-11-28T11:08:50.112806+0000","last_clean":"2023-11-28T11:08:50.112806+0000","last_became_active":"2023-11-28T11:08:45.524334+0000","last_became_peered":"2023-11-28T11:08:45.524334+0000","last_unstale":"2023-11-28T11:08:50.112806+0000","last_undegraded":"2023-11-28T11:08:50.112806+0000","last_fullsized":"2023-11-28T11:08:50.112806+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119778+0000","last_change":"2023-11-28T11:08:45.944621+0000","last_active":"2023-11-28T11:08:50.119778+0000","last_peered":"2023-11-28T11:08:50.119778+0000","last_clean":"2023-11-28T11:08:50.119778+0000","last_became_active":"2023-11-28T11:08:45.944368+0000","last_became_peered":"2023-11-28T11:08:45.944368+0000","last_unstale":"2023-11-28T11:08:50.119778+0000","last_undegraded":"2023-11-28T11:08:50.119778+0000","last_fullsized":"2023-11-28T11:08:50.119778+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:01.447773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570969+0000","last_change":"2023-11-28T11:08:50.131902+0000","last_active":"2023-11-28T11:08:51.570969+0000","last_peered":"2023-11-28T11:08:51.570969+0000","last_clean":"2023-11-28T11:08:51.570969+0000","last_became_active":"2023-11-28T11:08:50.131485+0000","last_became_peered":"2023-11-28T11:08:50.131485+0000","last_unstale":"2023-11-28T11:08:51.570969+0000","last_undegraded":"2023-11-28T11:08:51.570969+0000","last_fullsized":"2023-11-28T11:08:51.570969+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:41:08.804864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5,2],"acting":[6,0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110005+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:08:50.110005+0000","last_peered":"2023-11-28T11:08:50.110005+0000","last_clean":"2023-11-28T11:08:50.110005+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:08:50.110005+0000","last_undegraded":"2023-11-28T11:08:50.110005+0000","last_fullsized":"2023-11-28T11:08:50.110005+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.120247+0000","last_change":"2023-11-28T11:08:45.514598+0000","last_active":"2023-11-28T11:08:50.120247+0000","last_peered":"2023-11-28T11:08:50.120247+0000","last_clean":"2023-11-28T11:08:50.120247+0000","last_became_active":"2023-11-28T11:08:45.514348+0000","last_became_peered":"2023-11-28T11:08:45.514348+0000","last_unstale":"2023-11-28T11:08:50.120247+0000","last_undegraded":"2023-11-28T11:08:50.120247+0000","last_fullsized":"2023-11-28T11:08:50.120247+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:47:27.939003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103691+0000","last_change":"2023-11-28T11:08:45.950031+0000","last_active":"2023-11-28T11:08:50.103691+0000","last_peered":"2023-11-28T11:08:50.103691+0000","last_clean":"2023-11-28T11:08:50.103691+0000","last_became_active":"2023-11-28T11:08:45.949457+0000","last_became_peered":"2023-11-28T11:08:45.949457+0000","last_unstale":"2023-11-28T11:08:50.103691+0000","last_undegraded":"2023-11-28T11:08:50.103691+0000","last_fullsized":"2023-11-28T11:08:50.103691+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:34.416718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.437351+0000","last_change":"2023-11-28T11:08:50.124074+0000","last_active":"2023-11-28T11:08:51.437351+0000","last_peered":"2023-11-28T11:08:51.437351+0000","last_clean":"2023-11-28T11:08:51.437351+0000","last_became_active":"2023-11-28T11:08:50.123710+0000","last_became_peered":"2023-11-28T11:08:50.123710+0000","last_unstale":"2023-11-28T11:08:51.437351+0000","last_undegraded":"2023-11-28T11:08:51.437351+0000","last_fullsized":"2023-11-28T11:08:51.437351+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:42:35.306388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,6,5],"acting":[0,1,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.110239+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:08:50.110239+0000","last_peered":"2023-11-28T11:08:50.110239+0000","last_clean":"2023-11-28T11:08:50.110239+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:08:50.110239+0000","last_undegraded":"2023-11-28T11:08:50.110239+0000","last_fullsized":"2023-11-28T11:08:50.110239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105883+0000","last_change":"2023-11-28T11:08:45.514165+0000","last_active":"2023-11-28T11:08:50.105883+0000","last_peered":"2023-11-28T11:08:50.105883+0000","last_clean":"2023-11-28T11:08:50.105883+0000","last_became_active":"2023-11-28T11:08:45.513651+0000","last_became_peered":"2023-11-28T11:08:45.513651+0000","last_unstale":"2023-11-28T11:08:50.105883+0000","last_undegraded":"2023-11-28T11:08:50.105883+0000","last_fullsized":"2023-11-28T11:08:50.105883+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:12:47.005332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119668+0000","last_change":"2023-11-28T11:08:45.949610+0000","last_active":"2023-11-28T11:08:50.119668+0000","last_peered":"2023-11-28T11:08:50.119668+0000","last_clean":"2023-11-28T11:08:50.119668+0000","last_became_active":"2023-11-28T11:08:45.949348+0000","last_became_peered":"2023-11-28T11:08:45.949348+0000","last_unstale":"2023-11-28T11:08:50.119668+0000","last_undegraded":"2023-11-28T11:08:50.119668+0000","last_fullsized":"2023-11-28T11:08:50.119668+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:23:53.819338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570113+0000","last_change":"2023-11-28T11:08:50.135779+0000","last_active":"2023-11-28T11:08:51.570113+0000","last_peered":"2023-11-28T11:08:51.570113+0000","last_clean":"2023-11-28T11:08:51.570113+0000","last_became_active":"2023-11-28T11:08:50.131379+0000","last_became_peered":"2023-11-28T11:08:50.131379+0000","last_unstale":"2023-11-28T11:08:51.570113+0000","last_undegraded":"2023-11-28T11:08:51.570113+0000","last_fullsized":"2023-11-28T11:08:51.570113+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:42:35.306388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4,0],"acting":[1,3,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1b","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.103804+0000","last_change":"2023-11-28T11:08:45.523090+0000","last_active":"2023-11-28T11:08:50.103804+0000","last_peered":"2023-11-28T11:08:50.103804+0000","last_clean":"2023-11-28T11:08:50.103804+0000","last_became_active":"2023-11-28T11:08:45.522445+0000","last_became_peered":"2023-11-28T11:08:45.522445+0000","last_unstale":"2023-11-28T11:08:50.103804+0000","last_undegraded":"2023-11-28T11:08:50.103804+0000","last_fullsized":"2023-11-28T11:08:50.103804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:14:54.289645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108643+0000","last_change":"2023-11-28T11:08:45.943849+0000","last_active":"2023-11-28T11:08:50.108643+0000","last_peered":"2023-11-28T11:08:50.108643+0000","last_clean":"2023-11-28T11:08:50.108643+0000","last_became_active":"2023-11-28T11:08:45.943426+0000","last_became_peered":"2023-11-28T11:08:45.943426+0000","last_unstale":"2023-11-28T11:08:50.108643+0000","last_undegraded":"2023-11-28T11:08:50.108643+0000","last_fullsized":"2023-11-28T11:08:50.108643+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:10:29.591607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.570960+0000","last_change":"2023-11-28T11:08:50.129403+0000","last_active":"2023-11-28T11:08:51.570960+0000","last_peered":"2023-11-28T11:08:51.570960+0000","last_clean":"2023-11-28T11:08:51.570960+0000","last_became_active":"2023-11-28T11:08:50.128984+0000","last_became_peered":"2023-11-28T11:08:50.128984+0000","last_unstale":"2023-11-28T11:08:51.570960+0000","last_undegraded":"2023-11-28T11:08:51.570960+0000","last_fullsized":"2023-11-28T11:08:51.570960+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:54.778391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1,6],"acting":[7,4,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105418+0000","last_change":"2023-11-28T11:08:45.514673+0000","last_active":"2023-11-28T11:08:50.105418+0000","last_peered":"2023-11-28T11:08:50.105418+0000","last_clean":"2023-11-28T11:08:50.105418+0000","last_became_active":"2023-11-28T11:08:45.513328+0000","last_became_peered":"2023-11-28T11:08:45.513328+0000","last_unstale":"2023-11-28T11:08:50.105418+0000","last_undegraded":"2023-11-28T11:08:50.105418+0000","last_fullsized":"2023-11-28T11:08:50.105418+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:55.864512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112696+0000","last_change":"2023-11-28T11:08:45.948904+0000","last_active":"2023-11-28T11:08:50.112696+0000","last_peered":"2023-11-28T11:08:50.112696+0000","last_clean":"2023-11-28T11:08:50.112696+0000","last_became_active":"2023-11-28T11:08:45.948629+0000","last_became_peered":"2023-11-28T11:08:45.948629+0000","last_unstale":"2023-11-28T11:08:50.112696+0000","last_undegraded":"2023-11-28T11:08:50.112696+0000","last_fullsized":"2023-11-28T11:08:50.112696+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:17:55.031649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.537746+0000","last_change":"2023-11-28T11:08:50.124117+0000","last_active":"2023-11-28T11:08:50.537746+0000","last_peered":"2023-11-28T11:08:50.537746+0000","last_clean":"2023-11-28T11:08:50.537746+0000","last_became_active":"2023-11-28T11:08:50.123257+0000","last_became_peered":"2023-11-28T11:08:50.123257+0000","last_unstale":"2023-11-28T11:08:50.537746+0000","last_undegraded":"2023-11-28T11:08:50.537746+0000","last_fullsized":"2023-11-28T11:08:50.537746+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:52:24.842089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7,4],"acting":[5,0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1d","version":"20'2","reported_seq":17,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.105770+0000","last_change":"2023-11-28T11:08:45.511721+0000","last_active":"2023-11-28T11:08:50.105770+0000","last_peered":"2023-11-28T11:08:50.105770+0000","last_clean":"2023-11-28T11:08:50.105770+0000","last_became_active":"2023-11-28T11:08:45.511413+0000","last_became_peered":"2023-11-28T11:08:45.511413+0000","last_unstale":"2023-11-28T11:08:50.105770+0000","last_undegraded":"2023-11-28T11:08:50.105770+0000","last_fullsized":"2023-11-28T11:08:50.105770+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:24.093818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.111702+0000","last_change":"2023-11-28T11:08:45.945561+0000","last_active":"2023-11-28T11:08:50.111702+0000","last_peered":"2023-11-28T11:08:50.111702+0000","last_clean":"2023-11-28T11:08:50.111702+0000","last_became_active":"2023-11-28T11:08:45.945101+0000","last_became_peered":"2023-11-28T11:08:45.945101+0000","last_unstale":"2023-11-28T11:08:50.111702+0000","last_undegraded":"2023-11-28T11:08:50.111702+0000","last_fullsized":"2023-11-28T11:08:50.111702+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:00.598900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.535418+0000","last_change":"2023-11-28T11:08:50.132859+0000","last_active":"2023-11-28T11:08:50.535418+0000","last_peered":"2023-11-28T11:08:50.535418+0000","last_clean":"2023-11-28T11:08:50.535418+0000","last_became_active":"2023-11-28T11:08:50.132397+0000","last_became_peered":"2023-11-28T11:08:50.132397+0000","last_unstale":"2023-11-28T11:08:50.535418+0000","last_undegraded":"2023-11-28T11:08:50.535418+0000","last_fullsized":"2023-11-28T11:08:50.535418+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:36:07.333534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1,2],"acting":[4,6,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"20'1","reported_seq":15,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112935+0000","last_change":"2023-11-28T11:08:45.511864+0000","last_active":"2023-11-28T11:08:50.112935+0000","last_peered":"2023-11-28T11:08:50.112935+0000","last_clean":"2023-11-28T11:08:50.112935+0000","last_became_active":"2023-11-28T11:08:45.511500+0000","last_became_peered":"2023-11-28T11:08:45.511500+0000","last_unstale":"2023-11-28T11:08:50.112935+0000","last_undegraded":"2023-11-28T11:08:50.112935+0000","last_fullsized":"2023-11-28T11:08:50.112935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:31:55.864512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108100+0000","last_change":"2023-11-28T11:08:45.945158+0000","last_active":"2023-11-28T11:08:50.108100+0000","last_peered":"2023-11-28T11:08:50.108100+0000","last_clean":"2023-11-28T11:08:50.108100+0000","last_became_active":"2023-11-28T11:08:45.944341+0000","last_became_peered":"2023-11-28T11:08:45.944341+0000","last_unstale":"2023-11-28T11:08:50.108100+0000","last_undegraded":"2023-11-28T11:08:50.108100+0000","last_fullsized":"2023-11-28T11:08:50.108100+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:38:45.112861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":12,"reported_epoch":25,"state":"active+clean","last_fresh":"2023-11-28T11:08:51.569813+0000","last_change":"2023-11-28T11:08:50.136091+0000","last_active":"2023-11-28T11:08:51.569813+0000","last_peered":"2023-11-28T11:08:51.569813+0000","last_clean":"2023-11-28T11:08:51.569813+0000","last_became_active":"2023-11-28T11:08:50.135640+0000","last_became_peered":"2023-11-28T11:08:50.135640+0000","last_unstale":"2023-11-28T11:08:51.569813+0000","last_undegraded":"2023-11-28T11:08:51.569813+0000","last_fullsized":"2023-11-28T11:08:51.569813+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:49.091195+0000","last_clean_scrub_stamp":"2023-11-28T11:08:49.091195+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:19:52.258463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7,2],"acting":[1,5,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1f","version":"20'4","reported_seq":18,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.106438+0000","last_change":"2023-11-28T11:08:45.522945+0000","last_active":"2023-11-28T11:08:50.106438+0000","last_peered":"2023-11-28T11:08:50.106438+0000","last_clean":"2023-11-28T11:08:50.106438+0000","last_became_active":"2023-11-28T11:08:45.522100+0000","last_became_peered":"2023-11-28T11:08:45.522100+0000","last_unstale":"2023-11-28T11:08:50.106438+0000","last_undegraded":"2023-11-28T11:08:50.106438+0000","last_fullsized":"2023-11-28T11:08:50.106438+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:44.485404+0000","last_clean_scrub_stamp":"2023-11-28T11:08:44.485404+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:24.093818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.116376+0000","last_change":"2023-11-28T11:08:45.955608+0000","last_active":"2023-11-28T11:08:50.116376+0000","last_peered":"2023-11-28T11:08:50.116376+0000","last_clean":"2023-11-28T11:08:50.116376+0000","last_became_active":"2023-11-28T11:08:45.955293+0000","last_became_peered":"2023-11-28T11:08:45.955293+0000","last_unstale":"2023-11-28T11:08:50.116376+0000","last_undegraded":"2023-11-28T11:08:50.116376+0000","last_fullsized":"2023-11-28T11:08:50.116376+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:33.088404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.108532+0000","last_change":"2023-11-28T11:08:45.953710+0000","last_active":"2023-11-28T11:08:50.108532+0000","last_peered":"2023-11-28T11:08:50.108532+0000","last_clean":"2023-11-28T11:08:50.108532+0000","last_became_active":"2023-11-28T11:08:45.953349+0000","last_became_peered":"2023-11-28T11:08:45.953349+0000","last_unstale":"2023-11-28T11:08:50.108532+0000","last_undegraded":"2023-11-28T11:08:50.108532+0000","last_fullsized":"2023-11-28T11:08:50.108532+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:28:03.032281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.107845+0000","last_change":"2023-11-28T11:08:45.945358+0000","last_active":"2023-11-28T11:08:50.107845+0000","last_peered":"2023-11-28T11:08:50.107845+0000","last_clean":"2023-11-28T11:08:50.107845+0000","last_became_active":"2023-11-28T11:08:45.944619+0000","last_became_peered":"2023-11-28T11:08:45.944619+0000","last_unstale":"2023-11-28T11:08:50.107845+0000","last_undegraded":"2023-11-28T11:08:50.107845+0000","last_fullsized":"2023-11-28T11:08:50.107845+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:58:17.059825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.112795+0000","last_change":"2023-11-28T11:08:45.951315+0000","last_active":"2023-11-28T11:08:50.112795+0000","last_peered":"2023-11-28T11:08:50.112795+0000","last_clean":"2023-11-28T11:08:50.112795+0000","last_became_active":"2023-11-28T11:08:45.950992+0000","last_became_peered":"2023-11-28T11:08:45.950992+0000","last_unstale":"2023-11-28T11:08:50.112795+0000","last_undegraded":"2023-11-28T11:08:50.112795+0000","last_fullsized":"2023-11-28T11:08:50.112795+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:23:51.178080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":13,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.109781+0000","last_change":"2023-11-28T11:08:45.954451+0000","last_active":"2023-11-28T11:08:50.109781+0000","last_peered":"2023-11-28T11:08:50.109781+0000","last_clean":"2023-11-28T11:08:50.109781+0000","last_became_active":"2023-11-28T11:08:45.954162+0000","last_became_peered":"2023-11-28T11:08:45.954162+0000","last_unstale":"2023-11-28T11:08:50.109781+0000","last_undegraded":"2023-11-28T11:08:50.109781+0000","last_fullsized":"2023-11-28T11:08:50.109781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:54:39.158725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":14,"reported_epoch":24,"state":"active+clean","last_fresh":"2023-11-28T11:08:50.119893+0000","last_change":"2023-11-28T11:08:45.944860+0000","last_active":"2023-11-28T11:08:50.119893+0000","last_peered":"2023-11-28T11:08:50.119893+0000","last_clean":"2023-11-28T11:08:50.119893+0000","last_became_active":"2023-11-28T11:08:45.944591+0000","last_became_peered":"2023-11-28T11:08:45.944591+0000","last_unstale":"2023-11-28T11:08:50.119893+0000","last_undegraded":"2023-11-28T11:08:50.119893+0000","last_fullsized":"2023-11-28T11:08:50.119893+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:45.490884+0000","last_clean_scrub_stamp":"2023-11-28T11:08:45.490884+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:26:28.332988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":4528,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":48,"num_write_kb":18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":9056,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":42,"ondisk_log_size":42,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9260,"kb_used_data":868,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362580,"statfs":{"total":96636764160,"available":96627281920,"internally_reserved":0,"allocated":888832,"data_stored":703731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.2189999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.1360000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3820000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.3069999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49399999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574857,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8076,"kb_used_data":260,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363764,"statfs":{"total":96636764160,"available":96628494336,"internally_reserved":0,"allocated":266240,"data_stored":109401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.7450000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53600000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574857,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9232,"kb_used_data":840,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362608,"statfs":{"total":96636764160,"available":96627310592,"internally_reserved":0,"allocated":860160,"data_stored":699769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40500000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34799999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51900000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.246}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26600000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70199999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.496}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51800000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574858,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8072,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363768,"statfs":{"total":96636764160,"available":96628498432,"internally_reserved":0,"allocated":262144,"data_stored":108440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69999999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75}]}]},{"osd":3,"up_from":13,"seq":55834574857,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8144,"kb_used_data":264,"kb_used_omap":0,"kb_used_meta":7872,"kb_avail":94363696,"statfs":{"total":96636764160,"available":96628424704,"internally_reserved":0,"allocated":270336,"data_stored":109491,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8060928},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.965}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.0609999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.8899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.98}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":11.888}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.9490000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8224,"kb_used_data":280,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363616,"statfs":{"total":96636764160,"available":96628342784,"internally_reserved":0,"allocated":286720,"data_stored":113203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.626}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57599999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8084,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":7808,"kb_avail":94363756,"statfs":{"total":96636764160,"available":96628486144,"internally_reserved":0,"allocated":274432,"data_stored":110037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":7995392},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34399999999999997}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":566,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":3802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":3962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:09:09.500 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T11:09:09.500 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-28T11:09:09.501 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-28T11:09:09.501 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:09:09.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.657+0000 7ffb259b0700 1 Processor -- start 2023-11-28T11:09:09.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.657+0000 7ffb259b0700 1 -- start start 2023-11-28T11:09:09.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.658+0000 7ffb259b0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffb2015ca20 0x7ffb200b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.658+0000 7ffb259b0700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb200b2620 con 0x7ffb2006d0d0 2023-11-28T11:09:09.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.658+0000 7ffb259b0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb20071550 con 0x7ffb200b0d10 2023-11-28T11:09:09.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.658+0000 7ffb259b0700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb200721d0 con 0x7ffb2015ca20 2023-11-28T11:09:09.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.658+0000 7ffb1f7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ffb2006d0d0 0x7ffb2006d570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35410/0 (socket says 172.21.15.28:35410) 2023-11-28T11:09:09.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.658+0000 7ffb1f7fe700 1 -- 172.21.15.28:0/1879327981 learned_addr learned my addr 172.21.15.28:0/1879327981 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:09.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.659+0000 7ffb1e7fc700 1 --2- 172.21.15.28:0/1879327981 >> v2:172.21.15.59:3300/0 conn(0x7ffb2015ca20 0x7ffb200b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.659+0000 7ffb1dffb700 1 -- 172.21.15.28:0/1879327981 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345881378 0 0) 0x7ffb18001490 con 0x7ffb2006d0d0 2023-11-28T11:09:09.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.659+0000 7ffb1dffb700 1 -- 172.21.15.28:0/1879327981 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb080035b0 con 0x7ffb2006d0d0 2023-11-28T11:09:09.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.659+0000 7ffb1dffb700 1 -- 172.21.15.28:0/1879327981 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3907337217 0 0) 0x7ffb20071550 con 0x7ffb200b0d10 2023-11-28T11:09:09.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.659+0000 7ffb1dffb700 1 -- 172.21.15.28:0/1879327981 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb18001490 con 0x7ffb200b0d10 2023-11-28T11:09:09.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.659+0000 7ffb1e7fc700 1 -- 172.21.15.28:0/1879327981 >> v1:172.21.15.59:6789/0 conn(0x7ffb200b0d10 legacy=0x7ffb200b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.659+0000 7ffb1dffb700 1 -- 172.21.15.28:0/1879327981 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 758372510 0 0) 0x7ffb080035b0 con 0x7ffb2006d0d0 2023-11-28T11:09:09.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.659+0000 7ffb1e7fc700 1 -- 172.21.15.28:0/1879327981 >> v1:172.21.15.28:6789/0 conn(0x7ffb2006d0d0 legacy=0x7ffb2006d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.659+0000 7ffb1e7fc700 1 -- 172.21.15.28:0/1879327981 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb20071830 con 0x7ffb2015ca20 2023-11-28T11:09:09.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.659+0000 7ffb1e7fc700 1 --2- 172.21.15.28:0/1879327981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb2015ca20 0x7ffb200b20e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009840 tx=0x7ffb10009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd557c7e196dd988 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:09.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.659+0000 7ffb1dffb700 1 -- 172.21.15.28:0/1879327981 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb18001b80 con 0x7ffb2015ca20 2023-11-28T11:09:09.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.659+0000 7ffb1dffb700 1 -- 172.21.15.28:0/1879327981 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb10004030 con 0x7ffb2015ca20 2023-11-28T11:09:09.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.659+0000 7ffb1dffb700 1 -- 172.21.15.28:0/1879327981 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb100043a0 con 0x7ffb2015ca20 2023-11-28T11:09:09.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.660+0000 7ffb259b0700 1 -- 172.21.15.28:0/1879327981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb2015ca20 msgr2=0x7ffb200b20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.660+0000 7ffb259b0700 1 --2- 172.21.15.28:0/1879327981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb2015ca20 0x7ffb200b20e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7ffb10009840 tx=0x7ffb10009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:09.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.660+0000 7ffb259b0700 1 -- 172.21.15.28:0/1879327981 shutdown_connections 2023-11-28T11:09:09.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.660+0000 7ffb259b0700 1 --2- 172.21.15.28:0/1879327981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb2015ca20 0x7ffb200b20e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.660+0000 7ffb259b0700 1 -- 172.21.15.28:0/1879327981 >> 172.21.15.28:0/1879327981 conn(0x7ffb200cbd10 msgr2=0x7ffb200cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:09.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.660+0000 7ffb259b0700 1 -- 172.21.15.28:0/1879327981 shutdown_connections 2023-11-28T11:09:09.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.660+0000 7ffb259b0700 1 -- 172.21.15.28:0/1879327981 wait complete. 2023-11-28T11:09:09.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.661+0000 7ffb259b0700 1 Processor -- start 2023-11-28T11:09:09.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.661+0000 7ffb259b0700 1 -- start start 2023-11-28T11:09:09.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb259b0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb2006d0d0 0x7ffb2015bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb259b0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb200b0d10 0x7ffb20157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb259b0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb2015ca20 0x7ffb201f1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb259b0700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb20157c20 con 0x7ffb2006d0d0 2023-11-28T11:09:09.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb259b0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb201f2e90 con 0x7ffb2015ca20 2023-11-28T11:09:09.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb259b0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb201f20d0 con 0x7ffb200b0d10 2023-11-28T11:09:09.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb1effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb2006d0d0 0x7ffb2015bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb1effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb2006d0d0 0x7ffb2015bd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47354/0 (socket says 172.21.15.28:47354) 2023-11-28T11:09:09.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb1effd700 1 -- 172.21.15.28:0/701862770 learned_addr learned my addr 172.21.15.28:0/701862770 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:09.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb1e7fc700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb200b0d10 0x7ffb20157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb1f7fe700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb2015ca20 0x7ffb201f1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb1effd700 1 -- 172.21.15.28:0/701862770 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb200b0d10 msgr2=0x7ffb20157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb1effd700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb200b0d10 0x7ffb20157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb1effd700 1 -- 172.21.15.28:0/701862770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb2015ca20 msgr2=0x7ffb201f1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb1effd700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb2015ca20 0x7ffb201f1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb1effd700 1 -- 172.21.15.28:0/701862770 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb100094a0 con 0x7ffb2006d0d0 2023-11-28T11:09:09.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb1e7fc700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb200b0d10 0x7ffb20157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:09.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.662+0000 7ffb1f7fe700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb2015ca20 0x7ffb201f1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:09.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.663+0000 7ffb1effd700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb2006d0d0 0x7ffb2015bd60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400c500 tx=0x7ffb1400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:09.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.663+0000 7ffb07fff700 1 -- 172.21.15.28:0/701862770 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb140052d0 con 0x7ffb2006d0d0 2023-11-28T11:09:09.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.663+0000 7ffb07fff700 1 -- 172.21.15.28:0/701862770 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb14005430 con 0x7ffb2006d0d0 2023-11-28T11:09:09.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.663+0000 7ffb259b0700 1 -- 172.21.15.28:0/701862770 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb20071df0 con 0x7ffb2006d0d0 2023-11-28T11:09:09.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.663+0000 7ffb07fff700 1 -- 172.21.15.28:0/701862770 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb14003910 con 0x7ffb2006d0d0 2023-11-28T11:09:09.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.663+0000 7ffb259b0700 1 -- 172.21.15.28:0/701862770 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb20071b10 con 0x7ffb2006d0d0 2023-11-28T11:09:09.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.664+0000 7ffb259b0700 1 -- 172.21.15.28:0/701862770 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb200711e0 con 0x7ffb2006d0d0 2023-11-28T11:09:09.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.664+0000 7ffb07fff700 1 -- 172.21.15.28:0/701862770 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffb14019030 con 0x7ffb2006d0d0 2023-11-28T11:09:09.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.665+0000 7ffb07fff700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb0c0682f0 0x7ffb0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:09.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.665+0000 7ffb1e7fc700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb0c0682f0 0x7ffb0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:09.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.665+0000 7ffb07fff700 1 -- 172.21.15.28:0/701862770 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ffb14085420 con 0x7ffb2006d0d0 2023-11-28T11:09:09.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.665+0000 7ffb1e7fc700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb0c0682f0 0x7ffb0c06a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb201f65e0 tx=0x7ffb10013040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:09.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.669+0000 7ffb07fff700 1 -- 172.21.15.28:0/701862770 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffb14051970 con 0x7ffb2006d0d0 2023-11-28T11:09:09.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.908+0000 7ffb259b0700 1 -- 172.21.15.28:0/701862770 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ffb20066c60 con 0x7ffb2006d0d0 2023-11-28T11:09:09.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.908+0000 7ffb07fff700 1 -- 172.21.15.28:0/701862770 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ffb14054f90 con 0x7ffb2006d0d0 2023-11-28T11:09:09.909 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:09.909 INFO:teuthology.orchestra.run.smithi028.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-28T11:09:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.910+0000 7ffb259b0700 1 -- 172.21.15.28:0/701862770 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb0c0682f0 msgr2=0x7ffb0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.910+0000 7ffb259b0700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb0c0682f0 0x7ffb0c06a7a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb201f65e0 tx=0x7ffb10013040 comp rx=0 tx=0).stop 2023-11-28T11:09:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.910+0000 7ffb259b0700 1 -- 172.21.15.28:0/701862770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb2006d0d0 msgr2=0x7ffb2015bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.910+0000 7ffb259b0700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb2006d0d0 0x7ffb2015bd60 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ffb1400c500 tx=0x7ffb1400c8c0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.910+0000 7ffb259b0700 1 -- 172.21.15.28:0/701862770 shutdown_connections 2023-11-28T11:09:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.910+0000 7ffb259b0700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb0c0682f0 0x7ffb0c06a7a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.910+0000 7ffb259b0700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb2006d0d0 0x7ffb2015bd60 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.910+0000 7ffb259b0700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb2015ca20 0x7ffb201f1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.910+0000 7ffb259b0700 1 --2- 172.21.15.28:0/701862770 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb200b0d10 0x7ffb20157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:09.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.910+0000 7ffb259b0700 1 -- 172.21.15.28:0/701862770 >> 172.21.15.28:0/701862770 conn(0x7ffb200cbd10 msgr2=0x7ffb200b30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:09.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.910+0000 7ffb259b0700 1 -- 172.21.15.28:0/701862770 shutdown_connections 2023-11-28T11:09:09.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:09.910+0000 7ffb259b0700 1 -- 172.21.15.28:0/701862770 wait complete. 2023-11-28T11:09:09.923 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-28T11:09:09.924 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:10.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.052+0000 7f8cb3ad5700 1 Processor -- start 2023-11-28T11:09:10.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.053+0000 7f8cb3ad5700 1 -- start start 2023-11-28T11:09:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb3ad5700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8cac15a5a0 0x7f8cac06b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb3ad5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8cac0ca280 0x7f8cac0cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb3ad5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cac0cbf40 0x7f8cac0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb3ad5700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac0cb570 con 0x7f8cac15a5a0 2023-11-28T11:09:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb3ad5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cac0ca6f0 con 0x7f8cac0ca280 2023-11-28T11:09:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb3ad5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac0ca960 con 0x7f8cac0cbf40 2023-11-28T11:09:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb1871700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8cac15a5a0 0x7f8cac06b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb1871700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8cac15a5a0 0x7f8cac06b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47362/0 (socket says 172.21.15.28:47362) 2023-11-28T11:09:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb1871700 1 -- 172.21.15.28:0/1477052865 learned_addr learned my addr 172.21.15.28:0/1477052865 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb2072700 1 --2- 172.21.15.28:0/1477052865 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cac0cbf40 0x7f8cac0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb1070700 1 --2- 172.21.15.28:0/1477052865 >> v2:172.21.15.59:3300/0 conn(0x7f8cac0ca280 0x7f8cac0cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb1871700 1 -- 172.21.15.28:0/1477052865 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cac0cbf40 msgr2=0x7f8cac0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb1871700 1 --2- 172.21.15.28:0/1477052865 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cac0cbf40 0x7f8cac0cc4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb1871700 1 -- 172.21.15.28:0/1477052865 >> v2:172.21.15.59:3300/0 conn(0x7f8cac0ca280 msgr2=0x7f8cac0cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb1871700 1 --2- 172.21.15.28:0/1477052865 >> v2:172.21.15.59:3300/0 conn(0x7f8cac0ca280 0x7f8cac0cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb1871700 1 -- 172.21.15.28:0/1477052865 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cac0cac40 con 0x7f8cac15a5a0 2023-11-28T11:09:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.054+0000 7f8cb1871700 1 --2- 172.21.15.28:0/1477052865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cac15a5a0 0x7f8cac06b930 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0c6b20 tx=0x7f8ca0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8377f638e3ae683a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.055+0000 7f8cb086f700 1 -- 172.21.15.28:0/1477052865 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca000b870 con 0x7f8cac15a5a0 2023-11-28T11:09:10.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.055+0000 7f8cb086f700 1 -- 172.21.15.28:0/1477052865 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca000b9d0 con 0x7f8cac15a5a0 2023-11-28T11:09:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.055+0000 7f8cb086f700 1 -- 172.21.15.28:0/1477052865 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca000bd60 con 0x7f8cac15a5a0 2023-11-28T11:09:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.055+0000 7f8cb3ad5700 1 -- 172.21.15.28:0/1477052865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cac15a5a0 msgr2=0x7f8cac06b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.055+0000 7f8cb3ad5700 1 --2- 172.21.15.28:0/1477052865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cac15a5a0 0x7f8cac06b930 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f8cac0c6b20 tx=0x7f8ca0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.055+0000 7f8cb3ad5700 1 -- 172.21.15.28:0/1477052865 shutdown_connections 2023-11-28T11:09:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.055+0000 7f8cb3ad5700 1 --2- 172.21.15.28:0/1477052865 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cac0cbf40 0x7f8cac0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.055+0000 7f8cb3ad5700 1 --2- 172.21.15.28:0/1477052865 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cac15a5a0 0x7f8cac06b930 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.055+0000 7f8cb3ad5700 1 --2- 172.21.15.28:0/1477052865 >> v2:172.21.15.59:3300/0 conn(0x7f8cac0ca280 0x7f8cac0cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.055+0000 7f8cb3ad5700 1 -- 172.21.15.28:0/1477052865 >> 172.21.15.28:0/1477052865 conn(0x7f8cac0b1650 msgr2=0x7f8cac0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.059 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.055+0000 7f8cb3ad5700 1 -- 172.21.15.28:0/1477052865 shutdown_connections 2023-11-28T11:09:10.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.055+0000 7f8cb3ad5700 1 -- 172.21.15.28:0/1477052865 wait complete. 2023-11-28T11:09:10.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.056+0000 7f8cb3ad5700 1 Processor -- start 2023-11-28T11:09:10.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.057+0000 7f8cb3ad5700 1 -- start start 2023-11-28T11:09:10.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.057+0000 7f8cb3ad5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cac0ca280 0x7f8cac153050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.057+0000 7f8cb3ad5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cac0cbf40 0x7f8cac14dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.057+0000 7f8cb3ad5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cac15a5a0 0x7f8cac0b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.057+0000 7f8cb3ad5700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac154260 con 0x7f8cac0cbf40 2023-11-28T11:09:10.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.057+0000 7f8cb3ad5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac153650 con 0x7f8cac0ca280 2023-11-28T11:09:10.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.057+0000 7f8cb3ad5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cac153930 con 0x7f8cac15a5a0 2023-11-28T11:09:10.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.057+0000 7f8cb1070700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cac0cbf40 0x7f8cac14dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.057+0000 7f8cb1070700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cac0cbf40 0x7f8cac14dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47372/0 (socket says 172.21.15.28:47372) 2023-11-28T11:09:10.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.057+0000 7f8cb1070700 1 -- 172.21.15.28:0/3582399283 learned_addr learned my addr 172.21.15.28:0/3582399283 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:10.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.058+0000 7f8cb2072700 1 --2- 172.21.15.28:0/3582399283 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cac15a5a0 0x7f8cac0b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.058+0000 7f8cb1871700 1 --2- 172.21.15.28:0/3582399283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cac0ca280 0x7f8cac153050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.058+0000 7f8cb1070700 1 -- 172.21.15.28:0/3582399283 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cac15a5a0 msgr2=0x7f8cac0b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.058+0000 7f8cb1070700 1 --2- 172.21.15.28:0/3582399283 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cac15a5a0 0x7f8cac0b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.058+0000 7f8cb1070700 1 -- 172.21.15.28:0/3582399283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cac0ca280 msgr2=0x7f8cac153050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.058+0000 7f8cb1070700 1 --2- 172.21.15.28:0/3582399283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cac0ca280 0x7f8cac153050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.058+0000 7f8cb1070700 1 -- 172.21.15.28:0/3582399283 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca0009550 con 0x7f8cac0cbf40 2023-11-28T11:09:10.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.058+0000 7f8cb1070700 1 --2- 172.21.15.28:0/3582399283 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cac0cbf40 0x7f8cac14dd80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800c7f0 tx=0x7f8ca800cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.058+0000 7f8c9e7fc700 1 -- 172.21.15.28:0/3582399283 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8011800 con 0x7f8cac0cbf40 2023-11-28T11:09:10.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.058+0000 7f8c9e7fc700 1 -- 172.21.15.28:0/3582399283 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ca8011960 con 0x7f8cac0cbf40 2023-11-28T11:09:10.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.058+0000 7f8c9e7fc700 1 -- 172.21.15.28:0/3582399283 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ca8011cf0 con 0x7f8cac0cbf40 2023-11-28T11:09:10.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.058+0000 7f8cb3ad5700 1 -- 172.21.15.28:0/3582399283 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cac0cb200 con 0x7f8cac0cbf40 2023-11-28T11:09:10.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.058+0000 7f8cb3ad5700 1 -- 172.21.15.28:0/3582399283 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cac0caf20 con 0x7f8cac0cbf40 2023-11-28T11:09:10.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.059+0000 7f8cb3ad5700 1 -- 172.21.15.28:0/3582399283 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cac15a970 con 0x7f8cac0cbf40 2023-11-28T11:09:10.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.059+0000 7f8c9e7fc700 1 -- 172.21.15.28:0/3582399283 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8ca8021070 con 0x7f8cac0cbf40 2023-11-28T11:09:10.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.060+0000 7f8c9e7fc700 1 --2- 172.21.15.28:0/3582399283 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c900682f0 0x7f8c9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.060+0000 7f8c9e7fc700 1 -- 172.21.15.28:0/3582399283 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8ca8085e70 con 0x7f8cac0cbf40 2023-11-28T11:09:10.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.060+0000 7f8cb1871700 1 --2- 172.21.15.28:0/3582399283 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c900682f0 0x7f8c9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.060+0000 7f8cb1871700 1 --2- 172.21.15.28:0/3582399283 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c900682f0 0x7f8c9006a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0014040 tx=0x7f8ca0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.064+0000 7f8c9e7fc700 1 -- 172.21.15.28:0/3582399283 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8ca8052360 con 0x7f8cac0cbf40 2023-11-28T11:09:10.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.258+0000 7f8cb3ad5700 1 -- 172.21.15.28:0/3582399283 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8cac15d700 con 0x7f8cac0cbf40 2023-11-28T11:09:10.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.260+0000 7f8c9e7fc700 1 -- 172.21.15.28:0/3582399283 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f8ca801f020 con 0x7f8cac0cbf40 2023-11-28T11:09:10.263 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:10.263 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:50.531733+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:10.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.262+0000 7f8cb3ad5700 1 -- 172.21.15.28:0/3582399283 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c900682f0 msgr2=0x7f8c9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.262+0000 7f8cb3ad5700 1 --2- 172.21.15.28:0/3582399283 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c900682f0 0x7f8c9006a7a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f8ca0014040 tx=0x7f8ca0008030 comp rx=0 tx=0).stop 2023-11-28T11:09:10.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.262+0000 7f8cb3ad5700 1 -- 172.21.15.28:0/3582399283 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cac0cbf40 msgr2=0x7f8cac14dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.262+0000 7f8cb3ad5700 1 --2- 172.21.15.28:0/3582399283 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cac0cbf40 0x7f8cac14dd80 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f8ca800c7f0 tx=0x7f8ca800cbb0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.262+0000 7f8cb3ad5700 1 -- 172.21.15.28:0/3582399283 shutdown_connections 2023-11-28T11:09:10.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.262+0000 7f8cb3ad5700 1 --2- 172.21.15.28:0/3582399283 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c900682f0 0x7f8c9006a7a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.262+0000 7f8cb3ad5700 1 --2- 172.21.15.28:0/3582399283 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cac15a5a0 0x7f8cac0b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.262+0000 7f8cb3ad5700 1 --2- 172.21.15.28:0/3582399283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cac0ca280 0x7f8cac153050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.262+0000 7f8cb3ad5700 1 --2- 172.21.15.28:0/3582399283 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cac0cbf40 0x7f8cac14dd80 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.262+0000 7f8cb3ad5700 1 -- 172.21.15.28:0/3582399283 >> 172.21.15.28:0/3582399283 conn(0x7f8cac0b1650 msgr2=0x7f8cac0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.262+0000 7f8cb3ad5700 1 -- 172.21.15.28:0/3582399283 shutdown_connections 2023-11-28T11:09:10.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.262+0000 7f8cb3ad5700 1 -- 172.21.15.28:0/3582399283 wait complete. 2023-11-28T11:09:10.275 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:10.275 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:10.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.403+0000 7f531c733700 1 Processor -- start 2023-11-28T11:09:10.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.403+0000 7f531c733700 1 -- start start 2023-11-28T11:09:10.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531c733700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5314159500 0x7f53140cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531c733700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f531415dbf0 0x7f53140ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531c733700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f53140b1630 0x7f53140b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531c733700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53140b2010 con 0x7f53140b1630 2023-11-28T11:09:10.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531c733700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53140cede0 con 0x7f5314159500 2023-11-28T11:09:10.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531c733700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f531415e040 con 0x7f531415dbf0 2023-11-28T11:09:10.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531acd0700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f53140b1630 0x7f53140b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531acd0700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f53140b1630 0x7f53140b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47380/0 (socket says 172.21.15.28:47380) 2023-11-28T11:09:10.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531acd0700 1 -- 172.21.15.28:0/59705475 learned_addr learned my addr 172.21.15.28:0/59705475 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:10.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531a4cf700 1 --2- 172.21.15.28:0/59705475 >> v2:172.21.15.59:3300/0 conn(0x7f5314159500 0x7f53140cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f5319cce700 1 --2- 172.21.15.28:0/59705475 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f531415dbf0 0x7f53140ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531acd0700 1 -- 172.21.15.28:0/59705475 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f531415dbf0 msgr2=0x7f53140ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531acd0700 1 --2- 172.21.15.28:0/59705475 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f531415dbf0 0x7f53140ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531acd0700 1 -- 172.21.15.28:0/59705475 >> v2:172.21.15.59:3300/0 conn(0x7f5314159500 msgr2=0x7f53140cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531acd0700 1 --2- 172.21.15.28:0/59705475 >> v2:172.21.15.59:3300/0 conn(0x7f5314159500 0x7f53140cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531acd0700 1 -- 172.21.15.28:0/59705475 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53140ceff0 con 0x7f53140b1630 2023-11-28T11:09:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f531acd0700 1 --2- 172.21.15.28:0/59705475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53140b1630 0x7f53140b1ad0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f530c009e50 tx=0x7f530c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf92ba2315ad1c86 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f53194cd700 1 -- 172.21.15.28:0/59705475 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c00e070 con 0x7f53140b1630 2023-11-28T11:09:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f53194cd700 1 -- 172.21.15.28:0/59705475 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f530c00b980 con 0x7f53140b1630 2023-11-28T11:09:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.404+0000 7f53194cd700 1 -- 172.21.15.28:0/59705475 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530c00bd30 con 0x7f53140b1630 2023-11-28T11:09:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.405+0000 7f531c733700 1 -- 172.21.15.28:0/59705475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53140b1630 msgr2=0x7f53140b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.405+0000 7f531c733700 1 --2- 172.21.15.28:0/59705475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53140b1630 0x7f53140b1ad0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f530c009e50 tx=0x7f530c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:10.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.405+0000 7f531c733700 1 -- 172.21.15.28:0/59705475 shutdown_connections 2023-11-28T11:09:10.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.405+0000 7f531c733700 1 --2- 172.21.15.28:0/59705475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53140b1630 0x7f53140b1ad0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.405+0000 7f531c733700 1 --2- 172.21.15.28:0/59705475 >> v2:172.21.15.59:3300/0 conn(0x7f5314159500 0x7f53140cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.405+0000 7f531c733700 1 --2- 172.21.15.28:0/59705475 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f531415dbf0 0x7f53140ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.405+0000 7f531c733700 1 -- 172.21.15.28:0/59705475 >> 172.21.15.28:0/59705475 conn(0x7f53140cc630 msgr2=0x7f53140cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.405+0000 7f531c733700 1 -- 172.21.15.28:0/59705475 shutdown_connections 2023-11-28T11:09:10.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.405+0000 7f531c733700 1 -- 172.21.15.28:0/59705475 wait complete. 2023-11-28T11:09:10.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.407+0000 7f531c733700 1 Processor -- start 2023-11-28T11:09:10.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.407+0000 7f531c733700 1 -- start start 2023-11-28T11:09:10.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531c733700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53140b1630 0x7f53140b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531c733700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5314159500 0x7f531414f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531c733700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f531415dbf0 0x7f53140b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531c733700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53140b2eb0 con 0x7f53140b1630 2023-11-28T11:09:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531c733700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5314154ce0 con 0x7f531415dbf0 2023-11-28T11:09:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531c733700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5314154fc0 con 0x7f5314159500 2023-11-28T11:09:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531a4cf700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53140b1630 0x7f53140b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531a4cf700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53140b1630 0x7f53140b5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47382/0 (socket says 172.21.15.28:47382) 2023-11-28T11:09:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531a4cf700 1 -- 172.21.15.28:0/1924480123 learned_addr learned my addr 172.21.15.28:0/1924480123 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f5319cce700 1 --2- 172.21.15.28:0/1924480123 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5314159500 0x7f531414f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531acd0700 1 --2- 172.21.15.28:0/1924480123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f531415dbf0 0x7f53140b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531a4cf700 1 -- 172.21.15.28:0/1924480123 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5314159500 msgr2=0x7f531414f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531a4cf700 1 --2- 172.21.15.28:0/1924480123 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5314159500 0x7f531414f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531a4cf700 1 -- 172.21.15.28:0/1924480123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f531415dbf0 msgr2=0x7f53140b38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531a4cf700 1 --2- 172.21.15.28:0/1924480123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f531415dbf0 0x7f53140b38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531a4cf700 1 -- 172.21.15.28:0/1924480123 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c009b30 con 0x7f53140b1630 2023-11-28T11:09:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531acd0700 1 --2- 172.21.15.28:0/1924480123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f531415dbf0 0x7f53140b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531a4cf700 1 --2- 172.21.15.28:0/1924480123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53140b1630 0x7f53140b5200 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f531000c7f0 tx=0x7f531000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f530affd700 1 -- 172.21.15.28:0/1924480123 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310011800 con 0x7f53140b1630 2023-11-28T11:09:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f530affd700 1 -- 172.21.15.28:0/1924480123 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5310011960 con 0x7f53140b1630 2023-11-28T11:09:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f530affd700 1 -- 172.21.15.28:0/1924480123 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5310011cf0 con 0x7f53140b1630 2023-11-28T11:09:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.408+0000 7f531c733700 1 -- 172.21.15.28:0/1924480123 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f531415d110 con 0x7f53140b1630 2023-11-28T11:09:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.409+0000 7f531c733700 1 -- 172.21.15.28:0/1924480123 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53140cf2d0 con 0x7f53140b1630 2023-11-28T11:09:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.409+0000 7f531c733700 1 -- 172.21.15.28:0/1924480123 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f531415d3f0 con 0x7f53140b1630 2023-11-28T11:09:10.414 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.410+0000 7f530affd700 1 -- 172.21.15.28:0/1924480123 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5310021070 con 0x7f53140b1630 2023-11-28T11:09:10.414 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.410+0000 7f530affd700 1 --2- 172.21.15.28:0/1924480123 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f52f80682f0 0x7f52f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.414 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.410+0000 7f530affd700 1 -- 172.21.15.28:0/1924480123 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5310085cf0 con 0x7f53140b1630 2023-11-28T11:09:10.414 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.410+0000 7f5319cce700 1 --2- 172.21.15.28:0/1924480123 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f52f80682f0 0x7f52f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.414 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.411+0000 7f5319cce700 1 --2- 172.21.15.28:0/1924480123 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f52f80682f0 0x7f52f806a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f531415c6d0 tx=0x7f5304008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.415+0000 7f530affd700 1 -- 172.21.15.28:0/1924480123 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5310052180 con 0x7f53140b1630 2023-11-28T11:09:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.604+0000 7f531c733700 1 -- 172.21.15.28:0/1924480123 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f53140739f0 con 0x7f53140b1630 2023-11-28T11:09:10.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.605+0000 7f530affd700 1 -- 172.21.15.28:0/1924480123 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f531001f020 con 0x7f53140b1630 2023-11-28T11:09:10.605 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:09:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.606+0000 7f531c733700 1 -- 172.21.15.28:0/1924480123 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f52f80682f0 msgr2=0x7f52f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.606+0000 7f531c733700 1 --2- 172.21.15.28:0/1924480123 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f52f80682f0 0x7f52f806a7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f531415c6d0 tx=0x7f5304008040 comp rx=0 tx=0).stop 2023-11-28T11:09:10.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.606+0000 7f531c733700 1 -- 172.21.15.28:0/1924480123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53140b1630 msgr2=0x7f53140b5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.606+0000 7f531c733700 1 --2- 172.21.15.28:0/1924480123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53140b1630 0x7f53140b5200 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f531000c7f0 tx=0x7f531000cb00 comp rx=0 tx=0).stop 2023-11-28T11:09:10.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.606+0000 7f531c733700 1 -- 172.21.15.28:0/1924480123 shutdown_connections 2023-11-28T11:09:10.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.606+0000 7f531c733700 1 --2- 172.21.15.28:0/1924480123 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f52f80682f0 0x7f52f806a7a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.606+0000 7f531c733700 1 --2- 172.21.15.28:0/1924480123 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f53140b1630 0x7f53140b5200 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.607+0000 7f531c733700 1 --2- 172.21.15.28:0/1924480123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f531415dbf0 0x7f53140b38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.607+0000 7f531c733700 1 --2- 172.21.15.28:0/1924480123 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5314159500 0x7f531414f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.607+0000 7f531c733700 1 -- 172.21.15.28:0/1924480123 >> 172.21.15.28:0/1924480123 conn(0x7f53140cc630 msgr2=0x7f53140cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.607+0000 7f531c733700 1 -- 172.21.15.28:0/1924480123 shutdown_connections 2023-11-28T11:09:10.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.607+0000 7f531c733700 1 -- 172.21.15.28:0/1924480123 wait complete. 2023-11-28T11:09:10.619 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.747+0000 7ffb4179b700 1 Processor -- start 2023-11-28T11:09:10.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.747+0000 7ffb4179b700 1 -- start start 2023-11-28T11:09:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.747+0000 7ffb4179b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffb3c159500 0x7ffb3c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.747+0000 7ffb4179b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb3c15dbf0 0x7ffb3c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.747+0000 7ffb4179b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffb3c0b1630 0x7ffb3c0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.747+0000 7ffb4179b700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3c0b2010 con 0x7ffb3c0b1630 2023-11-28T11:09:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.747+0000 7ffb4179b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3c0cede0 con 0x7ffb3c159500 2023-11-28T11:09:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.747+0000 7ffb4179b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c15e040 con 0x7ffb3c15dbf0 2023-11-28T11:09:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.748+0000 7ffb3b7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffb3c0b1630 0x7ffb3c0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.748+0000 7ffb3b7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffb3c0b1630 0x7ffb3c0b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47394/0 (socket says 172.21.15.28:47394) 2023-11-28T11:09:10.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.748+0000 7ffb3b7fe700 1 -- 172.21.15.28:0/3365475558 learned_addr learned my addr 172.21.15.28:0/3365475558 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:10.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.748+0000 7ffb3a7fc700 1 --2- 172.21.15.28:0/3365475558 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb3c15dbf0 0x7ffb3c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.748+0000 7ffb3affd700 1 --2- 172.21.15.28:0/3365475558 >> v2:172.21.15.59:3300/0 conn(0x7ffb3c159500 0x7ffb3c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.748+0000 7ffb3b7fe700 1 -- 172.21.15.28:0/3365475558 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb3c15dbf0 msgr2=0x7ffb3c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:10.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.748+0000 7ffb3b7fe700 1 --2- 172.21.15.28:0/3365475558 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb3c15dbf0 0x7ffb3c0ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.748+0000 7ffb3b7fe700 1 -- 172.21.15.28:0/3365475558 >> v2:172.21.15.59:3300/0 conn(0x7ffb3c159500 msgr2=0x7ffb3c0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:10.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.748+0000 7ffb3b7fe700 1 --2- 172.21.15.28:0/3365475558 >> v2:172.21.15.59:3300/0 conn(0x7ffb3c159500 0x7ffb3c0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.748+0000 7ffb3b7fe700 1 -- 172.21.15.28:0/3365475558 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3c0ceff0 con 0x7ffb3c0b1630 2023-11-28T11:09:10.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.748+0000 7ffb3b7fe700 1 --2- 172.21.15.28:0/3365475558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c0b1630 0x7ffb3c0b1ad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009e50 tx=0x7ffb3400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99be8f48f46a2f59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.748+0000 7ffb39ffb700 1 -- 172.21.15.28:0/3365475558 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3400e070 con 0x7ffb3c0b1630 2023-11-28T11:09:10.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.748+0000 7ffb39ffb700 1 -- 172.21.15.28:0/3365475558 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb3400b980 con 0x7ffb3c0b1630 2023-11-28T11:09:10.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.748+0000 7ffb39ffb700 1 -- 172.21.15.28:0/3365475558 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3400bd30 con 0x7ffb3c0b1630 2023-11-28T11:09:10.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.749+0000 7ffb4179b700 1 -- 172.21.15.28:0/3365475558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c0b1630 msgr2=0x7ffb3c0b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.749+0000 7ffb4179b700 1 --2- 172.21.15.28:0/3365475558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c0b1630 0x7ffb3c0b1ad0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7ffb34009e50 tx=0x7ffb3400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:10.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.749+0000 7ffb4179b700 1 -- 172.21.15.28:0/3365475558 shutdown_connections 2023-11-28T11:09:10.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.749+0000 7ffb4179b700 1 --2- 172.21.15.28:0/3365475558 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c0b1630 0x7ffb3c0b1ad0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.749+0000 7ffb4179b700 1 --2- 172.21.15.28:0/3365475558 >> v2:172.21.15.59:3300/0 conn(0x7ffb3c159500 0x7ffb3c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.749+0000 7ffb4179b700 1 --2- 172.21.15.28:0/3365475558 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb3c15dbf0 0x7ffb3c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.749+0000 7ffb4179b700 1 -- 172.21.15.28:0/3365475558 >> 172.21.15.28:0/3365475558 conn(0x7ffb3c0cc630 msgr2=0x7ffb3c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.749+0000 7ffb4179b700 1 -- 172.21.15.28:0/3365475558 shutdown_connections 2023-11-28T11:09:10.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.749+0000 7ffb4179b700 1 -- 172.21.15.28:0/3365475558 wait complete. 2023-11-28T11:09:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.751+0000 7ffb4179b700 1 Processor -- start 2023-11-28T11:09:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.751+0000 7ffb4179b700 1 -- start start 2023-11-28T11:09:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.751+0000 7ffb4179b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c0b1630 0x7ffb3c0b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb4179b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb3c159500 0x7ffb3c06ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb4179b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb3c15dbf0 0x7ffb3c152c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb4179b700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c0b2da0 con 0x7ffb3c0b1630 2023-11-28T11:09:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb4179b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c0b3080 con 0x7ffb3c159500 2023-11-28T11:09:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb4179b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb3c0b3360 con 0x7ffb3c15dbf0 2023-11-28T11:09:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb3affd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c0b1630 0x7ffb3c0b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb3affd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c0b1630 0x7ffb3c0b42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47396/0 (socket says 172.21.15.28:47396) 2023-11-28T11:09:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb3affd700 1 -- 172.21.15.28:0/161669598 learned_addr learned my addr 172.21.15.28:0/161669598 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:10.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb3b7fe700 1 --2- 172.21.15.28:0/161669598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb3c15dbf0 0x7ffb3c152c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb3a7fc700 1 --2- 172.21.15.28:0/161669598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb3c159500 0x7ffb3c06ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb3affd700 1 -- 172.21.15.28:0/161669598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb3c15dbf0 msgr2=0x7ffb3c152c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb3affd700 1 --2- 172.21.15.28:0/161669598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb3c15dbf0 0x7ffb3c152c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb3affd700 1 -- 172.21.15.28:0/161669598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb3c159500 msgr2=0x7ffb3c06ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb3affd700 1 --2- 172.21.15.28:0/161669598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb3c159500 0x7ffb3c06ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb3affd700 1 -- 172.21.15.28:0/161669598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb34009b30 con 0x7ffb3c0b1630 2023-11-28T11:09:10.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb3affd700 1 --2- 172.21.15.28:0/161669598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c0b1630 0x7ffb3c0b42d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00d800 tx=0x7ffb2c00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb337fe700 1 -- 172.21.15.28:0/161669598 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c0125f0 con 0x7ffb3c0b1630 2023-11-28T11:09:10.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb337fe700 1 -- 172.21.15.28:0/161669598 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb2c012750 con 0x7ffb3c0b1630 2023-11-28T11:09:10.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb337fe700 1 -- 172.21.15.28:0/161669598 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb2c012ae0 con 0x7ffb3c0b1630 2023-11-28T11:09:10.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb4179b700 1 -- 172.21.15.28:0/161669598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb3c15d110 con 0x7ffb3c0b1630 2023-11-28T11:09:10.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.752+0000 7ffb4179b700 1 -- 172.21.15.28:0/161669598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb3c0cf2d0 con 0x7ffb3c0b1630 2023-11-28T11:09:10.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.753+0000 7ffb4179b700 1 -- 172.21.15.28:0/161669598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb3c15d3f0 con 0x7ffb3c0b1630 2023-11-28T11:09:10.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.754+0000 7ffb337fe700 1 -- 172.21.15.28:0/161669598 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffb2c012c40 con 0x7ffb3c0b1630 2023-11-28T11:09:10.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.754+0000 7ffb337fe700 1 --2- 172.21.15.28:0/161669598 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb1c068340 0x7ffb1c06a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:10.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.754+0000 7ffb337fe700 1 -- 172.21.15.28:0/161669598 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ffb2c085fb0 con 0x7ffb3c0b1630 2023-11-28T11:09:10.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.754+0000 7ffb3a7fc700 1 --2- 172.21.15.28:0/161669598 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb1c068340 0x7ffb1c06a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:10.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.755+0000 7ffb3a7fc700 1 --2- 172.21.15.28:0/161669598 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb1c068340 0x7ffb1c06a7f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0cc580 tx=0x7ffb24008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:10.763 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.759+0000 7ffb337fe700 1 -- 172.21.15.28:0/161669598 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffb2c052670 con 0x7ffb3c0b1630 2023-11-28T11:09:10.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.954+0000 7ffb4179b700 1 -- 172.21.15.28:0/161669598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb3c0739f0 con 0x7ffb3c0b1630 2023-11-28T11:09:10.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.954+0000 7ffb337fe700 1 -- 172.21.15.28:0/161669598 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb2c055c90 con 0x7ffb3c0b1630 2023-11-28T11:09:10.955 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:09:10.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.956+0000 7ffb4179b700 1 -- 172.21.15.28:0/161669598 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb1c068340 msgr2=0x7ffb1c06a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.956+0000 7ffb4179b700 1 --2- 172.21.15.28:0/161669598 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb1c068340 0x7ffb1c06a7f0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7ffb3c0cc580 tx=0x7ffb24008040 comp rx=0 tx=0).stop 2023-11-28T11:09:10.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.956+0000 7ffb4179b700 1 -- 172.21.15.28:0/161669598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c0b1630 msgr2=0x7ffb3c0b42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:10.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.956+0000 7ffb4179b700 1 --2- 172.21.15.28:0/161669598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c0b1630 0x7ffb3c0b42d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7ffb2c00d800 tx=0x7ffb2c00db10 comp rx=0 tx=0).stop 2023-11-28T11:09:10.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.956+0000 7ffb4179b700 1 -- 172.21.15.28:0/161669598 shutdown_connections 2023-11-28T11:09:10.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.956+0000 7ffb4179b700 1 --2- 172.21.15.28:0/161669598 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb1c068340 0x7ffb1c06a7f0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.956+0000 7ffb4179b700 1 --2- 172.21.15.28:0/161669598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffb3c0b1630 0x7ffb3c0b42d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.956+0000 7ffb4179b700 1 --2- 172.21.15.28:0/161669598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffb3c15dbf0 0x7ffb3c152c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.956+0000 7ffb4179b700 1 --2- 172.21.15.28:0/161669598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffb3c159500 0x7ffb3c06ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:10.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.956+0000 7ffb4179b700 1 -- 172.21.15.28:0/161669598 >> 172.21.15.28:0/161669598 conn(0x7ffb3c0cc630 msgr2=0x7ffb3c0b0e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:10.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.957+0000 7ffb4179b700 1 -- 172.21.15.28:0/161669598 shutdown_connections 2023-11-28T11:09:10.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:10.957+0000 7ffb4179b700 1 -- 172.21.15.28:0/161669598 wait complete. 2023-11-28T11:09:10.969 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:11.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.096+0000 7fe683e1b700 1 Processor -- start 2023-11-28T11:09:11.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.096+0000 7fe683e1b700 1 -- start start 2023-11-28T11:09:11.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.097+0000 7fe683e1b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe67c0b1d40 0x7fe67c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.097+0000 7fe683e1b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe67c15dbf0 0x7fe67c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.097+0000 7fe683e1b700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe67c0ceff0 con 0x7fe67c159500 2023-11-28T11:09:11.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.097+0000 7fe683e1b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe67c15e090 con 0x7fe67c15dbf0 2023-11-28T11:09:11.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.097+0000 7fe683e1b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0cede0 con 0x7fe67c0b1d40 2023-11-28T11:09:11.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.098+0000 7fe6813b6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe67c0b1d40 0x7fe67c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.098+0000 7fe6823b8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe67c15dbf0 0x7fe67c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.098+0000 7fe6813b6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe67c0b1d40 0x7fe67c0cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:53164/0 (socket says 172.21.15.28:53164) 2023-11-28T11:09:11.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.098+0000 7fe6813b6700 1 -- 172.21.15.28:0/136401417 learned_addr learned my addr 172.21.15.28:0/136401417 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:11.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.098+0000 7fe6813b6700 1 -- 172.21.15.28:0/136401417 >> v2:172.21.15.59:3300/0 conn(0x7fe67c15dbf0 msgr2=0x7fe67c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:11.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.098+0000 7fe6813b6700 1 --2- 172.21.15.28:0/136401417 >> v2:172.21.15.59:3300/0 conn(0x7fe67c15dbf0 0x7fe67c0ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.098+0000 7fe6813b6700 1 -- 172.21.15.28:0/136401417 >> v1:172.21.15.59:6789/0 conn(0x7fe67c159500 legacy=0x7fe67c0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.098+0000 7fe6813b6700 1 -- 172.21.15.28:0/136401417 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe67c0cf2d0 con 0x7fe67c0b1d40 2023-11-28T11:09:11.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.098+0000 7fe6823b8700 1 --2- 172.21.15.28:0/136401417 >> v2:172.21.15.59:3300/0 conn(0x7fe67c15dbf0 0x7fe67c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:11.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.098+0000 7fe6813b6700 1 --2- 172.21.15.28:0/136401417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe67c0b1d40 0x7fe67c0cd140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe6780098f0 tx=0x7fe678009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=81052a3af01d7af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.098+0000 7fe680bb5700 1 -- 172.21.15.28:0/136401417 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe678011830 con 0x7fe67c0b1d40 2023-11-28T11:09:11.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.099+0000 7fe680bb5700 1 -- 172.21.15.28:0/136401417 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe678011990 con 0x7fe67c0b1d40 2023-11-28T11:09:11.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.099+0000 7fe680bb5700 1 -- 172.21.15.28:0/136401417 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe678011d20 con 0x7fe67c0b1d40 2023-11-28T11:09:11.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.099+0000 7fe683e1b700 1 -- 172.21.15.28:0/136401417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe67c0b1d40 msgr2=0x7fe67c0cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.099+0000 7fe683e1b700 1 --2- 172.21.15.28:0/136401417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe67c0b1d40 0x7fe67c0cd140 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fe6780098f0 tx=0x7fe678009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:11.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.099+0000 7fe683e1b700 1 -- 172.21.15.28:0/136401417 shutdown_connections 2023-11-28T11:09:11.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.099+0000 7fe683e1b700 1 --2- 172.21.15.28:0/136401417 >> v2:172.21.15.59:3300/0 conn(0x7fe67c15dbf0 0x7fe67c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.099+0000 7fe683e1b700 1 --2- 172.21.15.28:0/136401417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe67c0b1d40 0x7fe67c0cd140 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.099+0000 7fe683e1b700 1 -- 172.21.15.28:0/136401417 >> 172.21.15.28:0/136401417 conn(0x7fe67c0cc630 msgr2=0x7fe67c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.114 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.099+0000 7fe683e1b700 1 -- 172.21.15.28:0/136401417 shutdown_connections 2023-11-28T11:09:11.114 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.099+0000 7fe683e1b700 1 -- 172.21.15.28:0/136401417 wait complete. 2023-11-28T11:09:11.115 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.100+0000 7fe683e1b700 1 Processor -- start 2023-11-28T11:09:11.115 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.100+0000 7fe683e1b700 1 -- start start 2023-11-28T11:09:11.115 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.101+0000 7fe683e1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe67c0b1d40 0x7fe67c1e4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.115 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.101+0000 7fe683e1b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe67c159500 0x7fe67c0b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.115 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.101+0000 7fe683e1b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe67c15dbf0 0x7fe67c0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.115 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.101+0000 7fe683e1b700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0b2f20 con 0x7fe67c15dbf0 2023-11-28T11:09:11.115 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.101+0000 7fe683e1b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0b3180 con 0x7fe67c0b1d40 2023-11-28T11:09:11.115 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.101+0000 7fe683e1b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe67c0b3460 con 0x7fe67c159500 2023-11-28T11:09:11.115 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.101+0000 7fe6823b8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe67c15dbf0 0x7fe67c0b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.115 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.101+0000 7fe6823b8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe67c15dbf0 0x7fe67c0b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47414/0 (socket says 172.21.15.28:47414) 2023-11-28T11:09:11.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.101+0000 7fe6823b8700 1 -- 172.21.15.28:0/180583757 learned_addr learned my addr 172.21.15.28:0/180583757 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:11.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.102+0000 7fe6813b6700 1 --2- 172.21.15.28:0/180583757 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe67c159500 0x7fe67c0b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.102+0000 7fe681bb7700 1 --2- 172.21.15.28:0/180583757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe67c0b1d40 0x7fe67c1e4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.102+0000 7fe6823b8700 1 -- 172.21.15.28:0/180583757 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe67c159500 msgr2=0x7fe67c0b4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.102+0000 7fe6823b8700 1 --2- 172.21.15.28:0/180583757 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe67c159500 0x7fe67c0b4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.102+0000 7fe6823b8700 1 -- 172.21.15.28:0/180583757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe67c0b1d40 msgr2=0x7fe67c1e4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.102+0000 7fe6823b8700 1 --2- 172.21.15.28:0/180583757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe67c0b1d40 0x7fe67c1e4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.102+0000 7fe6823b8700 1 -- 172.21.15.28:0/180583757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe674009a70 con 0x7fe67c15dbf0 2023-11-28T11:09:11.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.102+0000 7fe6823b8700 1 --2- 172.21.15.28:0/180583757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe67c15dbf0 0x7fe67c0b29e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe67400efc0 tx=0x7fe67400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.102+0000 7fe6727fc700 1 -- 172.21.15.28:0/180583757 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67400ca90 con 0x7fe67c15dbf0 2023-11-28T11:09:11.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.102+0000 7fe6727fc700 1 -- 172.21.15.28:0/180583757 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe67400f040 con 0x7fe67c15dbf0 2023-11-28T11:09:11.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.102+0000 7fe683e1b700 1 -- 172.21.15.28:0/180583757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe678009550 con 0x7fe67c15dbf0 2023-11-28T11:09:11.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.102+0000 7fe6727fc700 1 -- 172.21.15.28:0/180583757 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe67400ce10 con 0x7fe67c15dbf0 2023-11-28T11:09:11.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.102+0000 7fe683e1b700 1 -- 172.21.15.28:0/180583757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe67c15d3f0 con 0x7fe67c15dbf0 2023-11-28T11:09:11.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.103+0000 7fe683e1b700 1 -- 172.21.15.28:0/180583757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe67c15d110 con 0x7fe67c15dbf0 2023-11-28T11:09:11.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.104+0000 7fe6727fc700 1 -- 172.21.15.28:0/180583757 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe674014480 con 0x7fe67c15dbf0 2023-11-28T11:09:11.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.104+0000 7fe6727fc700 1 --2- 172.21.15.28:0/180583757 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe6600682f0 0x7fe66006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.104+0000 7fe6727fc700 1 -- 172.21.15.28:0/180583757 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe6740866f0 con 0x7fe67c15dbf0 2023-11-28T11:09:11.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.104+0000 7fe681bb7700 1 --2- 172.21.15.28:0/180583757 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe6600682f0 0x7fe66006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.104+0000 7fe681bb7700 1 --2- 172.21.15.28:0/180583757 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe6600682f0 0x7fe66006a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe66c007fd0 tx=0x7fe66c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.108+0000 7fe6727fc700 1 -- 172.21.15.28:0/180583757 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe674052c00 con 0x7fe67c15dbf0 2023-11-28T11:09:11.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.301+0000 7fe683e1b700 1 -- 172.21.15.28:0/180583757 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe67c15d6d0 con 0x7fe67c15dbf0 2023-11-28T11:09:11.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.302+0000 7fe6727fc700 1 -- 172.21.15.28:0/180583757 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe674009d70 con 0x7fe67c15dbf0 2023-11-28T11:09:11.302 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.304+0000 7fe683e1b700 1 -- 172.21.15.28:0/180583757 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe6600682f0 msgr2=0x7fe66006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.304+0000 7fe683e1b700 1 --2- 172.21.15.28:0/180583757 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe6600682f0 0x7fe66006a7a0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe66c007fd0 tx=0x7fe66c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.304+0000 7fe683e1b700 1 -- 172.21.15.28:0/180583757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe67c15dbf0 msgr2=0x7fe67c0b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.304+0000 7fe683e1b700 1 --2- 172.21.15.28:0/180583757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe67c15dbf0 0x7fe67c0b29e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fe67400efc0 tx=0x7fe67400c3a0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.304+0000 7fe683e1b700 1 -- 172.21.15.28:0/180583757 shutdown_connections 2023-11-28T11:09:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.304+0000 7fe683e1b700 1 --2- 172.21.15.28:0/180583757 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe6600682f0 0x7fe66006a7a0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.304+0000 7fe683e1b700 1 --2- 172.21.15.28:0/180583757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe67c15dbf0 0x7fe67c0b29e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.304+0000 7fe683e1b700 1 --2- 172.21.15.28:0/180583757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe67c0b1d40 0x7fe67c1e4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.304+0000 7fe683e1b700 1 --2- 172.21.15.28:0/180583757 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe67c159500 0x7fe67c0b4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.304+0000 7fe683e1b700 1 -- 172.21.15.28:0/180583757 >> 172.21.15.28:0/180583757 conn(0x7fe67c0cc630 msgr2=0x7fe67c154470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.304+0000 7fe683e1b700 1 -- 172.21.15.28:0/180583757 shutdown_connections 2023-11-28T11:09:11.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.304+0000 7fe683e1b700 1 -- 172.21.15.28:0/180583757 wait complete. 2023-11-28T11:09:11.317 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:11.444 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.444+0000 7f0723882700 1 Processor -- start 2023-11-28T11:09:11.445 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.444+0000 7f0723882700 1 -- start start 2023-11-28T11:09:11.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.445+0000 7f0723882700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f071c165670 0x7f071c167a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.445+0000 7f0723882700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c0ce550 con 0x7f071c0b1630 2023-11-28T11:09:11.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.445+0000 7f0723882700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f071c0ce830 con 0x7f071c0cf5f0 2023-11-28T11:09:11.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.445+0000 7f0723882700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f071c0ceb10 con 0x7f071c165670 2023-11-28T11:09:11.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.445+0000 7f072161e700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f071c0b1630 0x7f071c0b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35476/0 (socket says 172.21.15.28:35476) 2023-11-28T11:09:11.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.445+0000 7f072161e700 1 -- 172.21.15.28:0/4051474730 learned_addr learned my addr 172.21.15.28:0/4051474730 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:11.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.445+0000 7f0721e1f700 1 --2- 172.21.15.28:0/4051474730 >> v2:172.21.15.59:3300/0 conn(0x7f071c165670 0x7f071c167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.445+0000 7f0713fff700 1 -- 172.21.15.28:0/4051474730 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2178335295 0 0) 0x7f071c0ce550 con 0x7f071c0b1630 2023-11-28T11:09:11.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.445+0000 7f0713fff700 1 -- 172.21.15.28:0/4051474730 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07080035b0 con 0x7f071c0b1630 2023-11-28T11:09:11.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.446+0000 7f0713fff700 1 -- 172.21.15.28:0/4051474730 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2736716141 0 0) 0x7f071c0ce830 con 0x7f071c0cf5f0 2023-11-28T11:09:11.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.446+0000 7f0713fff700 1 -- 172.21.15.28:0/4051474730 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f071c0ce550 con 0x7f071c0cf5f0 2023-11-28T11:09:11.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.446+0000 7f0721e1f700 1 -- 172.21.15.28:0/4051474730 >> v1:172.21.15.59:6789/0 conn(0x7f071c0cf5f0 legacy=0x7f071c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.446+0000 7f0721e1f700 1 -- 172.21.15.28:0/4051474730 >> v1:172.21.15.28:6789/0 conn(0x7f071c0b1630 legacy=0x7f071c0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.446+0000 7f0713fff700 1 -- 172.21.15.28:0/4051474730 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3567326974 0 0) 0x7f07080035b0 con 0x7f071c0b1630 2023-11-28T11:09:11.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.446+0000 7f0721e1f700 1 -- 172.21.15.28:0/4051474730 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f071c0cedf0 con 0x7f071c165670 2023-11-28T11:09:11.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.446+0000 7f0721e1f700 1 --2- 172.21.15.28:0/4051474730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f071c165670 0x7f071c167a50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0714009dc0 tx=0x7f071400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a4afb99c06c93136 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.446+0000 7f0713fff700 1 -- 172.21.15.28:0/4051474730 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071400e070 con 0x7f071c165670 2023-11-28T11:09:11.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.446+0000 7f0713fff700 1 -- 172.21.15.28:0/4051474730 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071400bae0 con 0x7f071c165670 2023-11-28T11:09:11.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.446+0000 7f0713fff700 1 -- 172.21.15.28:0/4051474730 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071400be70 con 0x7f071c165670 2023-11-28T11:09:11.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.447+0000 7f0723882700 1 -- 172.21.15.28:0/4051474730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f071c165670 msgr2=0x7f071c167a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.447+0000 7f0723882700 1 --2- 172.21.15.28:0/4051474730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f071c165670 0x7f071c167a50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0714009dc0 tx=0x7f071400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:11.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.447+0000 7f0723882700 1 -- 172.21.15.28:0/4051474730 shutdown_connections 2023-11-28T11:09:11.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.447+0000 7f0723882700 1 --2- 172.21.15.28:0/4051474730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f071c165670 0x7f071c167a50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.447+0000 7f0723882700 1 -- 172.21.15.28:0/4051474730 >> 172.21.15.28:0/4051474730 conn(0x7f071c0cc630 msgr2=0x7f071c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.447+0000 7f0723882700 1 -- 172.21.15.28:0/4051474730 shutdown_connections 2023-11-28T11:09:11.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.447+0000 7f0723882700 1 -- 172.21.15.28:0/4051474730 wait complete. 2023-11-28T11:09:11.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.448+0000 7f0723882700 1 Processor -- start 2023-11-28T11:09:11.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.448+0000 7f0723882700 1 -- start start 2023-11-28T11:09:11.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0723882700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f071c0b1630 0x7f071c154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0723882700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f071c0cf5f0 0x7f071c153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0723882700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f071c165670 0x7f071c0c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0723882700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c0ae3b0 con 0x7f071c165670 2023-11-28T11:09:11.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0723882700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f071c0ac6d0 con 0x7f071c0b1630 2023-11-28T11:09:11.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0723882700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f071c0ac9b0 con 0x7f071c0cf5f0 2023-11-28T11:09:11.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0721e1f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f071c165670 0x7f071c0c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0721e1f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f071c165670 0x7f071c0c9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47428/0 (socket says 172.21.15.28:47428) 2023-11-28T11:09:11.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0721e1f700 1 -- 172.21.15.28:0/1416096655 learned_addr learned my addr 172.21.15.28:0/1416096655 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:11.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0720e1d700 1 --2- 172.21.15.28:0/1416096655 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f071c0cf5f0 0x7f071c153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f072161e700 1 --2- 172.21.15.28:0/1416096655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f071c0b1630 0x7f071c154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0721e1f700 1 -- 172.21.15.28:0/1416096655 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f071c0cf5f0 msgr2=0x7f071c153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0721e1f700 1 --2- 172.21.15.28:0/1416096655 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f071c0cf5f0 0x7f071c153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0721e1f700 1 -- 172.21.15.28:0/1416096655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f071c0b1630 msgr2=0x7f071c154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0721e1f700 1 --2- 172.21.15.28:0/1416096655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f071c0b1630 0x7f071c154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0721e1f700 1 -- 172.21.15.28:0/1416096655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0714009a70 con 0x7f071c165670 2023-11-28T11:09:11.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0721e1f700 1 --2- 172.21.15.28:0/1416096655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f071c165670 0x7f071c0c9290 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0714014040 tx=0x7f0714004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0711ffb700 1 -- 172.21.15.28:0/1416096655 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0714004510 con 0x7f071c165670 2023-11-28T11:09:11.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0711ffb700 1 -- 172.21.15.28:0/1416096655 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0714020070 con 0x7f071c165670 2023-11-28T11:09:11.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0723882700 1 -- 172.21.15.28:0/1416096655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071c0cb5b0 con 0x7f071c165670 2023-11-28T11:09:11.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.449+0000 7f0711ffb700 1 -- 172.21.15.28:0/1416096655 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07140048f0 con 0x7f071c165670 2023-11-28T11:09:11.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.450+0000 7f0723882700 1 -- 172.21.15.28:0/1416096655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f071c0cf0d0 con 0x7f071c165670 2023-11-28T11:09:11.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.451+0000 7f0723882700 1 -- 172.21.15.28:0/1416096655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071c0ce140 con 0x7f071c165670 2023-11-28T11:09:11.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.451+0000 7f0711ffb700 1 -- 172.21.15.28:0/1416096655 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0714012430 con 0x7f071c165670 2023-11-28T11:09:11.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.451+0000 7f0711ffb700 1 --2- 172.21.15.28:0/1416096655 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f07000682f0 0x7f070006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.452+0000 7f0711ffb700 1 -- 172.21.15.28:0/1416096655 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0714022070 con 0x7f071c165670 2023-11-28T11:09:11.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.452+0000 7f072161e700 1 --2- 172.21.15.28:0/1416096655 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f07000682f0 0x7f070006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.452+0000 7f072161e700 1 --2- 172.21.15.28:0/1416096655 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f07000682f0 0x7f070006a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f070c005910 tx=0x7f070c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.456+0000 7f0711ffb700 1 -- 172.21.15.28:0/1416096655 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f07140b95e0 con 0x7f071c165670 2023-11-28T11:09:11.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.647+0000 7f0723882700 1 -- 172.21.15.28:0/1416096655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f071c15ab30 con 0x7f071c165670 2023-11-28T11:09:11.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.648+0000 7f0711ffb700 1 -- 172.21.15.28:0/1416096655 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f071c15ab30 con 0x7f071c165670 2023-11-28T11:09:11.648 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:09:11.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.649+0000 7f0723882700 1 -- 172.21.15.28:0/1416096655 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f07000682f0 msgr2=0x7f070006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.649+0000 7f0723882700 1 --2- 172.21.15.28:0/1416096655 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f07000682f0 0x7f070006a7a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f070c005910 tx=0x7f070c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:11.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.649+0000 7f0723882700 1 -- 172.21.15.28:0/1416096655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f071c165670 msgr2=0x7f071c0c9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.649+0000 7f0723882700 1 --2- 172.21.15.28:0/1416096655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f071c165670 0x7f071c0c9290 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0714014040 tx=0x7f0714004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.650+0000 7f0723882700 1 -- 172.21.15.28:0/1416096655 shutdown_connections 2023-11-28T11:09:11.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.650+0000 7f0723882700 1 --2- 172.21.15.28:0/1416096655 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f07000682f0 0x7f070006a7a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.650+0000 7f0723882700 1 --2- 172.21.15.28:0/1416096655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f071c165670 0x7f071c0c9290 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.650+0000 7f0723882700 1 --2- 172.21.15.28:0/1416096655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f071c0b1630 0x7f071c154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.650+0000 7f0723882700 1 --2- 172.21.15.28:0/1416096655 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f071c0cf5f0 0x7f071c153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.650+0000 7f0723882700 1 -- 172.21.15.28:0/1416096655 >> 172.21.15.28:0/1416096655 conn(0x7f071c0cc630 msgr2=0x7f071c0cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.650+0000 7f0723882700 1 -- 172.21.15.28:0/1416096655 shutdown_connections 2023-11-28T11:09:11.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.650+0000 7f0723882700 1 -- 172.21.15.28:0/1416096655 wait complete. 2023-11-28T11:09:11.662 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:11.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.788+0000 7f5fc8e3f700 1 Processor -- start 2023-11-28T11:09:11.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.788+0000 7f5fc8e3f700 1 -- start start 2023-11-28T11:09:11.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.789+0000 7f5fc8e3f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fc4159500 0x7f5fc40cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.789+0000 7f5fc8e3f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5fc415dbf0 0x7f5fc40ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.789+0000 7f5fc8e3f700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc415e040 con 0x7f5fc40b1630 2023-11-28T11:09:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.789+0000 7f5fc8e3f700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc40cede0 con 0x7f5fc415dbf0 2023-11-28T11:09:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.789+0000 7f5fc8e3f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc40ceff0 con 0x7f5fc4159500 2023-11-28T11:09:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.790+0000 7f5fc1d9b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5fc415dbf0 0x7f5fc40ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.790+0000 7f5fc2d9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5fc40b1630 0x7f5fc406bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35492/0 (socket says 172.21.15.28:35492) 2023-11-28T11:09:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.790+0000 7f5fc2d9d700 1 -- 172.21.15.28:0/2212577681 learned_addr learned my addr 172.21.15.28:0/2212577681 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.790+0000 7f5fc259c700 1 --2- 172.21.15.28:0/2212577681 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fc4159500 0x7f5fc40cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.790+0000 7f5fc1d9b700 1 -- 172.21.15.28:0/2212577681 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fc4159500 msgr2=0x7f5fc40cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.790+0000 7f5fc1d9b700 1 --2- 172.21.15.28:0/2212577681 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fc4159500 0x7f5fc40cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.790+0000 7f5fc1d9b700 1 -- 172.21.15.28:0/2212577681 >> v1:172.21.15.28:6789/0 conn(0x7f5fc40b1630 legacy=0x7f5fc406bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.790+0000 7f5fc1d9b700 1 -- 172.21.15.28:0/2212577681 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc40cf2d0 con 0x7f5fc415dbf0 2023-11-28T11:09:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.790+0000 7f5fc159a700 1 -- 172.21.15.28:0/2212577681 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2159885230 0 0) 0x7f5fc415e040 con 0x7f5fc40b1630 2023-11-28T11:09:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.790+0000 7f5fc1d9b700 1 --2- 172.21.15.28:0/2212577681 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fc415dbf0 0x7f5fc40ce8a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009840 tx=0x7f5fac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b98bb7baa16a9892 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.790+0000 7f5fc159a700 1 -- 172.21.15.28:0/2212577681 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac011830 con 0x7f5fc415dbf0 2023-11-28T11:09:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.790+0000 7f5fc159a700 1 -- 172.21.15.28:0/2212577681 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac011990 con 0x7f5fc415dbf0 2023-11-28T11:09:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.790+0000 7f5fc159a700 1 -- 172.21.15.28:0/2212577681 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac011d20 con 0x7f5fc415dbf0 2023-11-28T11:09:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.791+0000 7f5fc8e3f700 1 -- 172.21.15.28:0/2212577681 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fc415dbf0 msgr2=0x7f5fc40ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.791+0000 7f5fc8e3f700 1 --2- 172.21.15.28:0/2212577681 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fc415dbf0 0x7f5fc40ce8a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f5fac009840 tx=0x7f5fac009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:11.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.791+0000 7f5fc8e3f700 1 -- 172.21.15.28:0/2212577681 shutdown_connections 2023-11-28T11:09:11.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.791+0000 7f5fc8e3f700 1 --2- 172.21.15.28:0/2212577681 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fc4159500 0x7f5fc40cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.791+0000 7f5fc8e3f700 1 --2- 172.21.15.28:0/2212577681 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fc415dbf0 0x7f5fc40ce8a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.791+0000 7f5fc8e3f700 1 -- 172.21.15.28:0/2212577681 >> 172.21.15.28:0/2212577681 conn(0x7f5fc40cc630 msgr2=0x7f5fc40cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.791+0000 7f5fc8e3f700 1 -- 172.21.15.28:0/2212577681 shutdown_connections 2023-11-28T11:09:11.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.791+0000 7f5fc8e3f700 1 -- 172.21.15.28:0/2212577681 wait complete. 2023-11-28T11:09:11.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.793+0000 7f5fc8e3f700 1 Processor -- start 2023-11-28T11:09:11.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.793+0000 7f5fc8e3f700 1 -- start start 2023-11-28T11:09:11.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc8e3f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fc40b1630 0x7f5fc414c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc8e3f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fc4159500 0x7f5fc414df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc8e3f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fc415dbf0 0x7f5fc40c32c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc8e3f700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc414ae00 con 0x7f5fc40b1630 2023-11-28T11:09:11.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc8e3f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc414e4a0 con 0x7f5fc4159500 2023-11-28T11:09:11.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc8e3f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc414cec0 con 0x7f5fc415dbf0 2023-11-28T11:09:11.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc259c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fc40b1630 0x7f5fc414c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc259c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fc40b1630 0x7f5fc414c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47438/0 (socket says 172.21.15.28:47438) 2023-11-28T11:09:11.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc259c700 1 -- 172.21.15.28:0/2720278704 learned_addr learned my addr 172.21.15.28:0/2720278704 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:11.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc1d9b700 1 --2- 172.21.15.28:0/2720278704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fc4159500 0x7f5fc414df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc2d9d700 1 --2- 172.21.15.28:0/2720278704 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fc415dbf0 0x7f5fc40c32c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc259c700 1 -- 172.21.15.28:0/2720278704 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fc415dbf0 msgr2=0x7f5fc40c32c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc259c700 1 --2- 172.21.15.28:0/2720278704 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fc415dbf0 0x7f5fc40c32c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc259c700 1 -- 172.21.15.28:0/2720278704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fc4159500 msgr2=0x7f5fc414df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc259c700 1 --2- 172.21.15.28:0/2720278704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fc4159500 0x7f5fc414df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc259c700 1 -- 172.21.15.28:0/2720278704 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac0094a0 con 0x7f5fc40b1630 2023-11-28T11:09:11.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc259c700 1 --2- 172.21.15.28:0/2720278704 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fc40b1630 0x7f5fc414c7c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400e840 tx=0x7f5fb400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fbaffd700 1 -- 172.21.15.28:0/2720278704 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4013610 con 0x7f5fc40b1630 2023-11-28T11:09:11.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fbaffd700 1 -- 172.21.15.28:0/2720278704 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb4013770 con 0x7f5fc40b1630 2023-11-28T11:09:11.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fc8e3f700 1 -- 172.21.15.28:0/2720278704 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc415d3f0 con 0x7f5fc40b1630 2023-11-28T11:09:11.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.794+0000 7f5fbaffd700 1 -- 172.21.15.28:0/2720278704 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb4013b20 con 0x7f5fc40b1630 2023-11-28T11:09:11.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.795+0000 7f5fc8e3f700 1 -- 172.21.15.28:0/2720278704 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc415d110 con 0x7f5fc40b1630 2023-11-28T11:09:11.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.796+0000 7f5fc8e3f700 1 -- 172.21.15.28:0/2720278704 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc415d6d0 con 0x7f5fc40b1630 2023-11-28T11:09:11.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.796+0000 7f5fbaffd700 1 -- 172.21.15.28:0/2720278704 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5fb4023070 con 0x7f5fc40b1630 2023-11-28T11:09:11.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.796+0000 7f5fbaffd700 1 --2- 172.21.15.28:0/2720278704 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fa40682f0 0x7f5fa406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:11.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.796+0000 7f5fbaffd700 1 -- 172.21.15.28:0/2720278704 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5fb4085bd0 con 0x7f5fc40b1630 2023-11-28T11:09:11.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.797+0000 7f5fc1d9b700 1 --2- 172.21.15.28:0/2720278704 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fa40682f0 0x7f5fa406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:11.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.797+0000 7f5fc1d9b700 1 --2- 172.21.15.28:0/2720278704 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fa40682f0 0x7f5fa406a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0090b0 tx=0x7f5fac008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:11.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.801+0000 7f5fbaffd700 1 -- 172.21.15.28:0/2720278704 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5fb4052140 con 0x7f5fc40b1630 2023-11-28T11:09:11.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.994+0000 7f5fc8e3f700 1 -- 172.21.15.28:0/2720278704 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5fc414ddb0 con 0x7f5fc40b1630 2023-11-28T11:09:11.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.994+0000 7f5fbaffd700 1 -- 172.21.15.28:0/2720278704 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5fb4055760 con 0x7f5fc40b1630 2023-11-28T11:09:11.995 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:11.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.996+0000 7f5fc8e3f700 1 -- 172.21.15.28:0/2720278704 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fa40682f0 msgr2=0x7f5fa406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.996+0000 7f5fc8e3f700 1 --2- 172.21.15.28:0/2720278704 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fa40682f0 0x7f5fa406a7a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5fac0090b0 tx=0x7f5fac008030 comp rx=0 tx=0).stop 2023-11-28T11:09:11.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.996+0000 7f5fc8e3f700 1 -- 172.21.15.28:0/2720278704 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fc40b1630 msgr2=0x7f5fc414c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:11.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.996+0000 7f5fc8e3f700 1 --2- 172.21.15.28:0/2720278704 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fc40b1630 0x7f5fc414c7c0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f5fb400e840 tx=0x7f5fb400ec00 comp rx=0 tx=0).stop 2023-11-28T11:09:11.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.996+0000 7f5fc8e3f700 1 -- 172.21.15.28:0/2720278704 shutdown_connections 2023-11-28T11:09:11.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.996+0000 7f5fc8e3f700 1 --2- 172.21.15.28:0/2720278704 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fa40682f0 0x7f5fa406a7a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.996+0000 7f5fc8e3f700 1 --2- 172.21.15.28:0/2720278704 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fc40b1630 0x7f5fc414c7c0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.996+0000 7f5fc8e3f700 1 --2- 172.21.15.28:0/2720278704 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fc415dbf0 0x7f5fc40c32c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.996+0000 7f5fc8e3f700 1 --2- 172.21.15.28:0/2720278704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fc4159500 0x7f5fc414df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.996+0000 7f5fc8e3f700 1 -- 172.21.15.28:0/2720278704 >> 172.21.15.28:0/2720278704 conn(0x7f5fc40cc630 msgr2=0x7f5fc40af9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.996+0000 7f5fc8e3f700 1 -- 172.21.15.28:0/2720278704 shutdown_connections 2023-11-28T11:09:11.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:11.997+0000 7f5fc8e3f700 1 -- 172.21.15.28:0/2720278704 wait complete. 2023-11-28T11:09:12.009 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:12.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.135+0000 7f059cf9d700 1 Processor -- start 2023-11-28T11:09:12.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.135+0000 7f059cf9d700 1 -- start start 2023-11-28T11:09:12.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.135+0000 7f059cf9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f05980cb050 0x7f05980b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.136+0000 7f059cf9d700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059815e820 con 0x7f059815a150 2023-11-28T11:09:12.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.136+0000 7f059cf9d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059815ecb0 con 0x7f05980b29c0 2023-11-28T11:09:12.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.136+0000 7f059cf9d700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05980b2e40 con 0x7f05980cb050 2023-11-28T11:09:12.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.136+0000 7f0596ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f05980cb050 0x7f05980b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.136+0000 7f05977fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f059815a150 0x7f05980ccab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35504/0 (socket says 172.21.15.28:35504) 2023-11-28T11:09:12.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.136+0000 7f05977fe700 1 -- 172.21.15.28:0/3958195172 learned_addr learned my addr 172.21.15.28:0/3958195172 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:12.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.136+0000 7f0596ffd700 1 -- 172.21.15.28:0/3958195172 >> v1:172.21.15.59:6789/0 conn(0x7f05980b29c0 legacy=0x7f059806dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.136+0000 7f0596ffd700 1 -- 172.21.15.28:0/3958195172 >> v1:172.21.15.28:6789/0 conn(0x7f059815a150 legacy=0x7f05980ccab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.136+0000 7f05967fc700 1 -- 172.21.15.28:0/3958195172 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3001170312 0 0) 0x7f059815e820 con 0x7f059815a150 2023-11-28T11:09:12.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.136+0000 7f0596ffd700 1 -- 172.21.15.28:0/3958195172 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059815def0 con 0x7f05980cb050 2023-11-28T11:09:12.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.136+0000 7f0596ffd700 1 --2- 172.21.15.28:0/3958195172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05980cb050 0x7f05980b2480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f058c009840 tx=0x7f058c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3281d1c707e02fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.136+0000 7f05967fc700 1 -- 172.21.15.28:0/3958195172 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c011830 con 0x7f05980cb050 2023-11-28T11:09:12.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.136+0000 7f05967fc700 1 -- 172.21.15.28:0/3958195172 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058c011990 con 0x7f05980cb050 2023-11-28T11:09:12.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.136+0000 7f05967fc700 1 -- 172.21.15.28:0/3958195172 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f058c011d20 con 0x7f05980cb050 2023-11-28T11:09:12.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.137+0000 7f059cf9d700 1 -- 172.21.15.28:0/3958195172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05980cb050 msgr2=0x7f05980b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.137+0000 7f059cf9d700 1 --2- 172.21.15.28:0/3958195172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05980cb050 0x7f05980b2480 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f058c009840 tx=0x7f058c009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:12.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.137+0000 7f059cf9d700 1 -- 172.21.15.28:0/3958195172 shutdown_connections 2023-11-28T11:09:12.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.137+0000 7f059cf9d700 1 --2- 172.21.15.28:0/3958195172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05980cb050 0x7f05980b2480 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.137+0000 7f059cf9d700 1 -- 172.21.15.28:0/3958195172 >> 172.21.15.28:0/3958195172 conn(0x7f05980cd480 msgr2=0x7f05980cd850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.137+0000 7f059cf9d700 1 -- 172.21.15.28:0/3958195172 shutdown_connections 2023-11-28T11:09:12.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.137+0000 7f059cf9d700 1 -- 172.21.15.28:0/3958195172 wait complete. 2023-11-28T11:09:12.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.138+0000 7f059cf9d700 1 Processor -- start 2023-11-28T11:09:12.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.139+0000 7f059cf9d700 1 -- start start 2023-11-28T11:09:12.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f059cf9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05980b29c0 0x7f0598159000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f059cf9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05980cb050 0x7f05980ce5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f059cf9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f059815a150 0x7f05980ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f059cf9d700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0598159540 con 0x7f05980b29c0 2023-11-28T11:09:12.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f059cf9d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05980c7ad0 con 0x7f059815a150 2023-11-28T11:09:12.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f059cf9d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05980c8930 con 0x7f05980cb050 2023-11-28T11:09:12.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f05977fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05980b29c0 0x7f0598159000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f05977fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05980b29c0 0x7f0598159000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47452/0 (socket says 172.21.15.28:47452) 2023-11-28T11:09:12.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f05977fe700 1 -- 172.21.15.28:0/1966676546 learned_addr learned my addr 172.21.15.28:0/1966676546 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:12.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f0596ffd700 1 --2- 172.21.15.28:0/1966676546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05980cb050 0x7f05980ce5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f0597fff700 1 --2- 172.21.15.28:0/1966676546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f059815a150 0x7f05980ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f05977fe700 1 -- 172.21.15.28:0/1966676546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05980cb050 msgr2=0x7f05980ce5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f05977fe700 1 --2- 172.21.15.28:0/1966676546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05980cb050 0x7f05980ce5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f05977fe700 1 -- 172.21.15.28:0/1966676546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f059815a150 msgr2=0x7f05980ceaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f05977fe700 1 --2- 172.21.15.28:0/1966676546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f059815a150 0x7f05980ceaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f05977fe700 1 -- 172.21.15.28:0/1966676546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f058c0094a0 con 0x7f05980b29c0 2023-11-28T11:09:12.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f0597fff700 1 --2- 172.21.15.28:0/1966676546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f059815a150 0x7f05980ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:12.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f05977fe700 1 --2- 172.21.15.28:0/1966676546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05980b29c0 0x7f0598159000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f058800c550 tx=0x7f058800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.140+0000 7f057ffff700 1 -- 172.21.15.28:0/1966676546 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588004c80 con 0x7f05980b29c0 2023-11-28T11:09:12.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.141+0000 7f057ffff700 1 -- 172.21.15.28:0/1966676546 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0588004de0 con 0x7f05980b29c0 2023-11-28T11:09:12.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.141+0000 7f059cf9d700 1 -- 172.21.15.28:0/1966676546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059815e4b0 con 0x7f05980b29c0 2023-11-28T11:09:12.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.141+0000 7f057ffff700 1 -- 172.21.15.28:0/1966676546 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0588007310 con 0x7f05980b29c0 2023-11-28T11:09:12.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.141+0000 7f059cf9d700 1 -- 172.21.15.28:0/1966676546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059815e1d0 con 0x7f05980b29c0 2023-11-28T11:09:12.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.142+0000 7f059cf9d700 1 -- 172.21.15.28:0/1966676546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059815a520 con 0x7f05980b29c0 2023-11-28T11:09:12.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.142+0000 7f057ffff700 1 -- 172.21.15.28:0/1966676546 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f058801a020 con 0x7f05980b29c0 2023-11-28T11:09:12.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.142+0000 7f057ffff700 1 --2- 172.21.15.28:0/1966676546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f05840682f0 0x7f058406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.142+0000 7f057ffff700 1 -- 172.21.15.28:0/1966676546 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f0588085ae0 con 0x7f05980b29c0 2023-11-28T11:09:12.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.142+0000 7f0596ffd700 1 --2- 172.21.15.28:0/1966676546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f05840682f0 0x7f058406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.143+0000 7f0596ffd700 1 --2- 172.21.15.28:0/1966676546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f05840682f0 0x7f058406a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f058c0090b0 tx=0x7f058c01c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.147+0000 7f057ffff700 1 -- 172.21.15.28:0/1966676546 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0588051fc0 con 0x7f05980b29c0 2023-11-28T11:09:12.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.387+0000 7f059cf9d700 1 -- 172.21.15.28:0/1966676546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f05980c8c10 con 0x7f05980b29c0 2023-11-28T11:09:12.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.387+0000 7f057ffff700 1 -- 172.21.15.28:0/1966676546 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4644 (secure 0 0 0) 0x7f05880555e0 con 0x7f05980b29c0 2023-11-28T11:09:12.389 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:12.389 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/3112877467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3112877467},{"type":"v1","addr":"172.21.15.59:6834","nonce":3112877467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3448236489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3448236489},{"type":"v1","addr":"172.21.15.28:6836","nonce":3448236489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4232269940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4232269940},{"type":"v1","addr":"172.21.15.28:6837","nonce":4232269940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:45.924816+0000","modified":"2023-11-28T11:09:03.234785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/3293053261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3293053261},{"type":"v1","addr":"172.21.15.59:6835","nonce":3293053261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:09:12.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.390+0000 7f059cf9d700 1 -- 172.21.15.28:0/1966676546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f05840682f0 msgr2=0x7f058406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.390+0000 7f059cf9d700 1 --2- 172.21.15.28:0/1966676546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f05840682f0 0x7f058406a7a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f058c0090b0 tx=0x7f058c01c040 comp rx=0 tx=0).stop 2023-11-28T11:09:12.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.390+0000 7f059cf9d700 1 -- 172.21.15.28:0/1966676546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05980b29c0 msgr2=0x7f0598159000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.390+0000 7f059cf9d700 1 --2- 172.21.15.28:0/1966676546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05980b29c0 0x7f0598159000 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f058800c550 tx=0x7f058800c860 comp rx=0 tx=0).stop 2023-11-28T11:09:12.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.390+0000 7f059cf9d700 1 -- 172.21.15.28:0/1966676546 shutdown_connections 2023-11-28T11:09:12.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.390+0000 7f059cf9d700 1 --2- 172.21.15.28:0/1966676546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f05840682f0 0x7f058406a7a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.390+0000 7f059cf9d700 1 --2- 172.21.15.28:0/1966676546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05980b29c0 0x7f0598159000 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.390+0000 7f059cf9d700 1 --2- 172.21.15.28:0/1966676546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f059815a150 0x7f05980ceaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.390+0000 7f059cf9d700 1 --2- 172.21.15.28:0/1966676546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05980cb050 0x7f05980ce5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.390+0000 7f059cf9d700 1 -- 172.21.15.28:0/1966676546 >> 172.21.15.28:0/1966676546 conn(0x7f05980cd480 msgr2=0x7f059806dd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.390+0000 7f059cf9d700 1 -- 172.21.15.28:0/1966676546 shutdown_connections 2023-11-28T11:09:12.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.390+0000 7f059cf9d700 1 -- 172.21.15.28:0/1966676546 wait complete. 2023-11-28T11:09:12.393 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 9 2023-11-28T11:09:12.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 9, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:08:45.924816+0000', 'modified': '2023-11-28T11:09:03.234785+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4296}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4296': {'gid': 4296, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6835/3293053261', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6833', 'nonce': 3293053261}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 3293053261}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4, 5], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:09:12.806 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:09:12.807 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T11:09:12.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.939+0000 7ffa6fc5c700 1 Processor -- start 2023-11-28T11:09:12.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.940+0000 7ffa6fc5c700 1 -- start start 2023-11-28T11:09:12.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.940+0000 7ffa6fc5c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa680b4fe0 0x7ffa6815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.940+0000 7ffa6fc5c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffa680cd4a0 0x7ffa680d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.941+0000 7ffa6fc5c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa6806abb0 con 0x7ffa6815a230 2023-11-28T11:09:12.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.941+0000 7ffa6fc5c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa6815a600 con 0x7ffa680cd4a0 2023-11-28T11:09:12.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.941+0000 7ffa6fc5c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa6806a280 con 0x7ffa680b4fe0 2023-11-28T11:09:12.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.941+0000 7ffa6e1f9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ffa680cd4a0 0x7ffa680d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.941+0000 7ffa6d1f7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa680b4fe0 0x7ffa6815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.941+0000 7ffa6d9f8700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ffa6815a230 0x7ffa680b48d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:46304/0 (socket says 172.21.15.28:46304) 2023-11-28T11:09:12.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.941+0000 7ffa6d9f8700 1 -- 172.21.15.28:0/3358954195 learned_addr learned my addr 172.21.15.28:0/3358954195 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:12.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.941+0000 7ffa6c9f6700 1 -- 172.21.15.28:0/3358954195 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990971738 0 0) 0x7ffa6806abb0 con 0x7ffa6815a230 2023-11-28T11:09:12.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.941+0000 7ffa6c9f6700 1 -- 172.21.15.28:0/3358954195 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffa540035b0 con 0x7ffa6815a230 2023-11-28T11:09:12.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.941+0000 7ffa6d1f7700 1 -- 172.21.15.28:0/3358954195 >> v2:172.21.15.59:3300/0 conn(0x7ffa680cd4a0 msgr2=0x7ffa680d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.941+0000 7ffa6d1f7700 1 --2- 172.21.15.28:0/3358954195 >> v2:172.21.15.59:3300/0 conn(0x7ffa680cd4a0 0x7ffa680d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.941+0000 7ffa6d1f7700 1 -- 172.21.15.28:0/3358954195 >> v1:172.21.15.59:6789/0 conn(0x7ffa6815a230 legacy=0x7ffa680b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.941+0000 7ffa6d1f7700 1 -- 172.21.15.28:0/3358954195 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa6806a560 con 0x7ffa680b4fe0 2023-11-28T11:09:12.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.941+0000 7ffa6e1f9700 1 --2- 172.21.15.28:0/3358954195 >> v2:172.21.15.59:3300/0 conn(0x7ffa680cd4a0 0x7ffa680d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:12.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.941+0000 7ffa6d1f7700 1 --2- 172.21.15.28:0/3358954195 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa680b4fe0 0x7ffa6815d6b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009840 tx=0x7ffa58009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28b79e443b1138b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.942+0000 7ffa6c9f6700 1 -- 172.21.15.28:0/3358954195 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58011830 con 0x7ffa680b4fe0 2023-11-28T11:09:12.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.942+0000 7ffa6c9f6700 1 -- 172.21.15.28:0/3358954195 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa58011990 con 0x7ffa680b4fe0 2023-11-28T11:09:12.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.942+0000 7ffa6c9f6700 1 -- 172.21.15.28:0/3358954195 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa58011d20 con 0x7ffa680b4fe0 2023-11-28T11:09:12.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.942+0000 7ffa6fc5c700 1 -- 172.21.15.28:0/3358954195 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa680b4fe0 msgr2=0x7ffa6815d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.942+0000 7ffa6fc5c700 1 --2- 172.21.15.28:0/3358954195 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa680b4fe0 0x7ffa6815d6b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ffa58009840 tx=0x7ffa58009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:12.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.942+0000 7ffa6fc5c700 1 -- 172.21.15.28:0/3358954195 shutdown_connections 2023-11-28T11:09:12.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.942+0000 7ffa6fc5c700 1 --2- 172.21.15.28:0/3358954195 >> v2:172.21.15.59:3300/0 conn(0x7ffa680cd4a0 0x7ffa680d0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.942+0000 7ffa6fc5c700 1 --2- 172.21.15.28:0/3358954195 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa680b4fe0 0x7ffa6815d6b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.942+0000 7ffa6fc5c700 1 -- 172.21.15.28:0/3358954195 >> 172.21.15.28:0/3358954195 conn(0x7ffa680cf8d0 msgr2=0x7ffa680cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.942+0000 7ffa6fc5c700 1 -- 172.21.15.28:0/3358954195 shutdown_connections 2023-11-28T11:09:12.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.942+0000 7ffa6fc5c700 1 -- 172.21.15.28:0/3358954195 wait complete. 2023-11-28T11:09:12.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.944+0000 7ffa6fc5c700 1 Processor -- start 2023-11-28T11:09:12.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.944+0000 7ffa6fc5c700 1 -- start start 2023-11-28T11:09:12.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.944+0000 7ffa6fc5c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa680b4fe0 0x7ffa680c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.944+0000 7ffa6fc5c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa680cd4a0 0x7ffa68152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.944+0000 7ffa6fc5c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa6815a230 0x7ffa68152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.944+0000 7ffa6fc5c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa681f2380 con 0x7ffa680b4fe0 2023-11-28T11:09:12.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.944+0000 7ffa6fc5c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa681f2620 con 0x7ffa680cd4a0 2023-11-28T11:09:12.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.944+0000 7ffa6fc5c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa681f2900 con 0x7ffa6815a230 2023-11-28T11:09:12.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.944+0000 7ffa6d9f8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa680b4fe0 0x7ffa680c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.945+0000 7ffa6d9f8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa680b4fe0 0x7ffa680c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47466/0 (socket says 172.21.15.28:47466) 2023-11-28T11:09:12.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.945+0000 7ffa6d9f8700 1 -- 172.21.15.28:0/3712810644 learned_addr learned my addr 172.21.15.28:0/3712810644 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:12.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.945+0000 7ffa6d1f7700 1 --2- 172.21.15.28:0/3712810644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa680cd4a0 0x7ffa68152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.945+0000 7ffa6e1f9700 1 --2- 172.21.15.28:0/3712810644 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa6815a230 0x7ffa68152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.945+0000 7ffa6d9f8700 1 -- 172.21.15.28:0/3712810644 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa6815a230 msgr2=0x7ffa68152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.945+0000 7ffa6d9f8700 1 --2- 172.21.15.28:0/3712810644 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa6815a230 0x7ffa68152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.945+0000 7ffa6d9f8700 1 -- 172.21.15.28:0/3712810644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa680cd4a0 msgr2=0x7ffa68152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.945+0000 7ffa6d9f8700 1 --2- 172.21.15.28:0/3712810644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa680cd4a0 0x7ffa68152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.945+0000 7ffa6d9f8700 1 -- 172.21.15.28:0/3712810644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa60009a70 con 0x7ffa680b4fe0 2023-11-28T11:09:12.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.945+0000 7ffa6d9f8700 1 --2- 172.21.15.28:0/3712810644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa680b4fe0 0x7ffa680c72e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffa64001eb0 tx=0x7ffa6400c7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.945+0000 7ffa5e7fc700 1 -- 172.21.15.28:0/3712810644 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6400cea0 con 0x7ffa680b4fe0 2023-11-28T11:09:12.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.945+0000 7ffa5e7fc700 1 -- 172.21.15.28:0/3712810644 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa64005d70 con 0x7ffa680b4fe0 2023-11-28T11:09:12.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.945+0000 7ffa6fc5c700 1 -- 172.21.15.28:0/3712810644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa580094a0 con 0x7ffa680b4fe0 2023-11-28T11:09:12.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.945+0000 7ffa5e7fc700 1 -- 172.21.15.28:0/3712810644 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa640072f0 con 0x7ffa680b4fe0 2023-11-28T11:09:12.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.945+0000 7ffa6fc5c700 1 -- 172.21.15.28:0/3712810644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa680cd920 con 0x7ffa680b4fe0 2023-11-28T11:09:12.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.947+0000 7ffa5e7fc700 1 -- 172.21.15.28:0/3712810644 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffa64019030 con 0x7ffa680b4fe0 2023-11-28T11:09:12.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.947+0000 7ffa5e7fc700 1 --2- 172.21.15.28:0/3712810644 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffa4c0682f0 0x7ffa4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.947+0000 7ffa5e7fc700 1 -- 172.21.15.28:0/3712810644 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ffa640852f0 con 0x7ffa680b4fe0 2023-11-28T11:09:12.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.948+0000 7ffa6d1f7700 1 --2- 172.21.15.28:0/3712810644 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffa4c0682f0 0x7ffa4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.948+0000 7ffa6d1f7700 1 --2- 172.21.15.28:0/3712810644 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffa4c0682f0 0x7ffa4c06a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffa680b53d0 tx=0x7ffa58008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.949+0000 7ffa7bde5b80 1 Processor -- start 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.950+0000 7ffa7bde5b80 1 -- start start 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.950+0000 7ffa7bde5b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2ef810 0x55569f2c5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.950+0000 7ffa7bde5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ed130 0x55569f2efbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.950+0000 7ffa7bde5b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2f0120 0x55569f30b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.950+0000 7ffa7bde5b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569f1afbc0 con 0x55569f2ef810 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.950+0000 7ffa7bde5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569f21e350 con 0x55569f2ed130 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.950+0000 7ffa7bde5b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55569f217880 con 0x55569f2f0120 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.950+0000 7ffa6d9f8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2ef810 0x55569f2c5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.950+0000 7ffa6d9f8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2ef810 0x55569f2c5a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47474/0 (socket says 172.21.15.28:47474) 2023-11-28T11:09:12.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.950+0000 7ffa6d9f8700 1 -- 172.21.15.28:0/640739896 learned_addr learned my addr 172.21.15.28:0/640739896 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.951+0000 7ffa6d9f8700 1 -- 172.21.15.28:0/640739896 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2f0120 msgr2=0x55569f30b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.951+0000 7ffa6d1f7700 1 --2- 172.21.15.28:0/640739896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ed130 0x55569f2efbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.951+0000 7ffa6e1f9700 1 --2- 172.21.15.28:0/640739896 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2f0120 0x55569f30b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.951+0000 7ffa6d9f8700 1 --2- 172.21.15.28:0/640739896 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2f0120 0x55569f30b770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.951+0000 7ffa6d9f8700 1 -- 172.21.15.28:0/640739896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ed130 msgr2=0x55569f2efbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.951+0000 7ffa6d9f8700 1 --2- 172.21.15.28:0/640739896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ed130 0x55569f2efbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.951+0000 7ffa6d9f8700 1 -- 172.21.15.28:0/640739896 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55569f0995e0 con 0x55569f2ef810 2023-11-28T11:09:12.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.951+0000 7ffa6d9f8700 1 --2- 172.21.15.28:0/640739896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2ef810 0x55569f2c5a20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa6405cf90 tx=0x7ffa6405d540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53790e091362d97 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.951+0000 7ffa4b7fe700 1 -- 172.21.15.28:0/640739896 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa640850d0 con 0x55569f2ef810 2023-11-28T11:09:12.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.951+0000 7ffa4b7fe700 1 -- 172.21.15.28:0/640739896 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6405d030 con 0x55569f2ef810 2023-11-28T11:09:12.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.951+0000 7ffa4b7fe700 1 -- 172.21.15.28:0/640739896 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa64068b70 con 0x55569f2ef810 2023-11-28T11:09:12.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.952+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/640739896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2ef810 msgr2=0x55569f2c5a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.952+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/640739896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2ef810 0x55569f2c5a20 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7ffa6405cf90 tx=0x7ffa6405d540 comp rx=0 tx=0).stop 2023-11-28T11:09:12.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.952+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/640739896 shutdown_connections 2023-11-28T11:09:12.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.952+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/640739896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2ef810 0x55569f2c5a20 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.952+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/640739896 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2f0120 0x55569f30b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.952+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/640739896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ed130 0x55569f2efbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.952+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/640739896 >> 172.21.15.28:0/640739896 conn(0x55569f2c2d60 msgr2=0x55569f2c36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.952+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/640739896 shutdown_connections 2023-11-28T11:09:12.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.952+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/640739896 wait complete. 2023-11-28T11:09:12.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.952+0000 7ffa7bde5b80 1 -- start start 2023-11-28T11:09:12.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.952+0000 7ffa7bde5b80 1 Processor -- start 2023-11-28T11:09:12.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa7bde5b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2ed130 0x55569f2e7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa7bde5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ef810 0x55569f30eda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa7bde5b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2f0120 0x55569f2ee5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa7bde5b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569f09b8d0 con 0x55569f2f0120 2023-11-28T11:09:12.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa7bde5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569f312480 con 0x55569f2ef810 2023-11-28T11:09:12.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa7bde5b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55569f2d6100 con 0x55569f2ed130 2023-11-28T11:09:12.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa7bde5b80 10 client.0 _ll_register_callbacks cb 0x55569f0bd240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:09:12.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa6e1f9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2f0120 0x55569f2ee5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa6e1f9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2f0120 0x55569f2ee5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47484/0 (socket says 172.21.15.28:47484) 2023-11-28T11:09:12.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa6e1f9700 1 -- 172.21.15.28:0/1788723515 learned_addr learned my addr 172.21.15.28:0/1788723515 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:12.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa6d9f8700 1 --2- 172.21.15.28:0/1788723515 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2ed130 0x55569f2e7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa6d1f7700 1 --2- 172.21.15.28:0/1788723515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ef810 0x55569f30eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa6e1f9700 1 -- 172.21.15.28:0/1788723515 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2ed130 msgr2=0x55569f2e7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa6e1f9700 1 --2- 172.21.15.28:0/1788723515 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2ed130 0x55569f2e7830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa6e1f9700 1 -- 172.21.15.28:0/1788723515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ef810 msgr2=0x55569f30eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa6e1f9700 1 --2- 172.21.15.28:0/1788723515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ef810 0x55569f30eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa6e1f9700 1 -- 172.21.15.28:0/1788723515 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa680cd920 con 0x55569f2f0120 2023-11-28T11:09:12.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa6d1f7700 1 --2- 172.21.15.28:0/1788723515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ef810 0x55569f30eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:12.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa6e1f9700 1 --2- 172.21.15.28:0/1788723515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2f0120 0x55569f2ee5f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffa60007e10 tx=0x7ffa60012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eacffe0c4f43649 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.954+0000 7ffa48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:09:12.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.955+0000 7ffa48ff9700 1 -- 172.21.15.28:0/1788723515 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000e040 con 0x55569f2f0120 2023-11-28T11:09:12.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.955+0000 7ffa48ff9700 1 -- 172.21.15.28:0/1788723515 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ffa60018470 con 0x55569f2f0120 2023-11-28T11:09:12.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.955+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/1788723515 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55569f1e5140 con 0x55569f2f0120 2023-11-28T11:09:12.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.955+0000 7ffa48ff9700 1 -- 172.21.15.28:0/1788723515 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa60021ea0 con 0x55569f2f0120 2023-11-28T11:09:12.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.955+0000 7ffa7bde5b80 10 client.4794 fetch_fsmap learned FSMap version 9 2023-11-28T11:09:12.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.955+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/1788723515 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55569f045d90 con 0x55569f2f0120 2023-11-28T11:09:12.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.955+0000 7ffa48ff9700 1 -- 172.21.15.28:0/1788723515 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55569f1e5140 con 0x55569f2f0120 2023-11-28T11:09:12.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.955+0000 7ffa48ff9700 1 -- 172.21.15.28:0/1788723515 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 9) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ffa60014040 con 0x55569f2f0120 2023-11-28T11:09:12.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.955+0000 7ffa7bde5b80 10 client.4794 fetch_fsmap finished waiting for FSMap version 9 2023-11-28T11:09:12.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.955+0000 7ffa7bde5b80 10 client.4794 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:09:12.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.955+0000 7ffa7bde5b80 20 client.4794 populate_metadata read hostname 'smithi028' 2023-11-28T11:09:12.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.955+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/1788723515 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x55569f310f50 0x55569f303840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.955+0000 7ffa7bde5b80 4 client.4794 mds_command: new command op to 4296 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:09:12.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.955+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/1788723515 --> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55569f09bf10 con 0x55569f310f50 2023-11-28T11:09:12.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.955+0000 7ffa6d9f8700 1 --2- 172.21.15.28:0/1788723515 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x55569f310f50 0x55569f303840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.956+0000 7ffa6d9f8700 1 --2- 172.21.15.28:0/1788723515 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x55569f310f50 0x55569f303840 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=969e268f7774fa05 server_cookie=26d2735fe9e024df in_seq=0 out_seq=0 2023-11-28T11:09:12.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.956+0000 7ffa48ff9700 10 client.4794 ms_handle_connect on v2:172.21.15.59:6833/3293053261 2023-11-28T11:09:12.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.956+0000 7ffa48ff9700 1 -- 172.21.15.28:0/1788723515 <== mds.0 v2:172.21.15.59:6833/3293053261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55569f09bf10 con 0x55569f310f50 2023-11-28T11:09:12.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.956+0000 7ffa48ff9700 10 client.4794 handle_command_reply: tid=0 2023-11-28T11:09:12.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.956+0000 7ffa7bde5b80 1 client.4794 shutdown 2023-11-28T11:09:12.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.957+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/1788723515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2f0120 msgr2=0x55569f2ee5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.957+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/1788723515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2f0120 0x55569f2ee5f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ffa60007e10 tx=0x7ffa60012520 comp rx=0 tx=0).stop 2023-11-28T11:09:12.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.957+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/1788723515 shutdown_connections 2023-11-28T11:09:12.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.957+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/1788723515 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x55569f310f50 0x55569f303840 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.957+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/1788723515 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2f0120 0x55569f2ee5f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.957+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/1788723515 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2ed130 0x55569f2e7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.957+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/1788723515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ef810 0x55569f30eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.957+0000 7ffa48ff9700 0 client.4794 ms_handle_reset on v2:172.21.15.59:6833/3293053261 2023-11-28T11:09:12.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.957+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/1788723515 >> 172.21.15.28:0/1788723515 conn(0x55569f2c2d60 msgr2=0x55569f2c5990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.957+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/1788723515 shutdown_connections 2023-11-28T11:09:12.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.957+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/1788723515 wait complete. 2023-11-28T11:09:12.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.957+0000 7ffa7bde5b80 20 client.4794 trim_cache size 0 max 16384 2023-11-28T11:09:12.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.975+0000 7ffa7bde5b80 1 Processor -- start 2023-11-28T11:09:12.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.975+0000 7ffa7bde5b80 1 -- start start 2023-11-28T11:09:12.979 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.975+0000 7ffa7bde5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ed130 0x55569f2d89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.979 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.975+0000 7ffa7bde5b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2ef810 0x55569f2d54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.979 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.975+0000 7ffa7bde5b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2f0120 0x55569f2d59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.979 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.975+0000 7ffa7bde5b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569f2c4230 con 0x55569f2f0120 2023-11-28T11:09:12.979 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.975+0000 7ffa7bde5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569f30f2f0 con 0x55569f2ed130 2023-11-28T11:09:12.979 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.975+0000 7ffa7bde5b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55569f2c04e0 con 0x55569f2ef810 2023-11-28T11:09:12.979 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.976+0000 7ffa6e1f9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2f0120 0x55569f2d59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.979 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.976+0000 7ffa6e1f9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2f0120 0x55569f2d59e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47492/0 (socket says 172.21.15.28:47492) 2023-11-28T11:09:12.979 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.976+0000 7ffa6e1f9700 1 -- 172.21.15.28:0/3393389439 learned_addr learned my addr 172.21.15.28:0/3393389439 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:12.979 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.976+0000 7ffa6d9f8700 1 --2- 172.21.15.28:0/3393389439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ed130 0x55569f2d89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.976+0000 7ffa6d1f7700 1 --2- 172.21.15.28:0/3393389439 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2ef810 0x55569f2d54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.976+0000 7ffa6e1f9700 1 -- 172.21.15.28:0/3393389439 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2ef810 msgr2=0x55569f2d54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.976+0000 7ffa6e1f9700 1 --2- 172.21.15.28:0/3393389439 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2ef810 0x55569f2d54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.976+0000 7ffa6e1f9700 1 -- 172.21.15.28:0/3393389439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ed130 msgr2=0x55569f2d89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.976+0000 7ffa6e1f9700 1 --2- 172.21.15.28:0/3393389439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ed130 0x55569f2d89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.976+0000 7ffa6e1f9700 1 -- 172.21.15.28:0/3393389439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55569f1de140 con 0x55569f2f0120 2023-11-28T11:09:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.976+0000 7ffa6e1f9700 1 --2- 172.21.15.28:0/3393389439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2f0120 0x55569f2d59e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffa60000f80 tx=0x7ffa60018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff6157c4a219ffc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.976+0000 7ffa4bfff700 1 -- 172.21.15.28:0/3393389439 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa6000e040 con 0x55569f2f0120 2023-11-28T11:09:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.976+0000 7ffa4bfff700 1 -- 172.21.15.28:0/3393389439 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa6000f040 con 0x55569f2f0120 2023-11-28T11:09:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.976+0000 7ffa4bfff700 1 -- 172.21.15.28:0/3393389439 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa60003c20 con 0x55569f2f0120 2023-11-28T11:09:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.977+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/3393389439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2f0120 msgr2=0x55569f2d59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.977+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/3393389439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2f0120 0x55569f2d59e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffa60000f80 tx=0x7ffa60018000 comp rx=0 tx=0).stop 2023-11-28T11:09:12.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.977+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/3393389439 shutdown_connections 2023-11-28T11:09:12.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.977+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/3393389439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2f0120 0x55569f2d59e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.977+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/3393389439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ed130 0x55569f2d89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.977+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/3393389439 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2ef810 0x55569f2d54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.977+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/3393389439 >> 172.21.15.28:0/3393389439 conn(0x55569f2c2d60 msgr2=0x55569f2c1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.977+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/3393389439 shutdown_connections 2023-11-28T11:09:12.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.977+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/3393389439 wait complete. 2023-11-28T11:09:12.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.977+0000 7ffa7bde5b80 1 -- start start 2023-11-28T11:09:12.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.977+0000 7ffa7bde5b80 1 Processor -- start 2023-11-28T11:09:12.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa7bde5b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2ed130 0x55569f2bee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa7bde5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ef810 0x55569f2d54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa7bde5b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2f0120 0x55569f2d59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa7bde5b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569f374730 con 0x55569f2ed130 2023-11-28T11:09:12.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa7bde5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55569f303960 con 0x55569f2ef810 2023-11-28T11:09:12.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa7bde5b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55569f373860 con 0x55569f2f0120 2023-11-28T11:09:12.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa7bde5b80 10 client.0 _ll_register_callbacks cb 0x55569f0bd240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:09:12.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa6d9f8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2ed130 0x55569f2bee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa6d9f8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2ed130 0x55569f2bee80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47494/0 (socket says 172.21.15.28:47494) 2023-11-28T11:09:12.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa6d9f8700 1 -- 172.21.15.28:0/4032131214 learned_addr learned my addr 172.21.15.28:0/4032131214 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:12.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa6d1f7700 1 --2- 172.21.15.28:0/4032131214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ef810 0x55569f2d54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa6e1f9700 1 --2- 172.21.15.28:0/4032131214 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2f0120 0x55569f2d59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa6d9f8700 1 -- 172.21.15.28:0/4032131214 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2f0120 msgr2=0x55569f2d59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa6d9f8700 1 --2- 172.21.15.28:0/4032131214 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2f0120 0x55569f2d59e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa6d9f8700 1 -- 172.21.15.28:0/4032131214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ef810 msgr2=0x55569f2d54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:12.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa6d9f8700 1 --2- 172.21.15.28:0/4032131214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ef810 0x55569f2d54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa6d9f8700 1 -- 172.21.15.28:0/4032131214 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55569f045d90 con 0x55569f2ed130 2023-11-28T11:09:12.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa6d1f7700 1 --2- 172.21.15.28:0/4032131214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ef810 0x55569f2d54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:12.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa6e1f9700 1 --2- 172.21.15.28:0/4032131214 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2f0120 0x55569f2d59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:12.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa6d9f8700 1 --2- 172.21.15.28:0/4032131214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2ed130 0x55569f2bee80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffa64055360 tx=0x7ffa64084000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d3fa213e462bac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:12.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:09:12.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.979+0000 7ffa48ff9700 1 -- 172.21.15.28:0/4032131214 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa640832c0 con 0x55569f2ed130 2023-11-28T11:09:12.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.980+0000 7ffa48ff9700 1 -- 172.21.15.28:0/4032131214 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7ffa6405d030 con 0x55569f2ed130 2023-11-28T11:09:12.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.980+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/4032131214 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55569f1d5160 con 0x55569f2ed130 2023-11-28T11:09:12.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.980+0000 7ffa48ff9700 1 -- 172.21.15.28:0/4032131214 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=9) v2 ==== 24+0+0 (secure 0 0 0) 0x7ffa6405d210 con 0x55569f2ed130 2023-11-28T11:09:12.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.980+0000 7ffa7bde5b80 10 client.4800 fetch_fsmap learned FSMap version 9 2023-11-28T11:09:12.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.980+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/4032131214 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=9,monmap=2+}) v3 -- 0x55569f06c200 con 0x55569f2ed130 2023-11-28T11:09:12.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.980+0000 7ffa48ff9700 1 -- 172.21.15.28:0/4032131214 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55569f1d5160 con 0x55569f2ed130 2023-11-28T11:09:12.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.980+0000 7ffa48ff9700 1 -- 172.21.15.28:0/4032131214 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 9) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ffa64068b70 con 0x55569f2ed130 2023-11-28T11:09:12.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.980+0000 7ffa7bde5b80 10 client.4800 fetch_fsmap finished waiting for FSMap version 9 2023-11-28T11:09:12.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.980+0000 7ffa7bde5b80 10 client.4800 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:09:12.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.980+0000 7ffa7bde5b80 20 client.4800 populate_metadata read hostname 'smithi028' 2023-11-28T11:09:12.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.980+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/4032131214 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x55569f370490 0x55569f370860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:12.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.980+0000 7ffa7bde5b80 4 client.4800 mds_command: new command op to 4296 tid=0[{"prefix": "status"}] 2023-11-28T11:09:12.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.980+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/4032131214 --> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55569f1bf170 con 0x55569f370490 2023-11-28T11:09:12.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.980+0000 7ffa6d1f7700 1 --2- 172.21.15.28:0/4032131214 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x55569f370490 0x55569f370860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:12.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa48ff9700 10 client.4800 ms_handle_connect on v2:172.21.15.59:6833/3293053261 2023-11-28T11:09:12.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa6d1f7700 1 --2- 172.21.15.28:0/4032131214 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x55569f370490 0x55569f370860 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6c2c0b57ceb5d9f server_cookie=595850f204f6ab7 in_seq=0 out_seq=0 2023-11-28T11:09:12.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa48ff9700 1 -- 172.21.15.28:0/4032131214 <== mds.0 v2:172.21.15.59:6833/3293053261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x55569f1bf170 con 0x55569f370490 2023-11-28T11:09:12.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa48ff9700 10 client.4800 handle_command_reply: tid=0 2023-11-28T11:09:12.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa7bde5b80 1 client.4800 shutdown 2023-11-28T11:09:12.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/4032131214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2ed130 msgr2=0x55569f2bee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/4032131214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2ed130 0x55569f2bee80 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffa64055360 tx=0x7ffa64084000 comp rx=0 tx=0).stop 2023-11-28T11:09:12.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/4032131214 shutdown_connections 2023-11-28T11:09:12.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/4032131214 >> [v2:172.21.15.59:6833/3293053261,v1:172.21.15.59:6835/3293053261] conn(0x55569f370490 0x55569f370860 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/4032131214 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55569f2ed130 0x55569f2bee80 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa48ff9700 0 client.4800 ms_handle_reset on v2:172.21.15.59:6833/3293053261 2023-11-28T11:09:12.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/4032131214 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55569f2f0120 0x55569f2d59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa7bde5b80 1 --2- 172.21.15.28:0/4032131214 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55569f2ef810 0x55569f2d54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/4032131214 >> 172.21.15.28:0/4032131214 conn(0x55569f2c2d60 msgr2=0x55569f2c57e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/4032131214 shutdown_connections 2023-11-28T11:09:12.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa7bde5b80 1 -- 172.21.15.28:0/4032131214 wait complete. 2023-11-28T11:09:12.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.981+0000 7ffa7bde5b80 20 client.4800 trim_cache size 0 max 16384 2023-11-28T11:09:12.990 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:09:12.990 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "6e299adc-7a2a-4cf1-afa2-7ce104b29a51", 2023-11-28T11:09:12.990 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2023-11-28T11:09:12.990 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4296, 2023-11-28T11:09:12.990 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2023-11-28T11:09:12.991 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2023-11-28T11:09:12.991 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2023-11-28T11:09:12.991 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 27.055592465, 2023-11-28T11:09:12.991 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 9, 2023-11-28T11:09:12.991 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 25, 2023-11-28T11:09:12.991 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 25, 2023-11-28T11:09:12.991 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 32.473307792999996 2023-11-28T11:09:12.991 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:09:12.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.982+0000 7ffa6fc5c700 1 -- 172.21.15.28:0/3712810644 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffa4c0682f0 msgr2=0x7ffa4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.982+0000 7ffa6fc5c700 1 --2- 172.21.15.28:0/3712810644 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffa4c0682f0 0x7ffa4c06a7a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ffa680b53d0 tx=0x7ffa58008030 comp rx=0 tx=0).stop 2023-11-28T11:09:12.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.982+0000 7ffa6fc5c700 1 -- 172.21.15.28:0/3712810644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa680b4fe0 msgr2=0x7ffa680c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:12.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.982+0000 7ffa6fc5c700 1 --2- 172.21.15.28:0/3712810644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa680b4fe0 0x7ffa680c72e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7ffa64001eb0 tx=0x7ffa6400c7b0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.982+0000 7ffa6fc5c700 1 -- 172.21.15.28:0/3712810644 shutdown_connections 2023-11-28T11:09:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.982+0000 7ffa6fc5c700 1 --2- 172.21.15.28:0/3712810644 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffa4c0682f0 0x7ffa4c06a7a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.982+0000 7ffa6fc5c700 1 --2- 172.21.15.28:0/3712810644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffa680b4fe0 0x7ffa680c72e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.982+0000 7ffa6fc5c700 1 --2- 172.21.15.28:0/3712810644 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffa6815a230 0x7ffa68152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.982+0000 7ffa6fc5c700 1 --2- 172.21.15.28:0/3712810644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa680cd4a0 0x7ffa68152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.982+0000 7ffa6fc5c700 1 -- 172.21.15.28:0/3712810644 >> 172.21.15.28:0/3712810644 conn(0x7ffa680cf8d0 msgr2=0x7ffa68159450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.982+0000 7ffa6fc5c700 1 -- 172.21.15.28:0/3712810644 shutdown_connections 2023-11-28T11:09:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:12.982+0000 7ffa6fc5c700 1 -- 172.21.15.28:0/3712810644 wait complete. 2023-11-28T11:09:12.996 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-11-28T11:09:13.009 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-11-28T11:09:13.009 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-28T11:09:13.009 INFO:tasks.ceph_fuse:client.0 config is {} 2023-11-28T11:09:13.009 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:09:13.010 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:13.010 INFO:tasks.ceph_fuse:client.2 config is {} 2023-11-28T11:09:13.010 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:09:13.010 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T11:09:13.010 INFO:tasks.ceph_fuse:client.1 config is {} 2023-11-28T11:09:13.010 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:09:13.010 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T11:09:13.011 INFO:tasks.ceph_fuse:client.3 config is {} 2023-11-28T11:09:13.011 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-11-28T11:09:13.011 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T11:09:13.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:13.011 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2023-11-28T11:09:13.029 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-11-28T11:09:13.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:13.030 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-11-28T11:09:13.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:13.106 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link delete ceph-brx 2023-11-28T11:09:13.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:13.215 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-11-28T11:09:13.237 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:13.237 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2023-11-28T11:09:13.312 INFO:teuthology.orchestra.run.smithi059.stderr:Cannot find device "ceph-brx" 2023-11-28T11:09:13.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:09:13.315 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-11-28T11:09:13.315 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:09:13.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.441+0000 7f107ebe2700 1 Processor -- start 2023-11-28T11:09:13.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.441+0000 7f107ebe2700 1 -- start start 2023-11-28T11:09:13.445 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.442+0000 7f107ebe2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10781592b0 0x7f10780cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.445 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.442+0000 7f107ebe2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10780caf90 0x7f1078070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.442+0000 7f107ebe2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107815a4a0 con 0x7f1078070960 2023-11-28T11:09:13.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.442+0000 7f107ebe2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1078071310 con 0x7f10780caf90 2023-11-28T11:09:13.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.442+0000 7f107ebe2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107815a780 con 0x7f10781592b0 2023-11-28T11:09:13.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.442+0000 7f107d17f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f10780caf90 0x7f1078070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.442+0000 7f107c97e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10781592b0 0x7f10780cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.442+0000 7f1077fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f1078070960 0x7f10780ca880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:46346/0 (socket says 172.21.15.28:46346) 2023-11-28T11:09:13.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.442+0000 7f1077fff700 1 -- 172.21.15.28:0/3785943416 learned_addr learned my addr 172.21.15.28:0/3785943416 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:13.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.442+0000 7f10777fe700 1 -- 172.21.15.28:0/3785943416 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1695105186 0 0) 0x7f107815a4a0 con 0x7f1078070960 2023-11-28T11:09:13.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.442+0000 7f10777fe700 1 -- 172.21.15.28:0/3785943416 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10680035b0 con 0x7f1078070960 2023-11-28T11:09:13.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.443+0000 7f107d17f700 1 -- 172.21.15.28:0/3785943416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10781592b0 msgr2=0x7f10780cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:13.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.443+0000 7f107d17f700 1 --2- 172.21.15.28:0/3785943416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10781592b0 0x7f10780cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.443+0000 7f107d17f700 1 -- 172.21.15.28:0/3785943416 >> v1:172.21.15.59:6789/0 conn(0x7f1078070960 legacy=0x7f10780ca880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.443+0000 7f107d17f700 1 -- 172.21.15.28:0/3785943416 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10780ca080 con 0x7f10780caf90 2023-11-28T11:09:13.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.443+0000 7f107c97e700 1 --2- 172.21.15.28:0/3785943416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10781592b0 0x7f10780cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:13.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.443+0000 7f107d17f700 1 --2- 172.21.15.28:0/3785943416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10780caf90 0x7f1078070dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f107000bfa0 tx=0x7f107000d330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=641d433b69de4efd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.443+0000 7f10777fe700 1 -- 172.21.15.28:0/3785943416 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1070010070 con 0x7f10780caf90 2023-11-28T11:09:13.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.443+0000 7f10777fe700 1 -- 172.21.15.28:0/3785943416 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f107000b2e0 con 0x7f10780caf90 2023-11-28T11:09:13.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.443+0000 7f10777fe700 1 -- 172.21.15.28:0/3785943416 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f107000dc00 con 0x7f10780caf90 2023-11-28T11:09:13.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.443+0000 7f107ebe2700 1 -- 172.21.15.28:0/3785943416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10780caf90 msgr2=0x7f1078070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.443+0000 7f107ebe2700 1 --2- 172.21.15.28:0/3785943416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10780caf90 0x7f1078070dd0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f107000bfa0 tx=0x7f107000d330 comp rx=0 tx=0).stop 2023-11-28T11:09:13.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.443+0000 7f107ebe2700 1 -- 172.21.15.28:0/3785943416 shutdown_connections 2023-11-28T11:09:13.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.443+0000 7f107ebe2700 1 --2- 172.21.15.28:0/3785943416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10780caf90 0x7f1078070dd0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.443+0000 7f107ebe2700 1 --2- 172.21.15.28:0/3785943416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10781592b0 0x7f10780cc5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.444+0000 7f107ebe2700 1 -- 172.21.15.28:0/3785943416 >> 172.21.15.28:0/3785943416 conn(0x7f10780ce9e0 msgr2=0x7f10780cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:13.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.444+0000 7f107ebe2700 1 -- 172.21.15.28:0/3785943416 shutdown_connections 2023-11-28T11:09:13.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.444+0000 7f107ebe2700 1 -- 172.21.15.28:0/3785943416 wait complete. 2023-11-28T11:09:13.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.445+0000 7f107ebe2700 1 Processor -- start 2023-11-28T11:09:13.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.445+0000 7f107ebe2700 1 -- start start 2023-11-28T11:09:13.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107ebe2700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1078070960 0x7f10780b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107ebe2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10780caf90 0x7f10781481c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107ebe2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10781592b0 0x7f107814acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107ebe2700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107814c580 con 0x7f1078070960 2023-11-28T11:09:13.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107ebe2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1078149cb0 con 0x7f10780caf90 2023-11-28T11:09:13.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107ebe2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107814c000 con 0x7f10781592b0 2023-11-28T11:09:13.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107c97e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1078070960 0x7f10780b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107c97e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1078070960 0x7f10780b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47508/0 (socket says 172.21.15.28:47508) 2023-11-28T11:09:13.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107c97e700 1 -- 172.21.15.28:0/2753867782 learned_addr learned my addr 172.21.15.28:0/2753867782 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:13.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107d17f700 1 --2- 172.21.15.28:0/2753867782 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10781592b0 0x7f107814acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f1077fff700 1 --2- 172.21.15.28:0/2753867782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10780caf90 0x7f10781481c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107c97e700 1 -- 172.21.15.28:0/2753867782 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10781592b0 msgr2=0x7f107814acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107c97e700 1 --2- 172.21.15.28:0/2753867782 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10781592b0 0x7f107814acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107c97e700 1 -- 172.21.15.28:0/2753867782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10780caf90 msgr2=0x7f10781481c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107c97e700 1 --2- 172.21.15.28:0/2753867782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10780caf90 0x7f10781481c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107c97e700 1 -- 172.21.15.28:0/2753867782 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f106c0094a0 con 0x7f1078070960 2023-11-28T11:09:13.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f1077fff700 1 --2- 172.21.15.28:0/2753867782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10780caf90 0x7f10781481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:13.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f107c97e700 1 --2- 172.21.15.28:0/2753867782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1078070960 0x7f10780b18b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f106c00ebe0 tx=0x7f106c00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.446+0000 7f10757fa700 1 -- 172.21.15.28:0/2753867782 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c00c850 con 0x7f1078070960 2023-11-28T11:09:13.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.447+0000 7f10757fa700 1 -- 172.21.15.28:0/2753867782 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c00c9b0 con 0x7f1078070960 2023-11-28T11:09:13.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.447+0000 7f107ebe2700 1 -- 172.21.15.28:0/2753867782 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107000bc80 con 0x7f1078070960 2023-11-28T11:09:13.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.447+0000 7f10757fa700 1 -- 172.21.15.28:0/2753867782 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c00cd40 con 0x7f1078070960 2023-11-28T11:09:13.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.447+0000 7f107ebe2700 1 -- 172.21.15.28:0/2753867782 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10780c9d90 con 0x7f1078070960 2023-11-28T11:09:13.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.447+0000 7f107ebe2700 1 -- 172.21.15.28:0/2753867782 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10780ca360 con 0x7f1078070960 2023-11-28T11:09:13.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.448+0000 7f10757fa700 1 -- 172.21.15.28:0/2753867782 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f106c019030 con 0x7f1078070960 2023-11-28T11:09:13.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.448+0000 7f10757fa700 1 --2- 172.21.15.28:0/2753867782 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f105c0682f0 0x7f105c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:13.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.448+0000 7f10757fa700 1 -- 172.21.15.28:0/2753867782 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f106c085fb0 con 0x7f1078070960 2023-11-28T11:09:13.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.448+0000 7f1077fff700 1 --2- 172.21.15.28:0/2753867782 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f105c0682f0 0x7f105c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:13.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.449+0000 7f1077fff700 1 --2- 172.21.15.28:0/2753867782 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f105c0682f0 0x7f105c06a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f10781f15a0 tx=0x7f1064009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:13.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.453+0000 7f10757fa700 1 -- 172.21.15.28:0/2753867782 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f106c052430 con 0x7f1078070960 2023-11-28T11:09:13.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.692+0000 7f107ebe2700 1 -- 172.21.15.28:0/2753867782 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f107815d1d0 con 0x7f1078070960 2023-11-28T11:09:13.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.692+0000 7f10757fa700 1 -- 172.21.15.28:0/2753867782 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f106c017070 con 0x7f1078070960 2023-11-28T11:09:13.694 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T11:09:13.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.694+0000 7f107ebe2700 1 -- 172.21.15.28:0/2753867782 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f105c0682f0 msgr2=0x7f105c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.694+0000 7f107ebe2700 1 --2- 172.21.15.28:0/2753867782 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f105c0682f0 0x7f105c06a7a0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f10781f15a0 tx=0x7f1064009040 comp rx=0 tx=0).stop 2023-11-28T11:09:13.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.694+0000 7f107ebe2700 1 -- 172.21.15.28:0/2753867782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1078070960 msgr2=0x7f10780b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:13.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.694+0000 7f107ebe2700 1 --2- 172.21.15.28:0/2753867782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1078070960 0x7f10780b18b0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f106c00ebe0 tx=0x7f106c00eef0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.694+0000 7f107ebe2700 1 -- 172.21.15.28:0/2753867782 shutdown_connections 2023-11-28T11:09:13.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.694+0000 7f107ebe2700 1 --2- 172.21.15.28:0/2753867782 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f105c0682f0 0x7f105c06a7a0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.694+0000 7f107ebe2700 1 --2- 172.21.15.28:0/2753867782 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1078070960 0x7f10780b18b0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.694+0000 7f107ebe2700 1 --2- 172.21.15.28:0/2753867782 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10781592b0 0x7f107814acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.694+0000 7f107ebe2700 1 --2- 172.21.15.28:0/2753867782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10780caf90 0x7f10781481c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:13.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.694+0000 7f107ebe2700 1 -- 172.21.15.28:0/2753867782 >> 172.21.15.28:0/2753867782 conn(0x7f10780ce9e0 msgr2=0x7f10780ca780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:13.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.694+0000 7f107ebe2700 1 -- 172.21.15.28:0/2753867782 shutdown_connections 2023-11-28T11:09:13.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:13.694+0000 7f107ebe2700 1 -- 172.21.15.28:0/2753867782 wait complete. 2023-11-28T11:09:13.708 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:09:13.708 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:09:13.708 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2023-11-28T11:09:13.708 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:09:13.708 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:13.709 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:09:13.709 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:09:13.709 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:09:13.709 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:09:13.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:13.709 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2023-11-28T11:09:13.726 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:13.727 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:13.727 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:13.727 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:13.727 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:13.727 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:13.727 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:13.727 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:13.727 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:13.727 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:13.727 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:09:13.728 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:13.728 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T11:09:13.728 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42305sec preferred_lft 42305sec 2023-11-28T11:09:13.728 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2023-11-28T11:09:13.728 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:13.728 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:13.728 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:13.728 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T11:09:13.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:13.729 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:09:13.729 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:09:13.729 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add name ceph-brx type bridge 2023-11-28T11:09:13.729 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip addr flush dev ceph-brx 2023-11-28T11:09:13.729 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set ceph-brx up 2023-11-28T11:09:13.729 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T11:09:13.729 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:09:13.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:13.847 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:09:13.874 INFO:teuthology.orchestra.run.smithi028.stdout:1 2023-11-28T11:09:13.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:13.875 DEBUG:teuthology.orchestra.run.smithi028:> route 2023-11-28T11:09:13.934 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2023-11-28T11:09:13.935 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:09:13.935 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T11:09:13.935 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T11:09:13.935 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:09:13.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:13.935 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:09:13.935 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:09:13.935 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T11:09:13.936 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T11:09:13.936 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T11:09:13.936 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:09:14.043 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:14.043 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2023-11-28T11:09:14.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:14.060 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list-id 2023-11-28T11:09:14.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:14.118 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:09:14.118 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:09:14.119 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:09:14.119 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-28T11:09:14.120 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:09:14.214 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-28T11:09:14.214 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:14.214 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:09:14.214 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:09:14.214 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-28T11:09:14.215 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T11:09:14.215 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-28T11:09:14.215 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-28T11:09:14.215 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-28T11:09:14.215 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:09:14.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:14.408 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:09:14.408 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:09:14.408 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2023-11-28T11:09:14.408 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T11:09:14.408 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:09:14.532 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T11:09:14.532 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:14.532 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:14.557 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:09:14.557 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:14.558 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:14.614 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2023-11-28T11:09:14.682 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:09:14.683 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:09:14.749 INFO:teuthology.orchestra.run.smithi028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:09:14.750 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:09:14.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:14.751 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2023-11-28T11:09:14.808 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T11:09:14.808 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/) 2023-11-28T11:09:14.851 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2023-11-28T11:09:14.877 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:09:14.878 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:09:14.942 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:2023-11-28T11:09:14.940+0000 7fc78fb0f5c0 -1 init, newargv = 0x55d00a760670 newargc=15 2023-11-28T11:09:14.942 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[114408]: starting ceph client 2023-11-28T11:09:14.947 INFO:teuthology.orchestra.run.smithi028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:09:14.947 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[114408]: starting fuse 2023-11-28T11:09:14.948 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:09:14.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:14.948 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2023-11-28T11:09:15.006 INFO:teuthology.orchestra.run.smithi028.stdout:47 2023-11-28T11:09:15.006 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T11:09:15.006 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:15.006 DEBUG:teuthology.orchestra.run.smithi028:> import glob 2023-11-28T11:09:15.006 DEBUG:teuthology.orchestra.run.smithi028:> import re 2023-11-28T11:09:15.006 DEBUG:teuthology.orchestra.run.smithi028:> import os 2023-11-28T11:09:15.006 DEBUG:teuthology.orchestra.run.smithi028:> import subprocess 2023-11-28T11:09:15.006 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:09:15.007 DEBUG:teuthology.orchestra.run.smithi028:> def _find_admin_socket(client_name): 2023-11-28T11:09:15.007 DEBUG:teuthology.orchestra.run.smithi028:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T11:09:15.007 DEBUG:teuthology.orchestra.run.smithi028:> files = glob.glob(asok_path) 2023-11-28T11:09:15.007 DEBUG:teuthology.orchestra.run.smithi028:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T11:09:15.007 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:09:15.007 DEBUG:teuthology.orchestra.run.smithi028:> # Given a non-glob path, it better be there 2023-11-28T11:09:15.007 DEBUG:teuthology.orchestra.run.smithi028:> if "*" not in asok_path: 2023-11-28T11:09:15.007 DEBUG:teuthology.orchestra.run.smithi028:> assert(len(files) == 1) 2023-11-28T11:09:15.007 DEBUG:teuthology.orchestra.run.smithi028:> return files[0] 2023-11-28T11:09:15.007 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:09:15.007 DEBUG:teuthology.orchestra.run.smithi028:> for f in files: 2023-11-28T11:09:15.007 DEBUG:teuthology.orchestra.run.smithi028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T11:09:15.007 DEBUG:teuthology.orchestra.run.smithi028:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T11:09:15.007 DEBUG:teuthology.orchestra.run.smithi028:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T11:09:15.008 DEBUG:teuthology.orchestra.run.smithi028:> contents = proc_f.read() 2023-11-28T11:09:15.008 DEBUG:teuthology.orchestra.run.smithi028:> if mountpoint in contents: 2023-11-28T11:09:15.008 DEBUG:teuthology.orchestra.run.smithi028:> return f 2023-11-28T11:09:15.008 DEBUG:teuthology.orchestra.run.smithi028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T11:09:15.008 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:09:15.008 DEBUG:teuthology.orchestra.run.smithi028:> print(_find_admin_socket("client.0")) 2023-11-28T11:09:15.008 DEBUG:teuthology.orchestra.run.smithi028:> ' 2023-11-28T11:09:15.130 INFO:teuthology.orchestra.run.smithi028.stdout:/var/run/ceph/ceph-client.0.114408.asok 2023-11-28T11:09:15.313 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.114408.asok 2023-11-28T11:09:15.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:15.313 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.114408.asok status 2023-11-28T11:09:15.475 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:09:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "metadata": { 2023-11-28T11:09:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T11:09:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T11:09:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "entity_id": "0", 2023-11-28T11:09:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "hostname": "smithi028", 2023-11-28T11:09:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T11:09:15.475 INFO:teuthology.orchestra.run.smithi028.stdout: "pid": "114408", 2023-11-28T11:09:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "root": "/" 2023-11-28T11:09:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:09:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_count": 0, 2023-11-28T11:09:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_pinned_count": 0, 2023-11-28T11:09:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4809, 2023-11-28T11:09:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "inst": { 2023-11-28T11:09:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "name": { 2023-11-28T11:09:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "client", 2023-11-28T11:09:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "num": 4809 2023-11-28T11:09:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:09:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2023-11-28T11:09:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2023-11-28T11:09:15.476 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2023-11-28T11:09:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1906161052 2023-11-28T11:09:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:09:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:09:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2023-11-28T11:09:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2023-11-28T11:09:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2023-11-28T11:09:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1906161052 2023-11-28T11:09:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:09:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "inst_str": "client.4809 192.168.0.1:0/1906161052", 2023-11-28T11:09:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "addr_str": "192.168.0.1:0/1906161052", 2023-11-28T11:09:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "inode_count": 1, 2023-11-28T11:09:15.477 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_epoch": 9, 2023-11-28T11:09:15.478 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch": 25, 2023-11-28T11:09:15.478 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch_barrier": 0, 2023-11-28T11:09:15.478 INFO:teuthology.orchestra.run.smithi028.stdout: "blocklisted": false, 2023-11-28T11:09:15.478 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs" 2023-11-28T11:09:15.478 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:09:15.485 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:09:15.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.610+0000 7fc92be42700 1 Processor -- start 2023-11-28T11:09:15.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.610+0000 7fc92be42700 1 -- start start 2023-11-28T11:09:15.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.610+0000 7fc92be42700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc924159590 0x7fc92406e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc92be42700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92415db70 0x7fc9240ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc92be42700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92415d120 con 0x7fc9240ce890 2023-11-28T11:09:15.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc92be42700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92415d6e0 con 0x7fc924159590 2023-11-28T11:09:15.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc92be42700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9240aef60 con 0x7fc92415db70 2023-11-28T11:09:15.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc92a3df700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc9240ce890 0x7fc9240caf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35588/0 (socket says 172.21.15.28:35588) 2023-11-28T11:09:15.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc92a3df700 1 -- 172.21.15.28:0/2433104639 learned_addr learned my addr 172.21.15.28:0/2433104639 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:15.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc929bde700 1 --2- 172.21.15.28:0/2433104639 >> v2:172.21.15.59:3300/0 conn(0x7fc924159590 0x7fc92406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc9293dd700 1 --2- 172.21.15.28:0/2433104639 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92415db70 0x7fc9240ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 -- 172.21.15.28:0/2433104639 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2345601775 0 0) 0x7fc92415d120 con 0x7fc9240ce890 2023-11-28T11:09:15.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 -- 172.21.15.28:0/2433104639 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9100057c0 con 0x7fc9240ce890 2023-11-28T11:09:15.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 -- 172.21.15.28:0/2433104639 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1808089856 0 0) 0x7fc9100057c0 con 0x7fc9240ce890 2023-11-28T11:09:15.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 -- 172.21.15.28:0/2433104639 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92415db70 msgr2=0x7fc9240ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:15.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 --2- 172.21.15.28:0/2433104639 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92415db70 0x7fc9240ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 -- 172.21.15.28:0/2433104639 >> v2:172.21.15.59:3300/0 conn(0x7fc924159590 msgr2=0x7fc92406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:15.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 --2- 172.21.15.28:0/2433104639 >> v2:172.21.15.59:3300/0 conn(0x7fc924159590 0x7fc92406e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 -- 172.21.15.28:0/2433104639 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9240af240 con 0x7fc9240ce890 2023-11-28T11:09:15.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 -- 172.21.15.28:0/2433104639 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fc91c0019f0 con 0x7fc9240ce890 2023-11-28T11:09:15.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 -- 172.21.15.28:0/2433104639 >> v1:172.21.15.28:6789/0 conn(0x7fc9240ce890 legacy=0x7fc9240caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 --2- 172.21.15.28:0/2433104639 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc910005c10 0x7fc910006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 --2- 172.21.15.28:0/2433104639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9100065f0 0x7fc910006ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 --2- 172.21.15.28:0/2433104639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc910007130 0x7fc91000dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 -- 172.21.15.28:0/2433104639 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92415d6e0 con 0x7fc9100065f0 2023-11-28T11:09:15.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 -- 172.21.15.28:0/2433104639 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9240aef60 con 0x7fc910007130 2023-11-28T11:09:15.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc928bdc700 1 -- 172.21.15.28:0/2433104639 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91000e160 con 0x7fc910005c10 2023-11-28T11:09:15.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc9293dd700 1 --2- 172.21.15.28:0/2433104639 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92415db70 0x7fc9240ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:15.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.611+0000 7fc929bde700 1 --2- 172.21.15.28:0/2433104639 >> v2:172.21.15.59:3300/0 conn(0x7fc924159590 0x7fc92406e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:15.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.612+0000 7fc9293dd700 1 --2- 172.21.15.28:0/2433104639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9100065f0 0x7fc910006ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.612+0000 7fc92a3df700 1 --2- 172.21.15.28:0/2433104639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc910007130 0x7fc91000dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.612+0000 7fc929bde700 1 --2- 172.21.15.28:0/2433104639 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc910005c10 0x7fc910006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.612+0000 7fc9293dd700 1 -- 172.21.15.28:0/2433104639 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc910005c10 msgr2=0x7fc910006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.612+0000 7fc9293dd700 1 --2- 172.21.15.28:0/2433104639 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc910005c10 0x7fc910006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.612+0000 7fc9293dd700 1 -- 172.21.15.28:0/2433104639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc910007130 msgr2=0x7fc91000dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.612+0000 7fc9293dd700 1 --2- 172.21.15.28:0/2433104639 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc910007130 0x7fc91000dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.612+0000 7fc9293dd700 1 -- 172.21.15.28:0/2433104639 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9100057c0 con 0x7fc9100065f0 2023-11-28T11:09:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.612+0000 7fc9293dd700 1 --2- 172.21.15.28:0/2433104639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9100065f0 0x7fc910006ac0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc9240cacc0 tx=0x7fc910007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a5ac9fa939bfbc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.612+0000 7fc928bdc700 1 -- 172.21.15.28:0/2433104639 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918004030 con 0x7fc9100065f0 2023-11-28T11:09:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.612+0000 7fc928bdc700 1 -- 172.21.15.28:0/2433104639 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc918004190 con 0x7fc9100065f0 2023-11-28T11:09:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.612+0000 7fc928bdc700 1 -- 172.21.15.28:0/2433104639 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc918004540 con 0x7fc9100065f0 2023-11-28T11:09:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.613+0000 7fc92be42700 1 -- 172.21.15.28:0/2433104639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9100065f0 msgr2=0x7fc910006ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.613+0000 7fc92be42700 1 --2- 172.21.15.28:0/2433104639 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9100065f0 0x7fc910006ac0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fc9240cacc0 tx=0x7fc910007a90 comp rx=0 tx=0).stop 2023-11-28T11:09:15.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.613+0000 7fc92a3df700 1 -- 172.21.15.28:0/2433104639 reap_dead start 2023-11-28T11:09:15.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.613+0000 7fc92be42700 1 -- 172.21.15.28:0/2433104639 shutdown_connections 2023-11-28T11:09:15.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.613+0000 7fc92be42700 1 -- 172.21.15.28:0/2433104639 >> 172.21.15.28:0/2433104639 conn(0x7fc9240b1650 msgr2=0x7fc9240b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:15.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.613+0000 7fc92be42700 1 -- 172.21.15.28:0/2433104639 shutdown_connections 2023-11-28T11:09:15.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.613+0000 7fc92be42700 1 -- 172.21.15.28:0/2433104639 wait complete. 2023-11-28T11:09:15.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.614+0000 7fc92be42700 1 Processor -- start 2023-11-28T11:09:15.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.615+0000 7fc92be42700 1 -- start start 2023-11-28T11:09:15.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.615+0000 7fc92be42700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9100065f0 0x7fc924151960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.615+0000 7fc92be42700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9240c7ec0 0x7fc9240b5550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.615+0000 7fc92be42700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9240c9460 0x7fc9240cb560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc92be42700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9240c8fd0 con 0x7fc9100065f0 2023-11-28T11:09:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc92be42700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9240b5a90 con 0x7fc9240c7ec0 2023-11-28T11:09:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc92be42700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9240c83c0 con 0x7fc9240c9460 2023-11-28T11:09:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc929bde700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9100065f0 0x7fc924151960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc929bde700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9100065f0 0x7fc924151960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47538/0 (socket says 172.21.15.28:47538) 2023-11-28T11:09:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc929bde700 1 -- 172.21.15.28:0/2951000016 learned_addr learned my addr 172.21.15.28:0/2951000016 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc9293dd700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9240c7ec0 0x7fc9240b5550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc92a3df700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9240c9460 0x7fc9240cb560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc929bde700 1 -- 172.21.15.28:0/2951000016 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9240c9460 msgr2=0x7fc9240cb560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc929bde700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9240c9460 0x7fc9240cb560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc929bde700 1 -- 172.21.15.28:0/2951000016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9240c7ec0 msgr2=0x7fc9240b5550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc929bde700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9240c7ec0 0x7fc9240b5550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc929bde700 1 -- 172.21.15.28:0/2951000016 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92415dfe0 con 0x7fc9100065f0 2023-11-28T11:09:15.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc92a3df700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9240c9460 0x7fc9240cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:15.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc9293dd700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9240c7ec0 0x7fc9240b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:15.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc929bde700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9100065f0 0x7fc924151960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc920011cd0 tx=0x7fc92000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc9167fc700 1 -- 172.21.15.28:0/2951000016 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92000fad0 con 0x7fc9100065f0 2023-11-28T11:09:15.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc9167fc700 1 -- 172.21.15.28:0/2951000016 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92000fc30 con 0x7fc9100065f0 2023-11-28T11:09:15.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc92be42700 1 -- 172.21.15.28:0/2951000016 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92415d370 con 0x7fc9100065f0 2023-11-28T11:09:15.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc9167fc700 1 -- 172.21.15.28:0/2951000016 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc920004030 con 0x7fc9100065f0 2023-11-28T11:09:15.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.616+0000 7fc92be42700 1 -- 172.21.15.28:0/2951000016 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc924159960 con 0x7fc9100065f0 2023-11-28T11:09:15.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.617+0000 7fc92be42700 1 -- 172.21.15.28:0/2951000016 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9240675a0 con 0x7fc9100065f0 2023-11-28T11:09:15.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.618+0000 7fc9167fc700 1 -- 172.21.15.28:0/2951000016 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc920024070 con 0x7fc9100065f0 2023-11-28T11:09:15.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.618+0000 7fc9167fc700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9080682f0 0x7fc90806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:15.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.618+0000 7fc9167fc700 1 -- 172.21.15.28:0/2951000016 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fc920086ac0 con 0x7fc9100065f0 2023-11-28T11:09:15.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.618+0000 7fc9293dd700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9080682f0 0x7fc90806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:15.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.619+0000 7fc9293dd700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9080682f0 0x7fc90806a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc9100069e0 tx=0x7fc918008070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:15.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.622+0000 7fc9167fc700 1 -- 172.21.15.28:0/2951000016 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc920052fc0 con 0x7fc9100065f0 2023-11-28T11:09:15.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.859+0000 7fc92be42700 1 -- 172.21.15.28:0/2951000016 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc924154260 con 0x7fc9100065f0 2023-11-28T11:09:15.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.860+0000 7fc9167fc700 1 -- 172.21.15.28:0/2951000016 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7fc920022070 con 0x7fc9100065f0 2023-11-28T11:09:15.861 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T11:09:15.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.862+0000 7fc92be42700 1 -- 172.21.15.28:0/2951000016 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9080682f0 msgr2=0x7fc90806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.862+0000 7fc92be42700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9080682f0 0x7fc90806a7a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc9100069e0 tx=0x7fc918008070 comp rx=0 tx=0).stop 2023-11-28T11:09:15.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.862+0000 7fc92be42700 1 -- 172.21.15.28:0/2951000016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9100065f0 msgr2=0x7fc924151960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:15.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.862+0000 7fc92be42700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9100065f0 0x7fc924151960 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fc920011cd0 tx=0x7fc92000f520 comp rx=0 tx=0).stop 2023-11-28T11:09:15.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.862+0000 7fc92be42700 1 -- 172.21.15.28:0/2951000016 shutdown_connections 2023-11-28T11:09:15.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.862+0000 7fc92be42700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9080682f0 0x7fc90806a7a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.862+0000 7fc92be42700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9100065f0 0x7fc924151960 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.862+0000 7fc92be42700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9240c9460 0x7fc9240cb560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.862+0000 7fc92be42700 1 --2- 172.21.15.28:0/2951000016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9240c7ec0 0x7fc9240b5550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:15.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.862+0000 7fc92be42700 1 -- 172.21.15.28:0/2951000016 >> 172.21.15.28:0/2951000016 conn(0x7fc9240b1650 msgr2=0x7fc9241533e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:15.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.862+0000 7fc92be42700 1 -- 172.21.15.28:0/2951000016 shutdown_connections 2023-11-28T11:09:15.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:15.862+0000 7fc92be42700 1 -- 172.21.15.28:0/2951000016 wait complete. 2023-11-28T11:09:15.875 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:09:15.876 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:09:15.876 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2023-11-28T11:09:15.876 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-28T11:09:15.876 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T11:09:15.876 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:09:15.876 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:09:15.876 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:09:15.876 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-28T11:09:15.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:15.876 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2023-11-28T11:09:15.894 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:15.894 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:15.894 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:15.894 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:15.894 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:15.894 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:15.894 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:15.895 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:15.895 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:15.895 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:15.895 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:09:15.895 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:15.895 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T11:09:15.895 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42303sec preferred_lft 42303sec 2023-11-28T11:09:15.895 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2023-11-28T11:09:15.895 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:15.895 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:15.895 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:15.895 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:09:15.895 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 62:ec:54:d3:7a:f0 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:15.896 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:09:15.896 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:15.896 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::407f:32ff:fe2d:9b2f/64 scope link 2023-11-28T11:09:15.896 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:15.896 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:09:15.896 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 62:ec:54:d3:7a:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:09:15.896 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::60ec:54ff:fed3:7af0/64 scope link tentative 2023-11-28T11:09:15.896 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:15.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:15.896 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:09:15.960 INFO:teuthology.orchestra.run.smithi028.stdout:1 2023-11-28T11:09:15.961 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:15.961 DEBUG:teuthology.orchestra.run.smithi028:> route 2023-11-28T11:09:16.021 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2023-11-28T11:09:16.021 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:09:16.021 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T11:09:16.021 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T11:09:16.021 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:09:16.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:16.021 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:09:16.022 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:09:16.022 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T11:09:16.022 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T11:09:16.022 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T11:09:16.022 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:09:16.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:16.128 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2023-11-28T11:09:16.145 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:09:16.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:16.146 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list-id 2023-11-28T11:09:16.202 INFO:teuthology.orchestra.run.smithi028.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-28T11:09:16.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:16.203 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:09:16.203 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:09:16.203 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:09:16.203 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-28T11:09:16.203 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:09:16.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:16.299 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T11:09:16.336 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:16.337 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:16.337 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:16.337 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:16.337 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:16.337 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:16.338 INFO:teuthology.orchestra.run.smithi028.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:09:16.338 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 66:4f:15:73:08:a7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:09:16.338 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:09:16.338 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:16.338 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::644f:15ff:fe73:8a7/64 scope link 2023-11-28T11:09:16.338 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:16.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:16.350 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T11:09:16.429 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:16.429 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:16.429 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:16.429 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:16.429 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:16.430 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:16.430 INFO:teuthology.orchestra.run.smithi028.stdout:2: veth0@if9: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:09:16.430 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 66:4f:15:73:08:a7 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:09:16.430 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:09:16.430 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:16.430 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::644f:15ff:fe73:8a7/64 scope link 2023-11-28T11:09:16.430 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:16.443 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-28T11:09:16.443 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:16.443 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:09:16.443 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:09:16.443 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-28T11:09:16.443 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T11:09:16.444 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-28T11:09:16.444 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-28T11:09:16.444 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-28T11:09:16.444 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:09:16.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:16.722 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:09:16.722 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:09:16.722 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.1 up 2023-11-28T11:09:16.722 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T11:09:16.723 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:09:16.839 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-11-28T11:09:16.839 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:16.840 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-11-28T11:09:16.871 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-11-28T11:09:16.872 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:16.872 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-11-28T11:09:16.929 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2023-11-28T11:09:16.998 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:09:16.998 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:09:17.065 INFO:teuthology.orchestra.run.smithi028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:09:17.065 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:09:17.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:17.066 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2023-11-28T11:09:17.123 INFO:teuthology.orchestra.run.smithi028.stdout:47 2023-11-28T11:09:17.124 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2023-11-28T11:09:17.124 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2 --client_mountpoint=/) 2023-11-28T11:09:17.166 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2023-11-28T11:09:17.193 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:09:17.194 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:09:17.256 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi028.stderr:2023-11-28T11:09:17.254+0000 7f37740fe5c0 -1 init, newargv = 0x5595ac95d970 newargc=15 2023-11-28T11:09:17.256 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi028.stderr:ceph-fuse[115062]: starting ceph client 2023-11-28T11:09:17.261 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi028.stderr:ceph-fuse[115062]: starting fuse 2023-11-28T11:09:17.264 INFO:teuthology.orchestra.run.smithi028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:09:17.265 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:09:17.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:17.265 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2023-11-28T11:09:17.323 INFO:teuthology.orchestra.run.smithi028.stdout:47 2023-11-28T11:09:17.323 INFO:teuthology.orchestra.run.smithi028.stdout:48 2023-11-28T11:09:17.323 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2023-11-28T11:09:17.323 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:17.323 DEBUG:teuthology.orchestra.run.smithi028:> import glob 2023-11-28T11:09:17.323 DEBUG:teuthology.orchestra.run.smithi028:> import re 2023-11-28T11:09:17.324 DEBUG:teuthology.orchestra.run.smithi028:> import os 2023-11-28T11:09:17.324 DEBUG:teuthology.orchestra.run.smithi028:> import subprocess 2023-11-28T11:09:17.324 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:09:17.324 DEBUG:teuthology.orchestra.run.smithi028:> def _find_admin_socket(client_name): 2023-11-28T11:09:17.324 DEBUG:teuthology.orchestra.run.smithi028:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-11-28T11:09:17.324 DEBUG:teuthology.orchestra.run.smithi028:> files = glob.glob(asok_path) 2023-11-28T11:09:17.324 DEBUG:teuthology.orchestra.run.smithi028:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-11-28T11:09:17.324 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:09:17.324 DEBUG:teuthology.orchestra.run.smithi028:> # Given a non-glob path, it better be there 2023-11-28T11:09:17.324 DEBUG:teuthology.orchestra.run.smithi028:> if "*" not in asok_path: 2023-11-28T11:09:17.324 DEBUG:teuthology.orchestra.run.smithi028:> assert(len(files) == 1) 2023-11-28T11:09:17.324 DEBUG:teuthology.orchestra.run.smithi028:> return files[0] 2023-11-28T11:09:17.324 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:09:17.325 DEBUG:teuthology.orchestra.run.smithi028:> for f in files: 2023-11-28T11:09:17.325 DEBUG:teuthology.orchestra.run.smithi028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T11:09:17.325 DEBUG:teuthology.orchestra.run.smithi028:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T11:09:17.325 DEBUG:teuthology.orchestra.run.smithi028:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T11:09:17.325 DEBUG:teuthology.orchestra.run.smithi028:> contents = proc_f.read() 2023-11-28T11:09:17.325 DEBUG:teuthology.orchestra.run.smithi028:> if mountpoint in contents: 2023-11-28T11:09:17.325 DEBUG:teuthology.orchestra.run.smithi028:> return f 2023-11-28T11:09:17.325 DEBUG:teuthology.orchestra.run.smithi028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T11:09:17.325 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:09:17.325 DEBUG:teuthology.orchestra.run.smithi028:> print(_find_admin_socket("client.2")) 2023-11-28T11:09:17.325 DEBUG:teuthology.orchestra.run.smithi028:> ' 2023-11-28T11:09:17.448 INFO:teuthology.orchestra.run.smithi028.stdout:/var/run/ceph/ceph-client.2.115062.asok 2023-11-28T11:09:17.630 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.115062.asok 2023-11-28T11:09:17.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:17.630 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.115062.asok status 2023-11-28T11:09:17.790 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:09:17.790 INFO:teuthology.orchestra.run.smithi028.stdout: "metadata": { 2023-11-28T11:09:17.790 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T11:09:17.790 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T11:09:17.790 INFO:teuthology.orchestra.run.smithi028.stdout: "entity_id": "2", 2023-11-28T11:09:17.790 INFO:teuthology.orchestra.run.smithi028.stdout: "hostname": "smithi028", 2023-11-28T11:09:17.791 INFO:teuthology.orchestra.run.smithi028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-11-28T11:09:17.791 INFO:teuthology.orchestra.run.smithi028.stdout: "pid": "115062", 2023-11-28T11:09:17.791 INFO:teuthology.orchestra.run.smithi028.stdout: "root": "/" 2023-11-28T11:09:17.791 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:09:17.791 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_count": 0, 2023-11-28T11:09:17.791 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_pinned_count": 0, 2023-11-28T11:09:17.791 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 4821, 2023-11-28T11:09:17.792 INFO:teuthology.orchestra.run.smithi028.stdout: "inst": { 2023-11-28T11:09:17.792 INFO:teuthology.orchestra.run.smithi028.stdout: "name": { 2023-11-28T11:09:17.792 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "client", 2023-11-28T11:09:17.792 INFO:teuthology.orchestra.run.smithi028.stdout: "num": 4821 2023-11-28T11:09:17.792 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:09:17.792 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2023-11-28T11:09:17.792 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2023-11-28T11:09:17.793 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.2:0", 2023-11-28T11:09:17.793 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3218438733 2023-11-28T11:09:17.793 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:09:17.793 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:09:17.793 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2023-11-28T11:09:17.793 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2023-11-28T11:09:17.793 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.2:0", 2023-11-28T11:09:17.794 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 3218438733 2023-11-28T11:09:17.794 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:09:17.794 INFO:teuthology.orchestra.run.smithi028.stdout: "inst_str": "client.4821 192.168.0.2:0/3218438733", 2023-11-28T11:09:17.794 INFO:teuthology.orchestra.run.smithi028.stdout: "addr_str": "192.168.0.2:0/3218438733", 2023-11-28T11:09:17.794 INFO:teuthology.orchestra.run.smithi028.stdout: "inode_count": 1, 2023-11-28T11:09:17.794 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_epoch": 9, 2023-11-28T11:09:17.794 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch": 25, 2023-11-28T11:09:17.795 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch_barrier": 0, 2023-11-28T11:09:17.795 INFO:teuthology.orchestra.run.smithi028.stdout: "blocklisted": false, 2023-11-28T11:09:17.795 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs" 2023-11-28T11:09:17.795 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:09:17.800 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:09:17.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.923+0000 7f2195ea3700 1 Processor -- start 2023-11-28T11:09:17.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.923+0000 7f2195ea3700 1 -- start start 2023-11-28T11:09:17.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.924+0000 7f2195ea3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f219015a6c0 0x7f21900cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.924+0000 7f2195ea3700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21900cfb90 con 0x7f21900d0020 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.924+0000 7f2195ea3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21900cb930 con 0x7f21900b2660 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.924+0000 7f2195ea3700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21900cf260 con 0x7f219015a6c0 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.924+0000 7f218effd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f21900d0020 0x7f21900ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35626/0 (socket says 172.21.15.28:35626) 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.924+0000 7f218effd700 1 -- 172.21.15.28:0/3043074663 learned_addr learned my addr 172.21.15.28:0/3043074663 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.924+0000 7f218f7fe700 1 --2- 172.21.15.28:0/3043074663 >> v2:172.21.15.59:3300/0 conn(0x7f219015a6c0 0x7f21900cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.924+0000 7f218e7fc700 1 -- 172.21.15.28:0/3043074663 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 743610053 0 0) 0x7f21900cfb90 con 0x7f21900d0020 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.924+0000 7f218e7fc700 1 -- 172.21.15.28:0/3043074663 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f217c0035b0 con 0x7f21900d0020 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 -- 172.21.15.28:0/3043074663 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 930853444 0 0) 0x7f217c0035b0 con 0x7f21900d0020 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 -- 172.21.15.28:0/3043074663 >> v2:172.21.15.59:3300/0 conn(0x7f219015a6c0 msgr2=0x7f21900cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:17.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 --2- 172.21.15.28:0/3043074663 >> v2:172.21.15.59:3300/0 conn(0x7f219015a6c0 0x7f21900cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 -- 172.21.15.28:0/3043074663 >> v1:172.21.15.59:6789/0 conn(0x7f21900b2660 legacy=0x7f21900b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 -- 172.21.15.28:0/3043074663 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21900cf540 con 0x7f21900d0020 2023-11-28T11:09:17.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 -- 172.21.15.28:0/3043074663 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f2178001670 con 0x7f21900d0020 2023-11-28T11:09:17.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 -- 172.21.15.28:0/3043074663 >> v1:172.21.15.28:6789/0 conn(0x7f21900d0020 legacy=0x7f21900ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 --2- 172.21.15.28:0/3043074663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f217c003a00 0x7f217c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 --2- 172.21.15.28:0/3043074663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f217c004850 0x7f217c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 --2- 172.21.15.28:0/3043074663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f217c005200 0x7f217c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 -- 172.21.15.28:0/3043074663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21900cf260 con 0x7f217c003a00 2023-11-28T11:09:17.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 -- 172.21.15.28:0/3043074663 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f217c00c060 con 0x7f217c004850 2023-11-28T11:09:17.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 -- 172.21.15.28:0/3043074663 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f217c00c220 con 0x7f217c005200 2023-11-28T11:09:17.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218f7fe700 1 --2- 172.21.15.28:0/3043074663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f217c003a00 0x7f217c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218effd700 1 --2- 172.21.15.28:0/3043074663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f217c004850 0x7f217c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218ffff700 1 --2- 172.21.15.28:0/3043074663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f217c005200 0x7f217c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218f7fe700 1 -- 172.21.15.28:0/3043074663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f217c005200 msgr2=0x7f217c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:17.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218f7fe700 1 --2- 172.21.15.28:0/3043074663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f217c005200 0x7f217c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218f7fe700 1 -- 172.21.15.28:0/3043074663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f217c004850 msgr2=0x7f217c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:17.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218f7fe700 1 --2- 172.21.15.28:0/3043074663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f217c004850 0x7f217c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218f7fe700 1 -- 172.21.15.28:0/3043074663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f217c0035b0 con 0x7f217c003a00 2023-11-28T11:09:17.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218effd700 1 --2- 172.21.15.28:0/3043074663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f217c004850 0x7f217c004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:17.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218ffff700 1 -- 172.21.15.28:0/3043074663 reap_dead start 2023-11-28T11:09:17.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218f7fe700 1 --2- 172.21.15.28:0/3043074663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f217c003a00 0x7f217c003e10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2180009b60 tx=0x7f217c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74d1ec8f30e8a50 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:17.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 -- 172.21.15.28:0/3043074663 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218000e6c0 con 0x7f217c003a00 2023-11-28T11:09:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 -- 172.21.15.28:0/3043074663 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f218000e820 con 0x7f217c003a00 2023-11-28T11:09:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.925+0000 7f218e7fc700 1 -- 172.21.15.28:0/3043074663 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f218000ebb0 con 0x7f217c003a00 2023-11-28T11:09:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.926+0000 7f2195ea3700 1 -- 172.21.15.28:0/3043074663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f217c003a00 msgr2=0x7f217c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.926+0000 7f2195ea3700 1 --2- 172.21.15.28:0/3043074663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f217c003a00 0x7f217c003e10 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f2180009b60 tx=0x7f217c005880 comp rx=0 tx=0).stop 2023-11-28T11:09:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.926+0000 7f2195ea3700 1 -- 172.21.15.28:0/3043074663 shutdown_connections 2023-11-28T11:09:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.926+0000 7f2195ea3700 1 --2- 172.21.15.28:0/3043074663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f217c003a00 0x7f217c003e10 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.926+0000 7f2195ea3700 1 -- 172.21.15.28:0/3043074663 >> 172.21.15.28:0/3043074663 conn(0x7f21900cd660 msgr2=0x7f21900cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.926+0000 7f2195ea3700 1 -- 172.21.15.28:0/3043074663 shutdown_connections 2023-11-28T11:09:17.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.926+0000 7f2195ea3700 1 -- 172.21.15.28:0/3043074663 wait complete. 2023-11-28T11:09:17.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.928+0000 7f2195ea3700 1 Processor -- start 2023-11-28T11:09:17.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.928+0000 7f2195ea3700 1 -- start start 2023-11-28T11:09:17.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f2195ea3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f217c003a00 0x7f21900ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f2195ea3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2190158380 0x7f21901587f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f2195ea3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21900a7780 0x7f21900a7bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f2195ea3700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2190158d30 con 0x7f21900a7780 2023-11-28T11:09:17.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f2195ea3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21900a8530 con 0x7f2190158380 2023-11-28T11:09:17.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f2195ea3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21900a8810 con 0x7f217c003a00 2023-11-28T11:09:17.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f218ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21900a7780 0x7f21900a7bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f218ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21900a7780 0x7f21900a7bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47582/0 (socket says 172.21.15.28:47582) 2023-11-28T11:09:17.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f218ffff700 1 -- 172.21.15.28:0/284312442 learned_addr learned my addr 172.21.15.28:0/284312442 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:17.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f218effd700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2190158380 0x7f21901587f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f218f7fe700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f217c003a00 0x7f21900ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f218ffff700 1 -- 172.21.15.28:0/284312442 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f217c003a00 msgr2=0x7f21900ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f218ffff700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f217c003a00 0x7f21900ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f218ffff700 1 -- 172.21.15.28:0/284312442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2190158380 msgr2=0x7f21901587f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:17.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f218ffff700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2190158380 0x7f21901587f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f218ffff700 1 -- 172.21.15.28:0/284312442 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21900d0470 con 0x7f21900a7780 2023-11-28T11:09:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f218effd700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2190158380 0x7f21901587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f218f7fe700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f217c003a00 0x7f21900ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.929+0000 7f218ffff700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21900a7780 0x7f21900a7bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2190072970 tx=0x7f218800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.930+0000 7f2187fff700 1 -- 172.21.15.28:0/284312442 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188003c10 con 0x7f21900a7780 2023-11-28T11:09:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.930+0000 7f2187fff700 1 -- 172.21.15.28:0/284312442 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2188003d70 con 0x7f21900a7780 2023-11-28T11:09:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.930+0000 7f2195ea3700 1 -- 172.21.15.28:0/284312442 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21900cf820 con 0x7f21900a7780 2023-11-28T11:09:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.930+0000 7f2187fff700 1 -- 172.21.15.28:0/284312442 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2188005310 con 0x7f21900a7780 2023-11-28T11:09:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.930+0000 7f2195ea3700 1 -- 172.21.15.28:0/284312442 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21900d0600 con 0x7f21900a7780 2023-11-28T11:09:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.930+0000 7f2195ea3700 1 -- 172.21.15.28:0/284312442 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21900c7d40 con 0x7f21900a7780 2023-11-28T11:09:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.931+0000 7f2187fff700 1 -- 172.21.15.28:0/284312442 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2188005470 con 0x7f21900a7780 2023-11-28T11:09:17.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.931+0000 7f2187fff700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f21700682f0 0x7f217006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:17.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.931+0000 7f2187fff700 1 -- 172.21.15.28:0/284312442 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2188086140 con 0x7f21900a7780 2023-11-28T11:09:17.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.931+0000 7f218f7fe700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f21700682f0 0x7f217006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:17.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.932+0000 7f218f7fe700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f21700682f0 0x7f217006a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f21800090b0 tx=0x7f2180008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:17.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:17.936+0000 7f2187fff700 1 -- 172.21.15.28:0/284312442 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f21900c7d40 con 0x7f21900a7780 2023-11-28T11:09:18.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:18.177+0000 7f2195ea3700 1 -- 172.21.15.28:0/284312442 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f21900c7d40 con 0x7f21900a7780 2023-11-28T11:09:18.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:18.177+0000 7f2187fff700 1 -- 172.21.15.28:0/284312442 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f21900c7d40 con 0x7f21900a7780 2023-11-28T11:09:18.179 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T11:09:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:18.179+0000 7f2195ea3700 1 -- 172.21.15.28:0/284312442 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f21700682f0 msgr2=0x7f217006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:18.179+0000 7f2195ea3700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f21700682f0 0x7f217006a7a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f21800090b0 tx=0x7f2180008030 comp rx=0 tx=0).stop 2023-11-28T11:09:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:18.179+0000 7f2195ea3700 1 -- 172.21.15.28:0/284312442 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21900a7780 msgr2=0x7f21900a7bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:18.179+0000 7f2195ea3700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21900a7780 0x7f21900a7bf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2190072970 tx=0x7f218800ee80 comp rx=0 tx=0).stop 2023-11-28T11:09:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:18.179+0000 7f2195ea3700 1 -- 172.21.15.28:0/284312442 shutdown_connections 2023-11-28T11:09:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:18.179+0000 7f2195ea3700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f21700682f0 0x7f217006a7a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:18.179+0000 7f2195ea3700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21900a7780 0x7f21900a7bf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:18.179+0000 7f2195ea3700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f217c003a00 0x7f21900ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:18.179+0000 7f2195ea3700 1 --2- 172.21.15.28:0/284312442 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2190158380 0x7f21901587f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:18.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:18.179+0000 7f2195ea3700 1 -- 172.21.15.28:0/284312442 >> 172.21.15.28:0/284312442 conn(0x7f21900cd660 msgr2=0x7f21900cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:18.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:18.179+0000 7f2195ea3700 1 -- 172.21.15.28:0/284312442 shutdown_connections 2023-11-28T11:09:18.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:18.179+0000 7f2195ea3700 1 -- 172.21.15.28:0/284312442 wait complete. 2023-11-28T11:09:18.192 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:09:18.192 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:09:18.193 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2023-11-28T11:09:18.193 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-28T11:09:18.193 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T11:09:18.193 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:09:18.193 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:09:18.193 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:09:18.193 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-28T11:09:18.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:18.193 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2023-11-28T11:09:18.215 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:18.215 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:18.216 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:18.216 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:18.216 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:09:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T11:09:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42315sec preferred_lft 42315sec 2023-11-28T11:09:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2023-11-28T11:09:18.216 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:18.216 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:18.217 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:18.217 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T11:09:18.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:18.217 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-28T11:09:18.217 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-28T11:09:18.217 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add name ceph-brx type bridge 2023-11-28T11:09:18.217 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr flush dev ceph-brx 2023-11-28T11:09:18.217 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set ceph-brx up 2023-11-28T11:09:18.217 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T11:09:18.218 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-28T11:09:18.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:18.393 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:09:18.422 INFO:teuthology.orchestra.run.smithi059.stdout:1 2023-11-28T11:09:18.424 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:18.424 DEBUG:teuthology.orchestra.run.smithi059:> route 2023-11-28T11:09:18.500 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2023-11-28T11:09:18.500 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:09:18.500 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T11:09:18.500 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T11:09:18.500 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:09:18.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:18.501 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-28T11:09:18.501 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-28T11:09:18.501 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T11:09:18.501 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T11:09:18.501 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T11:09:18.501 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-28T11:09:18.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:18.762 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-11-28T11:09:18.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:18.779 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2023-11-28T11:09:18.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:18.836 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-28T11:09:18.836 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-28T11:09:18.837 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:09:18.837 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-28T11:09:18.837 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-28T11:09:18.933 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-28T11:09:18.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:18.933 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-28T11:09:18.933 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-28T11:09:18.934 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-28T11:09:18.934 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T11:09:18.934 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-28T11:09:18.934 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-28T11:09:18.934 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-28T11:09:18.934 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-28T11:09:19.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:19.151 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-28T11:09:19.151 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-28T11:09:19.151 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2023-11-28T11:09:19.151 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T11:09:19.151 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-28T11:09:19.257 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-11-28T11:09:19.257 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:19.257 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-11-28T11:09:19.279 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-11-28T11:09:19.279 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:19.280 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-11-28T11:09:19.336 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2023-11-28T11:09:19.404 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:09:19.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:09:19.472 INFO:teuthology.orchestra.run.smithi059.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:09:19.473 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:09:19.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:19.475 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2023-11-28T11:09:19.531 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T11:09:19.531 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1 --client_mountpoint=/) 2023-11-28T11:09:19.573 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2023-11-28T11:09:19.600 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:09:19.600 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:09:19.667 INFO:teuthology.orchestra.run.smithi059.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:09:19.668 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:09:19.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:19.669 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2023-11-28T11:09:19.707 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi059.stderr:2023-11-28T11:09:19.706+0000 7f40a0f365c0 -1 init, newargv = 0x559a1812b660 newargc=15 2023-11-28T11:09:19.708 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi059.stderr:ceph-fuse[110065]: starting ceph client 2023-11-28T11:09:19.712 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi059.stderr:ceph-fuse[110065]: starting fuse 2023-11-28T11:09:19.725 INFO:teuthology.orchestra.run.smithi059.stdout:47 2023-11-28T11:09:19.725 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T11:09:19.725 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:19.726 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2023-11-28T11:09:19.726 DEBUG:teuthology.orchestra.run.smithi059:> import re 2023-11-28T11:09:19.726 DEBUG:teuthology.orchestra.run.smithi059:> import os 2023-11-28T11:09:19.726 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2023-11-28T11:09:19.726 DEBUG:teuthology.orchestra.run.smithi059:> 2023-11-28T11:09:19.726 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2023-11-28T11:09:19.727 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-11-28T11:09:19.727 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2023-11-28T11:09:19.727 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-11-28T11:09:19.727 DEBUG:teuthology.orchestra.run.smithi059:> 2023-11-28T11:09:19.727 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2023-11-28T11:09:19.728 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2023-11-28T11:09:19.728 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2023-11-28T11:09:19.728 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2023-11-28T11:09:19.728 DEBUG:teuthology.orchestra.run.smithi059:> 2023-11-28T11:09:19.728 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2023-11-28T11:09:19.728 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T11:09:19.728 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T11:09:19.728 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T11:09:19.729 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2023-11-28T11:09:19.729 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2023-11-28T11:09:19.729 DEBUG:teuthology.orchestra.run.smithi059:> return f 2023-11-28T11:09:19.729 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T11:09:19.729 DEBUG:teuthology.orchestra.run.smithi059:> 2023-11-28T11:09:19.729 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.1")) 2023-11-28T11:09:19.730 DEBUG:teuthology.orchestra.run.smithi059:> ' 2023-11-28T11:09:19.850 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.1.110065.asok 2023-11-28T11:09:20.033 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.110065.asok 2023-11-28T11:09:20.033 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:20.033 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.110065.asok status 2023-11-28T11:09:20.196 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-28T11:09:20.196 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2023-11-28T11:09:20.196 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T11:09:20.196 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T11:09:20.196 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "1", 2023-11-28T11:09:20.196 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2023-11-28T11:09:20.196 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-11-28T11:09:20.196 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "110065", 2023-11-28T11:09:20.197 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2023-11-28T11:09:20.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-28T11:09:20.197 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2023-11-28T11:09:20.197 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2023-11-28T11:09:20.197 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 4702, 2023-11-28T11:09:20.197 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2023-11-28T11:09:20.197 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2023-11-28T11:09:20.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2023-11-28T11:09:20.197 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 4702 2023-11-28T11:09:20.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-28T11:09:20.197 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2023-11-28T11:09:20.198 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2023-11-28T11:09:20.198 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2023-11-28T11:09:20.198 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2772146442 2023-11-28T11:09:20.198 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-28T11:09:20.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-28T11:09:20.198 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2023-11-28T11:09:20.198 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2023-11-28T11:09:20.198 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2023-11-28T11:09:20.198 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2772146442 2023-11-28T11:09:20.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-28T11:09:20.198 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.4702 192.168.0.1:0/2772146442", 2023-11-28T11:09:20.198 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/2772146442", 2023-11-28T11:09:20.199 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2023-11-28T11:09:20.199 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 9, 2023-11-28T11:09:20.199 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 25, 2023-11-28T11:09:20.199 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2023-11-28T11:09:20.199 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2023-11-28T11:09:20.199 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2023-11-28T11:09:20.199 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-28T11:09:20.206 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:09:20.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.343+0000 7f6bc36eb700 1 Processor -- start 2023-11-28T11:09:20.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.343+0000 7f6bc36eb700 1 -- start start 2023-11-28T11:09:20.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.344+0000 7f6bc36eb700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6bbc06e400 0x7f6bbc15db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.344+0000 7f6bc36eb700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bbc15e0b0 con 0x7f6bbc0ce890 2023-11-28T11:09:20.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.344+0000 7f6bc36eb700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bbc15d120 con 0x7f6bbc159590 2023-11-28T11:09:20.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.344+0000 7f6bc36eb700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bbc15d6e0 con 0x7f6bbc06e400 2023-11-28T11:09:20.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.344+0000 7f6bc0c86700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6bbc06e400 0x7f6bbc15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.345+0000 7f6bc0c86700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6bbc06e400 0x7f6bbc15db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47588/0 (socket says 172.21.15.28:47588) 2023-11-28T11:09:20.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.345+0000 7f6bc0c86700 1 -- 172.21.15.28:0/3632832634 learned_addr learned my addr 172.21.15.28:0/3632832634 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:20.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.345+0000 7f6bc1c88700 1 --1- 172.21.15.28:0/3632832634 >> v1:172.21.15.28:6789/0 conn(0x7f6bbc0ce890 0x7f6bbc0caf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35648/0 (socket says 172.21.15.28:35648) 2023-11-28T11:09:20.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.345+0000 7f6bc0c86700 1 -- 172.21.15.28:0/3632832634 >> v1:172.21.15.59:6789/0 conn(0x7f6bbc159590 legacy=0x7f6bbc0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.345+0000 7f6bc0c86700 1 -- 172.21.15.28:0/3632832634 >> v1:172.21.15.28:6789/0 conn(0x7f6bbc0ce890 legacy=0x7f6bbc0caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.345+0000 7f6bc0c86700 1 -- 172.21.15.28:0/3632832634 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bbc0aef60 con 0x7f6bbc06e400 2023-11-28T11:09:20.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.345+0000 7f6bc0c86700 1 --2- 172.21.15.28:0/3632832634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bbc06e400 0x7f6bbc15db70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80098f0 tx=0x7f6bb8009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5feebf60e6f35e0a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.345+0000 7f6bb3fff700 1 -- 172.21.15.28:0/3632832634 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb800b8a0 con 0x7f6bbc06e400 2023-11-28T11:09:20.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.345+0000 7f6bb3fff700 1 -- 172.21.15.28:0/3632832634 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb800ba00 con 0x7f6bbc06e400 2023-11-28T11:09:20.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.345+0000 7f6bb3fff700 1 -- 172.21.15.28:0/3632832634 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb800bd90 con 0x7f6bbc06e400 2023-11-28T11:09:20.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.346+0000 7f6bc36eb700 1 -- 172.21.15.28:0/3632832634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bbc06e400 msgr2=0x7f6bbc15db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.346+0000 7f6bc36eb700 1 --2- 172.21.15.28:0/3632832634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bbc06e400 0x7f6bbc15db70 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f6bb80098f0 tx=0x7f6bb8009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:20.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.346+0000 7f6bc36eb700 1 -- 172.21.15.28:0/3632832634 shutdown_connections 2023-11-28T11:09:20.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.346+0000 7f6bc36eb700 1 --2- 172.21.15.28:0/3632832634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bbc06e400 0x7f6bbc15db70 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.346+0000 7f6bc36eb700 1 -- 172.21.15.28:0/3632832634 >> 172.21.15.28:0/3632832634 conn(0x7f6bbc0b1650 msgr2=0x7f6bbc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:20.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.346+0000 7f6bc36eb700 1 -- 172.21.15.28:0/3632832634 shutdown_connections 2023-11-28T11:09:20.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.346+0000 7f6bc36eb700 1 -- 172.21.15.28:0/3632832634 wait complete. 2023-11-28T11:09:20.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.347+0000 7f6bc36eb700 1 Processor -- start 2023-11-28T11:09:20.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.347+0000 7f6bc36eb700 1 -- start start 2023-11-28T11:09:20.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc36eb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6bbc06e400 0x7f6bbc06b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc36eb700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bbc0ce890 0x7f6bbc0acd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc36eb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bbc159590 0x7f6bbc1f2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc36eb700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc1f60b0 con 0x7f6bbc0ce890 2023-11-28T11:09:20.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc36eb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc0ad2b0 con 0x7f6bbc159590 2023-11-28T11:09:20.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc36eb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bbc1f5cf0 con 0x7f6bbc06e400 2023-11-28T11:09:20.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc0c86700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bbc0ce890 0x7f6bbc0acd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc0c86700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bbc0ce890 0x7f6bbc0acd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47594/0 (socket says 172.21.15.28:47594) 2023-11-28T11:09:20.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc0c86700 1 -- 172.21.15.28:0/3446633872 learned_addr learned my addr 172.21.15.28:0/3446633872 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:20.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc1c88700 1 --2- 172.21.15.28:0/3446633872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bbc159590 0x7f6bbc1f2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc0c86700 1 -- 172.21.15.28:0/3446633872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6bbc06e400 msgr2=0x7f6bbc06b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc1487700 1 --2- 172.21.15.28:0/3446633872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6bbc06e400 0x7f6bbc06b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc0c86700 1 --2- 172.21.15.28:0/3446633872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6bbc06e400 0x7f6bbc06b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc0c86700 1 -- 172.21.15.28:0/3446633872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bbc159590 msgr2=0x7f6bbc1f2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc0c86700 1 --2- 172.21.15.28:0/3446633872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bbc159590 0x7f6bbc1f2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.348+0000 7f6bc0c86700 1 -- 172.21.15.28:0/3446633872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb8009550 con 0x7f6bbc0ce890 2023-11-28T11:09:20.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.349+0000 7f6bc0c86700 1 --2- 172.21.15.28:0/3446633872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bbc0ce890 0x7f6bbc0acd70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8000c00 tx=0x7f6bb8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.349+0000 7f6bb1ffb700 1 -- 172.21.15.28:0/3446633872 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8004570 con 0x7f6bbc0ce890 2023-11-28T11:09:20.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.349+0000 7f6bb1ffb700 1 -- 172.21.15.28:0/3446633872 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb801f070 con 0x7f6bbc0ce890 2023-11-28T11:09:20.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.349+0000 7f6bc36eb700 1 -- 172.21.15.28:0/3446633872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bbc15d370 con 0x7f6bbc0ce890 2023-11-28T11:09:20.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.349+0000 7f6bb1ffb700 1 -- 172.21.15.28:0/3446633872 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb80038a0 con 0x7f6bbc0ce890 2023-11-28T11:09:20.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.349+0000 7f6bc36eb700 1 -- 172.21.15.28:0/3446633872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bbc0af240 con 0x7f6bbc0ce890 2023-11-28T11:09:20.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.350+0000 7f6bc36eb700 1 -- 172.21.15.28:0/3446633872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bbc159960 con 0x7f6bbc0ce890 2023-11-28T11:09:20.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.351+0000 7f6bb1ffb700 1 -- 172.21.15.28:0/3446633872 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6bb802a020 con 0x7f6bbc0ce890 2023-11-28T11:09:20.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.351+0000 7f6bb1ffb700 1 --2- 172.21.15.28:0/3446633872 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6ba00682f0 0x7f6ba006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:20.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.351+0000 7f6bb1ffb700 1 -- 172.21.15.28:0/3446633872 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f6bb8087600 con 0x7f6bbc0ce890 2023-11-28T11:09:20.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.351+0000 7f6bc1487700 1 --2- 172.21.15.28:0/3446633872 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6ba00682f0 0x7f6ba006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:20.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.351+0000 7f6bc1487700 1 --2- 172.21.15.28:0/3446633872 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6ba00682f0 0x7f6ba006a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6bac000f60 tx=0x7f6bac009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:20.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.355+0000 7f6bb1ffb700 1 -- 172.21.15.28:0/3446633872 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6bb8053b70 con 0x7f6bbc0ce890 2023-11-28T11:09:20.598 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.596+0000 7f6bc36eb700 1 -- 172.21.15.28:0/3446633872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6bbc0675a0 con 0x7f6bbc0ce890 2023-11-28T11:09:20.598 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.597+0000 7f6bb1ffb700 1 -- 172.21.15.28:0/3446633872 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v9) v1 ==== 53+0+88 (secure 0 0 0) 0x7f6bb8057190 con 0x7f6bbc0ce890 2023-11-28T11:09:20.599 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-11-28T11:09:20.600 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.599+0000 7f6bc36eb700 1 -- 172.21.15.28:0/3446633872 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6ba00682f0 msgr2=0x7f6ba006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.601 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.599+0000 7f6bc36eb700 1 --2- 172.21.15.28:0/3446633872 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6ba00682f0 0x7f6ba006a7a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f6bac000f60 tx=0x7f6bac009040 comp rx=0 tx=0).stop 2023-11-28T11:09:20.601 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.599+0000 7f6bc36eb700 1 -- 172.21.15.28:0/3446633872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bbc0ce890 msgr2=0x7f6bbc0acd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:20.601 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.599+0000 7f6bc36eb700 1 --2- 172.21.15.28:0/3446633872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bbc0ce890 0x7f6bbc0acd70 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f6bb8000c00 tx=0x7f6bb8004750 comp rx=0 tx=0).stop 2023-11-28T11:09:20.601 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.599+0000 7f6bc36eb700 1 -- 172.21.15.28:0/3446633872 shutdown_connections 2023-11-28T11:09:20.601 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.599+0000 7f6bc36eb700 1 --2- 172.21.15.28:0/3446633872 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6ba00682f0 0x7f6ba006a7a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.601 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.599+0000 7f6bc36eb700 1 --2- 172.21.15.28:0/3446633872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6bbc159590 0x7f6bbc1f2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.599+0000 7f6bc36eb700 1 --2- 172.21.15.28:0/3446633872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6bbc06e400 0x7f6bbc06b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.599+0000 7f6bc36eb700 1 --2- 172.21.15.28:0/3446633872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6bbc0ce890 0x7f6bbc0acd70 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:20.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.599+0000 7f6bc36eb700 1 -- 172.21.15.28:0/3446633872 >> 172.21.15.28:0/3446633872 conn(0x7f6bbc0b1650 msgr2=0x7f6bbc0af850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:20.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.599+0000 7f6bc36eb700 1 -- 172.21.15.28:0/3446633872 shutdown_connections 2023-11-28T11:09:20.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:20.599+0000 7f6bc36eb700 1 -- 172.21.15.28:0/3446633872 wait complete. 2023-11-28T11:09:20.612 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:09:20.612 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:09:20.612 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2023-11-28T11:09:20.612 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-28T11:09:20.613 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T11:09:20.613 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:09:20.613 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T11:09:20.613 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:09:20.613 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-28T11:09:20.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:20.613 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2023-11-28T11:09:20.632 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:20.633 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:20.633 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:20.633 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:20.633 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:20.633 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:20.633 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:20.633 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:20.633 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:20.633 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:20.633 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:09:20.633 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:20.634 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T11:09:20.634 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42313sec preferred_lft 42313sec 2023-11-28T11:09:20.634 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link 2023-11-28T11:09:20.634 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:20.634 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:09:20.634 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:20.634 INFO:teuthology.orchestra.run.smithi059.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:09:20.634 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether d6:2c:b7:b9:23:4b brd ff:ff:ff:ff:ff:ff 2023-11-28T11:09:20.634 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:09:20.634 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:20.634 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::b424:acff:fec7:20c2/64 scope link 2023-11-28T11:09:20.634 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:20.635 INFO:teuthology.orchestra.run.smithi059.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:09:20.635 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether d6:2c:b7:b9:23:4b brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:09:20.635 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::d42c:b7ff:feb9:234b/64 scope link tentative 2023-11-28T11:09:20.635 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:20.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:20.635 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:09:20.702 INFO:teuthology.orchestra.run.smithi059.stdout:1 2023-11-28T11:09:20.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:20.703 DEBUG:teuthology.orchestra.run.smithi059:> route 2023-11-28T11:09:20.765 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2023-11-28T11:09:20.765 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:09:20.765 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T11:09:20.765 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T11:09:20.765 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:09:20.766 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:20.766 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-28T11:09:20.766 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-28T11:09:20.766 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T11:09:20.766 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T11:09:20.767 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T11:09:20.767 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-28T11:09:20.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:20.876 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-11-28T11:09:20.894 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T11:09:20.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:20.895 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2023-11-28T11:09:20.914 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:09:20.918 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:09:20.952 INFO:teuthology.orchestra.run.smithi059.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-28T11:09:20.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:20.952 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-28T11:09:20.952 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-28T11:09:20.952 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T11:09:20.953 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-28T11:09:20.953 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-28T11:09:21.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:21.085 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T11:09:21.124 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:21.124 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:21.125 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:21.125 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:21.125 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:21.125 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:21.125 INFO:teuthology.orchestra.run.smithi059.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:09:21.125 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 82:16:f9:e5:b3:60 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:09:21.125 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:09:21.125 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:21.126 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::8016:f9ff:fee5:b360/64 scope link 2023-11-28T11:09:21.126 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:21.137 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:21.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T11:09:21.222 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:09:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:09:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:09:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2023-11-28T11:09:21.222 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:21.223 INFO:teuthology.orchestra.run.smithi059.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:09:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 82:16:f9:e5:b3:60 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T11:09:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T11:09:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::8016:f9ff:fee5:b360/64 scope link 2023-11-28T11:09:21.223 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:09:21.235 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-28T11:09:21.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:21.235 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-28T11:09:21.235 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-28T11:09:21.235 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-28T11:09:21.235 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T11:09:21.235 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-28T11:09:21.236 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-28T11:09:21.236 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-28T11:09:21.236 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-28T11:09:21.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:21.441 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec bash -c ' 2023-11-28T11:09:21.441 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2023-11-28T11:09:21.441 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.1 up 2023-11-28T11:09:21.441 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T11:09:21.442 DEBUG:teuthology.orchestra.run.smithi059:> ') 2023-11-28T11:09:21.575 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-11-28T11:09:21.575 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:21.576 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-11-28T11:09:21.598 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-11-28T11:09:21.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:09:21.598 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-11-28T11:09:21.655 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2023-11-28T11:09:21.722 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:09:21.723 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:09:21.789 INFO:teuthology.orchestra.run.smithi059.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:09:21.790 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:09:21.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:21.791 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2023-11-28T11:09:21.848 INFO:teuthology.orchestra.run.smithi059.stdout:47 2023-11-28T11:09:21.849 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [47] 2023-11-28T11:09:21.849 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3 --client_mountpoint=/) 2023-11-28T11:09:21.892 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2023-11-28T11:09:21.920 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:09:21.920 DEBUG:teuthology.orchestra.run.smithi059:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:09:21.981 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi059.stderr:2023-11-28T11:09:21.979+0000 7f15a281b5c0 -1 init, newargv = 0x55870b6448b0 newargc=15 2023-11-28T11:09:21.981 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi059.stderr:ceph-fuse[110673]: starting ceph client 2023-11-28T11:09:21.986 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi059.stderr:ceph-fuse[110673]: starting fuse 2023-11-28T11:09:21.988 INFO:teuthology.orchestra.run.smithi059.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:09:21.989 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:09:21.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:21.990 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2023-11-28T11:09:22.047 INFO:teuthology.orchestra.run.smithi059.stdout:47 2023-11-28T11:09:22.047 INFO:teuthology.orchestra.run.smithi059.stdout:48 2023-11-28T11:09:22.048 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47, 48] 2023-11-28T11:09:22.048 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:09:22.048 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2023-11-28T11:09:22.048 DEBUG:teuthology.orchestra.run.smithi059:> import re 2023-11-28T11:09:22.048 DEBUG:teuthology.orchestra.run.smithi059:> import os 2023-11-28T11:09:22.048 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2023-11-28T11:09:22.048 DEBUG:teuthology.orchestra.run.smithi059:> 2023-11-28T11:09:22.048 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2023-11-28T11:09:22.048 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-11-28T11:09:22.048 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2023-11-28T11:09:22.049 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-11-28T11:09:22.049 DEBUG:teuthology.orchestra.run.smithi059:> 2023-11-28T11:09:22.049 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2023-11-28T11:09:22.049 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2023-11-28T11:09:22.049 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2023-11-28T11:09:22.049 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2023-11-28T11:09:22.049 DEBUG:teuthology.orchestra.run.smithi059:> 2023-11-28T11:09:22.049 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2023-11-28T11:09:22.049 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T11:09:22.049 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T11:09:22.049 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T11:09:22.049 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2023-11-28T11:09:22.049 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2023-11-28T11:09:22.050 DEBUG:teuthology.orchestra.run.smithi059:> return f 2023-11-28T11:09:22.050 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T11:09:22.050 DEBUG:teuthology.orchestra.run.smithi059:> 2023-11-28T11:09:22.050 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.3")) 2023-11-28T11:09:22.050 DEBUG:teuthology.orchestra.run.smithi059:> ' 2023-11-28T11:09:22.176 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.3.110673.asok 2023-11-28T11:09:22.358 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.110673.asok 2023-11-28T11:09:22.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.359 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.110673.asok status 2023-11-28T11:09:22.519 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2023-11-28T11:09:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2023-11-28T11:09:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T11:09:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T11:09:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "3", 2023-11-28T11:09:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2023-11-28T11:09:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-11-28T11:09:22.519 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "110673", 2023-11-28T11:09:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2023-11-28T11:09:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-28T11:09:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2023-11-28T11:09:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2023-11-28T11:09:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 4714, 2023-11-28T11:09:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2023-11-28T11:09:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2023-11-28T11:09:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2023-11-28T11:09:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 4714 2023-11-28T11:09:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-28T11:09:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2023-11-28T11:09:22.520 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2023-11-28T11:09:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.2:0", 2023-11-28T11:09:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 11899908 2023-11-28T11:09:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: } 2023-11-28T11:09:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-28T11:09:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2023-11-28T11:09:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2023-11-28T11:09:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.2:0", 2023-11-28T11:09:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 11899908 2023-11-28T11:09:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2023-11-28T11:09:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.4714 192.168.0.2:0/11899908", 2023-11-28T11:09:22.521 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.2:0/11899908", 2023-11-28T11:09:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2023-11-28T11:09:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 9, 2023-11-28T11:09:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 25, 2023-11-28T11:09:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2023-11-28T11:09:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2023-11-28T11:09:22.522 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2023-11-28T11:09:22.525 INFO:teuthology.orchestra.run.smithi059.stdout:} 2023-11-28T11:09:22.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.529 DEBUG:teuthology.orchestra.run.smithi028:> stat --file-system '--printf=%T 2023-11-28T11:09:22.529 DEBUG:teuthology.orchestra.run.smithi028:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:22.551 INFO:teuthology.orchestra.run.smithi028.stdout:fuseblk 2023-11-28T11:09:22.551 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:22.551 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.551 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:22.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.625 DEBUG:teuthology.orchestra.run.smithi028:> stat --file-system '--printf=%T 2023-11-28T11:09:22.625 DEBUG:teuthology.orchestra.run.smithi028:> ' -- /home/ubuntu/cephtest/mnt.2 2023-11-28T11:09:22.686 INFO:teuthology.orchestra.run.smithi028.stdout:fuseblk 2023-11-28T11:09:22.687 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-11-28T11:09:22.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.687 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-28T11:09:22.757 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.758 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2023-11-28T11:09:22.758 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.1 2023-11-28T11:09:22.776 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2023-11-28T11:09:22.777 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-11-28T11:09:22.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.777 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-28T11:09:22.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.846 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2023-11-28T11:09:22.846 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.3 2023-11-28T11:09:22.905 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2023-11-28T11:09:22.905 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-11-28T11:09:22.906 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:22.906 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-28T11:09:22.973 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-28T11:09:22.990 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:23.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.127+0000 7f21fcc65700 1 Processor -- start 2023-11-28T11:09:23.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.127+0000 7f21fcc65700 1 -- start start 2023-11-28T11:09:23.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.128+0000 7f21fcc65700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f21f815d1e0 0x7f21f815d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.128+0000 7f21fcc65700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21f806aa00 0x7f21f80cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.128+0000 7f21fcc65700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f806ae80 con 0x7f21f806a560 2023-11-28T11:09:23.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.128+0000 7f21fcc65700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f80cdec0 con 0x7f21f815d1e0 2023-11-28T11:09:23.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.128+0000 7f21fcc65700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80c9cd0 con 0x7f21f806aa00 2023-11-28T11:09:23.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.129+0000 7f21f659c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f21f815d1e0 0x7f21f815d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.129+0000 7f21f5d9b700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f21f806a560 0x7f21f80cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35662/0 (socket says 172.21.15.28:35662) 2023-11-28T11:09:23.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.129+0000 7f21f5d9b700 1 -- 172.21.15.28:0/211376396 learned_addr learned my addr 172.21.15.28:0/211376396 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:23.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.129+0000 7f21f6d9d700 1 --2- 172.21.15.28:0/211376396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21f806aa00 0x7f21f80cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.129+0000 7f21f659c700 1 -- 172.21.15.28:0/211376396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21f806aa00 msgr2=0x7f21f80cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:23.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.129+0000 7f21f659c700 1 --2- 172.21.15.28:0/211376396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21f806aa00 0x7f21f80cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.129+0000 7f21f659c700 1 -- 172.21.15.28:0/211376396 >> v1:172.21.15.28:6789/0 conn(0x7f21f806a560 legacy=0x7f21f80cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.129+0000 7f21f659c700 1 -- 172.21.15.28:0/211376396 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f80c9fb0 con 0x7f21f815d1e0 2023-11-28T11:09:23.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.129+0000 7f21f559a700 1 -- 172.21.15.28:0/211376396 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1758500193 0 0) 0x7f21f806ae80 con 0x7f21f806a560 2023-11-28T11:09:23.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.129+0000 7f21f6d9d700 1 --2- 172.21.15.28:0/211376396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21f806aa00 0x7f21f80cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:23.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.129+0000 7f21f659c700 1 --2- 172.21.15.28:0/211376396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f815d1e0 0x7f21f815d5b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009840 tx=0x7f21ec009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cca61a5c2793fb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.129+0000 7f21f559a700 1 -- 172.21.15.28:0/211376396 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec011830 con 0x7f21f815d1e0 2023-11-28T11:09:23.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.129+0000 7f21f559a700 1 -- 172.21.15.28:0/211376396 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ec011990 con 0x7f21f815d1e0 2023-11-28T11:09:23.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.129+0000 7f21f559a700 1 -- 172.21.15.28:0/211376396 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec011d20 con 0x7f21f815d1e0 2023-11-28T11:09:23.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.130+0000 7f21fcc65700 1 -- 172.21.15.28:0/211376396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f815d1e0 msgr2=0x7f21f815d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.130+0000 7f21fcc65700 1 --2- 172.21.15.28:0/211376396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f815d1e0 0x7f21f815d5b0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f21ec009840 tx=0x7f21ec009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:23.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.130+0000 7f21fcc65700 1 -- 172.21.15.28:0/211376396 shutdown_connections 2023-11-28T11:09:23.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.130+0000 7f21fcc65700 1 --2- 172.21.15.28:0/211376396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21f806aa00 0x7f21f80cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.130+0000 7f21fcc65700 1 --2- 172.21.15.28:0/211376396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f815d1e0 0x7f21f815d5b0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.130+0000 7f21fcc65700 1 -- 172.21.15.28:0/211376396 >> 172.21.15.28:0/211376396 conn(0x7f21f80b0dd0 msgr2=0x7f21f80b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:23.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.130+0000 7f21fcc65700 1 -- 172.21.15.28:0/211376396 shutdown_connections 2023-11-28T11:09:23.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.130+0000 7f21fcc65700 1 -- 172.21.15.28:0/211376396 wait complete. 2023-11-28T11:09:23.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.131+0000 7f21fcc65700 1 Processor -- start 2023-11-28T11:09:23.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.131+0000 7f21fcc65700 1 -- start start 2023-11-28T11:09:23.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.132+0000 7f21fcc65700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21f806a560 0x7f21f80b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.132+0000 7f21fcc65700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f806aa00 0x7f21f8159550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.132+0000 7f21fcc65700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21f815d1e0 0x7f21f80b3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.132+0000 7f21fcc65700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f815a1e0 con 0x7f21f806aa00 2023-11-28T11:09:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.132+0000 7f21fcc65700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80b2780 con 0x7f21f806a560 2023-11-28T11:09:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.132+0000 7f21fcc65700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f80b2a60 con 0x7f21f815d1e0 2023-11-28T11:09:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.132+0000 7f21f5d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f806aa00 0x7f21f8159550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.132+0000 7f21f5d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f806aa00 0x7f21f8159550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47610/0 (socket says 172.21.15.28:47610) 2023-11-28T11:09:23.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.132+0000 7f21f5d9b700 1 -- 172.21.15.28:0/532472253 learned_addr learned my addr 172.21.15.28:0/532472253 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:23.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.132+0000 7f21f659c700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21f806a560 0x7f21f80b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.133+0000 7f21f6d9d700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21f815d1e0 0x7f21f80b3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.133+0000 7f21f5d9b700 1 -- 172.21.15.28:0/532472253 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21f815d1e0 msgr2=0x7f21f80b3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.133+0000 7f21f5d9b700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21f815d1e0 0x7f21f80b3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.133+0000 7f21f5d9b700 1 -- 172.21.15.28:0/532472253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21f806a560 msgr2=0x7f21f80b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.133+0000 7f21f5d9b700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21f806a560 0x7f21f80b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.133+0000 7f21f5d9b700 1 -- 172.21.15.28:0/532472253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21ec0094a0 con 0x7f21f806aa00 2023-11-28T11:09:23.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.133+0000 7f21f659c700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21f806a560 0x7f21f80b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:23.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.133+0000 7f21f6d9d700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21f815d1e0 0x7f21f80b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:23.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.133+0000 7f21f5d9b700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f806aa00 0x7f21f8159550 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f21e800c550 tx=0x7f21e800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.133+0000 7f21e6ffd700 1 -- 172.21.15.28:0/532472253 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e8004c80 con 0x7f21f806aa00 2023-11-28T11:09:23.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.133+0000 7f21e6ffd700 1 -- 172.21.15.28:0/532472253 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e8004e00 con 0x7f21f806aa00 2023-11-28T11:09:23.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.133+0000 7f21fcc65700 1 -- 172.21.15.28:0/532472253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f80ca570 con 0x7f21f806aa00 2023-11-28T11:09:23.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.133+0000 7f21e6ffd700 1 -- 172.21.15.28:0/532472253 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e8007310 con 0x7f21f806aa00 2023-11-28T11:09:23.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.133+0000 7f21fcc65700 1 -- 172.21.15.28:0/532472253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f80ca290 con 0x7f21f806aa00 2023-11-28T11:09:23.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.134+0000 7f21fcc65700 1 -- 172.21.15.28:0/532472253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f80ca850 con 0x7f21f806aa00 2023-11-28T11:09:23.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.134+0000 7f21e6ffd700 1 -- 172.21.15.28:0/532472253 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f21e801a020 con 0x7f21f806aa00 2023-11-28T11:09:23.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.135+0000 7f21e6ffd700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f21d80682f0 0x7f21d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.148 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.135+0000 7f21e6ffd700 1 -- 172.21.15.28:0/532472253 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f21e8085c30 con 0x7f21f806aa00 2023-11-28T11:09:23.148 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.135+0000 7f21f659c700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f21d80682f0 0x7f21d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.148 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.135+0000 7f21f659c700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f21d80682f0 0x7f21d806a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f21f80b0a10 tx=0x7f21ec008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.139+0000 7f21e6ffd700 1 -- 172.21.15.28:0/532472253 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f21e80521f0 con 0x7f21f806aa00 2023-11-28T11:09:23.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.336+0000 7f21fcc65700 1 -- 172.21.15.28:0/532472253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21f80b1970 con 0x7f21f806aa00 2023-11-28T11:09:23.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.337+0000 7f21e6ffd700 1 -- 172.21.15.28:0/532472253 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f21e8055810 con 0x7f21f806aa00 2023-11-28T11:09:23.342 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:23.342 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:50.531733+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.339+0000 7f21fcc65700 1 -- 172.21.15.28:0/532472253 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f21d80682f0 msgr2=0x7f21d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.339+0000 7f21fcc65700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f21d80682f0 0x7f21d806a7a0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f21f80b0a10 tx=0x7f21ec008030 comp rx=0 tx=0).stop 2023-11-28T11:09:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.339+0000 7f21fcc65700 1 -- 172.21.15.28:0/532472253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f806aa00 msgr2=0x7f21f8159550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.339+0000 7f21fcc65700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f806aa00 0x7f21f8159550 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f21e800c550 tx=0x7f21e800c910 comp rx=0 tx=0).stop 2023-11-28T11:09:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.339+0000 7f21fcc65700 1 -- 172.21.15.28:0/532472253 shutdown_connections 2023-11-28T11:09:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.339+0000 7f21fcc65700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f21d80682f0 0x7f21d806a7a0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.339+0000 7f21fcc65700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f21f815d1e0 0x7f21f80b3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.339+0000 7f21fcc65700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21f806a560 0x7f21f80b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.339+0000 7f21fcc65700 1 --2- 172.21.15.28:0/532472253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f21f806aa00 0x7f21f8159550 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.339+0000 7f21fcc65700 1 -- 172.21.15.28:0/532472253 >> 172.21.15.28:0/532472253 conn(0x7f21f80b0dd0 msgr2=0x7f21f815ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:23.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.339+0000 7f21fcc65700 1 -- 172.21.15.28:0/532472253 shutdown_connections 2023-11-28T11:09:23.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.339+0000 7f21fcc65700 1 -- 172.21.15.28:0/532472253 wait complete. 2023-11-28T11:09:23.353 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:23.354 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:23.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.481+0000 7fe83215a700 1 Processor -- start 2023-11-28T11:09:23.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.481+0000 7fe83215a700 1 -- start start 2023-11-28T11:09:23.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.482+0000 7fe83215a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe82c159500 0x7fe82c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.482+0000 7fe83215a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe82c0b1d40 0x7fe82c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.482+0000 7fe83215a700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe82c15e040 con 0x7fe82c15dbf0 2023-11-28T11:09:23.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.482+0000 7fe83215a700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82c0cede0 con 0x7fe82c159500 2023-11-28T11:09:23.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.482+0000 7fe83215a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe82c0ceff0 con 0x7fe82c0b1d40 2023-11-28T11:09:23.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.482+0000 7fe82b7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe82c159500 0x7fe82c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.482+0000 7fe82affd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe82c15dbf0 0x7fe82c0b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35674/0 (socket says 172.21.15.28:35674) 2023-11-28T11:09:23.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.482+0000 7fe82b7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe82c159500 0x7fe82c0cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47616/0 (socket says 172.21.15.28:47616) 2023-11-28T11:09:23.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.482+0000 7fe82affd700 1 -- 172.21.15.28:0/2716694064 learned_addr learned my addr 172.21.15.28:0/2716694064 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:23.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.482+0000 7fe82bfff700 1 --2- 172.21.15.28:0/2716694064 >> v2:172.21.15.59:3300/0 conn(0x7fe82c0b1d40 0x7fe82c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.483+0000 7fe82b7fe700 1 -- 172.21.15.28:0/2716694064 >> v2:172.21.15.59:3300/0 conn(0x7fe82c0b1d40 msgr2=0x7fe82c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:23.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.483+0000 7fe82b7fe700 1 --2- 172.21.15.28:0/2716694064 >> v2:172.21.15.59:3300/0 conn(0x7fe82c0b1d40 0x7fe82c0ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.483+0000 7fe82b7fe700 1 -- 172.21.15.28:0/2716694064 >> v1:172.21.15.28:6789/0 conn(0x7fe82c15dbf0 legacy=0x7fe82c0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.483+0000 7fe82a7fc700 1 -- 172.21.15.28:0/2716694064 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2811836141 0 0) 0x7fe82c15e040 con 0x7fe82c15dbf0 2023-11-28T11:09:23.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.483+0000 7fe82b7fe700 1 -- 172.21.15.28:0/2716694064 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82c0cf2d0 con 0x7fe82c159500 2023-11-28T11:09:23.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.483+0000 7fe82b7fe700 1 --2- 172.21.15.28:0/2716694064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe82c159500 0x7fe82c0cd140 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0cfb50 tx=0x7fe81c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5907b97eb07984 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.483+0000 7fe82a7fc700 1 -- 172.21.15.28:0/2716694064 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c011830 con 0x7fe82c159500 2023-11-28T11:09:23.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.483+0000 7fe82a7fc700 1 -- 172.21.15.28:0/2716694064 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe81c011990 con 0x7fe82c159500 2023-11-28T11:09:23.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.483+0000 7fe82a7fc700 1 -- 172.21.15.28:0/2716694064 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe81c011d20 con 0x7fe82c159500 2023-11-28T11:09:23.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.483+0000 7fe83215a700 1 -- 172.21.15.28:0/2716694064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe82c159500 msgr2=0x7fe82c0cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.483+0000 7fe83215a700 1 --2- 172.21.15.28:0/2716694064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe82c159500 0x7fe82c0cd140 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe82c0cfb50 tx=0x7fe81c009b20 comp rx=0 tx=0).stop 2023-11-28T11:09:23.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.484+0000 7fe83215a700 1 -- 172.21.15.28:0/2716694064 shutdown_connections 2023-11-28T11:09:23.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.484+0000 7fe83215a700 1 --2- 172.21.15.28:0/2716694064 >> v2:172.21.15.59:3300/0 conn(0x7fe82c0b1d40 0x7fe82c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.484+0000 7fe83215a700 1 --2- 172.21.15.28:0/2716694064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe82c159500 0x7fe82c0cd140 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.484+0000 7fe83215a700 1 -- 172.21.15.28:0/2716694064 >> 172.21.15.28:0/2716694064 conn(0x7fe82c0cc630 msgr2=0x7fe82c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:23.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.484+0000 7fe83215a700 1 -- 172.21.15.28:0/2716694064 shutdown_connections 2023-11-28T11:09:23.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.484+0000 7fe83215a700 1 -- 172.21.15.28:0/2716694064 wait complete. 2023-11-28T11:09:23.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.485+0000 7fe83215a700 1 Processor -- start 2023-11-28T11:09:23.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.485+0000 7fe83215a700 1 -- start start 2023-11-28T11:09:23.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe83215a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe82c0b1d40 0x7fe82c0c6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe83215a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe82c159500 0x7fe82c0b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe83215a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe82c15dbf0 0x7fe82c14d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe83215a700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c0b3930 con 0x7fe82c159500 2023-11-28T11:09:23.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe83215a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c0b4770 con 0x7fe82c15dbf0 2023-11-28T11:09:23.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe83215a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe82c14d5d0 con 0x7fe82c0b1d40 2023-11-28T11:09:23.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe82affd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe82c159500 0x7fe82c0b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe82affd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe82c159500 0x7fe82c0b33f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47624/0 (socket says 172.21.15.28:47624) 2023-11-28T11:09:23.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe82affd700 1 -- 172.21.15.28:0/477591084 learned_addr learned my addr 172.21.15.28:0/477591084 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:23.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe82bfff700 1 --2- 172.21.15.28:0/477591084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe82c15dbf0 0x7fe82c14d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe82affd700 1 -- 172.21.15.28:0/477591084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe82c0b1d40 msgr2=0x7fe82c0c6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe82b7fe700 1 --2- 172.21.15.28:0/477591084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe82c0b1d40 0x7fe82c0c6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe82affd700 1 --2- 172.21.15.28:0/477591084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe82c0b1d40 0x7fe82c0c6020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe82affd700 1 -- 172.21.15.28:0/477591084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe82c15dbf0 msgr2=0x7fe82c14d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe82affd700 1 --2- 172.21.15.28:0/477591084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe82c15dbf0 0x7fe82c14d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe82affd700 1 -- 172.21.15.28:0/477591084 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81c0094a0 con 0x7fe82c159500 2023-11-28T11:09:23.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe82bfff700 1 --2- 172.21.15.28:0/477591084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe82c15dbf0 0x7fe82c14d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:23.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.486+0000 7fe82affd700 1 --2- 172.21.15.28:0/477591084 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe82c159500 0x7fe82c0b33f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe81400c550 tx=0x7fe81400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.487+0000 7fe823fff700 1 -- 172.21.15.28:0/477591084 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe814004c80 con 0x7fe82c159500 2023-11-28T11:09:23.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.487+0000 7fe823fff700 1 -- 172.21.15.28:0/477591084 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe814004de0 con 0x7fe82c159500 2023-11-28T11:09:23.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.487+0000 7fe83215a700 1 -- 172.21.15.28:0/477591084 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe82c15d3f0 con 0x7fe82c159500 2023-11-28T11:09:23.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.487+0000 7fe823fff700 1 -- 172.21.15.28:0/477591084 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8140072d0 con 0x7fe82c159500 2023-11-28T11:09:23.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.487+0000 7fe83215a700 1 -- 172.21.15.28:0/477591084 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe82c15d110 con 0x7fe82c159500 2023-11-28T11:09:23.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.487+0000 7fe83215a700 1 -- 172.21.15.28:0/477591084 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82c15d6d0 con 0x7fe82c159500 2023-11-28T11:09:23.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.488+0000 7fe823fff700 1 -- 172.21.15.28:0/477591084 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe81401a020 con 0x7fe82c159500 2023-11-28T11:09:23.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.488+0000 7fe823fff700 1 --2- 172.21.15.28:0/477591084 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe80c0682f0 0x7fe80c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.488+0000 7fe823fff700 1 -- 172.21.15.28:0/477591084 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe814085bf0 con 0x7fe82c159500 2023-11-28T11:09:23.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.488+0000 7fe82b7fe700 1 --2- 172.21.15.28:0/477591084 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe80c0682f0 0x7fe80c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.489+0000 7fe82b7fe700 1 --2- 172.21.15.28:0/477591084 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe80c0682f0 0x7fe80c06a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe81c000c00 tx=0x7fe81c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.493+0000 7fe823fff700 1 -- 172.21.15.28:0/477591084 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe814052210 con 0x7fe82c159500 2023-11-28T11:09:23.685 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.684+0000 7fe83215a700 1 -- 172.21.15.28:0/477591084 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe82c0b4920 con 0x7fe82c159500 2023-11-28T11:09:23.686 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.684+0000 7fe823fff700 1 -- 172.21.15.28:0/477591084 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe82c0b4920 con 0x7fe82c159500 2023-11-28T11:09:23.686 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:09:23.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.686+0000 7fe83215a700 1 -- 172.21.15.28:0/477591084 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe80c0682f0 msgr2=0x7fe80c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.686+0000 7fe83215a700 1 --2- 172.21.15.28:0/477591084 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe80c0682f0 0x7fe80c06a7a0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe81c000c00 tx=0x7fe81c008030 comp rx=0 tx=0).stop 2023-11-28T11:09:23.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.686+0000 7fe83215a700 1 -- 172.21.15.28:0/477591084 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe82c159500 msgr2=0x7fe82c0b33f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.686+0000 7fe83215a700 1 --2- 172.21.15.28:0/477591084 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe82c159500 0x7fe82c0b33f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe81400c550 tx=0x7fe81400c910 comp rx=0 tx=0).stop 2023-11-28T11:09:23.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.686+0000 7fe83215a700 1 -- 172.21.15.28:0/477591084 shutdown_connections 2023-11-28T11:09:23.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.686+0000 7fe83215a700 1 --2- 172.21.15.28:0/477591084 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe80c0682f0 0x7fe80c06a7a0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.686+0000 7fe83215a700 1 --2- 172.21.15.28:0/477591084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe82c15dbf0 0x7fe82c14d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.686+0000 7fe83215a700 1 --2- 172.21.15.28:0/477591084 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe82c0b1d40 0x7fe82c0c6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.686+0000 7fe83215a700 1 --2- 172.21.15.28:0/477591084 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe82c159500 0x7fe82c0b33f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.687+0000 7fe83215a700 1 -- 172.21.15.28:0/477591084 >> 172.21.15.28:0/477591084 conn(0x7fe82c0cc630 msgr2=0x7fe82c0cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:23.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.687+0000 7fe83215a700 1 -- 172.21.15.28:0/477591084 shutdown_connections 2023-11-28T11:09:23.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.687+0000 7fe83215a700 1 -- 172.21.15.28:0/477591084 wait complete. 2023-11-28T11:09:23.701 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:23.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.827+0000 7fa1fdca7700 1 Processor -- start 2023-11-28T11:09:23.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.827+0000 7fa1fdca7700 1 -- start start 2023-11-28T11:09:23.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.828+0000 7fa1fdca7700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa1f80ce560 0x7fa1f81589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.828+0000 7fa1fdca7700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa1f806e900 0x7fa1f806ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.828+0000 7fa1fdca7700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1f806e470 con 0x7fa1f806a1e0 2023-11-28T11:09:23.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.828+0000 7fa1fdca7700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f80ce930 con 0x7fa1f806e900 2023-11-28T11:09:23.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.828+0000 7fa1fdca7700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1f806de60 con 0x7fa1f80ce560 2023-11-28T11:09:23.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.828+0000 7fa1f6ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa1f806e900 0x7fa1f806ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.828+0000 7fa1f7fff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa1f806a1e0 0x7fa1f806a640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35692/0 (socket says 172.21.15.28:35692) 2023-11-28T11:09:23.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.828+0000 7fa1f6ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa1f806e900 0x7fa1f806ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47630/0 (socket says 172.21.15.28:47630) 2023-11-28T11:09:23.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.828+0000 7fa1f7fff700 1 -- 172.21.15.28:0/937091983 learned_addr learned my addr 172.21.15.28:0/937091983 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:23.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.828+0000 7fa1f77fe700 1 --2- 172.21.15.28:0/937091983 >> v2:172.21.15.59:3300/0 conn(0x7fa1f80ce560 0x7fa1f81589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.828+0000 7fa1f6ffd700 1 -- 172.21.15.28:0/937091983 >> v2:172.21.15.59:3300/0 conn(0x7fa1f80ce560 msgr2=0x7fa1f81589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:23.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.828+0000 7fa1f6ffd700 1 --2- 172.21.15.28:0/937091983 >> v2:172.21.15.59:3300/0 conn(0x7fa1f80ce560 0x7fa1f81589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.828+0000 7fa1f6ffd700 1 -- 172.21.15.28:0/937091983 >> v1:172.21.15.28:6789/0 conn(0x7fa1f806a1e0 legacy=0x7fa1f806a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.828+0000 7fa1f67fc700 1 -- 172.21.15.28:0/937091983 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3689406738 0 0) 0x7fa1f806e470 con 0x7fa1f806a1e0 2023-11-28T11:09:23.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.828+0000 7fa1f6ffd700 1 -- 172.21.15.28:0/937091983 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1f80ceb40 con 0x7fa1f806e900 2023-11-28T11:09:23.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.829+0000 7fa1f6ffd700 1 --2- 172.21.15.28:0/937091983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1f806e900 0x7fa1f806ed70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80b1ef0 tx=0x7fa1e000bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8ecf577961edc1a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.829+0000 7fa1f67fc700 1 -- 172.21.15.28:0/937091983 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e000dc70 con 0x7fa1f806e900 2023-11-28T11:09:23.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.829+0000 7fa1f67fc700 1 -- 172.21.15.28:0/937091983 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e000ddd0 con 0x7fa1f806e900 2023-11-28T11:09:23.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.829+0000 7fa1f67fc700 1 -- 172.21.15.28:0/937091983 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e0010690 con 0x7fa1f806e900 2023-11-28T11:09:23.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.829+0000 7fa1fdca7700 1 -- 172.21.15.28:0/937091983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1f806e900 msgr2=0x7fa1f806ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.829+0000 7fa1fdca7700 1 --2- 172.21.15.28:0/937091983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1f806e900 0x7fa1f806ed70 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fa1f80b1ef0 tx=0x7fa1e000bdc0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.829+0000 7fa1fdca7700 1 -- 172.21.15.28:0/937091983 shutdown_connections 2023-11-28T11:09:23.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.829+0000 7fa1fdca7700 1 --2- 172.21.15.28:0/937091983 >> v2:172.21.15.59:3300/0 conn(0x7fa1f80ce560 0x7fa1f81589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.829+0000 7fa1fdca7700 1 --2- 172.21.15.28:0/937091983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1f806e900 0x7fa1f806ed70 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.829+0000 7fa1fdca7700 1 -- 172.21.15.28:0/937091983 >> 172.21.15.28:0/937091983 conn(0x7fa1f80b0df0 msgr2=0x7fa1f80b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:23.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.829+0000 7fa1fdca7700 1 -- 172.21.15.28:0/937091983 shutdown_connections 2023-11-28T11:09:23.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.829+0000 7fa1fdca7700 1 -- 172.21.15.28:0/937091983 wait complete. 2023-11-28T11:09:23.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.831+0000 7fa1fdca7700 1 Processor -- start 2023-11-28T11:09:23.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.831+0000 7fa1fdca7700 1 -- start start 2023-11-28T11:09:23.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.832+0000 7fa1fdca7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1f806a1e0 0x7fa1f81509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.832+0000 7fa1fdca7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1f806e900 0x7fa1f8151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.832+0000 7fa1fdca7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1f80ce560 0x7fa1f81597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.832+0000 7fa1fdca7700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f8151eb0 con 0x7fa1f80ce560 2023-11-28T11:09:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.832+0000 7fa1fdca7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f80c7e20 con 0x7fa1f806a1e0 2023-11-28T11:09:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.832+0000 7fa1fdca7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1f81520e0 con 0x7fa1f806e900 2023-11-28T11:09:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.832+0000 7fa1f7fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1f80ce560 0x7fa1f81597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.832+0000 7fa1f7fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1f80ce560 0x7fa1f81597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47640/0 (socket says 172.21.15.28:47640) 2023-11-28T11:09:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.832+0000 7fa1f7fff700 1 -- 172.21.15.28:0/2544863437 learned_addr learned my addr 172.21.15.28:0/2544863437 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.833+0000 7fa1f6ffd700 1 --2- 172.21.15.28:0/2544863437 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1f806e900 0x7fa1f8151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.833+0000 7fa1f77fe700 1 --2- 172.21.15.28:0/2544863437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1f806a1e0 0x7fa1f81509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.833+0000 7fa1f7fff700 1 -- 172.21.15.28:0/2544863437 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1f806e900 msgr2=0x7fa1f8151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.833+0000 7fa1f7fff700 1 --2- 172.21.15.28:0/2544863437 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1f806e900 0x7fa1f8151970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.833+0000 7fa1f7fff700 1 -- 172.21.15.28:0/2544863437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1f806a1e0 msgr2=0x7fa1f81509f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:23.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.833+0000 7fa1f7fff700 1 --2- 172.21.15.28:0/2544863437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1f806a1e0 0x7fa1f81509f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:23.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.833+0000 7fa1f7fff700 1 -- 172.21.15.28:0/2544863437 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e000b740 con 0x7fa1f80ce560 2023-11-28T11:09:23.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.833+0000 7fa1f77fe700 1 --2- 172.21.15.28:0/2544863437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1f806a1e0 0x7fa1f81509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:23.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.833+0000 7fa1f7fff700 1 --2- 172.21.15.28:0/2544863437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1f80ce560 0x7fa1f81597b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0002230 tx=0x7fa1f000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.833+0000 7fa1effff700 1 -- 172.21.15.28:0/2544863437 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f000ac20 con 0x7fa1f80ce560 2023-11-28T11:09:23.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.833+0000 7fa1effff700 1 -- 172.21.15.28:0/2544863437 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1f000ada0 con 0x7fa1f80ce560 2023-11-28T11:09:23.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.833+0000 7fa1fdca7700 1 -- 172.21.15.28:0/2544863437 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1f806e100 con 0x7fa1f80ce560 2023-11-28T11:09:23.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.833+0000 7fa1effff700 1 -- 172.21.15.28:0/2544863437 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1f0007310 con 0x7fa1f80ce560 2023-11-28T11:09:23.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.833+0000 7fa1fdca7700 1 -- 172.21.15.28:0/2544863437 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1f80cee20 con 0x7fa1f80ce560 2023-11-28T11:09:23.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.834+0000 7fa1fdca7700 1 -- 172.21.15.28:0/2544863437 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1f80c9d90 con 0x7fa1f80ce560 2023-11-28T11:09:23.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.834+0000 7fa1effff700 1 -- 172.21.15.28:0/2544863437 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa1f001a030 con 0x7fa1f80ce560 2023-11-28T11:09:23.839 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.835+0000 7fa1effff700 1 --2- 172.21.15.28:0/2544863437 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa1d80682f0 0x7fa1d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:23.839 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.835+0000 7fa1effff700 1 -- 172.21.15.28:0/2544863437 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa1f0085d40 con 0x7fa1f80ce560 2023-11-28T11:09:23.839 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.835+0000 7fa1f77fe700 1 --2- 172.21.15.28:0/2544863437 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa1d80682f0 0x7fa1d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:23.839 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.835+0000 7fa1f77fe700 1 --2- 172.21.15.28:0/2544863437 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa1d80682f0 0x7fa1d806a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009820 tx=0x7fa1e8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:23.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:23.839+0000 7fa1effff700 1 -- 172.21.15.28:0/2544863437 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa1f00522b0 con 0x7fa1f80ce560 2023-11-28T11:09:24.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.033+0000 7fa1fdca7700 1 -- 172.21.15.28:0/2544863437 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa1f80c7f90 con 0x7fa1f80ce560 2023-11-28T11:09:24.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.033+0000 7fa1effff700 1 -- 172.21.15.28:0/2544863437 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa1f00180a0 con 0x7fa1f80ce560 2023-11-28T11:09:24.035 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:09:24.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.035+0000 7fa1fdca7700 1 -- 172.21.15.28:0/2544863437 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa1d80682f0 msgr2=0x7fa1d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.035+0000 7fa1fdca7700 1 --2- 172.21.15.28:0/2544863437 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa1d80682f0 0x7fa1d806a7a0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8009820 tx=0x7fa1e8008040 comp rx=0 tx=0).stop 2023-11-28T11:09:24.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.035+0000 7fa1fdca7700 1 -- 172.21.15.28:0/2544863437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1f80ce560 msgr2=0x7fa1f81597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.035+0000 7fa1fdca7700 1 --2- 172.21.15.28:0/2544863437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1f80ce560 0x7fa1f81597b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa1f0002230 tx=0x7fa1f000a060 comp rx=0 tx=0).stop 2023-11-28T11:09:24.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.035+0000 7fa1fdca7700 1 -- 172.21.15.28:0/2544863437 shutdown_connections 2023-11-28T11:09:24.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.035+0000 7fa1fdca7700 1 --2- 172.21.15.28:0/2544863437 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa1d80682f0 0x7fa1d806a7a0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.035+0000 7fa1fdca7700 1 --2- 172.21.15.28:0/2544863437 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1f80ce560 0x7fa1f81597b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.035+0000 7fa1fdca7700 1 --2- 172.21.15.28:0/2544863437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1f806a1e0 0x7fa1f81509f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.035+0000 7fa1fdca7700 1 --2- 172.21.15.28:0/2544863437 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1f806e900 0x7fa1f8151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.035+0000 7fa1fdca7700 1 -- 172.21.15.28:0/2544863437 >> 172.21.15.28:0/2544863437 conn(0x7fa1f80b0df0 msgr2=0x7fa1f80b4590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:24.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.035+0000 7fa1fdca7700 1 -- 172.21.15.28:0/2544863437 shutdown_connections 2023-11-28T11:09:24.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.035+0000 7fa1fdca7700 1 -- 172.21.15.28:0/2544863437 wait complete. 2023-11-28T11:09:24.049 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.175+0000 7f7d74419700 1 Processor -- start 2023-11-28T11:09:24.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.175+0000 7f7d74419700 1 -- start start 2023-11-28T11:09:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.176+0000 7f7d74419700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d6c06b900 0x7f7d6c0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.176+0000 7f7d74419700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d6c0ae730 con 0x7f7d6c15a640 2023-11-28T11:09:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.176+0000 7f7d74419700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d6c0c3e10 con 0x7f7d6c0cd140 2023-11-28T11:09:24.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.176+0000 7f7d74419700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c0cb210 con 0x7f7d6c06b900 2023-11-28T11:09:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.176+0000 7f7d721b5700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7d6c15a640 0x7f7d6c0c3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35702/0 (socket says 172.21.15.28:35702) 2023-11-28T11:09:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.176+0000 7f7d721b5700 1 -- 172.21.15.28:0/3759618614 learned_addr learned my addr 172.21.15.28:0/3759618614 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.176+0000 7f7d729b6700 1 --2- 172.21.15.28:0/3759618614 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d6c06b900 0x7f7d6c0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.176+0000 7f7d711b3700 1 -- 172.21.15.28:0/3759618614 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2495051189 0 0) 0x7f7d6c0ae730 con 0x7f7d6c15a640 2023-11-28T11:09:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d711b3700 1 -- 172.21.15.28:0/3759618614 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d580035b0 con 0x7f7d6c15a640 2023-11-28T11:09:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d711b3700 1 -- 172.21.15.28:0/3759618614 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2539781809 0 0) 0x7f7d580035b0 con 0x7f7d6c15a640 2023-11-28T11:09:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d711b3700 1 -- 172.21.15.28:0/3759618614 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d6c06b900 msgr2=0x7f7d6c0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d711b3700 1 --2- 172.21.15.28:0/3759618614 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d6c06b900 0x7f7d6c0cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d711b3700 1 -- 172.21.15.28:0/3759618614 >> v1:172.21.15.59:6789/0 conn(0x7f7d6c0cd140 legacy=0x7f7d6c06b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d711b3700 1 -- 172.21.15.28:0/3759618614 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c06e9e0 con 0x7f7d6c15a640 2023-11-28T11:09:24.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d711b3700 1 -- 172.21.15.28:0/3759618614 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f7d5c001670 con 0x7f7d6c15a640 2023-11-28T11:09:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d711b3700 1 -- 172.21.15.28:0/3759618614 >> v1:172.21.15.28:6789/0 conn(0x7f7d6c15a640 legacy=0x7f7d6c0c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d711b3700 1 --2- 172.21.15.28:0/3759618614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d58003a30 0x7f7d58003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d711b3700 1 --2- 172.21.15.28:0/3759618614 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d6c06b900 0x7f7d58004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d711b3700 1 --2- 172.21.15.28:0/3759618614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d58004d90 0x7f7d58005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d711b3700 1 -- 172.21.15.28:0/3759618614 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c0cb210 con 0x7f7d58003a30 2023-11-28T11:09:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d711b3700 1 -- 172.21.15.28:0/3759618614 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5800ba10 con 0x7f7d58004d90 2023-11-28T11:09:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d711b3700 1 -- 172.21.15.28:0/3759618614 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d5800bbd0 con 0x7f7d6c06b900 2023-11-28T11:09:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d721b5700 1 --2- 172.21.15.28:0/3759618614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d58003a30 0x7f7d58003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d719b4700 1 --2- 172.21.15.28:0/3759618614 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d6c06b900 0x7f7d58004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d729b6700 1 --2- 172.21.15.28:0/3759618614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d58004d90 0x7f7d58005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d721b5700 1 -- 172.21.15.28:0/3759618614 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d6c06b900 msgr2=0x7f7d58004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:24.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d721b5700 1 --2- 172.21.15.28:0/3759618614 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d6c06b900 0x7f7d58004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d721b5700 1 -- 172.21.15.28:0/3759618614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d58004d90 msgr2=0x7f7d58005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d721b5700 1 --2- 172.21.15.28:0/3759618614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d58004d90 0x7f7d58005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d721b5700 1 -- 172.21.15.28:0/3759618614 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d580035b0 con 0x7f7d58003a30 2023-11-28T11:09:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d729b6700 1 --2- 172.21.15.28:0/3759618614 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d58004d90 0x7f7d58005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.177+0000 7f7d721b5700 1 --2- 172.21.15.28:0/3759618614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d58003a30 0x7f7d58003e00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00c2f0 tx=0x7f7d58005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d15e0405a159cb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.178+0000 7f7d711b3700 1 -- 172.21.15.28:0/3759618614 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c00ce30 con 0x7f7d58003a30 2023-11-28T11:09:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.178+0000 7f7d711b3700 1 -- 172.21.15.28:0/3759618614 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5c005960 con 0x7f7d58003a30 2023-11-28T11:09:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.178+0000 7f7d711b3700 1 -- 172.21.15.28:0/3759618614 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5c005cf0 con 0x7f7d58003a30 2023-11-28T11:09:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.178+0000 7f7d74419700 1 -- 172.21.15.28:0/3759618614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d58003a30 msgr2=0x7f7d58003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.178+0000 7f7d74419700 1 --2- 172.21.15.28:0/3759618614 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d58003a30 0x7f7d58003e00 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c00c2f0 tx=0x7f7d58005880 comp rx=0 tx=0).stop 2023-11-28T11:09:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.178+0000 7f7d729b6700 1 -- 172.21.15.28:0/3759618614 reap_dead start 2023-11-28T11:09:24.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.178+0000 7f7d74419700 1 -- 172.21.15.28:0/3759618614 shutdown_connections 2023-11-28T11:09:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.178+0000 7f7d74419700 1 -- 172.21.15.28:0/3759618614 >> 172.21.15.28:0/3759618614 conn(0x7f7d6c0cc630 msgr2=0x7f7d6c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.178+0000 7f7d74419700 1 -- 172.21.15.28:0/3759618614 shutdown_connections 2023-11-28T11:09:24.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.178+0000 7f7d74419700 1 -- 172.21.15.28:0/3759618614 wait complete. 2023-11-28T11:09:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.180+0000 7f7d74419700 1 Processor -- start 2023-11-28T11:09:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.180+0000 7f7d74419700 1 -- start start 2023-11-28T11:09:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d74419700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d58003a30 0x7f7d6c1f3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d74419700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d6c158c50 0x7f7d6c15b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d74419700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d6c155690 0x7f7d6c14cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d74419700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1f3ad0 con 0x7f7d6c155690 2023-11-28T11:09:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d74419700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1f3db0 con 0x7f7d58003a30 2023-11-28T11:09:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d74419700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d6c1f4090 con 0x7f7d6c158c50 2023-11-28T11:09:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d729b6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d6c155690 0x7f7d6c14cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d729b6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d6c155690 0x7f7d6c14cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47660/0 (socket says 172.21.15.28:47660) 2023-11-28T11:09:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d729b6700 1 -- 172.21.15.28:0/2350743828 learned_addr learned my addr 172.21.15.28:0/2350743828 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d721b5700 1 --2- 172.21.15.28:0/2350743828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d58003a30 0x7f7d6c1f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d729b6700 1 -- 172.21.15.28:0/2350743828 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d6c158c50 msgr2=0x7f7d6c15b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d719b4700 1 --2- 172.21.15.28:0/2350743828 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d6c158c50 0x7f7d6c15b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d729b6700 1 --2- 172.21.15.28:0/2350743828 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d6c158c50 0x7f7d6c15b1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d729b6700 1 -- 172.21.15.28:0/2350743828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d58003a30 msgr2=0x7f7d6c1f3480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d729b6700 1 --2- 172.21.15.28:0/2350743828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d58003a30 0x7f7d6c1f3480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d729b6700 1 -- 172.21.15.28:0/2350743828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d6c0aec60 con 0x7f7d6c155690 2023-11-28T11:09:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d721b5700 1 --2- 172.21.15.28:0/2350743828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d58003a30 0x7f7d6c1f3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.181+0000 7f7d729b6700 1 --2- 172.21.15.28:0/2350743828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d6c155690 0x7f7d6c14cc10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c06ae70 tx=0x7f7d6400f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.182+0000 7f7d62ffd700 1 -- 172.21.15.28:0/2350743828 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6400fd60 con 0x7f7d6c155690 2023-11-28T11:09:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.182+0000 7f7d62ffd700 1 -- 172.21.15.28:0/2350743828 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d64011040 con 0x7f7d6c155690 2023-11-28T11:09:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.182+0000 7f7d74419700 1 -- 172.21.15.28:0/2350743828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d6c0ae980 con 0x7f7d6c155690 2023-11-28T11:09:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.182+0000 7f7d62ffd700 1 -- 172.21.15.28:0/2350743828 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d6400fee0 con 0x7f7d6c155690 2023-11-28T11:09:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.182+0000 7f7d74419700 1 -- 172.21.15.28:0/2350743828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d6c0aef40 con 0x7f7d6c155690 2023-11-28T11:09:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.183+0000 7f7d74419700 1 -- 172.21.15.28:0/2350743828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6c0739f0 con 0x7f7d6c155690 2023-11-28T11:09:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.183+0000 7f7d62ffd700 1 -- 172.21.15.28:0/2350743828 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7d6401e030 con 0x7f7d6c155690 2023-11-28T11:09:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.184+0000 7f7d62ffd700 1 --2- 172.21.15.28:0/2350743828 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7d50068320 0x7f7d5006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.184+0000 7f7d721b5700 1 --2- 172.21.15.28:0/2350743828 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7d50068320 0x7f7d5006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.189 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.184+0000 7f7d62ffd700 1 -- 172.21.15.28:0/2350743828 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f7d64085d00 con 0x7f7d6c155690 2023-11-28T11:09:24.189 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.184+0000 7f7d721b5700 1 --2- 172.21.15.28:0/2350743828 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7d50068320 0x7f7d5006a7d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c015040 tx=0x7f7d5c007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.189 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.188+0000 7f7d62ffd700 1 -- 172.21.15.28:0/2350743828 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7d64089030 con 0x7f7d6c155690 2023-11-28T11:09:24.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.380+0000 7f7d74419700 1 -- 172.21.15.28:0/2350743828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7d6c0cafd0 con 0x7f7d6c155690 2023-11-28T11:09:24.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.381+0000 7f7d62ffd700 1 -- 172.21.15.28:0/2350743828 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7d64052270 con 0x7f7d6c155690 2023-11-28T11:09:24.383 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:24.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.383+0000 7f7d74419700 1 -- 172.21.15.28:0/2350743828 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7d50068320 msgr2=0x7f7d5006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.383+0000 7f7d74419700 1 --2- 172.21.15.28:0/2350743828 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7d50068320 0x7f7d5006a7d0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7d5c015040 tx=0x7f7d5c007000 comp rx=0 tx=0).stop 2023-11-28T11:09:24.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.383+0000 7f7d74419700 1 -- 172.21.15.28:0/2350743828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d6c155690 msgr2=0x7f7d6c14cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.383+0000 7f7d74419700 1 --2- 172.21.15.28:0/2350743828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d6c155690 0x7f7d6c14cc10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7d6c06ae70 tx=0x7f7d6400f3b0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.383+0000 7f7d74419700 1 -- 172.21.15.28:0/2350743828 shutdown_connections 2023-11-28T11:09:24.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.383+0000 7f7d74419700 1 --2- 172.21.15.28:0/2350743828 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7d50068320 0x7f7d5006a7d0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.383+0000 7f7d74419700 1 --2- 172.21.15.28:0/2350743828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7d6c155690 0x7f7d6c14cc10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.383+0000 7f7d74419700 1 --2- 172.21.15.28:0/2350743828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d58003a30 0x7f7d6c1f3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.383+0000 7f7d74419700 1 --2- 172.21.15.28:0/2350743828 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7d6c158c50 0x7f7d6c15b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.383+0000 7f7d74419700 1 -- 172.21.15.28:0/2350743828 >> 172.21.15.28:0/2350743828 conn(0x7f7d6c0cc630 msgr2=0x7f7d6c0b1bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:24.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.383+0000 7f7d74419700 1 -- 172.21.15.28:0/2350743828 shutdown_connections 2023-11-28T11:09:24.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.383+0000 7f7d74419700 1 -- 172.21.15.28:0/2350743828 wait complete. 2023-11-28T11:09:24.396 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:24.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.522+0000 7f3e0ff3e700 1 Processor -- start 2023-11-28T11:09:24.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.522+0000 7f3e0ff3e700 1 -- start start 2023-11-28T11:09:24.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.523+0000 7f3e0ff3e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e081589c0 0x7f3e080ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.523+0000 7f3e0ff3e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3e08158d90 0x7f3e0806aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.523+0000 7f3e0ff3e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3e0806e900 0x7f3e0806ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.523+0000 7f3e0ff3e700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0806e470 con 0x7f3e08158d90 2023-11-28T11:09:24.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.523+0000 7f3e0ff3e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e0806de60 con 0x7f3e0806e900 2023-11-28T11:09:24.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.523+0000 7f3e0ff3e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e080ceb40 con 0x7f3e081589c0 2023-11-28T11:09:24.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.523+0000 7f3e0d4d9700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3e08158d90 0x7f3e0806aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.523+0000 7f3e0d4d9700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3e08158d90 0x7f3e0806aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47664/0 (socket says 172.21.15.28:47664) 2023-11-28T11:09:24.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.523+0000 7f3e0d4d9700 1 -- 172.21.15.28:0/1091187423 learned_addr learned my addr 172.21.15.28:0/1091187423 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:24.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.523+0000 7f3e0dcda700 1 --2- 172.21.15.28:0/1091187423 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e081589c0 0x7f3e080ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.523+0000 7f3e0e4db700 1 --2- 172.21.15.28:0/1091187423 >> v2:172.21.15.59:3300/0 conn(0x7f3e0806e900 0x7f3e0806ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.523+0000 7f3e0d4d9700 1 -- 172.21.15.28:0/1091187423 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e081589c0 msgr2=0x7f3e080ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:24.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.523+0000 7f3e0d4d9700 1 --2- 172.21.15.28:0/1091187423 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e081589c0 0x7f3e080ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.523+0000 7f3e0d4d9700 1 -- 172.21.15.28:0/1091187423 >> v2:172.21.15.59:3300/0 conn(0x7f3e0806e900 msgr2=0x7f3e0806ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:24.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.523+0000 7f3e0d4d9700 1 --2- 172.21.15.28:0/1091187423 >> v2:172.21.15.59:3300/0 conn(0x7f3e0806e900 0x7f3e0806ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.524+0000 7f3e0d4d9700 1 -- 172.21.15.28:0/1091187423 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e080cee20 con 0x7f3e08158d90 2023-11-28T11:09:24.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.524+0000 7f3e0d4d9700 1 --2- 172.21.15.28:0/1091187423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e08158d90 0x7f3e0806aa70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3e080cd6a0 tx=0x7f3e04009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ce348da67fca702 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.524+0000 7f3e0ccd8700 1 -- 172.21.15.28:0/1091187423 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0400b870 con 0x7f3e08158d90 2023-11-28T11:09:24.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.524+0000 7f3e0ccd8700 1 -- 172.21.15.28:0/1091187423 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0400b9d0 con 0x7f3e08158d90 2023-11-28T11:09:24.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.524+0000 7f3e0ccd8700 1 -- 172.21.15.28:0/1091187423 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0400bd60 con 0x7f3e08158d90 2023-11-28T11:09:24.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.524+0000 7f3e0ff3e700 1 -- 172.21.15.28:0/1091187423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e08158d90 msgr2=0x7f3e0806aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.524+0000 7f3e0ff3e700 1 --2- 172.21.15.28:0/1091187423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e08158d90 0x7f3e0806aa70 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f3e080cd6a0 tx=0x7f3e04009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.524+0000 7f3e0ff3e700 1 -- 172.21.15.28:0/1091187423 shutdown_connections 2023-11-28T11:09:24.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.524+0000 7f3e0ff3e700 1 --2- 172.21.15.28:0/1091187423 >> v2:172.21.15.59:3300/0 conn(0x7f3e0806e900 0x7f3e0806ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.524+0000 7f3e0ff3e700 1 --2- 172.21.15.28:0/1091187423 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e081589c0 0x7f3e080ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.524+0000 7f3e0ff3e700 1 --2- 172.21.15.28:0/1091187423 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e08158d90 0x7f3e0806aa70 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.524+0000 7f3e0ff3e700 1 -- 172.21.15.28:0/1091187423 >> 172.21.15.28:0/1091187423 conn(0x7f3e080b0df0 msgr2=0x7f3e080b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:24.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.524+0000 7f3e0ff3e700 1 -- 172.21.15.28:0/1091187423 shutdown_connections 2023-11-28T11:09:24.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.524+0000 7f3e0ff3e700 1 -- 172.21.15.28:0/1091187423 wait complete. 2023-11-28T11:09:24.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.526+0000 7f3e0ff3e700 1 Processor -- start 2023-11-28T11:09:24.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.526+0000 7f3e0ff3e700 1 -- start start 2023-11-28T11:09:24.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.526+0000 7f3e0ff3e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e0806e900 0x7f3e081f33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.526+0000 7f3e0ff3e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e081589c0 0x7f3e081f5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.526+0000 7f3e0ff3e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e08158d90 0x7f3e080c4930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.526+0000 7f3e0ff3e700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e081e2e00 con 0x7f3e08158d90 2023-11-28T11:09:24.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.526+0000 7f3e0ff3e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e080c54a0 con 0x7f3e0806e900 2023-11-28T11:09:24.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.526+0000 7f3e0ff3e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e081e24d0 con 0x7f3e081589c0 2023-11-28T11:09:24.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3e0e4db700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e08158d90 0x7f3e080c4930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3e0e4db700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e08158d90 0x7f3e080c4930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47674/0 (socket says 172.21.15.28:47674) 2023-11-28T11:09:24.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3e0e4db700 1 -- 172.21.15.28:0/4288531679 learned_addr learned my addr 172.21.15.28:0/4288531679 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:24.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3e0dcda700 1 --2- 172.21.15.28:0/4288531679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e0806e900 0x7f3e081f33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3e0d4d9700 1 --2- 172.21.15.28:0/4288531679 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e081589c0 0x7f3e081f5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3e0e4db700 1 -- 172.21.15.28:0/4288531679 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e081589c0 msgr2=0x7f3e081f5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3e0e4db700 1 --2- 172.21.15.28:0/4288531679 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e081589c0 0x7f3e081f5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3e0e4db700 1 -- 172.21.15.28:0/4288531679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e0806e900 msgr2=0x7f3e081f33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3e0e4db700 1 --2- 172.21.15.28:0/4288531679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e0806e900 0x7f3e081f33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3e0e4db700 1 -- 172.21.15.28:0/4288531679 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e04009550 con 0x7f3e08158d90 2023-11-28T11:09:24.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3e0e4db700 1 --2- 172.21.15.28:0/4288531679 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e08158d90 0x7f3e080c4930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3e00007e60 tx=0x7f3e0000e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3dfe7fc700 1 -- 172.21.15.28:0/4288531679 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0000ed30 con 0x7f3e08158d90 2023-11-28T11:09:24.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3dfe7fc700 1 -- 172.21.15.28:0/4288531679 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e00014420 con 0x7f3e08158d90 2023-11-28T11:09:24.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3e0ff3e700 1 -- 172.21.15.28:0/4288531679 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e0806afb0 con 0x7f3e08158d90 2023-11-28T11:09:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3dfe7fc700 1 -- 172.21.15.28:0/4288531679 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e000147d0 con 0x7f3e08158d90 2023-11-28T11:09:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.527+0000 7f3e0ff3e700 1 -- 172.21.15.28:0/4288531679 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e0806e100 con 0x7f3e08158d90 2023-11-28T11:09:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.528+0000 7f3e0ff3e700 1 -- 172.21.15.28:0/4288531679 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e080c9d90 con 0x7f3e08158d90 2023-11-28T11:09:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.529+0000 7f3dfe7fc700 1 -- 172.21.15.28:0/4288531679 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3e00012050 con 0x7f3e08158d90 2023-11-28T11:09:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.529+0000 7f3dfe7fc700 1 --2- 172.21.15.28:0/4288531679 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3dec0682f0 0x7f3dec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.529+0000 7f3e0dcda700 1 --2- 172.21.15.28:0/4288531679 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3dec0682f0 0x7f3dec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.529+0000 7f3dfe7fc700 1 -- 172.21.15.28:0/4288531679 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3e000864a0 con 0x7f3e08158d90 2023-11-28T11:09:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.530+0000 7f3e0dcda700 1 --2- 172.21.15.28:0/4288531679 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3dec0682f0 0x7f3dec06a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3df8006b20 tx=0x7f3df8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.534+0000 7f3dfe7fc700 1 -- 172.21.15.28:0/4288531679 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3e00052a10 con 0x7f3e08158d90 2023-11-28T11:09:24.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.730+0000 7f3e0ff3e700 1 -- 172.21.15.28:0/4288531679 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3e080739f0 con 0x7f3e08158d90 2023-11-28T11:09:24.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.730+0000 7f3dfe7fc700 1 -- 172.21.15.28:0/4288531679 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3e00056030 con 0x7f3e08158d90 2023-11-28T11:09:24.732 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:09:24.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.733+0000 7f3e0ff3e700 1 -- 172.21.15.28:0/4288531679 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3dec0682f0 msgr2=0x7f3dec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.733+0000 7f3e0ff3e700 1 --2- 172.21.15.28:0/4288531679 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3dec0682f0 0x7f3dec06a7a0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f3df8006b20 tx=0x7f3df8008040 comp rx=0 tx=0).stop 2023-11-28T11:09:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.733+0000 7f3e0ff3e700 1 -- 172.21.15.28:0/4288531679 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e08158d90 msgr2=0x7f3e080c4930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.733+0000 7f3e0ff3e700 1 --2- 172.21.15.28:0/4288531679 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e08158d90 0x7f3e080c4930 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3e00007e60 tx=0x7f3e0000e3c0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.733+0000 7f3e0ff3e700 1 -- 172.21.15.28:0/4288531679 shutdown_connections 2023-11-28T11:09:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.733+0000 7f3e0ff3e700 1 --2- 172.21.15.28:0/4288531679 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3dec0682f0 0x7f3dec06a7a0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.733+0000 7f3e0ff3e700 1 --2- 172.21.15.28:0/4288531679 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e08158d90 0x7f3e080c4930 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.733+0000 7f3e0ff3e700 1 --2- 172.21.15.28:0/4288531679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e0806e900 0x7f3e081f33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.733+0000 7f3e0ff3e700 1 --2- 172.21.15.28:0/4288531679 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3e081589c0 0x7f3e081f5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.733+0000 7f3e0ff3e700 1 -- 172.21.15.28:0/4288531679 >> 172.21.15.28:0/4288531679 conn(0x7f3e080b0df0 msgr2=0x7f3e0806b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.733+0000 7f3e0ff3e700 1 -- 172.21.15.28:0/4288531679 shutdown_connections 2023-11-28T11:09:24.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.733+0000 7f3e0ff3e700 1 -- 172.21.15.28:0/4288531679 wait complete. 2023-11-28T11:09:24.746 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:24.874 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.873+0000 7fa7e6e19700 1 Processor -- start 2023-11-28T11:09:24.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.873+0000 7fa7e6e19700 1 -- start start 2023-11-28T11:09:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7e6e19700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7e00cb630 0x7fa7e00b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7e6e19700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e00cd2a0 con 0x7fa7e00b62d0 2023-11-28T11:09:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7e6e19700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7e00d0400 con 0x7fa7e00b2bc0 2023-11-28T11:09:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7e6e19700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e00cba80 con 0x7fa7e00cb630 2023-11-28T11:09:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7e4bb5700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa7e00b62d0 0x7fa7e006f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35736/0 (socket says 172.21.15.28:35736) 2023-11-28T11:09:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7e4bb5700 1 -- 172.21.15.28:0/967052117 learned_addr learned my addr 172.21.15.28:0/967052117 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7df7fe700 1 -- 172.21.15.28:0/967052117 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3407116358 0 0) 0x7fa7e00cd2a0 con 0x7fa7e00b62d0 2023-11-28T11:09:24.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7df7fe700 1 -- 172.21.15.28:0/967052117 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7d00035b0 con 0x7fa7e00b62d0 2023-11-28T11:09:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7dffff700 1 --2- 172.21.15.28:0/967052117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7e00cb630 0x7fa7e00b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7df7fe700 1 -- 172.21.15.28:0/967052117 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3061303826 0 0) 0x7fa7d00035b0 con 0x7fa7e00b62d0 2023-11-28T11:09:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7df7fe700 1 -- 172.21.15.28:0/967052117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7e00cb630 msgr2=0x7fa7e00b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7df7fe700 1 --2- 172.21.15.28:0/967052117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7e00cb630 0x7fa7e00b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7df7fe700 1 -- 172.21.15.28:0/967052117 >> v1:172.21.15.59:6789/0 conn(0x7fa7e00b2bc0 legacy=0x7fa7e00ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7df7fe700 1 -- 172.21.15.28:0/967052117 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e00d06e0 con 0x7fa7e00b62d0 2023-11-28T11:09:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7df7fe700 1 -- 172.21.15.28:0/967052117 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fa7d4001670 con 0x7fa7e00b62d0 2023-11-28T11:09:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.874+0000 7fa7df7fe700 1 -- 172.21.15.28:0/967052117 >> v1:172.21.15.28:6789/0 conn(0x7fa7e00b62d0 legacy=0x7fa7e006f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7df7fe700 1 --2- 172.21.15.28:0/967052117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7e00cb630 0x7fa7d0003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7df7fe700 1 --2- 172.21.15.28:0/967052117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d0003fb0 0x7fa7d0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7df7fe700 1 --2- 172.21.15.28:0/967052117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7d0004d90 0x7fa7d0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7df7fe700 1 -- 172.21.15.28:0/967052117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d000bac0 con 0x7fa7d0003fb0 2023-11-28T11:09:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7df7fe700 1 -- 172.21.15.28:0/967052117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d000bc30 con 0x7fa7d0004d90 2023-11-28T11:09:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7df7fe700 1 -- 172.21.15.28:0/967052117 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d000bdf0 con 0x7fa7e00cb630 2023-11-28T11:09:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7dffff700 1 --2- 172.21.15.28:0/967052117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d0003fb0 0x7fa7d0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7e4bb5700 1 --2- 172.21.15.28:0/967052117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7e00cb630 0x7fa7d0003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7e53b6700 1 --2- 172.21.15.28:0/967052117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7d0004d90 0x7fa7d0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7dffff700 1 -- 172.21.15.28:0/967052117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7e00cb630 msgr2=0x7fa7d0003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7dffff700 1 --2- 172.21.15.28:0/967052117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7e00cb630 0x7fa7d0003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7dffff700 1 -- 172.21.15.28:0/967052117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7d0004d90 msgr2=0x7fa7d0005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7dffff700 1 --2- 172.21.15.28:0/967052117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7d0004d90 0x7fa7d0005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7dffff700 1 -- 172.21.15.28:0/967052117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d00035b0 con 0x7fa7d0003fb0 2023-11-28T11:09:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7dffff700 1 --2- 172.21.15.28:0/967052117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d0003fb0 0x7fa7d0004850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009b30 tx=0x7fa7d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11ae64fd5a3f3ec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7df7fe700 1 -- 172.21.15.28:0/967052117 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc00e6c0 con 0x7fa7d0003fb0 2023-11-28T11:09:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7df7fe700 1 -- 172.21.15.28:0/967052117 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7cc00e820 con 0x7fa7d0003fb0 2023-11-28T11:09:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.875+0000 7fa7df7fe700 1 -- 172.21.15.28:0/967052117 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7cc00eb90 con 0x7fa7d0003fb0 2023-11-28T11:09:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.876+0000 7fa7e6e19700 1 -- 172.21.15.28:0/967052117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d0003fb0 msgr2=0x7fa7d0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.876+0000 7fa7e6e19700 1 --2- 172.21.15.28:0/967052117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7d0003fb0 0x7fa7d0004850 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fa7cc009b30 tx=0x7fa7d0005880 comp rx=0 tx=0).stop 2023-11-28T11:09:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.876+0000 7fa7e53b6700 1 -- 172.21.15.28:0/967052117 reap_dead start 2023-11-28T11:09:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.876+0000 7fa7e6e19700 1 -- 172.21.15.28:0/967052117 shutdown_connections 2023-11-28T11:09:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.876+0000 7fa7e6e19700 1 -- 172.21.15.28:0/967052117 >> 172.21.15.28:0/967052117 conn(0x7fa7e00cd680 msgr2=0x7fa7e00cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.876+0000 7fa7e6e19700 1 -- 172.21.15.28:0/967052117 shutdown_connections 2023-11-28T11:09:24.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.876+0000 7fa7e6e19700 1 -- 172.21.15.28:0/967052117 wait complete. 2023-11-28T11:09:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.877+0000 7fa7e6e19700 1 Processor -- start 2023-11-28T11:09:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.877+0000 7fa7e6e19700 1 -- start start 2023-11-28T11:09:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.878+0000 7fa7e6e19700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7d0003fb0 0x7fa7e01585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.878+0000 7fa7e6e19700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7e006d400 0x7fa7e015d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.878+0000 7fa7e6e19700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7e006d950 0x7fa7e015d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.878+0000 7fa7e6e19700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e00c4730 con 0x7fa7e006d950 2023-11-28T11:09:24.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.878+0000 7fa7e6e19700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e006d1e0 con 0x7fa7d0003fb0 2023-11-28T11:09:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.878+0000 7fa7e6e19700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7e00c7990 con 0x7fa7e006d400 2023-11-28T11:09:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.878+0000 7fa7e53b6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7e006d950 0x7fa7e015d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.878+0000 7fa7e53b6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7e006d950 0x7fa7e015d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47694/0 (socket says 172.21.15.28:47694) 2023-11-28T11:09:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.878+0000 7fa7e53b6700 1 -- 172.21.15.28:0/1861222924 learned_addr learned my addr 172.21.15.28:0/1861222924 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.879+0000 7fa7dffff700 1 --2- 172.21.15.28:0/1861222924 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7e006d400 0x7fa7e015d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.879+0000 7fa7e53b6700 1 -- 172.21.15.28:0/1861222924 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7e006d400 msgr2=0x7fa7e015d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.879+0000 7fa7e4bb5700 1 --2- 172.21.15.28:0/1861222924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7d0003fb0 0x7fa7e01585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.879+0000 7fa7e53b6700 1 --2- 172.21.15.28:0/1861222924 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7e006d400 0x7fa7e015d060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.879+0000 7fa7e53b6700 1 -- 172.21.15.28:0/1861222924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7d0003fb0 msgr2=0x7fa7e01585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.879+0000 7fa7e53b6700 1 --2- 172.21.15.28:0/1861222924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7d0003fb0 0x7fa7e01585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.879+0000 7fa7e53b6700 1 -- 172.21.15.28:0/1861222924 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7e00b66a0 con 0x7fa7e006d950 2023-11-28T11:09:24.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.879+0000 7fa7e53b6700 1 --2- 172.21.15.28:0/1861222924 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7e006d950 0x7fa7e015d5a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0066a20 tx=0x7fa7d800d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.879+0000 7fa7dd7fa700 1 -- 172.21.15.28:0/1861222924 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d8003c10 con 0x7fa7e006d950 2023-11-28T11:09:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.879+0000 7fa7dd7fa700 1 -- 172.21.15.28:0/1861222924 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d8003d90 con 0x7fa7e006d950 2023-11-28T11:09:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.879+0000 7fa7e6e19700 1 -- 172.21.15.28:0/1861222924 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7e00cffb0 con 0x7fa7e006d950 2023-11-28T11:09:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.879+0000 7fa7dd7fa700 1 -- 172.21.15.28:0/1861222924 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d800dbd0 con 0x7fa7e006d950 2023-11-28T11:09:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.879+0000 7fa7e6e19700 1 -- 172.21.15.28:0/1861222924 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7e00cce50 con 0x7fa7e006d950 2023-11-28T11:09:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.880+0000 7fa7e6e19700 1 -- 172.21.15.28:0/1861222924 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7e015dae0 con 0x7fa7e006d950 2023-11-28T11:09:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.881+0000 7fa7dd7fa700 1 -- 172.21.15.28:0/1861222924 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa7d800dd30 con 0x7fa7e006d950 2023-11-28T11:09:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.881+0000 7fa7dd7fa700 1 --2- 172.21.15.28:0/1861222924 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa7c40682f0 0x7fa7c406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.881+0000 7fa7dd7fa700 1 -- 172.21.15.28:0/1861222924 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa7d80863b0 con 0x7fa7e006d950 2023-11-28T11:09:24.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.881+0000 7fa7e4bb5700 1 --2- 172.21.15.28:0/1861222924 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa7c40682f0 0x7fa7c406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:24.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.882+0000 7fa7e4bb5700 1 --2- 172.21.15.28:0/1861222924 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa7c40682f0 0x7fa7c406a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4007cf0 tx=0x7fa7d4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:24.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:24.885+0000 7fa7dd7fa700 1 -- 172.21.15.28:0/1861222924 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa7d8052920 con 0x7fa7e006d950 2023-11-28T11:09:25.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.076+0000 7fa7e6e19700 1 -- 172.21.15.28:0/1861222924 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa7e00739f0 con 0x7fa7e006d950 2023-11-28T11:09:25.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.076+0000 7fa7dd7fa700 1 -- 172.21.15.28:0/1861222924 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa7d8055f40 con 0x7fa7e006d950 2023-11-28T11:09:25.078 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:25.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.078+0000 7fa7e6e19700 1 -- 172.21.15.28:0/1861222924 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa7c40682f0 msgr2=0x7fa7c406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.078+0000 7fa7e6e19700 1 --2- 172.21.15.28:0/1861222924 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa7c40682f0 0x7fa7c406a7a0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa7d4007cf0 tx=0x7fa7d4009040 comp rx=0 tx=0).stop 2023-11-28T11:09:25.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.078+0000 7fa7e6e19700 1 -- 172.21.15.28:0/1861222924 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7e006d950 msgr2=0x7fa7e015d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.078+0000 7fa7e6e19700 1 --2- 172.21.15.28:0/1861222924 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7e006d950 0x7fa7e015d5a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa7e0066a20 tx=0x7fa7d800d430 comp rx=0 tx=0).stop 2023-11-28T11:09:25.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.078+0000 7fa7e6e19700 1 -- 172.21.15.28:0/1861222924 shutdown_connections 2023-11-28T11:09:25.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.078+0000 7fa7e6e19700 1 --2- 172.21.15.28:0/1861222924 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa7c40682f0 0x7fa7c406a7a0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.078+0000 7fa7e6e19700 1 --2- 172.21.15.28:0/1861222924 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa7e006d950 0x7fa7e015d5a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.078+0000 7fa7e6e19700 1 --2- 172.21.15.28:0/1861222924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa7d0003fb0 0x7fa7e01585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.078+0000 7fa7e6e19700 1 --2- 172.21.15.28:0/1861222924 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa7e006d400 0x7fa7e015d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.078+0000 7fa7e6e19700 1 -- 172.21.15.28:0/1861222924 >> 172.21.15.28:0/1861222924 conn(0x7fa7e00cd680 msgr2=0x7fa7e00b03f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.078+0000 7fa7e6e19700 1 -- 172.21.15.28:0/1861222924 shutdown_connections 2023-11-28T11:09:25.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.078+0000 7fa7e6e19700 1 -- 172.21.15.28:0/1861222924 wait complete. 2023-11-28T11:09:25.092 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:25.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.215+0000 7fb266338700 1 Processor -- start 2023-11-28T11:09:25.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.215+0000 7fb266338700 1 -- start start 2023-11-28T11:09:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb266338700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb260161090 0x7fb2600ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb266338700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb2600caeb0 0x7fb2600cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb266338700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2600ce340 con 0x7fb26015c990 2023-11-28T11:09:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb266338700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2600cebe0 con 0x7fb2600caeb0 2023-11-28T11:09:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb266338700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2600cf780 con 0x7fb260161090 2023-11-28T11:09:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb2648d5700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb2600caeb0 0x7fb2600cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb2648d5700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb2600caeb0 0x7fb2600cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47702/0 (socket says 172.21.15.28:47702) 2023-11-28T11:09:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb2648d5700 1 -- 172.21.15.28:0/1169315078 learned_addr learned my addr 172.21.15.28:0/1169315078 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb25ffff700 1 --1- 172.21.15.28:0/1169315078 >> v1:172.21.15.28:6789/0 conn(0x7fb26015c990 0x7fb2600cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35756/0 (socket says 172.21.15.28:35756) 2023-11-28T11:09:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb25f7fe700 1 --2- 172.21.15.28:0/1169315078 >> v2:172.21.15.59:3300/0 conn(0x7fb260161090 0x7fb2600ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb2648d5700 1 -- 172.21.15.28:0/1169315078 >> v2:172.21.15.59:3300/0 conn(0x7fb260161090 msgr2=0x7fb2600ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:25.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb2648d5700 1 --2- 172.21.15.28:0/1169315078 >> v2:172.21.15.59:3300/0 conn(0x7fb260161090 0x7fb2600ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb2648d5700 1 -- 172.21.15.28:0/1169315078 >> v1:172.21.15.28:6789/0 conn(0x7fb26015c990 legacy=0x7fb2600cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb2648d5700 1 -- 172.21.15.28:0/1169315078 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2600cb3f0 con 0x7fb2600caeb0 2023-11-28T11:09:25.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb2648d5700 1 --2- 172.21.15.28:0/1169315078 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2600caeb0 0x7fb2600cc310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb26006ab20 tx=0x7fb25800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d913470b1cf0dafb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb25effd700 1 -- 172.21.15.28:0/1169315078 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25800e070 con 0x7fb2600caeb0 2023-11-28T11:09:25.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb25effd700 1 -- 172.21.15.28:0/1169315078 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb25800b9c0 con 0x7fb2600caeb0 2023-11-28T11:09:25.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.216+0000 7fb25effd700 1 -- 172.21.15.28:0/1169315078 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25800bd70 con 0x7fb2600caeb0 2023-11-28T11:09:25.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.217+0000 7fb266338700 1 -- 172.21.15.28:0/1169315078 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2600caeb0 msgr2=0x7fb2600cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.217+0000 7fb266338700 1 --2- 172.21.15.28:0/1169315078 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2600caeb0 0x7fb2600cc310 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fb26006ab20 tx=0x7fb25800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:25.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.217+0000 7fb266338700 1 -- 172.21.15.28:0/1169315078 shutdown_connections 2023-11-28T11:09:25.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.217+0000 7fb266338700 1 --2- 172.21.15.28:0/1169315078 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb2600caeb0 0x7fb2600cc310 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.217+0000 7fb266338700 1 --2- 172.21.15.28:0/1169315078 >> v2:172.21.15.59:3300/0 conn(0x7fb260161090 0x7fb2600ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.217+0000 7fb266338700 1 -- 172.21.15.28:0/1169315078 >> 172.21.15.28:0/1169315078 conn(0x7fb2600b1650 msgr2=0x7fb2600b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.217+0000 7fb266338700 1 -- 172.21.15.28:0/1169315078 shutdown_connections 2023-11-28T11:09:25.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.217+0000 7fb266338700 1 -- 172.21.15.28:0/1169315078 wait complete. 2023-11-28T11:09:25.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.218+0000 7fb266338700 1 Processor -- start 2023-11-28T11:09:25.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.218+0000 7fb266338700 1 -- start start 2023-11-28T11:09:25.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.219+0000 7fb266338700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2600caeb0 0x7fb260159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.219+0000 7fb266338700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb26015c990 0x7fb2600c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.219+0000 7fb266338700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb260161090 0x7fb2601f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.219+0000 7fb266338700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26015a220 con 0x7fb260161090 2023-11-28T11:09:25.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.219+0000 7fb266338700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2600c61c0 con 0x7fb26015c990 2023-11-28T11:09:25.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.219+0000 7fb266338700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2600c64a0 con 0x7fb2600caeb0 2023-11-28T11:09:25.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.219+0000 7fb2648d5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb260161090 0x7fb2601f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.219+0000 7fb2648d5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb260161090 0x7fb2601f6420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47708/0 (socket says 172.21.15.28:47708) 2023-11-28T11:09:25.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.219+0000 7fb2648d5700 1 -- 172.21.15.28:0/3181870262 learned_addr learned my addr 172.21.15.28:0/3181870262 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:25.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.219+0000 7fb25ffff700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2600caeb0 0x7fb260159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.219+0000 7fb25f7fe700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb26015c990 0x7fb2600c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.220+0000 7fb2648d5700 1 -- 172.21.15.28:0/3181870262 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2600caeb0 msgr2=0x7fb260159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.220+0000 7fb2648d5700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2600caeb0 0x7fb260159980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.220+0000 7fb2648d5700 1 -- 172.21.15.28:0/3181870262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb26015c990 msgr2=0x7fb2600c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.220+0000 7fb2648d5700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb26015c990 0x7fb2600c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.220+0000 7fb2648d5700 1 -- 172.21.15.28:0/3181870262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb258009b30 con 0x7fb260161090 2023-11-28T11:09:25.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.220+0000 7fb25ffff700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2600caeb0 0x7fb260159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:25.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.220+0000 7fb25f7fe700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb26015c990 0x7fb2600c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:25.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.220+0000 7fb2648d5700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb260161090 0x7fb2601f6420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb25800b370 tx=0x7fb258004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.220+0000 7fb25cff9700 1 -- 172.21.15.28:0/3181870262 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2580092e0 con 0x7fb260161090 2023-11-28T11:09:25.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.220+0000 7fb25cff9700 1 -- 172.21.15.28:0/3181870262 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2580099b0 con 0x7fb260161090 2023-11-28T11:09:25.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.220+0000 7fb266338700 1 -- 172.21.15.28:0/3181870262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2600ce870 con 0x7fb260161090 2023-11-28T11:09:25.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.220+0000 7fb25cff9700 1 -- 172.21.15.28:0/3181870262 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb25801c680 con 0x7fb260161090 2023-11-28T11:09:25.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.220+0000 7fb266338700 1 -- 172.21.15.28:0/3181870262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2600ce590 con 0x7fb260161090 2023-11-28T11:09:25.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.221+0000 7fb266338700 1 -- 172.21.15.28:0/3181870262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2600cc850 con 0x7fb260161090 2023-11-28T11:09:25.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.221+0000 7fb25cff9700 1 -- 172.21.15.28:0/3181870262 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb258022030 con 0x7fb260161090 2023-11-28T11:09:25.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.222+0000 7fb25cff9700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb24c0682f0 0x7fb24c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.222+0000 7fb25cff9700 1 -- 172.21.15.28:0/3181870262 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fb2580878f0 con 0x7fb260161090 2023-11-28T11:09:25.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.222+0000 7fb25ffff700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb24c0682f0 0x7fb24c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.222+0000 7fb25ffff700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb24c0682f0 0x7fb24c06a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb2600b1440 tx=0x7fb250009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.226+0000 7fb25cff9700 1 -- 172.21.15.28:0/3181870262 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb258053e60 con 0x7fb260161090 2023-11-28T11:09:25.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.422+0000 7fb266338700 1 -- 172.21.15.28:0/3181870262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2600675a0 con 0x7fb260161090 2023-11-28T11:09:25.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.423+0000 7fb25cff9700 1 -- 172.21.15.28:0/3181870262 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19642 (secure 0 0 0) 0x7fb258057480 con 0x7fb260161090 2023-11-28T11:09:25.425 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:25.425 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:50.531733+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.425+0000 7fb266338700 1 -- 172.21.15.28:0/3181870262 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb24c0682f0 msgr2=0x7fb24c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.425+0000 7fb266338700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb24c0682f0 0x7fb24c06a7a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb2600b1440 tx=0x7fb250009040 comp rx=0 tx=0).stop 2023-11-28T11:09:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.425+0000 7fb266338700 1 -- 172.21.15.28:0/3181870262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb260161090 msgr2=0x7fb2601f6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.425+0000 7fb266338700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb260161090 0x7fb2601f6420 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fb25800b370 tx=0x7fb258004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.425+0000 7fb266338700 1 -- 172.21.15.28:0/3181870262 shutdown_connections 2023-11-28T11:09:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.425+0000 7fb266338700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb24c0682f0 0x7fb24c06a7a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.425+0000 7fb266338700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb260161090 0x7fb2601f6420 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.425+0000 7fb266338700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb2600caeb0 0x7fb260159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.425+0000 7fb266338700 1 --2- 172.21.15.28:0/3181870262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb26015c990 0x7fb2600c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.425+0000 7fb266338700 1 -- 172.21.15.28:0/3181870262 >> 172.21.15.28:0/3181870262 conn(0x7fb2600b1650 msgr2=0x7fb2600c6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.425+0000 7fb266338700 1 -- 172.21.15.28:0/3181870262 shutdown_connections 2023-11-28T11:09:25.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.425+0000 7fb266338700 1 -- 172.21.15.28:0/3181870262 wait complete. 2023-11-28T11:09:25.440 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:25.441 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:25.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.568+0000 7fbd2f49b700 1 Processor -- start 2023-11-28T11:09:25.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.568+0000 7fbd2f49b700 1 -- start start 2023-11-28T11:09:25.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.569+0000 7fbd2f49b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbd280b62d0 0x7fbd280cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.569+0000 7fbd2f49b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd2806f8b0 0x7fbd280b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.569+0000 7fbd2f49b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd280b2bc0 0x7fbd280ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.569+0000 7fbd2f49b700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd280b3020 con 0x7fbd280b2bc0 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.569+0000 7fbd2f49b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd280cd2a0 con 0x7fbd280b62d0 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.569+0000 7fbd2f49b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd280d0400 con 0x7fbd2806f8b0 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.569+0000 7fbd2da38700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd280b2bc0 0x7fbd280ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.569+0000 7fbd2da38700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbd280b2bc0 0x7fbd280ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47716/0 (socket says 172.21.15.28:47716) 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.569+0000 7fbd2da38700 1 -- 172.21.15.28:0/1395696911 learned_addr learned my addr 172.21.15.28:0/1395696911 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.570+0000 7fbd2ca36700 1 --2- 172.21.15.28:0/1395696911 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd2806f8b0 0x7fbd280b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.570+0000 7fbd2d237700 1 --2- 172.21.15.28:0/1395696911 >> v2:172.21.15.59:3300/0 conn(0x7fbd280b62d0 0x7fbd280cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.570+0000 7fbd2da38700 1 -- 172.21.15.28:0/1395696911 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd2806f8b0 msgr2=0x7fbd280b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.570+0000 7fbd2da38700 1 --2- 172.21.15.28:0/1395696911 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd2806f8b0 0x7fbd280b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.570+0000 7fbd2da38700 1 -- 172.21.15.28:0/1395696911 >> v2:172.21.15.59:3300/0 conn(0x7fbd280b62d0 msgr2=0x7fbd280cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.570+0000 7fbd2da38700 1 --2- 172.21.15.28:0/1395696911 >> v2:172.21.15.59:3300/0 conn(0x7fbd280b62d0 0x7fbd280cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.570+0000 7fbd2da38700 1 -- 172.21.15.28:0/1395696911 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd280d06e0 con 0x7fbd280b2bc0 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.570+0000 7fbd2da38700 1 --2- 172.21.15.28:0/1395696911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd280b2bc0 0x7fbd280ce7d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbd280720d0 tx=0x7fbd2000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1b50ebfa4e764a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.570+0000 7fbd1ffff700 1 -- 172.21.15.28:0/1395696911 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2000e070 con 0x7fbd280b2bc0 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.570+0000 7fbd1ffff700 1 -- 172.21.15.28:0/1395696911 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd2000b9c0 con 0x7fbd280b2bc0 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.570+0000 7fbd1ffff700 1 -- 172.21.15.28:0/1395696911 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd2000bd70 con 0x7fbd280b2bc0 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.571+0000 7fbd2f49b700 1 -- 172.21.15.28:0/1395696911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd280b2bc0 msgr2=0x7fbd280ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.571+0000 7fbd2f49b700 1 --2- 172.21.15.28:0/1395696911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd280b2bc0 0x7fbd280ce7d0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fbd280720d0 tx=0x7fbd2000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.571+0000 7fbd2f49b700 1 -- 172.21.15.28:0/1395696911 shutdown_connections 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.571+0000 7fbd2f49b700 1 --2- 172.21.15.28:0/1395696911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd280b2bc0 0x7fbd280ce7d0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.571+0000 7fbd2f49b700 1 --2- 172.21.15.28:0/1395696911 >> v2:172.21.15.59:3300/0 conn(0x7fbd280b62d0 0x7fbd280cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.571+0000 7fbd2f49b700 1 --2- 172.21.15.28:0/1395696911 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd2806f8b0 0x7fbd280b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.571+0000 7fbd2f49b700 1 -- 172.21.15.28:0/1395696911 >> 172.21.15.28:0/1395696911 conn(0x7fbd280cd680 msgr2=0x7fbd280cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.571+0000 7fbd2f49b700 1 -- 172.21.15.28:0/1395696911 shutdown_connections 2023-11-28T11:09:25.575 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.571+0000 7fbd2f49b700 1 -- 172.21.15.28:0/1395696911 wait complete. 2023-11-28T11:09:25.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.572+0000 7fbd2f49b700 1 Processor -- start 2023-11-28T11:09:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.572+0000 7fbd2f49b700 1 -- start start 2023-11-28T11:09:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2f49b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd2806f8b0 0x7fbd281f2c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2f49b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd280b2bc0 0x7fbd280ca9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2f49b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd280b62d0 0x7fbd280c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2f49b700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd281f1b90 con 0x7fbd280b62d0 2023-11-28T11:09:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2f49b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd281f1e70 con 0x7fbd280b2bc0 2023-11-28T11:09:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2f49b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd281f2150 con 0x7fbd2806f8b0 2023-11-28T11:09:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2da38700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd280b62d0 0x7fbd280c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2da38700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd280b62d0 0x7fbd280c89a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47722/0 (socket says 172.21.15.28:47722) 2023-11-28T11:09:25.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2da38700 1 -- 172.21.15.28:0/2791510627 learned_addr learned my addr 172.21.15.28:0/2791510627 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2d237700 1 --2- 172.21.15.28:0/2791510627 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd2806f8b0 0x7fbd281f2c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2ca36700 1 --2- 172.21.15.28:0/2791510627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd280b2bc0 0x7fbd280ca9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2da38700 1 -- 172.21.15.28:0/2791510627 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd2806f8b0 msgr2=0x7fbd281f2c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2da38700 1 --2- 172.21.15.28:0/2791510627 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd2806f8b0 0x7fbd281f2c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2da38700 1 -- 172.21.15.28:0/2791510627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd280b2bc0 msgr2=0x7fbd280ca9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2da38700 1 --2- 172.21.15.28:0/2791510627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd280b2bc0 0x7fbd280ca9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2da38700 1 -- 172.21.15.28:0/2791510627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd20009b30 con 0x7fbd280b62d0 2023-11-28T11:09:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2d237700 1 --2- 172.21.15.28:0/2791510627 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd2806f8b0 0x7fbd281f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd2da38700 1 --2- 172.21.15.28:0/2791510627 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd280b62d0 0x7fbd280c89a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbd28066a20 tx=0x7fbd20004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd1dffb700 1 -- 172.21.15.28:0/2791510627 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20004250 con 0x7fbd280b62d0 2023-11-28T11:09:25.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.573+0000 7fbd1dffb700 1 -- 172.21.15.28:0/2791510627 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd200043d0 con 0x7fbd280b62d0 2023-11-28T11:09:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.574+0000 7fbd2f49b700 1 -- 172.21.15.28:0/2791510627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd280b66a0 con 0x7fbd280b62d0 2023-11-28T11:09:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.574+0000 7fbd1dffb700 1 -- 172.21.15.28:0/2791510627 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd20004780 con 0x7fbd280b62d0 2023-11-28T11:09:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.574+0000 7fbd2f49b700 1 -- 172.21.15.28:0/2791510627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd280cffb0 con 0x7fbd280b62d0 2023-11-28T11:09:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.575+0000 7fbd2f49b700 1 -- 172.21.15.28:0/2791510627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd2806fd00 con 0x7fbd280b62d0 2023-11-28T11:09:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.575+0000 7fbd1dffb700 1 -- 172.21.15.28:0/2791510627 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbd2001c440 con 0x7fbd280b62d0 2023-11-28T11:09:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.575+0000 7fbd1dffb700 1 --2- 172.21.15.28:0/2791510627 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbd0c0682f0 0x7fbd0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.576+0000 7fbd1dffb700 1 -- 172.21.15.28:0/2791510627 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fbd20087630 con 0x7fbd280b62d0 2023-11-28T11:09:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.576+0000 7fbd2d237700 1 --2- 172.21.15.28:0/2791510627 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbd0c0682f0 0x7fbd0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.576+0000 7fbd2d237700 1 --2- 172.21.15.28:0/2791510627 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbd0c0682f0 0x7fbd0c06a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009a00 tx=0x7fbd18008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.580+0000 7fbd1dffb700 1 -- 172.21.15.28:0/2791510627 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbd20053b40 con 0x7fbd280b62d0 2023-11-28T11:09:25.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.772+0000 7fbd2f49b700 1 -- 172.21.15.28:0/2791510627 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fbd280739f0 con 0x7fbd280b62d0 2023-11-28T11:09:25.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.773+0000 7fbd1dffb700 1 -- 172.21.15.28:0/2791510627 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7fbd20057160 con 0x7fbd280b62d0 2023-11-28T11:09:25.774 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:09:25.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.775+0000 7fbd2f49b700 1 -- 172.21.15.28:0/2791510627 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbd0c0682f0 msgr2=0x7fbd0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.775+0000 7fbd2f49b700 1 --2- 172.21.15.28:0/2791510627 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbd0c0682f0 0x7fbd0c06a7a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fbd18009a00 tx=0x7fbd18008040 comp rx=0 tx=0).stop 2023-11-28T11:09:25.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.775+0000 7fbd2f49b700 1 -- 172.21.15.28:0/2791510627 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd280b62d0 msgr2=0x7fbd280c89a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.775+0000 7fbd2f49b700 1 --2- 172.21.15.28:0/2791510627 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd280b62d0 0x7fbd280c89a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fbd28066a20 tx=0x7fbd20004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.775+0000 7fbd2f49b700 1 -- 172.21.15.28:0/2791510627 shutdown_connections 2023-11-28T11:09:25.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.775+0000 7fbd2f49b700 1 --2- 172.21.15.28:0/2791510627 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbd0c0682f0 0x7fbd0c06a7a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.775+0000 7fbd2f49b700 1 --2- 172.21.15.28:0/2791510627 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbd280b62d0 0x7fbd280c89a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.775+0000 7fbd2f49b700 1 --2- 172.21.15.28:0/2791510627 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbd2806f8b0 0x7fbd281f2c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.775+0000 7fbd2f49b700 1 --2- 172.21.15.28:0/2791510627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbd280b2bc0 0x7fbd280ca9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.775+0000 7fbd2f49b700 1 -- 172.21.15.28:0/2791510627 >> 172.21.15.28:0/2791510627 conn(0x7fbd280cd680 msgr2=0x7fbd280ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.775+0000 7fbd2f49b700 1 -- 172.21.15.28:0/2791510627 shutdown_connections 2023-11-28T11:09:25.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.775+0000 7fbd2f49b700 1 -- 172.21.15.28:0/2791510627 wait complete. 2023-11-28T11:09:25.789 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:25.916 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.915+0000 7fe05165a700 1 Processor -- start 2023-11-28T11:09:25.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.915+0000 7fe05165a700 1 -- start start 2023-11-28T11:09:25.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe05165a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe04c15c640 0x7fe04c0cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe05165a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe04c0d2820 0x7fe04c0b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe05165a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe04c06bf30 0x7fe04c06c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe05165a700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04c15ca10 con 0x7fe04c06bf30 2023-11-28T11:09:25.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe05165a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe04c06c890 con 0x7fe04c0d2820 2023-11-28T11:09:25.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe05165a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c0d1470 con 0x7fe04c15c640 2023-11-28T11:09:25.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe04b7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe04c06bf30 0x7fe04c06c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe04b7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe04c06bf30 0x7fe04c06c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47728/0 (socket says 172.21.15.28:47728) 2023-11-28T11:09:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe04b7fe700 1 -- 172.21.15.28:0/2038633100 learned_addr learned my addr 172.21.15.28:0/2038633100 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe04affd700 1 --2- 172.21.15.28:0/2038633100 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe04c15c640 0x7fe04c0cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe04b7fe700 1 -- 172.21.15.28:0/2038633100 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe04c15c640 msgr2=0x7fe04c0cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe04a7fc700 1 --2- 172.21.15.28:0/2038633100 >> v2:172.21.15.59:3300/0 conn(0x7fe04c0d2820 0x7fe04c0b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe04b7fe700 1 --2- 172.21.15.28:0/2038633100 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe04c15c640 0x7fe04c0cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe04b7fe700 1 -- 172.21.15.28:0/2038633100 >> v2:172.21.15.59:3300/0 conn(0x7fe04c0d2820 msgr2=0x7fe04c0b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe04b7fe700 1 --2- 172.21.15.28:0/2038633100 >> v2:172.21.15.59:3300/0 conn(0x7fe04c0d2820 0x7fe04c0b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe04b7fe700 1 -- 172.21.15.28:0/2038633100 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe04c0cd4e0 con 0x7fe04c06bf30 2023-11-28T11:09:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe04affd700 1 --2- 172.21.15.28:0/2038633100 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe04c15c640 0x7fe04c0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe04a7fc700 1 --2- 172.21.15.28:0/2038633100 >> v2:172.21.15.59:3300/0 conn(0x7fe04c0d2820 0x7fe04c0b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.916+0000 7fe04b7fe700 1 --2- 172.21.15.28:0/2038633100 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe04c06bf30 0x7fe04c06c350 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe04c06d450 tx=0x7fe04400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f570c3dba6db622 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.917+0000 7fe049ffb700 1 -- 172.21.15.28:0/2038633100 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04400e070 con 0x7fe04c06bf30 2023-11-28T11:09:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.917+0000 7fe049ffb700 1 -- 172.21.15.28:0/2038633100 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe04400b9c0 con 0x7fe04c06bf30 2023-11-28T11:09:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.917+0000 7fe049ffb700 1 -- 172.21.15.28:0/2038633100 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe04400bd70 con 0x7fe04c06bf30 2023-11-28T11:09:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.917+0000 7fe05165a700 1 -- 172.21.15.28:0/2038633100 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe04c06bf30 msgr2=0x7fe04c06c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.917+0000 7fe05165a700 1 --2- 172.21.15.28:0/2038633100 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe04c06bf30 0x7fe04c06c350 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fe04c06d450 tx=0x7fe04400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.917+0000 7fe05165a700 1 -- 172.21.15.28:0/2038633100 shutdown_connections 2023-11-28T11:09:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.917+0000 7fe05165a700 1 --2- 172.21.15.28:0/2038633100 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe04c06bf30 0x7fe04c06c350 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.917+0000 7fe05165a700 1 --2- 172.21.15.28:0/2038633100 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe04c15c640 0x7fe04c0cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.917+0000 7fe05165a700 1 --2- 172.21.15.28:0/2038633100 >> v2:172.21.15.59:3300/0 conn(0x7fe04c0d2820 0x7fe04c0b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.917+0000 7fe05165a700 1 -- 172.21.15.28:0/2038633100 >> 172.21.15.28:0/2038633100 conn(0x7fe04c0d2df0 msgr2=0x7fe04c0d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.917+0000 7fe05165a700 1 -- 172.21.15.28:0/2038633100 shutdown_connections 2023-11-28T11:09:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.917+0000 7fe05165a700 1 -- 172.21.15.28:0/2038633100 wait complete. 2023-11-28T11:09:25.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.918+0000 7fe05165a700 1 Processor -- start 2023-11-28T11:09:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.919+0000 7fe05165a700 1 -- start start 2023-11-28T11:09:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.919+0000 7fe05165a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe04c06bf30 0x7fe04c0c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.919+0000 7fe05165a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe04c0d2820 0x7fe04c06a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.919+0000 7fe05165a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe04c15c640 0x7fe04c06ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.919+0000 7fe05165a700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c153b40 con 0x7fe04c15c640 2023-11-28T11:09:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.919+0000 7fe05165a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c0b5c20 con 0x7fe04c06bf30 2023-11-28T11:09:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.919+0000 7fe05165a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe04c0c6ff0 con 0x7fe04c0d2820 2023-11-28T11:09:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.919+0000 7fe04b7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe04c15c640 0x7fe04c06ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.919+0000 7fe04b7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe04c15c640 0x7fe04c06ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47734/0 (socket says 172.21.15.28:47734) 2023-11-28T11:09:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.919+0000 7fe04b7fe700 1 -- 172.21.15.28:0/4130828474 learned_addr learned my addr 172.21.15.28:0/4130828474 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.919+0000 7fe04a7fc700 1 --2- 172.21.15.28:0/4130828474 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe04c0d2820 0x7fe04c06a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.919+0000 7fe04affd700 1 --2- 172.21.15.28:0/4130828474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe04c06bf30 0x7fe04c0c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.920+0000 7fe04b7fe700 1 -- 172.21.15.28:0/4130828474 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe04c0d2820 msgr2=0x7fe04c06a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.920+0000 7fe04b7fe700 1 --2- 172.21.15.28:0/4130828474 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe04c0d2820 0x7fe04c06a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.920+0000 7fe04b7fe700 1 -- 172.21.15.28:0/4130828474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe04c06bf30 msgr2=0x7fe04c0c68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.920+0000 7fe04b7fe700 1 --2- 172.21.15.28:0/4130828474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe04c06bf30 0x7fe04c0c68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.920+0000 7fe04b7fe700 1 -- 172.21.15.28:0/4130828474 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe044009b30 con 0x7fe04c15c640 2023-11-28T11:09:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.920+0000 7fe04b7fe700 1 --2- 172.21.15.28:0/4130828474 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe04c15c640 0x7fe04c06ac30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe04400b370 tx=0x7fe044004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.920+0000 7fe0337fe700 1 -- 172.21.15.28:0/4130828474 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe044004250 con 0x7fe04c15c640 2023-11-28T11:09:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.920+0000 7fe0337fe700 1 -- 172.21.15.28:0/4130828474 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0440043d0 con 0x7fe04c15c640 2023-11-28T11:09:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.920+0000 7fe0337fe700 1 -- 172.21.15.28:0/4130828474 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe044004780 con 0x7fe04c15c640 2023-11-28T11:09:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.920+0000 7fe05165a700 1 -- 172.21.15.28:0/4130828474 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe04c0d1710 con 0x7fe04c15c640 2023-11-28T11:09:25.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.920+0000 7fe05165a700 1 -- 172.21.15.28:0/4130828474 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe04c0cd7c0 con 0x7fe04c15c640 2023-11-28T11:09:25.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.921+0000 7fe05165a700 1 -- 172.21.15.28:0/4130828474 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe04c0cf580 con 0x7fe04c15c640 2023-11-28T11:09:25.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.921+0000 7fe0337fe700 1 -- 172.21.15.28:0/4130828474 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe04402b070 con 0x7fe04c15c640 2023-11-28T11:09:25.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.922+0000 7fe0337fe700 1 --2- 172.21.15.28:0/4130828474 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0380682f0 0x7fe03806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:25.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.922+0000 7fe0337fe700 1 -- 172.21.15.28:0/4130828474 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fe044087340 con 0x7fe04c15c640 2023-11-28T11:09:25.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.922+0000 7fe04affd700 1 --2- 172.21.15.28:0/4130828474 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0380682f0 0x7fe03806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:25.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.922+0000 7fe04affd700 1 --2- 172.21.15.28:0/4130828474 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0380682f0 0x7fe03806a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe040006fd0 tx=0x7fe040008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:25.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:25.926+0000 7fe0337fe700 1 -- 172.21.15.28:0/4130828474 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe0440538b0 con 0x7fe04c15c640 2023-11-28T11:09:26.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.118+0000 7fe05165a700 1 -- 172.21.15.28:0/4130828474 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe04c0739f0 con 0x7fe04c15c640 2023-11-28T11:09:26.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.118+0000 7fe0337fe700 1 -- 172.21.15.28:0/4130828474 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe044056ed0 con 0x7fe04c15c640 2023-11-28T11:09:26.120 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:09:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.120+0000 7fe05165a700 1 -- 172.21.15.28:0/4130828474 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0380682f0 msgr2=0x7fe03806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.120+0000 7fe05165a700 1 --2- 172.21.15.28:0/4130828474 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0380682f0 0x7fe03806a7a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fe040006fd0 tx=0x7fe040008040 comp rx=0 tx=0).stop 2023-11-28T11:09:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.120+0000 7fe05165a700 1 -- 172.21.15.28:0/4130828474 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe04c15c640 msgr2=0x7fe04c06ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.120+0000 7fe05165a700 1 --2- 172.21.15.28:0/4130828474 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe04c15c640 0x7fe04c06ac30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe04400b370 tx=0x7fe044004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.120+0000 7fe05165a700 1 -- 172.21.15.28:0/4130828474 shutdown_connections 2023-11-28T11:09:26.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.120+0000 7fe05165a700 1 --2- 172.21.15.28:0/4130828474 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0380682f0 0x7fe03806a7a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.120+0000 7fe05165a700 1 --2- 172.21.15.28:0/4130828474 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe04c15c640 0x7fe04c06ac30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.120+0000 7fe05165a700 1 --2- 172.21.15.28:0/4130828474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe04c06bf30 0x7fe04c0c68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.120+0000 7fe05165a700 1 --2- 172.21.15.28:0/4130828474 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe04c0d2820 0x7fe04c06a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.120+0000 7fe05165a700 1 -- 172.21.15.28:0/4130828474 >> 172.21.15.28:0/4130828474 conn(0x7fe04c0d2df0 msgr2=0x7fe04c0d31c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.120+0000 7fe05165a700 1 -- 172.21.15.28:0/4130828474 shutdown_connections 2023-11-28T11:09:26.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.120+0000 7fe05165a700 1 -- 172.21.15.28:0/4130828474 wait complete. 2023-11-28T11:09:26.134 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:26.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.262+0000 7f1582a39700 1 Processor -- start 2023-11-28T11:09:26.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.262+0000 7f1582a39700 1 -- start start 2023-11-28T11:09:26.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.263+0000 7f1582a39700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f157c0b62d0 0x7f157c0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.263+0000 7f1582a39700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f157c06f8b0 0x7f157c0b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.263+0000 7f1582a39700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f157c0b2fe0 con 0x7f157c0b2bc0 2023-11-28T11:09:26.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.263+0000 7f1582a39700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f157c0cd2a0 con 0x7f157c0b62d0 2023-11-28T11:09:26.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.263+0000 7f1582a39700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f157c0d0400 con 0x7f157c06f8b0 2023-11-28T11:09:26.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.263+0000 7f157bfff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f157c0b62d0 0x7f157c0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.263+0000 7f157bfff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f157c0b62d0 0x7f157c0cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47738/0 (socket says 172.21.15.28:47738) 2023-11-28T11:09:26.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.263+0000 7f157bfff700 1 -- 172.21.15.28:0/3255920797 learned_addr learned my addr 172.21.15.28:0/3255920797 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:26.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.263+0000 7f1580fd6700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f157c0b2bc0 0x7f157c0ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35800/0 (socket says 172.21.15.28:35800) 2023-11-28T11:09:26.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.263+0000 7f157b7fe700 1 --2- 172.21.15.28:0/3255920797 >> v2:172.21.15.59:3300/0 conn(0x7f157c06f8b0 0x7f157c0b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.263+0000 7f157bfff700 1 -- 172.21.15.28:0/3255920797 >> v2:172.21.15.59:3300/0 conn(0x7f157c06f8b0 msgr2=0x7f157c0b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:26.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.263+0000 7f157bfff700 1 --2- 172.21.15.28:0/3255920797 >> v2:172.21.15.59:3300/0 conn(0x7f157c06f8b0 0x7f157c0b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.263+0000 7f157bfff700 1 -- 172.21.15.28:0/3255920797 >> v1:172.21.15.28:6789/0 conn(0x7f157c0b2bc0 legacy=0x7f157c0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.263+0000 7f157bfff700 1 -- 172.21.15.28:0/3255920797 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c0d06e0 con 0x7f157c0b62d0 2023-11-28T11:09:26.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.263+0000 7f157affd700 1 -- 172.21.15.28:0/3255920797 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 365296558 0 0) 0x7f157c0b2fe0 con 0x7f157c0b2bc0 2023-11-28T11:09:26.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.264+0000 7f157bfff700 1 --2- 172.21.15.28:0/3255920797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f157c0b62d0 0x7f157c0cb630 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f157c0cc8e0 tx=0x7f156c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5377baf26580ad7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.264+0000 7f157affd700 1 -- 172.21.15.28:0/3255920797 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00b870 con 0x7f157c0b62d0 2023-11-28T11:09:26.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.264+0000 7f157affd700 1 -- 172.21.15.28:0/3255920797 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156c00b9d0 con 0x7f157c0b62d0 2023-11-28T11:09:26.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.264+0000 7f157affd700 1 -- 172.21.15.28:0/3255920797 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156c00bd60 con 0x7f157c0b62d0 2023-11-28T11:09:26.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.264+0000 7f1582a39700 1 -- 172.21.15.28:0/3255920797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f157c0b62d0 msgr2=0x7f157c0cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.264+0000 7f1582a39700 1 --2- 172.21.15.28:0/3255920797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f157c0b62d0 0x7f157c0cb630 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f157c0cc8e0 tx=0x7f156c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.264+0000 7f1582a39700 1 -- 172.21.15.28:0/3255920797 shutdown_connections 2023-11-28T11:09:26.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.264+0000 7f1582a39700 1 --2- 172.21.15.28:0/3255920797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f157c0b62d0 0x7f157c0cb630 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.264+0000 7f1582a39700 1 --2- 172.21.15.28:0/3255920797 >> v2:172.21.15.59:3300/0 conn(0x7f157c06f8b0 0x7f157c0b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.264+0000 7f1582a39700 1 -- 172.21.15.28:0/3255920797 >> 172.21.15.28:0/3255920797 conn(0x7f157c0cd680 msgr2=0x7f157c0cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.264+0000 7f1582a39700 1 -- 172.21.15.28:0/3255920797 shutdown_connections 2023-11-28T11:09:26.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.264+0000 7f1582a39700 1 -- 172.21.15.28:0/3255920797 wait complete. 2023-11-28T11:09:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.265+0000 7f1582a39700 1 Processor -- start 2023-11-28T11:09:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.266+0000 7f1582a39700 1 -- start start 2023-11-28T11:09:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.266+0000 7f1582a39700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f157c06f8b0 0x7f157c13dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.266+0000 7f1582a39700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f157c0b2bc0 0x7f157c0c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.266+0000 7f1582a39700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f157c0b62d0 0x7f157c1f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.266+0000 7f1582a39700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c157230 con 0x7f157c0b62d0 2023-11-28T11:09:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.266+0000 7f1582a39700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f157c1565f0 con 0x7f157c0b2bc0 2023-11-28T11:09:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.266+0000 7f1582a39700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f157c156890 con 0x7f157c06f8b0 2023-11-28T11:09:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f1580fd6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f157c0b62d0 0x7f157c1f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f1580fd6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f157c0b62d0 0x7f157c1f34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47748/0 (socket says 172.21.15.28:47748) 2023-11-28T11:09:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f1580fd6700 1 -- 172.21.15.28:0/3274084811 learned_addr learned my addr 172.21.15.28:0/3274084811 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f157bfff700 1 --2- 172.21.15.28:0/3274084811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f157c06f8b0 0x7f157c13dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f157b7fe700 1 --2- 172.21.15.28:0/3274084811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f157c0b2bc0 0x7f157c0c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f1580fd6700 1 -- 172.21.15.28:0/3274084811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f157c06f8b0 msgr2=0x7f157c13dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f1580fd6700 1 --2- 172.21.15.28:0/3274084811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f157c06f8b0 0x7f157c13dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f1580fd6700 1 -- 172.21.15.28:0/3274084811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f157c0b2bc0 msgr2=0x7f157c0c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f1580fd6700 1 --2- 172.21.15.28:0/3274084811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f157c0b2bc0 0x7f157c0c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f1580fd6700 1 -- 172.21.15.28:0/3274084811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f156c009550 con 0x7f157c0b62d0 2023-11-28T11:09:26.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f157b7fe700 1 --2- 172.21.15.28:0/3274084811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f157c0b2bc0 0x7f157c0c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f1580fd6700 1 --2- 172.21.15.28:0/3274084811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f157c0b62d0 0x7f157c1f34b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1574002230 tx=0x7f157400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f1578ff9700 1 -- 172.21.15.28:0/3274084811 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f157400ab70 con 0x7f157c0b62d0 2023-11-28T11:09:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f1578ff9700 1 -- 172.21.15.28:0/3274084811 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f157400acf0 con 0x7f157c0b62d0 2023-11-28T11:09:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f1582a39700 1 -- 172.21.15.28:0/3274084811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f157c06fd30 con 0x7f157c0b62d0 2023-11-28T11:09:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f1578ff9700 1 -- 172.21.15.28:0/3274084811 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1574007310 con 0x7f157c0b62d0 2023-11-28T11:09:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.267+0000 7f1582a39700 1 -- 172.21.15.28:0/3274084811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f157c0cffb0 con 0x7f157c0b62d0 2023-11-28T11:09:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.268+0000 7f1582a39700 1 -- 172.21.15.28:0/3274084811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f157c0b66a0 con 0x7f157c0b62d0 2023-11-28T11:09:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.269+0000 7f1578ff9700 1 -- 172.21.15.28:0/3274084811 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f157401a030 con 0x7f157c0b62d0 2023-11-28T11:09:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.269+0000 7f1578ff9700 1 --2- 172.21.15.28:0/3274084811 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15680682f0 0x7f156806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.269+0000 7f1578ff9700 1 -- 172.21.15.28:0/3274084811 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1574085c80 con 0x7f157c0b62d0 2023-11-28T11:09:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.269+0000 7f157bfff700 1 --2- 172.21.15.28:0/3274084811 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15680682f0 0x7f156806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.270+0000 7f157bfff700 1 --2- 172.21.15.28:0/3274084811 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15680682f0 0x7f156806a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f156c014040 tx=0x7f156c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.273+0000 7f1578ff9700 1 -- 172.21.15.28:0/3274084811 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f15740521f0 con 0x7f157c0b62d0 2023-11-28T11:09:26.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.466+0000 7f1582a39700 1 -- 172.21.15.28:0/3274084811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f157c157370 con 0x7f157c0b62d0 2023-11-28T11:09:26.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.466+0000 7f1578ff9700 1 -- 172.21.15.28:0/3274084811 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1574018020 con 0x7f157c0b62d0 2023-11-28T11:09:26.468 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:26.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.468+0000 7f1582a39700 1 -- 172.21.15.28:0/3274084811 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15680682f0 msgr2=0x7f156806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.468+0000 7f1582a39700 1 --2- 172.21.15.28:0/3274084811 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15680682f0 0x7f156806a7a0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f156c014040 tx=0x7f156c008030 comp rx=0 tx=0).stop 2023-11-28T11:09:26.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.468+0000 7f1582a39700 1 -- 172.21.15.28:0/3274084811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f157c0b62d0 msgr2=0x7f157c1f34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.468+0000 7f1582a39700 1 --2- 172.21.15.28:0/3274084811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f157c0b62d0 0x7f157c1f34b0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1574002230 tx=0x7f157400cf50 comp rx=0 tx=0).stop 2023-11-28T11:09:26.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.468+0000 7f1582a39700 1 -- 172.21.15.28:0/3274084811 shutdown_connections 2023-11-28T11:09:26.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.468+0000 7f1582a39700 1 --2- 172.21.15.28:0/3274084811 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15680682f0 0x7f156806a7a0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.468+0000 7f1582a39700 1 --2- 172.21.15.28:0/3274084811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f157c0b62d0 0x7f157c1f34b0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.468+0000 7f1582a39700 1 --2- 172.21.15.28:0/3274084811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f157c06f8b0 0x7f157c13dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.468+0000 7f1582a39700 1 --2- 172.21.15.28:0/3274084811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f157c0b2bc0 0x7f157c0c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.468+0000 7f1582a39700 1 -- 172.21.15.28:0/3274084811 >> 172.21.15.28:0/3274084811 conn(0x7f157c0cd680 msgr2=0x7f157c15a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.469+0000 7f1582a39700 1 -- 172.21.15.28:0/3274084811 shutdown_connections 2023-11-28T11:09:26.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.469+0000 7f1582a39700 1 -- 172.21.15.28:0/3274084811 wait complete. 2023-11-28T11:09:26.482 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:26.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.608+0000 7f101ae10700 1 Processor -- start 2023-11-28T11:09:26.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.608+0000 7f101ae10700 1 -- start start 2023-11-28T11:09:26.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.609+0000 7f101ae10700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f101415c890 0x7f10140b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.609+0000 7f101ae10700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1014160d30 0x7f101415d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.609+0000 7f101ae10700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f10140ce7b0 0x7f10140cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.609+0000 7f101ae10700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10140b2ba0 con 0x7f10140ce7b0 2023-11-28T11:09:26.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.609+0000 7f101ae10700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10140cf170 con 0x7f101415c890 2023-11-28T11:09:26.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.609+0000 7f101ae10700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101415d2d0 con 0x7f1014160d30 2023-11-28T11:09:26.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.609+0000 7f10193ad700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f10140ce7b0 0x7f10140cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.609+0000 7f10193ad700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f10140ce7b0 0x7f10140cec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47754/0 (socket says 172.21.15.28:47754) 2023-11-28T11:09:26.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.609+0000 7f10193ad700 1 -- 172.21.15.28:0/189133307 learned_addr learned my addr 172.21.15.28:0/189133307 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:26.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f1018bac700 1 --2- 172.21.15.28:0/189133307 >> v2:172.21.15.59:3300/0 conn(0x7f101415c890 0x7f10140b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f1013fff700 1 --2- 172.21.15.28:0/189133307 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1014160d30 0x7f101415d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.614 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f10193ad700 1 -- 172.21.15.28:0/189133307 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1014160d30 msgr2=0x7f101415d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:26.614 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f10193ad700 1 --2- 172.21.15.28:0/189133307 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1014160d30 0x7f101415d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.614 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f10193ad700 1 -- 172.21.15.28:0/189133307 >> v2:172.21.15.59:3300/0 conn(0x7f101415c890 msgr2=0x7f10140b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:26.614 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f10193ad700 1 --2- 172.21.15.28:0/189133307 >> v2:172.21.15.59:3300/0 conn(0x7f101415c890 0x7f10140b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.614 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f10193ad700 1 -- 172.21.15.28:0/189133307 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f101415dca0 con 0x7f10140ce7b0 2023-11-28T11:09:26.614 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f1013fff700 1 --2- 172.21.15.28:0/189133307 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1014160d30 0x7f101415d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:26.614 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f10193ad700 1 --2- 172.21.15.28:0/189133307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10140ce7b0 0x7f10140cec30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f10140b37d0 tx=0x7f100c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bac3936210fcb4b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.614 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f10137fe700 1 -- 172.21.15.28:0/189133307 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c00e070 con 0x7f10140ce7b0 2023-11-28T11:09:26.614 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f10137fe700 1 -- 172.21.15.28:0/189133307 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100c00b9c0 con 0x7f10140ce7b0 2023-11-28T11:09:26.614 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f10137fe700 1 -- 172.21.15.28:0/189133307 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100c00bd70 con 0x7f10140ce7b0 2023-11-28T11:09:26.614 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f101ae10700 1 -- 172.21.15.28:0/189133307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10140ce7b0 msgr2=0x7f10140cec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f101ae10700 1 --2- 172.21.15.28:0/189133307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10140ce7b0 0x7f10140cec30 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f10140b37d0 tx=0x7f100c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:26.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f101ae10700 1 -- 172.21.15.28:0/189133307 shutdown_connections 2023-11-28T11:09:26.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f101ae10700 1 --2- 172.21.15.28:0/189133307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f10140ce7b0 0x7f10140cec30 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f101ae10700 1 --2- 172.21.15.28:0/189133307 >> v2:172.21.15.59:3300/0 conn(0x7f101415c890 0x7f10140b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.610+0000 7f101ae10700 1 --2- 172.21.15.28:0/189133307 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1014160d30 0x7f101415d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.611+0000 7f101ae10700 1 -- 172.21.15.28:0/189133307 >> 172.21.15.28:0/189133307 conn(0x7f101406e9b0 msgr2=0x7f10140cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.611+0000 7f101ae10700 1 -- 172.21.15.28:0/189133307 shutdown_connections 2023-11-28T11:09:26.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.611+0000 7f101ae10700 1 -- 172.21.15.28:0/189133307 wait complete. 2023-11-28T11:09:26.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.612+0000 7f101ae10700 1 Processor -- start 2023-11-28T11:09:26.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.612+0000 7f101ae10700 1 -- start start 2023-11-28T11:09:26.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f101ae10700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10140ce7b0 0x7f101415f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f101ae10700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101415c890 0x7f101415ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f101ae10700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014160d30 0x7f10141f1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f101ae10700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1014158850 con 0x7f101415c890 2023-11-28T11:09:26.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f101ae10700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10140b3a10 con 0x7f1014160d30 2023-11-28T11:09:26.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f101ae10700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101415f470 con 0x7f10140ce7b0 2023-11-28T11:09:26.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f1013fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101415c890 0x7f101415ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f1013fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101415c890 0x7f101415ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47758/0 (socket says 172.21.15.28:47758) 2023-11-28T11:09:26.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f1013fff700 1 -- 172.21.15.28:0/1230721235 learned_addr learned my addr 172.21.15.28:0/1230721235 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:26.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f1018bac700 1 --2- 172.21.15.28:0/1230721235 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10140ce7b0 0x7f101415f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f10193ad700 1 --2- 172.21.15.28:0/1230721235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014160d30 0x7f10141f1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f1013fff700 1 -- 172.21.15.28:0/1230721235 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10140ce7b0 msgr2=0x7f101415f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f1013fff700 1 --2- 172.21.15.28:0/1230721235 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10140ce7b0 0x7f101415f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f1013fff700 1 -- 172.21.15.28:0/1230721235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014160d30 msgr2=0x7f10141f1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f1013fff700 1 --2- 172.21.15.28:0/1230721235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014160d30 0x7f10141f1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f1013fff700 1 -- 172.21.15.28:0/1230721235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c009b30 con 0x7f101415c890 2023-11-28T11:09:26.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f1013fff700 1 --2- 172.21.15.28:0/1230721235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101415c890 0x7f101415ef30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f100000c7f0 tx=0x7f100000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f10117fa700 1 -- 172.21.15.28:0/1230721235 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000011800 con 0x7f101415c890 2023-11-28T11:09:26.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f10117fa700 1 -- 172.21.15.28:0/1230721235 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1000011960 con 0x7f101415c890 2023-11-28T11:09:26.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f10117fa700 1 -- 172.21.15.28:0/1230721235 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1000011cf0 con 0x7f101415c890 2023-11-28T11:09:26.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f101ae10700 1 -- 172.21.15.28:0/1230721235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10140b2d90 con 0x7f101415c890 2023-11-28T11:09:26.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.613+0000 7f101ae10700 1 -- 172.21.15.28:0/1230721235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10141611a0 con 0x7f101415c890 2023-11-28T11:09:26.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.614+0000 7f101ae10700 1 -- 172.21.15.28:0/1230721235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10140b3070 con 0x7f101415c890 2023-11-28T11:09:26.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.615+0000 7f10117fa700 1 -- 172.21.15.28:0/1230721235 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1000021070 con 0x7f101415c890 2023-11-28T11:09:26.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.615+0000 7f10117fa700 1 --2- 172.21.15.28:0/1230721235 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0ff80682f0 0x7f0ff806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.615+0000 7f10117fa700 1 -- 172.21.15.28:0/1230721235 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1000085f10 con 0x7f101415c890 2023-11-28T11:09:26.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.615+0000 7f1018bac700 1 --2- 172.21.15.28:0/1230721235 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0ff80682f0 0x7f0ff806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.616+0000 7f1018bac700 1 --2- 172.21.15.28:0/1230721235 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0ff80682f0 0x7f0ff806a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1008009950 tx=0x7f1008008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.619+0000 7f10117fa700 1 -- 172.21.15.28:0/1230721235 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1000052310 con 0x7f101415c890 2023-11-28T11:09:26.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.810+0000 7f101ae10700 1 -- 172.21.15.28:0/1230721235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f10140739f0 con 0x7f101415c890 2023-11-28T11:09:26.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.811+0000 7f10117fa700 1 -- 172.21.15.28:0/1230721235 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f100001f070 con 0x7f101415c890 2023-11-28T11:09:26.812 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:09:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.813+0000 7f101ae10700 1 -- 172.21.15.28:0/1230721235 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0ff80682f0 msgr2=0x7f0ff806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.813+0000 7f101ae10700 1 --2- 172.21.15.28:0/1230721235 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0ff80682f0 0x7f0ff806a7a0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1008009950 tx=0x7f1008008040 comp rx=0 tx=0).stop 2023-11-28T11:09:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.813+0000 7f101ae10700 1 -- 172.21.15.28:0/1230721235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101415c890 msgr2=0x7f101415ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.813+0000 7f101ae10700 1 --2- 172.21.15.28:0/1230721235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101415c890 0x7f101415ef30 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f100000c7f0 tx=0x7f100000cb00 comp rx=0 tx=0).stop 2023-11-28T11:09:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.813+0000 7f101ae10700 1 -- 172.21.15.28:0/1230721235 shutdown_connections 2023-11-28T11:09:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.813+0000 7f101ae10700 1 --2- 172.21.15.28:0/1230721235 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0ff80682f0 0x7f0ff806a7a0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.813+0000 7f101ae10700 1 --2- 172.21.15.28:0/1230721235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1014160d30 0x7f10141f1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.813+0000 7f101ae10700 1 --2- 172.21.15.28:0/1230721235 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f10140ce7b0 0x7f101415f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.813+0000 7f101ae10700 1 --2- 172.21.15.28:0/1230721235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f101415c890 0x7f101415ef30 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.813+0000 7f101ae10700 1 -- 172.21.15.28:0/1230721235 >> 172.21.15.28:0/1230721235 conn(0x7f101406e9b0 msgr2=0x7f10140d0680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.813+0000 7f101ae10700 1 -- 172.21.15.28:0/1230721235 shutdown_connections 2023-11-28T11:09:26.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.813+0000 7f101ae10700 1 -- 172.21.15.28:0/1230721235 wait complete. 2023-11-28T11:09:26.827 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.953+0000 7fa2d4140700 1 Processor -- start 2023-11-28T11:09:26.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.953+0000 7fa2d4140700 1 -- start start 2023-11-28T11:09:26.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.954+0000 7fa2d4140700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2cc0cf5f0 0x7fa2cc0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.954+0000 7fa2d4140700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2cc0ce550 con 0x7fa2cc0cb3e0 2023-11-28T11:09:26.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.954+0000 7fa2d4140700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2cc0ce830 con 0x7fa2cc0b1630 2023-11-28T11:09:26.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.954+0000 7fa2d4140700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc0cfa60 con 0x7fa2cc0cf5f0 2023-11-28T11:09:26.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.954+0000 7fa2d26dd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa2cc0cb3e0 0x7fa2cc06afc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35826/0 (socket says 172.21.15.28:35826) 2023-11-28T11:09:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.954+0000 7fa2d26dd700 1 -- 172.21.15.28:0/2751353137 learned_addr learned my addr 172.21.15.28:0/2751353137 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.954+0000 7fa2d16db700 1 --2- 172.21.15.28:0/2751353137 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2cc0cf5f0 0x7fa2cc0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.955+0000 7fa2d0eda700 1 -- 172.21.15.28:0/2751353137 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 427483781 0 0) 0x7fa2cc0ce550 con 0x7fa2cc0cb3e0 2023-11-28T11:09:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.955+0000 7fa2d0eda700 1 -- 172.21.15.28:0/2751353137 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa2b80035b0 con 0x7fa2cc0cb3e0 2023-11-28T11:09:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.955+0000 7fa2d16db700 1 -- 172.21.15.28:0/2751353137 >> v1:172.21.15.59:6789/0 conn(0x7fa2cc0b1630 legacy=0x7fa2cc0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.955+0000 7fa2d0eda700 1 -- 172.21.15.28:0/2751353137 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25583977 0 0) 0x7fa2cc0ce830 con 0x7fa2cc0b1630 2023-11-28T11:09:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.955+0000 7fa2d16db700 1 -- 172.21.15.28:0/2751353137 >> v1:172.21.15.28:6789/0 conn(0x7fa2cc0cb3e0 legacy=0x7fa2cc06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.955+0000 7fa2d16db700 1 -- 172.21.15.28:0/2751353137 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2cc0ceb10 con 0x7fa2cc0cf5f0 2023-11-28T11:09:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.955+0000 7fa2d16db700 1 --2- 172.21.15.28:0/2751353137 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2cc0cf5f0 0x7fa2cc0caea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009870 tx=0x7fa2c8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=787065b7548c1154 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.955+0000 7fa2d0eda700 1 -- 172.21.15.28:0/2751353137 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8011830 con 0x7fa2cc0cf5f0 2023-11-28T11:09:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.955+0000 7fa2d0eda700 1 -- 172.21.15.28:0/2751353137 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c8011990 con 0x7fa2cc0cf5f0 2023-11-28T11:09:26.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.955+0000 7fa2d0eda700 1 -- 172.21.15.28:0/2751353137 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c8011d00 con 0x7fa2cc0cf5f0 2023-11-28T11:09:26.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.956+0000 7fa2d4140700 1 -- 172.21.15.28:0/2751353137 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2cc0cf5f0 msgr2=0x7fa2cc0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.956+0000 7fa2d4140700 1 --2- 172.21.15.28:0/2751353137 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2cc0cf5f0 0x7fa2cc0caea0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa2c8009870 tx=0x7fa2c8009b80 comp rx=0 tx=0).stop 2023-11-28T11:09:26.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.956+0000 7fa2d4140700 1 -- 172.21.15.28:0/2751353137 shutdown_connections 2023-11-28T11:09:26.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.956+0000 7fa2d4140700 1 --2- 172.21.15.28:0/2751353137 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2cc0cf5f0 0x7fa2cc0caea0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.956+0000 7fa2d4140700 1 -- 172.21.15.28:0/2751353137 >> 172.21.15.28:0/2751353137 conn(0x7fa2cc0cc630 msgr2=0x7fa2cc0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:26.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.956+0000 7fa2d4140700 1 -- 172.21.15.28:0/2751353137 shutdown_connections 2023-11-28T11:09:26.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.956+0000 7fa2d4140700 1 -- 172.21.15.28:0/2751353137 wait complete. 2023-11-28T11:09:26.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.957+0000 7fa2d4140700 1 Processor -- start 2023-11-28T11:09:26.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.957+0000 7fa2d4140700 1 -- start start 2023-11-28T11:09:26.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.958+0000 7fa2d4140700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2cc0b1630 0x7fa2cc14e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.958+0000 7fa2d4140700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2cc0cb3e0 0x7fa2cc0b53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.958+0000 7fa2d4140700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2cc0cf5f0 0x7fa2cc0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.958+0000 7fa2d4140700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc15eaf0 con 0x7fa2cc0cf5f0 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.958+0000 7fa2d4140700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc14e5e0 con 0x7fa2cc0cb3e0 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.958+0000 7fa2d4140700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2cc0b5920 con 0x7fa2cc0b1630 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.958+0000 7fa2d26dd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2cc0cf5f0 0x7fa2cc0ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2d26dd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2cc0cf5f0 0x7fa2cc0ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47774/0 (socket says 172.21.15.28:47774) 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2d26dd700 1 -- 172.21.15.28:0/474852801 learned_addr learned my addr 172.21.15.28:0/474852801 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2d16db700 1 --2- 172.21.15.28:0/474852801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2cc0cb3e0 0x7fa2cc0b53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2d1edc700 1 --2- 172.21.15.28:0/474852801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2cc0b1630 0x7fa2cc14e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2d26dd700 1 -- 172.21.15.28:0/474852801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2cc0b1630 msgr2=0x7fa2cc14e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2d26dd700 1 --2- 172.21.15.28:0/474852801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2cc0b1630 0x7fa2cc14e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2d26dd700 1 -- 172.21.15.28:0/474852801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2cc0cb3e0 msgr2=0x7fa2cc0b53e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:26.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2d26dd700 1 --2- 172.21.15.28:0/474852801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2cc0cb3e0 0x7fa2cc0b53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:26.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2d26dd700 1 -- 172.21.15.28:0/474852801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2c80094a0 con 0x7fa2cc0cf5f0 2023-11-28T11:09:26.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2d1edc700 1 --2- 172.21.15.28:0/474852801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2cc0b1630 0x7fa2cc14e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:26.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2d26dd700 1 --2- 172.21.15.28:0/474852801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2cc0cf5f0 0x7fa2cc0ab8c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400cb70 tx=0x7fa2c400cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2c27fc700 1 -- 172.21.15.28:0/474852801 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c400ab10 con 0x7fa2cc0cf5f0 2023-11-28T11:09:26.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2c27fc700 1 -- 172.21.15.28:0/474852801 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2c400ac70 con 0x7fa2cc0cf5f0 2023-11-28T11:09:26.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2d4140700 1 -- 172.21.15.28:0/474852801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2cc0cf0d0 con 0x7fa2cc0cf5f0 2023-11-28T11:09:26.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2c27fc700 1 -- 172.21.15.28:0/474852801 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2c4012400 con 0x7fa2cc0cf5f0 2023-11-28T11:09:26.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.959+0000 7fa2d4140700 1 -- 172.21.15.28:0/474852801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2cc0cedf0 con 0x7fa2cc0cf5f0 2023-11-28T11:09:26.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.960+0000 7fa2d4140700 1 -- 172.21.15.28:0/474852801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2cc0ce140 con 0x7fa2cc0cf5f0 2023-11-28T11:09:26.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.961+0000 7fa2c27fc700 1 -- 172.21.15.28:0/474852801 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa2c4019030 con 0x7fa2cc0cf5f0 2023-11-28T11:09:26.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.961+0000 7fa2c27fc700 1 --2- 172.21.15.28:0/474852801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa2b00682f0 0x7fa2b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:26.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.961+0000 7fa2c27fc700 1 -- 172.21.15.28:0/474852801 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa2c4085c40 con 0x7fa2cc0cf5f0 2023-11-28T11:09:26.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.961+0000 7fa2d1edc700 1 --2- 172.21.15.28:0/474852801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa2b00682f0 0x7fa2b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:26.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.961+0000 7fa2d1edc700 1 --2- 172.21.15.28:0/474852801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa2b00682f0 0x7fa2b006a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc000f60 tx=0x7fa2bc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:26.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:26.965+0000 7fa2c27fc700 1 -- 172.21.15.28:0/474852801 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa2c40521b0 con 0x7fa2cc0cf5f0 2023-11-28T11:09:27.160 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.158+0000 7fa2d4140700 1 -- 172.21.15.28:0/474852801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa2cc15ec30 con 0x7fa2cc0cf5f0 2023-11-28T11:09:27.160 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.158+0000 7fa2c27fc700 1 -- 172.21.15.28:0/474852801 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa2c4017020 con 0x7fa2cc0cf5f0 2023-11-28T11:09:27.160 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:27.162 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.160+0000 7fa2d4140700 1 -- 172.21.15.28:0/474852801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa2b00682f0 msgr2=0x7fa2b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.162 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.160+0000 7fa2d4140700 1 --2- 172.21.15.28:0/474852801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa2b00682f0 0x7fa2b006a7a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fa2bc000f60 tx=0x7fa2bc009040 comp rx=0 tx=0).stop 2023-11-28T11:09:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.160+0000 7fa2d4140700 1 -- 172.21.15.28:0/474852801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2cc0cf5f0 msgr2=0x7fa2cc0ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.160+0000 7fa2d4140700 1 --2- 172.21.15.28:0/474852801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2cc0cf5f0 0x7fa2cc0ab8c0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa2c400cb70 tx=0x7fa2c400cf30 comp rx=0 tx=0).stop 2023-11-28T11:09:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.161+0000 7fa2d4140700 1 -- 172.21.15.28:0/474852801 shutdown_connections 2023-11-28T11:09:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.161+0000 7fa2d4140700 1 --2- 172.21.15.28:0/474852801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa2b00682f0 0x7fa2b006a7a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.161+0000 7fa2d4140700 1 --2- 172.21.15.28:0/474852801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa2cc0cf5f0 0x7fa2cc0ab8c0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.161+0000 7fa2d4140700 1 --2- 172.21.15.28:0/474852801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa2cc0b1630 0x7fa2cc14e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.161+0000 7fa2d4140700 1 --2- 172.21.15.28:0/474852801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa2cc0cb3e0 0x7fa2cc0b53e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.161+0000 7fa2d4140700 1 -- 172.21.15.28:0/474852801 >> 172.21.15.28:0/474852801 conn(0x7fa2cc0cc630 msgr2=0x7fa2cc0b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.161+0000 7fa2d4140700 1 -- 172.21.15.28:0/474852801 shutdown_connections 2023-11-28T11:09:27.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.161+0000 7fa2d4140700 1 -- 172.21.15.28:0/474852801 wait complete. 2023-11-28T11:09:27.175 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:27.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.298+0000 7f19b7a9e700 1 Processor -- start 2023-11-28T11:09:27.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.298+0000 7f19b7a9e700 1 -- start start 2023-11-28T11:09:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.299+0000 7f19b7a9e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19b015c990 0x7f19b00cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.299+0000 7f19b7a9e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f19b00cc310 0x7f19b00cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.299+0000 7f19b7a9e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19b00ce340 con 0x7f19b0161090 2023-11-28T11:09:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.299+0000 7f19b7a9e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19b00cebe0 con 0x7f19b00cc310 2023-11-28T11:09:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.299+0000 7f19b7a9e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b00cccf0 con 0x7f19b015c990 2023-11-28T11:09:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.299+0000 7f19b603b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f19b00cc310 0x7f19b00cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.299+0000 7f19b583a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19b015c990 0x7f19b00cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.299+0000 7f19b603b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f19b00cc310 0x7f19b00cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:35072/0 (socket says 172.21.15.28:35072) 2023-11-28T11:09:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.299+0000 7f19b5039700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f19b0161090 0x7f19b00ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:46624/0 (socket says 172.21.15.28:46624) 2023-11-28T11:09:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.299+0000 7f19b603b700 1 -- 172.21.15.28:0/2434603341 learned_addr learned my addr 172.21.15.28:0/2434603341 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:27.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.300+0000 7f19b583a700 1 -- 172.21.15.28:0/2434603341 >> v2:172.21.15.59:3300/0 conn(0x7f19b00cc310 msgr2=0x7f19b00cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.300+0000 7f19b583a700 1 --2- 172.21.15.28:0/2434603341 >> v2:172.21.15.59:3300/0 conn(0x7f19b00cc310 0x7f19b00cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.300+0000 7f19b583a700 1 -- 172.21.15.28:0/2434603341 >> v1:172.21.15.59:6789/0 conn(0x7f19b0161090 legacy=0x7f19b00ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.300+0000 7f19b583a700 1 -- 172.21.15.28:0/2434603341 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b00cb3f0 con 0x7f19b015c990 2023-11-28T11:09:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.300+0000 7f19b603b700 1 --2- 172.21.15.28:0/2434603341 >> v2:172.21.15.59:3300/0 conn(0x7f19b00cc310 0x7f19b00cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.300+0000 7f19b583a700 1 --2- 172.21.15.28:0/2434603341 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19b015c990 0x7f19b00cf070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009840 tx=0x7f19a0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=51bfbac53e00680b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.300+0000 7f19b4838700 1 -- 172.21.15.28:0/2434603341 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0011830 con 0x7f19b015c990 2023-11-28T11:09:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.300+0000 7f19b4838700 1 -- 172.21.15.28:0/2434603341 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a0011990 con 0x7f19b015c990 2023-11-28T11:09:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.300+0000 7f19b4838700 1 -- 172.21.15.28:0/2434603341 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a0011d20 con 0x7f19b015c990 2023-11-28T11:09:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.300+0000 7f19b7a9e700 1 -- 172.21.15.28:0/2434603341 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19b015c990 msgr2=0x7f19b00cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.300+0000 7f19b7a9e700 1 --2- 172.21.15.28:0/2434603341 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19b015c990 0x7f19b00cf070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009840 tx=0x7f19a0009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.301+0000 7f19b7a9e700 1 -- 172.21.15.28:0/2434603341 shutdown_connections 2023-11-28T11:09:27.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.301+0000 7f19b7a9e700 1 --2- 172.21.15.28:0/2434603341 >> v2:172.21.15.59:3300/0 conn(0x7f19b00cc310 0x7f19b00cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.301+0000 7f19b7a9e700 1 --2- 172.21.15.28:0/2434603341 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19b015c990 0x7f19b00cf070 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.301+0000 7f19b7a9e700 1 -- 172.21.15.28:0/2434603341 >> 172.21.15.28:0/2434603341 conn(0x7f19b00b1650 msgr2=0x7f19b00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:27.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.301+0000 7f19b7a9e700 1 -- 172.21.15.28:0/2434603341 shutdown_connections 2023-11-28T11:09:27.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.301+0000 7f19b7a9e700 1 -- 172.21.15.28:0/2434603341 wait complete. 2023-11-28T11:09:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.302+0000 7f19b7a9e700 1 Processor -- start 2023-11-28T11:09:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.302+0000 7f19b7a9e700 1 -- start start 2023-11-28T11:09:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b7a9e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19b00cc310 0x7f19b0159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b7a9e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19b015c990 0x7f19b00c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b7a9e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b0161090 0x7f19b006c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b7a9e700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b01f6160 con 0x7f19b0161090 2023-11-28T11:09:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b7a9e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19b01f6400 con 0x7f19b00cc310 2023-11-28T11:09:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b7a9e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19b01f66e0 con 0x7f19b015c990 2023-11-28T11:09:27.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b603b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b0161090 0x7f19b006c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b583a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19b00cc310 0x7f19b0159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b603b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b0161090 0x7f19b006c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47788/0 (socket says 172.21.15.28:47788) 2023-11-28T11:09:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b603b700 1 -- 172.21.15.28:0/1297449696 learned_addr learned my addr 172.21.15.28:0/1297449696 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b5039700 1 --2- 172.21.15.28:0/1297449696 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19b015c990 0x7f19b00c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b603b700 1 -- 172.21.15.28:0/1297449696 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19b015c990 msgr2=0x7f19b00c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b603b700 1 --2- 172.21.15.28:0/1297449696 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19b015c990 0x7f19b00c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b603b700 1 -- 172.21.15.28:0/1297449696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19b00cc310 msgr2=0x7f19b0159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b603b700 1 --2- 172.21.15.28:0/1297449696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19b00cc310 0x7f19b0159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b603b700 1 -- 172.21.15.28:0/1297449696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19a8009b30 con 0x7f19b0161090 2023-11-28T11:09:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b603b700 1 --2- 172.21.15.28:0/1297449696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b0161090 0x7f19b006c860 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f19a800efc0 tx=0x7f19a800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19a67fc700 1 -- 172.21.15.28:0/1297449696 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a8018070 con 0x7f19b0161090 2023-11-28T11:09:27.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19a67fc700 1 -- 172.21.15.28:0/1297449696 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19a800f040 con 0x7f19b0161090 2023-11-28T11:09:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19a67fc700 1 -- 172.21.15.28:0/1297449696 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19a8003c40 con 0x7f19b0161090 2023-11-28T11:09:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b7a9e700 1 -- 172.21.15.28:0/1297449696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19a00094a0 con 0x7f19b0161090 2023-11-28T11:09:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.303+0000 7f19b7a9e700 1 -- 172.21.15.28:0/1297449696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19b00ce870 con 0x7f19b0161090 2023-11-28T11:09:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.304+0000 7f19b7a9e700 1 -- 172.21.15.28:0/1297449696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19b00ce590 con 0x7f19b0161090 2023-11-28T11:09:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.305+0000 7f19a67fc700 1 -- 172.21.15.28:0/1297449696 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f19a800c880 con 0x7f19b0161090 2023-11-28T11:09:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.305+0000 7f19a67fc700 1 --2- 172.21.15.28:0/1297449696 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f19940682f0 0x7f199406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.305+0000 7f19a67fc700 1 -- 172.21.15.28:0/1297449696 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f19a80861a0 con 0x7f19b0161090 2023-11-28T11:09:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.305+0000 7f19b583a700 1 --2- 172.21.15.28:0/1297449696 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f19940682f0 0x7f199406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.306+0000 7f19b583a700 1 --2- 172.21.15.28:0/1297449696 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f19940682f0 0x7f199406a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009fd0 tx=0x7f19a0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:27.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.310+0000 7f19a67fc700 1 -- 172.21.15.28:0/1297449696 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f19a8052710 con 0x7f19b0161090 2023-11-28T11:09:27.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.504+0000 7f19b7a9e700 1 -- 172.21.15.28:0/1297449696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f19b01f29b0 con 0x7f19b0161090 2023-11-28T11:09:27.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.505+0000 7f19a67fc700 1 -- 172.21.15.28:0/1297449696 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f19a8009ce0 con 0x7f19b0161090 2023-11-28T11:09:27.507 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:27.507 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:50.531733+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.507+0000 7f19b7a9e700 1 -- 172.21.15.28:0/1297449696 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f19940682f0 msgr2=0x7f199406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.507+0000 7f19b7a9e700 1 --2- 172.21.15.28:0/1297449696 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f19940682f0 0x7f199406a7a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f19a0009fd0 tx=0x7f19a0008030 comp rx=0 tx=0).stop 2023-11-28T11:09:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.507+0000 7f19b7a9e700 1 -- 172.21.15.28:0/1297449696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b0161090 msgr2=0x7f19b006c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.507+0000 7f19b7a9e700 1 --2- 172.21.15.28:0/1297449696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b0161090 0x7f19b006c860 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f19a800efc0 tx=0x7f19a800c3a0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.507+0000 7f19b7a9e700 1 -- 172.21.15.28:0/1297449696 shutdown_connections 2023-11-28T11:09:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.507+0000 7f19b7a9e700 1 --2- 172.21.15.28:0/1297449696 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f19940682f0 0x7f199406a7a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.507+0000 7f19b7a9e700 1 --2- 172.21.15.28:0/1297449696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f19b0161090 0x7f19b006c860 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.507+0000 7f19b7a9e700 1 --2- 172.21.15.28:0/1297449696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f19b00cc310 0x7f19b0159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.507+0000 7f19b7a9e700 1 --2- 172.21.15.28:0/1297449696 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f19b015c990 0x7f19b00c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.507+0000 7f19b7a9e700 1 -- 172.21.15.28:0/1297449696 >> 172.21.15.28:0/1297449696 conn(0x7f19b00b1650 msgr2=0x7f19b015bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.507+0000 7f19b7a9e700 1 -- 172.21.15.28:0/1297449696 shutdown_connections 2023-11-28T11:09:27.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.507+0000 7f19b7a9e700 1 -- 172.21.15.28:0/1297449696 wait complete. 2023-11-28T11:09:27.521 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:27.521 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:27.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.647+0000 7f3ea5912700 1 Processor -- start 2023-11-28T11:09:27.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.647+0000 7f3ea5912700 1 -- start start 2023-11-28T11:09:27.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.648+0000 7f3ea5912700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3ea015c640 0x7f3ea00cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.648+0000 7f3ea5912700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ea00d2820 0x7f3ea00b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.648+0000 7f3ea5912700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ea00d1470 con 0x7f3ea006bf30 2023-11-28T11:09:27.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.648+0000 7f3ea5912700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ea015ca10 con 0x7f3ea015c640 2023-11-28T11:09:27.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.648+0000 7f3ea5912700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00cd4e0 con 0x7f3ea00d2820 2023-11-28T11:09:27.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.648+0000 7f3e9f7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3ea006bf30 0x7f3ea006c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35854/0 (socket says 172.21.15.28:35854) 2023-11-28T11:09:27.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.648+0000 7f3e9f7fe700 1 -- 172.21.15.28:0/712953956 learned_addr learned my addr 172.21.15.28:0/712953956 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:27.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.648+0000 7f3e9effd700 1 --2- 172.21.15.28:0/712953956 >> v2:172.21.15.59:3300/0 conn(0x7f3ea015c640 0x7f3ea00cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.648+0000 7f3e9dffb700 1 -- 172.21.15.28:0/712953956 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 378911149 0 0) 0x7f3ea00d1470 con 0x7f3ea006bf30 2023-11-28T11:09:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.648+0000 7f3e9dffb700 1 -- 172.21.15.28:0/712953956 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3e880035b0 con 0x7f3ea006bf30 2023-11-28T11:09:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.648+0000 7f3e9e7fc700 1 --2- 172.21.15.28:0/712953956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ea00d2820 0x7f3ea00b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.648+0000 7f3e9dffb700 1 -- 172.21.15.28:0/712953956 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 503300134 0 0) 0x7f3e880035b0 con 0x7f3ea006bf30 2023-11-28T11:09:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 -- 172.21.15.28:0/712953956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ea00d2820 msgr2=0x7f3ea00b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 --2- 172.21.15.28:0/712953956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ea00d2820 0x7f3ea00b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 -- 172.21.15.28:0/712953956 >> v2:172.21.15.59:3300/0 conn(0x7f3ea015c640 msgr2=0x7f3ea00cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 --2- 172.21.15.28:0/712953956 >> v2:172.21.15.59:3300/0 conn(0x7f3ea015c640 0x7f3ea00cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 -- 172.21.15.28:0/712953956 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea00cd7c0 con 0x7f3ea006bf30 2023-11-28T11:09:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 -- 172.21.15.28:0/712953956 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f3e980019f0 con 0x7f3ea006bf30 2023-11-28T11:09:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 -- 172.21.15.28:0/712953956 >> v1:172.21.15.28:6789/0 conn(0x7f3ea006bf30 legacy=0x7f3ea006c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 --2- 172.21.15.28:0/712953956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ea00d2820 0x7f3e88003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 --2- 172.21.15.28:0/712953956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e88003f80 0x7f3e88004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 --2- 172.21.15.28:0/712953956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e88004d90 0x7f3e88005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 -- 172.21.15.28:0/712953956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea015ca10 con 0x7f3e88003f80 2023-11-28T11:09:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 -- 172.21.15.28:0/712953956 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00cd4e0 con 0x7f3e88004d90 2023-11-28T11:09:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 -- 172.21.15.28:0/712953956 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e8800ba40 con 0x7f3ea00d2820 2023-11-28T11:09:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9e7fc700 1 --2- 172.21.15.28:0/712953956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e88003f80 0x7f3e88004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9effd700 1 --2- 172.21.15.28:0/712953956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ea00d2820 0x7f3e88003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9f7fe700 1 --2- 172.21.15.28:0/712953956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e88004d90 0x7f3e88005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9e7fc700 1 -- 172.21.15.28:0/712953956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ea00d2820 msgr2=0x7f3e88003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9e7fc700 1 --2- 172.21.15.28:0/712953956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ea00d2820 0x7f3e88003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9e7fc700 1 -- 172.21.15.28:0/712953956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e88004d90 msgr2=0x7f3e88005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:27.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9e7fc700 1 --2- 172.21.15.28:0/712953956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e88004d90 0x7f3e88005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9e7fc700 1 -- 172.21.15.28:0/712953956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e880035b0 con 0x7f3e88003f80 2023-11-28T11:09:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9f7fe700 1 --2- 172.21.15.28:0/712953956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e88004d90 0x7f3e88005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9e7fc700 1 --2- 172.21.15.28:0/712953956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e88003f80 0x7f3e88004850 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009b60 tx=0x7f3e88005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5a55bc5ffd94f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 -- 172.21.15.28:0/712953956 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000e6c0 con 0x7f3e88003f80 2023-11-28T11:09:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 -- 172.21.15.28:0/712953956 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9000e820 con 0x7f3e88003f80 2023-11-28T11:09:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.649+0000 7f3e9dffb700 1 -- 172.21.15.28:0/712953956 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9000eb90 con 0x7f3e88003f80 2023-11-28T11:09:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.650+0000 7f3ea5912700 1 -- 172.21.15.28:0/712953956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e88003f80 msgr2=0x7f3e88004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.650+0000 7f3ea5912700 1 --2- 172.21.15.28:0/712953956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e88003f80 0x7f3e88004850 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f3e90009b60 tx=0x7f3e88005880 comp rx=0 tx=0).stop 2023-11-28T11:09:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.650+0000 7f3e9f7fe700 1 -- 172.21.15.28:0/712953956 reap_dead start 2023-11-28T11:09:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.650+0000 7f3ea5912700 1 -- 172.21.15.28:0/712953956 shutdown_connections 2023-11-28T11:09:27.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.650+0000 7f3ea5912700 1 -- 172.21.15.28:0/712953956 >> 172.21.15.28:0/712953956 conn(0x7f3ea00d2df0 msgr2=0x7f3ea00d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:27.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.650+0000 7f3ea5912700 1 -- 172.21.15.28:0/712953956 shutdown_connections 2023-11-28T11:09:27.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.650+0000 7f3ea5912700 1 -- 172.21.15.28:0/712953956 wait complete. 2023-11-28T11:09:27.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.651+0000 7f3ea5912700 1 Processor -- start 2023-11-28T11:09:27.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.651+0000 7f3ea5912700 1 -- start start 2023-11-28T11:09:27.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3ea5912700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e88003f80 0x7f3ea00c6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3ea5912700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ea00b0c00 0x7f3ea00b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3ea5912700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ea015e5f0 0x7f3ea015ea40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3ea5912700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea015f5e0 con 0x7f3e88003f80 2023-11-28T11:09:27.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3ea5912700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea00b15b0 con 0x7f3ea00b0c00 2023-11-28T11:09:27.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3ea5912700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ea015f300 con 0x7f3ea015e5f0 2023-11-28T11:09:27.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3e9effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e88003f80 0x7f3ea00c6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3e9effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e88003f80 0x7f3ea00c6b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47804/0 (socket says 172.21.15.28:47804) 2023-11-28T11:09:27.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3e9effd700 1 -- 172.21.15.28:0/3747986281 learned_addr learned my addr 172.21.15.28:0/3747986281 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:27.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3e9f7fe700 1 --2- 172.21.15.28:0/3747986281 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ea015e5f0 0x7f3ea015ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3e9e7fc700 1 --2- 172.21.15.28:0/3747986281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ea00b0c00 0x7f3ea00b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3e9effd700 1 -- 172.21.15.28:0/3747986281 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ea015e5f0 msgr2=0x7f3ea015ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3e9effd700 1 --2- 172.21.15.28:0/3747986281 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ea015e5f0 0x7f3ea015ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3e9effd700 1 -- 172.21.15.28:0/3747986281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ea00b0c00 msgr2=0x7f3ea00b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3e9effd700 1 --2- 172.21.15.28:0/3747986281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ea00b0c00 0x7f3ea00b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3e9effd700 1 -- 172.21.15.28:0/3747986281 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ea00cf580 con 0x7f3e88003f80 2023-11-28T11:09:27.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.652+0000 7f3e9e7fc700 1 --2- 172.21.15.28:0/3747986281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ea00b0c00 0x7f3ea00b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:27.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.653+0000 7f3e9effd700 1 --2- 172.21.15.28:0/3747986281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e88003f80 0x7f3ea00c6b60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3e94011ad0 tx=0x7f3e94011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:27.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.653+0000 7f3e87fff700 1 -- 172.21.15.28:0/3747986281 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e9400fa60 con 0x7f3e88003f80 2023-11-28T11:09:27.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.653+0000 7f3e87fff700 1 -- 172.21.15.28:0/3747986281 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e9400fbc0 con 0x7f3e88003f80 2023-11-28T11:09:27.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.653+0000 7f3ea5912700 1 -- 172.21.15.28:0/3747986281 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ea00d1710 con 0x7f3e88003f80 2023-11-28T11:09:27.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.653+0000 7f3e87fff700 1 -- 172.21.15.28:0/3747986281 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e94014440 con 0x7f3e88003f80 2023-11-28T11:09:27.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.653+0000 7f3ea5912700 1 -- 172.21.15.28:0/3747986281 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ea00cdaa0 con 0x7f3e88003f80 2023-11-28T11:09:27.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.654+0000 7f3ea5912700 1 -- 172.21.15.28:0/3747986281 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ea00739f0 con 0x7f3e88003f80 2023-11-28T11:09:27.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.654+0000 7f3e87fff700 1 -- 172.21.15.28:0/3747986281 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3e940145a0 con 0x7f3e88003f80 2023-11-28T11:09:27.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.655+0000 7f3e87fff700 1 --2- 172.21.15.28:0/3747986281 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3e8c068320 0x7f3e8c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.655+0000 7f3e87fff700 1 -- 172.21.15.28:0/3747986281 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f3e94086830 con 0x7f3e88003f80 2023-11-28T11:09:27.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.655+0000 7f3e9e7fc700 1 --2- 172.21.15.28:0/3747986281 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3e8c068320 0x7f3e8c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.655+0000 7f3e9e7fc700 1 --2- 172.21.15.28:0/3747986281 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3e8c068320 0x7f3e8c06a7d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00b8540 tx=0x7f3e90008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:27.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.659+0000 7f3e87fff700 1 -- 172.21.15.28:0/3747986281 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3e94052d40 con 0x7f3e88003f80 2023-11-28T11:09:27.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.852+0000 7f3ea5912700 1 -- 172.21.15.28:0/3747986281 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f3ea00b4630 con 0x7f3e88003f80 2023-11-28T11:09:27.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.852+0000 7f3e87fff700 1 -- 172.21.15.28:0/3747986281 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f3e94056360 con 0x7f3e88003f80 2023-11-28T11:09:27.854 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:09:27.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.854+0000 7f3ea5912700 1 -- 172.21.15.28:0/3747986281 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3e8c068320 msgr2=0x7f3e8c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.854+0000 7f3ea5912700 1 --2- 172.21.15.28:0/3747986281 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3e8c068320 0x7f3e8c06a7d0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3ea00b8540 tx=0x7f3e90008030 comp rx=0 tx=0).stop 2023-11-28T11:09:27.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.854+0000 7f3ea5912700 1 -- 172.21.15.28:0/3747986281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e88003f80 msgr2=0x7f3ea00c6b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:27.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.854+0000 7f3ea5912700 1 --2- 172.21.15.28:0/3747986281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e88003f80 0x7f3ea00c6b60 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3e94011ad0 tx=0x7f3e94011e90 comp rx=0 tx=0).stop 2023-11-28T11:09:27.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.854+0000 7f3ea5912700 1 -- 172.21.15.28:0/3747986281 shutdown_connections 2023-11-28T11:09:27.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.854+0000 7f3ea5912700 1 --2- 172.21.15.28:0/3747986281 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3e8c068320 0x7f3e8c06a7d0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.854+0000 7f3ea5912700 1 --2- 172.21.15.28:0/3747986281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3e88003f80 0x7f3ea00c6b60 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.854+0000 7f3ea5912700 1 --2- 172.21.15.28:0/3747986281 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ea015e5f0 0x7f3ea015ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.854+0000 7f3ea5912700 1 --2- 172.21.15.28:0/3747986281 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ea00b0c00 0x7f3ea00b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:27.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.855+0000 7f3ea5912700 1 -- 172.21.15.28:0/3747986281 >> 172.21.15.28:0/3747986281 conn(0x7f3ea00d2df0 msgr2=0x7f3ea015a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:27.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.855+0000 7f3ea5912700 1 -- 172.21.15.28:0/3747986281 shutdown_connections 2023-11-28T11:09:27.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.855+0000 7f3ea5912700 1 -- 172.21.15.28:0/3747986281 wait complete. 2023-11-28T11:09:27.868 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:27.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.995+0000 7f273e6ab700 1 Processor -- start 2023-11-28T11:09:27.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.995+0000 7f273e6ab700 1 -- start start 2023-11-28T11:09:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.995+0000 7f273e6ab700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27380b1630 0x7f27380cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.995+0000 7f273e6ab700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f273806afc0 0x7f273806b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f273e6ab700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27380ce550 con 0x7f27380b1a00 2023-11-28T11:09:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f273e6ab700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27380ce830 con 0x7f273806afc0 2023-11-28T11:09:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f273e6ab700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27380ceb10 con 0x7f27380b1630 2023-11-28T11:09:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f273cc48700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f273806afc0 0x7f273806b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f2737fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27380b1630 0x7f27380cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f2737fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27380b1630 0x7f27380cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:53570/0 (socket says 172.21.15.28:53570) 2023-11-28T11:09:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f273cc48700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f273806afc0 0x7f273806b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:35104/0 (socket says 172.21.15.28:35104) 2023-11-28T11:09:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f2737fff700 1 -- 172.21.15.28:0/1489070888 learned_addr learned my addr 172.21.15.28:0/1489070888 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f2736ffd700 1 -- 172.21.15.28:0/1489070888 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940958447 0 0) 0x7f27380ce550 con 0x7f27380b1a00 2023-11-28T11:09:27.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f2736ffd700 1 -- 172.21.15.28:0/1489070888 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27200035b0 con 0x7f27380b1a00 2023-11-28T11:09:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f2737fff700 1 -- 172.21.15.28:0/1489070888 >> v2:172.21.15.59:3300/0 conn(0x7f273806afc0 msgr2=0x7f273806b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f2737fff700 1 --2- 172.21.15.28:0/1489070888 >> v2:172.21.15.59:3300/0 conn(0x7f273806afc0 0x7f273806b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f2737fff700 1 -- 172.21.15.28:0/1489070888 >> v1:172.21.15.59:6789/0 conn(0x7f27380b1a00 legacy=0x7f27380caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f2737fff700 1 -- 172.21.15.28:0/1489070888 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27380cedf0 con 0x7f27380b1630 2023-11-28T11:09:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f273cc48700 1 --2- 172.21.15.28:0/1489070888 >> v2:172.21.15.59:3300/0 conn(0x7f273806afc0 0x7f273806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.996+0000 7f2737fff700 1 --2- 172.21.15.28:0/1489070888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27380b1630 0x7f27380cf5f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f27280098f0 tx=0x7f2728009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fc1758edd1455384 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.997+0000 7f2736ffd700 1 -- 172.21.15.28:0/1489070888 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2728011830 con 0x7f27380b1630 2023-11-28T11:09:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.997+0000 7f2736ffd700 1 -- 172.21.15.28:0/1489070888 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2728011990 con 0x7f27380b1630 2023-11-28T11:09:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.997+0000 7f2736ffd700 1 -- 172.21.15.28:0/1489070888 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2728011d20 con 0x7f27380b1630 2023-11-28T11:09:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.997+0000 7f273e6ab700 1 -- 172.21.15.28:0/1489070888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27380b1630 msgr2=0x7f27380cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.997+0000 7f273e6ab700 1 --2- 172.21.15.28:0/1489070888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27380b1630 0x7f27380cf5f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f27280098f0 tx=0x7f2728009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:28.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.997+0000 7f273e6ab700 1 -- 172.21.15.28:0/1489070888 shutdown_connections 2023-11-28T11:09:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.997+0000 7f273e6ab700 1 --2- 172.21.15.28:0/1489070888 >> v2:172.21.15.59:3300/0 conn(0x7f273806afc0 0x7f273806b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.997+0000 7f273e6ab700 1 --2- 172.21.15.28:0/1489070888 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27380b1630 0x7f27380cf5f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.997+0000 7f273e6ab700 1 -- 172.21.15.28:0/1489070888 >> 172.21.15.28:0/1489070888 conn(0x7f27380cc630 msgr2=0x7f27380cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.997+0000 7f273e6ab700 1 -- 172.21.15.28:0/1489070888 shutdown_connections 2023-11-28T11:09:28.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.997+0000 7f273e6ab700 1 -- 172.21.15.28:0/1489070888 wait complete. 2023-11-28T11:09:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.998+0000 7f273e6ab700 1 Processor -- start 2023-11-28T11:09:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.999+0000 7f273e6ab700 1 -- start start 2023-11-28T11:09:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.999+0000 7f273e6ab700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f273806afc0 0x7f2738156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.999+0000 7f273e6ab700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27380b1630 0x7f27380c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.999+0000 7f273e6ab700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27380b1a00 0x7f27380b44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.999+0000 7f273e6ab700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2738157120 con 0x7f273806afc0 2023-11-28T11:09:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.999+0000 7f273e6ab700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27380c50e0 con 0x7f27380b1630 2023-11-28T11:09:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:27.999+0000 7f273e6ab700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27381575f0 con 0x7f27380b1a00 2023-11-28T11:09:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f2737fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f273806afc0 0x7f2738156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f2737fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f273806afc0 0x7f2738156be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47816/0 (socket says 172.21.15.28:47816) 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f2737fff700 1 -- 172.21.15.28:0/3885328309 learned_addr learned my addr 172.21.15.28:0/3885328309 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f27377fe700 1 --2- 172.21.15.28:0/3885328309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27380b1630 0x7f27380c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f273cc48700 1 --2- 172.21.15.28:0/3885328309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27380b1a00 0x7f27380b44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f2737fff700 1 -- 172.21.15.28:0/3885328309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27380b1a00 msgr2=0x7f27380b44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f2737fff700 1 --2- 172.21.15.28:0/3885328309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27380b1a00 0x7f27380b44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f2737fff700 1 -- 172.21.15.28:0/3885328309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27380b1630 msgr2=0x7f27380c7550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f2737fff700 1 --2- 172.21.15.28:0/3885328309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27380b1630 0x7f27380c7550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f2737fff700 1 -- 172.21.15.28:0/3885328309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2730009a70 con 0x7f273806afc0 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f273cc48700 1 --2- 172.21.15.28:0/3885328309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27380b1a00 0x7f27380b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f2737fff700 1 --2- 172.21.15.28:0/3885328309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f273806afc0 0x7f2738156be0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2728009c40 tx=0x7f2728004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f2734ff9700 1 -- 172.21.15.28:0/3885328309 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2728004380 con 0x7f273806afc0 2023-11-28T11:09:28.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f2734ff9700 1 -- 172.21.15.28:0/3885328309 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f272801f070 con 0x7f273806afc0 2023-11-28T11:09:28.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f273e6ab700 1 -- 172.21.15.28:0/3885328309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2728009550 con 0x7f273806afc0 2023-11-28T11:09:28.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f2734ff9700 1 -- 172.21.15.28:0/3885328309 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27280036a0 con 0x7f273806afc0 2023-11-28T11:09:28.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.000+0000 7f273e6ab700 1 -- 172.21.15.28:0/3885328309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27380b1e70 con 0x7f273806afc0 2023-11-28T11:09:28.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.001+0000 7f273e6ab700 1 -- 172.21.15.28:0/3885328309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27380cf0d0 con 0x7f273806afc0 2023-11-28T11:09:28.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.002+0000 7f2734ff9700 1 -- 172.21.15.28:0/3885328309 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f272802a020 con 0x7f273806afc0 2023-11-28T11:09:28.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.002+0000 7f2734ff9700 1 --2- 172.21.15.28:0/3885328309 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f27240682f0 0x7f272406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.002+0000 7f2734ff9700 1 -- 172.21.15.28:0/3885328309 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2728087530 con 0x7f273806afc0 2023-11-28T11:09:28.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.002+0000 7f27377fe700 1 --2- 172.21.15.28:0/3885328309 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f27240682f0 0x7f272406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.002+0000 7f27377fe700 1 --2- 172.21.15.28:0/3885328309 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f27240682f0 0x7f272406a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f27380b58c0 tx=0x7f272c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.006+0000 7f2734ff9700 1 -- 172.21.15.28:0/3885328309 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f27280539c0 con 0x7f273806afc0 2023-11-28T11:09:28.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.199+0000 7f273e6ab700 1 -- 172.21.15.28:0/3885328309 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f27380cb5b0 con 0x7f273806afc0 2023-11-28T11:09:28.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.199+0000 7f2734ff9700 1 -- 172.21.15.28:0/3885328309 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2728056fe0 con 0x7f273806afc0 2023-11-28T11:09:28.201 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:09:28.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.201+0000 7f273e6ab700 1 -- 172.21.15.28:0/3885328309 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f27240682f0 msgr2=0x7f272406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.201+0000 7f273e6ab700 1 --2- 172.21.15.28:0/3885328309 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f27240682f0 0x7f272406a7a0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f27380b58c0 tx=0x7f272c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:28.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.201+0000 7f273e6ab700 1 -- 172.21.15.28:0/3885328309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f273806afc0 msgr2=0x7f2738156be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.201+0000 7f273e6ab700 1 --2- 172.21.15.28:0/3885328309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f273806afc0 0x7f2738156be0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2728009c40 tx=0x7f2728004750 comp rx=0 tx=0).stop 2023-11-28T11:09:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.201+0000 7f273e6ab700 1 -- 172.21.15.28:0/3885328309 shutdown_connections 2023-11-28T11:09:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.201+0000 7f273e6ab700 1 --2- 172.21.15.28:0/3885328309 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f27240682f0 0x7f272406a7a0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.201+0000 7f273e6ab700 1 --2- 172.21.15.28:0/3885328309 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f273806afc0 0x7f2738156be0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.201+0000 7f273e6ab700 1 --2- 172.21.15.28:0/3885328309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f27380b1a00 0x7f27380b44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.201+0000 7f273e6ab700 1 --2- 172.21.15.28:0/3885328309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27380b1630 0x7f27380c7550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.201+0000 7f273e6ab700 1 -- 172.21.15.28:0/3885328309 >> 172.21.15.28:0/3885328309 conn(0x7f27380cc630 msgr2=0x7f27380ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:28.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.201+0000 7f273e6ab700 1 -- 172.21.15.28:0/3885328309 shutdown_connections 2023-11-28T11:09:28.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.201+0000 7f273e6ab700 1 -- 172.21.15.28:0/3885328309 wait complete. 2023-11-28T11:09:28.215 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:28.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.341+0000 7f6745e34700 1 Processor -- start 2023-11-28T11:09:28.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.341+0000 7f6745e34700 1 -- start start 2023-11-28T11:09:28.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.342+0000 7f6745e34700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6740159500 0x7f67400cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.342+0000 7f6745e34700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67400b1d40 0x7f67400ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.342+0000 7f6745e34700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f674015e040 con 0x7f674015dbf0 2023-11-28T11:09:28.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.342+0000 7f6745e34700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67400cede0 con 0x7f6740159500 2023-11-28T11:09:28.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.342+0000 7f6745e34700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67400ceff0 con 0x7f67400b1d40 2023-11-28T11:09:28.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.342+0000 7f673effd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f674015dbf0 0x7f67400b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35882/0 (socket says 172.21.15.28:35882) 2023-11-28T11:09:28.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.342+0000 7f673effd700 1 -- 172.21.15.28:0/1402342671 learned_addr learned my addr 172.21.15.28:0/1402342671 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:28.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673f7fe700 1 --2- 172.21.15.28:0/1402342671 >> v2:172.21.15.59:3300/0 conn(0x7f6740159500 0x7f67400cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673ffff700 1 --2- 172.21.15.28:0/1402342671 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67400b1d40 0x7f67400ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 -- 172.21.15.28:0/1402342671 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2251146383 0 0) 0x7f674015e040 con 0x7f674015dbf0 2023-11-28T11:09:28.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 -- 172.21.15.28:0/1402342671 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f672c0035b0 con 0x7f674015dbf0 2023-11-28T11:09:28.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 -- 172.21.15.28:0/1402342671 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1977855505 0 0) 0x7f672c0035b0 con 0x7f674015dbf0 2023-11-28T11:09:28.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 -- 172.21.15.28:0/1402342671 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67400b1d40 msgr2=0x7f67400ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:28.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 --2- 172.21.15.28:0/1402342671 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67400b1d40 0x7f67400ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 -- 172.21.15.28:0/1402342671 >> v2:172.21.15.59:3300/0 conn(0x7f6740159500 msgr2=0x7f67400cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:28.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 --2- 172.21.15.28:0/1402342671 >> v2:172.21.15.59:3300/0 conn(0x7f6740159500 0x7f67400cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 -- 172.21.15.28:0/1402342671 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67400cf2d0 con 0x7f674015dbf0 2023-11-28T11:09:28.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 -- 172.21.15.28:0/1402342671 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f6728001670 con 0x7f674015dbf0 2023-11-28T11:09:28.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 -- 172.21.15.28:0/1402342671 >> v1:172.21.15.28:6789/0 conn(0x7f674015dbf0 legacy=0x7f67400b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 --2- 172.21.15.28:0/1402342671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f672c003a00 0x7f672c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 --2- 172.21.15.28:0/1402342671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f672c004430 0x7f672c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 --2- 172.21.15.28:0/1402342671 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67400b1d40 0x7f672c004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 -- 172.21.15.28:0/1402342671 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67400cede0 con 0x7f672c004430 2023-11-28T11:09:28.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 -- 172.21.15.28:0/1402342671 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67400ceff0 con 0x7f672c003a00 2023-11-28T11:09:28.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673e7fc700 1 -- 172.21.15.28:0/1402342671 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672c00ba10 con 0x7f67400b1d40 2023-11-28T11:09:28.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673effd700 1 --2- 172.21.15.28:0/1402342671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f672c004430 0x7f672c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673f7fe700 1 --2- 172.21.15.28:0/1402342671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f672c003a00 0x7f672c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673ffff700 1 --2- 172.21.15.28:0/1402342671 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67400b1d40 0x7f672c004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673effd700 1 -- 172.21.15.28:0/1402342671 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67400b1d40 msgr2=0x7f672c004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:28.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673effd700 1 --2- 172.21.15.28:0/1402342671 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f67400b1d40 0x7f672c004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673effd700 1 -- 172.21.15.28:0/1402342671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f672c003a00 msgr2=0x7f672c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:28.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673effd700 1 --2- 172.21.15.28:0/1402342671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f672c003a00 0x7f672c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.343+0000 7f673effd700 1 -- 172.21.15.28:0/1402342671 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f672c0035b0 con 0x7f672c004430 2023-11-28T11:09:28.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.344+0000 7f673effd700 1 --2- 172.21.15.28:0/1402342671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f672c004430 0x7f672c0048f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f67400cd680 tx=0x7f672c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e23da2f0f3a1e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.344+0000 7f673e7fc700 1 -- 172.21.15.28:0/1402342671 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672800ce20 con 0x7f672c004430 2023-11-28T11:09:28.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.344+0000 7f673e7fc700 1 -- 172.21.15.28:0/1402342671 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6728011380 con 0x7f672c004430 2023-11-28T11:09:28.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.344+0000 7f673e7fc700 1 -- 172.21.15.28:0/1402342671 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728005ba0 con 0x7f672c004430 2023-11-28T11:09:28.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.344+0000 7f6745e34700 1 -- 172.21.15.28:0/1402342671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f672c004430 msgr2=0x7f672c0048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.344+0000 7f6745e34700 1 --2- 172.21.15.28:0/1402342671 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f672c004430 0x7f672c0048f0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f67400cd680 tx=0x7f672c005880 comp rx=0 tx=0).stop 2023-11-28T11:09:28.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.344+0000 7f673ffff700 1 -- 172.21.15.28:0/1402342671 reap_dead start 2023-11-28T11:09:28.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.344+0000 7f6745e34700 1 -- 172.21.15.28:0/1402342671 shutdown_connections 2023-11-28T11:09:28.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.344+0000 7f6745e34700 1 -- 172.21.15.28:0/1402342671 >> 172.21.15.28:0/1402342671 conn(0x7f67400cc630 msgr2=0x7f67400cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:28.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.344+0000 7f6745e34700 1 -- 172.21.15.28:0/1402342671 shutdown_connections 2023-11-28T11:09:28.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.344+0000 7f6745e34700 1 -- 172.21.15.28:0/1402342671 wait complete. 2023-11-28T11:09:28.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.346+0000 7f6745e34700 1 Processor -- start 2023-11-28T11:09:28.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.346+0000 7f6745e34700 1 -- start start 2023-11-28T11:09:28.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f6745e34700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f672c004430 0x7f6740141bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f6745e34700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67400c54f0 0x7f67400c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f6745e34700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67400c9af0 0x7f674006be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f6745e34700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67400c5960 con 0x7f67400c54f0 2023-11-28T11:09:28.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f6745e34700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6740159970 con 0x7f67400c9af0 2023-11-28T11:09:28.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f6745e34700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6740159c50 con 0x7f672c004430 2023-11-28T11:09:28.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f673effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67400c54f0 0x7f67400c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f673effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67400c54f0 0x7f67400c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47838/0 (socket says 172.21.15.28:47838) 2023-11-28T11:09:28.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f673effd700 1 -- 172.21.15.28:0/590839734 learned_addr learned my addr 172.21.15.28:0/590839734 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:28.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f673f7fe700 1 --2- 172.21.15.28:0/590839734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f672c004430 0x7f6740141bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f673ffff700 1 --2- 172.21.15.28:0/590839734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67400c9af0 0x7f674006be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f673effd700 1 -- 172.21.15.28:0/590839734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f672c004430 msgr2=0x7f6740141bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f673effd700 1 --2- 172.21.15.28:0/590839734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f672c004430 0x7f6740141bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f673effd700 1 -- 172.21.15.28:0/590839734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67400c9af0 msgr2=0x7f674006be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f673effd700 1 --2- 172.21.15.28:0/590839734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67400c9af0 0x7f674006be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f673effd700 1 -- 172.21.15.28:0/590839734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674015d3f0 con 0x7f67400c54f0 2023-11-28T11:09:28.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f673effd700 1 --2- 172.21.15.28:0/590839734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67400c54f0 0x7f67400c95b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6728011530 tx=0x7f6728000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f6737fff700 1 -- 172.21.15.28:0/590839734 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67280072d0 con 0x7f67400c54f0 2023-11-28T11:09:28.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f6737fff700 1 -- 172.21.15.28:0/590839734 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6728007450 con 0x7f67400c54f0 2023-11-28T11:09:28.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f6745e34700 1 -- 172.21.15.28:0/590839734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f674015d110 con 0x7f67400c54f0 2023-11-28T11:09:28.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.347+0000 7f6737fff700 1 -- 172.21.15.28:0/590839734 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6728007800 con 0x7f67400c54f0 2023-11-28T11:09:28.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.348+0000 7f6745e34700 1 -- 172.21.15.28:0/590839734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f674015d6d0 con 0x7f67400c54f0 2023-11-28T11:09:28.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.349+0000 7f6745e34700 1 -- 172.21.15.28:0/590839734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674015a860 con 0x7f67400c54f0 2023-11-28T11:09:28.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.349+0000 7f6737fff700 1 -- 172.21.15.28:0/590839734 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6728019d70 con 0x7f67400c54f0 2023-11-28T11:09:28.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.349+0000 7f6737fff700 1 --2- 172.21.15.28:0/590839734 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f67200682f0 0x7f672006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.349+0000 7f6737fff700 1 -- 172.21.15.28:0/590839734 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f672808fa90 con 0x7f67400c54f0 2023-11-28T11:09:28.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.349+0000 7f673f7fe700 1 --2- 172.21.15.28:0/590839734 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f67200682f0 0x7f672006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.350+0000 7f673f7fe700 1 --2- 172.21.15.28:0/590839734 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f67200682f0 0x7f672006a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6730009c60 tx=0x7f6730009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.354+0000 7f6737fff700 1 -- 172.21.15.28:0/590839734 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f672805c000 con 0x7f67400c54f0 2023-11-28T11:09:28.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.546+0000 7f6745e34700 1 -- 172.21.15.28:0/590839734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f67400739f0 con 0x7f67400c54f0 2023-11-28T11:09:28.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.547+0000 7f6737fff700 1 -- 172.21.15.28:0/590839734 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6728007dc0 con 0x7f67400c54f0 2023-11-28T11:09:28.548 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:28.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.549+0000 7f6745e34700 1 -- 172.21.15.28:0/590839734 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f67200682f0 msgr2=0x7f672006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.549+0000 7f6745e34700 1 --2- 172.21.15.28:0/590839734 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f67200682f0 0x7f672006a7a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6730009c60 tx=0x7f6730009210 comp rx=0 tx=0).stop 2023-11-28T11:09:28.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.549+0000 7f6745e34700 1 -- 172.21.15.28:0/590839734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67400c54f0 msgr2=0x7f67400c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.549+0000 7f6745e34700 1 --2- 172.21.15.28:0/590839734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67400c54f0 0x7f67400c95b0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6728011530 tx=0x7f6728000fe0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.549+0000 7f6745e34700 1 -- 172.21.15.28:0/590839734 shutdown_connections 2023-11-28T11:09:28.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.549+0000 7f6745e34700 1 --2- 172.21.15.28:0/590839734 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f67200682f0 0x7f672006a7a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.549+0000 7f6745e34700 1 --2- 172.21.15.28:0/590839734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67400c9af0 0x7f674006be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.549+0000 7f6745e34700 1 --2- 172.21.15.28:0/590839734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f672c004430 0x7f6740141bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.549+0000 7f6745e34700 1 --2- 172.21.15.28:0/590839734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67400c54f0 0x7f67400c95b0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.549+0000 7f6745e34700 1 -- 172.21.15.28:0/590839734 >> 172.21.15.28:0/590839734 conn(0x7f67400cc630 msgr2=0x7f67400af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:28.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.549+0000 7f6745e34700 1 -- 172.21.15.28:0/590839734 shutdown_connections 2023-11-28T11:09:28.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.549+0000 7f6745e34700 1 -- 172.21.15.28:0/590839734 wait complete. 2023-11-28T11:09:28.562 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:28.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.687+0000 7f1f95fb3700 1 Processor -- start 2023-11-28T11:09:28.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.687+0000 7f1f95fb3700 1 -- start start 2023-11-28T11:09:28.691 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.689+0000 7f1f95fb3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1f900b62d0 0x7f1f900cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.689+0000 7f1f95fb3700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1f900b2d90 0x7f1f900ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.689+0000 7f1f95fb3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f900cd2a0 con 0x7f1f9006f8b0 2023-11-28T11:09:28.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.689+0000 7f1f95fb3700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900d0400 con 0x7f1f900b2d90 2023-11-28T11:09:28.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.689+0000 7f1f95fb3700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f900d06e0 con 0x7f1f900b62d0 2023-11-28T11:09:28.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.689+0000 7f1f8ffff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1f900b2d90 0x7f1f900ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.689+0000 7f1f8ffff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1f900b2d90 0x7f1f900ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47848/0 (socket says 172.21.15.28:47848) 2023-11-28T11:09:28.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.689+0000 7f1f8ffff700 1 -- 172.21.15.28:0/2837398951 learned_addr learned my addr 172.21.15.28:0/2837398951 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:28.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.689+0000 7f1f8f7fe700 1 --2- 172.21.15.28:0/2837398951 >> v2:172.21.15.59:3300/0 conn(0x7f1f900b62d0 0x7f1f900cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.689+0000 7f1f8ffff700 1 -- 172.21.15.28:0/2837398951 >> v2:172.21.15.59:3300/0 conn(0x7f1f900b62d0 msgr2=0x7f1f900cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:28.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.689+0000 7f1f8ffff700 1 --2- 172.21.15.28:0/2837398951 >> v2:172.21.15.59:3300/0 conn(0x7f1f900b62d0 0x7f1f900cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.689+0000 7f1f8ffff700 1 -- 172.21.15.28:0/2837398951 >> v1:172.21.15.59:6789/0 conn(0x7f1f9006f8b0 legacy=0x7f1f900b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.689+0000 7f1f8ffff700 1 -- 172.21.15.28:0/2837398951 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f900cffb0 con 0x7f1f900b2d90 2023-11-28T11:09:28.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.689+0000 7f1f8ffff700 1 --2- 172.21.15.28:0/2837398951 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f900b2d90 0x7f1f900ce7d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1f900720d0 tx=0x7f1f8800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9ccae1db2471e92 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.690+0000 7f1f8e7fc700 1 -- 172.21.15.28:0/2837398951 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8800e070 con 0x7f1f900b2d90 2023-11-28T11:09:28.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.690+0000 7f1f8e7fc700 1 -- 172.21.15.28:0/2837398951 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f8800b9c0 con 0x7f1f900b2d90 2023-11-28T11:09:28.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.690+0000 7f1f8e7fc700 1 -- 172.21.15.28:0/2837398951 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f8800bd70 con 0x7f1f900b2d90 2023-11-28T11:09:28.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.690+0000 7f1f95fb3700 1 -- 172.21.15.28:0/2837398951 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f900b2d90 msgr2=0x7f1f900ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.690+0000 7f1f95fb3700 1 --2- 172.21.15.28:0/2837398951 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f900b2d90 0x7f1f900ce7d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f1f900720d0 tx=0x7f1f8800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:28.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.690+0000 7f1f95fb3700 1 -- 172.21.15.28:0/2837398951 shutdown_connections 2023-11-28T11:09:28.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.690+0000 7f1f95fb3700 1 --2- 172.21.15.28:0/2837398951 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f900b2d90 0x7f1f900ce7d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.690+0000 7f1f95fb3700 1 --2- 172.21.15.28:0/2837398951 >> v2:172.21.15.59:3300/0 conn(0x7f1f900b62d0 0x7f1f900cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.690+0000 7f1f95fb3700 1 -- 172.21.15.28:0/2837398951 >> 172.21.15.28:0/2837398951 conn(0x7f1f900cd680 msgr2=0x7f1f900cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:28.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.690+0000 7f1f95fb3700 1 -- 172.21.15.28:0/2837398951 shutdown_connections 2023-11-28T11:09:28.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.690+0000 7f1f95fb3700 1 -- 172.21.15.28:0/2837398951 wait complete. 2023-11-28T11:09:28.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.691+0000 7f1f95fb3700 1 Processor -- start 2023-11-28T11:09:28.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.692+0000 7f1f95fb3700 1 -- start start 2023-11-28T11:09:28.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.692+0000 7f1f95fb3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f9006f8b0 0x7f1f900c6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.692+0000 7f1f95fb3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f900b2d90 0x7f1f90157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.692+0000 7f1f95fb3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f900b62d0 0x7f1f90158170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.692+0000 7f1f95fb3700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f901f34d0 con 0x7f1f900b2d90 2023-11-28T11:09:28.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.692+0000 7f1f95fb3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f901f4010 con 0x7f1f9006f8b0 2023-11-28T11:09:28.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.692+0000 7f1f95fb3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f901f36e0 con 0x7f1f900b62d0 2023-11-28T11:09:28.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.692+0000 7f1f8effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f900b2d90 0x7f1f90157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.692+0000 7f1f8effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f900b2d90 0x7f1f90157c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47850/0 (socket says 172.21.15.28:47850) 2023-11-28T11:09:28.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.692+0000 7f1f8effd700 1 -- 172.21.15.28:0/3319815610 learned_addr learned my addr 172.21.15.28:0/3319815610 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:28.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.693+0000 7f1f8f7fe700 1 --2- 172.21.15.28:0/3319815610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f9006f8b0 0x7f1f900c6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.693+0000 7f1f8ffff700 1 --2- 172.21.15.28:0/3319815610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f900b62d0 0x7f1f90158170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.693+0000 7f1f8effd700 1 -- 172.21.15.28:0/3319815610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f900b62d0 msgr2=0x7f1f90158170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.693+0000 7f1f8effd700 1 --2- 172.21.15.28:0/3319815610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f900b62d0 0x7f1f90158170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.693+0000 7f1f8effd700 1 -- 172.21.15.28:0/3319815610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f9006f8b0 msgr2=0x7f1f900c6ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.693+0000 7f1f8effd700 1 --2- 172.21.15.28:0/3319815610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f9006f8b0 0x7f1f900c6ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.693+0000 7f1f8effd700 1 -- 172.21.15.28:0/3319815610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f88009b30 con 0x7f1f900b2d90 2023-11-28T11:09:28.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.693+0000 7f1f8ffff700 1 --2- 172.21.15.28:0/3319815610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f900b62d0 0x7f1f90158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:28.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.693+0000 7f1f8effd700 1 --2- 172.21.15.28:0/3319815610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f900b2d90 0x7f1f90157c30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000c450 tx=0x7f1f8000c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.693+0000 7f1f87fff700 1 -- 172.21.15.28:0/3319815610 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f800052d0 con 0x7f1f900b2d90 2023-11-28T11:09:28.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.693+0000 7f1f87fff700 1 -- 172.21.15.28:0/3319815610 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f80005430 con 0x7f1f900b2d90 2023-11-28T11:09:28.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.693+0000 7f1f95fb3700 1 -- 172.21.15.28:0/3319815610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f900b66a0 con 0x7f1f900b2d90 2023-11-28T11:09:28.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.693+0000 7f1f87fff700 1 -- 172.21.15.28:0/3319815610 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f800038c0 con 0x7f1f900b2d90 2023-11-28T11:09:28.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.693+0000 7f1f95fb3700 1 -- 172.21.15.28:0/3319815610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9006fd30 con 0x7f1f900b2d90 2023-11-28T11:09:28.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.694+0000 7f1f95fb3700 1 -- 172.21.15.28:0/3319815610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f900cce50 con 0x7f1f900b2d90 2023-11-28T11:09:28.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.694+0000 7f1f87fff700 1 -- 172.21.15.28:0/3319815610 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1f80019030 con 0x7f1f900b2d90 2023-11-28T11:09:28.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.695+0000 7f1f87fff700 1 --2- 172.21.15.28:0/3319815610 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f700682f0 0x7f1f7006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:28.699 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.695+0000 7f1f87fff700 1 -- 172.21.15.28:0/3319815610 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f1f80085230 con 0x7f1f900b2d90 2023-11-28T11:09:28.699 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.695+0000 7f1f8f7fe700 1 --2- 172.21.15.28:0/3319815610 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f700682f0 0x7f1f7006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:28.699 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.695+0000 7f1f8f7fe700 1 --2- 172.21.15.28:0/3319815610 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f700682f0 0x7f1f7006a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009930 tx=0x7f1f78008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:28.700 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.699+0000 7f1f87fff700 1 -- 172.21.15.28:0/3319815610 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1f800517a0 con 0x7f1f900b2d90 2023-11-28T11:09:28.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.889+0000 7f1f95fb3700 1 -- 172.21.15.28:0/3319815610 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1f901f4180 con 0x7f1f900b2d90 2023-11-28T11:09:28.891 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.889+0000 7f1f87fff700 1 -- 172.21.15.28:0/3319815610 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1f80054dc0 con 0x7f1f900b2d90 2023-11-28T11:09:28.891 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:09:28.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.891+0000 7f1f95fb3700 1 -- 172.21.15.28:0/3319815610 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f700682f0 msgr2=0x7f1f7006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.891+0000 7f1f95fb3700 1 --2- 172.21.15.28:0/3319815610 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f700682f0 0x7f1f7006a7a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009930 tx=0x7f1f78008040 comp rx=0 tx=0).stop 2023-11-28T11:09:28.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.891+0000 7f1f95fb3700 1 -- 172.21.15.28:0/3319815610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f900b2d90 msgr2=0x7f1f90157c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:28.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.891+0000 7f1f95fb3700 1 --2- 172.21.15.28:0/3319815610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f900b2d90 0x7f1f90157c30 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000c450 tx=0x7f1f8000c810 comp rx=0 tx=0).stop 2023-11-28T11:09:28.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.891+0000 7f1f95fb3700 1 -- 172.21.15.28:0/3319815610 shutdown_connections 2023-11-28T11:09:28.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.891+0000 7f1f95fb3700 1 --2- 172.21.15.28:0/3319815610 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f700682f0 0x7f1f7006a7a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.891+0000 7f1f95fb3700 1 --2- 172.21.15.28:0/3319815610 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f900b62d0 0x7f1f90158170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.891+0000 7f1f95fb3700 1 --2- 172.21.15.28:0/3319815610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f9006f8b0 0x7f1f900c6ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.891+0000 7f1f95fb3700 1 --2- 172.21.15.28:0/3319815610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f900b2d90 0x7f1f90157c30 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:28.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.891+0000 7f1f95fb3700 1 -- 172.21.15.28:0/3319815610 >> 172.21.15.28:0/3319815610 conn(0x7f1f900cd680 msgr2=0x7f1f900b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:28.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.891+0000 7f1f95fb3700 1 -- 172.21.15.28:0/3319815610 shutdown_connections 2023-11-28T11:09:28.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:28.892+0000 7f1f95fb3700 1 -- 172.21.15.28:0/3319815610 wait complete. 2023-11-28T11:09:28.905 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:29.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.031+0000 7f78b84d0700 1 Processor -- start 2023-11-28T11:09:29.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.031+0000 7f78b84d0700 1 -- start start 2023-11-28T11:09:29.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.032+0000 7f78b84d0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f78b01589c0 0x7f78b00ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.032+0000 7f78b84d0700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f78b0158d90 0x7f78b006aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.032+0000 7f78b84d0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f78b006a1e0 0x7f78b006e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.032+0000 7f78b84d0700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b006e470 con 0x7f78b0158d90 2023-11-28T11:09:29.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.032+0000 7f78b84d0700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78b006de60 con 0x7f78b006a1e0 2023-11-28T11:09:29.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.032+0000 7f78b84d0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00ceb40 con 0x7f78b01589c0 2023-11-28T11:09:29.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.032+0000 7f78b5a6b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f78b0158d90 0x7f78b006aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.032+0000 7f78b5a6b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f78b0158d90 0x7f78b006aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47858/0 (socket says 172.21.15.28:47858) 2023-11-28T11:09:29.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.032+0000 7f78b5a6b700 1 -- 172.21.15.28:0/3468420839 learned_addr learned my addr 172.21.15.28:0/3468420839 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:29.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.033+0000 7f78b626c700 1 --2- 172.21.15.28:0/3468420839 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f78b01589c0 0x7f78b00ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.033+0000 7f78b6a6d700 1 --2- 172.21.15.28:0/3468420839 >> v2:172.21.15.59:3300/0 conn(0x7f78b006a1e0 0x7f78b006e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.033+0000 7f78b5a6b700 1 -- 172.21.15.28:0/3468420839 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f78b01589c0 msgr2=0x7f78b00ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:29.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.033+0000 7f78b5a6b700 1 --2- 172.21.15.28:0/3468420839 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f78b01589c0 0x7f78b00ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.033+0000 7f78b5a6b700 1 -- 172.21.15.28:0/3468420839 >> v2:172.21.15.59:3300/0 conn(0x7f78b006a1e0 msgr2=0x7f78b006e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:29.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.033+0000 7f78b5a6b700 1 --2- 172.21.15.28:0/3468420839 >> v2:172.21.15.59:3300/0 conn(0x7f78b006a1e0 0x7f78b006e900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.033+0000 7f78b5a6b700 1 -- 172.21.15.28:0/3468420839 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78b00cee20 con 0x7f78b0158d90 2023-11-28T11:09:29.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.033+0000 7f78b626c700 1 --2- 172.21.15.28:0/3468420839 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f78b01589c0 0x7f78b00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:29.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.033+0000 7f78b6a6d700 1 --2- 172.21.15.28:0/3468420839 >> v2:172.21.15.59:3300/0 conn(0x7f78b006a1e0 0x7f78b006e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:29.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.033+0000 7f78b5a6b700 1 --2- 172.21.15.28:0/3468420839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78b0158d90 0x7f78b006aa70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f78b00cd6a0 tx=0x7f78ac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65d70cc3fbb3ad5d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.033+0000 7f78b526a700 1 -- 172.21.15.28:0/3468420839 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00b870 con 0x7f78b0158d90 2023-11-28T11:09:29.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.033+0000 7f78b526a700 1 -- 172.21.15.28:0/3468420839 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ac00b9d0 con 0x7f78b0158d90 2023-11-28T11:09:29.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.033+0000 7f78b526a700 1 -- 172.21.15.28:0/3468420839 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac00bd60 con 0x7f78b0158d90 2023-11-28T11:09:29.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.034+0000 7f78b84d0700 1 -- 172.21.15.28:0/3468420839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78b0158d90 msgr2=0x7f78b006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.034+0000 7f78b84d0700 1 --2- 172.21.15.28:0/3468420839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78b0158d90 0x7f78b006aa70 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f78b00cd6a0 tx=0x7f78ac009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.034+0000 7f78b84d0700 1 -- 172.21.15.28:0/3468420839 shutdown_connections 2023-11-28T11:09:29.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.034+0000 7f78b84d0700 1 --2- 172.21.15.28:0/3468420839 >> v2:172.21.15.59:3300/0 conn(0x7f78b006a1e0 0x7f78b006e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.034+0000 7f78b84d0700 1 --2- 172.21.15.28:0/3468420839 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f78b01589c0 0x7f78b00ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.034+0000 7f78b84d0700 1 --2- 172.21.15.28:0/3468420839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78b0158d90 0x7f78b006aa70 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.034+0000 7f78b84d0700 1 -- 172.21.15.28:0/3468420839 >> 172.21.15.28:0/3468420839 conn(0x7f78b00b0df0 msgr2=0x7f78b00b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:29.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.034+0000 7f78b84d0700 1 -- 172.21.15.28:0/3468420839 shutdown_connections 2023-11-28T11:09:29.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.034+0000 7f78b84d0700 1 -- 172.21.15.28:0/3468420839 wait complete. 2023-11-28T11:09:29.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.035+0000 7f78b84d0700 1 Processor -- start 2023-11-28T11:09:29.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.035+0000 7f78b84d0700 1 -- start start 2023-11-28T11:09:29.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b84d0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f78b006a1e0 0x7f78b0152ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b84d0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78b01589c0 0x7f78b00c54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b84d0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78b0158d90 0x7f78b00b2ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b84d0700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00c61b0 con 0x7f78b01589c0 2023-11-28T11:09:29.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b84d0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00c5ed0 con 0x7f78b0158d90 2023-11-28T11:09:29.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b84d0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78b00b2510 con 0x7f78b006a1e0 2023-11-28T11:09:29.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b5a6b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78b01589c0 0x7f78b00c54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b5a6b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78b01589c0 0x7f78b00c54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47862/0 (socket says 172.21.15.28:47862) 2023-11-28T11:09:29.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b5a6b700 1 -- 172.21.15.28:0/57832892 learned_addr learned my addr 172.21.15.28:0/57832892 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:29.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b626c700 1 --2- 172.21.15.28:0/57832892 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f78b006a1e0 0x7f78b0152ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b6a6d700 1 --2- 172.21.15.28:0/57832892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78b0158d90 0x7f78b00b2ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b5a6b700 1 -- 172.21.15.28:0/57832892 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f78b006a1e0 msgr2=0x7f78b0152ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b5a6b700 1 --2- 172.21.15.28:0/57832892 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f78b006a1e0 0x7f78b0152ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b5a6b700 1 -- 172.21.15.28:0/57832892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78b0158d90 msgr2=0x7f78b00b2ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b5a6b700 1 --2- 172.21.15.28:0/57832892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78b0158d90 0x7f78b00b2ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b5a6b700 1 -- 172.21.15.28:0/57832892 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78ac009550 con 0x7f78b01589c0 2023-11-28T11:09:29.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b6a6d700 1 --2- 172.21.15.28:0/57832892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78b0158d90 0x7f78b00b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:29.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.036+0000 7f78b5a6b700 1 --2- 172.21.15.28:0/57832892 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78b01589c0 0x7f78b00c54e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f78ac014040 tx=0x7f78ac004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.037+0000 7f78a6ffd700 1 -- 172.21.15.28:0/57832892 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac004520 con 0x7f78b01589c0 2023-11-28T11:09:29.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.037+0000 7f78a6ffd700 1 -- 172.21.15.28:0/57832892 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78ac01e070 con 0x7f78b01589c0 2023-11-28T11:09:29.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.037+0000 7f78a6ffd700 1 -- 172.21.15.28:0/57832892 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78ac0038e0 con 0x7f78b01589c0 2023-11-28T11:09:29.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.037+0000 7f78b84d0700 1 -- 172.21.15.28:0/57832892 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78b006afb0 con 0x7f78b01589c0 2023-11-28T11:09:29.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.037+0000 7f78b84d0700 1 -- 172.21.15.28:0/57832892 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78b006e100 con 0x7f78b01589c0 2023-11-28T11:09:29.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.037+0000 7f78b84d0700 1 -- 172.21.15.28:0/57832892 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78b00c9d90 con 0x7f78b01589c0 2023-11-28T11:09:29.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.038+0000 7f78a6ffd700 1 -- 172.21.15.28:0/57832892 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f78ac00f050 con 0x7f78b01589c0 2023-11-28T11:09:29.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.038+0000 7f78a6ffd700 1 --2- 172.21.15.28:0/57832892 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f78940682f0 0x7f789406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.038+0000 7f78a6ffd700 1 -- 172.21.15.28:0/57832892 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f78ac020070 con 0x7f78b01589c0 2023-11-28T11:09:29.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.038+0000 7f78b626c700 1 --2- 172.21.15.28:0/57832892 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f78940682f0 0x7f789406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.039+0000 7f78b626c700 1 --2- 172.21.15.28:0/57832892 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f78940682f0 0x7f789406a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f78a0006fd0 tx=0x7f78a0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.043+0000 7f78a6ffd700 1 -- 172.21.15.28:0/57832892 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f78ac053b00 con 0x7f78b01589c0 2023-11-28T11:09:29.239 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.237+0000 7f78b84d0700 1 -- 172.21.15.28:0/57832892 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f78b00739f0 con 0x7f78b01589c0 2023-11-28T11:09:29.239 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.238+0000 7f78a6ffd700 1 -- 172.21.15.28:0/57832892 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7f78ac057120 con 0x7f78b01589c0 2023-11-28T11:09:29.239 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:29.241 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.240+0000 7f78b84d0700 1 -- 172.21.15.28:0/57832892 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f78940682f0 msgr2=0x7f789406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.241 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.240+0000 7f78b84d0700 1 --2- 172.21.15.28:0/57832892 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f78940682f0 0x7f789406a7a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f78a0006fd0 tx=0x7f78a0008040 comp rx=0 tx=0).stop 2023-11-28T11:09:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.240+0000 7f78b84d0700 1 -- 172.21.15.28:0/57832892 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78b01589c0 msgr2=0x7f78b00c54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.240+0000 7f78b84d0700 1 --2- 172.21.15.28:0/57832892 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78b01589c0 0x7f78b00c54e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f78ac014040 tx=0x7f78ac004750 comp rx=0 tx=0).stop 2023-11-28T11:09:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.240+0000 7f78b84d0700 1 -- 172.21.15.28:0/57832892 shutdown_connections 2023-11-28T11:09:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.240+0000 7f78b84d0700 1 --2- 172.21.15.28:0/57832892 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f78940682f0 0x7f789406a7a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.240+0000 7f78b84d0700 1 --2- 172.21.15.28:0/57832892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78b0158d90 0x7f78b00b2ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.240+0000 7f78b84d0700 1 --2- 172.21.15.28:0/57832892 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f78b006a1e0 0x7f78b0152ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.240+0000 7f78b84d0700 1 --2- 172.21.15.28:0/57832892 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f78b01589c0 0x7f78b00c54e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.240+0000 7f78b84d0700 1 -- 172.21.15.28:0/57832892 >> 172.21.15.28:0/57832892 conn(0x7f78b00b0df0 msgr2=0x7f78b00af400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.240+0000 7f78b84d0700 1 -- 172.21.15.28:0/57832892 shutdown_connections 2023-11-28T11:09:29.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.240+0000 7f78b84d0700 1 -- 172.21.15.28:0/57832892 wait complete. 2023-11-28T11:09:29.254 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:29.378 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.377+0000 7f5b23274700 1 Processor -- start 2023-11-28T11:09:29.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.377+0000 7f5b23274700 1 -- start start 2023-11-28T11:09:29.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b23274700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b1c0cb230 0x7f5b1c0ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b23274700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5b1c0b2660 0x7f5b1c0b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b23274700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b1c0cfb90 con 0x7f5b1c15a6c0 2023-11-28T11:09:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b23274700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b1c0b3020 con 0x7f5b1c0b2660 2023-11-28T11:09:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b23274700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c0cb680 con 0x7f5b1c0cb230 2023-11-28T11:09:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b21811700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5b1c0b2660 0x7f5b1c0b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b21010700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5b1c15a6c0 0x7f5b1c0d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35928/0 (socket says 172.21.15.28:35928) 2023-11-28T11:09:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b21811700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5b1c0b2660 0x7f5b1c0b2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47874/0 (socket says 172.21.15.28:47874) 2023-11-28T11:09:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b21010700 1 -- 172.21.15.28:0/4246713351 learned_addr learned my addr 172.21.15.28:0/4246713351 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b2080f700 1 --2- 172.21.15.28:0/4246713351 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b1c0cb230 0x7f5b1c0ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b21811700 1 -- 172.21.15.28:0/4246713351 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b1c0cb230 msgr2=0x7f5b1c0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b21811700 1 --2- 172.21.15.28:0/4246713351 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b1c0cb230 0x7f5b1c0ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b21811700 1 -- 172.21.15.28:0/4246713351 >> v1:172.21.15.28:6789/0 conn(0x7f5b1c15a6c0 legacy=0x7f5b1c0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b21811700 1 -- 172.21.15.28:0/4246713351 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b1c0cb930 con 0x7f5b1c0b2660 2023-11-28T11:09:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b13fff700 1 -- 172.21.15.28:0/4246713351 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 747009479 0 0) 0x7f5b1c0cfb90 con 0x7f5b1c15a6c0 2023-11-28T11:09:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.378+0000 7f5b21811700 1 --2- 172.21.15.28:0/4246713351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b1c0b2660 0x7f5b1c0b2ae0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009d90 tx=0x7f5b1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ab9ca7c8c6b180 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.379+0000 7f5b13fff700 1 -- 172.21.15.28:0/4246713351 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b1400e070 con 0x7f5b1c0b2660 2023-11-28T11:09:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.379+0000 7f5b13fff700 1 -- 172.21.15.28:0/4246713351 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b1400bb20 con 0x7f5b1c0b2660 2023-11-28T11:09:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.379+0000 7f5b13fff700 1 -- 172.21.15.28:0/4246713351 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b14012430 con 0x7f5b1c0b2660 2023-11-28T11:09:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.379+0000 7f5b23274700 1 -- 172.21.15.28:0/4246713351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b1c0b2660 msgr2=0x7f5b1c0b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.379+0000 7f5b23274700 1 --2- 172.21.15.28:0/4246713351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b1c0b2660 0x7f5b1c0b2ae0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f5b14009d90 tx=0x7f5b1400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.379+0000 7f5b23274700 1 -- 172.21.15.28:0/4246713351 shutdown_connections 2023-11-28T11:09:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.379+0000 7f5b23274700 1 --2- 172.21.15.28:0/4246713351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b1c0b2660 0x7f5b1c0b2ae0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.379+0000 7f5b23274700 1 --2- 172.21.15.28:0/4246713351 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b1c0cb230 0x7f5b1c0ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.379+0000 7f5b23274700 1 -- 172.21.15.28:0/4246713351 >> 172.21.15.28:0/4246713351 conn(0x7f5b1c0cd660 msgr2=0x7f5b1c0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:29.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.379+0000 7f5b23274700 1 -- 172.21.15.28:0/4246713351 shutdown_connections 2023-11-28T11:09:29.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.379+0000 7f5b23274700 1 -- 172.21.15.28:0/4246713351 wait complete. 2023-11-28T11:09:29.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.380+0000 7f5b23274700 1 Processor -- start 2023-11-28T11:09:29.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.381+0000 7f5b23274700 1 -- start start 2023-11-28T11:09:29.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.381+0000 7f5b23274700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b1c0b2660 0x7f5b1c0c8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.381+0000 7f5b23274700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b1c0cb230 0x7f5b1c06d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.381+0000 7f5b23274700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b1c15a6c0 0x7f5b1c1f4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.381+0000 7f5b23274700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c06dbd0 con 0x7f5b1c0b2660 2023-11-28T11:09:29.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.381+0000 7f5b23274700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c0c9050 con 0x7f5b1c15a6c0 2023-11-28T11:09:29.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.381+0000 7f5b23274700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b1c0c8d30 con 0x7f5b1c0cb230 2023-11-28T11:09:29.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.381+0000 7f5b21010700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b1c0b2660 0x7f5b1c0c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.381+0000 7f5b21010700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b1c0b2660 0x7f5b1c0c8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47878/0 (socket says 172.21.15.28:47878) 2023-11-28T11:09:29.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.381+0000 7f5b21010700 1 -- 172.21.15.28:0/3356660779 learned_addr learned my addr 172.21.15.28:0/3356660779 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:29.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.381+0000 7f5b2080f700 1 --2- 172.21.15.28:0/3356660779 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b1c0cb230 0x7f5b1c06d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.381+0000 7f5b21811700 1 --2- 172.21.15.28:0/3356660779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b1c15a6c0 0x7f5b1c1f4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.382+0000 7f5b21010700 1 -- 172.21.15.28:0/3356660779 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b1c0cb230 msgr2=0x7f5b1c06d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.382+0000 7f5b21010700 1 --2- 172.21.15.28:0/3356660779 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b1c0cb230 0x7f5b1c06d690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.382+0000 7f5b21010700 1 -- 172.21.15.28:0/3356660779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b1c15a6c0 msgr2=0x7f5b1c1f4ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.382+0000 7f5b21010700 1 --2- 172.21.15.28:0/3356660779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b1c15a6c0 0x7f5b1c1f4ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.382+0000 7f5b21010700 1 -- 172.21.15.28:0/3356660779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b14009a70 con 0x7f5b1c0b2660 2023-11-28T11:09:29.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.382+0000 7f5b21811700 1 --2- 172.21.15.28:0/3356660779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b1c15a6c0 0x7f5b1c1f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:29.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.382+0000 7f5b21010700 1 --2- 172.21.15.28:0/3356660779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b1c0b2660 0x7f5b1c0c8680 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5b180011e0 tx=0x7f5b1800ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.382+0000 7f5b11ffb700 1 -- 172.21.15.28:0/3356660779 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18004c80 con 0x7f5b1c0b2660 2023-11-28T11:09:29.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.382+0000 7f5b11ffb700 1 -- 172.21.15.28:0/3356660779 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b18004de0 con 0x7f5b1c0b2660 2023-11-28T11:09:29.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.382+0000 7f5b23274700 1 -- 172.21.15.28:0/3356660779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b1c0cf540 con 0x7f5b1c0b2660 2023-11-28T11:09:29.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.382+0000 7f5b11ffb700 1 -- 172.21.15.28:0/3356660779 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b18007310 con 0x7f5b1c0b2660 2023-11-28T11:09:29.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.382+0000 7f5b23274700 1 -- 172.21.15.28:0/3356660779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b1c0cf260 con 0x7f5b1c0b2660 2023-11-28T11:09:29.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.383+0000 7f5b23274700 1 -- 172.21.15.28:0/3356660779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b1c0cf820 con 0x7f5b1c0b2660 2023-11-28T11:09:29.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.383+0000 7f5b11ffb700 1 -- 172.21.15.28:0/3356660779 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5b1801a020 con 0x7f5b1c0b2660 2023-11-28T11:09:29.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.384+0000 7f5b11ffb700 1 --2- 172.21.15.28:0/3356660779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5b000682f0 0x7f5b0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.384+0000 7f5b11ffb700 1 -- 172.21.15.28:0/3356660779 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f5b18085ba0 con 0x7f5b1c0b2660 2023-11-28T11:09:29.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.384+0000 7f5b2080f700 1 --2- 172.21.15.28:0/3356660779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5b000682f0 0x7f5b0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.384+0000 7f5b2080f700 1 --2- 172.21.15.28:0/3356660779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5b000682f0 0x7f5b0006a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0b51c0 tx=0x7f5b0c009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.388+0000 7f5b11ffb700 1 -- 172.21.15.28:0/3356660779 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5b18052110 con 0x7f5b1c0b2660 2023-11-28T11:09:29.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.584+0000 7f5b23274700 1 -- 172.21.15.28:0/3356660779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b1c0675a0 con 0x7f5b1c0b2660 2023-11-28T11:09:29.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.586+0000 7f5b11ffb700 1 -- 172.21.15.28:0/3356660779 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f5b18055730 con 0x7f5b1c0b2660 2023-11-28T11:09:29.587 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:29.588 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:50.531733+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:29.589 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.587+0000 7f5b23274700 1 -- 172.21.15.28:0/3356660779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5b000682f0 msgr2=0x7f5b0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.587+0000 7f5b23274700 1 --2- 172.21.15.28:0/3356660779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5b000682f0 0x7f5b0006a7a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b1c0b51c0 tx=0x7f5b0c009210 comp rx=0 tx=0).stop 2023-11-28T11:09:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.587+0000 7f5b23274700 1 -- 172.21.15.28:0/3356660779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b1c0b2660 msgr2=0x7f5b1c0c8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.587+0000 7f5b23274700 1 --2- 172.21.15.28:0/3356660779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b1c0b2660 0x7f5b1c0c8680 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f5b180011e0 tx=0x7f5b1800ca30 comp rx=0 tx=0).stop 2023-11-28T11:09:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.588+0000 7f5b23274700 1 -- 172.21.15.28:0/3356660779 shutdown_connections 2023-11-28T11:09:29.590 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.588+0000 7f5b23274700 1 --2- 172.21.15.28:0/3356660779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5b000682f0 0x7f5b0006a7a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.591 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.588+0000 7f5b23274700 1 --2- 172.21.15.28:0/3356660779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b1c0b2660 0x7f5b1c0c8680 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.591 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.588+0000 7f5b23274700 1 --2- 172.21.15.28:0/3356660779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b1c15a6c0 0x7f5b1c1f4ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.591 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.588+0000 7f5b23274700 1 --2- 172.21.15.28:0/3356660779 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b1c0cb230 0x7f5b1c06d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.591 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.588+0000 7f5b23274700 1 -- 172.21.15.28:0/3356660779 >> 172.21.15.28:0/3356660779 conn(0x7f5b1c0cd660 msgr2=0x7f5b1c0b61d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:29.591 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.588+0000 7f5b23274700 1 -- 172.21.15.28:0/3356660779 shutdown_connections 2023-11-28T11:09:29.591 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.588+0000 7f5b23274700 1 -- 172.21.15.28:0/3356660779 wait complete. 2023-11-28T11:09:29.601 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:29.602 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:29.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.728+0000 7f29189c4700 1 Processor -- start 2023-11-28T11:09:29.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.728+0000 7f29189c4700 1 -- start start 2023-11-28T11:09:29.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f29189c4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f29141589c0 0x7f29140ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f29189c4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2914158d90 0x7f291406aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f29189c4700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f291406e900 0x7f291406ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f29189c4700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291406e470 con 0x7f291406e900 2023-11-28T11:09:29.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f29189c4700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f291406de60 con 0x7f29141589c0 2023-11-28T11:09:29.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f29189c4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29140ceb40 con 0x7f2914158d90 2023-11-28T11:09:29.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f2912d9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f291406e900 0x7f291406ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f2912d9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f291406e900 0x7f291406ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47888/0 (socket says 172.21.15.28:47888) 2023-11-28T11:09:29.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f2912d9d700 1 -- 172.21.15.28:0/4067579314 learned_addr learned my addr 172.21.15.28:0/4067579314 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:29.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f2911d9b700 1 --2- 172.21.15.28:0/4067579314 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2914158d90 0x7f291406aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f291259c700 1 --2- 172.21.15.28:0/4067579314 >> v2:172.21.15.59:3300/0 conn(0x7f29141589c0 0x7f29140ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f2912d9d700 1 -- 172.21.15.28:0/4067579314 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2914158d90 msgr2=0x7f291406aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:29.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f2912d9d700 1 --2- 172.21.15.28:0/4067579314 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2914158d90 0x7f291406aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f2912d9d700 1 -- 172.21.15.28:0/4067579314 >> v2:172.21.15.59:3300/0 conn(0x7f29141589c0 msgr2=0x7f29140ce560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:29.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f2912d9d700 1 --2- 172.21.15.28:0/4067579314 >> v2:172.21.15.59:3300/0 conn(0x7f29141589c0 0x7f29140ce560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f2912d9d700 1 -- 172.21.15.28:0/4067579314 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29140cee20 con 0x7f291406e900 2023-11-28T11:09:29.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.729+0000 7f291259c700 1 --2- 172.21.15.28:0/4067579314 >> v2:172.21.15.59:3300/0 conn(0x7f29141589c0 0x7f29140ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:29.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.730+0000 7f2912d9d700 1 --2- 172.21.15.28:0/4067579314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f291406e900 0x7f291406ed80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f29140b1ef0 tx=0x7f290c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ceb8959e36276d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.730+0000 7f291159a700 1 -- 172.21.15.28:0/4067579314 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c00e070 con 0x7f291406e900 2023-11-28T11:09:29.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.730+0000 7f291159a700 1 -- 172.21.15.28:0/4067579314 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290c00b9c0 con 0x7f291406e900 2023-11-28T11:09:29.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.730+0000 7f291159a700 1 -- 172.21.15.28:0/4067579314 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290c00bd70 con 0x7f291406e900 2023-11-28T11:09:29.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.730+0000 7f29189c4700 1 -- 172.21.15.28:0/4067579314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f291406e900 msgr2=0x7f291406ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.730+0000 7f29189c4700 1 --2- 172.21.15.28:0/4067579314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f291406e900 0x7f291406ed80 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f29140b1ef0 tx=0x7f290c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:29.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.730+0000 7f29189c4700 1 -- 172.21.15.28:0/4067579314 shutdown_connections 2023-11-28T11:09:29.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.730+0000 7f29189c4700 1 --2- 172.21.15.28:0/4067579314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f291406e900 0x7f291406ed80 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.730+0000 7f29189c4700 1 --2- 172.21.15.28:0/4067579314 >> v2:172.21.15.59:3300/0 conn(0x7f29141589c0 0x7f29140ce560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.730+0000 7f29189c4700 1 --2- 172.21.15.28:0/4067579314 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2914158d90 0x7f291406aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.730+0000 7f29189c4700 1 -- 172.21.15.28:0/4067579314 >> 172.21.15.28:0/4067579314 conn(0x7f29140b0df0 msgr2=0x7f29140b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:29.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.730+0000 7f29189c4700 1 -- 172.21.15.28:0/4067579314 shutdown_connections 2023-11-28T11:09:29.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.730+0000 7f29189c4700 1 -- 172.21.15.28:0/4067579314 wait complete. 2023-11-28T11:09:29.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.732+0000 7f29189c4700 1 Processor -- start 2023-11-28T11:09:29.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.732+0000 7f29189c4700 1 -- start start 2023-11-28T11:09:29.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.732+0000 7f29189c4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f291406e900 0x7f291414e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.732+0000 7f29189c4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29141589c0 0x7f29140b45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f29189c4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2914158d90 0x7f2914163a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f29189c4700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29140b5000 con 0x7f291406e900 2023-11-28T11:09:29.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f29189c4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2914150300 con 0x7f2914158d90 2023-11-28T11:09:29.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f29189c4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f291414f9d0 con 0x7f29141589c0 2023-11-28T11:09:29.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f291259c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f291406e900 0x7f291414e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f291259c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f291406e900 0x7f291414e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47890/0 (socket says 172.21.15.28:47890) 2023-11-28T11:09:29.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f291259c700 1 -- 172.21.15.28:0/1711895982 learned_addr learned my addr 172.21.15.28:0/1711895982 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:29.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f2911d9b700 1 --2- 172.21.15.28:0/1711895982 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29141589c0 0x7f29140b45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f2912d9d700 1 --2- 172.21.15.28:0/1711895982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2914158d90 0x7f2914163a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f291259c700 1 -- 172.21.15.28:0/1711895982 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29141589c0 msgr2=0x7f29140b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f291259c700 1 --2- 172.21.15.28:0/1711895982 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29141589c0 0x7f29140b45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f291259c700 1 -- 172.21.15.28:0/1711895982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2914158d90 msgr2=0x7f2914163a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f291259c700 1 --2- 172.21.15.28:0/1711895982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2914158d90 0x7f2914163a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f291259c700 1 -- 172.21.15.28:0/1711895982 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c009b30 con 0x7f291406e900 2023-11-28T11:09:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f291259c700 1 --2- 172.21.15.28:0/1711895982 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f291406e900 0x7f291414e3a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f290800c740 tx=0x7f290800ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f2902ffd700 1 -- 172.21.15.28:0/1711895982 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908011800 con 0x7f291406e900 2023-11-28T11:09:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f2902ffd700 1 -- 172.21.15.28:0/1711895982 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2908011960 con 0x7f291406e900 2023-11-28T11:09:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f29189c4700 1 -- 172.21.15.28:0/1711895982 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f291406afb0 con 0x7f291406e900 2023-11-28T11:09:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f2902ffd700 1 -- 172.21.15.28:0/1711895982 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2908011cd0 con 0x7f291406e900 2023-11-28T11:09:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.733+0000 7f29189c4700 1 -- 172.21.15.28:0/1711895982 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f291406e100 con 0x7f291406e900 2023-11-28T11:09:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.734+0000 7f29189c4700 1 -- 172.21.15.28:0/1711895982 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29140739f0 con 0x7f291406e900 2023-11-28T11:09:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.735+0000 7f2902ffd700 1 -- 172.21.15.28:0/1711895982 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2908020070 con 0x7f291406e900 2023-11-28T11:09:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.735+0000 7f2902ffd700 1 --2- 172.21.15.28:0/1711895982 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f28f40682f0 0x7f28f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:29.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.735+0000 7f2902ffd700 1 -- 172.21.15.28:0/1711895982 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f2908085bc0 con 0x7f291406e900 2023-11-28T11:09:29.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.735+0000 7f2911d9b700 1 --2- 172.21.15.28:0/1711895982 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f28f40682f0 0x7f28f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:29.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.736+0000 7f2911d9b700 1 --2- 172.21.15.28:0/1711895982 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f28f40682f0 0x7f28f406a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2914071d90 tx=0x7f2904008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:29.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.739+0000 7f2902ffd700 1 -- 172.21.15.28:0/1711895982 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2908052130 con 0x7f291406e900 2023-11-28T11:09:29.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.932+0000 7f29189c4700 1 -- 172.21.15.28:0/1711895982 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f29140cd700 con 0x7f291406e900 2023-11-28T11:09:29.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.932+0000 7f2902ffd700 1 -- 172.21.15.28:0/1711895982 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2908055750 con 0x7f291406e900 2023-11-28T11:09:29.934 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:09:29.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.934+0000 7f29189c4700 1 -- 172.21.15.28:0/1711895982 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f28f40682f0 msgr2=0x7f28f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.934+0000 7f29189c4700 1 --2- 172.21.15.28:0/1711895982 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f28f40682f0 0x7f28f406a7a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2914071d90 tx=0x7f2904008040 comp rx=0 tx=0).stop 2023-11-28T11:09:29.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.934+0000 7f29189c4700 1 -- 172.21.15.28:0/1711895982 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f291406e900 msgr2=0x7f291414e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:29.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.934+0000 7f29189c4700 1 --2- 172.21.15.28:0/1711895982 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f291406e900 0x7f291414e3a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f290800c740 tx=0x7f290800ca50 comp rx=0 tx=0).stop 2023-11-28T11:09:29.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.934+0000 7f29189c4700 1 -- 172.21.15.28:0/1711895982 shutdown_connections 2023-11-28T11:09:29.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.934+0000 7f29189c4700 1 --2- 172.21.15.28:0/1711895982 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f28f40682f0 0x7f28f406a7a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.934+0000 7f29189c4700 1 --2- 172.21.15.28:0/1711895982 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f291406e900 0x7f291414e3a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.934+0000 7f29189c4700 1 --2- 172.21.15.28:0/1711895982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2914158d90 0x7f2914163a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.935+0000 7f29189c4700 1 --2- 172.21.15.28:0/1711895982 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f29141589c0 0x7f29140b45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:29.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.935+0000 7f29189c4700 1 -- 172.21.15.28:0/1711895982 >> 172.21.15.28:0/1711895982 conn(0x7f29140b0df0 msgr2=0x7f2914153640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:29.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.935+0000 7f29189c4700 1 -- 172.21.15.28:0/1711895982 shutdown_connections 2023-11-28T11:09:29.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:29.935+0000 7f29189c4700 1 -- 172.21.15.28:0/1711895982 wait complete. 2023-11-28T11:09:29.948 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:30.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.074+0000 7f01068e6700 1 Processor -- start 2023-11-28T11:09:30.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.074+0000 7f01068e6700 1 -- start start 2023-11-28T11:09:30.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.075+0000 7f01068e6700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0100159500 0x7f01000cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.075+0000 7f01068e6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01000b1d40 0x7f01000ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.075+0000 7f01068e6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f010015e040 con 0x7f010015dbf0 2023-11-28T11:09:30.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.075+0000 7f01068e6700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01000cede0 con 0x7f0100159500 2023-11-28T11:09:30.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.075+0000 7f01068e6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01000ceff0 con 0x7f01000b1d40 2023-11-28T11:09:30.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.075+0000 7f00fffff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0100159500 0x7f01000cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.075+0000 7f00fffff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0100159500 0x7f01000cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47898/0 (socket says 172.21.15.28:47898) 2023-11-28T11:09:30.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.075+0000 7f00fffff700 1 -- 172.21.15.28:0/1489109811 learned_addr learned my addr 172.21.15.28:0/1489109811 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:30.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.075+0000 7f0104e83700 1 --2- 172.21.15.28:0/1489109811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01000b1d40 0x7f01000ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.075+0000 7f00fffff700 1 -- 172.21.15.28:0/1489109811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01000b1d40 msgr2=0x7f01000ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.075+0000 7f00fffff700 1 --2- 172.21.15.28:0/1489109811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01000b1d40 0x7f01000ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.075+0000 7f00fffff700 1 -- 172.21.15.28:0/1489109811 >> v1:172.21.15.59:6789/0 conn(0x7f010015dbf0 legacy=0x7f01000b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.076+0000 7f00fffff700 1 -- 172.21.15.28:0/1489109811 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01000cf2d0 con 0x7f0100159500 2023-11-28T11:09:30.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.076+0000 7f0104e83700 1 --2- 172.21.15.28:0/1489109811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01000b1d40 0x7f01000ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:30.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.076+0000 7f00fffff700 1 --2- 172.21.15.28:0/1489109811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0100159500 0x7f01000cd140 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f00f00098f0 tx=0x7f00f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a259ab95d0bbcd6f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.076+0000 7f00feffd700 1 -- 172.21.15.28:0/1489109811 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0004230 con 0x7f0100159500 2023-11-28T11:09:30.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.076+0000 7f00feffd700 1 -- 172.21.15.28:0/1489109811 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f0004390 con 0x7f0100159500 2023-11-28T11:09:30.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.076+0000 7f00feffd700 1 -- 172.21.15.28:0/1489109811 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f00036a0 con 0x7f0100159500 2023-11-28T11:09:30.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.076+0000 7f01068e6700 1 -- 172.21.15.28:0/1489109811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0100159500 msgr2=0x7f01000cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.076+0000 7f01068e6700 1 --2- 172.21.15.28:0/1489109811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0100159500 0x7f01000cd140 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f00f00098f0 tx=0x7f00f0009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:30.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.076+0000 7f01068e6700 1 -- 172.21.15.28:0/1489109811 shutdown_connections 2023-11-28T11:09:30.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.076+0000 7f01068e6700 1 --2- 172.21.15.28:0/1489109811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0100159500 0x7f01000cd140 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.076+0000 7f01068e6700 1 --2- 172.21.15.28:0/1489109811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f01000b1d40 0x7f01000ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.076+0000 7f01068e6700 1 -- 172.21.15.28:0/1489109811 >> 172.21.15.28:0/1489109811 conn(0x7f01000cc630 msgr2=0x7f01000cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:30.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.076+0000 7f01068e6700 1 -- 172.21.15.28:0/1489109811 shutdown_connections 2023-11-28T11:09:30.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.077+0000 7f01068e6700 1 -- 172.21.15.28:0/1489109811 wait complete. 2023-11-28T11:09:30.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.078+0000 7f01068e6700 1 Processor -- start 2023-11-28T11:09:30.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.078+0000 7f01068e6700 1 -- start start 2023-11-28T11:09:30.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f01068e6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01000b1d40 0x7f01000b53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f01068e6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0100159500 0x7f0100154090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f01068e6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f010015dbf0 0x7f0100157fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f01068e6700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01000b4350 con 0x7f01000b1d40 2023-11-28T11:09:30.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f01068e6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01000b4630 con 0x7f010015dbf0 2023-11-28T11:09:30.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f01068e6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01000b4910 con 0x7f0100159500 2023-11-28T11:09:30.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f00fffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01000b1d40 0x7f01000b53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f00fffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01000b1d40 0x7f01000b53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47902/0 (socket says 172.21.15.28:47902) 2023-11-28T11:09:30.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f00fffff700 1 -- 172.21.15.28:0/4227078497 learned_addr learned my addr 172.21.15.28:0/4227078497 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:30.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f0104e83700 1 --2- 172.21.15.28:0/4227078497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f010015dbf0 0x7f0100157fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f00ff7fe700 1 --2- 172.21.15.28:0/4227078497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0100159500 0x7f0100154090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f00fffff700 1 -- 172.21.15.28:0/4227078497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0100159500 msgr2=0x7f0100154090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f00fffff700 1 --2- 172.21.15.28:0/4227078497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0100159500 0x7f0100154090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f00fffff700 1 -- 172.21.15.28:0/4227078497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f010015dbf0 msgr2=0x7f0100157fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f00fffff700 1 --2- 172.21.15.28:0/4227078497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f010015dbf0 0x7f0100157fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f00fffff700 1 -- 172.21.15.28:0/4227078497 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f0009550 con 0x7f01000b1d40 2023-11-28T11:09:30.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f00ff7fe700 1 --2- 172.21.15.28:0/4227078497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0100159500 0x7f0100154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:30.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f00fffff700 1 --2- 172.21.15.28:0/4227078497 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01000b1d40 0x7f01000b53f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009130 tx=0x7f00f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f00fcff9700 1 -- 172.21.15.28:0/4227078497 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f0003d80 con 0x7f01000b1d40 2023-11-28T11:09:30.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f00fcff9700 1 -- 172.21.15.28:0/4227078497 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f001f070 con 0x7f01000b1d40 2023-11-28T11:09:30.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f01068e6700 1 -- 172.21.15.28:0/4227078497 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f010015d3f0 con 0x7f01000b1d40 2023-11-28T11:09:30.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f00fcff9700 1 -- 172.21.15.28:0/4227078497 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f000b8a0 con 0x7f01000b1d40 2023-11-28T11:09:30.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.079+0000 7f01068e6700 1 -- 172.21.15.28:0/4227078497 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f010015d110 con 0x7f01000b1d40 2023-11-28T11:09:30.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.081+0000 7f01068e6700 1 -- 172.21.15.28:0/4227078497 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f010015d6d0 con 0x7f01000b1d40 2023-11-28T11:09:30.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.081+0000 7f00fcff9700 1 -- 172.21.15.28:0/4227078497 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f00f002a020 con 0x7f01000b1d40 2023-11-28T11:09:30.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.081+0000 7f00fcff9700 1 --2- 172.21.15.28:0/4227078497 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00e00682f0 0x7f00e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.082+0000 7f00fcff9700 1 -- 172.21.15.28:0/4227078497 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f00f0087840 con 0x7f01000b1d40 2023-11-28T11:09:30.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.082+0000 7f00ff7fe700 1 --2- 172.21.15.28:0/4227078497 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00e00682f0 0x7f00e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.082+0000 7f00ff7fe700 1 --2- 172.21.15.28:0/4227078497 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00e00682f0 0x7f00e006a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f01001f3ac0 tx=0x7f00e8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.086+0000 7f00fcff9700 1 -- 172.21.15.28:0/4227078497 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f00f0053cc0 con 0x7f01000b1d40 2023-11-28T11:09:30.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.280+0000 7f01068e6700 1 -- 172.21.15.28:0/4227078497 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f010015d6d0 con 0x7f01000b1d40 2023-11-28T11:09:30.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.280+0000 7f00fcff9700 1 -- 172.21.15.28:0/4227078497 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v25) v1 ==== 93+0+10 (secure 0 0 0) 0x7f00f00572e0 con 0x7f01000b1d40 2023-11-28T11:09:30.282 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:09:30.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.282+0000 7f01068e6700 1 -- 172.21.15.28:0/4227078497 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00e00682f0 msgr2=0x7f00e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.282+0000 7f01068e6700 1 --2- 172.21.15.28:0/4227078497 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00e00682f0 0x7f00e006a7a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f01001f3ac0 tx=0x7f00e8009040 comp rx=0 tx=0).stop 2023-11-28T11:09:30.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.282+0000 7f01068e6700 1 -- 172.21.15.28:0/4227078497 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01000b1d40 msgr2=0x7f01000b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.282+0000 7f01068e6700 1 --2- 172.21.15.28:0/4227078497 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01000b1d40 0x7f01000b53f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f00f0009130 tx=0x7f00f0004750 comp rx=0 tx=0).stop 2023-11-28T11:09:30.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.282+0000 7f01068e6700 1 -- 172.21.15.28:0/4227078497 shutdown_connections 2023-11-28T11:09:30.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.282+0000 7f01068e6700 1 --2- 172.21.15.28:0/4227078497 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00e00682f0 0x7f00e006a7a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.282+0000 7f01068e6700 1 --2- 172.21.15.28:0/4227078497 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f01000b1d40 0x7f01000b53f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.282+0000 7f01068e6700 1 --2- 172.21.15.28:0/4227078497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f010015dbf0 0x7f0100157fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.282+0000 7f01068e6700 1 --2- 172.21.15.28:0/4227078497 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0100159500 0x7f0100154090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.283+0000 7f01068e6700 1 -- 172.21.15.28:0/4227078497 >> 172.21.15.28:0/4227078497 conn(0x7f01000cc630 msgr2=0x7f010006bbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:30.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.283+0000 7f01068e6700 1 -- 172.21.15.28:0/4227078497 shutdown_connections 2023-11-28T11:09:30.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.283+0000 7f01068e6700 1 -- 172.21.15.28:0/4227078497 wait complete. 2023-11-28T11:09:30.296 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:30.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.422+0000 7f17d5b57700 1 Processor -- start 2023-11-28T11:09:30.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.422+0000 7f17d5b57700 1 -- start start 2023-11-28T11:09:30.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.423+0000 7f17d5b57700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17d0159500 0x7f17d00cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17d5b57700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f17d00b1d40 0x7f17d00ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17d5b57700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17d015e040 con 0x7f17d015dbf0 2023-11-28T11:09:30.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17d5b57700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17d00cede0 con 0x7f17d00b1d40 2023-11-28T11:09:30.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17d5b57700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d00ceff0 con 0x7f17d0159500 2023-11-28T11:09:30.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ceffd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f17d015dbf0 0x7f17d00b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35970/0 (socket says 172.21.15.28:35970) 2023-11-28T11:09:30.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ceffd700 1 -- 172.21.15.28:0/2096574797 learned_addr learned my addr 172.21.15.28:0/2096574797 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:30.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 -- 172.21.15.28:0/2096574797 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461377979 0 0) 0x7f17d015e040 con 0x7f17d015dbf0 2023-11-28T11:09:30.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17cf7fe700 1 --2- 172.21.15.28:0/2096574797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17d0159500 0x7f17d00cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17cffff700 1 --2- 172.21.15.28:0/2096574797 >> v2:172.21.15.59:3300/0 conn(0x7f17d00b1d40 0x7f17d00ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 -- 172.21.15.28:0/2096574797 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17bc0035b0 con 0x7f17d015dbf0 2023-11-28T11:09:30.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 -- 172.21.15.28:0/2096574797 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1893477710 0 0) 0x7f17bc0035b0 con 0x7f17d015dbf0 2023-11-28T11:09:30.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 -- 172.21.15.28:0/2096574797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17d0159500 msgr2=0x7f17d00cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 --2- 172.21.15.28:0/2096574797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17d0159500 0x7f17d00cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 -- 172.21.15.28:0/2096574797 >> v2:172.21.15.59:3300/0 conn(0x7f17d00b1d40 msgr2=0x7f17d00ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 --2- 172.21.15.28:0/2096574797 >> v2:172.21.15.59:3300/0 conn(0x7f17d00b1d40 0x7f17d00ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 -- 172.21.15.28:0/2096574797 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d00cf2d0 con 0x7f17d015dbf0 2023-11-28T11:09:30.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 -- 172.21.15.28:0/2096574797 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f17c0001670 con 0x7f17d015dbf0 2023-11-28T11:09:30.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 -- 172.21.15.28:0/2096574797 >> v1:172.21.15.28:6789/0 conn(0x7f17d015dbf0 legacy=0x7f17d00b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 --2- 172.21.15.28:0/2096574797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17bc003a00 0x7f17bc003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 --2- 172.21.15.28:0/2096574797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17bc004430 0x7f17bc0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 --2- 172.21.15.28:0/2096574797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17d0159500 0x7f17bc004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 -- 172.21.15.28:0/2096574797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d00cede0 con 0x7f17bc004430 2023-11-28T11:09:30.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 -- 172.21.15.28:0/2096574797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d00ceff0 con 0x7f17bc003a00 2023-11-28T11:09:30.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ce7fc700 1 -- 172.21.15.28:0/2096574797 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17bc00ba10 con 0x7f17d0159500 2023-11-28T11:09:30.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.424+0000 7f17ceffd700 1 --2- 172.21.15.28:0/2096574797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17bc004430 0x7f17bc0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.425+0000 7f17cf7fe700 1 --2- 172.21.15.28:0/2096574797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17bc003a00 0x7f17bc003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.425+0000 7f17cffff700 1 --2- 172.21.15.28:0/2096574797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17d0159500 0x7f17bc004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.425+0000 7f17ceffd700 1 -- 172.21.15.28:0/2096574797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17d0159500 msgr2=0x7f17bc004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.425+0000 7f17ceffd700 1 --2- 172.21.15.28:0/2096574797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17d0159500 0x7f17bc004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.425+0000 7f17ceffd700 1 -- 172.21.15.28:0/2096574797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17bc003a00 msgr2=0x7f17bc003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.425+0000 7f17ceffd700 1 --2- 172.21.15.28:0/2096574797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17bc003a00 0x7f17bc003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.425+0000 7f17ceffd700 1 -- 172.21.15.28:0/2096574797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc0035b0 con 0x7f17bc004430 2023-11-28T11:09:30.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.425+0000 7f17ceffd700 1 --2- 172.21.15.28:0/2096574797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17bc004430 0x7f17bc0048f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f17c000c320 tx=0x7f17bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3821894ce2f4d422 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.425+0000 7f17ce7fc700 1 -- 172.21.15.28:0/2096574797 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c000ce60 con 0x7f17bc004430 2023-11-28T11:09:30.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.425+0000 7f17ce7fc700 1 -- 172.21.15.28:0/2096574797 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c0005960 con 0x7f17bc004430 2023-11-28T11:09:30.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.425+0000 7f17ce7fc700 1 -- 172.21.15.28:0/2096574797 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c0005cf0 con 0x7f17bc004430 2023-11-28T11:09:30.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.426+0000 7f17d5b57700 1 -- 172.21.15.28:0/2096574797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17bc004430 msgr2=0x7f17bc0048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.426+0000 7f17d5b57700 1 --2- 172.21.15.28:0/2096574797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17bc004430 0x7f17bc0048f0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f17c000c320 tx=0x7f17bc005880 comp rx=0 tx=0).stop 2023-11-28T11:09:30.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.426+0000 7f17cffff700 1 -- 172.21.15.28:0/2096574797 reap_dead start 2023-11-28T11:09:30.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.426+0000 7f17d5b57700 1 -- 172.21.15.28:0/2096574797 shutdown_connections 2023-11-28T11:09:30.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.426+0000 7f17d5b57700 1 -- 172.21.15.28:0/2096574797 >> 172.21.15.28:0/2096574797 conn(0x7f17d00cc630 msgr2=0x7f17d00cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:30.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.426+0000 7f17d5b57700 1 -- 172.21.15.28:0/2096574797 shutdown_connections 2023-11-28T11:09:30.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.426+0000 7f17d5b57700 1 -- 172.21.15.28:0/2096574797 wait complete. 2023-11-28T11:09:30.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.427+0000 7f17d5b57700 1 Processor -- start 2023-11-28T11:09:30.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.428+0000 7f17d5b57700 1 -- start start 2023-11-28T11:09:30.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.428+0000 7f17d5b57700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17bc004430 0x7f17d00b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.428+0000 7f17d5b57700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17d0151af0 0x7f17d00b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17d5b57700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17d015b640 0x7f17d00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17d5b57700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d015c780 con 0x7f17d015b640 2023-11-28T11:09:30.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17d5b57700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17d015bc00 con 0x7f17d0151af0 2023-11-28T11:09:30.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17d5b57700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17d015bee0 con 0x7f17bc004430 2023-11-28T11:09:30.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17cffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17d015b640 0x7f17d00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17cffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17d015b640 0x7f17d00b4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47926/0 (socket says 172.21.15.28:47926) 2023-11-28T11:09:30.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17cffff700 1 -- 172.21.15.28:0/1415388335 learned_addr learned my addr 172.21.15.28:0/1415388335 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:30.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17cf7fe700 1 --2- 172.21.15.28:0/1415388335 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17bc004430 0x7f17d00b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17ceffd700 1 --2- 172.21.15.28:0/1415388335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17d0151af0 0x7f17d00b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17cffff700 1 -- 172.21.15.28:0/1415388335 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17bc004430 msgr2=0x7f17d00b1490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17cffff700 1 --2- 172.21.15.28:0/1415388335 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17bc004430 0x7f17d00b1490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17cffff700 1 -- 172.21.15.28:0/1415388335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17d0151af0 msgr2=0x7f17d00b49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17cffff700 1 --2- 172.21.15.28:0/1415388335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17d0151af0 0x7f17d00b49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17cffff700 1 -- 172.21.15.28:0/1415388335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17d015d3f0 con 0x7f17d015b640 2023-11-28T11:09:30.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17cffff700 1 --2- 172.21.15.28:0/1415388335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17d015b640 0x7f17d00b4f30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f17d00d0c50 tx=0x7f17c8009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17c7fff700 1 -- 172.21.15.28:0/1415388335 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8003e30 con 0x7f17d015b640 2023-11-28T11:09:30.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17c7fff700 1 -- 172.21.15.28:0/1415388335 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17c800e040 con 0x7f17d015b640 2023-11-28T11:09:30.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17d5b57700 1 -- 172.21.15.28:0/1415388335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17d015d110 con 0x7f17d015b640 2023-11-28T11:09:30.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17c7fff700 1 -- 172.21.15.28:0/1415388335 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17c8012930 con 0x7f17d015b640 2023-11-28T11:09:30.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.429+0000 7f17d5b57700 1 -- 172.21.15.28:0/1415388335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17d015d6d0 con 0x7f17d015b640 2023-11-28T11:09:30.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.430+0000 7f17d5b57700 1 -- 172.21.15.28:0/1415388335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17d00739f0 con 0x7f17d015b640 2023-11-28T11:09:30.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.431+0000 7f17c7fff700 1 -- 172.21.15.28:0/1415388335 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f17c8021030 con 0x7f17d015b640 2023-11-28T11:09:30.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.431+0000 7f17c7fff700 1 --2- 172.21.15.28:0/1415388335 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f17b00682f0 0x7f17b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.431+0000 7f17c7fff700 1 -- 172.21.15.28:0/1415388335 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f17c8087b70 con 0x7f17d015b640 2023-11-28T11:09:30.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.431+0000 7f17cf7fe700 1 --2- 172.21.15.28:0/1415388335 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f17b00682f0 0x7f17b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.432+0000 7f17cf7fe700 1 --2- 172.21.15.28:0/1415388335 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f17b00682f0 0x7f17b006a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f17d01550a0 tx=0x7f17b8009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.435+0000 7f17c7fff700 1 -- 172.21.15.28:0/1415388335 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f17c80540e0 con 0x7f17d015b640 2023-11-28T11:09:30.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.630+0000 7f17d5b57700 1 -- 172.21.15.28:0/1415388335 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f17d00b2230 con 0x7f17d015b640 2023-11-28T11:09:30.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.631+0000 7f17c7fff700 1 -- 172.21.15.28:0/1415388335 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v25) v1 ==== 104+0+11 (secure 0 0 0) 0x7f17c8057700 con 0x7f17d015b640 2023-11-28T11:09:30.632 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:30.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.633+0000 7f17d5b57700 1 -- 172.21.15.28:0/1415388335 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f17b00682f0 msgr2=0x7f17b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.633+0000 7f17d5b57700 1 --2- 172.21.15.28:0/1415388335 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f17b00682f0 0x7f17b006a7a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f17d01550a0 tx=0x7f17b8009210 comp rx=0 tx=0).stop 2023-11-28T11:09:30.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.633+0000 7f17d5b57700 1 -- 172.21.15.28:0/1415388335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17d015b640 msgr2=0x7f17d00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.633+0000 7f17d5b57700 1 --2- 172.21.15.28:0/1415388335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17d015b640 0x7f17d00b4f30 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f17d00d0c50 tx=0x7f17c8009ea0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.633+0000 7f17d5b57700 1 -- 172.21.15.28:0/1415388335 shutdown_connections 2023-11-28T11:09:30.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.633+0000 7f17d5b57700 1 --2- 172.21.15.28:0/1415388335 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f17b00682f0 0x7f17b006a7a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.633+0000 7f17d5b57700 1 --2- 172.21.15.28:0/1415388335 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f17d015b640 0x7f17d00b4f30 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.633+0000 7f17d5b57700 1 --2- 172.21.15.28:0/1415388335 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f17bc004430 0x7f17d00b1490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.633+0000 7f17d5b57700 1 --2- 172.21.15.28:0/1415388335 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f17d0151af0 0x7f17d00b49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.633+0000 7f17d5b57700 1 -- 172.21.15.28:0/1415388335 >> 172.21.15.28:0/1415388335 conn(0x7f17d00cc630 msgr2=0x7f17d00cd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:30.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.633+0000 7f17d5b57700 1 -- 172.21.15.28:0/1415388335 shutdown_connections 2023-11-28T11:09:30.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.633+0000 7f17d5b57700 1 -- 172.21.15.28:0/1415388335 wait complete. 2023-11-28T11:09:30.646 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:30.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.772+0000 7f68fcc51700 1 Processor -- start 2023-11-28T11:09:30.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.772+0000 7f68fcc51700 1 -- start start 2023-11-28T11:09:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.773+0000 7f68fcc51700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f68f80b2680 0x7f68f80b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.773+0000 7f68fcc51700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68f80cbbf0 0x7f68f80cc050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.773+0000 7f68fcc51700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f806f940 con 0x7f68f815fc00 2023-11-28T11:09:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.773+0000 7f68fcc51700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f80b3030 con 0x7f68f80b2680 2023-11-28T11:09:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.773+0000 7f68fcc51700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f806fc20 con 0x7f68f80cbbf0 2023-11-28T11:09:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.773+0000 7f68f659c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f68f815fc00 0x7f68f806eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:35988/0 (socket says 172.21.15.28:35988) 2023-11-28T11:09:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.773+0000 7f68f659c700 1 -- 172.21.15.28:0/96227658 learned_addr learned my addr 172.21.15.28:0/96227658 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:30.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f5d9b700 1 --2- 172.21.15.28:0/96227658 >> v2:172.21.15.59:3300/0 conn(0x7f68f80b2680 0x7f68f80b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f6d9d700 1 --2- 172.21.15.28:0/96227658 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68f80cbbf0 0x7f68f80cc050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 -- 172.21.15.28:0/96227658 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2169252216 0 0) 0x7f68f806f940 con 0x7f68f815fc00 2023-11-28T11:09:30.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 -- 172.21.15.28:0/96227658 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e00035b0 con 0x7f68f815fc00 2023-11-28T11:09:30.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 -- 172.21.15.28:0/96227658 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1796402749 0 0) 0x7f68e00035b0 con 0x7f68f815fc00 2023-11-28T11:09:30.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 -- 172.21.15.28:0/96227658 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68f80cbbf0 msgr2=0x7f68f80cc050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 --2- 172.21.15.28:0/96227658 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68f80cbbf0 0x7f68f80cc050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 -- 172.21.15.28:0/96227658 >> v2:172.21.15.59:3300/0 conn(0x7f68f80b2680 msgr2=0x7f68f80b2af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 --2- 172.21.15.28:0/96227658 >> v2:172.21.15.59:3300/0 conn(0x7f68f80b2680 0x7f68f80b2af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 -- 172.21.15.28:0/96227658 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f80cb3f0 con 0x7f68f815fc00 2023-11-28T11:09:30.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 -- 172.21.15.28:0/96227658 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f68e8001670 con 0x7f68f815fc00 2023-11-28T11:09:30.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 -- 172.21.15.28:0/96227658 >> v1:172.21.15.28:6789/0 conn(0x7f68f815fc00 legacy=0x7f68f806eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 --2- 172.21.15.28:0/96227658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68e0003a00 0x7f68e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 --2- 172.21.15.28:0/96227658 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68f80cbbf0 0x7f68e0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 --2- 172.21.15.28:0/96227658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0004d90 0x7f68e0005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 -- 172.21.15.28:0/96227658 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80b3030 con 0x7f68e0004d90 2023-11-28T11:09:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 -- 172.21.15.28:0/96227658 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f806fc20 con 0x7f68e0003a00 2023-11-28T11:09:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f559a700 1 -- 172.21.15.28:0/96227658 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e000ba10 con 0x7f68f80cbbf0 2023-11-28T11:09:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f6d9d700 1 --2- 172.21.15.28:0/96227658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0004d90 0x7f68e0005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f5d9b700 1 --2- 172.21.15.28:0/96227658 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68f80cbbf0 0x7f68e0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.774+0000 7f68f659c700 1 --2- 172.21.15.28:0/96227658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68e0003a00 0x7f68e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.775+0000 7f68f6d9d700 1 -- 172.21.15.28:0/96227658 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68f80cbbf0 msgr2=0x7f68e0004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.775+0000 7f68f6d9d700 1 --2- 172.21.15.28:0/96227658 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68f80cbbf0 0x7f68e0004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.775+0000 7f68f6d9d700 1 -- 172.21.15.28:0/96227658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68e0003a00 msgr2=0x7f68e0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:30.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.775+0000 7f68f6d9d700 1 --2- 172.21.15.28:0/96227658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68e0003a00 0x7f68e0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.775+0000 7f68f6d9d700 1 -- 172.21.15.28:0/96227658 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e00035b0 con 0x7f68e0004d90 2023-11-28T11:09:30.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.775+0000 7f68f659c700 1 --2- 172.21.15.28:0/96227658 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68e0003a00 0x7f68e0003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:30.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.775+0000 7f68f6d9d700 1 --2- 172.21.15.28:0/96227658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0004d90 0x7f68e0005200 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f68f0009fc0 tx=0x7f68e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fee6cf0bcb5f85 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.775+0000 7f68f559a700 1 -- 172.21.15.28:0/96227658 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f000ebd0 con 0x7f68e0004d90 2023-11-28T11:09:30.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.775+0000 7f68f559a700 1 -- 172.21.15.28:0/96227658 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f000f040 con 0x7f68e0004d90 2023-11-28T11:09:30.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.775+0000 7f68f559a700 1 -- 172.21.15.28:0/96227658 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f0014480 con 0x7f68e0004d90 2023-11-28T11:09:30.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.775+0000 7f68fcc51700 1 -- 172.21.15.28:0/96227658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0004d90 msgr2=0x7f68e0005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.775+0000 7f68fcc51700 1 --2- 172.21.15.28:0/96227658 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0004d90 0x7f68e0005200 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f68f0009fc0 tx=0x7f68e0005880 comp rx=0 tx=0).stop 2023-11-28T11:09:30.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.775+0000 7f68f6d9d700 1 -- 172.21.15.28:0/96227658 reap_dead start 2023-11-28T11:09:30.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.775+0000 7f68fcc51700 1 -- 172.21.15.28:0/96227658 shutdown_connections 2023-11-28T11:09:30.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.775+0000 7f68fcc51700 1 -- 172.21.15.28:0/96227658 >> 172.21.15.28:0/96227658 conn(0x7f68f80cd680 msgr2=0x7f68f80cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:30.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.776+0000 7f68fcc51700 1 -- 172.21.15.28:0/96227658 shutdown_connections 2023-11-28T11:09:30.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.776+0000 7f68fcc51700 1 -- 172.21.15.28:0/96227658 wait complete. 2023-11-28T11:09:30.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.777+0000 7f68fcc51700 1 Processor -- start 2023-11-28T11:09:30.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.777+0000 7f68fcc51700 1 -- start start 2023-11-28T11:09:30.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68fcc51700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68e0004d90 0x7f68f8155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68fcc51700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f80b5c00 0x7f68f80b6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68fcc51700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f80c6040 0x7f68f80c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68fcc51700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80c78a0 con 0x7f68f80c6040 2023-11-28T11:09:30.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68fcc51700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80b65b0 con 0x7f68f80b5c00 2023-11-28T11:09:30.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68fcc51700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f80c69f0 con 0x7f68e0004d90 2023-11-28T11:09:30.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68f6d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f80c6040 0x7f68f80c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68f6d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f80c6040 0x7f68f80c64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47946/0 (socket says 172.21.15.28:47946) 2023-11-28T11:09:30.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68f6d9d700 1 -- 172.21.15.28:0/3804392153 learned_addr learned my addr 172.21.15.28:0/3804392153 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:30.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68f659c700 1 --2- 172.21.15.28:0/3804392153 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68e0004d90 0x7f68f8155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68f5d9b700 1 --2- 172.21.15.28:0/3804392153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f80b5c00 0x7f68f80b6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68f6d9d700 1 -- 172.21.15.28:0/3804392153 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68e0004d90 msgr2=0x7f68f8155270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68f6d9d700 1 --2- 172.21.15.28:0/3804392153 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68e0004d90 0x7f68f8155270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68f6d9d700 1 -- 172.21.15.28:0/3804392153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f80b5c00 msgr2=0x7f68f80b6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68f6d9d700 1 --2- 172.21.15.28:0/3804392153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f80b5c00 0x7f68f80b6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68f6d9d700 1 -- 172.21.15.28:0/3804392153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f815ffd0 con 0x7f68f80c6040 2023-11-28T11:09:30.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68f6d9d700 1 --2- 172.21.15.28:0/3804392153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f80c6040 0x7f68f80c64b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f68e00046d0 tx=0x7f68f0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68e6ffd700 1 -- 172.21.15.28:0/3804392153 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f0007000 con 0x7f68f80c6040 2023-11-28T11:09:30.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68e6ffd700 1 -- 172.21.15.28:0/3804392153 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f0022070 con 0x7f68f80c6040 2023-11-28T11:09:30.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68fcc51700 1 -- 172.21.15.28:0/3804392153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f80cb6d0 con 0x7f68f80c6040 2023-11-28T11:09:30.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.778+0000 7f68e6ffd700 1 -- 172.21.15.28:0/3804392153 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f0029480 con 0x7f68f80c6040 2023-11-28T11:09:30.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.779+0000 7f68fcc51700 1 -- 172.21.15.28:0/3804392153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f806f5b0 con 0x7f68f80c6040 2023-11-28T11:09:30.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.780+0000 7f68fcc51700 1 -- 172.21.15.28:0/3804392153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f80c79e0 con 0x7f68f80c6040 2023-11-28T11:09:30.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.780+0000 7f68e6ffd700 1 -- 172.21.15.28:0/3804392153 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f68f00295e0 con 0x7f68f80c6040 2023-11-28T11:09:30.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.780+0000 7f68e6ffd700 1 --2- 172.21.15.28:0/3804392153 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68d80682f0 0x7f68d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:30.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.781+0000 7f68f659c700 1 --2- 172.21.15.28:0/3804392153 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68d80682f0 0x7f68d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:30.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.781+0000 7f68e6ffd700 1 -- 172.21.15.28:0/3804392153 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f68f0024070 con 0x7f68f80c6040 2023-11-28T11:09:30.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.781+0000 7f68f659c700 1 --2- 172.21.15.28:0/3804392153 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68d80682f0 0x7f68d806a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68e8000fb0 tx=0x7f68e8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:30.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.785+0000 7f68e6ffd700 1 -- 172.21.15.28:0/3804392153 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f68f0054870 con 0x7f68f80c6040 2023-11-28T11:09:30.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.978+0000 7f68fcc51700 1 -- 172.21.15.28:0/3804392153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f68f80739f0 con 0x7f68f80c6040 2023-11-28T11:09:30.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.978+0000 7f68e6ffd700 1 -- 172.21.15.28:0/3804392153 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v25) v1 ==== 100+0+11 (secure 0 0 0) 0x7f68f0057e90 con 0x7f68f80c6040 2023-11-28T11:09:30.980 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:09:30.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.980+0000 7f68fcc51700 1 -- 172.21.15.28:0/3804392153 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68d80682f0 msgr2=0x7f68d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.980+0000 7f68fcc51700 1 --2- 172.21.15.28:0/3804392153 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68d80682f0 0x7f68d806a7a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f68e8000fb0 tx=0x7f68e8009040 comp rx=0 tx=0).stop 2023-11-28T11:09:30.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.980+0000 7f68fcc51700 1 -- 172.21.15.28:0/3804392153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f80c6040 msgr2=0x7f68f80c64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:30.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.980+0000 7f68fcc51700 1 --2- 172.21.15.28:0/3804392153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f80c6040 0x7f68f80c64b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f68e00046d0 tx=0x7f68f0009750 comp rx=0 tx=0).stop 2023-11-28T11:09:30.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.980+0000 7f68fcc51700 1 -- 172.21.15.28:0/3804392153 shutdown_connections 2023-11-28T11:09:30.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.980+0000 7f68fcc51700 1 --2- 172.21.15.28:0/3804392153 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68d80682f0 0x7f68d806a7a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.980+0000 7f68fcc51700 1 --2- 172.21.15.28:0/3804392153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68f80c6040 0x7f68f80c64b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.980+0000 7f68fcc51700 1 --2- 172.21.15.28:0/3804392153 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68e0004d90 0x7f68f8155270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.980+0000 7f68fcc51700 1 --2- 172.21.15.28:0/3804392153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f80b5c00 0x7f68f80b6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:30.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.980+0000 7f68fcc51700 1 -- 172.21.15.28:0/3804392153 >> 172.21.15.28:0/3804392153 conn(0x7f68f80cd680 msgr2=0x7f68f80cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:30.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.981+0000 7f68fcc51700 1 -- 172.21.15.28:0/3804392153 shutdown_connections 2023-11-28T11:09:30.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:30.981+0000 7f68fcc51700 1 -- 172.21.15.28:0/3804392153 wait complete. 2023-11-28T11:09:30.994 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:31.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.120+0000 7fa3746db700 1 Processor -- start 2023-11-28T11:09:31.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.120+0000 7fa3746db700 1 -- start start 2023-11-28T11:09:31.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa3746db700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa36c0c9950 0x7fa36c15d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa3746db700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36c0cae80 con 0x7fa36c15c880 2023-11-28T11:09:31.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa3746db700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36c0c9de0 con 0x7fa36c15c4b0 2023-11-28T11:09:31.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa3746db700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c15dba0 con 0x7fa36c0c9950 2023-11-28T11:09:31.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa371c76700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fa36c15c880 0x7fa36c0cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36010/0 (socket says 172.21.15.28:36010) 2023-11-28T11:09:31.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa371c76700 1 -- 172.21.15.28:0/922508520 learned_addr learned my addr 172.21.15.28:0/922508520 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:31.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa372c78700 1 --2- 172.21.15.28:0/922508520 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa36c0c9950 0x7fa36c15d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa371475700 1 -- 172.21.15.28:0/922508520 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960758844 0 0) 0x7fa36c0cae80 con 0x7fa36c15c880 2023-11-28T11:09:31.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa371475700 1 -- 172.21.15.28:0/922508520 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3580035b0 con 0x7fa36c15c880 2023-11-28T11:09:31.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa371475700 1 -- 172.21.15.28:0/922508520 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3033472023 0 0) 0x7fa3580035b0 con 0x7fa36c15c880 2023-11-28T11:09:31.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa371475700 1 -- 172.21.15.28:0/922508520 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa36c0c9950 msgr2=0x7fa36c15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:31.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa371475700 1 --2- 172.21.15.28:0/922508520 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa36c0c9950 0x7fa36c15d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa371475700 1 -- 172.21.15.28:0/922508520 >> v1:172.21.15.59:6789/0 conn(0x7fa36c15c4b0 legacy=0x7fa36c06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa371475700 1 -- 172.21.15.28:0/922508520 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c0ca0c0 con 0x7fa36c15c880 2023-11-28T11:09:31.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa371475700 1 -- 172.21.15.28:0/922508520 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fa35c001670 con 0x7fa36c15c880 2023-11-28T11:09:31.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa371475700 1 -- 172.21.15.28:0/922508520 >> v1:172.21.15.28:6789/0 conn(0x7fa36c15c880 legacy=0x7fa36c0cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa371475700 1 --2- 172.21.15.28:0/922508520 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa36c0c9950 0x7fa358003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.121+0000 7fa371475700 1 --2- 172.21.15.28:0/922508520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa358003f70 0x7fa358004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa371475700 1 --2- 172.21.15.28:0/922508520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa358004d90 0x7fa358005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa371475700 1 -- 172.21.15.28:0/922508520 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c15dba0 con 0x7fa358003f70 2023-11-28T11:09:31.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa371475700 1 -- 172.21.15.28:0/922508520 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa35800ba40 con 0x7fa358004d90 2023-11-28T11:09:31.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa371475700 1 -- 172.21.15.28:0/922508520 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa35800bc00 con 0x7fa36c0c9950 2023-11-28T11:09:31.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa371c76700 1 --2- 172.21.15.28:0/922508520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa358003f70 0x7fa358004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa372477700 1 --2- 172.21.15.28:0/922508520 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa36c0c9950 0x7fa358003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa372c78700 1 --2- 172.21.15.28:0/922508520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa358004d90 0x7fa358005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa371c76700 1 -- 172.21.15.28:0/922508520 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa36c0c9950 msgr2=0x7fa358003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:31.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa371c76700 1 --2- 172.21.15.28:0/922508520 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa36c0c9950 0x7fa358003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa371c76700 1 -- 172.21.15.28:0/922508520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa358004d90 msgr2=0x7fa358005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:31.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa371c76700 1 --2- 172.21.15.28:0/922508520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa358004d90 0x7fa358005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa371c76700 1 -- 172.21.15.28:0/922508520 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3580035b0 con 0x7fa358003f70 2023-11-28T11:09:31.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa371c76700 1 --2- 172.21.15.28:0/922508520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa358003f70 0x7fa358004850 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00c2f0 tx=0x7fa358005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e70e951f409bec3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:31.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa371475700 1 -- 172.21.15.28:0/922508520 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00ce30 con 0x7fa358003f70 2023-11-28T11:09:31.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa371475700 1 -- 172.21.15.28:0/922508520 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c005960 con 0x7fa358003f70 2023-11-28T11:09:31.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.122+0000 7fa371475700 1 -- 172.21.15.28:0/922508520 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c005d10 con 0x7fa358003f70 2023-11-28T11:09:31.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.123+0000 7fa3746db700 1 -- 172.21.15.28:0/922508520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa358003f70 msgr2=0x7fa358004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.123+0000 7fa3746db700 1 --2- 172.21.15.28:0/922508520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa358003f70 0x7fa358004850 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00c2f0 tx=0x7fa358005880 comp rx=0 tx=0).stop 2023-11-28T11:09:31.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.123+0000 7fa372c78700 1 -- 172.21.15.28:0/922508520 reap_dead start 2023-11-28T11:09:31.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.123+0000 7fa3746db700 1 -- 172.21.15.28:0/922508520 shutdown_connections 2023-11-28T11:09:31.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.123+0000 7fa3746db700 1 -- 172.21.15.28:0/922508520 >> 172.21.15.28:0/922508520 conn(0x7fa36c0b0df0 msgr2=0x7fa36c0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:31.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.123+0000 7fa3746db700 1 -- 172.21.15.28:0/922508520 shutdown_connections 2023-11-28T11:09:31.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.123+0000 7fa3746db700 1 -- 172.21.15.28:0/922508520 wait complete. 2023-11-28T11:09:31.130 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.124+0000 7fa3746db700 1 Processor -- start 2023-11-28T11:09:31.130 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.124+0000 7fa3746db700 1 -- start start 2023-11-28T11:09:31.130 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa3746db700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa358003f70 0x7fa36c14da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.130 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa3746db700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa36c0b3b00 0x7fa36c0b3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa3746db700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa36c15ee90 0x7fa36c15f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa3746db700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c0b44b0 con 0x7fa36c0b3b00 2023-11-28T11:09:31.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa3746db700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c15f840 con 0x7fa36c15ee90 2023-11-28T11:09:31.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa3746db700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c0ab8c0 con 0x7fa358003f70 2023-11-28T11:09:31.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa371c76700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa36c0b3b00 0x7fa36c0b3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa371c76700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa36c0b3b00 0x7fa36c0b3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47964/0 (socket says 172.21.15.28:47964) 2023-11-28T11:09:31.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa371c76700 1 -- 172.21.15.28:0/3787333885 learned_addr learned my addr 172.21.15.28:0/3787333885 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:31.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa371c76700 1 -- 172.21.15.28:0/3787333885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa358003f70 msgr2=0x7fa36c14da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa372477700 1 --2- 172.21.15.28:0/3787333885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa358003f70 0x7fa36c14da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa372c78700 1 --2- 172.21.15.28:0/3787333885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa36c15ee90 0x7fa36c15f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa371c76700 1 --2- 172.21.15.28:0/3787333885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa358003f70 0x7fa36c14da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa371c76700 1 -- 172.21.15.28:0/3787333885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa36c15ee90 msgr2=0x7fa36c15f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa371c76700 1 --2- 172.21.15.28:0/3787333885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa36c15ee90 0x7fa36c15f300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa371c76700 1 -- 172.21.15.28:0/3787333885 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c0ca680 con 0x7fa36c0b3b00 2023-11-28T11:09:31.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.125+0000 7fa371c76700 1 --2- 172.21.15.28:0/3787333885 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa36c0b3b00 0x7fa36c0b3f70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00c8e0 tx=0x7fa35c001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:31.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.126+0000 7fa362ffd700 1 -- 172.21.15.28:0/3787333885 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c001660 con 0x7fa36c0b3b00 2023-11-28T11:09:31.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.126+0000 7fa362ffd700 1 -- 172.21.15.28:0/3787333885 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c004a80 con 0x7fa36c0b3b00 2023-11-28T11:09:31.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.126+0000 7fa3746db700 1 -- 172.21.15.28:0/3787333885 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36c0ca3a0 con 0x7fa36c0b3b00 2023-11-28T11:09:31.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.126+0000 7fa362ffd700 1 -- 172.21.15.28:0/3787333885 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c004e30 con 0x7fa36c0b3b00 2023-11-28T11:09:31.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.126+0000 7fa3746db700 1 -- 172.21.15.28:0/3787333885 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36c0ca960 con 0x7fa36c0b3b00 2023-11-28T11:09:31.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.127+0000 7fa3746db700 1 -- 172.21.15.28:0/3787333885 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36c0739f0 con 0x7fa36c0b3b00 2023-11-28T11:09:31.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.127+0000 7fa362ffd700 1 -- 172.21.15.28:0/3787333885 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa35c02e070 con 0x7fa36c0b3b00 2023-11-28T11:09:31.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.127+0000 7fa362ffd700 1 --2- 172.21.15.28:0/3787333885 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3500682f0 0x7fa35006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.128+0000 7fa362ffd700 1 -- 172.21.15.28:0/3787333885 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fa35c087770 con 0x7fa36c0b3b00 2023-11-28T11:09:31.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.128+0000 7fa372477700 1 --2- 172.21.15.28:0/3787333885 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3500682f0 0x7fa35006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.128+0000 7fa372477700 1 --2- 172.21.15.28:0/3787333885 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3500682f0 0x7fa35006a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa368000c00 tx=0x7fa368009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:31.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.132+0000 7fa362ffd700 1 -- 172.21.15.28:0/3787333885 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa35c053d40 con 0x7fa36c0b3b00 2023-11-28T11:09:31.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.326+0000 7fa3746db700 1 -- 172.21.15.28:0/3787333885 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa36c0b5260 con 0x7fa36c0b3b00 2023-11-28T11:09:31.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.326+0000 7fa362ffd700 1 -- 172.21.15.28:0/3787333885 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v25) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa35c007da0 con 0x7fa36c0b3b00 2023-11-28T11:09:31.328 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:31.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.328+0000 7fa3746db700 1 -- 172.21.15.28:0/3787333885 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3500682f0 msgr2=0x7fa35006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.328+0000 7fa3746db700 1 --2- 172.21.15.28:0/3787333885 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3500682f0 0x7fa35006a7a0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa368000c00 tx=0x7fa368009040 comp rx=0 tx=0).stop 2023-11-28T11:09:31.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.328+0000 7fa3746db700 1 -- 172.21.15.28:0/3787333885 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa36c0b3b00 msgr2=0x7fa36c0b3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.328+0000 7fa3746db700 1 --2- 172.21.15.28:0/3787333885 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa36c0b3b00 0x7fa36c0b3f70 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fa35c00c8e0 tx=0x7fa35c001a00 comp rx=0 tx=0).stop 2023-11-28T11:09:31.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.328+0000 7fa3746db700 1 -- 172.21.15.28:0/3787333885 shutdown_connections 2023-11-28T11:09:31.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.328+0000 7fa3746db700 1 --2- 172.21.15.28:0/3787333885 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3500682f0 0x7fa35006a7a0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.328+0000 7fa3746db700 1 --2- 172.21.15.28:0/3787333885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa36c15ee90 0x7fa36c15f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.328+0000 7fa3746db700 1 --2- 172.21.15.28:0/3787333885 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa358003f70 0x7fa36c14da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.328+0000 7fa3746db700 1 --2- 172.21.15.28:0/3787333885 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa36c0b3b00 0x7fa36c0b3f70 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.328+0000 7fa3746db700 1 -- 172.21.15.28:0/3787333885 >> 172.21.15.28:0/3787333885 conn(0x7fa36c0b0df0 msgr2=0x7fa36c0cd830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:31.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.328+0000 7fa3746db700 1 -- 172.21.15.28:0/3787333885 shutdown_connections 2023-11-28T11:09:31.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.328+0000 7fa3746db700 1 -- 172.21.15.28:0/3787333885 wait complete. 2023-11-28T11:09:31.353 INFO:tasks.cephfs_test_runner:Starting test: test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:09:31.353 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T11:09:31.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.478+0000 7f8c8e4b4700 1 Processor -- start 2023-11-28T11:09:31.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.479+0000 7f8c8e4b4700 1 -- start start 2023-11-28T11:09:31.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.479+0000 7f8c8e4b4700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8c880ccbc0 0x7f8c880b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.479+0000 7f8c8e4b4700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c880ca990 con 0x7f8c8815a650 2023-11-28T11:09:31.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.479+0000 7f8c8e4b4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c880caf50 con 0x7f8c8815aa20 2023-11-28T11:09:31.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.479+0000 7f8c8e4b4700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c880aef10 con 0x7f8c880ccbc0 2023-11-28T11:09:31.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.479+0000 7f8c8ca51700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8c880ccbc0 0x7f8c880b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.479+0000 7f8c87fff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8c8815a650 0x7f8c880ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36028/0 (socket says 172.21.15.28:36028) 2023-11-28T11:09:31.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.479+0000 7f8c8ca51700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8c880ccbc0 0x7f8c880b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47974/0 (socket says 172.21.15.28:47974) 2023-11-28T11:09:31.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.479+0000 7f8c87fff700 1 -- 172.21.15.28:0/3968130600 learned_addr learned my addr 172.21.15.28:0/3968130600 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:31.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.480+0000 7f8c8ca51700 1 -- 172.21.15.28:0/3968130600 >> v1:172.21.15.59:6789/0 conn(0x7f8c8815aa20 legacy=0x7f8c880cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:31.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.480+0000 7f8c8ca51700 1 -- 172.21.15.28:0/3968130600 >> v1:172.21.15.28:6789/0 conn(0x7f8c8815a650 legacy=0x7f8c880ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.480+0000 7f8c8ca51700 1 -- 172.21.15.28:0/3968130600 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c880af1f0 con 0x7f8c880ccbc0 2023-11-28T11:09:31.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.480+0000 7f8c86ffd700 1 -- 172.21.15.28:0/3968130600 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3103739157 0 0) 0x7f8c880ca990 con 0x7f8c8815a650 2023-11-28T11:09:31.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.480+0000 7f8c8ca51700 1 --2- 172.21.15.28:0/3968130600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c880ccbc0 0x7f8c880b2950 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009d90 tx=0x7f8c8000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10cb39c1594d0950 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:31.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.480+0000 7f8c86ffd700 1 -- 172.21.15.28:0/3968130600 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c8000e070 con 0x7f8c880ccbc0 2023-11-28T11:09:31.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.480+0000 7f8c86ffd700 1 -- 172.21.15.28:0/3968130600 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c8000bb90 con 0x7f8c880ccbc0 2023-11-28T11:09:31.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.480+0000 7f8c86ffd700 1 -- 172.21.15.28:0/3968130600 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c80003a20 con 0x7f8c880ccbc0 2023-11-28T11:09:31.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.481+0000 7f8c8e4b4700 1 -- 172.21.15.28:0/3968130600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c880ccbc0 msgr2=0x7f8c880b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.481+0000 7f8c8e4b4700 1 --2- 172.21.15.28:0/3968130600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c880ccbc0 0x7f8c880b2950 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f8c80009d90 tx=0x7f8c8000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:31.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.481+0000 7f8c8e4b4700 1 -- 172.21.15.28:0/3968130600 shutdown_connections 2023-11-28T11:09:31.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.481+0000 7f8c8e4b4700 1 --2- 172.21.15.28:0/3968130600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c880ccbc0 0x7f8c880b2950 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.481+0000 7f8c8e4b4700 1 -- 172.21.15.28:0/3968130600 >> 172.21.15.28:0/3968130600 conn(0x7f8c880b1650 msgr2=0x7f8c880b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:31.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.481+0000 7f8c8e4b4700 1 -- 172.21.15.28:0/3968130600 shutdown_connections 2023-11-28T11:09:31.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.481+0000 7f8c8e4b4700 1 -- 172.21.15.28:0/3968130600 wait complete. 2023-11-28T11:09:31.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.482+0000 7f8c8e4b4700 1 Processor -- start 2023-11-28T11:09:31.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.482+0000 7f8c8e4b4700 1 -- start start 2023-11-28T11:09:31.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c8e4b4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c880ccbc0 0x7f8c880c5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c8e4b4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8815a650 0x7f8c880cf9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c8e4b4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8c8815aa20 0x7f8c880cff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c8e4b4700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c880c6000 con 0x7f8c8815a650 2023-11-28T11:09:31.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c8e4b4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c881e4cb0 con 0x7f8c880ccbc0 2023-11-28T11:09:31.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c8e4b4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c881e40a0 con 0x7f8c8815aa20 2023-11-28T11:09:31.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c877fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8815a650 0x7f8c880cf9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c877fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8815a650 0x7f8c880cf9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47978/0 (socket says 172.21.15.28:47978) 2023-11-28T11:09:31.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c877fe700 1 -- 172.21.15.28:0/2701866336 learned_addr learned my addr 172.21.15.28:0/2701866336 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:31.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c87fff700 1 --2- 172.21.15.28:0/2701866336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c880ccbc0 0x7f8c880c5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c8ca51700 1 --2- 172.21.15.28:0/2701866336 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8c8815aa20 0x7f8c880cff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c877fe700 1 -- 172.21.15.28:0/2701866336 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8c8815aa20 msgr2=0x7f8c880cff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c877fe700 1 --2- 172.21.15.28:0/2701866336 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8c8815aa20 0x7f8c880cff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c877fe700 1 -- 172.21.15.28:0/2701866336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c880ccbc0 msgr2=0x7f8c880c5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:31.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c877fe700 1 --2- 172.21.15.28:0/2701866336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c880ccbc0 0x7f8c880c5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:31.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c877fe700 1 -- 172.21.15.28:0/2701866336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c80009a70 con 0x7f8c8815a650 2023-11-28T11:09:31.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.483+0000 7f8c877fe700 1 --2- 172.21.15.28:0/2701866336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8815a650 0x7f8c880cf9f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000c500 tx=0x7f8c7000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:31.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.484+0000 7f8c84ff9700 1 -- 172.21.15.28:0/2701866336 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70011790 con 0x7f8c8815a650 2023-11-28T11:09:31.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.484+0000 7f8c84ff9700 1 -- 172.21.15.28:0/2701866336 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c700118f0 con 0x7f8c8815a650 2023-11-28T11:09:31.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.484+0000 7f8c8e4b4700 1 -- 172.21.15.28:0/2701866336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c880b5500 con 0x7f8c8815a650 2023-11-28T11:09:31.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.484+0000 7f8c84ff9700 1 -- 172.21.15.28:0/2701866336 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c70011ca0 con 0x7f8c8815a650 2023-11-28T11:09:31.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.484+0000 7f8c8e4b4700 1 -- 172.21.15.28:0/2701866336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c880cabe0 con 0x7f8c8815a650 2023-11-28T11:09:31.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.484+0000 7f8c8e4b4700 1 -- 172.21.15.28:0/2701866336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c880b5690 con 0x7f8c8815a650 2023-11-28T11:09:31.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.485+0000 7f8c84ff9700 1 -- 172.21.15.28:0/2701866336 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8c70010050 con 0x7f8c8815a650 2023-11-28T11:09:31.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.485+0000 7f8c84ff9700 1 --2- 172.21.15.28:0/2701866336 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c680682f0 0x7f8c6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:31.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.486+0000 7f8c84ff9700 1 -- 172.21.15.28:0/2701866336 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f8c70085c80 con 0x7f8c8815a650 2023-11-28T11:09:31.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.486+0000 7f8c87fff700 1 --2- 172.21.15.28:0/2701866336 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c680682f0 0x7f8c6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:31.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.486+0000 7f8c87fff700 1 --2- 172.21.15.28:0/2701866336 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c680682f0 0x7f8c6806a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8c78000f60 tx=0x7f8c78007be0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:31.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.490+0000 7f8c84ff9700 1 -- 172.21.15.28:0/2701866336 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8c700521f0 con 0x7f8c8815a650 2023-11-28T11:09:31.766 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:31.765+0000 7f8c8e4b4700 1 -- 172.21.15.28:0/2701866336 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f8c880675a0 con 0x7f8c8815a650 2023-11-28T11:09:32.248 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.247+0000 7f8c84ff9700 1 -- 172.21.15.28:0/2701866336 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v61) v1 ==== 145+0+0 (secure 0 0 0) 0x7f8c70055810 con 0x7f8c8815a650 2023-11-28T11:09:32.251 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.249+0000 7f8c8e4b4700 1 -- 172.21.15.28:0/2701866336 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c680682f0 msgr2=0x7f8c6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.252 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.249+0000 7f8c8e4b4700 1 --2- 172.21.15.28:0/2701866336 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c680682f0 0x7f8c6806a7a0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f8c78000f60 tx=0x7f8c78007be0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.252 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.249+0000 7f8c8e4b4700 1 -- 172.21.15.28:0/2701866336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8815a650 msgr2=0x7f8c880cf9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.252 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.249+0000 7f8c8e4b4700 1 --2- 172.21.15.28:0/2701866336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8815a650 0x7f8c880cf9f0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f8c7000c500 tx=0x7f8c7000c8c0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.252 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.249+0000 7f8c8e4b4700 1 -- 172.21.15.28:0/2701866336 shutdown_connections 2023-11-28T11:09:32.252 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.249+0000 7f8c8e4b4700 1 --2- 172.21.15.28:0/2701866336 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c680682f0 0x7f8c6806a7a0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.252 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.249+0000 7f8c8e4b4700 1 --2- 172.21.15.28:0/2701866336 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8c8815aa20 0x7f8c880cff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.252 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.250+0000 7f8c8e4b4700 1 --2- 172.21.15.28:0/2701866336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c880ccbc0 0x7f8c880c5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.252 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.250+0000 7f8c8e4b4700 1 --2- 172.21.15.28:0/2701866336 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c8815a650 0x7f8c880cf9f0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.252 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.250+0000 7f8c8e4b4700 1 -- 172.21.15.28:0/2701866336 >> 172.21.15.28:0/2701866336 conn(0x7f8c880b1650 msgr2=0x7f8c8806ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:32.252 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.250+0000 7f8c8e4b4700 1 -- 172.21.15.28:0/2701866336 shutdown_connections 2023-11-28T11:09:32.252 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.250+0000 7f8c8e4b4700 1 -- 172.21.15.28:0/2701866336 wait complete. 2023-11-28T11:09:32.266 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:09:32.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.400+0000 7fefe62c9700 1 Processor -- start 2023-11-28T11:09:32.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.400+0000 7fefe62c9700 1 -- start start 2023-11-28T11:09:32.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.401+0000 7fefe62c9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fefe015c880 0x7fefe015d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.401+0000 7fefe62c9700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fefe00c9950 0x7fefe00cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.401+0000 7fefe62c9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefe00cae80 con 0x7fefe015c4b0 2023-11-28T11:09:32.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.401+0000 7fefe62c9700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefe015dba0 con 0x7fefe00c9950 2023-11-28T11:09:32.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.401+0000 7fefe62c9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe00c9de0 con 0x7fefe015c880 2023-11-28T11:09:32.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.401+0000 7fefe4866700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fefe00c9950 0x7fefe00cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.401+0000 7fefe4866700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fefe00c9950 0x7fefe00cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47988/0 (socket says 172.21.15.28:47988) 2023-11-28T11:09:32.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.401+0000 7fefe4866700 1 -- 172.21.15.28:0/260107744 learned_addr learned my addr 172.21.15.28:0/260107744 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:32.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.401+0000 7fefdf7fe700 1 --2- 172.21.15.28:0/260107744 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fefe015c880 0x7fefe015d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.401+0000 7fefe4866700 1 -- 172.21.15.28:0/260107744 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fefe015c880 msgr2=0x7fefe015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:32.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.401+0000 7fefe4866700 1 --2- 172.21.15.28:0/260107744 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fefe015c880 0x7fefe015d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.401+0000 7fefe4866700 1 -- 172.21.15.28:0/260107744 >> v1:172.21.15.59:6789/0 conn(0x7fefe015c4b0 legacy=0x7fefe006a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.401+0000 7fefe4866700 1 -- 172.21.15.28:0/260107744 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefe00ca0c0 con 0x7fefe00c9950 2023-11-28T11:09:32.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.401+0000 7fefe4866700 1 --2- 172.21.15.28:0/260107744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefe00c9950 0x7fefe00cced0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009e50 tx=0x7fefd800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=593850819b7b2db2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:32.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.402+0000 7fefdeffd700 1 -- 172.21.15.28:0/260107744 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd800e070 con 0x7fefe00c9950 2023-11-28T11:09:32.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.402+0000 7fefdeffd700 1 -- 172.21.15.28:0/260107744 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd800b910 con 0x7fefe00c9950 2023-11-28T11:09:32.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.402+0000 7fefdeffd700 1 -- 172.21.15.28:0/260107744 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd800bcc0 con 0x7fefe00c9950 2023-11-28T11:09:32.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.402+0000 7fefe62c9700 1 -- 172.21.15.28:0/260107744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefe00c9950 msgr2=0x7fefe00cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.402+0000 7fefe62c9700 1 --2- 172.21.15.28:0/260107744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefe00c9950 0x7fefe00cced0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fefd8009e50 tx=0x7fefd800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:32.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.402+0000 7fefe62c9700 1 -- 172.21.15.28:0/260107744 shutdown_connections 2023-11-28T11:09:32.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.402+0000 7fefe62c9700 1 --2- 172.21.15.28:0/260107744 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefe00c9950 0x7fefe00cced0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.402+0000 7fefe62c9700 1 --2- 172.21.15.28:0/260107744 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fefe015c880 0x7fefe015d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.402+0000 7fefe62c9700 1 -- 172.21.15.28:0/260107744 >> 172.21.15.28:0/260107744 conn(0x7fefe00b0df0 msgr2=0x7fefe00b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:32.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.402+0000 7fefe62c9700 1 -- 172.21.15.28:0/260107744 shutdown_connections 2023-11-28T11:09:32.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.402+0000 7fefe62c9700 1 -- 172.21.15.28:0/260107744 wait complete. 2023-11-28T11:09:32.409 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.404+0000 7fefe62c9700 1 Processor -- start 2023-11-28T11:09:32.409 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.404+0000 7fefe62c9700 1 -- start start 2023-11-28T11:09:32.409 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefe62c9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefe00c9950 0x7fefe00b2130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.409 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefe62c9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefe015c4b0 0x7fefe014e110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefe62c9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fefe015c880 0x7fefe00ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefe62c9700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe00acd00 con 0x7fefe015c4b0 2023-11-28T11:09:32.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefe62c9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefe014e650 con 0x7fefe00c9950 2023-11-28T11:09:32.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefe62c9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefe00ac0f0 con 0x7fefe015c880 2023-11-28T11:09:32.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefdf7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefe015c4b0 0x7fefe014e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefdf7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefe015c4b0 0x7fefe014e110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:47992/0 (socket says 172.21.15.28:47992) 2023-11-28T11:09:32.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefdf7fe700 1 -- 172.21.15.28:0/1954414829 learned_addr learned my addr 172.21.15.28:0/1954414829 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:32.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefe4866700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fefe015c880 0x7fefe00ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefdffff700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefe00c9950 0x7fefe00b2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefdf7fe700 1 -- 172.21.15.28:0/1954414829 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fefe015c880 msgr2=0x7fefe00ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefdf7fe700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fefe015c880 0x7fefe00ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefdf7fe700 1 -- 172.21.15.28:0/1954414829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefe00c9950 msgr2=0x7fefe00b2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefdf7fe700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefe00c9950 0x7fefe00b2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefdf7fe700 1 -- 172.21.15.28:0/1954414829 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd8009b30 con 0x7fefe015c4b0 2023-11-28T11:09:32.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefe4866700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fefe015c880 0x7fefe00ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:32.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.405+0000 7fefdffff700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefe00c9950 0x7fefe00b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:32.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.406+0000 7fefdf7fe700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefe015c4b0 0x7fefe014e110 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fefd000e870 tx=0x7fefd000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:32.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.406+0000 7fefdcff9700 1 -- 172.21.15.28:0/1954414829 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0009590 con 0x7fefe015c4b0 2023-11-28T11:09:32.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.406+0000 7fefdcff9700 1 -- 172.21.15.28:0/1954414829 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd000ee70 con 0x7fefe015c4b0 2023-11-28T11:09:32.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.406+0000 7fefe62c9700 1 -- 172.21.15.28:0/1954414829 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefe00ca680 con 0x7fefe015c4b0 2023-11-28T11:09:32.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.406+0000 7fefdcff9700 1 -- 172.21.15.28:0/1954414829 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd000ca50 con 0x7fefe015c4b0 2023-11-28T11:09:32.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.406+0000 7fefe62c9700 1 -- 172.21.15.28:0/1954414829 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefe00ca3a0 con 0x7fefe015c4b0 2023-11-28T11:09:32.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.407+0000 7fefe62c9700 1 -- 172.21.15.28:0/1954414829 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefe00ca960 con 0x7fefe015c4b0 2023-11-28T11:09:32.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.407+0000 7fefdcff9700 1 -- 172.21.15.28:0/1954414829 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fefd0023070 con 0x7fefe015c4b0 2023-11-28T11:09:32.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.408+0000 7fefdcff9700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fefcc0682f0 0x7fefcc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.408+0000 7fefdcff9700 1 -- 172.21.15.28:0/1954414829 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7fefd0086130 con 0x7fefe015c4b0 2023-11-28T11:09:32.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.408+0000 7fefdffff700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fefcc0682f0 0x7fefcc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.408+0000 7fefdffff700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fefcc0682f0 0x7fefcc06a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fefd4000f60 tx=0x7fefd4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:32.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.412+0000 7fefdcff9700 1 -- 172.21.15.28:0/1954414829 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fefd00526a0 con 0x7fefe015c4b0 2023-11-28T11:09:32.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.607+0000 7fefe62c9700 1 -- 172.21.15.28:0/1954414829 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fefe00739f0 con 0x7fefe015c4b0 2023-11-28T11:09:32.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.611+0000 7fefdcff9700 1 -- 172.21.15.28:0/1954414829 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7fefd0055cc0 con 0x7fefe015c4b0 2023-11-28T11:09:32.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.613+0000 7fefe62c9700 1 -- 172.21.15.28:0/1954414829 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fefcc0682f0 msgr2=0x7fefcc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.613+0000 7fefe62c9700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fefcc0682f0 0x7fefcc06a7a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fefd4000f60 tx=0x7fefd4009040 comp rx=0 tx=0).stop 2023-11-28T11:09:32.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.613+0000 7fefe62c9700 1 -- 172.21.15.28:0/1954414829 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefe015c4b0 msgr2=0x7fefe014e110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.613+0000 7fefe62c9700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefe015c4b0 0x7fefe014e110 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fefd000e870 tx=0x7fefd000ec30 comp rx=0 tx=0).stop 2023-11-28T11:09:32.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.613+0000 7fefe62c9700 1 -- 172.21.15.28:0/1954414829 shutdown_connections 2023-11-28T11:09:32.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.613+0000 7fefe62c9700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fefcc0682f0 0x7fefcc06a7a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.613+0000 7fefe62c9700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fefe015c880 0x7fefe00ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.613+0000 7fefe62c9700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefe00c9950 0x7fefe00b2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.613+0000 7fefe62c9700 1 --2- 172.21.15.28:0/1954414829 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fefe015c4b0 0x7fefe014e110 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.613+0000 7fefe62c9700 1 -- 172.21.15.28:0/1954414829 >> 172.21.15.28:0/1954414829 conn(0x7fefe00b0df0 msgr2=0x7fefe006a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:32.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.613+0000 7fefe62c9700 1 -- 172.21.15.28:0/1954414829 shutdown_connections 2023-11-28T11:09:32.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.613+0000 7fefe62c9700 1 -- 172.21.15.28:0/1954414829 wait complete. 2023-11-28T11:09:32.626 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:09:32.627 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:09:32.663 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:09:32.711 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:09:32.711 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:09:32.711 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:32.729 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:09:32.729 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:32.788 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:32.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.950+0000 7f42dfe4b700 1 Processor -- start 2023-11-28T11:09:32.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.951+0000 7f42dfe4b700 1 -- start start 2023-11-28T11:09:32.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.951+0000 7f42dfe4b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f42d80cb230 0x7f42d80ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.951+0000 7f42dfe4b700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d80cfb90 con 0x7f42d815a6c0 2023-11-28T11:09:32.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dfe4b700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42d80cb680 con 0x7f42d80b2660 2023-11-28T11:09:32.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dfe4b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42d80cb930 con 0x7f42d80cb230 2023-11-28T11:09:32.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42ddbe7700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f42d815a6c0 0x7f42d80d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36056/0 (socket says 172.21.15.28:36056) 2023-11-28T11:09:32.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42ddbe7700 1 -- 172.21.15.28:0/3214956210 learned_addr learned my addr 172.21.15.28:0/3214956210 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:32.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dd3e6700 1 --2- 172.21.15.28:0/3214956210 >> v2:172.21.15.59:3300/0 conn(0x7f42d80cb230 0x7f42d80ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dcbe5700 1 -- 172.21.15.28:0/3214956210 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3639025447 0 0) 0x7f42d80cfb90 con 0x7f42d815a6c0 2023-11-28T11:09:32.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dcbe5700 1 -- 172.21.15.28:0/3214956210 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42c40035b0 con 0x7f42d815a6c0 2023-11-28T11:09:32.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dcbe5700 1 -- 172.21.15.28:0/3214956210 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4056086345 0 0) 0x7f42c40035b0 con 0x7f42d815a6c0 2023-11-28T11:09:32.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dcbe5700 1 -- 172.21.15.28:0/3214956210 >> v2:172.21.15.59:3300/0 conn(0x7f42d80cb230 msgr2=0x7f42d80ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:32.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dcbe5700 1 --2- 172.21.15.28:0/3214956210 >> v2:172.21.15.59:3300/0 conn(0x7f42d80cb230 0x7f42d80ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dcbe5700 1 -- 172.21.15.28:0/3214956210 >> v1:172.21.15.59:6789/0 conn(0x7f42d80b2660 legacy=0x7f42d80b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dcbe5700 1 -- 172.21.15.28:0/3214956210 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d80cf260 con 0x7f42d815a6c0 2023-11-28T11:09:32.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dcbe5700 1 -- 172.21.15.28:0/3214956210 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f42d4001670 con 0x7f42d815a6c0 2023-11-28T11:09:32.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dcbe5700 1 -- 172.21.15.28:0/3214956210 >> v1:172.21.15.28:6789/0 conn(0x7f42d815a6c0 legacy=0x7f42d80d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dcbe5700 1 --2- 172.21.15.28:0/3214956210 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42c4003a00 0x7f42c4003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dcbe5700 1 --2- 172.21.15.28:0/3214956210 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c40043f0 0x7f42c40048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dcbe5700 1 --2- 172.21.15.28:0/3214956210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42c40050b0 0x7f42c400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dcbe5700 1 -- 172.21.15.28:0/3214956210 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80cb930 con 0x7f42c40043f0 2023-11-28T11:09:32.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dcbe5700 1 -- 172.21.15.28:0/3214956210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c400bf50 con 0x7f42c40050b0 2023-11-28T11:09:32.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dcbe5700 1 -- 172.21.15.28:0/3214956210 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c400c110 con 0x7f42c4003a00 2023-11-28T11:09:32.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.952+0000 7f42dd3e6700 1 --2- 172.21.15.28:0/3214956210 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c40043f0 0x7f42c40048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.953+0000 7f42ddbe7700 1 --2- 172.21.15.28:0/3214956210 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42c4003a00 0x7f42c4003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.953+0000 7f42de3e8700 1 --2- 172.21.15.28:0/3214956210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42c40050b0 0x7f42c400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.953+0000 7f42dd3e6700 1 -- 172.21.15.28:0/3214956210 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42c4003a00 msgr2=0x7f42c4003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:32.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.953+0000 7f42dd3e6700 1 --2- 172.21.15.28:0/3214956210 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42c4003a00 0x7f42c4003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.953+0000 7f42dd3e6700 1 -- 172.21.15.28:0/3214956210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42c40050b0 msgr2=0x7f42c400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:32.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.953+0000 7f42dd3e6700 1 --2- 172.21.15.28:0/3214956210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42c40050b0 0x7f42c400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.953+0000 7f42dd3e6700 1 -- 172.21.15.28:0/3214956210 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c40035b0 con 0x7f42c40043f0 2023-11-28T11:09:32.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.953+0000 7f42de3e8700 1 --2- 172.21.15.28:0/3214956210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42c40050b0 0x7f42c400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:32.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.953+0000 7f42ddbe7700 1 --2- 172.21.15.28:0/3214956210 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42c4003a00 0x7f42c4003dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:32.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.953+0000 7f42de3e8700 1 -- 172.21.15.28:0/3214956210 reap_dead start 2023-11-28T11:09:32.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.953+0000 7f42dd3e6700 1 --2- 172.21.15.28:0/3214956210 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c40043f0 0x7f42c40048b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009b60 tx=0x7f42c4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feb8cabdb12dac57 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:32.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.953+0000 7f42dcbe5700 1 -- 172.21.15.28:0/3214956210 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc00e6c0 con 0x7f42c40043f0 2023-11-28T11:09:32.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.953+0000 7f42dcbe5700 1 -- 172.21.15.28:0/3214956210 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42cc00e820 con 0x7f42c40043f0 2023-11-28T11:09:32.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.953+0000 7f42dcbe5700 1 -- 172.21.15.28:0/3214956210 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42cc00eb90 con 0x7f42c40043f0 2023-11-28T11:09:32.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.954+0000 7f42dfe4b700 1 -- 172.21.15.28:0/3214956210 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c40043f0 msgr2=0x7f42c40048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.954+0000 7f42dfe4b700 1 --2- 172.21.15.28:0/3214956210 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c40043f0 0x7f42c40048b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f42cc009b60 tx=0x7f42c4005880 comp rx=0 tx=0).stop 2023-11-28T11:09:32.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.954+0000 7f42dfe4b700 1 -- 172.21.15.28:0/3214956210 shutdown_connections 2023-11-28T11:09:32.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.954+0000 7f42dfe4b700 1 --2- 172.21.15.28:0/3214956210 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c40043f0 0x7f42c40048b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.954+0000 7f42dfe4b700 1 -- 172.21.15.28:0/3214956210 >> 172.21.15.28:0/3214956210 conn(0x7f42d80cd660 msgr2=0x7f42d80cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:32.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.954+0000 7f42dfe4b700 1 -- 172.21.15.28:0/3214956210 shutdown_connections 2023-11-28T11:09:32.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.954+0000 7f42dfe4b700 1 -- 172.21.15.28:0/3214956210 wait complete. 2023-11-28T11:09:32.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.955+0000 7f42dfe4b700 1 Processor -- start 2023-11-28T11:09:32.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.955+0000 7f42dfe4b700 1 -- start start 2023-11-28T11:09:32.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42dfe4b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c40043f0 0x7f42d815cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42dfe4b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42d80b1810 0x7f42d80c8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42dfe4b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42d80b1c80 0x7f42d80c9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42dfe4b700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80c9b50 con 0x7f42c40043f0 2023-11-28T11:09:32.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42dfe4b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42d81564a0 con 0x7f42d80b1c80 2023-11-28T11:09:32.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42dfe4b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42d80c9870 con 0x7f42d80b1810 2023-11-28T11:09:32.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42ddbe7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c40043f0 0x7f42d815cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42ddbe7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c40043f0 0x7f42d815cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48010/0 (socket says 172.21.15.28:48010) 2023-11-28T11:09:32.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42ddbe7700 1 -- 172.21.15.28:0/3593657346 learned_addr learned my addr 172.21.15.28:0/3593657346 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:32.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42ddbe7700 1 -- 172.21.15.28:0/3593657346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42d80b1810 msgr2=0x7f42d80c8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42ddbe7700 1 --2- 172.21.15.28:0/3593657346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42d80b1810 0x7f42d80c8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42ddbe7700 1 -- 172.21.15.28:0/3593657346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42d80b1c80 msgr2=0x7f42d80c9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:32.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42ddbe7700 1 --2- 172.21.15.28:0/3593657346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42d80b1c80 0x7f42d80c9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:32.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42ddbe7700 1 -- 172.21.15.28:0/3593657346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d80cf820 con 0x7f42c40043f0 2023-11-28T11:09:32.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42ddbe7700 1 --2- 172.21.15.28:0/3593657346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c40043f0 0x7f42d815cf10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f42d4001bd0 tx=0x7f42d400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:32.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42ca7fc700 1 -- 172.21.15.28:0/3593657346 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d4005e20 con 0x7f42c40043f0 2023-11-28T11:09:32.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42ca7fc700 1 -- 172.21.15.28:0/3593657346 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d400c700 con 0x7f42c40043f0 2023-11-28T11:09:32.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42dfe4b700 1 -- 172.21.15.28:0/3593657346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42d80cf540 con 0x7f42c40043f0 2023-11-28T11:09:32.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42ca7fc700 1 -- 172.21.15.28:0/3593657346 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d400ca70 con 0x7f42c40043f0 2023-11-28T11:09:32.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.956+0000 7f42dfe4b700 1 -- 172.21.15.28:0/3593657346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42d80b2ae0 con 0x7f42c40043f0 2023-11-28T11:09:32.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.957+0000 7f42dfe4b700 1 -- 172.21.15.28:0/3593657346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42d80675a0 con 0x7f42c40043f0 2023-11-28T11:09:32.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.958+0000 7f42ca7fc700 1 -- 172.21.15.28:0/3593657346 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f42d400cbd0 con 0x7f42c40043f0 2023-11-28T11:09:32.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.958+0000 7f42ca7fc700 1 --2- 172.21.15.28:0/3593657346 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42bc0682f0 0x7f42bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:32.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.958+0000 7f42ca7fc700 1 -- 172.21.15.28:0/3593657346 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f42d4085e60 con 0x7f42c40043f0 2023-11-28T11:09:32.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.958+0000 7f42dd3e6700 1 --2- 172.21.15.28:0/3593657346 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42bc0682f0 0x7f42bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:32.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.959+0000 7f42dd3e6700 1 --2- 172.21.15.28:0/3593657346 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42bc0682f0 0x7f42bc06a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0090b0 tx=0x7f42cc013000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:32.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:32.963+0000 7f42ca7fc700 1 -- 172.21.15.28:0/3593657346 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f42d40523d0 con 0x7f42c40043f0 2023-11-28T11:09:33.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.155+0000 7f42dfe4b700 1 -- 172.21.15.28:0/3593657346 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42d80ceb50 con 0x7f42c40043f0 2023-11-28T11:09:33.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.157+0000 7f42ca7fc700 1 -- 172.21.15.28:0/3593657346 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v25) v1 ==== 74+0+19642 (secure 0 0 0) 0x7f42d400ce60 con 0x7f42c40043f0 2023-11-28T11:09:33.159 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:33.159 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:08:50.531733+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:33.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.159+0000 7f42dfe4b700 1 -- 172.21.15.28:0/3593657346 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42bc0682f0 msgr2=0x7f42bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.159+0000 7f42dfe4b700 1 --2- 172.21.15.28:0/3593657346 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42bc0682f0 0x7f42bc06a7a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f42cc0090b0 tx=0x7f42cc013000 comp rx=0 tx=0).stop 2023-11-28T11:09:33.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.159+0000 7f42dfe4b700 1 -- 172.21.15.28:0/3593657346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c40043f0 msgr2=0x7f42d815cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.159+0000 7f42dfe4b700 1 --2- 172.21.15.28:0/3593657346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c40043f0 0x7f42d815cf10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f42d4001bd0 tx=0x7f42d400ee80 comp rx=0 tx=0).stop 2023-11-28T11:09:33.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.159+0000 7f42dfe4b700 1 -- 172.21.15.28:0/3593657346 shutdown_connections 2023-11-28T11:09:33.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.159+0000 7f42dfe4b700 1 --2- 172.21.15.28:0/3593657346 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42bc0682f0 0x7f42bc06a7a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.159+0000 7f42dfe4b700 1 --2- 172.21.15.28:0/3593657346 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42c40043f0 0x7f42d815cf10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.159+0000 7f42dfe4b700 1 --2- 172.21.15.28:0/3593657346 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42d80b1c80 0x7f42d80c9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.159+0000 7f42dfe4b700 1 --2- 172.21.15.28:0/3593657346 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42d80b1810 0x7f42d80c8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.162 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.159+0000 7f42dfe4b700 1 -- 172.21.15.28:0/3593657346 >> 172.21.15.28:0/3593657346 conn(0x7f42d80cd660 msgr2=0x7f42d80cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:33.162 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.159+0000 7f42dfe4b700 1 -- 172.21.15.28:0/3593657346 shutdown_connections 2023-11-28T11:09:33.162 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.159+0000 7f42dfe4b700 1 -- 172.21.15.28:0/3593657346 wait complete. 2023-11-28T11:09:33.173 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:33.173 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:33.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.301+0000 7f83a23ea700 1 Processor -- start 2023-11-28T11:09:33.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.301+0000 7f83a23ea700 1 -- start start 2023-11-28T11:09:33.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.302+0000 7f83a23ea700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f839c06dd40 0x7f839c06f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.302+0000 7f83a23ea700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f839c15a860 con 0x7f839c15a490 2023-11-28T11:09:33.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.302+0000 7f83a23ea700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f839c06b510 con 0x7f839c0b2150 2023-11-28T11:09:33.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.302+0000 7f83a23ea700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f839c0c9a80 con 0x7f839c06dd40 2023-11-28T11:09:33.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.302+0000 7f839bfff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f839c15a490 0x7f839c06d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36080/0 (socket says 172.21.15.28:36080) 2023-11-28T11:09:33.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.302+0000 7f839bfff700 1 -- 172.21.15.28:0/3933287320 learned_addr learned my addr 172.21.15.28:0/3933287320 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:33.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.302+0000 7f83a0987700 1 --2- 172.21.15.28:0/3933287320 >> v2:172.21.15.59:3300/0 conn(0x7f839c06dd40 0x7f839c06f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.302+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329153236 0 0) 0x7f839c15a860 con 0x7f839c15a490 2023-11-28T11:09:33.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.302+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83880035b0 con 0x7f839c15a490 2023-11-28T11:09:33.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 420620686 0 0) 0x7f839c06b510 con 0x7f839c0b2150 2023-11-28T11:09:33.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f839c15a860 con 0x7f839c0b2150 2023-11-28T11:09:33.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4070377466 0 0) 0x7f83880035b0 con 0x7f839c15a490 2023-11-28T11:09:33.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 >> v2:172.21.15.59:3300/0 conn(0x7f839c06dd40 msgr2=0x7f839c06f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:33.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839affd700 1 --2- 172.21.15.28:0/3933287320 >> v2:172.21.15.59:3300/0 conn(0x7f839c06dd40 0x7f839c06f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 >> v1:172.21.15.59:6789/0 conn(0x7f839c0b2150 legacy=0x7f839c0ca520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c06b230 con 0x7f839c15a490 2023-11-28T11:09:33.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f838c001670 con 0x7f839c15a490 2023-11-28T11:09:33.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 >> v1:172.21.15.28:6789/0 conn(0x7f839c15a490 legacy=0x7f839c06d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839affd700 1 --2- 172.21.15.28:0/3933287320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8388003b90 0x7f8388003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839affd700 1 --2- 172.21.15.28:0/3933287320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8388004850 0x7f8388004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839affd700 1 --2- 172.21.15.28:0/3933287320 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8388005200 0x7f838800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c0c9a80 con 0x7f8388004850 2023-11-28T11:09:33.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838800c110 con 0x7f8388003b90 2023-11-28T11:09:33.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838800c2d0 con 0x7f8388005200 2023-11-28T11:09:33.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839b7fe700 1 --2- 172.21.15.28:0/3933287320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8388004850 0x7f8388004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839bfff700 1 --2- 172.21.15.28:0/3933287320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8388003b90 0x7f8388003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f83a0987700 1 --2- 172.21.15.28:0/3933287320 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8388005200 0x7f838800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839b7fe700 1 -- 172.21.15.28:0/3933287320 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8388005200 msgr2=0x7f838800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:33.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839b7fe700 1 --2- 172.21.15.28:0/3933287320 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8388005200 0x7f838800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839b7fe700 1 -- 172.21.15.28:0/3933287320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8388003b90 msgr2=0x7f8388003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:33.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839b7fe700 1 --2- 172.21.15.28:0/3933287320 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8388003b90 0x7f8388003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839b7fe700 1 -- 172.21.15.28:0/3933287320 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83880035b0 con 0x7f8388004850 2023-11-28T11:09:33.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f83a0987700 1 --2- 172.21.15.28:0/3933287320 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8388005200 0x7f838800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:33.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f83a0987700 1 -- 172.21.15.28:0/3933287320 reap_dead start 2023-11-28T11:09:33.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.303+0000 7f839b7fe700 1 --2- 172.21.15.28:0/3933287320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8388004850 0x7f8388004cc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8384007fd0 tx=0x7f8388005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de7f2865e2715433 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:33.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.304+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384001430 con 0x7f8388004850 2023-11-28T11:09:33.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.304+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83840052d0 con 0x7f8388004850 2023-11-28T11:09:33.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.304+0000 7f839affd700 1 -- 172.21.15.28:0/3933287320 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8384001d40 con 0x7f8388004850 2023-11-28T11:09:33.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.304+0000 7f83a23ea700 1 -- 172.21.15.28:0/3933287320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8388004850 msgr2=0x7f8388004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.304+0000 7f83a23ea700 1 --2- 172.21.15.28:0/3933287320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8388004850 0x7f8388004cc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f8384007fd0 tx=0x7f8388005880 comp rx=0 tx=0).stop 2023-11-28T11:09:33.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.304+0000 7f83a23ea700 1 -- 172.21.15.28:0/3933287320 shutdown_connections 2023-11-28T11:09:33.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.304+0000 7f83a23ea700 1 --2- 172.21.15.28:0/3933287320 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8388004850 0x7f8388004cc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.304+0000 7f83a23ea700 1 -- 172.21.15.28:0/3933287320 >> 172.21.15.28:0/3933287320 conn(0x7f839c0b0df0 msgr2=0x7f839c0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:33.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.304+0000 7f83a23ea700 1 -- 172.21.15.28:0/3933287320 shutdown_connections 2023-11-28T11:09:33.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.304+0000 7f83a23ea700 1 -- 172.21.15.28:0/3933287320 wait complete. 2023-11-28T11:09:33.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.305+0000 7f83a23ea700 1 Processor -- start 2023-11-28T11:09:33.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.306+0000 7f83a23ea700 1 -- start start 2023-11-28T11:09:33.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.306+0000 7f83a23ea700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8388004850 0x7f839c1f0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.306+0000 7f83a23ea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f839c0af720 0x7f839c1f26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.306+0000 7f83a23ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839c0c8980 0x7f839c0c8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.306+0000 7f83a23ea700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c1562c0 con 0x7f8388004850 2023-11-28T11:09:33.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.306+0000 7f83a23ea700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f839c0c9360 con 0x7f839c0c8980 2023-11-28T11:09:33.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.306+0000 7f83a23ea700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f839c1f0e70 con 0x7f839c0af720 2023-11-28T11:09:33.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.306+0000 7f839bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8388004850 0x7f839c1f0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.307+0000 7f839bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8388004850 0x7f839c1f0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48034/0 (socket says 172.21.15.28:48034) 2023-11-28T11:09:33.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.307+0000 7f839bfff700 1 -- 172.21.15.28:0/3536851654 learned_addr learned my addr 172.21.15.28:0/3536851654 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:33.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.307+0000 7f839b7fe700 1 --2- 172.21.15.28:0/3536851654 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f839c0af720 0x7f839c1f26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.307+0000 7f83a0987700 1 --2- 172.21.15.28:0/3536851654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839c0c8980 0x7f839c0c8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.307+0000 7f839bfff700 1 -- 172.21.15.28:0/3536851654 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f839c0af720 msgr2=0x7f839c1f26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.307+0000 7f839bfff700 1 --2- 172.21.15.28:0/3536851654 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f839c0af720 0x7f839c1f26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.307+0000 7f839bfff700 1 -- 172.21.15.28:0/3536851654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839c0c8980 msgr2=0x7f839c0c8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.307+0000 7f839bfff700 1 --2- 172.21.15.28:0/3536851654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839c0c8980 0x7f839c0c8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.307+0000 7f839bfff700 1 -- 172.21.15.28:0/3536851654 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f839c0ca000 con 0x7f8388004850 2023-11-28T11:09:33.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.307+0000 7f839bfff700 1 --2- 172.21.15.28:0/3536851654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8388004850 0x7f839c1f0930 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f838c007340 tx=0x7f838c007650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:33.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.307+0000 7f8398ff9700 1 -- 172.21.15.28:0/3536851654 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c001090 con 0x7f8388004850 2023-11-28T11:09:33.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.307+0000 7f8398ff9700 1 -- 172.21.15.28:0/3536851654 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f838c00f660 con 0x7f8388004850 2023-11-28T11:09:33.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.307+0000 7f8398ff9700 1 -- 172.21.15.28:0/3536851654 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838c00f9f0 con 0x7f8388004850 2023-11-28T11:09:33.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.307+0000 7f83a23ea700 1 -- 172.21.15.28:0/3536851654 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f839c0c9d20 con 0x7f8388004850 2023-11-28T11:09:33.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.307+0000 7f83a23ea700 1 -- 172.21.15.28:0/3536851654 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f839c06ae90 con 0x7f8388004850 2023-11-28T11:09:33.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.308+0000 7f83a23ea700 1 -- 172.21.15.28:0/3536851654 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f839c156410 con 0x7f8388004850 2023-11-28T11:09:33.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.309+0000 7f8398ff9700 1 -- 172.21.15.28:0/3536851654 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f838c00fb50 con 0x7f8388004850 2023-11-28T11:09:33.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.309+0000 7f8398ff9700 1 --2- 172.21.15.28:0/3536851654 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f837c0682f0 0x7f837c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.309+0000 7f8398ff9700 1 -- 172.21.15.28:0/3536851654 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 6813+0+0 (secure 0 0 0) 0x7f838c085f30 con 0x7f8388004850 2023-11-28T11:09:33.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.309+0000 7f839b7fe700 1 --2- 172.21.15.28:0/3536851654 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f837c0682f0 0x7f837c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.309+0000 7f839b7fe700 1 --2- 172.21.15.28:0/3536851654 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f837c0682f0 0x7f837c06a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8384000c00 tx=0x7f838400a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:33.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.313+0000 7f8398ff9700 1 -- 172.21.15.28:0/3536851654 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f839c156410 con 0x7f8388004850 2023-11-28T11:09:33.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.505+0000 7f83a23ea700 1 -- 172.21.15.28:0/3536851654 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f839c156410 con 0x7f8388004850 2023-11-28T11:09:33.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.505+0000 7f8398ff9700 1 -- 172.21.15.28:0/3536851654 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v25) v1 ==== 92+0+10 (secure 0 0 0) 0x7f839c156410 con 0x7f8388004850 2023-11-28T11:09:33.507 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:09:33.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.508+0000 7f83a23ea700 1 -- 172.21.15.28:0/3536851654 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f837c0682f0 msgr2=0x7f837c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.508+0000 7f83a23ea700 1 --2- 172.21.15.28:0/3536851654 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f837c0682f0 0x7f837c06a7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8384000c00 tx=0x7f838400a000 comp rx=0 tx=0).stop 2023-11-28T11:09:33.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.508+0000 7f83a23ea700 1 -- 172.21.15.28:0/3536851654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8388004850 msgr2=0x7f839c1f0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.508+0000 7f83a23ea700 1 --2- 172.21.15.28:0/3536851654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8388004850 0x7f839c1f0930 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f838c007340 tx=0x7f838c007650 comp rx=0 tx=0).stop 2023-11-28T11:09:33.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.508+0000 7f83a23ea700 1 -- 172.21.15.28:0/3536851654 shutdown_connections 2023-11-28T11:09:33.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.508+0000 7f83a23ea700 1 --2- 172.21.15.28:0/3536851654 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f837c0682f0 0x7f837c06a7a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.508+0000 7f83a23ea700 1 --2- 172.21.15.28:0/3536851654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8388004850 0x7f839c1f0930 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.508+0000 7f83a23ea700 1 --2- 172.21.15.28:0/3536851654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f839c0c8980 0x7f839c0c8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.508+0000 7f83a23ea700 1 --2- 172.21.15.28:0/3536851654 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f839c0af720 0x7f839c1f26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.508+0000 7f83a23ea700 1 -- 172.21.15.28:0/3536851654 >> 172.21.15.28:0/3536851654 conn(0x7f839c0b0df0 msgr2=0x7f839c06d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:33.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.508+0000 7f83a23ea700 1 -- 172.21.15.28:0/3536851654 shutdown_connections 2023-11-28T11:09:33.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.508+0000 7f83a23ea700 1 -- 172.21.15.28:0/3536851654 wait complete. 2023-11-28T11:09:33.522 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:33.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.651+0000 7fdc068dd700 1 Processor -- start 2023-11-28T11:09:33.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.652+0000 7fdc068dd700 1 -- start start 2023-11-28T11:09:33.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.652+0000 7fdc068dd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc00159500 0x7fdc000cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.652+0000 7fdc068dd700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc0015e040 con 0x7fdc0015dbf0 2023-11-28T11:09:33.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.652+0000 7fdc068dd700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc000ceff0 con 0x7fdc000b1d40 2023-11-28T11:09:33.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.652+0000 7fdc068dd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000cf2d0 con 0x7fdc00159500 2023-11-28T11:09:33.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.652+0000 7fdbff7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdc0015dbf0 0x7fdc000b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36100/0 (socket says 172.21.15.28:36100) 2023-11-28T11:09:33.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.652+0000 7fdbff7fe700 1 -- 172.21.15.28:0/2662674633 learned_addr learned my addr 172.21.15.28:0/2662674633 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:33.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfeffd700 1 -- 172.21.15.28:0/2662674633 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1837099279 0 0) 0x7fdc0015e040 con 0x7fdc0015dbf0 2023-11-28T11:09:33.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfeffd700 1 -- 172.21.15.28:0/2662674633 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdbe80035b0 con 0x7fdc0015dbf0 2023-11-28T11:09:33.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfffff700 1 --2- 172.21.15.28:0/2662674633 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc00159500 0x7fdc000cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfeffd700 1 -- 172.21.15.28:0/2662674633 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2562341276 0 0) 0x7fdbe80035b0 con 0x7fdc0015dbf0 2023-11-28T11:09:33.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfeffd700 1 -- 172.21.15.28:0/2662674633 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc00159500 msgr2=0x7fdc000cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:33.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfeffd700 1 --2- 172.21.15.28:0/2662674633 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc00159500 0x7fdc000cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfeffd700 1 -- 172.21.15.28:0/2662674633 >> v1:172.21.15.59:6789/0 conn(0x7fdc000b1d40 legacy=0x7fdc0006bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfeffd700 1 -- 172.21.15.28:0/2662674633 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0015d110 con 0x7fdc0015dbf0 2023-11-28T11:09:33.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfeffd700 1 -- 172.21.15.28:0/2662674633 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fdbf0001670 con 0x7fdc0015dbf0 2023-11-28T11:09:33.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfeffd700 1 -- 172.21.15.28:0/2662674633 >> v1:172.21.15.28:6789/0 conn(0x7fdc0015dbf0 legacy=0x7fdc000b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfeffd700 1 --2- 172.21.15.28:0/2662674633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbe8003a30 0x7fdbe8003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfeffd700 1 --2- 172.21.15.28:0/2662674633 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc00159500 0x7fdbe8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfeffd700 1 --2- 172.21.15.28:0/2662674633 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbe8004d90 0x7fdbe8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfeffd700 1 -- 172.21.15.28:0/2662674633 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000cf2d0 con 0x7fdbe8004d90 2023-11-28T11:09:33.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfeffd700 1 -- 172.21.15.28:0/2662674633 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe800bb50 con 0x7fdbe8003a30 2023-11-28T11:09:33.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfeffd700 1 -- 172.21.15.28:0/2662674633 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbe800bce0 con 0x7fdc00159500 2023-11-28T11:09:33.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdc04e7a700 1 --2- 172.21.15.28:0/2662674633 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbe8004d90 0x7fdbe8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbfffff700 1 --2- 172.21.15.28:0/2662674633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbe8003a30 0x7fdbe8003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.653+0000 7fdbff7fe700 1 --2- 172.21.15.28:0/2662674633 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc00159500 0x7fdbe8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.654+0000 7fdc04e7a700 1 -- 172.21.15.28:0/2662674633 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc00159500 msgr2=0x7fdbe8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:33.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.654+0000 7fdc04e7a700 1 --2- 172.21.15.28:0/2662674633 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc00159500 0x7fdbe8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.654+0000 7fdc04e7a700 1 -- 172.21.15.28:0/2662674633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbe8003a30 msgr2=0x7fdbe8003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:33.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.654+0000 7fdc04e7a700 1 --2- 172.21.15.28:0/2662674633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbe8003a30 0x7fdbe8003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.654+0000 7fdc04e7a700 1 -- 172.21.15.28:0/2662674633 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe80035b0 con 0x7fdbe8004d90 2023-11-28T11:09:33.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.654+0000 7fdc04e7a700 1 --2- 172.21.15.28:0/2662674633 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbe8004d90 0x7fdbe8005200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800c240 tx=0x7fdbe8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1afed2e233c80df server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:33.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.654+0000 7fdbfeffd700 1 -- 172.21.15.28:0/2662674633 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8001df0 con 0x7fdbe8004d90 2023-11-28T11:09:33.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.654+0000 7fdbfeffd700 1 -- 172.21.15.28:0/2662674633 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf800cda0 con 0x7fdbe8004d90 2023-11-28T11:09:33.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.654+0000 7fdbfeffd700 1 -- 172.21.15.28:0/2662674633 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8003c40 con 0x7fdbe8004d90 2023-11-28T11:09:33.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.654+0000 7fdc068dd700 1 -- 172.21.15.28:0/2662674633 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbe8004d90 msgr2=0x7fdbe8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.654+0000 7fdc068dd700 1 --2- 172.21.15.28:0/2662674633 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbe8004d90 0x7fdbe8005200 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800c240 tx=0x7fdbe8005880 comp rx=0 tx=0).stop 2023-11-28T11:09:33.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.654+0000 7fdc04e7a700 1 -- 172.21.15.28:0/2662674633 reap_dead start 2023-11-28T11:09:33.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.654+0000 7fdc068dd700 1 -- 172.21.15.28:0/2662674633 shutdown_connections 2023-11-28T11:09:33.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.654+0000 7fdc068dd700 1 -- 172.21.15.28:0/2662674633 >> 172.21.15.28:0/2662674633 conn(0x7fdc000cc630 msgr2=0x7fdc000cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:33.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.654+0000 7fdc068dd700 1 -- 172.21.15.28:0/2662674633 shutdown_connections 2023-11-28T11:09:33.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.655+0000 7fdc068dd700 1 -- 172.21.15.28:0/2662674633 wait complete. 2023-11-28T11:09:33.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.656+0000 7fdc068dd700 1 Processor -- start 2023-11-28T11:09:33.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.656+0000 7fdc068dd700 1 -- start start 2023-11-28T11:09:33.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdc068dd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc00144d50 0x7fdc000b0350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdc068dd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc00145120 0x7fdc000b0890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdc068dd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc000b45d0 0x7fdc000b4a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdc068dd700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc00142850 con 0x7fdc00144d50 2023-11-28T11:09:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdc068dd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000b5850 con 0x7fdc00145120 2023-11-28T11:09:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdc068dd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc000b5290 con 0x7fdc000b45d0 2023-11-28T11:09:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdbfffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc00144d50 0x7fdc000b0350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdbfffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc00144d50 0x7fdc000b0350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48052/0 (socket says 172.21.15.28:48052) 2023-11-28T11:09:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdbfffff700 1 -- 172.21.15.28:0/3558054659 learned_addr learned my addr 172.21.15.28:0/3558054659 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdc04e7a700 1 --2- 172.21.15.28:0/3558054659 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc000b45d0 0x7fdc000b4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdbff7fe700 1 --2- 172.21.15.28:0/3558054659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc00145120 0x7fdc000b0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdbfffff700 1 -- 172.21.15.28:0/3558054659 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc000b45d0 msgr2=0x7fdc000b4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdbfffff700 1 --2- 172.21.15.28:0/3558054659 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc000b45d0 0x7fdc000b4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdbfffff700 1 -- 172.21.15.28:0/3558054659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc00145120 msgr2=0x7fdc000b0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdbfffff700 1 --2- 172.21.15.28:0/3558054659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc00145120 0x7fdc000b0890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdbfffff700 1 -- 172.21.15.28:0/3558054659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0015d6d0 con 0x7fdc00144d50 2023-11-28T11:09:33.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdbff7fe700 1 --2- 172.21.15.28:0/3558054659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc00145120 0x7fdc000b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:33.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdbfffff700 1 --2- 172.21.15.28:0/3558054659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc00144d50 0x7fdc000b0350 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4011aa0 tx=0x7fdbf4011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:33.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdbfcff9700 1 -- 172.21.15.28:0/3558054659 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf400f940 con 0x7fdc00144d50 2023-11-28T11:09:33.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdbfcff9700 1 -- 172.21.15.28:0/3558054659 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf400faa0 con 0x7fdc00144d50 2023-11-28T11:09:33.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdc068dd700 1 -- 172.21.15.28:0/3558054659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0015d3f0 con 0x7fdc00144d50 2023-11-28T11:09:33.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdbfcff9700 1 -- 172.21.15.28:0/3558054659 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf400fe50 con 0x7fdc00144d50 2023-11-28T11:09:33.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.657+0000 7fdc068dd700 1 -- 172.21.15.28:0/3558054659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc001598d0 con 0x7fdc00144d50 2023-11-28T11:09:33.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.658+0000 7fdc068dd700 1 -- 172.21.15.28:0/3558054659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc00142b30 con 0x7fdc00144d50 2023-11-28T11:09:33.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.659+0000 7fdbfcff9700 1 -- 172.21.15.28:0/3558054659 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdbf4014440 con 0x7fdc00144d50 2023-11-28T11:09:33.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.659+0000 7fdbfcff9700 1 --2- 172.21.15.28:0/3558054659 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdbec0682f0 0x7fdbec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:33.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.659+0000 7fdbfcff9700 1 -- 172.21.15.28:0/3558054659 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdbf40869f0 con 0x7fdc00144d50 2023-11-28T11:09:33.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.659+0000 7fdbff7fe700 1 --2- 172.21.15.28:0/3558054659 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdbec0682f0 0x7fdbec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:33.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.660+0000 7fdbff7fe700 1 --2- 172.21.15.28:0/3558054659 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdbec0682f0 0x7fdbec06a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdc00155630 tx=0x7fdbf0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:33.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.663+0000 7fdbfcff9700 1 -- 172.21.15.28:0/3558054659 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdbf4052f40 con 0x7fdc00144d50 2023-11-28T11:09:33.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.855+0000 7fdc068dd700 1 -- 172.21.15.28:0/3558054659 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc000739f0 con 0x7fdc00144d50 2023-11-28T11:09:33.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.855+0000 7fdbfcff9700 1 -- 172.21.15.28:0/3558054659 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v26) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdbf4056560 con 0x7fdc00144d50 2023-11-28T11:09:33.857 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:09:33.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.857+0000 7fdc068dd700 1 -- 172.21.15.28:0/3558054659 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdbec0682f0 msgr2=0x7fdbec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.857+0000 7fdc068dd700 1 --2- 172.21.15.28:0/3558054659 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdbec0682f0 0x7fdbec06a7a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdc00155630 tx=0x7fdbf0009040 comp rx=0 tx=0).stop 2023-11-28T11:09:33.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.857+0000 7fdc068dd700 1 -- 172.21.15.28:0/3558054659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc00144d50 msgr2=0x7fdc000b0350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:33.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.857+0000 7fdc068dd700 1 --2- 172.21.15.28:0/3558054659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc00144d50 0x7fdc000b0350 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fdbf4011aa0 tx=0x7fdbf4011db0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.857+0000 7fdc068dd700 1 -- 172.21.15.28:0/3558054659 shutdown_connections 2023-11-28T11:09:33.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.857+0000 7fdc068dd700 1 --2- 172.21.15.28:0/3558054659 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdbec0682f0 0x7fdbec06a7a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.857+0000 7fdc068dd700 1 --2- 172.21.15.28:0/3558054659 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc00144d50 0x7fdc000b0350 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.857+0000 7fdc068dd700 1 --2- 172.21.15.28:0/3558054659 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc000b45d0 0x7fdc000b4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.857+0000 7fdc068dd700 1 --2- 172.21.15.28:0/3558054659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc00145120 0x7fdc000b0890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:33.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.857+0000 7fdc068dd700 1 -- 172.21.15.28:0/3558054659 >> 172.21.15.28:0/3558054659 conn(0x7fdc000cc630 msgr2=0x7fdc000ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:33.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.857+0000 7fdc068dd700 1 -- 172.21.15.28:0/3558054659 shutdown_connections 2023-11-28T11:09:33.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:33.857+0000 7fdc068dd700 1 -- 172.21.15.28:0/3558054659 wait complete. 2023-11-28T11:09:33.871 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:34.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.000+0000 7fa32b127700 1 Processor -- start 2023-11-28T11:09:34.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.000+0000 7fa32b127700 1 -- start start 2023-11-28T11:09:34.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.001+0000 7fa32b127700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3240b1630 0x7fa3240cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.001+0000 7fa32b127700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3240ce550 con 0x7fa3240b1a00 2023-11-28T11:09:34.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.001+0000 7fa32b127700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3240ce830 con 0x7fa32406afc0 2023-11-28T11:09:34.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.001+0000 7fa32b127700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3240ceb10 con 0x7fa3240b1630 2023-11-28T11:09:34.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.001+0000 7fa328ec3700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3240b1630 0x7fa3240cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.001+0000 7fa328ec3700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3240b1630 0x7fa3240cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48060/0 (socket says 172.21.15.28:48060) 2023-11-28T11:09:34.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.001+0000 7fa328ec3700 1 -- 172.21.15.28:0/1750126984 learned_addr learned my addr 172.21.15.28:0/1750126984 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:34.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.001+0000 7fa328ec3700 1 -- 172.21.15.28:0/1750126984 >> v1:172.21.15.59:6789/0 conn(0x7fa32406afc0 legacy=0x7fa32406b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:34.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.001+0000 7fa328ec3700 1 -- 172.21.15.28:0/1750126984 >> v1:172.21.15.28:6789/0 conn(0x7fa3240b1a00 legacy=0x7fa3240caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.001+0000 7fa328ec3700 1 -- 172.21.15.28:0/1750126984 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3240cedf0 con 0x7fa3240b1630 2023-11-28T11:09:34.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.001+0000 7fa328ec3700 1 --2- 172.21.15.28:0/1750126984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240b1630 0x7fa3240cf5f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa318009920 tx=0x7fa318009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bda4b382197b3ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:34.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.001+0000 7fa3237fe700 1 -- 172.21.15.28:0/1750126984 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318004230 con 0x7fa3240b1630 2023-11-28T11:09:34.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.001+0000 7fa3237fe700 1 -- 172.21.15.28:0/1750126984 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa318004390 con 0x7fa3240b1630 2023-11-28T11:09:34.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.002+0000 7fa3237fe700 1 -- 172.21.15.28:0/1750126984 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa318009e70 con 0x7fa3240b1630 2023-11-28T11:09:34.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.002+0000 7fa32b127700 1 -- 172.21.15.28:0/1750126984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240b1630 msgr2=0x7fa3240cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.002+0000 7fa32b127700 1 --2- 172.21.15.28:0/1750126984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240b1630 0x7fa3240cf5f0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa318009920 tx=0x7fa318009c30 comp rx=0 tx=0).stop 2023-11-28T11:09:34.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.002+0000 7fa32b127700 1 -- 172.21.15.28:0/1750126984 shutdown_connections 2023-11-28T11:09:34.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.002+0000 7fa32b127700 1 --2- 172.21.15.28:0/1750126984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240b1630 0x7fa3240cf5f0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.002+0000 7fa32b127700 1 -- 172.21.15.28:0/1750126984 >> 172.21.15.28:0/1750126984 conn(0x7fa3240cc630 msgr2=0x7fa3240cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:34.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.002+0000 7fa32b127700 1 -- 172.21.15.28:0/1750126984 shutdown_connections 2023-11-28T11:09:34.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.002+0000 7fa32b127700 1 -- 172.21.15.28:0/1750126984 wait complete. 2023-11-28T11:09:34.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.003+0000 7fa32b127700 1 Processor -- start 2023-11-28T11:09:34.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.003+0000 7fa32b127700 1 -- start start 2023-11-28T11:09:34.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.004+0000 7fa32b127700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32406afc0 0x7fa3241f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.004+0000 7fa32b127700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3240b1630 0x7fa324157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.004+0000 7fa32b127700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240b1a00 0x7fa324156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.004+0000 7fa32b127700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3241f2dd0 con 0x7fa3240b1a00 2023-11-28T11:09:34.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.004+0000 7fa32b127700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3241581a0 con 0x7fa3240b1630 2023-11-28T11:09:34.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.004+0000 7fa32b127700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240b4490 con 0x7fa32406afc0 2023-11-28T11:09:34.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.004+0000 7fa3296c4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240b1a00 0x7fa324156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.004+0000 7fa3296c4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240b1a00 0x7fa324156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48070/0 (socket says 172.21.15.28:48070) 2023-11-28T11:09:34.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.004+0000 7fa3296c4700 1 -- 172.21.15.28:0/2397302834 learned_addr learned my addr 172.21.15.28:0/2397302834 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:34.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.005+0000 7fa323fff700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3240b1630 0x7fa324157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.005+0000 7fa3296c4700 1 -- 172.21.15.28:0/2397302834 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32406afc0 msgr2=0x7fa3241f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.005+0000 7fa328ec3700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32406afc0 0x7fa3241f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.005+0000 7fa3296c4700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32406afc0 0x7fa3241f3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.005+0000 7fa3296c4700 1 -- 172.21.15.28:0/2397302834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3240b1630 msgr2=0x7fa324157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.005+0000 7fa3296c4700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3240b1630 0x7fa324157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.005+0000 7fa3296c4700 1 -- 172.21.15.28:0/2397302834 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa318009550 con 0x7fa3240b1a00 2023-11-28T11:09:34.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.005+0000 7fa323fff700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3240b1630 0x7fa324157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:34.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.005+0000 7fa328ec3700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32406afc0 0x7fa3241f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:34.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.005+0000 7fa3296c4700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240b1a00 0x7fa324156f90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00cc10 tx=0x7fa31c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:34.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.005+0000 7fa3217fa700 1 -- 172.21.15.28:0/2397302834 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c00abb0 con 0x7fa3240b1a00 2023-11-28T11:09:34.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.005+0000 7fa3217fa700 1 -- 172.21.15.28:0/2397302834 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31c00ad10 con 0x7fa3240b1a00 2023-11-28T11:09:34.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.005+0000 7fa32b127700 1 -- 172.21.15.28:0/2397302834 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3240b1ea0 con 0x7fa3240b1a00 2023-11-28T11:09:34.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.005+0000 7fa3217fa700 1 -- 172.21.15.28:0/2397302834 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c012660 con 0x7fa3240b1a00 2023-11-28T11:09:34.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.005+0000 7fa32b127700 1 -- 172.21.15.28:0/2397302834 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3240cf0d0 con 0x7fa3240b1a00 2023-11-28T11:09:34.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.006+0000 7fa32b127700 1 -- 172.21.15.28:0/2397302834 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3240cb5b0 con 0x7fa3240b1a00 2023-11-28T11:09:34.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.006+0000 7fa3217fa700 1 -- 172.21.15.28:0/2397302834 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa31c022070 con 0x7fa3240b1a00 2023-11-28T11:09:34.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.007+0000 7fa3217fa700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3080682f0 0x7fa30806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.007+0000 7fa3217fa700 1 -- 172.21.15.28:0/2397302834 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa31c086690 con 0x7fa3240b1a00 2023-11-28T11:09:34.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.007+0000 7fa328ec3700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3080682f0 0x7fa30806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.007+0000 7fa328ec3700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3080682f0 0x7fa30806a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa318009c70 tx=0x7fa318008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:34.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.011+0000 7fa3217fa700 1 -- 172.21.15.28:0/2397302834 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa31c052bc0 con 0x7fa3240b1a00 2023-11-28T11:09:34.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.205+0000 7fa32b127700 1 -- 172.21.15.28:0/2397302834 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa3240739f0 con 0x7fa3240b1a00 2023-11-28T11:09:34.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.205+0000 7fa3217fa700 1 -- 172.21.15.28:0/2397302834 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v26) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa31c016020 con 0x7fa3240b1a00 2023-11-28T11:09:34.207 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:34.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.207+0000 7fa32b127700 1 -- 172.21.15.28:0/2397302834 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3080682f0 msgr2=0x7fa30806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.207+0000 7fa32b127700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3080682f0 0x7fa30806a7a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa318009c70 tx=0x7fa318008030 comp rx=0 tx=0).stop 2023-11-28T11:09:34.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.207+0000 7fa32b127700 1 -- 172.21.15.28:0/2397302834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240b1a00 msgr2=0x7fa324156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.207+0000 7fa32b127700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240b1a00 0x7fa324156f90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa31c00cc10 tx=0x7fa31c00a060 comp rx=0 tx=0).stop 2023-11-28T11:09:34.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.207+0000 7fa32b127700 1 -- 172.21.15.28:0/2397302834 shutdown_connections 2023-11-28T11:09:34.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.207+0000 7fa32b127700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3080682f0 0x7fa30806a7a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.207+0000 7fa32b127700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240b1a00 0x7fa324156f90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.207+0000 7fa32b127700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32406afc0 0x7fa3241f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.207+0000 7fa32b127700 1 --2- 172.21.15.28:0/2397302834 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3240b1630 0x7fa324157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.207+0000 7fa32b127700 1 -- 172.21.15.28:0/2397302834 >> 172.21.15.28:0/2397302834 conn(0x7fa3240cc630 msgr2=0x7fa3240cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:34.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.207+0000 7fa32b127700 1 -- 172.21.15.28:0/2397302834 shutdown_connections 2023-11-28T11:09:34.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.207+0000 7fa32b127700 1 -- 172.21.15.28:0/2397302834 wait complete. 2023-11-28T11:09:34.221 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:34.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.347+0000 7f6029528700 1 Processor -- start 2023-11-28T11:09:34.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.348+0000 7f6029528700 1 -- start start 2023-11-28T11:09:34.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.348+0000 7f6029528700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f602415fc00 0x7f602406eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.348+0000 7f6029528700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f602406f3e0 0x7f60240b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.348+0000 7f6029528700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f60240b2bc0 0x7f60241602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.348+0000 7f6029528700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60240b3020 con 0x7f602406f3e0 2023-11-28T11:09:34.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.348+0000 7f6029528700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f602406f940 con 0x7f602415fc00 2023-11-28T11:09:34.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.348+0000 7f6029528700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602406fc20 con 0x7f60240b2bc0 2023-11-28T11:09:34.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.349+0000 7f60227fc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f602406f3e0 0x7f60240b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.349+0000 7f60227fc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f602406f3e0 0x7f60240b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48074/0 (socket says 172.21.15.28:48074) 2023-11-28T11:09:34.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.349+0000 7f60227fc700 1 -- 172.21.15.28:0/1308246289 learned_addr learned my addr 172.21.15.28:0/1308246289 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:34.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.349+0000 7f60237fe700 1 --2- 172.21.15.28:0/1308246289 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f60240b2bc0 0x7f60241602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.349+0000 7f6022ffd700 1 --2- 172.21.15.28:0/1308246289 >> v2:172.21.15.59:3300/0 conn(0x7f602415fc00 0x7f602406eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.349+0000 7f60227fc700 1 -- 172.21.15.28:0/1308246289 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f60240b2bc0 msgr2=0x7f60241602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:34.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.349+0000 7f60227fc700 1 --2- 172.21.15.28:0/1308246289 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f60240b2bc0 0x7f60241602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.349+0000 7f60227fc700 1 -- 172.21.15.28:0/1308246289 >> v2:172.21.15.59:3300/0 conn(0x7f602415fc00 msgr2=0x7f602406eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:34.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.349+0000 7f60227fc700 1 --2- 172.21.15.28:0/1308246289 >> v2:172.21.15.59:3300/0 conn(0x7f602415fc00 0x7f602406eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.349+0000 7f60227fc700 1 -- 172.21.15.28:0/1308246289 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60240cb3f0 con 0x7f602406f3e0 2023-11-28T11:09:34.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.349+0000 7f60227fc700 1 --2- 172.21.15.28:0/1308246289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f602406f3e0 0x7f60240b2680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f60240d0b60 tx=0x7f600c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d355051d986b19 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:34.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.349+0000 7f6021ffb700 1 -- 172.21.15.28:0/1308246289 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c00b870 con 0x7f602406f3e0 2023-11-28T11:09:34.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.349+0000 7f6021ffb700 1 -- 172.21.15.28:0/1308246289 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600c00b9d0 con 0x7f602406f3e0 2023-11-28T11:09:34.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.349+0000 7f6021ffb700 1 -- 172.21.15.28:0/1308246289 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600c00bd60 con 0x7f602406f3e0 2023-11-28T11:09:34.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.350+0000 7f6029528700 1 -- 172.21.15.28:0/1308246289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f602406f3e0 msgr2=0x7f60240b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.350+0000 7f6029528700 1 --2- 172.21.15.28:0/1308246289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f602406f3e0 0x7f60240b2680 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f60240d0b60 tx=0x7f600c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.350+0000 7f6029528700 1 -- 172.21.15.28:0/1308246289 shutdown_connections 2023-11-28T11:09:34.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.350+0000 7f6029528700 1 --2- 172.21.15.28:0/1308246289 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f60240b2bc0 0x7f60241602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.350+0000 7f6029528700 1 --2- 172.21.15.28:0/1308246289 >> v2:172.21.15.59:3300/0 conn(0x7f602415fc00 0x7f602406eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.350+0000 7f6029528700 1 --2- 172.21.15.28:0/1308246289 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f602406f3e0 0x7f60240b2680 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.350+0000 7f6029528700 1 -- 172.21.15.28:0/1308246289 >> 172.21.15.28:0/1308246289 conn(0x7f60240cd680 msgr2=0x7f60240cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:34.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.350+0000 7f6029528700 1 -- 172.21.15.28:0/1308246289 shutdown_connections 2023-11-28T11:09:34.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.350+0000 7f6029528700 1 -- 172.21.15.28:0/1308246289 wait complete. 2023-11-28T11:09:34.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.351+0000 7f6029528700 1 Processor -- start 2023-11-28T11:09:34.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.351+0000 7f6029528700 1 -- start start 2023-11-28T11:09:34.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f6029528700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f602406f3e0 0x7f60240c70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f6029528700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f60240b2bc0 0x7f602415be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f6029528700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f602415fc00 0x7f60240ca920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f6029528700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6024159280 con 0x7f602415fc00 2023-11-28T11:09:34.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f6029528700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f602415add0 con 0x7f602406f3e0 2023-11-28T11:09:34.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f6029528700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f602415b0b0 con 0x7f60240b2bc0 2023-11-28T11:09:34.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f60237fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f602415fc00 0x7f60240ca920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f60237fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f602415fc00 0x7f60240ca920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48084/0 (socket says 172.21.15.28:48084) 2023-11-28T11:09:34.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f60237fe700 1 -- 172.21.15.28:0/2050194559 learned_addr learned my addr 172.21.15.28:0/2050194559 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:34.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f6022ffd700 1 --2- 172.21.15.28:0/2050194559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f602406f3e0 0x7f60240c70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f60227fc700 1 --2- 172.21.15.28:0/2050194559 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f60240b2bc0 0x7f602415be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f60237fe700 1 -- 172.21.15.28:0/2050194559 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f60240b2bc0 msgr2=0x7f602415be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f60237fe700 1 --2- 172.21.15.28:0/2050194559 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f60240b2bc0 0x7f602415be70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f60237fe700 1 -- 172.21.15.28:0/2050194559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f602406f3e0 msgr2=0x7f60240c70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f60237fe700 1 --2- 172.21.15.28:0/2050194559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f602406f3e0 0x7f60240c70e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f60237fe700 1 -- 172.21.15.28:0/2050194559 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c009550 con 0x7f602415fc00 2023-11-28T11:09:34.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.352+0000 7f6022ffd700 1 --2- 172.21.15.28:0/2050194559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f602406f3e0 0x7f60240c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:34.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.353+0000 7f60237fe700 1 --2- 172.21.15.28:0/2050194559 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f602415fc00 0x7f60240ca920 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f601c007e40 tx=0x7f601c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:34.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.353+0000 7f601b7fe700 1 -- 172.21.15.28:0/2050194559 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c00eea0 con 0x7f602415fc00 2023-11-28T11:09:34.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.353+0000 7f601b7fe700 1 -- 172.21.15.28:0/2050194559 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f601c014420 con 0x7f602415fc00 2023-11-28T11:09:34.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.353+0000 7f6029528700 1 -- 172.21.15.28:0/2050194559 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f602415ffd0 con 0x7f602415fc00 2023-11-28T11:09:34.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.353+0000 7f601b7fe700 1 -- 172.21.15.28:0/2050194559 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f601c0147b0 con 0x7f602415fc00 2023-11-28T11:09:34.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.353+0000 7f6029528700 1 -- 172.21.15.28:0/2050194559 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60240cb6d0 con 0x7f602415fc00 2023-11-28T11:09:34.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.354+0000 7f6029528700 1 -- 172.21.15.28:0/2050194559 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60240ce340 con 0x7f602415fc00 2023-11-28T11:09:34.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.354+0000 7f601b7fe700 1 -- 172.21.15.28:0/2050194559 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f601c012050 con 0x7f602415fc00 2023-11-28T11:09:34.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.354+0000 7f601b7fe700 1 --2- 172.21.15.28:0/2050194559 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60040682f0 0x7f600406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.355+0000 7f601b7fe700 1 -- 172.21.15.28:0/2050194559 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f601c086710 con 0x7f602415fc00 2023-11-28T11:09:34.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.355+0000 7f6022ffd700 1 --2- 172.21.15.28:0/2050194559 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60040682f0 0x7f600406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.355+0000 7f6022ffd700 1 --2- 172.21.15.28:0/2050194559 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60040682f0 0x7f600406a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f60140097a0 tx=0x7f6014008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:34.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.359+0000 7f601b7fe700 1 -- 172.21.15.28:0/2050194559 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f601c052bc0 con 0x7f602415fc00 2023-11-28T11:09:34.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.551+0000 7f6029528700 1 -- 172.21.15.28:0/2050194559 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f60240739f0 con 0x7f602415fc00 2023-11-28T11:09:34.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.551+0000 7f601b7fe700 1 -- 172.21.15.28:0/2050194559 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v26) v1 ==== 100+0+11 (secure 0 0 0) 0x7f601c0561e0 con 0x7f602415fc00 2023-11-28T11:09:34.553 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:09:34.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.553+0000 7f6029528700 1 -- 172.21.15.28:0/2050194559 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60040682f0 msgr2=0x7f600406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.553+0000 7f6029528700 1 --2- 172.21.15.28:0/2050194559 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60040682f0 0x7f600406a7a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f60140097a0 tx=0x7f6014008040 comp rx=0 tx=0).stop 2023-11-28T11:09:34.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.553+0000 7f6029528700 1 -- 172.21.15.28:0/2050194559 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f602415fc00 msgr2=0x7f60240ca920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.553+0000 7f6029528700 1 --2- 172.21.15.28:0/2050194559 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f602415fc00 0x7f60240ca920 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f601c007e40 tx=0x7f601c00e3c0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.554+0000 7f6029528700 1 -- 172.21.15.28:0/2050194559 shutdown_connections 2023-11-28T11:09:34.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.554+0000 7f6029528700 1 --2- 172.21.15.28:0/2050194559 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f60040682f0 0x7f600406a7a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.554+0000 7f6029528700 1 --2- 172.21.15.28:0/2050194559 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f602415fc00 0x7f60240ca920 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.554+0000 7f6029528700 1 --2- 172.21.15.28:0/2050194559 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f602406f3e0 0x7f60240c70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.554+0000 7f6029528700 1 --2- 172.21.15.28:0/2050194559 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f60240b2bc0 0x7f602415be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.554+0000 7f6029528700 1 -- 172.21.15.28:0/2050194559 >> 172.21.15.28:0/2050194559 conn(0x7f60240cd680 msgr2=0x7f60241609f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:34.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.554+0000 7f6029528700 1 -- 172.21.15.28:0/2050194559 shutdown_connections 2023-11-28T11:09:34.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.554+0000 7f6029528700 1 -- 172.21.15.28:0/2050194559 wait complete. 2023-11-28T11:09:34.568 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:34.699 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.697+0000 7f0907857700 1 Processor -- start 2023-11-28T11:09:34.699 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.698+0000 7f0907857700 1 -- start start 2023-11-28T11:09:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.698+0000 7f0907857700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09000cd3e0 0x7f090006a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.698+0000 7f0907857700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f09000d0fe0 0x7f09000d1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.698+0000 7f0907857700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f090006bb90 con 0x7f090015fa10 2023-11-28T11:09:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.698+0000 7f0907857700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09000cfa50 con 0x7f09000d0fe0 2023-11-28T11:09:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.698+0000 7f0907857700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09000d0ae0 con 0x7f09000cd3e0 2023-11-28T11:09:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.698+0000 7f0905df4700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f09000d0fe0 0x7f09000d1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.699+0000 7f0905df4700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f09000d0fe0 0x7f09000d1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48092/0 (socket says 172.21.15.28:48092) 2023-11-28T11:09:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.699+0000 7f0905df4700 1 -- 172.21.15.28:0/2959327281 learned_addr learned my addr 172.21.15.28:0/2959327281 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.699+0000 7f0905df4700 1 -- 172.21.15.28:0/2959327281 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09000cd3e0 msgr2=0x7f090006a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.699+0000 7f0904df2700 1 --2- 172.21.15.28:0/2959327281 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09000cd3e0 0x7f090006a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.699+0000 7f0905df4700 1 --2- 172.21.15.28:0/2959327281 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09000cd3e0 0x7f090006a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.699+0000 7f0905df4700 1 -- 172.21.15.28:0/2959327281 >> v1:172.21.15.28:6789/0 conn(0x7f090015fa10 legacy=0x7f090006a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.699+0000 7f0905df4700 1 -- 172.21.15.28:0/2959327281 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09000cfca0 con 0x7f09000d0fe0 2023-11-28T11:09:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.699+0000 7f0904df2700 1 --2- 172.21.15.28:0/2959327281 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09000cd3e0 0x7f090006a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:34.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.699+0000 7f0905df4700 1 --2- 172.21.15.28:0/2959327281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09000d0fe0 0x7f09000d1460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009e50 tx=0x7f08f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=944dbbafa58572e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:34.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.699+0000 7f08f3fff700 1 -- 172.21.15.28:0/2959327281 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f800e070 con 0x7f09000d0fe0 2023-11-28T11:09:34.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.699+0000 7f08f3fff700 1 -- 172.21.15.28:0/2959327281 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f800b9c0 con 0x7f09000d0fe0 2023-11-28T11:09:34.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.699+0000 7f08f3fff700 1 -- 172.21.15.28:0/2959327281 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f800bd70 con 0x7f09000d0fe0 2023-11-28T11:09:34.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.700+0000 7f0907857700 1 -- 172.21.15.28:0/2959327281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09000d0fe0 msgr2=0x7f09000d1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.700+0000 7f0907857700 1 --2- 172.21.15.28:0/2959327281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09000d0fe0 0x7f09000d1460 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f08f8009e50 tx=0x7f08f800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:34.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.700+0000 7f0907857700 1 -- 172.21.15.28:0/2959327281 shutdown_connections 2023-11-28T11:09:34.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.700+0000 7f0907857700 1 --2- 172.21.15.28:0/2959327281 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f09000d0fe0 0x7f09000d1460 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.700+0000 7f0907857700 1 --2- 172.21.15.28:0/2959327281 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09000cd3e0 0x7f090006a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.700+0000 7f0907857700 1 -- 172.21.15.28:0/2959327281 >> 172.21.15.28:0/2959327281 conn(0x7f090006ce40 msgr2=0x7f09000b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:34.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.700+0000 7f0907857700 1 -- 172.21.15.28:0/2959327281 shutdown_connections 2023-11-28T11:09:34.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.700+0000 7f0907857700 1 -- 172.21.15.28:0/2959327281 wait complete. 2023-11-28T11:09:34.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.701+0000 7f0907857700 1 Processor -- start 2023-11-28T11:09:34.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.701+0000 7f0907857700 1 -- start start 2023-11-28T11:09:34.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.702+0000 7f0907857700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09000cd3e0 0x7f09000c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.702+0000 7f0907857700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09000d0fe0 0x7f09000cc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.702+0000 7f0907857700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090015fa10 0x7f0900158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.702+0000 7f0907857700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09000caeb0 con 0x7f090015fa10 2023-11-28T11:09:34.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.702+0000 7f0907857700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09000c98b0 con 0x7f09000cd3e0 2023-11-28T11:09:34.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.702+0000 7f0907857700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09000cb100 con 0x7f09000d0fe0 2023-11-28T11:09:34.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.702+0000 7f0905df4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090015fa10 0x7f0900158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.702+0000 7f0905df4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090015fa10 0x7f0900158770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48098/0 (socket says 172.21.15.28:48098) 2023-11-28T11:09:34.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.702+0000 7f0905df4700 1 -- 172.21.15.28:0/2950085618 learned_addr learned my addr 172.21.15.28:0/2950085618 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:34.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.702+0000 7f0905df4700 1 -- 172.21.15.28:0/2950085618 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09000d0fe0 msgr2=0x7f09000cc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.702+0000 7f0905df4700 1 --2- 172.21.15.28:0/2950085618 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09000d0fe0 0x7f09000cc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.702+0000 7f0905df4700 1 -- 172.21.15.28:0/2950085618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09000cd3e0 msgr2=0x7f09000c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.702+0000 7f0905df4700 1 --2- 172.21.15.28:0/2950085618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09000cd3e0 0x7f09000c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.702+0000 7f0905df4700 1 -- 172.21.15.28:0/2950085618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08f8009b30 con 0x7f090015fa10 2023-11-28T11:09:34.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.702+0000 7f0905df4700 1 --2- 172.21.15.28:0/2950085618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090015fa10 0x7f0900158770 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f090006c050 tx=0x7f08f8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:34.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.703+0000 7f08f1ffb700 1 -- 172.21.15.28:0/2950085618 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f80092e0 con 0x7f090015fa10 2023-11-28T11:09:34.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.703+0000 7f08f1ffb700 1 -- 172.21.15.28:0/2950085618 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08f80099b0 con 0x7f090015fa10 2023-11-28T11:09:34.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.703+0000 7f08f1ffb700 1 -- 172.21.15.28:0/2950085618 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08f801c680 con 0x7f090015fa10 2023-11-28T11:09:34.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.703+0000 7f0907857700 1 -- 172.21.15.28:0/2950085618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09000cd850 con 0x7f090015fa10 2023-11-28T11:09:34.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.703+0000 7f0907857700 1 -- 172.21.15.28:0/2950085618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09000cff80 con 0x7f090015fa10 2023-11-28T11:09:34.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.704+0000 7f0907857700 1 -- 172.21.15.28:0/2950085618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f090015fde0 con 0x7f090015fa10 2023-11-28T11:09:34.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.704+0000 7f08f1ffb700 1 -- 172.21.15.28:0/2950085618 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f08f802c070 con 0x7f090015fa10 2023-11-28T11:09:34.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.705+0000 7f08f1ffb700 1 --2- 172.21.15.28:0/2950085618 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f08e40682f0 0x7f08e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:34.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.705+0000 7f08f1ffb700 1 -- 172.21.15.28:0/2950085618 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f08f8087da0 con 0x7f090015fa10 2023-11-28T11:09:34.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.705+0000 7f09055f3700 1 --2- 172.21.15.28:0/2950085618 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f08e40682f0 0x7f08e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:34.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.705+0000 7f09055f3700 1 --2- 172.21.15.28:0/2950085618 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f08e40682f0 0x7f08e406a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f08fc000f60 tx=0x7f08fc007a40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:34.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.709+0000 7f08f1ffb700 1 -- 172.21.15.28:0/2950085618 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f08f80541e0 con 0x7f090015fa10 2023-11-28T11:09:34.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.906+0000 7f0907857700 1 -- 172.21.15.28:0/2950085618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f090006b8e0 con 0x7f090015fa10 2023-11-28T11:09:34.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.906+0000 7f08f1ffb700 1 -- 172.21.15.28:0/2950085618 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f08f8057800 con 0x7f090015fa10 2023-11-28T11:09:34.908 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:34.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.908+0000 7f0907857700 1 -- 172.21.15.28:0/2950085618 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f08e40682f0 msgr2=0x7f08e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.908+0000 7f0907857700 1 --2- 172.21.15.28:0/2950085618 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f08e40682f0 0x7f08e406a7a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f08fc000f60 tx=0x7f08fc007a40 comp rx=0 tx=0).stop 2023-11-28T11:09:34.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.908+0000 7f0907857700 1 -- 172.21.15.28:0/2950085618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090015fa10 msgr2=0x7f0900158770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:34.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.908+0000 7f0907857700 1 --2- 172.21.15.28:0/2950085618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090015fa10 0x7f0900158770 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f090006c050 tx=0x7f08f8004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.909+0000 7f0907857700 1 -- 172.21.15.28:0/2950085618 shutdown_connections 2023-11-28T11:09:34.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.909+0000 7f0907857700 1 --2- 172.21.15.28:0/2950085618 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f08e40682f0 0x7f08e406a7a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.909+0000 7f0907857700 1 --2- 172.21.15.28:0/2950085618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f090015fa10 0x7f0900158770 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.909+0000 7f0907857700 1 --2- 172.21.15.28:0/2950085618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09000cd3e0 0x7f09000c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.909+0000 7f0907857700 1 --2- 172.21.15.28:0/2950085618 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f09000d0fe0 0x7f09000cc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:34.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.909+0000 7f0907857700 1 -- 172.21.15.28:0/2950085618 >> 172.21.15.28:0/2950085618 conn(0x7f090006ce40 msgr2=0x7f09000d30a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:34.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.909+0000 7f0907857700 1 -- 172.21.15.28:0/2950085618 shutdown_connections 2023-11-28T11:09:34.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:34.909+0000 7f0907857700 1 -- 172.21.15.28:0/2950085618 wait complete. 2023-11-28T11:09:34.923 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:35.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.047+0000 7f417081a700 1 Processor -- start 2023-11-28T11:09:35.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.047+0000 7f417081a700 1 -- start start 2023-11-28T11:09:35.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f417081a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f416c159590 0x7f416c06e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f417081a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f416c15db70 0x7f416c0ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f417081a700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f416c15d120 con 0x7f416c0ce890 2023-11-28T11:09:35.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f417081a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f416c15d6e0 con 0x7f416c159590 2023-11-28T11:09:35.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f417081a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416c0aef60 con 0x7f416c15db70 2023-11-28T11:09:35.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f416ad9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f416c0ce890 0x7f416c0caf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36164/0 (socket says 172.21.15.28:36164) 2023-11-28T11:09:35.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f416ad9d700 1 -- 172.21.15.28:0/1362559216 learned_addr learned my addr 172.21.15.28:0/1362559216 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:35.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f416a59c700 1 --2- 172.21.15.28:0/1362559216 >> v2:172.21.15.59:3300/0 conn(0x7f416c159590 0x7f416c06e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f4169d9b700 1 --2- 172.21.15.28:0/1362559216 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f416c15db70 0x7f416c0ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f416959a700 1 -- 172.21.15.28:0/1362559216 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1995219348 0 0) 0x7f416c15d120 con 0x7f416c0ce890 2023-11-28T11:09:35.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f416959a700 1 -- 172.21.15.28:0/1362559216 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41540035b0 con 0x7f416c0ce890 2023-11-28T11:09:35.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f416959a700 1 -- 172.21.15.28:0/1362559216 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 735804810 0 0) 0x7f41540035b0 con 0x7f416c0ce890 2023-11-28T11:09:35.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f416959a700 1 -- 172.21.15.28:0/1362559216 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f416c15db70 msgr2=0x7f416c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:35.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f416959a700 1 --2- 172.21.15.28:0/1362559216 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f416c15db70 0x7f416c0ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f416959a700 1 -- 172.21.15.28:0/1362559216 >> v2:172.21.15.59:3300/0 conn(0x7f416c159590 msgr2=0x7f416c06e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:35.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f416959a700 1 --2- 172.21.15.28:0/1362559216 >> v2:172.21.15.59:3300/0 conn(0x7f416c159590 0x7f416c06e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f416959a700 1 -- 172.21.15.28:0/1362559216 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c0af240 con 0x7f416c0ce890 2023-11-28T11:09:35.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f416959a700 1 -- 172.21.15.28:0/1362559216 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f41640019f0 con 0x7f416c0ce890 2023-11-28T11:09:35.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f416959a700 1 -- 172.21.15.28:0/1362559216 >> v1:172.21.15.28:6789/0 conn(0x7f416c0ce890 legacy=0x7f416c0caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.048+0000 7f416959a700 1 --2- 172.21.15.28:0/1362559216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4154003a00 0x7f4154003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416959a700 1 --2- 172.21.15.28:0/1362559216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4154004430 0x7f41540048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416959a700 1 --2- 172.21.15.28:0/1362559216 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f416c15db70 0x7f4154004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416959a700 1 -- 172.21.15.28:0/1362559216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c15d6e0 con 0x7f4154003a00 2023-11-28T11:09:35.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416959a700 1 -- 172.21.15.28:0/1362559216 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c0aef60 con 0x7f4154004430 2023-11-28T11:09:35.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416959a700 1 -- 172.21.15.28:0/1362559216 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415400ba10 con 0x7f416c15db70 2023-11-28T11:09:35.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416a59c700 1 --2- 172.21.15.28:0/1362559216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4154003a00 0x7f4154003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f4169d9b700 1 --2- 172.21.15.28:0/1362559216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4154004430 0x7f41540048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416ad9d700 1 --2- 172.21.15.28:0/1362559216 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f416c15db70 0x7f4154004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416a59c700 1 -- 172.21.15.28:0/1362559216 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f416c15db70 msgr2=0x7f4154004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:35.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416a59c700 1 --2- 172.21.15.28:0/1362559216 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f416c15db70 0x7f4154004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416a59c700 1 -- 172.21.15.28:0/1362559216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4154004430 msgr2=0x7f41540048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:35.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416a59c700 1 --2- 172.21.15.28:0/1362559216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4154004430 0x7f41540048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416a59c700 1 -- 172.21.15.28:0/1362559216 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41540035b0 con 0x7f4154003a00 2023-11-28T11:09:35.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416a59c700 1 --2- 172.21.15.28:0/1362559216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4154003a00 0x7f4154003e10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f415c009b60 tx=0x7f4154005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6a42d32ee9985fd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:35.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416959a700 1 -- 172.21.15.28:0/1362559216 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c00e6c0 con 0x7f4154003a00 2023-11-28T11:09:35.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416959a700 1 -- 172.21.15.28:0/1362559216 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f415c00e820 con 0x7f4154003a00 2023-11-28T11:09:35.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.049+0000 7f416959a700 1 -- 172.21.15.28:0/1362559216 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f415c00eb90 con 0x7f4154003a00 2023-11-28T11:09:35.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.050+0000 7f417081a700 1 -- 172.21.15.28:0/1362559216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4154003a00 msgr2=0x7f4154003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.050+0000 7f417081a700 1 --2- 172.21.15.28:0/1362559216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4154003a00 0x7f4154003e10 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f415c009b60 tx=0x7f4154005880 comp rx=0 tx=0).stop 2023-11-28T11:09:35.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.050+0000 7f416ad9d700 1 -- 172.21.15.28:0/1362559216 reap_dead start 2023-11-28T11:09:35.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.050+0000 7f417081a700 1 -- 172.21.15.28:0/1362559216 shutdown_connections 2023-11-28T11:09:35.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.050+0000 7f417081a700 1 -- 172.21.15.28:0/1362559216 >> 172.21.15.28:0/1362559216 conn(0x7f416c0b1650 msgr2=0x7f416c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:35.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.050+0000 7f417081a700 1 -- 172.21.15.28:0/1362559216 shutdown_connections 2023-11-28T11:09:35.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.050+0000 7f417081a700 1 -- 172.21.15.28:0/1362559216 wait complete. 2023-11-28T11:09:35.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.051+0000 7f417081a700 1 Processor -- start 2023-11-28T11:09:35.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.051+0000 7f417081a700 1 -- start start 2023-11-28T11:09:35.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f417081a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4154003a00 0x7f416c162f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f417081a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f416c158750 0x7f416c14af80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f417081a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f416c0b4960 0x7f416c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f417081a700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c164810 con 0x7f416c0b4960 2023-11-28T11:09:35.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f417081a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f416c0b5310 con 0x7f416c158750 2023-11-28T11:09:35.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f417081a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f416c0c6820 con 0x7f4154003a00 2023-11-28T11:09:35.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f416ad9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f416c0b4960 0x7f416c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f416ad9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f416c0b4960 0x7f416c0b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48118/0 (socket says 172.21.15.28:48118) 2023-11-28T11:09:35.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f416ad9d700 1 -- 172.21.15.28:0/1797481108 learned_addr learned my addr 172.21.15.28:0/1797481108 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:35.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f4169d9b700 1 --2- 172.21.15.28:0/1797481108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f416c158750 0x7f416c14af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f416a59c700 1 --2- 172.21.15.28:0/1797481108 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4154003a00 0x7f416c162f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f416ad9d700 1 -- 172.21.15.28:0/1797481108 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4154003a00 msgr2=0x7f416c162f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f416ad9d700 1 --2- 172.21.15.28:0/1797481108 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4154003a00 0x7f416c162f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f416ad9d700 1 -- 172.21.15.28:0/1797481108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f416c158750 msgr2=0x7f416c14af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f416ad9d700 1 --2- 172.21.15.28:0/1797481108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f416c158750 0x7f416c14af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.052+0000 7f416ad9d700 1 -- 172.21.15.28:0/1797481108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f416c15dfe0 con 0x7f416c0b4960 2023-11-28T11:09:35.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.053+0000 7f416ad9d700 1 --2- 172.21.15.28:0/1797481108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f416c0b4960 0x7f416c0b4dd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f416c06bee0 tx=0x7f416400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:35.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.053+0000 7f415affd700 1 -- 172.21.15.28:0/1797481108 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f416400ad30 con 0x7f416c0b4960 2023-11-28T11:09:35.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.053+0000 7f415affd700 1 -- 172.21.15.28:0/1797481108 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f416400ae90 con 0x7f416c0b4960 2023-11-28T11:09:35.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.053+0000 7f417081a700 1 -- 172.21.15.28:0/1797481108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f416c15d370 con 0x7f416c0b4960 2023-11-28T11:09:35.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.053+0000 7f415affd700 1 -- 172.21.15.28:0/1797481108 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164013630 con 0x7f416c0b4960 2023-11-28T11:09:35.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.053+0000 7f417081a700 1 -- 172.21.15.28:0/1797481108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416c159960 con 0x7f416c0b4960 2023-11-28T11:09:35.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.054+0000 7f417081a700 1 -- 172.21.15.28:0/1797481108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f416c0675a0 con 0x7f416c0b4960 2023-11-28T11:09:35.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.054+0000 7f415affd700 1 -- 172.21.15.28:0/1797481108 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4164023070 con 0x7f416c0b4960 2023-11-28T11:09:35.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.055+0000 7f415affd700 1 --2- 172.21.15.28:0/1797481108 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f414c0682f0 0x7f414c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.055+0000 7f415affd700 1 -- 172.21.15.28:0/1797481108 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4164086a20 con 0x7f416c0b4960 2023-11-28T11:09:35.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.055+0000 7f416a59c700 1 --2- 172.21.15.28:0/1797481108 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f414c0682f0 0x7f414c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.055+0000 7f416a59c700 1 --2- 172.21.15.28:0/1797481108 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f414c0682f0 0x7f414c06a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f415c0090b0 tx=0x7f415c01f040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:35.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.059+0000 7f415affd700 1 -- 172.21.15.28:0/1797481108 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4164052f50 con 0x7f416c0b4960 2023-11-28T11:09:35.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.267+0000 7f417081a700 1 -- 172.21.15.28:0/1797481108 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f416c06e490 con 0x7f416c0b4960 2023-11-28T11:09:35.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.268+0000 7f415affd700 1 -- 172.21.15.28:0/1797481108 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f4164017080 con 0x7f416c0b4960 2023-11-28T11:09:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.270+0000 7f417081a700 1 -- 172.21.15.28:0/1797481108 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f414c0682f0 msgr2=0x7f414c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.270+0000 7f417081a700 1 --2- 172.21.15.28:0/1797481108 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f414c0682f0 0x7f414c06a7a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f415c0090b0 tx=0x7f415c01f040 comp rx=0 tx=0).stop 2023-11-28T11:09:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.270+0000 7f417081a700 1 -- 172.21.15.28:0/1797481108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f416c0b4960 msgr2=0x7f416c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.270+0000 7f417081a700 1 --2- 172.21.15.28:0/1797481108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f416c0b4960 0x7f416c0b4dd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f416c06bee0 tx=0x7f416400a620 comp rx=0 tx=0).stop 2023-11-28T11:09:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.270+0000 7f417081a700 1 -- 172.21.15.28:0/1797481108 shutdown_connections 2023-11-28T11:09:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.270+0000 7f417081a700 1 --2- 172.21.15.28:0/1797481108 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f414c0682f0 0x7f414c06a7a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.270+0000 7f417081a700 1 --2- 172.21.15.28:0/1797481108 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f416c0b4960 0x7f416c0b4dd0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.270+0000 7f417081a700 1 --2- 172.21.15.28:0/1797481108 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4154003a00 0x7f416c162f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.270+0000 7f417081a700 1 --2- 172.21.15.28:0/1797481108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f416c158750 0x7f416c14af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.270+0000 7f417081a700 1 -- 172.21.15.28:0/1797481108 >> 172.21.15.28:0/1797481108 conn(0x7f416c0b1650 msgr2=0x7f416c0b0870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.270+0000 7f417081a700 1 -- 172.21.15.28:0/1797481108 shutdown_connections 2023-11-28T11:09:35.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.270+0000 7f417081a700 1 -- 172.21.15.28:0/1797481108 wait complete. 2023-11-28T11:09:35.273 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2023-11-28T11:09:35.284 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:09:35.284 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:35.302 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:35.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.465+0000 7faf3d0ee700 1 Processor -- start 2023-11-28T11:09:35.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.465+0000 7faf3d0ee700 1 -- start start 2023-11-28T11:09:35.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.465+0000 7faf3d0ee700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf380b2660 0x7faf380ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.465+0000 7faf3d0ee700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7faf380cecf0 0x7faf3806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.465+0000 7faf3d0ee700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf380b30b0 con 0x7faf3815c560 2023-11-28T11:09:35.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.465+0000 7faf3d0ee700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf3806c020 con 0x7faf380cecf0 2023-11-28T11:09:35.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.465+0000 7faf3d0ee700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf380cf130 con 0x7faf380b2660 2023-11-28T11:09:35.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.466+0000 7faf3759e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7faf380cecf0 0x7faf3806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.466+0000 7faf3759e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7faf380cecf0 0x7faf3806c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48126/0 (socket says 172.21.15.28:48126) 2023-11-28T11:09:35.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.466+0000 7faf3759e700 1 -- 172.21.15.28:0/284617382 learned_addr learned my addr 172.21.15.28:0/284617382 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:35.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.466+0000 7faf3659c700 1 --2- 172.21.15.28:0/284617382 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf380b2660 0x7faf380ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.466+0000 7faf3759e700 1 -- 172.21.15.28:0/284617382 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf380b2660 msgr2=0x7faf380ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:35.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.466+0000 7faf3759e700 1 --2- 172.21.15.28:0/284617382 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf380b2660 0x7faf380ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.466+0000 7faf3759e700 1 -- 172.21.15.28:0/284617382 >> v1:172.21.15.59:6789/0 conn(0x7faf3815c560 legacy=0x7faf3806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.466+0000 7faf3759e700 1 -- 172.21.15.28:0/284617382 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380d09c0 con 0x7faf380cecf0 2023-11-28T11:09:35.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.466+0000 7faf3759e700 1 --2- 172.21.15.28:0/284617382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf380cecf0 0x7faf3806c4b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7faf30009e50 tx=0x7faf3000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26ce6e0ed9fde6cb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:35.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.466+0000 7faf35d9b700 1 -- 172.21.15.28:0/284617382 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3000e070 con 0x7faf380cecf0 2023-11-28T11:09:35.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.466+0000 7faf35d9b700 1 -- 172.21.15.28:0/284617382 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3000b9c0 con 0x7faf380cecf0 2023-11-28T11:09:35.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.466+0000 7faf35d9b700 1 -- 172.21.15.28:0/284617382 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3000bd70 con 0x7faf380cecf0 2023-11-28T11:09:35.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.467+0000 7faf3d0ee700 1 -- 172.21.15.28:0/284617382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf380cecf0 msgr2=0x7faf3806c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.467+0000 7faf3d0ee700 1 --2- 172.21.15.28:0/284617382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf380cecf0 0x7faf3806c4b0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7faf30009e50 tx=0x7faf3000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:35.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.467+0000 7faf3d0ee700 1 -- 172.21.15.28:0/284617382 shutdown_connections 2023-11-28T11:09:35.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.467+0000 7faf3d0ee700 1 --2- 172.21.15.28:0/284617382 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf380cecf0 0x7faf3806c4b0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.467+0000 7faf3d0ee700 1 --2- 172.21.15.28:0/284617382 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf380b2660 0x7faf380ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.467+0000 7faf3d0ee700 1 -- 172.21.15.28:0/284617382 >> 172.21.15.28:0/284617382 conn(0x7faf380cd660 msgr2=0x7faf380cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:35.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.467+0000 7faf3d0ee700 1 -- 172.21.15.28:0/284617382 shutdown_connections 2023-11-28T11:09:35.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.467+0000 7faf3d0ee700 1 -- 172.21.15.28:0/284617382 wait complete. 2023-11-28T11:09:35.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.468+0000 7faf3d0ee700 1 Processor -- start 2023-11-28T11:09:35.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.468+0000 7faf3d0ee700 1 -- start start 2023-11-28T11:09:35.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.469+0000 7faf3d0ee700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf380b2660 0x7faf3814ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.469+0000 7faf3d0ee700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf380cecf0 0x7faf380ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.469+0000 7faf3d0ee700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf3815c560 0x7faf380add00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.469+0000 7faf3d0ee700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf380ae5d0 con 0x7faf380cecf0 2023-11-28T11:09:35.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.469+0000 7faf3d0ee700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf380ae8b0 con 0x7faf3815c560 2023-11-28T11:09:35.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.469+0000 7faf3d0ee700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf380aeb90 con 0x7faf380b2660 2023-11-28T11:09:35.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.469+0000 7faf3659c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf380cecf0 0x7faf380ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.469+0000 7faf3659c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf380cecf0 0x7faf380ad7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48130/0 (socket says 172.21.15.28:48130) 2023-11-28T11:09:35.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.469+0000 7faf3659c700 1 -- 172.21.15.28:0/2254734012 learned_addr learned my addr 172.21.15.28:0/2254734012 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:35.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.469+0000 7faf36d9d700 1 --2- 172.21.15.28:0/2254734012 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf380b2660 0x7faf3814ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.469+0000 7faf3659c700 1 -- 172.21.15.28:0/2254734012 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf380b2660 msgr2=0x7faf3814ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.469+0000 7faf3659c700 1 --2- 172.21.15.28:0/2254734012 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf380b2660 0x7faf3814ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.469+0000 7faf3659c700 1 -- 172.21.15.28:0/2254734012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf3815c560 msgr2=0x7faf380add00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.469+0000 7faf3759e700 1 --2- 172.21.15.28:0/2254734012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf3815c560 0x7faf380add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.470+0000 7faf3659c700 1 --2- 172.21.15.28:0/2254734012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf3815c560 0x7faf380add00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.470+0000 7faf3659c700 1 -- 172.21.15.28:0/2254734012 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf30009b30 con 0x7faf380cecf0 2023-11-28T11:09:35.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.470+0000 7faf3759e700 1 --2- 172.21.15.28:0/2254734012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf3815c560 0x7faf380add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:35.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.470+0000 7faf3659c700 1 --2- 172.21.15.28:0/2254734012 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf380cecf0 0x7faf380ad7c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00e810 tx=0x7faf2c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:35.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.470+0000 7faf1f7fe700 1 -- 172.21.15.28:0/2254734012 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c013610 con 0x7faf380cecf0 2023-11-28T11:09:35.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.470+0000 7faf1f7fe700 1 -- 172.21.15.28:0/2254734012 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c013770 con 0x7faf380cecf0 2023-11-28T11:09:35.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.470+0000 7faf3d0ee700 1 -- 172.21.15.28:0/2254734012 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf380b2dd0 con 0x7faf380cecf0 2023-11-28T11:09:35.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.470+0000 7faf1f7fe700 1 -- 172.21.15.28:0/2254734012 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c013b20 con 0x7faf380cecf0 2023-11-28T11:09:35.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.470+0000 7faf3d0ee700 1 -- 172.21.15.28:0/2254734012 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf380b2af0 con 0x7faf380cecf0 2023-11-28T11:09:35.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.471+0000 7faf3d0ee700 1 -- 172.21.15.28:0/2254734012 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf3806bcb0 con 0x7faf380cecf0 2023-11-28T11:09:35.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.471+0000 7faf1f7fe700 1 -- 172.21.15.28:0/2254734012 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faf2c023070 con 0x7faf380cecf0 2023-11-28T11:09:35.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.472+0000 7faf1f7fe700 1 --2- 172.21.15.28:0/2254734012 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faf240682f0 0x7faf2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.472+0000 7faf1f7fe700 1 -- 172.21.15.28:0/2254734012 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7faf2c085ce0 con 0x7faf380cecf0 2023-11-28T11:09:35.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.472+0000 7faf36d9d700 1 --2- 172.21.15.28:0/2254734012 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faf240682f0 0x7faf2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.472+0000 7faf36d9d700 1 --2- 172.21.15.28:0/2254734012 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faf240682f0 0x7faf2406a7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faf28007fd0 tx=0x7faf28009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:35.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.476+0000 7faf1f7fe700 1 -- 172.21.15.28:0/2254734012 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faf2c052190 con 0x7faf380cecf0 2023-11-28T11:09:35.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.671+0000 7faf3d0ee700 1 -- 172.21.15.28:0/2254734012 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7faf380b4730 con 0x7faf380cecf0 2023-11-28T11:09:35.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.673+0000 7faf1f7fe700 1 -- 172.21.15.28:0/2254734012 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19737 (secure 0 0 0) 0x7faf2c0557b0 con 0x7faf380cecf0 2023-11-28T11:09:35.674 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:35.675 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:09:34.614188+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.675+0000 7faf3d0ee700 1 -- 172.21.15.28:0/2254734012 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faf240682f0 msgr2=0x7faf2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.675+0000 7faf3d0ee700 1 --2- 172.21.15.28:0/2254734012 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faf240682f0 0x7faf2406a7a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7faf28007fd0 tx=0x7faf28009040 comp rx=0 tx=0).stop 2023-11-28T11:09:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.675+0000 7faf3d0ee700 1 -- 172.21.15.28:0/2254734012 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf380cecf0 msgr2=0x7faf380ad7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.675+0000 7faf3d0ee700 1 --2- 172.21.15.28:0/2254734012 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf380cecf0 0x7faf380ad7c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faf2c00e810 tx=0x7faf2c00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.675+0000 7faf3d0ee700 1 -- 172.21.15.28:0/2254734012 shutdown_connections 2023-11-28T11:09:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.675+0000 7faf3d0ee700 1 --2- 172.21.15.28:0/2254734012 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faf240682f0 0x7faf2406a7a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.675+0000 7faf3d0ee700 1 --2- 172.21.15.28:0/2254734012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf3815c560 0x7faf380add00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.675+0000 7faf3d0ee700 1 --2- 172.21.15.28:0/2254734012 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf380b2660 0x7faf3814ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.675+0000 7faf3d0ee700 1 --2- 172.21.15.28:0/2254734012 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf380cecf0 0x7faf380ad7c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.675+0000 7faf3d0ee700 1 -- 172.21.15.28:0/2254734012 >> 172.21.15.28:0/2254734012 conn(0x7faf380cd660 msgr2=0x7faf380b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:35.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.675+0000 7faf3d0ee700 1 -- 172.21.15.28:0/2254734012 shutdown_connections 2023-11-28T11:09:35.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.675+0000 7faf3d0ee700 1 -- 172.21.15.28:0/2254734012 wait complete. 2023-11-28T11:09:35.689 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:35.690 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:35.817 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.816+0000 7f85a8b41700 1 Processor -- start 2023-11-28T11:09:35.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.816+0000 7f85a8b41700 1 -- start start 2023-11-28T11:09:35.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.817+0000 7f85a8b41700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f85a40b1d40 0x7f85a40cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.817+0000 7f85a8b41700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85a415dbf0 0x7f85a40ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.817+0000 7f85a8b41700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a40ceff0 con 0x7f85a4159500 2023-11-28T11:09:35.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.817+0000 7f85a8b41700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85a415e090 con 0x7f85a40b1d40 2023-11-28T11:09:35.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.817+0000 7f85a8b41700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a40cede0 con 0x7f85a415dbf0 2023-11-28T11:09:35.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.817+0000 7f85a259c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f85a4159500 0x7f85a40b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36194/0 (socket says 172.21.15.28:36194) 2023-11-28T11:09:35.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.817+0000 7f85a259c700 1 -- 172.21.15.28:0/321636872 learned_addr learned my addr 172.21.15.28:0/321636872 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:35.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.817+0000 7f85a2d9d700 1 --2- 172.21.15.28:0/321636872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85a415dbf0 0x7f85a40ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.817+0000 7f85a1d9b700 1 --2- 172.21.15.28:0/321636872 >> v2:172.21.15.59:3300/0 conn(0x7f85a40b1d40 0x7f85a40cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.817+0000 7f85a159a700 1 -- 172.21.15.28:0/321636872 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4293213844 0 0) 0x7f85a40ceff0 con 0x7f85a4159500 2023-11-28T11:09:35.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.817+0000 7f85a159a700 1 -- 172.21.15.28:0/321636872 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f858c0035b0 con 0x7f85a4159500 2023-11-28T11:09:35.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.818+0000 7f85a2d9d700 1 -- 172.21.15.28:0/321636872 >> v2:172.21.15.59:3300/0 conn(0x7f85a40b1d40 msgr2=0x7f85a40cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:35.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.818+0000 7f85a2d9d700 1 --2- 172.21.15.28:0/321636872 >> v2:172.21.15.59:3300/0 conn(0x7f85a40b1d40 0x7f85a40cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.818+0000 7f85a2d9d700 1 -- 172.21.15.28:0/321636872 >> v1:172.21.15.28:6789/0 conn(0x7f85a4159500 legacy=0x7f85a40b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.818+0000 7f85a2d9d700 1 -- 172.21.15.28:0/321636872 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a40cf2d0 con 0x7f85a415dbf0 2023-11-28T11:09:35.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.818+0000 7f85a2d9d700 1 --2- 172.21.15.28:0/321636872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85a415dbf0 0x7f85a40ce8a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f859c009d90 tx=0x7f859c00b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=753d7840d84fc64b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:35.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.818+0000 7f85a159a700 1 -- 172.21.15.28:0/321636872 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c00e070 con 0x7f85a415dbf0 2023-11-28T11:09:35.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.818+0000 7f85a159a700 1 -- 172.21.15.28:0/321636872 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859c00bb20 con 0x7f85a415dbf0 2023-11-28T11:09:35.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.818+0000 7f85a159a700 1 -- 172.21.15.28:0/321636872 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c00be90 con 0x7f85a415dbf0 2023-11-28T11:09:35.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.819+0000 7f85a8b41700 1 -- 172.21.15.28:0/321636872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85a415dbf0 msgr2=0x7f85a40ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.819+0000 7f85a8b41700 1 --2- 172.21.15.28:0/321636872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85a415dbf0 0x7f85a40ce8a0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f859c009d90 tx=0x7f859c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:35.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.819+0000 7f85a8b41700 1 -- 172.21.15.28:0/321636872 shutdown_connections 2023-11-28T11:09:35.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.819+0000 7f85a8b41700 1 --2- 172.21.15.28:0/321636872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85a415dbf0 0x7f85a40ce8a0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.819+0000 7f85a8b41700 1 --2- 172.21.15.28:0/321636872 >> v2:172.21.15.59:3300/0 conn(0x7f85a40b1d40 0x7f85a40cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.819+0000 7f85a8b41700 1 -- 172.21.15.28:0/321636872 >> 172.21.15.28:0/321636872 conn(0x7f85a40cc630 msgr2=0x7f85a40cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:35.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.819+0000 7f85a8b41700 1 -- 172.21.15.28:0/321636872 shutdown_connections 2023-11-28T11:09:35.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.819+0000 7f85a8b41700 1 -- 172.21.15.28:0/321636872 wait complete. 2023-11-28T11:09:35.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.820+0000 7f85a8b41700 1 Processor -- start 2023-11-28T11:09:35.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.820+0000 7f85a8b41700 1 -- start start 2023-11-28T11:09:35.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a8b41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a40b1d40 0x7f85a41f4080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a8b41700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85a4159500 0x7f85a4154f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a8b41700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85a415dbf0 0x7f85a41558e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a8b41700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a40c5450 con 0x7f85a4159500 2023-11-28T11:09:35.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a8b41700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a40c45f0 con 0x7f85a40b1d40 2023-11-28T11:09:35.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a8b41700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a40c4840 con 0x7f85a415dbf0 2023-11-28T11:09:35.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a1d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85a4159500 0x7f85a4154f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a1d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85a4159500 0x7f85a4154f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48142/0 (socket says 172.21.15.28:48142) 2023-11-28T11:09:35.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a1d9b700 1 -- 172.21.15.28:0/3787582911 learned_addr learned my addr 172.21.15.28:0/3787582911 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:35.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a2d9d700 1 --2- 172.21.15.28:0/3787582911 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85a415dbf0 0x7f85a41558e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a259c700 1 --2- 172.21.15.28:0/3787582911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a40b1d40 0x7f85a41f4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a1d9b700 1 -- 172.21.15.28:0/3787582911 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85a415dbf0 msgr2=0x7f85a41558e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a1d9b700 1 --2- 172.21.15.28:0/3787582911 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85a415dbf0 0x7f85a41558e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a1d9b700 1 -- 172.21.15.28:0/3787582911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a40b1d40 msgr2=0x7f85a41f4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:35.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a1d9b700 1 --2- 172.21.15.28:0/3787582911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a40b1d40 0x7f85a41f4080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:35.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a1d9b700 1 -- 172.21.15.28:0/3787582911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f859c009a70 con 0x7f85a4159500 2023-11-28T11:09:35.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.821+0000 7f85a2d9d700 1 --2- 172.21.15.28:0/3787582911 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85a415dbf0 0x7f85a41558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:35.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.822+0000 7f85a1d9b700 1 --2- 172.21.15.28:0/3787582911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85a4159500 0x7f85a4154f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f859400e870 tx=0x7f859400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:35.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.822+0000 7f8592ffd700 1 -- 172.21.15.28:0/3787582911 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859400c640 con 0x7f85a4159500 2023-11-28T11:09:35.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.822+0000 7f8592ffd700 1 -- 172.21.15.28:0/3787582911 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859400c7a0 con 0x7f85a4159500 2023-11-28T11:09:35.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.822+0000 7f85a8b41700 1 -- 172.21.15.28:0/3787582911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a415d3f0 con 0x7f85a4159500 2023-11-28T11:09:35.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.822+0000 7f8592ffd700 1 -- 172.21.15.28:0/3787582911 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859400cb10 con 0x7f85a4159500 2023-11-28T11:09:35.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.822+0000 7f85a8b41700 1 -- 172.21.15.28:0/3787582911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a415d110 con 0x7f85a4159500 2023-11-28T11:09:35.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.823+0000 7f85a8b41700 1 -- 172.21.15.28:0/3787582911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a415d6d0 con 0x7f85a4159500 2023-11-28T11:09:35.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.823+0000 7f8592ffd700 1 -- 172.21.15.28:0/3787582911 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8594019030 con 0x7f85a4159500 2023-11-28T11:09:35.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.824+0000 7f8592ffd700 1 --2- 172.21.15.28:0/3787582911 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f85840682f0 0x7f858406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:35.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.824+0000 7f8592ffd700 1 -- 172.21.15.28:0/3787582911 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8594085e00 con 0x7f85a4159500 2023-11-28T11:09:35.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.824+0000 7f85a259c700 1 --2- 172.21.15.28:0/3787582911 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f85840682f0 0x7f858406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:35.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.824+0000 7f85a259c700 1 --2- 172.21.15.28:0/3787582911 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f85840682f0 0x7f858406a7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8598005910 tx=0x7f8598009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:35.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:35.828+0000 7f8592ffd700 1 -- 172.21.15.28:0/3787582911 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f85940522b0 con 0x7f85a4159500 2023-11-28T11:09:36.021 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.020+0000 7f85a8b41700 1 -- 172.21.15.28:0/3787582911 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f85a40739f0 con 0x7f85a4159500 2023-11-28T11:09:36.022 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.020+0000 7f8592ffd700 1 -- 172.21.15.28:0/3787582911 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f85940558d0 con 0x7f85a4159500 2023-11-28T11:09:36.022 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:09:36.024 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.022+0000 7f85a8b41700 1 -- 172.21.15.28:0/3787582911 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f85840682f0 msgr2=0x7f858406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.024 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.022+0000 7f85a8b41700 1 --2- 172.21.15.28:0/3787582911 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f85840682f0 0x7f858406a7a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f8598005910 tx=0x7f8598009040 comp rx=0 tx=0).stop 2023-11-28T11:09:36.024 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.022+0000 7f85a8b41700 1 -- 172.21.15.28:0/3787582911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85a4159500 msgr2=0x7f85a4154f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.024 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.022+0000 7f85a8b41700 1 --2- 172.21.15.28:0/3787582911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85a4159500 0x7f85a4154f70 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f859400e870 tx=0x7f859400ec30 comp rx=0 tx=0).stop 2023-11-28T11:09:36.024 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.022+0000 7f85a8b41700 1 -- 172.21.15.28:0/3787582911 shutdown_connections 2023-11-28T11:09:36.024 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.022+0000 7f85a8b41700 1 --2- 172.21.15.28:0/3787582911 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f85840682f0 0x7f858406a7a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.024 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.022+0000 7f85a8b41700 1 --2- 172.21.15.28:0/3787582911 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f85a415dbf0 0x7f85a41558e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.022+0000 7f85a8b41700 1 --2- 172.21.15.28:0/3787582911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f85a40b1d40 0x7f85a41f4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.022+0000 7f85a8b41700 1 --2- 172.21.15.28:0/3787582911 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f85a4159500 0x7f85a4154f70 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.022+0000 7f85a8b41700 1 -- 172.21.15.28:0/3787582911 >> 172.21.15.28:0/3787582911 conn(0x7f85a40cc630 msgr2=0x7f85a40cbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:36.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.022+0000 7f85a8b41700 1 -- 172.21.15.28:0/3787582911 shutdown_connections 2023-11-28T11:09:36.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.022+0000 7f85a8b41700 1 -- 172.21.15.28:0/3787582911 wait complete. 2023-11-28T11:09:36.036 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:36.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.162+0000 7fc45ae81700 1 Processor -- start 2023-11-28T11:09:36.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.162+0000 7fc45ae81700 1 -- start start 2023-11-28T11:09:36.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.163+0000 7fc45ae81700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc45415a490 0x7fc45406f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.163+0000 7fc45ae81700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc45406dd40 0x7fc4540ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.163+0000 7fc45ae81700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc45415a860 con 0x7fc4540b2150 2023-11-28T11:09:36.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.163+0000 7fc45ae81700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45406b510 con 0x7fc45406dd40 2023-11-28T11:09:36.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.163+0000 7fc45ae81700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4540c9a80 con 0x7fc45415a490 2023-11-28T11:09:36.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.163+0000 7fc45941e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc45406dd40 0x7fc4540ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.163+0000 7fc45941e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc45406dd40 0x7fc4540ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48154/0 (socket says 172.21.15.28:48154) 2023-11-28T11:09:36.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.163+0000 7fc45941e700 1 -- 172.21.15.28:0/1418000128 learned_addr learned my addr 172.21.15.28:0/1418000128 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:36.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.163+0000 7fc458c1d700 1 --2- 172.21.15.28:0/1418000128 >> v2:172.21.15.59:3300/0 conn(0x7fc45415a490 0x7fc45406f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.164+0000 7fc45941e700 1 -- 172.21.15.28:0/1418000128 >> v2:172.21.15.59:3300/0 conn(0x7fc45415a490 msgr2=0x7fc45406f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:36.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.164+0000 7fc45941e700 1 --2- 172.21.15.28:0/1418000128 >> v2:172.21.15.59:3300/0 conn(0x7fc45415a490 0x7fc45406f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.164+0000 7fc45941e700 1 -- 172.21.15.28:0/1418000128 >> v1:172.21.15.59:6789/0 conn(0x7fc4540b2150 legacy=0x7fc45406d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.164+0000 7fc45941e700 1 -- 172.21.15.28:0/1418000128 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45406b230 con 0x7fc45406dd40 2023-11-28T11:09:36.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.164+0000 7fc45941e700 1 --2- 172.21.15.28:0/1418000128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc45406dd40 0x7fc4540ca520 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc4540cf1b0 tx=0x7fc44c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974b4ebec512e6f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:36.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.164+0000 7fc4537fe700 1 -- 172.21.15.28:0/1418000128 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c00e070 con 0x7fc45406dd40 2023-11-28T11:09:36.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.164+0000 7fc4537fe700 1 -- 172.21.15.28:0/1418000128 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44c00b910 con 0x7fc45406dd40 2023-11-28T11:09:36.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.164+0000 7fc4537fe700 1 -- 172.21.15.28:0/1418000128 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c00bcc0 con 0x7fc45406dd40 2023-11-28T11:09:36.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.164+0000 7fc45ae81700 1 -- 172.21.15.28:0/1418000128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc45406dd40 msgr2=0x7fc4540ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.164+0000 7fc45ae81700 1 --2- 172.21.15.28:0/1418000128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc45406dd40 0x7fc4540ca520 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc4540cf1b0 tx=0x7fc44c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:36.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.164+0000 7fc45ae81700 1 -- 172.21.15.28:0/1418000128 shutdown_connections 2023-11-28T11:09:36.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.164+0000 7fc45ae81700 1 --2- 172.21.15.28:0/1418000128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc45406dd40 0x7fc4540ca520 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.164+0000 7fc45ae81700 1 --2- 172.21.15.28:0/1418000128 >> v2:172.21.15.59:3300/0 conn(0x7fc45415a490 0x7fc45406f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.165+0000 7fc45ae81700 1 -- 172.21.15.28:0/1418000128 >> 172.21.15.28:0/1418000128 conn(0x7fc4540b0df0 msgr2=0x7fc4540b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:36.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.165+0000 7fc45ae81700 1 -- 172.21.15.28:0/1418000128 shutdown_connections 2023-11-28T11:09:36.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.165+0000 7fc45ae81700 1 -- 172.21.15.28:0/1418000128 wait complete. 2023-11-28T11:09:36.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.166+0000 7fc45ae81700 1 Processor -- start 2023-11-28T11:09:36.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.166+0000 7fc45ae81700 1 -- start start 2023-11-28T11:09:36.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc45ae81700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc45406dd40 0x7fc45414fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc45ae81700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4540b2150 0x7fc4540c8a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc45ae81700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc45415a490 0x7fc4540c6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc45ae81700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc454150ec0 con 0x7fc45406dd40 2023-11-28T11:09:36.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc45ae81700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc45415bad0 con 0x7fc4540b2150 2023-11-28T11:09:36.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc45ae81700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4541503c0 con 0x7fc45415a490 2023-11-28T11:09:36.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc458c1d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc45406dd40 0x7fc45414fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc458c1d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc45406dd40 0x7fc45414fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48156/0 (socket says 172.21.15.28:48156) 2023-11-28T11:09:36.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc458c1d700 1 -- 172.21.15.28:0/1695632600 learned_addr learned my addr 172.21.15.28:0/1695632600 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:36.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc45941e700 1 --2- 172.21.15.28:0/1695632600 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc45415a490 0x7fc4540c6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc453fff700 1 --2- 172.21.15.28:0/1695632600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4540b2150 0x7fc4540c8a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc458c1d700 1 -- 172.21.15.28:0/1695632600 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc45415a490 msgr2=0x7fc4540c6bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc458c1d700 1 --2- 172.21.15.28:0/1695632600 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc45415a490 0x7fc4540c6bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc458c1d700 1 -- 172.21.15.28:0/1695632600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4540b2150 msgr2=0x7fc4540c8a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc458c1d700 1 --2- 172.21.15.28:0/1695632600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4540b2150 0x7fc4540c8a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc458c1d700 1 -- 172.21.15.28:0/1695632600 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc44c009b30 con 0x7fc45406dd40 2023-11-28T11:09:36.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc458c1d700 1 --2- 172.21.15.28:0/1695632600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc45406dd40 0x7fc45414fe80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc44800e9e0 tx=0x7fc44800ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:36.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc4517fa700 1 -- 172.21.15.28:0/1695632600 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc448009710 con 0x7fc45406dd40 2023-11-28T11:09:36.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc4517fa700 1 -- 172.21.15.28:0/1695632600 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44800ee80 con 0x7fc45406dd40 2023-11-28T11:09:36.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc45ae81700 1 -- 172.21.15.28:0/1695632600 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4540ca000 con 0x7fc45406dd40 2023-11-28T11:09:36.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.167+0000 7fc4517fa700 1 -- 172.21.15.28:0/1695632600 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44800ca50 con 0x7fc45406dd40 2023-11-28T11:09:36.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.168+0000 7fc45ae81700 1 -- 172.21.15.28:0/1695632600 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4540c9d20 con 0x7fc45406dd40 2023-11-28T11:09:36.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.168+0000 7fc45ae81700 1 -- 172.21.15.28:0/1695632600 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45406ae90 con 0x7fc45406dd40 2023-11-28T11:09:36.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.169+0000 7fc4517fa700 1 -- 172.21.15.28:0/1695632600 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc448019030 con 0x7fc45406dd40 2023-11-28T11:09:36.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.169+0000 7fc4517fa700 1 --2- 172.21.15.28:0/1695632600 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc4380682f0 0x7fc43806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.169+0000 7fc4517fa700 1 -- 172.21.15.28:0/1695632600 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc448085ed0 con 0x7fc45406dd40 2023-11-28T11:09:36.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.169+0000 7fc453fff700 1 --2- 172.21.15.28:0/1695632600 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc4380682f0 0x7fc43806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.170+0000 7fc453fff700 1 --2- 172.21.15.28:0/1695632600 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc4380682f0 0x7fc43806a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc454154620 tx=0x7fc440009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:36.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.174+0000 7fc4517fa700 1 -- 172.21.15.28:0/1695632600 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc448052380 con 0x7fc45406dd40 2023-11-28T11:09:36.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.367+0000 7fc45ae81700 1 -- 172.21.15.28:0/1695632600 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc4540739f0 con 0x7fc45406dd40 2023-11-28T11:09:36.369 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.367+0000 7fc4517fa700 1 -- 172.21.15.28:0/1695632600 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc4480559a0 con 0x7fc45406dd40 2023-11-28T11:09:36.369 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:09:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.369+0000 7fc45ae81700 1 -- 172.21.15.28:0/1695632600 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc4380682f0 msgr2=0x7fc43806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.369+0000 7fc45ae81700 1 --2- 172.21.15.28:0/1695632600 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc4380682f0 0x7fc43806a7a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc454154620 tx=0x7fc440009040 comp rx=0 tx=0).stop 2023-11-28T11:09:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.369+0000 7fc45ae81700 1 -- 172.21.15.28:0/1695632600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc45406dd40 msgr2=0x7fc45414fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.369+0000 7fc45ae81700 1 --2- 172.21.15.28:0/1695632600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc45406dd40 0x7fc45414fe80 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fc44800e9e0 tx=0x7fc44800ecf0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.369+0000 7fc45ae81700 1 -- 172.21.15.28:0/1695632600 shutdown_connections 2023-11-28T11:09:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.369+0000 7fc45ae81700 1 --2- 172.21.15.28:0/1695632600 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc4380682f0 0x7fc43806a7a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.369+0000 7fc45ae81700 1 --2- 172.21.15.28:0/1695632600 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc45406dd40 0x7fc45414fe80 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.371 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.369+0000 7fc45ae81700 1 --2- 172.21.15.28:0/1695632600 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc45415a490 0x7fc4540c6bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.369+0000 7fc45ae81700 1 --2- 172.21.15.28:0/1695632600 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4540b2150 0x7fc4540c8a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.369+0000 7fc45ae81700 1 -- 172.21.15.28:0/1695632600 >> 172.21.15.28:0/1695632600 conn(0x7fc4540b0df0 msgr2=0x7fc4540b0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:36.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.369+0000 7fc45ae81700 1 -- 172.21.15.28:0/1695632600 shutdown_connections 2023-11-28T11:09:36.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.369+0000 7fc45ae81700 1 -- 172.21.15.28:0/1695632600 wait complete. 2023-11-28T11:09:36.383 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:36.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.509+0000 7f1f6bfaa700 1 Processor -- start 2023-11-28T11:09:36.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.509+0000 7f1f6bfaa700 1 -- start start 2023-11-28T11:09:36.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f6bfaa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f6406bf30 0x7f1f6406c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f6bfaa700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f640b6350 con 0x7f1f640d2820 2023-11-28T11:09:36.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f6bfaa700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f6406c870 con 0x7f1f6415c640 2023-11-28T11:09:36.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f6bfaa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6415ca10 con 0x7f1f6406bf30 2023-11-28T11:09:36.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f69545700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1f640d2820 0x7f1f640b5c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36224/0 (socket says 172.21.15.28:36224) 2023-11-28T11:09:36.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f69545700 1 -- 172.21.15.28:0/1428458732 learned_addr learned my addr 172.21.15.28:0/1428458732 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:36.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f6a547700 1 --2- 172.21.15.28:0/1428458732 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f6406bf30 0x7f1f6406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f68d44700 1 -- 172.21.15.28:0/1428458732 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3675312769 0 0) 0x7f1f640b6350 con 0x7f1f640d2820 2023-11-28T11:09:36.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f68d44700 1 -- 172.21.15.28:0/1428458732 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f500035b0 con 0x7f1f640d2820 2023-11-28T11:09:36.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f68d44700 1 -- 172.21.15.28:0/1428458732 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4251512243 0 0) 0x7f1f500035b0 con 0x7f1f640d2820 2023-11-28T11:09:36.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f68d44700 1 -- 172.21.15.28:0/1428458732 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f6406bf30 msgr2=0x7f1f6406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:36.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f68d44700 1 --2- 172.21.15.28:0/1428458732 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f6406bf30 0x7f1f6406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f68d44700 1 -- 172.21.15.28:0/1428458732 >> v1:172.21.15.59:6789/0 conn(0x7f1f6415c640 legacy=0x7f1f640cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f68d44700 1 -- 172.21.15.28:0/1428458732 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f640d1470 con 0x7f1f640d2820 2023-11-28T11:09:36.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f68d44700 1 -- 172.21.15.28:0/1428458732 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f1f54001670 con 0x7f1f640d2820 2023-11-28T11:09:36.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.510+0000 7f1f68d44700 1 -- 172.21.15.28:0/1428458732 >> v1:172.21.15.28:6789/0 conn(0x7f1f640d2820 legacy=0x7f1f640b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f68d44700 1 --2- 172.21.15.28:0/1428458732 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f6406bf30 0x7f1f50003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f68d44700 1 --2- 172.21.15.28:0/1428458732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f50003fb0 0x7f1f50004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f68d44700 1 --2- 172.21.15.28:0/1428458732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f50004d90 0x7f1f50005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f68d44700 1 -- 172.21.15.28:0/1428458732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f6415ca10 con 0x7f1f50004d90 2023-11-28T11:09:36.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f68d44700 1 -- 172.21.15.28:0/1428458732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5000bb80 con 0x7f1f50003fb0 2023-11-28T11:09:36.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f68d44700 1 -- 172.21.15.28:0/1428458732 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f5000bd40 con 0x7f1f6406bf30 2023-11-28T11:09:36.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f6a547700 1 --2- 172.21.15.28:0/1428458732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f50004d90 0x7f1f50005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f69545700 1 --2- 172.21.15.28:0/1428458732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f50003fb0 0x7f1f50004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f69d46700 1 --2- 172.21.15.28:0/1428458732 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f6406bf30 0x7f1f50003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f6a547700 1 -- 172.21.15.28:0/1428458732 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f6406bf30 msgr2=0x7f1f50003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:36.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f6a547700 1 --2- 172.21.15.28:0/1428458732 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f6406bf30 0x7f1f50003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f6a547700 1 -- 172.21.15.28:0/1428458732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f50003fb0 msgr2=0x7f1f50004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:36.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f6a547700 1 --2- 172.21.15.28:0/1428458732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f50003fb0 0x7f1f50004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f6a547700 1 -- 172.21.15.28:0/1428458732 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f500035b0 con 0x7f1f50004d90 2023-11-28T11:09:36.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f6a547700 1 --2- 172.21.15.28:0/1428458732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f50004d90 0x7f1f50005200 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009f90 tx=0x7f1f50005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a38a559b0fd46fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:36.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f68d44700 1 -- 172.21.15.28:0/1428458732 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c00ec30 con 0x7f1f50004d90 2023-11-28T11:09:36.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f68d44700 1 -- 172.21.15.28:0/1428458732 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c00f040 con 0x7f1f50004d90 2023-11-28T11:09:36.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.511+0000 7f1f68d44700 1 -- 172.21.15.28:0/1428458732 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c014480 con 0x7f1f50004d90 2023-11-28T11:09:36.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.512+0000 7f1f6bfaa700 1 -- 172.21.15.28:0/1428458732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f50004d90 msgr2=0x7f1f50005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.512+0000 7f1f6bfaa700 1 --2- 172.21.15.28:0/1428458732 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f50004d90 0x7f1f50005200 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1f5c009f90 tx=0x7f1f50005880 comp rx=0 tx=0).stop 2023-11-28T11:09:36.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.512+0000 7f1f6a547700 1 -- 172.21.15.28:0/1428458732 reap_dead start 2023-11-28T11:09:36.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.512+0000 7f1f6bfaa700 1 -- 172.21.15.28:0/1428458732 shutdown_connections 2023-11-28T11:09:36.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.512+0000 7f1f6bfaa700 1 -- 172.21.15.28:0/1428458732 >> 172.21.15.28:0/1428458732 conn(0x7f1f640d2df0 msgr2=0x7f1f640d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:36.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.512+0000 7f1f6bfaa700 1 -- 172.21.15.28:0/1428458732 shutdown_connections 2023-11-28T11:09:36.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.512+0000 7f1f6bfaa700 1 -- 172.21.15.28:0/1428458732 wait complete. 2023-11-28T11:09:36.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.513+0000 7f1f6bfaa700 1 Processor -- start 2023-11-28T11:09:36.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.513+0000 7f1f6bfaa700 1 -- start start 2023-11-28T11:09:36.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.514+0000 7f1f6bfaa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f641542a0 0x7f1f640d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.514+0000 7f1f6bfaa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f64154670 0x7f1f640d2070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.514+0000 7f1f6bfaa700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f64156370 0x7f1f641567f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.514+0000 7f1f6bfaa700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f641f52c0 con 0x7f1f64156370 2023-11-28T11:09:36.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.514+0000 7f1f6bfaa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f641f5530 con 0x7f1f64154670 2023-11-28T11:09:36.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.514+0000 7f1f6bfaa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f641f5810 con 0x7f1f641542a0 2023-11-28T11:09:36.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.514+0000 7f1f6a547700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f64156370 0x7f1f641567f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.514+0000 7f1f6a547700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f64156370 0x7f1f641567f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48180/0 (socket says 172.21.15.28:48180) 2023-11-28T11:09:36.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.514+0000 7f1f6a547700 1 -- 172.21.15.28:0/3987220246 learned_addr learned my addr 172.21.15.28:0/3987220246 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:36.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.514+0000 7f1f6a547700 1 -- 172.21.15.28:0/3987220246 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f641542a0 msgr2=0x7f1f640d1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.514+0000 7f1f6a547700 1 --2- 172.21.15.28:0/3987220246 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f641542a0 0x7f1f640d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.514+0000 7f1f6a547700 1 -- 172.21.15.28:0/3987220246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f64154670 msgr2=0x7f1f640d2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.514+0000 7f1f6a547700 1 --2- 172.21.15.28:0/3987220246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f64154670 0x7f1f640d2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.514+0000 7f1f6a547700 1 -- 172.21.15.28:0/3987220246 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f640cd7c0 con 0x7f1f64156370 2023-11-28T11:09:36.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.514+0000 7f1f6a547700 1 --2- 172.21.15.28:0/3987220246 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f64156370 0x7f1f641567f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000bb50 tx=0x7f1f5c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:36.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.515+0000 7f1f5a7fc700 1 -- 172.21.15.28:0/3987220246 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c009410 con 0x7f1f64156370 2023-11-28T11:09:36.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.515+0000 7f1f5a7fc700 1 -- 172.21.15.28:0/3987220246 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f5c022070 con 0x7f1f64156370 2023-11-28T11:09:36.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.515+0000 7f1f5a7fc700 1 -- 172.21.15.28:0/3987220246 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f5c014cc0 con 0x7f1f64156370 2023-11-28T11:09:36.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.515+0000 7f1f6bfaa700 1 -- 172.21.15.28:0/3987220246 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f640cd4e0 con 0x7f1f64156370 2023-11-28T11:09:36.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.515+0000 7f1f6bfaa700 1 -- 172.21.15.28:0/3987220246 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f640d1710 con 0x7f1f64156370 2023-11-28T11:09:36.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.516+0000 7f1f6bfaa700 1 -- 172.21.15.28:0/3987220246 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f640739f0 con 0x7f1f64156370 2023-11-28T11:09:36.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.516+0000 7f1f5a7fc700 1 -- 172.21.15.28:0/3987220246 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1f5c01e430 con 0x7f1f64156370 2023-11-28T11:09:36.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.516+0000 7f1f5a7fc700 1 --2- 172.21.15.28:0/3987220246 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f48068320 0x7f1f4806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.518+0000 7f1f5a7fc700 1 -- 172.21.15.28:0/3987220246 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1f5c024070 con 0x7f1f64156370 2023-11-28T11:09:36.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.518+0000 7f1f69d46700 1 --2- 172.21.15.28:0/3987220246 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f48068320 0x7f1f4806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.518+0000 7f1f69d46700 1 --2- 172.21.15.28:0/3987220246 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f48068320 0x7f1f4806a7d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1f60000c00 tx=0x7f1f60009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:36.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.522+0000 7f1f5a7fc700 1 -- 172.21.15.28:0/3987220246 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1f5c053ad0 con 0x7f1f64156370 2023-11-28T11:09:36.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.718+0000 7f1f6bfaa700 1 -- 172.21.15.28:0/3987220246 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1f640d1090 con 0x7f1f64156370 2023-11-28T11:09:36.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.718+0000 7f1f5a7fc700 1 -- 172.21.15.28:0/3987220246 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1f5c0570f0 con 0x7f1f64156370 2023-11-28T11:09:36.719 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:36.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.720+0000 7f1f6bfaa700 1 -- 172.21.15.28:0/3987220246 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f48068320 msgr2=0x7f1f4806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.720+0000 7f1f6bfaa700 1 --2- 172.21.15.28:0/3987220246 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f48068320 0x7f1f4806a7d0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1f60000c00 tx=0x7f1f60009040 comp rx=0 tx=0).stop 2023-11-28T11:09:36.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.720+0000 7f1f6bfaa700 1 -- 172.21.15.28:0/3987220246 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f64156370 msgr2=0x7f1f641567f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.720+0000 7f1f6bfaa700 1 --2- 172.21.15.28:0/3987220246 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f64156370 0x7f1f641567f0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1f5000bb50 tx=0x7f1f5c009750 comp rx=0 tx=0).stop 2023-11-28T11:09:36.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.720+0000 7f1f6bfaa700 1 -- 172.21.15.28:0/3987220246 shutdown_connections 2023-11-28T11:09:36.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.720+0000 7f1f6bfaa700 1 --2- 172.21.15.28:0/3987220246 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1f48068320 0x7f1f4806a7d0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.720+0000 7f1f6bfaa700 1 --2- 172.21.15.28:0/3987220246 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1f64156370 0x7f1f641567f0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.720+0000 7f1f6bfaa700 1 --2- 172.21.15.28:0/3987220246 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1f641542a0 0x7f1f640d1b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.720+0000 7f1f6bfaa700 1 --2- 172.21.15.28:0/3987220246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1f64154670 0x7f1f640d2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.720+0000 7f1f6bfaa700 1 -- 172.21.15.28:0/3987220246 >> 172.21.15.28:0/3987220246 conn(0x7f1f640d2df0 msgr2=0x7f1f6406aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:36.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.720+0000 7f1f6bfaa700 1 -- 172.21.15.28:0/3987220246 shutdown_connections 2023-11-28T11:09:36.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.720+0000 7f1f6bfaa700 1 -- 172.21.15.28:0/3987220246 wait complete. 2023-11-28T11:09:36.733 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:36.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.860+0000 7fe9f7070700 1 Processor -- start 2023-11-28T11:09:36.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.860+0000 7fe9f7070700 1 -- start start 2023-11-28T11:09:36.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.861+0000 7fe9f7070700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe9f00b1630 0x7fe9f00cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.861+0000 7fe9f7070700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f00ce550 con 0x7fe9f00b1a00 2023-11-28T11:09:36.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.861+0000 7fe9f7070700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f00ce830 con 0x7fe9f006afc0 2023-11-28T11:09:36.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.861+0000 7fe9f7070700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f00ceb10 con 0x7fe9f00b1630 2023-11-28T11:09:36.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.861+0000 7fe9f4e0c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe9f00b1630 0x7fe9f00cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.861+0000 7fe9f4e0c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe9f00b1630 0x7fe9f00cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48184/0 (socket says 172.21.15.28:48184) 2023-11-28T11:09:36.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.861+0000 7fe9f4e0c700 1 -- 172.21.15.28:0/3551665268 learned_addr learned my addr 172.21.15.28:0/3551665268 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:36.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.862+0000 7fe9f4e0c700 1 -- 172.21.15.28:0/3551665268 >> v1:172.21.15.59:6789/0 conn(0x7fe9f006afc0 legacy=0x7fe9f006b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:36.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.862+0000 7fe9f4e0c700 1 -- 172.21.15.28:0/3551665268 >> v1:172.21.15.28:6789/0 conn(0x7fe9f00b1a00 legacy=0x7fe9f00caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.862+0000 7fe9f4e0c700 1 -- 172.21.15.28:0/3551665268 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f00cedf0 con 0x7fe9f00b1630 2023-11-28T11:09:36.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.862+0000 7fe9f4e0c700 1 --2- 172.21.15.28:0/3551665268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9f00b1630 0x7fe9f00cf5f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009920 tx=0x7fe9e4009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=520b97ca26079925 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:36.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.862+0000 7fe9ef7fe700 1 -- 172.21.15.28:0/3551665268 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4004230 con 0x7fe9f00b1630 2023-11-28T11:09:36.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.862+0000 7fe9ef7fe700 1 -- 172.21.15.28:0/3551665268 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e4004390 con 0x7fe9f00b1630 2023-11-28T11:09:36.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.862+0000 7fe9ef7fe700 1 -- 172.21.15.28:0/3551665268 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e4009e70 con 0x7fe9f00b1630 2023-11-28T11:09:36.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.862+0000 7fe9f7070700 1 -- 172.21.15.28:0/3551665268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9f00b1630 msgr2=0x7fe9f00cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.862+0000 7fe9f7070700 1 --2- 172.21.15.28:0/3551665268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9f00b1630 0x7fe9f00cf5f0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009920 tx=0x7fe9e4009c30 comp rx=0 tx=0).stop 2023-11-28T11:09:36.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.862+0000 7fe9f7070700 1 -- 172.21.15.28:0/3551665268 shutdown_connections 2023-11-28T11:09:36.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.863+0000 7fe9f7070700 1 --2- 172.21.15.28:0/3551665268 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9f00b1630 0x7fe9f00cf5f0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.863+0000 7fe9f7070700 1 -- 172.21.15.28:0/3551665268 >> 172.21.15.28:0/3551665268 conn(0x7fe9f00cc630 msgr2=0x7fe9f00cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:36.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.863+0000 7fe9f7070700 1 -- 172.21.15.28:0/3551665268 shutdown_connections 2023-11-28T11:09:36.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.863+0000 7fe9f7070700 1 -- 172.21.15.28:0/3551665268 wait complete. 2023-11-28T11:09:36.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.864+0000 7fe9f7070700 1 Processor -- start 2023-11-28T11:09:36.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.864+0000 7fe9f7070700 1 -- start start 2023-11-28T11:09:36.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f7070700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f006afc0 0x7fe9f01f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f7070700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9f00b1630 0x7fe9f0157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f7070700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9f00b1a00 0x7fe9f0156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f7070700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01f2dd0 con 0x7fe9f00b1a00 2023-11-28T11:09:36.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f7070700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f01581a0 con 0x7fe9f006afc0 2023-11-28T11:09:36.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f7070700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f00b4490 con 0x7fe9f00b1630 2023-11-28T11:09:36.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f560d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9f00b1a00 0x7fe9f0156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f560d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9f00b1a00 0x7fe9f0156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48194/0 (socket says 172.21.15.28:48194) 2023-11-28T11:09:36.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f560d700 1 -- 172.21.15.28:0/2822322159 learned_addr learned my addr 172.21.15.28:0/2822322159 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:36.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f4e0c700 1 --2- 172.21.15.28:0/2822322159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f006afc0 0x7fe9f01f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9effff700 1 --2- 172.21.15.28:0/2822322159 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9f00b1630 0x7fe9f0157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f560d700 1 -- 172.21.15.28:0/2822322159 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9f00b1630 msgr2=0x7fe9f0157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f560d700 1 --2- 172.21.15.28:0/2822322159 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9f00b1630 0x7fe9f0157c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f560d700 1 -- 172.21.15.28:0/2822322159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f006afc0 msgr2=0x7fe9f01f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:36.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f560d700 1 --2- 172.21.15.28:0/2822322159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f006afc0 0x7fe9f01f3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:36.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f560d700 1 -- 172.21.15.28:0/2822322159 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e4009550 con 0x7fe9f00b1a00 2023-11-28T11:09:36.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f560d700 1 --2- 172.21.15.28:0/2822322159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9f00b1a00 0x7fe9f0156f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800cc10 tx=0x7fe9e800cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:36.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9ed7fa700 1 -- 172.21.15.28:0/2822322159 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e800aa60 con 0x7fe9f00b1a00 2023-11-28T11:09:36.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9ed7fa700 1 -- 172.21.15.28:0/2822322159 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e800abc0 con 0x7fe9f00b1a00 2023-11-28T11:09:36.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f7070700 1 -- 172.21.15.28:0/2822322159 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f00b1ea0 con 0x7fe9f00b1a00 2023-11-28T11:09:36.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9ed7fa700 1 -- 172.21.15.28:0/2822322159 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8012420 con 0x7fe9f00b1a00 2023-11-28T11:09:36.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.865+0000 7fe9f7070700 1 -- 172.21.15.28:0/2822322159 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f00cf0d0 con 0x7fe9f00b1a00 2023-11-28T11:09:36.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.866+0000 7fe9f7070700 1 -- 172.21.15.28:0/2822322159 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f00cb5b0 con 0x7fe9f00b1a00 2023-11-28T11:09:36.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.867+0000 7fe9ed7fa700 1 -- 172.21.15.28:0/2822322159 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe9e8010050 con 0x7fe9f00b1a00 2023-11-28T11:09:36.874 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.867+0000 7fe9ed7fa700 1 --2- 172.21.15.28:0/2822322159 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe9d40682f0 0x7fe9d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:36.874 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.867+0000 7fe9f4e0c700 1 --2- 172.21.15.28:0/2822322159 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe9d40682f0 0x7fe9d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:36.874 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.867+0000 7fe9ed7fa700 1 -- 172.21.15.28:0/2822322159 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe9e8086160 con 0x7fe9f00b1a00 2023-11-28T11:09:36.874 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.868+0000 7fe9f4e0c700 1 --2- 172.21.15.28:0/2822322159 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe9d40682f0 0x7fe9d406a7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009130 tx=0x7fe9e4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:36.874 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:36.872+0000 7fe9ed7fa700 1 -- 172.21.15.28:0/2822322159 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe9e8052630 con 0x7fe9f00b1a00 2023-11-28T11:09:37.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.068+0000 7fe9f7070700 1 -- 172.21.15.28:0/2822322159 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe9f00739f0 con 0x7fe9f00b1a00 2023-11-28T11:09:37.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.069+0000 7fe9ed7fa700 1 -- 172.21.15.28:0/2822322159 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe9e8055c50 con 0x7fe9f00b1a00 2023-11-28T11:09:37.071 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:09:37.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.071+0000 7fe9f7070700 1 -- 172.21.15.28:0/2822322159 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe9d40682f0 msgr2=0x7fe9d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.071+0000 7fe9f7070700 1 --2- 172.21.15.28:0/2822322159 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe9d40682f0 0x7fe9d406a7a0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fe9e4009130 tx=0x7fe9e4008030 comp rx=0 tx=0).stop 2023-11-28T11:09:37.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.071+0000 7fe9f7070700 1 -- 172.21.15.28:0/2822322159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9f00b1a00 msgr2=0x7fe9f0156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.071+0000 7fe9f7070700 1 --2- 172.21.15.28:0/2822322159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9f00b1a00 0x7fe9f0156f90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800cc10 tx=0x7fe9e800cf20 comp rx=0 tx=0).stop 2023-11-28T11:09:37.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.071+0000 7fe9f7070700 1 -- 172.21.15.28:0/2822322159 shutdown_connections 2023-11-28T11:09:37.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.071+0000 7fe9f7070700 1 --2- 172.21.15.28:0/2822322159 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe9d40682f0 0x7fe9d406a7a0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.071+0000 7fe9f7070700 1 --2- 172.21.15.28:0/2822322159 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe9f00b1a00 0x7fe9f0156f90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.071+0000 7fe9f7070700 1 --2- 172.21.15.28:0/2822322159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe9f006afc0 0x7fe9f01f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.071+0000 7fe9f7070700 1 --2- 172.21.15.28:0/2822322159 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe9f00b1630 0x7fe9f0157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.071+0000 7fe9f7070700 1 -- 172.21.15.28:0/2822322159 >> 172.21.15.28:0/2822322159 conn(0x7fe9f00cc630 msgr2=0x7fe9f00cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:37.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.071+0000 7fe9f7070700 1 -- 172.21.15.28:0/2822322159 shutdown_connections 2023-11-28T11:09:37.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.071+0000 7fe9f7070700 1 -- 172.21.15.28:0/2822322159 wait complete. 2023-11-28T11:09:37.084 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:37.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.211+0000 7f811b296700 1 Processor -- start 2023-11-28T11:09:37.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.211+0000 7f811b296700 1 -- start start 2023-11-28T11:09:37.215 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.212+0000 7f811b296700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f81140cf5f0 0x7f81140caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.215 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.212+0000 7f811b296700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f81140cb3e0 0x7f811406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.212+0000 7f811b296700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81140ce550 con 0x7f81140b1630 2023-11-28T11:09:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.212+0000 7f811b296700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81140cfa60 con 0x7f81140cb3e0 2023-11-28T11:09:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.212+0000 7f811b296700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81140ce830 con 0x7f81140cf5f0 2023-11-28T11:09:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.212+0000 7f8119833700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f81140cb3e0 0x7f811406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.212+0000 7f8119833700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f81140cb3e0 0x7f811406afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48202/0 (socket says 172.21.15.28:48202) 2023-11-28T11:09:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.212+0000 7f8119833700 1 -- 172.21.15.28:0/2339393549 learned_addr learned my addr 172.21.15.28:0/2339393549 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.212+0000 7f8118831700 1 --2- 172.21.15.28:0/2339393549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f81140cf5f0 0x7f81140caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.212+0000 7f8119833700 1 -- 172.21.15.28:0/2339393549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f81140cf5f0 msgr2=0x7f81140caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:37.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.212+0000 7f8119833700 1 --2- 172.21.15.28:0/2339393549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f81140cf5f0 0x7f81140caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.213+0000 7f8119833700 1 -- 172.21.15.28:0/2339393549 >> v1:172.21.15.59:6789/0 conn(0x7f81140b1630 legacy=0x7f81140b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:37.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.213+0000 7f8119833700 1 -- 172.21.15.28:0/2339393549 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81140ceb10 con 0x7f81140cb3e0 2023-11-28T11:09:37.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.213+0000 7f8118831700 1 --2- 172.21.15.28:0/2339393549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f81140cf5f0 0x7f81140caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:37.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.213+0000 7f8119833700 1 --2- 172.21.15.28:0/2339393549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81140cb3e0 0x7f811406afc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f810c009e50 tx=0x7f810c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff331045661a1c7f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:37.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.213+0000 7f810bfff700 1 -- 172.21.15.28:0/2339393549 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c00e070 con 0x7f81140cb3e0 2023-11-28T11:09:37.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.213+0000 7f810bfff700 1 -- 172.21.15.28:0/2339393549 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f810c003a20 con 0x7f81140cb3e0 2023-11-28T11:09:37.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.213+0000 7f810bfff700 1 -- 172.21.15.28:0/2339393549 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f810c003dd0 con 0x7f81140cb3e0 2023-11-28T11:09:37.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.213+0000 7f811b296700 1 -- 172.21.15.28:0/2339393549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81140cb3e0 msgr2=0x7f811406afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.213+0000 7f811b296700 1 --2- 172.21.15.28:0/2339393549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81140cb3e0 0x7f811406afc0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f810c009e50 tx=0x7f810c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:37.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.213+0000 7f811b296700 1 -- 172.21.15.28:0/2339393549 shutdown_connections 2023-11-28T11:09:37.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.213+0000 7f811b296700 1 --2- 172.21.15.28:0/2339393549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81140cb3e0 0x7f811406afc0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.213+0000 7f811b296700 1 --2- 172.21.15.28:0/2339393549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f81140cf5f0 0x7f81140caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.213+0000 7f811b296700 1 -- 172.21.15.28:0/2339393549 >> 172.21.15.28:0/2339393549 conn(0x7f81140cc630 msgr2=0x7f81140cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:37.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.214+0000 7f811b296700 1 -- 172.21.15.28:0/2339393549 shutdown_connections 2023-11-28T11:09:37.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.214+0000 7f811b296700 1 -- 172.21.15.28:0/2339393549 wait complete. 2023-11-28T11:09:37.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.215+0000 7f811b296700 1 Processor -- start 2023-11-28T11:09:37.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.215+0000 7f811b296700 1 -- start start 2023-11-28T11:09:37.222 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f811b296700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81140b1630 0x7f81140abad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.222 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f811b296700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81140cb3e0 0x7f81140b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.222 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f811b296700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f81140cf5f0 0x7f81140c78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.222 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f811b296700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81140ac120 con 0x7f81140cb3e0 2023-11-28T11:09:37.222 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f811b296700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81140adf40 con 0x7f81140b1630 2023-11-28T11:09:37.222 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f811b296700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f811415b9f0 con 0x7f81140cf5f0 2023-11-28T11:09:37.222 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f8118831700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81140cb3e0 0x7f81140b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f8118831700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81140cb3e0 0x7f81140b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48206/0 (socket says 172.21.15.28:48206) 2023-11-28T11:09:37.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f8118831700 1 -- 172.21.15.28:0/2145438008 learned_addr learned my addr 172.21.15.28:0/2145438008 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:37.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f8119032700 1 --2- 172.21.15.28:0/2145438008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81140b1630 0x7f81140abad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f8119833700 1 --2- 172.21.15.28:0/2145438008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f81140cf5f0 0x7f81140c78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f8118831700 1 -- 172.21.15.28:0/2145438008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f81140cf5f0 msgr2=0x7f81140c78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f8118831700 1 --2- 172.21.15.28:0/2145438008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f81140cf5f0 0x7f81140c78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f8118831700 1 -- 172.21.15.28:0/2145438008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81140b1630 msgr2=0x7f81140abad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f8118831700 1 --2- 172.21.15.28:0/2145438008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81140b1630 0x7f81140abad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f8118831700 1 -- 172.21.15.28:0/2145438008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f810c009b30 con 0x7f81140cb3e0 2023-11-28T11:09:37.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f8118831700 1 --2- 172.21.15.28:0/2145438008 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81140cb3e0 0x7f81140b29e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f810400c820 tx=0x7f810400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:37.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f8109ffb700 1 -- 172.21.15.28:0/2145438008 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8104011800 con 0x7f81140cb3e0 2023-11-28T11:09:37.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f8109ffb700 1 -- 172.21.15.28:0/2145438008 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8104011960 con 0x7f81140cb3e0 2023-11-28T11:09:37.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f8109ffb700 1 -- 172.21.15.28:0/2145438008 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8104011cf0 con 0x7f81140cb3e0 2023-11-28T11:09:37.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.216+0000 7f811b296700 1 -- 172.21.15.28:0/2145438008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81140cf0d0 con 0x7f81140cb3e0 2023-11-28T11:09:37.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.217+0000 7f811b296700 1 -- 172.21.15.28:0/2145438008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81140cedf0 con 0x7f81140cb3e0 2023-11-28T11:09:37.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.218+0000 7f811b296700 1 -- 172.21.15.28:0/2145438008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81140ce140 con 0x7f81140cb3e0 2023-11-28T11:09:37.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.218+0000 7f8109ffb700 1 -- 172.21.15.28:0/2145438008 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8104017030 con 0x7f81140cb3e0 2023-11-28T11:09:37.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.218+0000 7f8109ffb700 1 --2- 172.21.15.28:0/2145438008 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80f80682f0 0x7f80f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.218+0000 7f8109ffb700 1 -- 172.21.15.28:0/2145438008 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8104086070 con 0x7f81140cb3e0 2023-11-28T11:09:37.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.218+0000 7f8119032700 1 --2- 172.21.15.28:0/2145438008 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80f80682f0 0x7f80f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.219+0000 7f8119032700 1 --2- 172.21.15.28:0/2145438008 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80f80682f0 0x7f80f806a7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8110007fd0 tx=0x7f8110007ec0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:37.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.223+0000 7f8109ffb700 1 -- 172.21.15.28:0/2145438008 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f81040525a0 con 0x7f81140cb3e0 2023-11-28T11:09:37.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.417+0000 7f811b296700 1 -- 172.21.15.28:0/2145438008 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f81140739f0 con 0x7f81140cb3e0 2023-11-28T11:09:37.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.418+0000 7f8109ffb700 1 -- 172.21.15.28:0/2145438008 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f810401f020 con 0x7f81140cb3e0 2023-11-28T11:09:37.418 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:37.420 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.420+0000 7f811b296700 1 -- 172.21.15.28:0/2145438008 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80f80682f0 msgr2=0x7f80f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.420+0000 7f811b296700 1 --2- 172.21.15.28:0/2145438008 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80f80682f0 0x7f80f806a7a0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8110007fd0 tx=0x7f8110007ec0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.420+0000 7f811b296700 1 -- 172.21.15.28:0/2145438008 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81140cb3e0 msgr2=0x7f81140b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.420+0000 7f811b296700 1 --2- 172.21.15.28:0/2145438008 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81140cb3e0 0x7f81140b29e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f810400c820 tx=0x7f810400cb30 comp rx=0 tx=0).stop 2023-11-28T11:09:37.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.420+0000 7f811b296700 1 -- 172.21.15.28:0/2145438008 shutdown_connections 2023-11-28T11:09:37.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.420+0000 7f811b296700 1 --2- 172.21.15.28:0/2145438008 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80f80682f0 0x7f80f806a7a0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.420+0000 7f811b296700 1 --2- 172.21.15.28:0/2145438008 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f81140cf5f0 0x7f81140c78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.420+0000 7f811b296700 1 --2- 172.21.15.28:0/2145438008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81140b1630 0x7f81140abad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.420+0000 7f811b296700 1 --2- 172.21.15.28:0/2145438008 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f81140cb3e0 0x7f81140b29e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.420+0000 7f811b296700 1 -- 172.21.15.28:0/2145438008 >> 172.21.15.28:0/2145438008 conn(0x7f81140cc630 msgr2=0x7f81140b0d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:37.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.420+0000 7f811b296700 1 -- 172.21.15.28:0/2145438008 shutdown_connections 2023-11-28T11:09:37.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.420+0000 7f811b296700 1 -- 172.21.15.28:0/2145438008 wait complete. 2023-11-28T11:09:37.433 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:37.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.558+0000 7fdc2c451700 1 Processor -- start 2023-11-28T11:09:37.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.558+0000 7fdc2c451700 1 -- start start 2023-11-28T11:09:37.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.559+0000 7fdc2c451700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc2415c990 0x7fdc240cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.559+0000 7fdc2c451700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdc24161090 0x7fdc240ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.559+0000 7fdc2c451700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc240ce340 con 0x7fdc240caeb0 2023-11-28T11:09:37.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.559+0000 7fdc2c451700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc240cebe0 con 0x7fdc24161090 2023-11-28T11:09:37.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.559+0000 7fdc2c451700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc240cb3f0 con 0x7fdc2415c990 2023-11-28T11:09:37.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.559+0000 7fdc2a9ee700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdc240caeb0 0x7fdc240cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36274/0 (socket says 172.21.15.28:36274) 2023-11-28T11:09:37.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.559+0000 7fdc2a9ee700 1 -- 172.21.15.28:0/3617378273 learned_addr learned my addr 172.21.15.28:0/3617378273 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:37.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.559+0000 7fdc299ec700 1 --2- 172.21.15.28:0/3617378273 >> v2:172.21.15.59:3300/0 conn(0x7fdc24161090 0x7fdc240ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.559+0000 7fdc2a1ed700 1 --2- 172.21.15.28:0/3617378273 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc2415c990 0x7fdc240cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.559+0000 7fdc291eb700 1 -- 172.21.15.28:0/3617378273 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 81097915 0 0) 0x7fdc240ce340 con 0x7fdc240caeb0 2023-11-28T11:09:37.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.559+0000 7fdc291eb700 1 -- 172.21.15.28:0/3617378273 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc100035b0 con 0x7fdc240caeb0 2023-11-28T11:09:37.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 -- 172.21.15.28:0/3617378273 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 544464435 0 0) 0x7fdc100035b0 con 0x7fdc240caeb0 2023-11-28T11:09:37.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 -- 172.21.15.28:0/3617378273 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc2415c990 msgr2=0x7fdc240cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:37.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 --2- 172.21.15.28:0/3617378273 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc2415c990 0x7fdc240cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 -- 172.21.15.28:0/3617378273 >> v2:172.21.15.59:3300/0 conn(0x7fdc24161090 msgr2=0x7fdc240ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:37.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 --2- 172.21.15.28:0/3617378273 >> v2:172.21.15.59:3300/0 conn(0x7fdc24161090 0x7fdc240ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 -- 172.21.15.28:0/3617378273 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc240ce590 con 0x7fdc240caeb0 2023-11-28T11:09:37.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 -- 172.21.15.28:0/3617378273 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fdc1c0019f0 con 0x7fdc240caeb0 2023-11-28T11:09:37.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 -- 172.21.15.28:0/3617378273 >> v1:172.21.15.28:6789/0 conn(0x7fdc240caeb0 legacy=0x7fdc240cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 --2- 172.21.15.28:0/3617378273 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc2415c990 0x7fdc10003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 --2- 172.21.15.28:0/3617378273 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc10003f80 0x7fdc10004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 --2- 172.21.15.28:0/3617378273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc10004d90 0x7fdc10005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 -- 172.21.15.28:0/3617378273 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc240cebe0 con 0x7fdc10003f80 2023-11-28T11:09:37.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 -- 172.21.15.28:0/3617378273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc240cb3f0 con 0x7fdc10004d90 2023-11-28T11:09:37.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 -- 172.21.15.28:0/3617378273 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1000baf0 con 0x7fdc2415c990 2023-11-28T11:09:37.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc299ec700 1 --2- 172.21.15.28:0/3617378273 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc10003f80 0x7fdc10004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc2a1ed700 1 --2- 172.21.15.28:0/3617378273 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc2415c990 0x7fdc10003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc2a9ee700 1 --2- 172.21.15.28:0/3617378273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc10004d90 0x7fdc10005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc299ec700 1 -- 172.21.15.28:0/3617378273 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc2415c990 msgr2=0x7fdc10003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:37.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc299ec700 1 --2- 172.21.15.28:0/3617378273 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc2415c990 0x7fdc10003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc299ec700 1 -- 172.21.15.28:0/3617378273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc10004d90 msgr2=0x7fdc10005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:37.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc299ec700 1 --2- 172.21.15.28:0/3617378273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc10004d90 0x7fdc10005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc299ec700 1 -- 172.21.15.28:0/3617378273 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc100035b0 con 0x7fdc10003f80 2023-11-28T11:09:37.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc2a9ee700 1 --2- 172.21.15.28:0/3617378273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc10004d90 0x7fdc10005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:37.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc299ec700 1 --2- 172.21.15.28:0/3617378273 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc10003f80 0x7fdc10004850 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdc241614c0 tx=0x7fdc10005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5e4eddb39e75f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:37.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 -- 172.21.15.28:0/3617378273 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc20009e70 con 0x7fdc10003f80 2023-11-28T11:09:37.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 -- 172.21.15.28:0/3617378273 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc2000e6c0 con 0x7fdc10003f80 2023-11-28T11:09:37.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.560+0000 7fdc291eb700 1 -- 172.21.15.28:0/3617378273 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc2000ea30 con 0x7fdc10003f80 2023-11-28T11:09:37.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.561+0000 7fdc2c451700 1 -- 172.21.15.28:0/3617378273 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc10003f80 msgr2=0x7fdc10004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.561+0000 7fdc2c451700 1 --2- 172.21.15.28:0/3617378273 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc10003f80 0x7fdc10004850 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fdc241614c0 tx=0x7fdc10005880 comp rx=0 tx=0).stop 2023-11-28T11:09:37.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.561+0000 7fdc2a9ee700 1 -- 172.21.15.28:0/3617378273 reap_dead start 2023-11-28T11:09:37.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.561+0000 7fdc2c451700 1 -- 172.21.15.28:0/3617378273 shutdown_connections 2023-11-28T11:09:37.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.561+0000 7fdc2c451700 1 -- 172.21.15.28:0/3617378273 >> 172.21.15.28:0/3617378273 conn(0x7fdc240b1650 msgr2=0x7fdc240b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:37.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.561+0000 7fdc2c451700 1 -- 172.21.15.28:0/3617378273 shutdown_connections 2023-11-28T11:09:37.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.561+0000 7fdc2c451700 1 -- 172.21.15.28:0/3617378273 wait complete. 2023-11-28T11:09:37.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.562+0000 7fdc2c451700 1 Processor -- start 2023-11-28T11:09:37.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.562+0000 7fdc2c451700 1 -- start start 2023-11-28T11:09:37.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.563+0000 7fdc2c451700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc10003f80 0x7fdc2414bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.563+0000 7fdc2c451700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc2415de20 0x7fdc2415e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.563+0000 7fdc2c451700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc240c3ad0 0x7fdc240c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.563+0000 7fdc2c451700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc240c44b0 con 0x7fdc10003f80 2023-11-28T11:09:37.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.563+0000 7fdc2c451700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2414c440 con 0x7fdc2415de20 2023-11-28T11:09:37.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.563+0000 7fdc2c451700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc2415e8e0 con 0x7fdc240c3ad0 2023-11-28T11:09:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.563+0000 7fdc2a1ed700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc10003f80 0x7fdc2414bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.563+0000 7fdc2a1ed700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc10003f80 0x7fdc2414bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48224/0 (socket says 172.21.15.28:48224) 2023-11-28T11:09:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.563+0000 7fdc2a1ed700 1 -- 172.21.15.28:0/391358737 learned_addr learned my addr 172.21.15.28:0/391358737 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.563+0000 7fdc2a1ed700 1 -- 172.21.15.28:0/391358737 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc240c3ad0 msgr2=0x7fdc240c3f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.564+0000 7fdc2a1ed700 1 --2- 172.21.15.28:0/391358737 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc240c3ad0 0x7fdc240c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.564+0000 7fdc2a1ed700 1 -- 172.21.15.28:0/391358737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc2415de20 msgr2=0x7fdc2415e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.564+0000 7fdc2a1ed700 1 --2- 172.21.15.28:0/391358737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc2415de20 0x7fdc2415e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.564+0000 7fdc2a1ed700 1 -- 172.21.15.28:0/391358737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc240cca20 con 0x7fdc10003f80 2023-11-28T11:09:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.564+0000 7fdc2a1ed700 1 --2- 172.21.15.28:0/391358737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc10003f80 0x7fdc2414bf00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdc14011ad0 tx=0x7fdc14011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.564+0000 7fdc1affd700 1 -- 172.21.15.28:0/391358737 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400f980 con 0x7fdc10003f80 2023-11-28T11:09:37.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.564+0000 7fdc1affd700 1 -- 172.21.15.28:0/391358737 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc1400fae0 con 0x7fdc10003f80 2023-11-28T11:09:37.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.564+0000 7fdc2c451700 1 -- 172.21.15.28:0/391358737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc240ce870 con 0x7fdc10003f80 2023-11-28T11:09:37.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.564+0000 7fdc1affd700 1 -- 172.21.15.28:0/391358737 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400fe90 con 0x7fdc10003f80 2023-11-28T11:09:37.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.564+0000 7fdc2c451700 1 -- 172.21.15.28:0/391358737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc240ccbb0 con 0x7fdc10003f80 2023-11-28T11:09:37.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.565+0000 7fdc2c451700 1 -- 172.21.15.28:0/391358737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc240675a0 con 0x7fdc10003f80 2023-11-28T11:09:37.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.565+0000 7fdc1affd700 1 -- 172.21.15.28:0/391358737 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdc14014440 con 0x7fdc10003f80 2023-11-28T11:09:37.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.566+0000 7fdc1affd700 1 --2- 172.21.15.28:0/391358737 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdc08068320 0x7fdc0806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:37.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.566+0000 7fdc1affd700 1 -- 172.21.15.28:0/391358737 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdc14086740 con 0x7fdc10003f80 2023-11-28T11:09:37.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.566+0000 7fdc299ec700 1 --2- 172.21.15.28:0/391358737 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdc08068320 0x7fdc0806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:37.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.566+0000 7fdc299ec700 1 --2- 172.21.15.28:0/391358737 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdc08068320 0x7fdc0806a7d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdc2406eec0 tx=0x7fdc20013000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:37.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.570+0000 7fdc1affd700 1 -- 172.21.15.28:0/391358737 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdc14052cd0 con 0x7fdc10003f80 2023-11-28T11:09:37.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.777+0000 7fdc2c451700 1 -- 172.21.15.28:0/391358737 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fdc2406ca90 con 0x7fdc10003f80 2023-11-28T11:09:37.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.777+0000 7fdc1affd700 1 -- 172.21.15.28:0/391358737 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fdc140562f0 con 0x7fdc10003f80 2023-11-28T11:09:37.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.779+0000 7fdc2c451700 1 -- 172.21.15.28:0/391358737 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdc08068320 msgr2=0x7fdc0806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.779+0000 7fdc2c451700 1 --2- 172.21.15.28:0/391358737 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdc08068320 0x7fdc0806a7d0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fdc2406eec0 tx=0x7fdc20013000 comp rx=0 tx=0).stop 2023-11-28T11:09:37.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.779+0000 7fdc2c451700 1 -- 172.21.15.28:0/391358737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc10003f80 msgr2=0x7fdc2414bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:37.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.779+0000 7fdc2c451700 1 --2- 172.21.15.28:0/391358737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc10003f80 0x7fdc2414bf00 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdc14011ad0 tx=0x7fdc14011e90 comp rx=0 tx=0).stop 2023-11-28T11:09:37.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.779+0000 7fdc2c451700 1 -- 172.21.15.28:0/391358737 shutdown_connections 2023-11-28T11:09:37.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.779+0000 7fdc2c451700 1 --2- 172.21.15.28:0/391358737 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdc08068320 0x7fdc0806a7d0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.779+0000 7fdc2c451700 1 --2- 172.21.15.28:0/391358737 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc10003f80 0x7fdc2414bf00 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.779+0000 7fdc2c451700 1 --2- 172.21.15.28:0/391358737 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc240c3ad0 0x7fdc240c3f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.779+0000 7fdc2c451700 1 --2- 172.21.15.28:0/391358737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc2415de20 0x7fdc2415e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:37.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.779+0000 7fdc2c451700 1 -- 172.21.15.28:0/391358737 >> 172.21.15.28:0/391358737 conn(0x7fdc240b1650 msgr2=0x7fdc240cce80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:37.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.779+0000 7fdc2c451700 1 -- 172.21.15.28:0/391358737 shutdown_connections 2023-11-28T11:09:37.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:37.779+0000 7fdc2c451700 1 -- 172.21.15.28:0/391358737 wait complete. 2023-11-28T11:09:37.781 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2023-11-28T11:09:37.792 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi028.front.sepia.ceph.com... 2023-11-28T11:09:37.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:37.792 DEBUG:teuthology.orchestra.run.smithi028:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T11:09:37.835 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:37.835 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:09:37.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:37.835 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:09:37.893 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:09:37.893 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:37.915 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:09:37.915 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:37.976 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:38.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.100+0000 7f2f55263700 1 Processor -- start 2023-11-28T11:09:38.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.100+0000 7f2f55263700 1 -- start start 2023-11-28T11:09:38.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.101+0000 7f2f55263700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2f5015a6c0 0x7f2f500cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.101+0000 7f2f55263700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f500b2660 0x7f2f500b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.101+0000 7f2f55263700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f500cfb90 con 0x7f2f500d0020 2023-11-28T11:09:38.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.101+0000 7f2f55263700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f500b3020 con 0x7f2f5015a6c0 2023-11-28T11:09:38.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.101+0000 7f2f55263700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500cb930 con 0x7f2f500b2660 2023-11-28T11:09:38.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.101+0000 7f2f4effd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2f5015a6c0 0x7f2f500cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.101+0000 7f2f4effd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2f5015a6c0 0x7f2f500cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48232/0 (socket says 172.21.15.28:48232) 2023-11-28T11:09:38.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.101+0000 7f2f4effd700 1 -- 172.21.15.28:0/3995469361 learned_addr learned my addr 172.21.15.28:0/3995469361 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:38.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.102+0000 7f2f4f7fe700 1 --2- 172.21.15.28:0/3995469361 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f500b2660 0x7f2f500b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.102+0000 7f2f4effd700 1 -- 172.21.15.28:0/3995469361 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f500b2660 msgr2=0x7f2f500b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.102+0000 7f2f4effd700 1 --2- 172.21.15.28:0/3995469361 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f500b2660 0x7f2f500b2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.102+0000 7f2f4effd700 1 -- 172.21.15.28:0/3995469361 >> v1:172.21.15.59:6789/0 conn(0x7f2f500d0020 legacy=0x7f2f500ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.102+0000 7f2f4effd700 1 -- 172.21.15.28:0/3995469361 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f500cf260 con 0x7f2f5015a6c0 2023-11-28T11:09:38.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.102+0000 7f2f4effd700 1 --2- 172.21.15.28:0/3995469361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f5015a6c0 0x7f2f500cb230 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2f440098f0 tx=0x7f2f44009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1700c5020b34252 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.102+0000 7f2f4dffb700 1 -- 172.21.15.28:0/3995469361 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44004230 con 0x7f2f5015a6c0 2023-11-28T11:09:38.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.102+0000 7f2f4dffb700 1 -- 172.21.15.28:0/3995469361 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f44004390 con 0x7f2f5015a6c0 2023-11-28T11:09:38.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.102+0000 7f2f4dffb700 1 -- 172.21.15.28:0/3995469361 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f440036a0 con 0x7f2f5015a6c0 2023-11-28T11:09:38.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.102+0000 7f2f55263700 1 -- 172.21.15.28:0/3995469361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f5015a6c0 msgr2=0x7f2f500cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.102+0000 7f2f55263700 1 --2- 172.21.15.28:0/3995469361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f5015a6c0 0x7f2f500cb230 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f2f440098f0 tx=0x7f2f44009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:38.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.103+0000 7f2f55263700 1 -- 172.21.15.28:0/3995469361 shutdown_connections 2023-11-28T11:09:38.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.103+0000 7f2f55263700 1 --2- 172.21.15.28:0/3995469361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f5015a6c0 0x7f2f500cb230 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.103+0000 7f2f55263700 1 --2- 172.21.15.28:0/3995469361 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f500b2660 0x7f2f500b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.103+0000 7f2f55263700 1 -- 172.21.15.28:0/3995469361 >> 172.21.15.28:0/3995469361 conn(0x7f2f500cd660 msgr2=0x7f2f500cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.103+0000 7f2f55263700 1 -- 172.21.15.28:0/3995469361 shutdown_connections 2023-11-28T11:09:38.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.103+0000 7f2f55263700 1 -- 172.21.15.28:0/3995469361 wait complete. 2023-11-28T11:09:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.104+0000 7f2f55263700 1 Processor -- start 2023-11-28T11:09:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.104+0000 7f2f55263700 1 -- start start 2023-11-28T11:09:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f55263700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f500b2660 0x7f2f501f2e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f55263700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f500d0020 0x7f2f501584e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f55263700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f5015a6c0 0x7f2f50158a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f55263700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500c66e0 con 0x7f2f5015a6c0 2023-11-28T11:09:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f55263700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500c69c0 con 0x7f2f500b2660 2023-11-28T11:09:38.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f55263700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f500c6ca0 con 0x7f2f500d0020 2023-11-28T11:09:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f4f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f5015a6c0 0x7f2f50158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f4f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f5015a6c0 0x7f2f50158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48242/0 (socket says 172.21.15.28:48242) 2023-11-28T11:09:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f4f7fe700 1 -- 172.21.15.28:0/2504381965 learned_addr learned my addr 172.21.15.28:0/2504381965 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f4effd700 1 --2- 172.21.15.28:0/2504381965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f500b2660 0x7f2f501f2e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f4e7fc700 1 --2- 172.21.15.28:0/2504381965 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f500d0020 0x7f2f501584e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f4f7fe700 1 -- 172.21.15.28:0/2504381965 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f500d0020 msgr2=0x7f2f501584e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f4f7fe700 1 --2- 172.21.15.28:0/2504381965 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f500d0020 0x7f2f501584e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f4f7fe700 1 -- 172.21.15.28:0/2504381965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f500b2660 msgr2=0x7f2f501f2e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f4f7fe700 1 --2- 172.21.15.28:0/2504381965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f500b2660 0x7f2f501f2e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f4f7fe700 1 -- 172.21.15.28:0/2504381965 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f44009550 con 0x7f2f5015a6c0 2023-11-28T11:09:38.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f4f7fe700 1 --2- 172.21.15.28:0/2504381965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f5015a6c0 0x7f2f50158a20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800cd70 tx=0x7f2f4800a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.105+0000 7f2f377fe700 1 -- 172.21.15.28:0/2504381965 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48003a20 con 0x7f2f5015a6c0 2023-11-28T11:09:38.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.106+0000 7f2f377fe700 1 -- 172.21.15.28:0/2504381965 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f48003b80 con 0x7f2f5015a6c0 2023-11-28T11:09:38.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.106+0000 7f2f55263700 1 -- 172.21.15.28:0/2504381965 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f500cf820 con 0x7f2f5015a6c0 2023-11-28T11:09:38.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.106+0000 7f2f377fe700 1 -- 172.21.15.28:0/2504381965 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48004020 con 0x7f2f5015a6c0 2023-11-28T11:09:38.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.106+0000 7f2f55263700 1 -- 172.21.15.28:0/2504381965 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f500cf540 con 0x7f2f5015a6c0 2023-11-28T11:09:38.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.107+0000 7f2f55263700 1 -- 172.21.15.28:0/2504381965 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f500d0450 con 0x7f2f5015a6c0 2023-11-28T11:09:38.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.107+0000 7f2f377fe700 1 -- 172.21.15.28:0/2504381965 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2f4801c020 con 0x7f2f5015a6c0 2023-11-28T11:09:38.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.107+0000 7f2f377fe700 1 --2- 172.21.15.28:0/2504381965 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2f3c0682f0 0x7f2f3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.107+0000 7f2f377fe700 1 -- 172.21.15.28:0/2504381965 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2f48012070 con 0x7f2f5015a6c0 2023-11-28T11:09:38.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.107+0000 7f2f4effd700 1 --2- 172.21.15.28:0/2504381965 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2f3c0682f0 0x7f2f3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.108+0000 7f2f4effd700 1 --2- 172.21.15.28:0/2504381965 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2f3c0682f0 0x7f2f3c06a7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009130 tx=0x7f2f44008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.111+0000 7f2f377fe700 1 -- 172.21.15.28:0/2504381965 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2f48052b70 con 0x7f2f5015a6c0 2023-11-28T11:09:38.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.313+0000 7f2f55263700 1 -- 172.21.15.28:0/2504381965 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f500675a0 con 0x7f2f5015a6c0 2023-11-28T11:09:38.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.314+0000 7f2f377fe700 1 -- 172.21.15.28:0/2504381965 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19737 (secure 0 0 0) 0x7f2f48056190 con 0x7f2f5015a6c0 2023-11-28T11:09:38.315 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:38.315 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:09:34.614188+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:38.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.316+0000 7f2f55263700 1 -- 172.21.15.28:0/2504381965 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2f3c0682f0 msgr2=0x7f2f3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.316+0000 7f2f55263700 1 --2- 172.21.15.28:0/2504381965 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2f3c0682f0 0x7f2f3c06a7a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2f44009130 tx=0x7f2f44008030 comp rx=0 tx=0).stop 2023-11-28T11:09:38.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.316+0000 7f2f55263700 1 -- 172.21.15.28:0/2504381965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f5015a6c0 msgr2=0x7f2f50158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.316+0000 7f2f55263700 1 --2- 172.21.15.28:0/2504381965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f5015a6c0 0x7f2f50158a20 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f2f4800cd70 tx=0x7f2f4800a570 comp rx=0 tx=0).stop 2023-11-28T11:09:38.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.316+0000 7f2f55263700 1 -- 172.21.15.28:0/2504381965 shutdown_connections 2023-11-28T11:09:38.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.316+0000 7f2f55263700 1 --2- 172.21.15.28:0/2504381965 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2f3c0682f0 0x7f2f3c06a7a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.316+0000 7f2f55263700 1 --2- 172.21.15.28:0/2504381965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f5015a6c0 0x7f2f50158a20 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.316+0000 7f2f55263700 1 --2- 172.21.15.28:0/2504381965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f500b2660 0x7f2f501f2e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.316+0000 7f2f55263700 1 --2- 172.21.15.28:0/2504381965 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f500d0020 0x7f2f501584e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.316+0000 7f2f55263700 1 -- 172.21.15.28:0/2504381965 >> 172.21.15.28:0/2504381965 conn(0x7f2f500cd660 msgr2=0x7f2f500b1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.317+0000 7f2f55263700 1 -- 172.21.15.28:0/2504381965 shutdown_connections 2023-11-28T11:09:38.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.317+0000 7f2f55263700 1 -- 172.21.15.28:0/2504381965 wait complete. 2023-11-28T11:09:38.329 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:38.330 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:38.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.457+0000 7f1433db2700 1 Processor -- start 2023-11-28T11:09:38.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.457+0000 7f1433db2700 1 -- start start 2023-11-28T11:09:38.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.458+0000 7f1433db2700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f142c0b2d90 0x7f142c0ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.458+0000 7f1433db2700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f142c0cd2a0 con 0x7f142c0b62d0 2023-11-28T11:09:38.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.458+0000 7f1433db2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f142c0d0400 con 0x7f142c0cb630 2023-11-28T11:09:38.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.458+0000 7f1433db2700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f142c0d06e0 con 0x7f142c0b2d90 2023-11-28T11:09:38.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.458+0000 7f143234f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f142c0b2d90 0x7f142c0ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.458+0000 7f143234f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f142c0b2d90 0x7f142c0ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48250/0 (socket says 172.21.15.28:48250) 2023-11-28T11:09:38.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.458+0000 7f143234f700 1 -- 172.21.15.28:0/4004523431 learned_addr learned my addr 172.21.15.28:0/4004523431 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:38.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.458+0000 7f143234f700 1 -- 172.21.15.28:0/4004523431 >> v1:172.21.15.59:6789/0 conn(0x7f142c0cb630 legacy=0x7f142c0b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.459+0000 7f143234f700 1 -- 172.21.15.28:0/4004523431 >> v1:172.21.15.28:6789/0 conn(0x7f142c0b62d0 legacy=0x7f142c06f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.459+0000 7f143234f700 1 -- 172.21.15.28:0/4004523431 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c0cffb0 con 0x7f142c0b2d90 2023-11-28T11:09:38.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.459+0000 7f143234f700 1 --2- 172.21.15.28:0/4004523431 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f142c0b2d90 0x7f142c0ce7d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1424009e50 tx=0x7f142400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ebce3bc0b61fbc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.459+0000 7f1430b4c700 1 -- 172.21.15.28:0/4004523431 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142400e070 con 0x7f142c0b2d90 2023-11-28T11:09:38.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.459+0000 7f1430b4c700 1 -- 172.21.15.28:0/4004523431 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1424003a20 con 0x7f142c0b2d90 2023-11-28T11:09:38.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.459+0000 7f1430b4c700 1 -- 172.21.15.28:0/4004523431 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1424003dd0 con 0x7f142c0b2d90 2023-11-28T11:09:38.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.459+0000 7f1433db2700 1 -- 172.21.15.28:0/4004523431 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f142c0b2d90 msgr2=0x7f142c0ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.459+0000 7f1433db2700 1 --2- 172.21.15.28:0/4004523431 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f142c0b2d90 0x7f142c0ce7d0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f1424009e50 tx=0x7f142400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:38.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.459+0000 7f1433db2700 1 -- 172.21.15.28:0/4004523431 shutdown_connections 2023-11-28T11:09:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.459+0000 7f1433db2700 1 --2- 172.21.15.28:0/4004523431 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f142c0b2d90 0x7f142c0ce7d0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.459+0000 7f1433db2700 1 -- 172.21.15.28:0/4004523431 >> 172.21.15.28:0/4004523431 conn(0x7f142c0cd680 msgr2=0x7f142c0cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.460+0000 7f1433db2700 1 -- 172.21.15.28:0/4004523431 shutdown_connections 2023-11-28T11:09:38.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.460+0000 7f1433db2700 1 -- 172.21.15.28:0/4004523431 wait complete. 2023-11-28T11:09:38.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.461+0000 7f1433db2700 1 Processor -- start 2023-11-28T11:09:38.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.461+0000 7f1433db2700 1 -- start start 2023-11-28T11:09:38.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.461+0000 7f1433db2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f142c0b2d90 0x7f142c14cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.461+0000 7f1433db2700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f142c0b62d0 0x7f142c159290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.461+0000 7f1433db2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c0cb630 0x7f142c1597d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.461+0000 7f1433db2700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c1e40a0 con 0x7f142c0b62d0 2023-11-28T11:09:38.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.461+0000 7f1433db2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f142c1e4380 con 0x7f142c0cb630 2023-11-28T11:09:38.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.461+0000 7f1433db2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f142c1e4660 con 0x7f142c0b2d90 2023-11-28T11:09:38.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f143134d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f142c0b62d0 0x7f142c159290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f143134d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f142c0b62d0 0x7f142c159290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48254/0 (socket says 172.21.15.28:48254) 2023-11-28T11:09:38.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f143134d700 1 -- 172.21.15.28:0/2258076801 learned_addr learned my addr 172.21.15.28:0/2258076801 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:38.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f143234f700 1 --2- 172.21.15.28:0/2258076801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c0cb630 0x7f142c1597d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f1431b4e700 1 --2- 172.21.15.28:0/2258076801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f142c0b2d90 0x7f142c14cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f143134d700 1 -- 172.21.15.28:0/2258076801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f142c0b2d90 msgr2=0x7f142c14cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f143134d700 1 --2- 172.21.15.28:0/2258076801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f142c0b2d90 0x7f142c14cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f143134d700 1 -- 172.21.15.28:0/2258076801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c0cb630 msgr2=0x7f142c1597d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f143134d700 1 --2- 172.21.15.28:0/2258076801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c0cb630 0x7f142c1597d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f143134d700 1 -- 172.21.15.28:0/2258076801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1424009b30 con 0x7f142c0b62d0 2023-11-28T11:09:38.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f143234f700 1 --2- 172.21.15.28:0/2258076801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c0cb630 0x7f142c1597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:38.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f143134d700 1 --2- 172.21.15.28:0/2258076801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f142c0b62d0 0x7f142c159290 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f142800b4a0 tx=0x7f142800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f14227fc700 1 -- 172.21.15.28:0/2258076801 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428011830 con 0x7f142c0b62d0 2023-11-28T11:09:38.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f14227fc700 1 -- 172.21.15.28:0/2258076801 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1428011990 con 0x7f142c0b62d0 2023-11-28T11:09:38.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f1433db2700 1 -- 172.21.15.28:0/2258076801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f142c0cce50 con 0x7f142c0b62d0 2023-11-28T11:09:38.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f14227fc700 1 -- 172.21.15.28:0/2258076801 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1428011d40 con 0x7f142c0b62d0 2023-11-28T11:09:38.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.462+0000 7f1433db2700 1 -- 172.21.15.28:0/2258076801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f142c0b66a0 con 0x7f142c0b62d0 2023-11-28T11:09:38.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.463+0000 7f1433db2700 1 -- 172.21.15.28:0/2258076801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f142c0ccfe0 con 0x7f142c0b62d0 2023-11-28T11:09:38.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.464+0000 7f14227fc700 1 -- 172.21.15.28:0/2258076801 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f142800f050 con 0x7f142c0b62d0 2023-11-28T11:09:38.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.464+0000 7f14227fc700 1 --2- 172.21.15.28:0/2258076801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f14100682f0 0x7f141006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.464+0000 7f14227fc700 1 -- 172.21.15.28:0/2258076801 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1428085ba0 con 0x7f142c0b62d0 2023-11-28T11:09:38.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.464+0000 7f1431b4e700 1 --2- 172.21.15.28:0/2258076801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f14100682f0 0x7f141006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.464+0000 7f1431b4e700 1 --2- 172.21.15.28:0/2258076801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f14100682f0 0x7f141006a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f141c0016c0 tx=0x7f141c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.468+0000 7f14227fc700 1 -- 172.21.15.28:0/2258076801 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1428052070 con 0x7f142c0b62d0 2023-11-28T11:09:38.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.659+0000 7f1433db2700 1 -- 172.21.15.28:0/2258076801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f142c0739f0 con 0x7f142c0b62d0 2023-11-28T11:09:38.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.659+0000 7f14227fc700 1 -- 172.21.15.28:0/2258076801 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1428015020 con 0x7f142c0b62d0 2023-11-28T11:09:38.660 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:09:38.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.661+0000 7f1433db2700 1 -- 172.21.15.28:0/2258076801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f14100682f0 msgr2=0x7f141006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.661+0000 7f1433db2700 1 --2- 172.21.15.28:0/2258076801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f14100682f0 0x7f141006a7a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f141c0016c0 tx=0x7f141c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:38.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.661+0000 7f1433db2700 1 -- 172.21.15.28:0/2258076801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f142c0b62d0 msgr2=0x7f142c159290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.661+0000 7f1433db2700 1 --2- 172.21.15.28:0/2258076801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f142c0b62d0 0x7f142c159290 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f142800b4a0 tx=0x7f142800b860 comp rx=0 tx=0).stop 2023-11-28T11:09:38.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.661+0000 7f1433db2700 1 -- 172.21.15.28:0/2258076801 shutdown_connections 2023-11-28T11:09:38.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.661+0000 7f1433db2700 1 --2- 172.21.15.28:0/2258076801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f14100682f0 0x7f141006a7a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.661+0000 7f1433db2700 1 --2- 172.21.15.28:0/2258076801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f142c0cb630 0x7f142c1597d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.661+0000 7f1433db2700 1 --2- 172.21.15.28:0/2258076801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f142c0b2d90 0x7f142c14cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.661+0000 7f1433db2700 1 --2- 172.21.15.28:0/2258076801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f142c0b62d0 0x7f142c159290 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.661+0000 7f1433db2700 1 -- 172.21.15.28:0/2258076801 >> 172.21.15.28:0/2258076801 conn(0x7f142c0cd680 msgr2=0x7f142c0b1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.661+0000 7f1433db2700 1 -- 172.21.15.28:0/2258076801 shutdown_connections 2023-11-28T11:09:38.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.661+0000 7f1433db2700 1 -- 172.21.15.28:0/2258076801 wait complete. 2023-11-28T11:09:38.674 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:38.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.802+0000 7f523d759700 1 Processor -- start 2023-11-28T11:09:38.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.802+0000 7f523d759700 1 -- start start 2023-11-28T11:09:38.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.803+0000 7f523d759700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5238159500 0x7f52380cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.803+0000 7f523d759700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f523815dbf0 0x7f52380ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.803+0000 7f523d759700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f52380b1630 0x7f52380b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.803+0000 7f523d759700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52380b2010 con 0x7f52380b1630 2023-11-28T11:09:38.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.803+0000 7f523d759700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52380cede0 con 0x7f5238159500 2023-11-28T11:09:38.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.803+0000 7f523d759700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f523815e040 con 0x7f523815dbf0 2023-11-28T11:09:38.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.803+0000 7f52377fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f52380b1630 0x7f52380b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.803+0000 7f52377fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f52380b1630 0x7f52380b1ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48264/0 (socket says 172.21.15.28:48264) 2023-11-28T11:09:38.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.803+0000 7f52377fe700 1 -- 172.21.15.28:0/1330718373 learned_addr learned my addr 172.21.15.28:0/1330718373 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:38.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.803+0000 7f5236ffd700 1 --2- 172.21.15.28:0/1330718373 >> v2:172.21.15.59:3300/0 conn(0x7f5238159500 0x7f52380cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.803+0000 7f52367fc700 1 --2- 172.21.15.28:0/1330718373 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f523815dbf0 0x7f52380ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f52377fe700 1 -- 172.21.15.28:0/1330718373 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f523815dbf0 msgr2=0x7f52380ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f52377fe700 1 --2- 172.21.15.28:0/1330718373 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f523815dbf0 0x7f52380ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f52377fe700 1 -- 172.21.15.28:0/1330718373 >> v2:172.21.15.59:3300/0 conn(0x7f5238159500 msgr2=0x7f52380cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:38.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f52377fe700 1 --2- 172.21.15.28:0/1330718373 >> v2:172.21.15.59:3300/0 conn(0x7f5238159500 0x7f52380cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f52377fe700 1 -- 172.21.15.28:0/1330718373 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52380ceff0 con 0x7f52380b1630 2023-11-28T11:09:38.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f52377fe700 1 --2- 172.21.15.28:0/1330718373 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52380b1630 0x7f52380b1ad0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f523000d150 tx=0x7f523000d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aef071f49bd3cecf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f5235ffb700 1 -- 172.21.15.28:0/1330718373 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5230010070 con 0x7f52380b1630 2023-11-28T11:09:38.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f5235ffb700 1 -- 172.21.15.28:0/1330718373 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f523000dcd0 con 0x7f52380b1630 2023-11-28T11:09:38.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f5235ffb700 1 -- 172.21.15.28:0/1330718373 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f523000b310 con 0x7f52380b1630 2023-11-28T11:09:38.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f523d759700 1 -- 172.21.15.28:0/1330718373 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52380b1630 msgr2=0x7f52380b1ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f523d759700 1 --2- 172.21.15.28:0/1330718373 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52380b1630 0x7f52380b1ad0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f523000d150 tx=0x7f523000d460 comp rx=0 tx=0).stop 2023-11-28T11:09:38.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f523d759700 1 -- 172.21.15.28:0/1330718373 shutdown_connections 2023-11-28T11:09:38.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f523d759700 1 --2- 172.21.15.28:0/1330718373 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f52380b1630 0x7f52380b1ad0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f523d759700 1 --2- 172.21.15.28:0/1330718373 >> v2:172.21.15.59:3300/0 conn(0x7f5238159500 0x7f52380cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f523d759700 1 --2- 172.21.15.28:0/1330718373 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f523815dbf0 0x7f52380ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.804+0000 7f523d759700 1 -- 172.21.15.28:0/1330718373 >> 172.21.15.28:0/1330718373 conn(0x7f52380cc630 msgr2=0x7f52380cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:38.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.805+0000 7f523d759700 1 -- 172.21.15.28:0/1330718373 shutdown_connections 2023-11-28T11:09:38.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.805+0000 7f523d759700 1 -- 172.21.15.28:0/1330718373 wait complete. 2023-11-28T11:09:38.810 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.806+0000 7f523d759700 1 Processor -- start 2023-11-28T11:09:38.810 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.806+0000 7f523d759700 1 -- start start 2023-11-28T11:09:38.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f523d759700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52380b1630 0x7f52381513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f523d759700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5238159500 0x7f52380ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f523d759700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f523815dbf0 0x7f52380abe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f523d759700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f523815a960 con 0x7f523815dbf0 2023-11-28T11:09:38.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f523d759700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52380ac430 con 0x7f5238159500 2023-11-28T11:09:38.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f523d759700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52380ac710 con 0x7f52380b1630 2023-11-28T11:09:38.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f52377fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f523815dbf0 0x7f52380abe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f52377fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f523815dbf0 0x7f52380abe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48270/0 (socket says 172.21.15.28:48270) 2023-11-28T11:09:38.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f52377fe700 1 -- 172.21.15.28:0/3634139663 learned_addr learned my addr 172.21.15.28:0/3634139663 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:38.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f52367fc700 1 --2- 172.21.15.28:0/3634139663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5238159500 0x7f52380ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f5236ffd700 1 --2- 172.21.15.28:0/3634139663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52380b1630 0x7f52381513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f52377fe700 1 -- 172.21.15.28:0/3634139663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52380b1630 msgr2=0x7f52381513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f52377fe700 1 --2- 172.21.15.28:0/3634139663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52380b1630 0x7f52381513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f52377fe700 1 -- 172.21.15.28:0/3634139663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5238159500 msgr2=0x7f52380ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:38.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f52377fe700 1 --2- 172.21.15.28:0/3634139663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5238159500 0x7f52380ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:38.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f52377fe700 1 -- 172.21.15.28:0/3634139663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523000bd40 con 0x7f523815dbf0 2023-11-28T11:09:38.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f52377fe700 1 --2- 172.21.15.28:0/3634139663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f523815dbf0 0x7f52380abe00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f523000d7b0 tx=0x7f5230004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f522f7fe700 1 -- 172.21.15.28:0/3634139663 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5230009a10 con 0x7f523815dbf0 2023-11-28T11:09:38.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f522f7fe700 1 -- 172.21.15.28:0/3634139663 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5230009b70 con 0x7f523815dbf0 2023-11-28T11:09:38.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f522f7fe700 1 -- 172.21.15.28:0/3634139663 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5230003e80 con 0x7f523815dbf0 2023-11-28T11:09:38.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.807+0000 7f523d759700 1 -- 172.21.15.28:0/3634139663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f523815d110 con 0x7f523815dbf0 2023-11-28T11:09:38.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.808+0000 7f523d759700 1 -- 172.21.15.28:0/3634139663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52380cf2d0 con 0x7f523815dbf0 2023-11-28T11:09:38.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.808+0000 7f523d759700 1 -- 172.21.15.28:0/3634139663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f523815d3f0 con 0x7f523815dbf0 2023-11-28T11:09:38.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.809+0000 7f522f7fe700 1 -- 172.21.15.28:0/3634139663 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5230004040 con 0x7f523815dbf0 2023-11-28T11:09:38.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.810+0000 7f522f7fe700 1 --2- 172.21.15.28:0/3634139663 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f52180682f0 0x7f521806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:38.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.810+0000 7f522f7fe700 1 -- 172.21.15.28:0/3634139663 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f523000de40 con 0x7f523815dbf0 2023-11-28T11:09:38.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.810+0000 7f5236ffd700 1 --2- 172.21.15.28:0/3634139663 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f52180682f0 0x7f521806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:38.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.810+0000 7f5236ffd700 1 --2- 172.21.15.28:0/3634139663 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f52180682f0 0x7f521806a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5228009b60 tx=0x7f5228009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:38.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:38.813+0000 7f522f7fe700 1 -- 172.21.15.28:0/3634139663 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5230054210 con 0x7f523815dbf0 2023-11-28T11:09:39.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.006+0000 7f523d759700 1 -- 172.21.15.28:0/3634139663 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f52380739f0 con 0x7f523815dbf0 2023-11-28T11:09:39.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.006+0000 7f522f7fe700 1 -- 172.21.15.28:0/3634139663 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5230057830 con 0x7f523815dbf0 2023-11-28T11:09:39.007 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.008+0000 7f523d759700 1 -- 172.21.15.28:0/3634139663 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f52180682f0 msgr2=0x7f521806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.008+0000 7f523d759700 1 --2- 172.21.15.28:0/3634139663 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f52180682f0 0x7f521806a7a0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5228009b60 tx=0x7f5228009210 comp rx=0 tx=0).stop 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.008+0000 7f523d759700 1 -- 172.21.15.28:0/3634139663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f523815dbf0 msgr2=0x7f52380abe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.008+0000 7f523d759700 1 --2- 172.21.15.28:0/3634139663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f523815dbf0 0x7f52380abe00 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f523000d7b0 tx=0x7f5230004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.008+0000 7f523d759700 1 -- 172.21.15.28:0/3634139663 shutdown_connections 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.008+0000 7f523d759700 1 --2- 172.21.15.28:0/3634139663 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f52180682f0 0x7f521806a7a0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.008+0000 7f523d759700 1 --2- 172.21.15.28:0/3634139663 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f523815dbf0 0x7f52380abe00 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.008+0000 7f523d759700 1 --2- 172.21.15.28:0/3634139663 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f52380b1630 0x7f52381513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.008+0000 7f523d759700 1 --2- 172.21.15.28:0/3634139663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5238159500 0x7f52380ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.008+0000 7f523d759700 1 -- 172.21.15.28:0/3634139663 >> 172.21.15.28:0/3634139663 conn(0x7f52380cc630 msgr2=0x7f52380cd730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:39.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.008+0000 7f523d759700 1 -- 172.21.15.28:0/3634139663 shutdown_connections 2023-11-28T11:09:39.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.008+0000 7f523d759700 1 -- 172.21.15.28:0/3634139663 wait complete. 2023-11-28T11:09:39.021 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:39.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.148+0000 7efd4b13d700 1 Processor -- start 2023-11-28T11:09:39.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.148+0000 7efd4b13d700 1 -- start start 2023-11-28T11:09:39.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.149+0000 7efd4b13d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efd440b1d40 0x7efd440cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.149+0000 7efd4b13d700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd440ceff0 con 0x7efd4415dbf0 2023-11-28T11:09:39.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.149+0000 7efd4b13d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd440cf2d0 con 0x7efd44159500 2023-11-28T11:09:39.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.149+0000 7efd4b13d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd4415e090 con 0x7efd440b1d40 2023-11-28T11:09:39.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.149+0000 7efd496da700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7efd4415dbf0 0x7efd4406bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36336/0 (socket says 172.21.15.28:36336) 2023-11-28T11:09:39.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.149+0000 7efd496da700 1 -- 172.21.15.28:0/795427105 learned_addr learned my addr 172.21.15.28:0/795427105 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:39.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.149+0000 7efd43fff700 1 --2- 172.21.15.28:0/795427105 >> v2:172.21.15.59:3300/0 conn(0x7efd440b1d40 0x7efd440cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.149+0000 7efd437fe700 1 -- 172.21.15.28:0/795427105 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1476704998 0 0) 0x7efd440ceff0 con 0x7efd4415dbf0 2023-11-28T11:09:39.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.150+0000 7efd437fe700 1 -- 172.21.15.28:0/795427105 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd340035b0 con 0x7efd4415dbf0 2023-11-28T11:09:39.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.150+0000 7efd43fff700 1 -- 172.21.15.28:0/795427105 >> v1:172.21.15.59:6789/0 conn(0x7efd44159500 legacy=0x7efd440b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.150+0000 7efd43fff700 1 -- 172.21.15.28:0/795427105 >> v1:172.21.15.28:6789/0 conn(0x7efd4415dbf0 legacy=0x7efd4406bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.150+0000 7efd43fff700 1 -- 172.21.15.28:0/795427105 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd4415d110 con 0x7efd440b1d40 2023-11-28T11:09:39.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.150+0000 7efd43fff700 1 --2- 172.21.15.28:0/795427105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd440b1d40 0x7efd440cd140 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efd30009840 tx=0x7efd30009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32aef2812caad393 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.150+0000 7efd437fe700 1 -- 172.21.15.28:0/795427105 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30011830 con 0x7efd440b1d40 2023-11-28T11:09:39.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.150+0000 7efd437fe700 1 -- 172.21.15.28:0/795427105 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd30011990 con 0x7efd440b1d40 2023-11-28T11:09:39.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.150+0000 7efd437fe700 1 -- 172.21.15.28:0/795427105 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd30011d00 con 0x7efd440b1d40 2023-11-28T11:09:39.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.151+0000 7efd4b13d700 1 -- 172.21.15.28:0/795427105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd440b1d40 msgr2=0x7efd440cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.151+0000 7efd4b13d700 1 --2- 172.21.15.28:0/795427105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd440b1d40 0x7efd440cd140 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7efd30009840 tx=0x7efd30009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:39.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.151+0000 7efd4b13d700 1 -- 172.21.15.28:0/795427105 shutdown_connections 2023-11-28T11:09:39.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.151+0000 7efd4b13d700 1 --2- 172.21.15.28:0/795427105 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd440b1d40 0x7efd440cd140 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.151+0000 7efd4b13d700 1 -- 172.21.15.28:0/795427105 >> 172.21.15.28:0/795427105 conn(0x7efd440cc630 msgr2=0x7efd440cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:39.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.151+0000 7efd4b13d700 1 -- 172.21.15.28:0/795427105 shutdown_connections 2023-11-28T11:09:39.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.151+0000 7efd4b13d700 1 -- 172.21.15.28:0/795427105 wait complete. 2023-11-28T11:09:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.152+0000 7efd4b13d700 1 Processor -- start 2023-11-28T11:09:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.152+0000 7efd4b13d700 1 -- start start 2023-11-28T11:09:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd4b13d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd440b1d40 0x7efd44151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd4b13d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd44159500 0x7efd440b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd4b13d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd4415dbf0 0x7efd440b0950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd4b13d700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd44151a80 con 0x7efd440b1d40 2023-11-28T11:09:39.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd4b13d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd440ab960 con 0x7efd44159500 2023-11-28T11:09:39.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd4b13d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd440abc40 con 0x7efd4415dbf0 2023-11-28T11:09:39.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd48ed9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd440b1d40 0x7efd44151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd48ed9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd440b1d40 0x7efd44151540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48280/0 (socket says 172.21.15.28:48280) 2023-11-28T11:09:39.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd48ed9700 1 -- 172.21.15.28:0/3627897812 learned_addr learned my addr 172.21.15.28:0/3627897812 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:39.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd43fff700 1 --2- 172.21.15.28:0/3627897812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd44159500 0x7efd440b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd496da700 1 --2- 172.21.15.28:0/3627897812 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd4415dbf0 0x7efd440b0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd48ed9700 1 -- 172.21.15.28:0/3627897812 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd4415dbf0 msgr2=0x7efd440b0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd48ed9700 1 --2- 172.21.15.28:0/3627897812 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd4415dbf0 0x7efd440b0950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd48ed9700 1 -- 172.21.15.28:0/3627897812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd44159500 msgr2=0x7efd440b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd48ed9700 1 --2- 172.21.15.28:0/3627897812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd44159500 0x7efd440b0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd48ed9700 1 -- 172.21.15.28:0/3627897812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd300094a0 con 0x7efd440b1d40 2023-11-28T11:09:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.153+0000 7efd48ed9700 1 --2- 172.21.15.28:0/3627897812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd440b1d40 0x7efd44151540 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efd3800c4e0 tx=0x7efd3800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.154+0000 7efd417fa700 1 -- 172.21.15.28:0/3627897812 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd380052d0 con 0x7efd440b1d40 2023-11-28T11:09:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.154+0000 7efd417fa700 1 -- 172.21.15.28:0/3627897812 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd38005430 con 0x7efd440b1d40 2023-11-28T11:09:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.154+0000 7efd4b13d700 1 -- 172.21.15.28:0/3627897812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd4415d6d0 con 0x7efd440b1d40 2023-11-28T11:09:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.154+0000 7efd417fa700 1 -- 172.21.15.28:0/3627897812 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd380038c0 con 0x7efd440b1d40 2023-11-28T11:09:39.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.154+0000 7efd4b13d700 1 -- 172.21.15.28:0/3627897812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd4415d3f0 con 0x7efd440b1d40 2023-11-28T11:09:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.155+0000 7efd4b13d700 1 -- 172.21.15.28:0/3627897812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd441598d0 con 0x7efd440b1d40 2023-11-28T11:09:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.155+0000 7efd417fa700 1 -- 172.21.15.28:0/3627897812 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efd38019030 con 0x7efd440b1d40 2023-11-28T11:09:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.156+0000 7efd417fa700 1 --2- 172.21.15.28:0/3627897812 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efd280682f0 0x7efd2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.156+0000 7efd43fff700 1 --2- 172.21.15.28:0/3627897812 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efd280682f0 0x7efd2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.156+0000 7efd417fa700 1 -- 172.21.15.28:0/3627897812 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efd38085370 con 0x7efd440b1d40 2023-11-28T11:09:39.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.156+0000 7efd43fff700 1 --2- 172.21.15.28:0/3627897812 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efd280682f0 0x7efd2806a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efd30009840 tx=0x7efd30008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.160 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.160+0000 7efd417fa700 1 -- 172.21.15.28:0/3627897812 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efd380518a0 con 0x7efd440b1d40 2023-11-28T11:09:39.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.355+0000 7efd4b13d700 1 -- 172.21.15.28:0/3627897812 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efd440739f0 con 0x7efd440b1d40 2023-11-28T11:09:39.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.356+0000 7efd417fa700 1 -- 172.21.15.28:0/3627897812 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7efd38054ec0 con 0x7efd440b1d40 2023-11-28T11:09:39.356 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:39.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.358+0000 7efd4b13d700 1 -- 172.21.15.28:0/3627897812 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efd280682f0 msgr2=0x7efd2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.358+0000 7efd4b13d700 1 --2- 172.21.15.28:0/3627897812 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efd280682f0 0x7efd2806a7a0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7efd30009840 tx=0x7efd30008030 comp rx=0 tx=0).stop 2023-11-28T11:09:39.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.358+0000 7efd4b13d700 1 -- 172.21.15.28:0/3627897812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd440b1d40 msgr2=0x7efd44151540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.358+0000 7efd4b13d700 1 --2- 172.21.15.28:0/3627897812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd440b1d40 0x7efd44151540 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7efd3800c4e0 tx=0x7efd3800c8a0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.358+0000 7efd4b13d700 1 -- 172.21.15.28:0/3627897812 shutdown_connections 2023-11-28T11:09:39.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.358+0000 7efd4b13d700 1 --2- 172.21.15.28:0/3627897812 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efd280682f0 0x7efd2806a7a0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.358+0000 7efd4b13d700 1 --2- 172.21.15.28:0/3627897812 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efd440b1d40 0x7efd44151540 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.358+0000 7efd4b13d700 1 --2- 172.21.15.28:0/3627897812 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efd4415dbf0 0x7efd440b0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.358+0000 7efd4b13d700 1 --2- 172.21.15.28:0/3627897812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd44159500 0x7efd440b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.358+0000 7efd4b13d700 1 -- 172.21.15.28:0/3627897812 >> 172.21.15.28:0/3627897812 conn(0x7efd440cc630 msgr2=0x7efd440d0300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:39.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.358+0000 7efd4b13d700 1 -- 172.21.15.28:0/3627897812 shutdown_connections 2023-11-28T11:09:39.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.358+0000 7efd4b13d700 1 -- 172.21.15.28:0/3627897812 wait complete. 2023-11-28T11:09:39.371 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:39.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.497+0000 7eff62dcf700 1 Processor -- start 2023-11-28T11:09:39.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.497+0000 7eff62dcf700 1 -- start start 2023-11-28T11:09:39.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.498+0000 7eff62dcf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7eff5c0b1d40 0x7eff5c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.498+0000 7eff62dcf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff5c15dbf0 0x7eff5c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.498+0000 7eff62dcf700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff5c0ceff0 con 0x7eff5c159500 2023-11-28T11:09:39.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.498+0000 7eff62dcf700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff5c15e090 con 0x7eff5c0b1d40 2023-11-28T11:09:39.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.498+0000 7eff62dcf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c0cede0 con 0x7eff5c15dbf0 2023-11-28T11:09:39.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.499+0000 7eff6136c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff5c15dbf0 0x7eff5c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.499+0000 7eff5bfff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7eff5c0b1d40 0x7eff5c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.499+0000 7eff6136c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff5c15dbf0 0x7eff5c0ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:54050/0 (socket says 172.21.15.28:54050) 2023-11-28T11:09:39.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.499+0000 7eff5bfff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7eff5c0b1d40 0x7eff5c0cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:35580/0 (socket says 172.21.15.28:35580) 2023-11-28T11:09:39.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.499+0000 7eff60b6b700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7eff5c159500 0x7eff5c0b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:47132/0 (socket says 172.21.15.28:47132) 2023-11-28T11:09:39.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.499+0000 7eff6136c700 1 -- 172.21.15.28:0/1658906265 learned_addr learned my addr 172.21.15.28:0/1658906265 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:39.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.499+0000 7eff6136c700 1 -- 172.21.15.28:0/1658906265 >> v2:172.21.15.59:3300/0 conn(0x7eff5c0b1d40 msgr2=0x7eff5c0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:39.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.499+0000 7eff6136c700 1 --2- 172.21.15.28:0/1658906265 >> v2:172.21.15.59:3300/0 conn(0x7eff5c0b1d40 0x7eff5c0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.499+0000 7eff6136c700 1 -- 172.21.15.28:0/1658906265 >> v1:172.21.15.59:6789/0 conn(0x7eff5c159500 legacy=0x7eff5c0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.499+0000 7eff6136c700 1 -- 172.21.15.28:0/1658906265 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff5c0cf2d0 con 0x7eff5c15dbf0 2023-11-28T11:09:39.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.499+0000 7eff5bfff700 1 --2- 172.21.15.28:0/1658906265 >> v2:172.21.15.59:3300/0 conn(0x7eff5c0b1d40 0x7eff5c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:39.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.499+0000 7eff6136c700 1 --2- 172.21.15.28:0/1658906265 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff5c15dbf0 0x7eff5c0ce8a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7eff54009e50 tx=0x7eff5400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=be13cb2792b101ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.499+0000 7eff5b7fe700 1 -- 172.21.15.28:0/1658906265 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5400e070 con 0x7eff5c15dbf0 2023-11-28T11:09:39.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.499+0000 7eff5b7fe700 1 -- 172.21.15.28:0/1658906265 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5400b910 con 0x7eff5c15dbf0 2023-11-28T11:09:39.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.499+0000 7eff5b7fe700 1 -- 172.21.15.28:0/1658906265 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5400bcc0 con 0x7eff5c15dbf0 2023-11-28T11:09:39.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.500+0000 7eff62dcf700 1 -- 172.21.15.28:0/1658906265 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff5c15dbf0 msgr2=0x7eff5c0ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.500+0000 7eff62dcf700 1 --2- 172.21.15.28:0/1658906265 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff5c15dbf0 0x7eff5c0ce8a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7eff54009e50 tx=0x7eff5400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:39.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.500+0000 7eff62dcf700 1 -- 172.21.15.28:0/1658906265 shutdown_connections 2023-11-28T11:09:39.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.500+0000 7eff62dcf700 1 --2- 172.21.15.28:0/1658906265 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff5c15dbf0 0x7eff5c0ce8a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.500+0000 7eff62dcf700 1 --2- 172.21.15.28:0/1658906265 >> v2:172.21.15.59:3300/0 conn(0x7eff5c0b1d40 0x7eff5c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.500+0000 7eff62dcf700 1 -- 172.21.15.28:0/1658906265 >> 172.21.15.28:0/1658906265 conn(0x7eff5c0cc630 msgr2=0x7eff5c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:39.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.500+0000 7eff62dcf700 1 -- 172.21.15.28:0/1658906265 shutdown_connections 2023-11-28T11:09:39.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.500+0000 7eff62dcf700 1 -- 172.21.15.28:0/1658906265 wait complete. 2023-11-28T11:09:39.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.501+0000 7eff62dcf700 1 Processor -- start 2023-11-28T11:09:39.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.501+0000 7eff62dcf700 1 -- start start 2023-11-28T11:09:39.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.502+0000 7eff62dcf700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff5c0b1d40 0x7eff5c157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.502+0000 7eff62dcf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff5c159500 0x7eff5c158510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.502+0000 7eff62dcf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff5c15dbf0 0x7eff5c1547b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.502+0000 7eff62dcf700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c156530 con 0x7eff5c0b1d40 2023-11-28T11:09:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.502+0000 7eff62dcf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c1558f0 con 0x7eff5c159500 2023-11-28T11:09:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.502+0000 7eff62dcf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff5c155b90 con 0x7eff5c15dbf0 2023-11-28T11:09:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.502+0000 7eff60b6b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff5c0b1d40 0x7eff5c157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.502+0000 7eff60b6b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff5c0b1d40 0x7eff5c157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48294/0 (socket says 172.21.15.28:48294) 2023-11-28T11:09:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.502+0000 7eff60b6b700 1 -- 172.21.15.28:0/597450712 learned_addr learned my addr 172.21.15.28:0/597450712 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.502+0000 7eff5bfff700 1 --2- 172.21.15.28:0/597450712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff5c159500 0x7eff5c158510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.502+0000 7eff6136c700 1 --2- 172.21.15.28:0/597450712 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff5c15dbf0 0x7eff5c1547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.503+0000 7eff60b6b700 1 -- 172.21.15.28:0/597450712 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff5c15dbf0 msgr2=0x7eff5c1547b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.503+0000 7eff60b6b700 1 --2- 172.21.15.28:0/597450712 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff5c15dbf0 0x7eff5c1547b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.503+0000 7eff60b6b700 1 -- 172.21.15.28:0/597450712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff5c159500 msgr2=0x7eff5c158510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.503+0000 7eff60b6b700 1 --2- 172.21.15.28:0/597450712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff5c159500 0x7eff5c158510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.503+0000 7eff60b6b700 1 -- 172.21.15.28:0/597450712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff480094a0 con 0x7eff5c0b1d40 2023-11-28T11:09:39.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.503+0000 7eff60b6b700 1 --2- 172.21.15.28:0/597450712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff5c0b1d40 0x7eff5c157fd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7eff5000c520 tx=0x7eff5000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.503+0000 7eff597fa700 1 -- 172.21.15.28:0/597450712 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff50005cb0 con 0x7eff5c0b1d40 2023-11-28T11:09:39.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.503+0000 7eff597fa700 1 -- 172.21.15.28:0/597450712 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff50005e10 con 0x7eff5c0b1d40 2023-11-28T11:09:39.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.503+0000 7eff62dcf700 1 -- 172.21.15.28:0/597450712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff54009b30 con 0x7eff5c0b1d40 2023-11-28T11:09:39.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.503+0000 7eff597fa700 1 -- 172.21.15.28:0/597450712 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff500072d0 con 0x7eff5c0b1d40 2023-11-28T11:09:39.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.503+0000 7eff62dcf700 1 -- 172.21.15.28:0/597450712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff5c15d3f0 con 0x7eff5c0b1d40 2023-11-28T11:09:39.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.504+0000 7eff62dcf700 1 -- 172.21.15.28:0/597450712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff5c15d110 con 0x7eff5c0b1d40 2023-11-28T11:09:39.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.504+0000 7eff597fa700 1 -- 172.21.15.28:0/597450712 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7eff50019030 con 0x7eff5c0b1d40 2023-11-28T11:09:39.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.505+0000 7eff597fa700 1 --2- 172.21.15.28:0/597450712 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7eff400682f0 0x7eff4006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.505+0000 7eff597fa700 1 -- 172.21.15.28:0/597450712 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7eff500852c0 con 0x7eff5c0b1d40 2023-11-28T11:09:39.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.505+0000 7eff5bfff700 1 --2- 172.21.15.28:0/597450712 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7eff400682f0 0x7eff4006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.505+0000 7eff5bfff700 1 --2- 172.21.15.28:0/597450712 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7eff400682f0 0x7eff4006a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7eff5c1f5e00 tx=0x7eff48008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.509+0000 7eff597fa700 1 -- 172.21.15.28:0/597450712 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7eff5004b5e0 con 0x7eff5c0b1d40 2023-11-28T11:09:39.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.704+0000 7eff62dcf700 1 -- 172.21.15.28:0/597450712 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7eff5c15d6d0 con 0x7eff5c0b1d40 2023-11-28T11:09:39.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.704+0000 7eff597fa700 1 -- 172.21.15.28:0/597450712 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7eff500170a0 con 0x7eff5c0b1d40 2023-11-28T11:09:39.705 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:09:39.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.706+0000 7eff62dcf700 1 -- 172.21.15.28:0/597450712 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7eff400682f0 msgr2=0x7eff4006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.706+0000 7eff62dcf700 1 --2- 172.21.15.28:0/597450712 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7eff400682f0 0x7eff4006a7a0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7eff5c1f5e00 tx=0x7eff48008040 comp rx=0 tx=0).stop 2023-11-28T11:09:39.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.706+0000 7eff62dcf700 1 -- 172.21.15.28:0/597450712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff5c0b1d40 msgr2=0x7eff5c157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.706+0000 7eff62dcf700 1 --2- 172.21.15.28:0/597450712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff5c0b1d40 0x7eff5c157fd0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7eff5000c520 tx=0x7eff5000c8e0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.706+0000 7eff62dcf700 1 -- 172.21.15.28:0/597450712 shutdown_connections 2023-11-28T11:09:39.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.706+0000 7eff62dcf700 1 --2- 172.21.15.28:0/597450712 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7eff400682f0 0x7eff4006a7a0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.706+0000 7eff62dcf700 1 --2- 172.21.15.28:0/597450712 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff5c0b1d40 0x7eff5c157fd0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.706+0000 7eff62dcf700 1 --2- 172.21.15.28:0/597450712 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff5c15dbf0 0x7eff5c1547b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.706+0000 7eff62dcf700 1 --2- 172.21.15.28:0/597450712 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff5c159500 0x7eff5c158510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.706+0000 7eff62dcf700 1 -- 172.21.15.28:0/597450712 >> 172.21.15.28:0/597450712 conn(0x7eff5c0cc630 msgr2=0x7eff5c0af3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:39.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.706+0000 7eff62dcf700 1 -- 172.21.15.28:0/597450712 shutdown_connections 2023-11-28T11:09:39.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.707+0000 7eff62dcf700 1 -- 172.21.15.28:0/597450712 wait complete. 2023-11-28T11:09:39.719 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:39.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.846+0000 7f366ab4c700 1 Processor -- start 2023-11-28T11:09:39.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.846+0000 7f366ab4c700 1 -- start start 2023-11-28T11:09:39.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.847+0000 7f366ab4c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f36640b1630 0x7f36640cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.847+0000 7f366ab4c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f36640b1a00 0x7f36640caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.847+0000 7f366ab4c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36640cb3e0 0x7f3664160350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.847+0000 7f366ab4c700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36640ce550 con 0x7f36640b1630 2023-11-28T11:09:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.847+0000 7f366ab4c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36640ce830 con 0x7f36640b1a00 2023-11-28T11:09:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.847+0000 7f366ab4c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36640ceb10 con 0x7f36640cb3e0 2023-11-28T11:09:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.847+0000 7f36688e8700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f36640b1630 0x7f36640cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.847+0000 7f36688e8700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f36640b1630 0x7f36640cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48302/0 (socket says 172.21.15.28:48302) 2023-11-28T11:09:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.847+0000 7f36688e8700 1 -- 172.21.15.28:0/3209759310 learned_addr learned my addr 172.21.15.28:0/3209759310 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.848+0000 7f36688e8700 1 -- 172.21.15.28:0/3209759310 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36640cb3e0 msgr2=0x7f3664160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.848+0000 7f3663fff700 1 --2- 172.21.15.28:0/3209759310 >> v2:172.21.15.59:3300/0 conn(0x7f36640b1a00 0x7f36640caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.848+0000 7f36690e9700 1 --2- 172.21.15.28:0/3209759310 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36640cb3e0 0x7f3664160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.848+0000 7f36688e8700 1 --2- 172.21.15.28:0/3209759310 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36640cb3e0 0x7f3664160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.848+0000 7f36688e8700 1 -- 172.21.15.28:0/3209759310 >> v2:172.21.15.59:3300/0 conn(0x7f36640b1a00 msgr2=0x7f36640caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:39.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.848+0000 7f36688e8700 1 --2- 172.21.15.28:0/3209759310 >> v2:172.21.15.59:3300/0 conn(0x7f36640b1a00 0x7f36640caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.848+0000 7f36688e8700 1 -- 172.21.15.28:0/3209759310 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36640cedf0 con 0x7f36640b1630 2023-11-28T11:09:39.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.848+0000 7f36690e9700 1 --2- 172.21.15.28:0/3209759310 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36640cb3e0 0x7f3664160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:39.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.848+0000 7f3663fff700 1 --2- 172.21.15.28:0/3209759310 >> v2:172.21.15.59:3300/0 conn(0x7f36640b1a00 0x7f36640caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:39.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.848+0000 7f36688e8700 1 --2- 172.21.15.28:0/3209759310 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36640b1630 0x7f36640cf5f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f36580098f0 tx=0x7f3658009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2254c3474c6e6e09 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.848+0000 7f36637fe700 1 -- 172.21.15.28:0/3209759310 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800b860 con 0x7f36640b1630 2023-11-28T11:09:39.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.848+0000 7f36637fe700 1 -- 172.21.15.28:0/3209759310 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365800b9c0 con 0x7f36640b1630 2023-11-28T11:09:39.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.848+0000 7f36637fe700 1 -- 172.21.15.28:0/3209759310 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365800bd50 con 0x7f36640b1630 2023-11-28T11:09:39.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.848+0000 7f366ab4c700 1 -- 172.21.15.28:0/3209759310 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36640b1630 msgr2=0x7f36640cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.848+0000 7f366ab4c700 1 --2- 172.21.15.28:0/3209759310 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36640b1630 0x7f36640cf5f0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f36580098f0 tx=0x7f3658009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:39.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.849+0000 7f366ab4c700 1 -- 172.21.15.28:0/3209759310 shutdown_connections 2023-11-28T11:09:39.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.849+0000 7f366ab4c700 1 --2- 172.21.15.28:0/3209759310 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36640cb3e0 0x7f3664160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.849+0000 7f366ab4c700 1 --2- 172.21.15.28:0/3209759310 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36640b1630 0x7f36640cf5f0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.849+0000 7f366ab4c700 1 --2- 172.21.15.28:0/3209759310 >> v2:172.21.15.59:3300/0 conn(0x7f36640b1a00 0x7f36640caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.849+0000 7f366ab4c700 1 -- 172.21.15.28:0/3209759310 >> 172.21.15.28:0/3209759310 conn(0x7f36640cc630 msgr2=0x7f36640cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:39.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.849+0000 7f366ab4c700 1 -- 172.21.15.28:0/3209759310 shutdown_connections 2023-11-28T11:09:39.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.849+0000 7f366ab4c700 1 -- 172.21.15.28:0/3209759310 wait complete. 2023-11-28T11:09:39.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.850+0000 7f366ab4c700 1 Processor -- start 2023-11-28T11:09:39.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.850+0000 7f366ab4c700 1 -- start start 2023-11-28T11:09:39.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f366ab4c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36640b1630 0x7f3664146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f366ab4c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36640b1a00 0x7f36640b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f366ab4c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36640cb3e0 0x7f36640c6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f366ab4c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366415d4e0 con 0x7f36640b1630 2023-11-28T11:09:39.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f366ab4c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36641f29c0 con 0x7f36640b1a00 2023-11-28T11:09:39.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f366ab4c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36640b4f00 con 0x7f36640cb3e0 2023-11-28T11:09:39.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f36688e8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36640b1630 0x7f3664146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f36688e8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36640b1630 0x7f3664146e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48308/0 (socket says 172.21.15.28:48308) 2023-11-28T11:09:39.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f36688e8700 1 -- 172.21.15.28:0/2768963254 learned_addr learned my addr 172.21.15.28:0/2768963254 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:39.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f3663fff700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36640b1a00 0x7f36640b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f36690e9700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36640cb3e0 0x7f36640c6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f36688e8700 1 -- 172.21.15.28:0/2768963254 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36640cb3e0 msgr2=0x7f36640c6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f36688e8700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36640cb3e0 0x7f36640c6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f36688e8700 1 -- 172.21.15.28:0/2768963254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36640b1a00 msgr2=0x7f36640b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:39.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f36688e8700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36640b1a00 0x7f36640b4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:39.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f36688e8700 1 -- 172.21.15.28:0/2768963254 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3658009550 con 0x7f36640b1630 2023-11-28T11:09:39.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f3663fff700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36640b1a00 0x7f36640b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:39.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f36690e9700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36640cb3e0 0x7f36640c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:39.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f36688e8700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36640b1630 0x7f3664146e30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3664066a20 tx=0x7f3658004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f36617fa700 1 -- 172.21.15.28:0/2768963254 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3658004480 con 0x7f36640b1630 2023-11-28T11:09:39.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.851+0000 7f36617fa700 1 -- 172.21.15.28:0/2768963254 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365801f070 con 0x7f36640b1630 2023-11-28T11:09:39.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.852+0000 7f366ab4c700 1 -- 172.21.15.28:0/2768963254 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36640b1ea0 con 0x7f36640b1630 2023-11-28T11:09:39.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.852+0000 7f36617fa700 1 -- 172.21.15.28:0/2768963254 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36580038e0 con 0x7f36640b1630 2023-11-28T11:09:39.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.852+0000 7f366ab4c700 1 -- 172.21.15.28:0/2768963254 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36640cf0d0 con 0x7f36640b1630 2023-11-28T11:09:39.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.853+0000 7f366ab4c700 1 -- 172.21.15.28:0/2768963254 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36640ce140 con 0x7f36640b1630 2023-11-28T11:09:39.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.853+0000 7f36617fa700 1 -- 172.21.15.28:0/2768963254 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f365802a020 con 0x7f36640b1630 2023-11-28T11:09:39.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.853+0000 7f36617fa700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f36480682f0 0x7f364806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:39.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.853+0000 7f36617fa700 1 -- 172.21.15.28:0/2768963254 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f36580875a0 con 0x7f36640b1630 2023-11-28T11:09:39.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.853+0000 7f3663fff700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f36480682f0 0x7f364806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:39.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.854+0000 7f3663fff700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f36480682f0 0x7f364806a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3664157c30 tx=0x7f3650008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:39.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:39.858+0000 7f36617fa700 1 -- 172.21.15.28:0/2768963254 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3658053ad0 con 0x7f36640b1630 2023-11-28T11:09:40.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.051+0000 7f366ab4c700 1 -- 172.21.15.28:0/2768963254 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f36640739f0 con 0x7f36640b1630 2023-11-28T11:09:40.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.051+0000 7f36617fa700 1 -- 172.21.15.28:0/2768963254 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f36580570f0 con 0x7f36640b1630 2023-11-28T11:09:40.052 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:40.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.053+0000 7f366ab4c700 1 -- 172.21.15.28:0/2768963254 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f36480682f0 msgr2=0x7f364806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.053+0000 7f366ab4c700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f36480682f0 0x7f364806a7a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3664157c30 tx=0x7f3650008040 comp rx=0 tx=0).stop 2023-11-28T11:09:40.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.053+0000 7f366ab4c700 1 -- 172.21.15.28:0/2768963254 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36640b1630 msgr2=0x7f3664146e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.053+0000 7f366ab4c700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36640b1630 0x7f3664146e30 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3664066a20 tx=0x7f3658004750 comp rx=0 tx=0).stop 2023-11-28T11:09:40.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.054+0000 7f366ab4c700 1 -- 172.21.15.28:0/2768963254 shutdown_connections 2023-11-28T11:09:40.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.054+0000 7f366ab4c700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f36480682f0 0x7f364806a7a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.054+0000 7f366ab4c700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f36640b1630 0x7f3664146e30 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.054+0000 7f366ab4c700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f36640cb3e0 0x7f36640c6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.054+0000 7f366ab4c700 1 --2- 172.21.15.28:0/2768963254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36640b1a00 0x7f36640b4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.054+0000 7f366ab4c700 1 -- 172.21.15.28:0/2768963254 >> 172.21.15.28:0/2768963254 conn(0x7f36640cc630 msgr2=0x7f36640cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:40.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.054+0000 7f366ab4c700 1 -- 172.21.15.28:0/2768963254 shutdown_connections 2023-11-28T11:09:40.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.054+0000 7f366ab4c700 1 -- 172.21.15.28:0/2768963254 wait complete. 2023-11-28T11:09:40.067 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:40.193 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.192+0000 7f30db3a8700 1 Processor -- start 2023-11-28T11:09:40.193 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.192+0000 7f30db3a8700 1 -- start start 2023-11-28T11:09:40.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.193+0000 7f30db3a8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f30d406e400 0x7f30d415db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.193+0000 7f30db3a8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30d40ce890 0x7f30d40ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.193+0000 7f30db3a8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30d415e0b0 con 0x7f30d4159590 2023-11-28T11:09:40.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.193+0000 7f30db3a8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30d40cf250 con 0x7f30d406e400 2023-11-28T11:09:40.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.193+0000 7f30db3a8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d415d120 con 0x7f30d40ce890 2023-11-28T11:09:40.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.194+0000 7f30d8943700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f30d406e400 0x7f30d415db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.194+0000 7f30d9945700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30d40ce890 0x7f30d40ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.194+0000 7f30d9945700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30d40ce890 0x7f30d40ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:54078/0 (socket says 172.21.15.28:54078) 2023-11-28T11:09:40.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.194+0000 7f30d8943700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f30d406e400 0x7f30d415db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:35608/0 (socket says 172.21.15.28:35608) 2023-11-28T11:09:40.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.194+0000 7f30d9945700 1 -- 172.21.15.28:0/2286210870 learned_addr learned my addr 172.21.15.28:0/2286210870 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:40.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.194+0000 7f30d9144700 1 --1- 172.21.15.28:0/2286210870 >> v1:172.21.15.59:6789/0 conn(0x7f30d4159590 0x7f30d40ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:47160/0 (socket says 172.21.15.28:47160) 2023-11-28T11:09:40.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.194+0000 7f30d8943700 1 -- 172.21.15.28:0/2286210870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30d40ce890 msgr2=0x7f30d40ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:40.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.194+0000 7f30d8943700 1 --2- 172.21.15.28:0/2286210870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30d40ce890 0x7f30d40ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.194+0000 7f30d8943700 1 -- 172.21.15.28:0/2286210870 >> v1:172.21.15.59:6789/0 conn(0x7f30d4159590 legacy=0x7f30d40ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.194+0000 7f30d8943700 1 -- 172.21.15.28:0/2286210870 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30d415d6e0 con 0x7f30d406e400 2023-11-28T11:09:40.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.194+0000 7f30d9945700 1 --2- 172.21.15.28:0/2286210870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30d40ce890 0x7f30d40ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:40.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.194+0000 7f30d8943700 1 --2- 172.21.15.28:0/2286210870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30d406e400 0x7f30d415db70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009840 tx=0x7f30d0009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a44f0ae5f1947ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.194+0000 7f30cbfff700 1 -- 172.21.15.28:0/2286210870 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0011830 con 0x7f30d406e400 2023-11-28T11:09:40.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.194+0000 7f30cbfff700 1 -- 172.21.15.28:0/2286210870 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30d0011990 con 0x7f30d406e400 2023-11-28T11:09:40.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.194+0000 7f30cbfff700 1 -- 172.21.15.28:0/2286210870 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0011d20 con 0x7f30d406e400 2023-11-28T11:09:40.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.195+0000 7f30db3a8700 1 -- 172.21.15.28:0/2286210870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30d406e400 msgr2=0x7f30d415db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.195+0000 7f30db3a8700 1 --2- 172.21.15.28:0/2286210870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30d406e400 0x7f30d415db70 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f30d0009840 tx=0x7f30d0009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:40.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.195+0000 7f30db3a8700 1 -- 172.21.15.28:0/2286210870 shutdown_connections 2023-11-28T11:09:40.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.195+0000 7f30db3a8700 1 --2- 172.21.15.28:0/2286210870 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30d40ce890 0x7f30d40ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.195+0000 7f30db3a8700 1 --2- 172.21.15.28:0/2286210870 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30d406e400 0x7f30d415db70 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.195+0000 7f30db3a8700 1 -- 172.21.15.28:0/2286210870 >> 172.21.15.28:0/2286210870 conn(0x7f30d40b1650 msgr2=0x7f30d40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:40.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.195+0000 7f30db3a8700 1 -- 172.21.15.28:0/2286210870 shutdown_connections 2023-11-28T11:09:40.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.195+0000 7f30db3a8700 1 -- 172.21.15.28:0/2286210870 wait complete. 2023-11-28T11:09:40.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.196+0000 7f30db3a8700 1 Processor -- start 2023-11-28T11:09:40.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.196+0000 7f30db3a8700 1 -- start start 2023-11-28T11:09:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.197+0000 7f30db3a8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30d406e400 0x7f30d4156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.197+0000 7f30db3a8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30d40ce890 0x7f30d41f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.197+0000 7f30db3a8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30d4159590 0x7f30d40c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.197+0000 7f30db3a8700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4142940 con 0x7f30d406e400 2023-11-28T11:09:40.201 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.197+0000 7f30db3a8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4142c20 con 0x7f30d40ce890 2023-11-28T11:09:40.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.197+0000 7f30db3a8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30d4142f00 con 0x7f30d4159590 2023-11-28T11:09:40.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.197+0000 7f30d9144700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30d406e400 0x7f30d4156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.197+0000 7f30d9144700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30d406e400 0x7f30d4156360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48322/0 (socket says 172.21.15.28:48322) 2023-11-28T11:09:40.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.197+0000 7f30d9144700 1 -- 172.21.15.28:0/1022382619 learned_addr learned my addr 172.21.15.28:0/1022382619 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:40.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.197+0000 7f30d9945700 1 --2- 172.21.15.28:0/1022382619 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30d4159590 0x7f30d40c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.197+0000 7f30d8943700 1 --2- 172.21.15.28:0/1022382619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30d40ce890 0x7f30d41f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.198+0000 7f30d9144700 1 -- 172.21.15.28:0/1022382619 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30d4159590 msgr2=0x7f30d40c5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.198+0000 7f30d9144700 1 --2- 172.21.15.28:0/1022382619 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30d4159590 0x7f30d40c5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.198+0000 7f30d9144700 1 -- 172.21.15.28:0/1022382619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30d40ce890 msgr2=0x7f30d41f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.198+0000 7f30d9144700 1 --2- 172.21.15.28:0/1022382619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30d40ce890 0x7f30d41f2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.198+0000 7f30d9144700 1 -- 172.21.15.28:0/1022382619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30cc009b30 con 0x7f30d406e400 2023-11-28T11:09:40.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.198+0000 7f30d9144700 1 --2- 172.21.15.28:0/1022382619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30d406e400 0x7f30d4156360 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f30c4001e50 tx=0x7f30c400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.198+0000 7f30c9ffb700 1 -- 172.21.15.28:0/1022382619 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c4005cb0 con 0x7f30d406e400 2023-11-28T11:09:40.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.198+0000 7f30db3a8700 1 -- 172.21.15.28:0/1022382619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30d00094a0 con 0x7f30d406e400 2023-11-28T11:09:40.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.198+0000 7f30c9ffb700 1 -- 172.21.15.28:0/1022382619 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30c4005e10 con 0x7f30d406e400 2023-11-28T11:09:40.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.198+0000 7f30c9ffb700 1 -- 172.21.15.28:0/1022382619 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30c40072d0 con 0x7f30d406e400 2023-11-28T11:09:40.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.198+0000 7f30db3a8700 1 -- 172.21.15.28:0/1022382619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30d40af240 con 0x7f30d406e400 2023-11-28T11:09:40.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.199+0000 7f30db3a8700 1 -- 172.21.15.28:0/1022382619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30d40aef60 con 0x7f30d406e400 2023-11-28T11:09:40.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.199+0000 7f30c9ffb700 1 -- 172.21.15.28:0/1022382619 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f30c4007430 con 0x7f30d406e400 2023-11-28T11:09:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.200+0000 7f30c9ffb700 1 --2- 172.21.15.28:0/1022382619 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f30b80682f0 0x7f30b806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.200+0000 7f30c9ffb700 1 -- 172.21.15.28:0/1022382619 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f30c4087be0 con 0x7f30d406e400 2023-11-28T11:09:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.200+0000 7f30d8943700 1 --2- 172.21.15.28:0/1022382619 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f30b80682f0 0x7f30b806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.200+0000 7f30d8943700 1 --2- 172.21.15.28:0/1022382619 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f30b80682f0 0x7f30b806a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f30d0000c00 tx=0x7f30d0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.204+0000 7f30c9ffb700 1 -- 172.21.15.28:0/1022382619 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f30c4053fa0 con 0x7f30d406e400 2023-11-28T11:09:40.414 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.413+0000 7f30db3a8700 1 -- 172.21.15.28:0/1022382619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f30d415d370 con 0x7f30d406e400 2023-11-28T11:09:40.414 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.414+0000 7f30c9ffb700 1 -- 172.21.15.28:0/1022382619 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f30c40011b0 con 0x7f30d406e400 2023-11-28T11:09:40.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.416+0000 7f30db3a8700 1 -- 172.21.15.28:0/1022382619 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f30b80682f0 msgr2=0x7f30b806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.416+0000 7f30db3a8700 1 --2- 172.21.15.28:0/1022382619 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f30b80682f0 0x7f30b806a7a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f30d0000c00 tx=0x7f30d0008030 comp rx=0 tx=0).stop 2023-11-28T11:09:40.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.416+0000 7f30db3a8700 1 -- 172.21.15.28:0/1022382619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30d406e400 msgr2=0x7f30d4156360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.416+0000 7f30db3a8700 1 --2- 172.21.15.28:0/1022382619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30d406e400 0x7f30d4156360 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f30c4001e50 tx=0x7f30c400c890 comp rx=0 tx=0).stop 2023-11-28T11:09:40.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.416+0000 7f30db3a8700 1 -- 172.21.15.28:0/1022382619 shutdown_connections 2023-11-28T11:09:40.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.416+0000 7f30db3a8700 1 --2- 172.21.15.28:0/1022382619 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f30b80682f0 0x7f30b806a7a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.416+0000 7f30db3a8700 1 --2- 172.21.15.28:0/1022382619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f30d406e400 0x7f30d4156360 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.416+0000 7f30db3a8700 1 --2- 172.21.15.28:0/1022382619 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f30d4159590 0x7f30d40c5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.416+0000 7f30db3a8700 1 --2- 172.21.15.28:0/1022382619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f30d40ce890 0x7f30d41f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.416+0000 7f30db3a8700 1 -- 172.21.15.28:0/1022382619 >> 172.21.15.28:0/1022382619 conn(0x7f30d40b1650 msgr2=0x7f30d40cb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:40.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.416+0000 7f30db3a8700 1 -- 172.21.15.28:0/1022382619 shutdown_connections 2023-11-28T11:09:40.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.416+0000 7f30db3a8700 1 -- 172.21.15.28:0/1022382619 wait complete. 2023-11-28T11:09:40.418 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2023-11-28T11:09:40.428 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:09:40.429 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:40.450 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:40.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.575+0000 7f509a8e9700 1 Processor -- start 2023-11-28T11:09:40.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.575+0000 7f509a8e9700 1 -- start start 2023-11-28T11:09:40.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.576+0000 7f509a8e9700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f509415aa20 0x7f50940b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.576+0000 7f509a8e9700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50940ca990 con 0x7f50940b5500 2023-11-28T11:09:40.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.576+0000 7f509a8e9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50940caf50 con 0x7f509415a650 2023-11-28T11:09:40.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.576+0000 7f509a8e9700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50940aef10 con 0x7f509415aa20 2023-11-28T11:09:40.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.576+0000 7f50937fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f509415aa20 0x7f50940b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.576+0000 7f50937fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f509415aa20 0x7f50940b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48330/0 (socket says 172.21.15.28:48330) 2023-11-28T11:09:40.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.576+0000 7f50937fe700 1 -- 172.21.15.28:0/1062929649 learned_addr learned my addr 172.21.15.28:0/1062929649 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:40.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.576+0000 7f50937fe700 1 -- 172.21.15.28:0/1062929649 >> v1:172.21.15.59:6789/0 conn(0x7f509415a650 legacy=0x7f50940ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:40.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.576+0000 7f50937fe700 1 -- 172.21.15.28:0/1062929649 >> v1:172.21.15.28:6789/0 conn(0x7f50940b5500 legacy=0x7f50940cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.576+0000 7f50937fe700 1 -- 172.21.15.28:0/1062929649 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50940af1f0 con 0x7f509415aa20 2023-11-28T11:09:40.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.576+0000 7f50937fe700 1 --2- 172.21.15.28:0/1062929649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f509415aa20 0x7f50940b2950 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f50840098f0 tx=0x7f5084009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f575143bdd5b8a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.577+0000 7f5092ffd700 1 -- 172.21.15.28:0/1062929649 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5084004230 con 0x7f509415aa20 2023-11-28T11:09:40.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.577+0000 7f5092ffd700 1 -- 172.21.15.28:0/1062929649 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5084004390 con 0x7f509415aa20 2023-11-28T11:09:40.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.577+0000 7f5092ffd700 1 -- 172.21.15.28:0/1062929649 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50840036a0 con 0x7f509415aa20 2023-11-28T11:09:40.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.577+0000 7f509a8e9700 1 -- 172.21.15.28:0/1062929649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f509415aa20 msgr2=0x7f50940b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.577+0000 7f509a8e9700 1 --2- 172.21.15.28:0/1062929649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f509415aa20 0x7f50940b2950 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f50840098f0 tx=0x7f5084009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:40.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.577+0000 7f509a8e9700 1 -- 172.21.15.28:0/1062929649 shutdown_connections 2023-11-28T11:09:40.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.577+0000 7f509a8e9700 1 --2- 172.21.15.28:0/1062929649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f509415aa20 0x7f50940b2950 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.577+0000 7f509a8e9700 1 -- 172.21.15.28:0/1062929649 >> 172.21.15.28:0/1062929649 conn(0x7f50940b1650 msgr2=0x7f50940b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:40.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.577+0000 7f509a8e9700 1 -- 172.21.15.28:0/1062929649 shutdown_connections 2023-11-28T11:09:40.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.577+0000 7f509a8e9700 1 -- 172.21.15.28:0/1062929649 wait complete. 2023-11-28T11:09:40.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.578+0000 7f509a8e9700 1 Processor -- start 2023-11-28T11:09:40.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.579+0000 7f509a8e9700 1 -- start start 2023-11-28T11:09:40.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.579+0000 7f509a8e9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50940b5500 0x7f50940cf270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.579+0000 7f509a8e9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f509415a650 0x7f50940cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.579+0000 7f509a8e9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f509415aa20 0x7f509414af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.579+0000 7f509a8e9700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50940cdfa0 con 0x7f509415aa20 2023-11-28T11:09:40.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.579+0000 7f509a8e9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f509414b4a0 con 0x7f50940b5500 2023-11-28T11:09:40.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.579+0000 7f509a8e9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50940cede0 con 0x7f509415a650 2023-11-28T11:09:40.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.579+0000 7f5098e86700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f509415aa20 0x7f509414af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.579+0000 7f5098e86700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f509415aa20 0x7f509414af60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48340/0 (socket says 172.21.15.28:48340) 2023-11-28T11:09:40.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.579+0000 7f5098e86700 1 -- 172.21.15.28:0/1222387960 learned_addr learned my addr 172.21.15.28:0/1222387960 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:40.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.580+0000 7f50937fe700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f509415a650 0x7f50940cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.580+0000 7f5093fff700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50940b5500 0x7f50940cf270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.580+0000 7f5098e86700 1 -- 172.21.15.28:0/1222387960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f509415a650 msgr2=0x7f50940cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.580+0000 7f5098e86700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f509415a650 0x7f50940cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.580+0000 7f5098e86700 1 -- 172.21.15.28:0/1222387960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50940b5500 msgr2=0x7f50940cf270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.580+0000 7f5098e86700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50940b5500 0x7f50940cf270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.580+0000 7f5098e86700 1 -- 172.21.15.28:0/1222387960 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5084009550 con 0x7f509415aa20 2023-11-28T11:09:40.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.580+0000 7f50937fe700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f509415a650 0x7f50940cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:40.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.580+0000 7f5093fff700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50940b5500 0x7f50940cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:40.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.580+0000 7f5098e86700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f509415aa20 0x7f509414af60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f508c00cc50 tx=0x7f508c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.580+0000 7f5090ff9700 1 -- 172.21.15.28:0/1222387960 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c00ad80 con 0x7f509415aa20 2023-11-28T11:09:40.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.580+0000 7f5090ff9700 1 -- 172.21.15.28:0/1222387960 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508c00ce50 con 0x7f509415aa20 2023-11-28T11:09:40.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.580+0000 7f509a8e9700 1 -- 172.21.15.28:0/1222387960 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50940ccbc0 con 0x7f509415aa20 2023-11-28T11:09:40.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.580+0000 7f5090ff9700 1 -- 172.21.15.28:0/1222387960 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508c003c40 con 0x7f509415aa20 2023-11-28T11:09:40.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.580+0000 7f509a8e9700 1 -- 172.21.15.28:0/1222387960 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50940cabe0 con 0x7f509415aa20 2023-11-28T11:09:40.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.581+0000 7f509a8e9700 1 -- 172.21.15.28:0/1222387960 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50940ccd50 con 0x7f509415aa20 2023-11-28T11:09:40.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.582+0000 7f5090ff9700 1 -- 172.21.15.28:0/1222387960 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f508c010050 con 0x7f509415aa20 2023-11-28T11:09:40.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.582+0000 7f5090ff9700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f50740682f0 0x7f507406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.582+0000 7f5090ff9700 1 -- 172.21.15.28:0/1222387960 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f508c085f20 con 0x7f509415aa20 2023-11-28T11:09:40.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.582+0000 7f5093fff700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f50740682f0 0x7f507406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.583+0000 7f5093fff700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f50740682f0 0x7f507406a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f507c003f40 tx=0x7f507c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.586+0000 7f5090ff9700 1 -- 172.21.15.28:0/1222387960 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f508c052450 con 0x7f509415aa20 2023-11-28T11:09:40.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.783+0000 7f509a8e9700 1 -- 172.21.15.28:0/1222387960 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f509415b170 con 0x7f509415aa20 2023-11-28T11:09:40.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.784+0000 7f5090ff9700 1 -- 172.21.15.28:0/1222387960 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19737 (secure 0 0 0) 0x7f508c055a70 con 0x7f509415aa20 2023-11-28T11:09:40.785 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:40.785 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:09:34.614188+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:40.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.787+0000 7f509a8e9700 1 -- 172.21.15.28:0/1222387960 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f50740682f0 msgr2=0x7f507406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.787+0000 7f509a8e9700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f50740682f0 0x7f507406a7a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f507c003f40 tx=0x7f507c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:40.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.787+0000 7f509a8e9700 1 -- 172.21.15.28:0/1222387960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f509415aa20 msgr2=0x7f509414af60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.787+0000 7f509a8e9700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f509415aa20 0x7f509414af60 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f508c00cc50 tx=0x7f508c00a340 comp rx=0 tx=0).stop 2023-11-28T11:09:40.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.787+0000 7f509a8e9700 1 -- 172.21.15.28:0/1222387960 shutdown_connections 2023-11-28T11:09:40.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.787+0000 7f509a8e9700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f50740682f0 0x7f507406a7a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.787+0000 7f509a8e9700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f509415aa20 0x7f509414af60 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.787+0000 7f509a8e9700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f50940b5500 0x7f50940cf270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.787+0000 7f509a8e9700 1 --2- 172.21.15.28:0/1222387960 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f509415a650 0x7f50940cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.787+0000 7f509a8e9700 1 -- 172.21.15.28:0/1222387960 >> 172.21.15.28:0/1222387960 conn(0x7f50940b1650 msgr2=0x7f50940b4960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:40.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.787+0000 7f509a8e9700 1 -- 172.21.15.28:0/1222387960 shutdown_connections 2023-11-28T11:09:40.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.787+0000 7f509a8e9700 1 -- 172.21.15.28:0/1222387960 wait complete. 2023-11-28T11:09:40.800 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:40.800 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:40.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.928+0000 7f0d72984700 1 Processor -- start 2023-11-28T11:09:40.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.928+0000 7f0d72984700 1 -- start start 2023-11-28T11:09:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.928+0000 7f0d72984700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0d6c0cd140 0x7f0d6c06b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.928+0000 7f0d72984700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d6c06d140 0x7f0d6c06b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.928+0000 7f0d72984700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d6c0ae730 con 0x7f0d6c15a640 2023-11-28T11:09:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.928+0000 7f0d72984700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d6c0cb210 con 0x7f0d6c0cd140 2023-11-28T11:09:40.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.928+0000 7f0d72984700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c0c3e10 con 0x7f0d6c06d140 2023-11-28T11:09:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6bfff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0d6c15a640 0x7f0d6c0c3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36402/0 (socket says 172.21.15.28:36402) 2023-11-28T11:09:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6bfff700 1 -- 172.21.15.28:0/3549833576 learned_addr learned my addr 172.21.15.28:0/3549833576 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d70f21700 1 --2- 172.21.15.28:0/3549833576 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d6c06d140 0x7f0d6c06b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6b7fe700 1 --2- 172.21.15.28:0/3549833576 >> v2:172.21.15.59:3300/0 conn(0x7f0d6c0cd140 0x7f0d6c06b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 -- 172.21.15.28:0/3549833576 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2380179710 0 0) 0x7f0d6c0ae730 con 0x7f0d6c15a640 2023-11-28T11:09:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 -- 172.21.15.28:0/3549833576 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d540035b0 con 0x7f0d6c15a640 2023-11-28T11:09:40.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 -- 172.21.15.28:0/3549833576 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 394140754 0 0) 0x7f0d540035b0 con 0x7f0d6c15a640 2023-11-28T11:09:40.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 -- 172.21.15.28:0/3549833576 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d6c06d140 msgr2=0x7f0d6c06b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:40.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 --2- 172.21.15.28:0/3549833576 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d6c06d140 0x7f0d6c06b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 -- 172.21.15.28:0/3549833576 >> v2:172.21.15.59:3300/0 conn(0x7f0d6c0cd140 msgr2=0x7f0d6c06b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:40.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 --2- 172.21.15.28:0/3549833576 >> v2:172.21.15.59:3300/0 conn(0x7f0d6c0cd140 0x7f0d6c06b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 -- 172.21.15.28:0/3549833576 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c06e9e0 con 0x7f0d6c15a640 2023-11-28T11:09:40.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 -- 172.21.15.28:0/3549833576 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f0d5c001670 con 0x7f0d6c15a640 2023-11-28T11:09:40.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6b7fe700 1 --2- 172.21.15.28:0/3549833576 >> v2:172.21.15.59:3300/0 conn(0x7f0d6c0cd140 0x7f0d6c06b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:40.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 -- 172.21.15.28:0/3549833576 >> v1:172.21.15.28:6789/0 conn(0x7f0d6c15a640 legacy=0x7f0d6c0c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 --2- 172.21.15.28:0/3549833576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d54003a00 0x7f0d54003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 --2- 172.21.15.28:0/3549833576 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d6c06d140 0x7f0d54004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 --2- 172.21.15.28:0/3549833576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d54004d90 0x7f0d54005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 -- 172.21.15.28:0/3549833576 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c0cb210 con 0x7f0d54003a00 2023-11-28T11:09:40.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 -- 172.21.15.28:0/3549833576 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c0c3e10 con 0x7f0d54004d90 2023-11-28T11:09:40.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6affd700 1 -- 172.21.15.28:0/3549833576 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5400ba10 con 0x7f0d6c06d140 2023-11-28T11:09:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6bfff700 1 --2- 172.21.15.28:0/3549833576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d54003a00 0x7f0d54003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d6b7fe700 1 --2- 172.21.15.28:0/3549833576 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d6c06d140 0x7f0d54004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.929+0000 7f0d70f21700 1 --2- 172.21.15.28:0/3549833576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d54004d90 0x7f0d54005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.930+0000 7f0d6bfff700 1 -- 172.21.15.28:0/3549833576 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d6c06d140 msgr2=0x7f0d54004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:40.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.930+0000 7f0d6bfff700 1 --2- 172.21.15.28:0/3549833576 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d6c06d140 0x7f0d54004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.930+0000 7f0d6bfff700 1 -- 172.21.15.28:0/3549833576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d54004d90 msgr2=0x7f0d54005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.930+0000 7f0d6bfff700 1 --2- 172.21.15.28:0/3549833576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d54004d90 0x7f0d54005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.930+0000 7f0d6bfff700 1 -- 172.21.15.28:0/3549833576 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d540035b0 con 0x7f0d54003a00 2023-11-28T11:09:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.930+0000 7f0d6bfff700 1 --2- 172.21.15.28:0/3549833576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d54003a00 0x7f0d54003e10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00c320 tx=0x7f0d54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0a0360269f55487 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.930+0000 7f0d6affd700 1 -- 172.21.15.28:0/3549833576 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c00ce00 con 0x7f0d54003a00 2023-11-28T11:09:40.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.930+0000 7f0d6affd700 1 -- 172.21.15.28:0/3549833576 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d5c005960 con 0x7f0d54003a00 2023-11-28T11:09:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.930+0000 7f0d6affd700 1 -- 172.21.15.28:0/3549833576 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d5c005d10 con 0x7f0d54003a00 2023-11-28T11:09:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.930+0000 7f0d72984700 1 -- 172.21.15.28:0/3549833576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d54003a00 msgr2=0x7f0d54003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.930+0000 7f0d72984700 1 --2- 172.21.15.28:0/3549833576 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d54003a00 0x7f0d54003e10 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f0d5c00c320 tx=0x7f0d54005880 comp rx=0 tx=0).stop 2023-11-28T11:09:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.930+0000 7f0d70f21700 1 -- 172.21.15.28:0/3549833576 reap_dead start 2023-11-28T11:09:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.931+0000 7f0d72984700 1 -- 172.21.15.28:0/3549833576 shutdown_connections 2023-11-28T11:09:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.931+0000 7f0d72984700 1 -- 172.21.15.28:0/3549833576 >> 172.21.15.28:0/3549833576 conn(0x7f0d6c0cc630 msgr2=0x7f0d6c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:40.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.931+0000 7f0d72984700 1 -- 172.21.15.28:0/3549833576 shutdown_connections 2023-11-28T11:09:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.931+0000 7f0d72984700 1 -- 172.21.15.28:0/3549833576 wait complete. 2023-11-28T11:09:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.932+0000 7f0d72984700 1 Processor -- start 2023-11-28T11:09:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.932+0000 7f0d72984700 1 -- start start 2023-11-28T11:09:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d72984700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d54003a00 0x7f0d6c155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d72984700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6c156080 0x7f0d6c15c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d72984700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d6c1588b0 0x7f0d6c158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d72984700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c0b34e0 con 0x7f0d6c156080 2023-11-28T11:09:40.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d72984700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c15cd70 con 0x7f0d6c1588b0 2023-11-28T11:09:40.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d72984700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d6c159260 con 0x7f0d54003a00 2023-11-28T11:09:40.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d6b7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6c156080 0x7f0d6c15c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d6b7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6c156080 0x7f0d6c15c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48358/0 (socket says 172.21.15.28:48358) 2023-11-28T11:09:40.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d6b7fe700 1 -- 172.21.15.28:0/3956828424 learned_addr learned my addr 172.21.15.28:0/3956828424 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:40.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d6b7fe700 1 -- 172.21.15.28:0/3956828424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d54003a00 msgr2=0x7f0d6c155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d6bfff700 1 --2- 172.21.15.28:0/3956828424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d54003a00 0x7f0d6c155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d6b7fe700 1 --2- 172.21.15.28:0/3956828424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d54003a00 0x7f0d6c155910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d70f21700 1 --2- 172.21.15.28:0/3956828424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d6c1588b0 0x7f0d6c158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d6b7fe700 1 -- 172.21.15.28:0/3956828424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d6c1588b0 msgr2=0x7f0d6c158d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d6b7fe700 1 --2- 172.21.15.28:0/3956828424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d6c1588b0 0x7f0d6c158d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d6b7fe700 1 -- 172.21.15.28:0/3956828424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d6c0aec60 con 0x7f0d6c156080 2023-11-28T11:09:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d6b7fe700 1 --2- 172.21.15.28:0/3956828424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6c156080 0x7f0d6c15c520 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0d60011d30 tx=0x7f0d6000f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.933+0000 7f0d68ff9700 1 -- 172.21.15.28:0/3956828424 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d6000fb30 con 0x7f0d6c156080 2023-11-28T11:09:40.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.934+0000 7f0d68ff9700 1 -- 172.21.15.28:0/3956828424 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d6000fc90 con 0x7f0d6c156080 2023-11-28T11:09:40.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.934+0000 7f0d72984700 1 -- 172.21.15.28:0/3956828424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d6c0ae980 con 0x7f0d6c156080 2023-11-28T11:09:40.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.934+0000 7f0d68ff9700 1 -- 172.21.15.28:0/3956828424 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d60004030 con 0x7f0d6c156080 2023-11-28T11:09:40.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.934+0000 7f0d72984700 1 -- 172.21.15.28:0/3956828424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d6c0aef40 con 0x7f0d6c156080 2023-11-28T11:09:40.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.935+0000 7f0d72984700 1 -- 172.21.15.28:0/3956828424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d6c0b3620 con 0x7f0d6c156080 2023-11-28T11:09:40.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.935+0000 7f0d68ff9700 1 -- 172.21.15.28:0/3956828424 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0d60024070 con 0x7f0d6c156080 2023-11-28T11:09:40.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.935+0000 7f0d68ff9700 1 --2- 172.21.15.28:0/3956828424 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0d580682f0 0x7f0d5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:40.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.935+0000 7f0d68ff9700 1 -- 172.21.15.28:0/3956828424 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0d600864f0 con 0x7f0d6c156080 2023-11-28T11:09:40.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.935+0000 7f0d6bfff700 1 --2- 172.21.15.28:0/3956828424 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0d580682f0 0x7f0d5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:40.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.936+0000 7f0d6bfff700 1 --2- 172.21.15.28:0/3956828424 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0d580682f0 0x7f0d5806a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c1594a0 tx=0x7f0d5c00a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:40.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:40.940+0000 7f0d68ff9700 1 -- 172.21.15.28:0/3956828424 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0d6c0b3620 con 0x7f0d6c156080 2023-11-28T11:09:41.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.132+0000 7f0d72984700 1 -- 172.21.15.28:0/3956828424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0d6c0b3620 con 0x7f0d6c156080 2023-11-28T11:09:41.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.133+0000 7f0d68ff9700 1 -- 172.21.15.28:0/3956828424 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0d6c0b3620 con 0x7f0d6c156080 2023-11-28T11:09:41.134 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:09:41.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.135+0000 7f0d72984700 1 -- 172.21.15.28:0/3956828424 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0d580682f0 msgr2=0x7f0d5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.135+0000 7f0d72984700 1 --2- 172.21.15.28:0/3956828424 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0d580682f0 0x7f0d5806a7a0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0d6c1594a0 tx=0x7f0d5c00a000 comp rx=0 tx=0).stop 2023-11-28T11:09:41.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.135+0000 7f0d72984700 1 -- 172.21.15.28:0/3956828424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6c156080 msgr2=0x7f0d6c15c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.135+0000 7f0d72984700 1 --2- 172.21.15.28:0/3956828424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6c156080 0x7f0d6c15c520 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f0d60011d30 tx=0x7f0d6000f520 comp rx=0 tx=0).stop 2023-11-28T11:09:41.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.135+0000 7f0d72984700 1 -- 172.21.15.28:0/3956828424 shutdown_connections 2023-11-28T11:09:41.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.135+0000 7f0d72984700 1 --2- 172.21.15.28:0/3956828424 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0d580682f0 0x7f0d5806a7a0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.135+0000 7f0d72984700 1 --2- 172.21.15.28:0/3956828424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0d6c1588b0 0x7f0d6c158d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.135+0000 7f0d72984700 1 --2- 172.21.15.28:0/3956828424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0d54003a00 0x7f0d6c155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.135+0000 7f0d72984700 1 --2- 172.21.15.28:0/3956828424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0d6c156080 0x7f0d6c15c520 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.135+0000 7f0d72984700 1 -- 172.21.15.28:0/3956828424 >> 172.21.15.28:0/3956828424 conn(0x7f0d6c0cc630 msgr2=0x7f0d6c0cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.135+0000 7f0d72984700 1 -- 172.21.15.28:0/3956828424 shutdown_connections 2023-11-28T11:09:41.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.135+0000 7f0d72984700 1 -- 172.21.15.28:0/3956828424 wait complete. 2023-11-28T11:09:41.148 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:41.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.275+0000 7ff2be20d700 1 Processor -- start 2023-11-28T11:09:41.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.275+0000 7ff2be20d700 1 -- start start 2023-11-28T11:09:41.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.276+0000 7ff2be20d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff2b80ca360 0x7ff2b806f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.276+0000 7ff2be20d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2b80cae50 0x7ff2b80cb2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.276+0000 7ff2be20d700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b80caab0 con 0x7ff2b815c950 2023-11-28T11:09:41.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.276+0000 7ff2be20d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b815cd20 con 0x7ff2b80ca360 2023-11-28T11:09:41.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.276+0000 7ff2be20d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b806a8f0 con 0x7ff2b80cae50 2023-11-28T11:09:41.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.276+0000 7ff2b77fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff2b815c950 0x7ff2b806a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36422/0 (socket says 172.21.15.28:36422) 2023-11-28T11:09:41.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.276+0000 7ff2b77fe700 1 -- 172.21.15.28:0/4203369458 learned_addr learned my addr 172.21.15.28:0/4203369458 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:41.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 -- 172.21.15.28:0/4203369458 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1926223486 0 0) 0x7ff2b80caab0 con 0x7ff2b815c950 2023-11-28T11:09:41.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2affff700 1 --2- 172.21.15.28:0/4203369458 >> v2:172.21.15.59:3300/0 conn(0x7ff2b80ca360 0x7ff2b806f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 -- 172.21.15.28:0/4203369458 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff2a40035b0 con 0x7ff2b815c950 2023-11-28T11:09:41.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b7fff700 1 --2- 172.21.15.28:0/4203369458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2b80cae50 0x7ff2b80cb2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 -- 172.21.15.28:0/4203369458 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2171719824 0 0) 0x7ff2a40035b0 con 0x7ff2b815c950 2023-11-28T11:09:41.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 -- 172.21.15.28:0/4203369458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2b80cae50 msgr2=0x7ff2b80cb2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:41.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 --2- 172.21.15.28:0/4203369458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2b80cae50 0x7ff2b80cb2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 -- 172.21.15.28:0/4203369458 >> v2:172.21.15.59:3300/0 conn(0x7ff2b80ca360 msgr2=0x7ff2b806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:41.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 --2- 172.21.15.28:0/4203369458 >> v2:172.21.15.59:3300/0 conn(0x7ff2b80ca360 0x7ff2b806f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 -- 172.21.15.28:0/4203369458 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b80cb7f0 con 0x7ff2b815c950 2023-11-28T11:09:41.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 -- 172.21.15.28:0/4203369458 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7ff2a8001670 con 0x7ff2b815c950 2023-11-28T11:09:41.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 -- 172.21.15.28:0/4203369458 >> v1:172.21.15.28:6789/0 conn(0x7ff2b815c950 legacy=0x7ff2b806a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 --2- 172.21.15.28:0/4203369458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2b80cae50 0x7ff2a4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 --2- 172.21.15.28:0/4203369458 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2a4003fb0 0x7ff2a4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 --2- 172.21.15.28:0/4203369458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a4004d90 0x7ff2a4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 -- 172.21.15.28:0/4203369458 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b815cd20 con 0x7ff2a4003fb0 2023-11-28T11:09:41.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 -- 172.21.15.28:0/4203369458 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b806a8f0 con 0x7ff2a4004d90 2023-11-28T11:09:41.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b6ffd700 1 -- 172.21.15.28:0/4203369458 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2a400dc20 con 0x7ff2b80cae50 2023-11-28T11:09:41.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2affff700 1 --2- 172.21.15.28:0/4203369458 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2a4003fb0 0x7ff2a4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b77fe700 1 --2- 172.21.15.28:0/4203369458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2b80cae50 0x7ff2a4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b7fff700 1 --2- 172.21.15.28:0/4203369458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a4004d90 0x7ff2a4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2affff700 1 -- 172.21.15.28:0/4203369458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2b80cae50 msgr2=0x7ff2a4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:41.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2affff700 1 --2- 172.21.15.28:0/4203369458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2b80cae50 0x7ff2a4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2affff700 1 -- 172.21.15.28:0/4203369458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a4004d90 msgr2=0x7ff2a4005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:41.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2affff700 1 --2- 172.21.15.28:0/4203369458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a4004d90 0x7ff2a4005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2affff700 1 -- 172.21.15.28:0/4203369458 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2a40035b0 con 0x7ff2a4003fb0 2023-11-28T11:09:41.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b7fff700 1 --2- 172.21.15.28:0/4203369458 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2a4004d90 0x7ff2a4005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:41.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.277+0000 7ff2b77fe700 1 --2- 172.21.15.28:0/4203369458 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2b80cae50 0x7ff2a4003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:41.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.278+0000 7ff2affff700 1 --2- 172.21.15.28:0/4203369458 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2a4003fb0 0x7ff2a4004850 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009b60 tx=0x7ff2a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9597dd7b743e9d30 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.278+0000 7ff2b6ffd700 1 -- 172.21.15.28:0/4203369458 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a000e6c0 con 0x7ff2a4003fb0 2023-11-28T11:09:41.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.278+0000 7ff2b6ffd700 1 -- 172.21.15.28:0/4203369458 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a000e820 con 0x7ff2a4003fb0 2023-11-28T11:09:41.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.278+0000 7ff2b6ffd700 1 -- 172.21.15.28:0/4203369458 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a000eb90 con 0x7ff2a4003fb0 2023-11-28T11:09:41.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.278+0000 7ff2be20d700 1 -- 172.21.15.28:0/4203369458 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2a4003fb0 msgr2=0x7ff2a4004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.278+0000 7ff2be20d700 1 --2- 172.21.15.28:0/4203369458 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2a4003fb0 0x7ff2a4004850 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0009b60 tx=0x7ff2a4005880 comp rx=0 tx=0).stop 2023-11-28T11:09:41.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.278+0000 7ff2b7fff700 1 -- 172.21.15.28:0/4203369458 reap_dead start 2023-11-28T11:09:41.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.278+0000 7ff2be20d700 1 -- 172.21.15.28:0/4203369458 shutdown_connections 2023-11-28T11:09:41.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.278+0000 7ff2be20d700 1 -- 172.21.15.28:0/4203369458 >> 172.21.15.28:0/4203369458 conn(0x7ff2b80b0df0 msgr2=0x7ff2b80b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.278+0000 7ff2be20d700 1 -- 172.21.15.28:0/4203369458 shutdown_connections 2023-11-28T11:09:41.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.278+0000 7ff2be20d700 1 -- 172.21.15.28:0/4203369458 wait complete. 2023-11-28T11:09:41.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.279+0000 7ff2be20d700 1 Processor -- start 2023-11-28T11:09:41.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.280+0000 7ff2be20d700 1 -- start start 2023-11-28T11:09:41.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.280+0000 7ff2be20d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2a4003fb0 0x7ff2b8144bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.280+0000 7ff2be20d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2b81ee3a0 0x7ff2b8144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.280+0000 7ff2be20d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2b81f1600 0x7ff2b81f1a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.280+0000 7ff2be20d700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b80c3300 con 0x7ff2a4003fb0 2023-11-28T11:09:41.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.280+0000 7ff2be20d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b80c35e0 con 0x7ff2b81f1600 2023-11-28T11:09:41.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.280+0000 7ff2be20d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b80c38c0 con 0x7ff2b81ee3a0 2023-11-28T11:09:41.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.280+0000 7ff2b77fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2a4003fb0 0x7ff2b8144bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.280+0000 7ff2b77fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2a4003fb0 0x7ff2b8144bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48376/0 (socket says 172.21.15.28:48376) 2023-11-28T11:09:41.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.280+0000 7ff2b77fe700 1 -- 172.21.15.28:0/3327715593 learned_addr learned my addr 172.21.15.28:0/3327715593 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:41.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.281+0000 7ff2affff700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2b81ee3a0 0x7ff2b8144340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.281+0000 7ff2b77fe700 1 -- 172.21.15.28:0/3327715593 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2b81ee3a0 msgr2=0x7ff2b8144340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.281+0000 7ff2b7fff700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2b81f1600 0x7ff2b81f1a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.281+0000 7ff2b77fe700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2b81ee3a0 0x7ff2b8144340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.281+0000 7ff2b77fe700 1 -- 172.21.15.28:0/3327715593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2b81f1600 msgr2=0x7ff2b81f1a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.288 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.281+0000 7ff2b77fe700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2b81f1600 0x7ff2b81f1a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.288 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.281+0000 7ff2b77fe700 1 -- 172.21.15.28:0/3327715593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b80ca7d0 con 0x7ff2a4003fb0 2023-11-28T11:09:41.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.281+0000 7ff2b7fff700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2b81f1600 0x7ff2b81f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:41.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.281+0000 7ff2affff700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2b81ee3a0 0x7ff2b8144340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:41.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.281+0000 7ff2b77fe700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2a4003fb0 0x7ff2b8144bc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800e8c0 tx=0x7ff2a800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.281+0000 7ff2b4ff9700 1 -- 172.21.15.28:0/3327715593 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a800eec0 con 0x7ff2a4003fb0 2023-11-28T11:09:41.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.281+0000 7ff2b4ff9700 1 -- 172.21.15.28:0/3327715593 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a800c7e0 con 0x7ff2a4003fb0 2023-11-28T11:09:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.281+0000 7ff2be20d700 1 -- 172.21.15.28:0/3327715593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2b806e770 con 0x7ff2a4003fb0 2023-11-28T11:09:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.281+0000 7ff2b4ff9700 1 -- 172.21.15.28:0/3327715593 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a800cb50 con 0x7ff2a4003fb0 2023-11-28T11:09:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.281+0000 7ff2be20d700 1 -- 172.21.15.28:0/3327715593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b806ea10 con 0x7ff2a4003fb0 2023-11-28T11:09:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.282+0000 7ff2be20d700 1 -- 172.21.15.28:0/3327715593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b81f0f40 con 0x7ff2a4003fb0 2023-11-28T11:09:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.282+0000 7ff2b4ff9700 1 -- 172.21.15.28:0/3327715593 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff2a800ccb0 con 0x7ff2a4003fb0 2023-11-28T11:09:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.283+0000 7ff2b4ff9700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff2980682f0 0x7ff29806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.283+0000 7ff2affff700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff2980682f0 0x7ff29806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.283+0000 7ff2b4ff9700 1 -- 172.21.15.28:0/3327715593 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff2a8086080 con 0x7ff2a4003fb0 2023-11-28T11:09:41.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.283+0000 7ff2affff700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff2980682f0 0x7ff29806a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80cb5a0 tx=0x7ff2a0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.287+0000 7ff2b4ff9700 1 -- 172.21.15.28:0/3327715593 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff2b81f0f40 con 0x7ff2a4003fb0 2023-11-28T11:09:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.478+0000 7ff2be20d700 1 -- 172.21.15.28:0/3327715593 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff2b81f0f40 con 0x7ff2a4003fb0 2023-11-28T11:09:41.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.479+0000 7ff2b4ff9700 1 -- 172.21.15.28:0/3327715593 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff2b81f0f40 con 0x7ff2a4003fb0 2023-11-28T11:09:41.480 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:09:41.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.481+0000 7ff2be20d700 1 -- 172.21.15.28:0/3327715593 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff2980682f0 msgr2=0x7ff29806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.481+0000 7ff2be20d700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff2980682f0 0x7ff29806a7a0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7ff2b80cb5a0 tx=0x7ff2a0008030 comp rx=0 tx=0).stop 2023-11-28T11:09:41.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.481+0000 7ff2be20d700 1 -- 172.21.15.28:0/3327715593 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2a4003fb0 msgr2=0x7ff2b8144bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.481+0000 7ff2be20d700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2a4003fb0 0x7ff2b8144bc0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7ff2a800e8c0 tx=0x7ff2a800ec80 comp rx=0 tx=0).stop 2023-11-28T11:09:41.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.481+0000 7ff2be20d700 1 -- 172.21.15.28:0/3327715593 shutdown_connections 2023-11-28T11:09:41.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.481+0000 7ff2be20d700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff2980682f0 0x7ff29806a7a0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.481+0000 7ff2be20d700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff2a4003fb0 0x7ff2b8144bc0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.481+0000 7ff2be20d700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2b81f1600 0x7ff2b81f1a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.481+0000 7ff2be20d700 1 --2- 172.21.15.28:0/3327715593 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff2b81ee3a0 0x7ff2b8144340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.481+0000 7ff2be20d700 1 -- 172.21.15.28:0/3327715593 >> 172.21.15.28:0/3327715593 conn(0x7ff2b80b0df0 msgr2=0x7ff2b8071920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.481+0000 7ff2be20d700 1 -- 172.21.15.28:0/3327715593 shutdown_connections 2023-11-28T11:09:41.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.481+0000 7ff2be20d700 1 -- 172.21.15.28:0/3327715593 wait complete. 2023-11-28T11:09:41.494 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:41.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.623+0000 7f2b2a28d700 1 Processor -- start 2023-11-28T11:09:41.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.623+0000 7f2b2a28d700 1 -- start start 2023-11-28T11:09:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.624+0000 7f2b2a28d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b24159500 0x7f2b240cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.624+0000 7f2b2a28d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2b240b1d40 0x7f2b240ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.624+0000 7f2b2a28d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b2415e040 con 0x7f2b2415dbf0 2023-11-28T11:09:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.624+0000 7f2b2a28d700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b240cede0 con 0x7f2b240b1d40 2023-11-28T11:09:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.624+0000 7f2b2a28d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b240ceff0 con 0x7f2b24159500 2023-11-28T11:09:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.624+0000 7f2b2882a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2b240b1d40 0x7f2b240ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.624+0000 7f2b2882a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2b240b1d40 0x7f2b240ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48388/0 (socket says 172.21.15.28:48388) 2023-11-28T11:09:41.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.624+0000 7f2b2882a700 1 -- 172.21.15.28:0/2092784247 learned_addr learned my addr 172.21.15.28:0/2092784247 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:41.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.624+0000 7f2b23fff700 1 --2- 172.21.15.28:0/2092784247 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b24159500 0x7f2b240cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.624+0000 7f2b2882a700 1 -- 172.21.15.28:0/2092784247 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b24159500 msgr2=0x7f2b240cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:41.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.624+0000 7f2b2882a700 1 --2- 172.21.15.28:0/2092784247 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b24159500 0x7f2b240cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.624+0000 7f2b2882a700 1 -- 172.21.15.28:0/2092784247 >> v1:172.21.15.59:6789/0 conn(0x7f2b2415dbf0 legacy=0x7f2b240b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.624+0000 7f2b2882a700 1 -- 172.21.15.28:0/2092784247 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b240cf2d0 con 0x7f2b240b1d40 2023-11-28T11:09:41.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.625+0000 7f2b2882a700 1 --2- 172.21.15.28:0/2092784247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b240b1d40 0x7f2b240ce8a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009e50 tx=0x7f2b1c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1580d790a58d98fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.625+0000 7f2b22ffd700 1 -- 172.21.15.28:0/2092784247 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c00e070 con 0x7f2b240b1d40 2023-11-28T11:09:41.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.625+0000 7f2b22ffd700 1 -- 172.21.15.28:0/2092784247 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c00b910 con 0x7f2b240b1d40 2023-11-28T11:09:41.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.625+0000 7f2b22ffd700 1 -- 172.21.15.28:0/2092784247 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c00bcc0 con 0x7f2b240b1d40 2023-11-28T11:09:41.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.625+0000 7f2b2a28d700 1 -- 172.21.15.28:0/2092784247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b240b1d40 msgr2=0x7f2b240ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.625+0000 7f2b2a28d700 1 --2- 172.21.15.28:0/2092784247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b240b1d40 0x7f2b240ce8a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c009e50 tx=0x7f2b1c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:41.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.625+0000 7f2b2a28d700 1 -- 172.21.15.28:0/2092784247 shutdown_connections 2023-11-28T11:09:41.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.625+0000 7f2b2a28d700 1 --2- 172.21.15.28:0/2092784247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b240b1d40 0x7f2b240ce8a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.625+0000 7f2b2a28d700 1 --2- 172.21.15.28:0/2092784247 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b24159500 0x7f2b240cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.625+0000 7f2b2a28d700 1 -- 172.21.15.28:0/2092784247 >> 172.21.15.28:0/2092784247 conn(0x7f2b240cc630 msgr2=0x7f2b240cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.625+0000 7f2b2a28d700 1 -- 172.21.15.28:0/2092784247 shutdown_connections 2023-11-28T11:09:41.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.625+0000 7f2b2a28d700 1 -- 172.21.15.28:0/2092784247 wait complete. 2023-11-28T11:09:41.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.627+0000 7f2b2a28d700 1 Processor -- start 2023-11-28T11:09:41.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.627+0000 7f2b2a28d700 1 -- start start 2023-11-28T11:09:41.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.627+0000 7f2b2a28d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b240b1d40 0x7f2b240b4c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.627+0000 7f2b2a28d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b24159500 0x7f2b240c5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.627+0000 7f2b2a28d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b2415dbf0 0x7f2b2406adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.627+0000 7f2b2a28d700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2415a5a0 con 0x7f2b2415dbf0 2023-11-28T11:09:41.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.627+0000 7f2b2a28d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2415a3e0 con 0x7f2b240b1d40 2023-11-28T11:09:41.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.627+0000 7f2b2a28d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b240b5140 con 0x7f2b24159500 2023-11-28T11:09:41.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b2882a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b2415dbf0 0x7f2b2406adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b2882a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b2415dbf0 0x7f2b2406adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48394/0 (socket says 172.21.15.28:48394) 2023-11-28T11:09:41.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b2882a700 1 -- 172.21.15.28:0/4139202705 learned_addr learned my addr 172.21.15.28:0/4139202705 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:41.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b23fff700 1 --2- 172.21.15.28:0/4139202705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b240b1d40 0x7f2b240b4c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b237fe700 1 --2- 172.21.15.28:0/4139202705 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b24159500 0x7f2b240c5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b2882a700 1 -- 172.21.15.28:0/4139202705 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b24159500 msgr2=0x7f2b240c5b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b2882a700 1 --2- 172.21.15.28:0/4139202705 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b24159500 0x7f2b240c5b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b2882a700 1 -- 172.21.15.28:0/4139202705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b240b1d40 msgr2=0x7f2b240b4c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b2882a700 1 --2- 172.21.15.28:0/4139202705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b240b1d40 0x7f2b240b4c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b2882a700 1 -- 172.21.15.28:0/4139202705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b1c009b30 con 0x7f2b2415dbf0 2023-11-28T11:09:41.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b2882a700 1 --2- 172.21.15.28:0/4139202705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b2415dbf0 0x7f2b2406adb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2b24066a20 tx=0x7f2b1c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b20ff9700 1 -- 172.21.15.28:0/4139202705 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c004800 con 0x7f2b2415dbf0 2023-11-28T11:09:41.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b20ff9700 1 -- 172.21.15.28:0/4139202705 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1c004960 con 0x7f2b2415dbf0 2023-11-28T11:09:41.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b20ff9700 1 -- 172.21.15.28:0/4139202705 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1c01b670 con 0x7f2b2415dbf0 2023-11-28T11:09:41.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b2a28d700 1 -- 172.21.15.28:0/4139202705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2415d3f0 con 0x7f2b2415dbf0 2023-11-28T11:09:41.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.628+0000 7f2b2a28d700 1 -- 172.21.15.28:0/4139202705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2415d110 con 0x7f2b2415dbf0 2023-11-28T11:09:41.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.629+0000 7f2b2a28d700 1 -- 172.21.15.28:0/4139202705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b2415d6d0 con 0x7f2b2415dbf0 2023-11-28T11:09:41.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.630+0000 7f2b20ff9700 1 -- 172.21.15.28:0/4139202705 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2b1c021030 con 0x7f2b2415dbf0 2023-11-28T11:09:41.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.630+0000 7f2b20ff9700 1 --2- 172.21.15.28:0/4139202705 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2b040682f0 0x7f2b0406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.630+0000 7f2b20ff9700 1 -- 172.21.15.28:0/4139202705 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2b1c087240 con 0x7f2b2415dbf0 2023-11-28T11:09:41.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.630+0000 7f2b23fff700 1 --2- 172.21.15.28:0/4139202705 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2b040682f0 0x7f2b0406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.631+0000 7f2b23fff700 1 --2- 172.21.15.28:0/4139202705 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2b040682f0 0x7f2b0406a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009be0 tx=0x7f2b14009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.635+0000 7f2b20ff9700 1 -- 172.21.15.28:0/4139202705 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2b1c053770 con 0x7f2b2415dbf0 2023-11-28T11:09:41.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.827+0000 7f2b2a28d700 1 -- 172.21.15.28:0/4139202705 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f2b2415a6e0 con 0x7f2b2415dbf0 2023-11-28T11:09:41.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.828+0000 7f2b20ff9700 1 -- 172.21.15.28:0/4139202705 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2b2415a6e0 con 0x7f2b2415dbf0 2023-11-28T11:09:41.828 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:41.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.830+0000 7f2b2a28d700 1 -- 172.21.15.28:0/4139202705 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2b040682f0 msgr2=0x7f2b0406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.830+0000 7f2b2a28d700 1 --2- 172.21.15.28:0/4139202705 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2b040682f0 0x7f2b0406a7a0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2b14009be0 tx=0x7f2b14009210 comp rx=0 tx=0).stop 2023-11-28T11:09:41.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.830+0000 7f2b2a28d700 1 -- 172.21.15.28:0/4139202705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b2415dbf0 msgr2=0x7f2b2406adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.830+0000 7f2b2a28d700 1 --2- 172.21.15.28:0/4139202705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b2415dbf0 0x7f2b2406adb0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f2b24066a20 tx=0x7f2b1c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.830+0000 7f2b2a28d700 1 -- 172.21.15.28:0/4139202705 shutdown_connections 2023-11-28T11:09:41.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.830+0000 7f2b2a28d700 1 --2- 172.21.15.28:0/4139202705 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2b040682f0 0x7f2b0406a7a0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.830+0000 7f2b2a28d700 1 --2- 172.21.15.28:0/4139202705 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2b2415dbf0 0x7f2b2406adb0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.830+0000 7f2b2a28d700 1 --2- 172.21.15.28:0/4139202705 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2b240b1d40 0x7f2b240b4c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.830+0000 7f2b2a28d700 1 --2- 172.21.15.28:0/4139202705 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2b24159500 0x7f2b240c5b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.830+0000 7f2b2a28d700 1 -- 172.21.15.28:0/4139202705 >> 172.21.15.28:0/4139202705 conn(0x7f2b240cc630 msgr2=0x7f2b240b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.830+0000 7f2b2a28d700 1 -- 172.21.15.28:0/4139202705 shutdown_connections 2023-11-28T11:09:41.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.830+0000 7f2b2a28d700 1 -- 172.21.15.28:0/4139202705 wait complete. 2023-11-28T11:09:41.842 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:41.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.971+0000 7f1daccea700 1 Processor -- start 2023-11-28T11:09:41.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.971+0000 7f1daccea700 1 -- start start 2023-11-28T11:09:41.974 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.972+0000 7f1daccea700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1da80cd140 0x7f1da806b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.974 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.972+0000 7f1daccea700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1da806d140 0x7f1da806b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.974 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.972+0000 7f1daccea700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da80ae730 con 0x7f1da815a640 2023-11-28T11:09:41.974 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.972+0000 7f1daccea700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da80cb210 con 0x7f1da806d140 2023-11-28T11:09:41.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.972+0000 7f1daccea700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da80c3e10 con 0x7f1da80cd140 2023-11-28T11:09:41.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.972+0000 7f1da6d9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1da806d140 0x7f1da806b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.972+0000 7f1da6d9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1da806d140 0x7f1da806b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48398/0 (socket says 172.21.15.28:48398) 2023-11-28T11:09:41.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.972+0000 7f1da659c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1da815a640 0x7f1da80c3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36456/0 (socket says 172.21.15.28:36456) 2023-11-28T11:09:41.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.972+0000 7f1da6d9d700 1 -- 172.21.15.28:0/3783782988 learned_addr learned my addr 172.21.15.28:0/3783782988 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:41.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.973+0000 7f1da5d9b700 1 --2- 172.21.15.28:0/3783782988 >> v2:172.21.15.59:3300/0 conn(0x7f1da80cd140 0x7f1da806b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.973+0000 7f1da6d9d700 1 -- 172.21.15.28:0/3783782988 >> v2:172.21.15.59:3300/0 conn(0x7f1da80cd140 msgr2=0x7f1da806b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:41.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.973+0000 7f1da6d9d700 1 --2- 172.21.15.28:0/3783782988 >> v2:172.21.15.59:3300/0 conn(0x7f1da80cd140 0x7f1da806b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.973+0000 7f1da6d9d700 1 -- 172.21.15.28:0/3783782988 >> v1:172.21.15.28:6789/0 conn(0x7f1da815a640 legacy=0x7f1da80c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.973+0000 7f1da6d9d700 1 -- 172.21.15.28:0/3783782988 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da806e9e0 con 0x7f1da806d140 2023-11-28T11:09:41.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.973+0000 7f1da6d9d700 1 --2- 172.21.15.28:0/3783782988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da806d140 0x7f1da806b730 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1da80d0c50 tx=0x7f1da000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58aeef9e2730c0e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.973+0000 7f1da559a700 1 -- 172.21.15.28:0/3783782988 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da000e070 con 0x7f1da806d140 2023-11-28T11:09:41.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.973+0000 7f1da559a700 1 -- 172.21.15.28:0/3783782988 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da000b9c0 con 0x7f1da806d140 2023-11-28T11:09:41.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.973+0000 7f1da559a700 1 -- 172.21.15.28:0/3783782988 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da000bd70 con 0x7f1da806d140 2023-11-28T11:09:41.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.973+0000 7f1daccea700 1 -- 172.21.15.28:0/3783782988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da806d140 msgr2=0x7f1da806b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.973+0000 7f1daccea700 1 --2- 172.21.15.28:0/3783782988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da806d140 0x7f1da806b730 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1da80d0c50 tx=0x7f1da000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:41.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.974+0000 7f1daccea700 1 -- 172.21.15.28:0/3783782988 shutdown_connections 2023-11-28T11:09:41.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.974+0000 7f1daccea700 1 --2- 172.21.15.28:0/3783782988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da806d140 0x7f1da806b730 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.974+0000 7f1daccea700 1 --2- 172.21.15.28:0/3783782988 >> v2:172.21.15.59:3300/0 conn(0x7f1da80cd140 0x7f1da806b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.974+0000 7f1daccea700 1 -- 172.21.15.28:0/3783782988 >> 172.21.15.28:0/3783782988 conn(0x7f1da80cc630 msgr2=0x7f1da80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:41.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.974+0000 7f1daccea700 1 -- 172.21.15.28:0/3783782988 shutdown_connections 2023-11-28T11:09:41.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.974+0000 7f1daccea700 1 -- 172.21.15.28:0/3783782988 wait complete. 2023-11-28T11:09:41.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.975+0000 7f1daccea700 1 Processor -- start 2023-11-28T11:09:41.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.975+0000 7f1daccea700 1 -- start start 2023-11-28T11:09:41.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1daccea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1da806d140 0x7f1da8155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1daccea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1da80cd140 0x7f1da81555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1daccea700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da815a640 0x7f1da814c730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1daccea700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80abbf0 con 0x7f1da815a640 2023-11-28T11:09:41.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1daccea700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80b02e0 con 0x7f1da806d140 2023-11-28T11:09:41.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1daccea700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da80b1400 con 0x7f1da80cd140 2023-11-28T11:09:41.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1da6d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da815a640 0x7f1da814c730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1da6d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da815a640 0x7f1da814c730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48404/0 (socket says 172.21.15.28:48404) 2023-11-28T11:09:41.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1da6d9d700 1 -- 172.21.15.28:0/3070355544 learned_addr learned my addr 172.21.15.28:0/3070355544 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:41.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1da659c700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1da806d140 0x7f1da8155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1da6d9d700 1 -- 172.21.15.28:0/3070355544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1da80cd140 msgr2=0x7f1da81555c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1da5d9b700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1da80cd140 0x7f1da81555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1da6d9d700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1da80cd140 0x7f1da81555c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1da6d9d700 1 -- 172.21.15.28:0/3070355544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1da806d140 msgr2=0x7f1da8155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:41.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1da6d9d700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1da806d140 0x7f1da8155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:41.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1da6d9d700 1 -- 172.21.15.28:0/3070355544 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da0009b30 con 0x7f1da815a640 2023-11-28T11:09:41.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1da659c700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1da806d140 0x7f1da8155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:41.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1da5d9b700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1da80cd140 0x7f1da81555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:41.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.976+0000 7f1da6d9d700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da815a640 0x7f1da814c730 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1da000b730 tx=0x7f1da0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.977+0000 7f1d96ffd700 1 -- 172.21.15.28:0/3070355544 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da00099b0 con 0x7f1da815a640 2023-11-28T11:09:41.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.977+0000 7f1d96ffd700 1 -- 172.21.15.28:0/3070355544 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1da00092e0 con 0x7f1da815a640 2023-11-28T11:09:41.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.977+0000 7f1daccea700 1 -- 172.21.15.28:0/3070355544 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da80aec60 con 0x7f1da815a640 2023-11-28T11:09:41.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.977+0000 7f1d96ffd700 1 -- 172.21.15.28:0/3070355544 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1da001b650 con 0x7f1da815a640 2023-11-28T11:09:41.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.977+0000 7f1daccea700 1 -- 172.21.15.28:0/3070355544 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da80ae980 con 0x7f1da815a640 2023-11-28T11:09:41.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.978+0000 7f1daccea700 1 -- 172.21.15.28:0/3070355544 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da80aef40 con 0x7f1da815a640 2023-11-28T11:09:41.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.978+0000 7f1d96ffd700 1 -- 172.21.15.28:0/3070355544 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1da0021030 con 0x7f1da815a640 2023-11-28T11:09:41.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.978+0000 7f1d96ffd700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1d880682f0 0x7f1d8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:41.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.978+0000 7f1d96ffd700 1 -- 172.21.15.28:0/3070355544 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1da00875b0 con 0x7f1da815a640 2023-11-28T11:09:41.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.979+0000 7f1da659c700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1d880682f0 0x7f1d8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:41.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.979+0000 7f1da659c700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1d880682f0 0x7f1d8806a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000f60 tx=0x7f1d9c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:41.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:41.983+0000 7f1d96ffd700 1 -- 172.21.15.28:0/3070355544 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1da0053ae0 con 0x7f1da815a640 2023-11-28T11:09:42.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.177+0000 7f1daccea700 1 -- 172.21.15.28:0/3070355544 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1da80739f0 con 0x7f1da815a640 2023-11-28T11:09:42.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.178+0000 7f1d96ffd700 1 -- 172.21.15.28:0/3070355544 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1da0057100 con 0x7f1da815a640 2023-11-28T11:09:42.179 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:09:42.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.180+0000 7f1daccea700 1 -- 172.21.15.28:0/3070355544 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1d880682f0 msgr2=0x7f1d8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.180+0000 7f1daccea700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1d880682f0 0x7f1d8806a7a0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1d9c000f60 tx=0x7f1d9c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:42.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.180+0000 7f1daccea700 1 -- 172.21.15.28:0/3070355544 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da815a640 msgr2=0x7f1da814c730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.180+0000 7f1daccea700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da815a640 0x7f1da814c730 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f1da000b730 tx=0x7f1da0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.180+0000 7f1daccea700 1 -- 172.21.15.28:0/3070355544 shutdown_connections 2023-11-28T11:09:42.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.180+0000 7f1daccea700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1d880682f0 0x7f1d8806a7a0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.180+0000 7f1daccea700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1da815a640 0x7f1da814c730 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.180+0000 7f1daccea700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1da806d140 0x7f1da8155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.180+0000 7f1daccea700 1 --2- 172.21.15.28:0/3070355544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1da80cd140 0x7f1da81555c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.180+0000 7f1daccea700 1 -- 172.21.15.28:0/3070355544 >> 172.21.15.28:0/3070355544 conn(0x7f1da80cc630 msgr2=0x7f1da80c3fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:42.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.180+0000 7f1daccea700 1 -- 172.21.15.28:0/3070355544 shutdown_connections 2023-11-28T11:09:42.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.180+0000 7f1daccea700 1 -- 172.21.15.28:0/3070355544 wait complete. 2023-11-28T11:09:42.193 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:42.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.320+0000 7f68fb791700 1 Processor -- start 2023-11-28T11:09:42.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.321+0000 7f68fb791700 1 -- start start 2023-11-28T11:09:42.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.321+0000 7f68fb791700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f68f415fc00 0x7f68f406eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.321+0000 7f68fb791700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f406f940 con 0x7f68f40b2d90 2023-11-28T11:09:42.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.321+0000 7f68fb791700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68f406fc20 con 0x7f68f406f3e0 2023-11-28T11:09:42.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.321+0000 7f68fb791700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68f40cb3f0 con 0x7f68f415fc00 2023-11-28T11:09:42.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.321+0000 7f68f9d2e700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f68f40b2d90 0x7f68f40cbbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36472/0 (socket says 172.21.15.28:36472) 2023-11-28T11:09:42.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.321+0000 7f68f9d2e700 1 -- 172.21.15.28:0/3994690743 learned_addr learned my addr 172.21.15.28:0/3994690743 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:42.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68f952d700 1 --2- 172.21.15.28:0/3994690743 >> v2:172.21.15.59:3300/0 conn(0x7f68f415fc00 0x7f68f406eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68ebfff700 1 -- 172.21.15.28:0/3994690743 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4039222589 0 0) 0x7f68f406f940 con 0x7f68f40b2d90 2023-11-28T11:09:42.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68ebfff700 1 -- 172.21.15.28:0/3994690743 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68e00035b0 con 0x7f68f40b2d90 2023-11-28T11:09:42.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68ebfff700 1 -- 172.21.15.28:0/3994690743 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1006589139 0 0) 0x7f68e00035b0 con 0x7f68f40b2d90 2023-11-28T11:09:42.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68ebfff700 1 -- 172.21.15.28:0/3994690743 >> v2:172.21.15.59:3300/0 conn(0x7f68f415fc00 msgr2=0x7f68f406eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:42.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68ebfff700 1 --2- 172.21.15.28:0/3994690743 >> v2:172.21.15.59:3300/0 conn(0x7f68f415fc00 0x7f68f406eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68ebfff700 1 -- 172.21.15.28:0/3994690743 >> v1:172.21.15.59:6789/0 conn(0x7f68f406f3e0 legacy=0x7f68f40b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68ebfff700 1 -- 172.21.15.28:0/3994690743 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f40cb6d0 con 0x7f68f40b2d90 2023-11-28T11:09:42.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68ebfff700 1 -- 172.21.15.28:0/3994690743 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f68ec0019f0 con 0x7f68f40b2d90 2023-11-28T11:09:42.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68ebfff700 1 -- 172.21.15.28:0/3994690743 >> v1:172.21.15.28:6789/0 conn(0x7f68f40b2d90 legacy=0x7f68f40cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68ebfff700 1 --2- 172.21.15.28:0/3994690743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68e0003a00 0x7f68e0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68ebfff700 1 --2- 172.21.15.28:0/3994690743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68e0004850 0x7f68e0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68ebfff700 1 --2- 172.21.15.28:0/3994690743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0005200 0x7f68e000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68ebfff700 1 -- 172.21.15.28:0/3994690743 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40cb3f0 con 0x7f68e0005200 2023-11-28T11:09:42.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68ebfff700 1 -- 172.21.15.28:0/3994690743 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e000c080 con 0x7f68e0003a00 2023-11-28T11:09:42.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68ebfff700 1 -- 172.21.15.28:0/3994690743 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e000c240 con 0x7f68e0004850 2023-11-28T11:09:42.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68f9d2e700 1 --2- 172.21.15.28:0/3994690743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0005200 0x7f68e000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68f952d700 1 --2- 172.21.15.28:0/3994690743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68e0003a00 0x7f68e0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68f8d2c700 1 --2- 172.21.15.28:0/3994690743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68e0004850 0x7f68e0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68f9d2e700 1 -- 172.21.15.28:0/3994690743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68e0004850 msgr2=0x7f68e0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:42.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68f9d2e700 1 --2- 172.21.15.28:0/3994690743 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68e0004850 0x7f68e0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68f9d2e700 1 -- 172.21.15.28:0/3994690743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68e0003a00 msgr2=0x7f68e0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:42.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.322+0000 7f68f9d2e700 1 --2- 172.21.15.28:0/3994690743 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68e0003a00 0x7f68e0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.323+0000 7f68f9d2e700 1 -- 172.21.15.28:0/3994690743 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e00035b0 con 0x7f68e0005200 2023-11-28T11:09:42.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.323+0000 7f68f9d2e700 1 -- 172.21.15.28:0/3994690743 reap_dead start 2023-11-28T11:09:42.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.323+0000 7f68f9d2e700 1 --2- 172.21.15.28:0/3994690743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0005200 0x7f68e000ba10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00a820 tx=0x7f68e0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e41f909e184785b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.323+0000 7f68ebfff700 1 -- 172.21.15.28:0/3994690743 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec00a2f0 con 0x7f68e0005200 2023-11-28T11:09:42.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.323+0000 7f68ebfff700 1 -- 172.21.15.28:0/3994690743 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68ec012600 con 0x7f68e0005200 2023-11-28T11:09:42.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.323+0000 7f68ebfff700 1 -- 172.21.15.28:0/3994690743 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68ec0129b0 con 0x7f68e0005200 2023-11-28T11:09:42.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.323+0000 7f68fb791700 1 -- 172.21.15.28:0/3994690743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0005200 msgr2=0x7f68e000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.323+0000 7f68fb791700 1 --2- 172.21.15.28:0/3994690743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0005200 0x7f68e000ba10 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f68ec00a820 tx=0x7f68e0005880 comp rx=0 tx=0).stop 2023-11-28T11:09:42.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.323+0000 7f68fb791700 1 -- 172.21.15.28:0/3994690743 shutdown_connections 2023-11-28T11:09:42.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.323+0000 7f68fb791700 1 --2- 172.21.15.28:0/3994690743 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0005200 0x7f68e000ba10 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.323+0000 7f68fb791700 1 -- 172.21.15.28:0/3994690743 >> 172.21.15.28:0/3994690743 conn(0x7f68f40cd680 msgr2=0x7f68f40cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:42.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.323+0000 7f68fb791700 1 -- 172.21.15.28:0/3994690743 shutdown_connections 2023-11-28T11:09:42.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.323+0000 7f68fb791700 1 -- 172.21.15.28:0/3994690743 wait complete. 2023-11-28T11:09:42.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.325+0000 7f68fb791700 1 Processor -- start 2023-11-28T11:09:42.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.325+0000 7f68fb791700 1 -- start start 2023-11-28T11:09:42.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.325+0000 7f68fb791700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0005200 0x7f68f40b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.325+0000 7f68fb791700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f40c9730 0x7f68f40c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.325+0000 7f68fb791700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68f4147720 0x7f68f4147b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.325+0000 7f68fb791700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40c4c00 con 0x7f68e0005200 2023-11-28T11:09:42.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.325+0000 7f68fb791700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40ca180 con 0x7f68f40c9730 2023-11-28T11:09:42.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.325+0000 7f68fb791700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68f40ca460 con 0x7f68f4147720 2023-11-28T11:09:42.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68f952d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0005200 0x7f68f40b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68f952d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0005200 0x7f68f40b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48422/0 (socket says 172.21.15.28:48422) 2023-11-28T11:09:42.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68f952d700 1 -- 172.21.15.28:0/1957668403 learned_addr learned my addr 172.21.15.28:0/1957668403 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:42.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68f8d2c700 1 --2- 172.21.15.28:0/1957668403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f40c9730 0x7f68f40c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68f9d2e700 1 --2- 172.21.15.28:0/1957668403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68f4147720 0x7f68f4147b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68f952d700 1 -- 172.21.15.28:0/1957668403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68f4147720 msgr2=0x7f68f4147b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68f952d700 1 --2- 172.21.15.28:0/1957668403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68f4147720 0x7f68f4147b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68f952d700 1 -- 172.21.15.28:0/1957668403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f40c9730 msgr2=0x7f68f40c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68f952d700 1 --2- 172.21.15.28:0/1957668403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f40c9730 0x7f68f40c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68f952d700 1 -- 172.21.15.28:0/1957668403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f40ce340 con 0x7f68e0005200 2023-11-28T11:09:42.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68f952d700 1 --2- 172.21.15.28:0/1957668403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0005200 0x7f68f40b3ca0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f68e40119a0 tx=0x7f68e4011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68e9ffb700 1 -- 172.21.15.28:0/1957668403 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e400f780 con 0x7f68e0005200 2023-11-28T11:09:42.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68e9ffb700 1 -- 172.21.15.28:0/1957668403 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68e400f8e0 con 0x7f68e0005200 2023-11-28T11:09:42.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68fb791700 1 -- 172.21.15.28:0/1957668403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68f415ffd0 con 0x7f68e0005200 2023-11-28T11:09:42.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68e9ffb700 1 -- 172.21.15.28:0/1957668403 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68e400fc70 con 0x7f68e0005200 2023-11-28T11:09:42.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.326+0000 7f68fb791700 1 -- 172.21.15.28:0/1957668403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68f406eea0 con 0x7f68e0005200 2023-11-28T11:09:42.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.328+0000 7f68fb791700 1 -- 172.21.15.28:0/1957668403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68f40739f0 con 0x7f68e0005200 2023-11-28T11:09:42.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.328+0000 7f68e9ffb700 1 -- 172.21.15.28:0/1957668403 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f68e4014440 con 0x7f68e0005200 2023-11-28T11:09:42.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.328+0000 7f68e9ffb700 1 --2- 172.21.15.28:0/1957668403 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68d80682f0 0x7f68d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.328+0000 7f68e9ffb700 1 -- 172.21.15.28:0/1957668403 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f68e40866b0 con 0x7f68e0005200 2023-11-28T11:09:42.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.328+0000 7f68f8d2c700 1 --2- 172.21.15.28:0/1957668403 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68d80682f0 0x7f68d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.329+0000 7f68f8d2c700 1 --2- 172.21.15.28:0/1957668403 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68d80682f0 0x7f68d806a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f68f40cacd0 tx=0x7f68f0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.333+0000 7f68e9ffb700 1 -- 172.21.15.28:0/1957668403 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f68e4052b80 con 0x7f68e0005200 2023-11-28T11:09:42.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.523+0000 7f68fb791700 1 -- 172.21.15.28:0/1957668403 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f68f40b6b80 con 0x7f68e0005200 2023-11-28T11:09:42.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.524+0000 7f68e9ffb700 1 -- 172.21.15.28:0/1957668403 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f68e40561a0 con 0x7f68e0005200 2023-11-28T11:09:42.524 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:42.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.526+0000 7f68fb791700 1 -- 172.21.15.28:0/1957668403 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68d80682f0 msgr2=0x7f68d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.526+0000 7f68fb791700 1 --2- 172.21.15.28:0/1957668403 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68d80682f0 0x7f68d806a7a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f68f40cacd0 tx=0x7f68f0009040 comp rx=0 tx=0).stop 2023-11-28T11:09:42.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.526+0000 7f68fb791700 1 -- 172.21.15.28:0/1957668403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0005200 msgr2=0x7f68f40b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.526+0000 7f68fb791700 1 --2- 172.21.15.28:0/1957668403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0005200 0x7f68f40b3ca0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f68e40119a0 tx=0x7f68e4011cb0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.526+0000 7f68fb791700 1 -- 172.21.15.28:0/1957668403 shutdown_connections 2023-11-28T11:09:42.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.526+0000 7f68fb791700 1 --2- 172.21.15.28:0/1957668403 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68d80682f0 0x7f68d806a7a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.526+0000 7f68fb791700 1 --2- 172.21.15.28:0/1957668403 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f68e0005200 0x7f68f40b3ca0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.526+0000 7f68fb791700 1 --2- 172.21.15.28:0/1957668403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68f4147720 0x7f68f4147b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.526+0000 7f68fb791700 1 --2- 172.21.15.28:0/1957668403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68f40c9730 0x7f68f40c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.526+0000 7f68fb791700 1 -- 172.21.15.28:0/1957668403 >> 172.21.15.28:0/1957668403 conn(0x7f68f40cd680 msgr2=0x7f68f40b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:42.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.526+0000 7f68fb791700 1 -- 172.21.15.28:0/1957668403 shutdown_connections 2023-11-28T11:09:42.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.526+0000 7f68fb791700 1 -- 172.21.15.28:0/1957668403 wait complete. 2023-11-28T11:09:42.539 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:42.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.663+0000 7f152fe95700 1 Processor -- start 2023-11-28T11:09:42.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.663+0000 7f152fe95700 1 -- start start 2023-11-28T11:09:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.664+0000 7f152fe95700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1528159590 0x7f152806e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.664+0000 7f152fe95700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15280ce890 0x7f15280ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.664+0000 7f152fe95700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f152815d120 con 0x7f152815db70 2023-11-28T11:09:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.664+0000 7f152fe95700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15280cf250 con 0x7f1528159590 2023-11-28T11:09:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.664+0000 7f152fe95700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f152815d6e0 con 0x7f15280ce890 2023-11-28T11:09:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.664+0000 7f152dc31700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1528159590 0x7f152806e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.664+0000 7f152dc31700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1528159590 0x7f152806e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48430/0 (socket says 172.21.15.28:48430) 2023-11-28T11:09:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.664+0000 7f152dc31700 1 -- 172.21.15.28:0/1041564441 learned_addr learned my addr 172.21.15.28:0/1041564441 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:42.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.664+0000 7f152e432700 1 --2- 172.21.15.28:0/1041564441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15280ce890 0x7f15280ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.664+0000 7f152dc31700 1 -- 172.21.15.28:0/1041564441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15280ce890 msgr2=0x7f15280ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.664+0000 7f152dc31700 1 --2- 172.21.15.28:0/1041564441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15280ce890 0x7f15280ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.664+0000 7f152dc31700 1 -- 172.21.15.28:0/1041564441 >> v1:172.21.15.59:6789/0 conn(0x7f152815db70 legacy=0x7f15280ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.664+0000 7f152dc31700 1 -- 172.21.15.28:0/1041564441 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15280aef60 con 0x7f1528159590 2023-11-28T11:09:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.664+0000 7f152dc31700 1 --2- 172.21.15.28:0/1041564441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1528159590 0x7f152806e400 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f15240098f0 tx=0x7f1524009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b163310408c3fa4d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.664+0000 7f152cc2f700 1 -- 172.21.15.28:0/1041564441 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1524004230 con 0x7f1528159590 2023-11-28T11:09:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.665+0000 7f152cc2f700 1 -- 172.21.15.28:0/1041564441 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1524004390 con 0x7f1528159590 2023-11-28T11:09:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.665+0000 7f152cc2f700 1 -- 172.21.15.28:0/1041564441 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15240036a0 con 0x7f1528159590 2023-11-28T11:09:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.665+0000 7f152fe95700 1 -- 172.21.15.28:0/1041564441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1528159590 msgr2=0x7f152806e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.665+0000 7f152fe95700 1 --2- 172.21.15.28:0/1041564441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1528159590 0x7f152806e400 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f15240098f0 tx=0x7f1524009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.665+0000 7f152fe95700 1 -- 172.21.15.28:0/1041564441 shutdown_connections 2023-11-28T11:09:42.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.665+0000 7f152fe95700 1 --2- 172.21.15.28:0/1041564441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1528159590 0x7f152806e400 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.665+0000 7f152fe95700 1 --2- 172.21.15.28:0/1041564441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15280ce890 0x7f15280ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.665+0000 7f152fe95700 1 -- 172.21.15.28:0/1041564441 >> 172.21.15.28:0/1041564441 conn(0x7f15280b1650 msgr2=0x7f15280b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:42.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.665+0000 7f152fe95700 1 -- 172.21.15.28:0/1041564441 shutdown_connections 2023-11-28T11:09:42.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.665+0000 7f152fe95700 1 -- 172.21.15.28:0/1041564441 wait complete. 2023-11-28T11:09:42.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.667+0000 7f152fe95700 1 Processor -- start 2023-11-28T11:09:42.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.667+0000 7f152fe95700 1 -- start start 2023-11-28T11:09:42.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152fe95700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15280ce890 0x7f1528150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152fe95700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1528159590 0x7f152806aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152fe95700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f152815db70 0x7f1528156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152fe95700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1528150ad0 con 0x7f1528159590 2023-11-28T11:09:42.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152fe95700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1528156920 con 0x7f152815db70 2023-11-28T11:09:42.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152fe95700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1528155d10 con 0x7f15280ce890 2023-11-28T11:09:42.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152d430700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1528159590 0x7f152806aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152d430700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1528159590 0x7f152806aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48438/0 (socket says 172.21.15.28:48438) 2023-11-28T11:09:42.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152d430700 1 -- 172.21.15.28:0/3210622802 learned_addr learned my addr 172.21.15.28:0/3210622802 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:42.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152dc31700 1 --2- 172.21.15.28:0/3210622802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15280ce890 0x7f1528150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152d430700 1 -- 172.21.15.28:0/3210622802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15280ce890 msgr2=0x7f1528150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152e432700 1 --2- 172.21.15.28:0/3210622802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f152815db70 0x7f1528156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152d430700 1 --2- 172.21.15.28:0/3210622802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15280ce890 0x7f1528150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152d430700 1 -- 172.21.15.28:0/3210622802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f152815db70 msgr2=0x7f1528156db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152d430700 1 --2- 172.21.15.28:0/3210622802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f152815db70 0x7f1528156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152d430700 1 -- 172.21.15.28:0/3210622802 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1524009550 con 0x7f1528159590 2023-11-28T11:09:42.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152dc31700 1 --2- 172.21.15.28:0/3210622802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15280ce890 0x7f1528150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:42.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152d430700 1 --2- 172.21.15.28:0/3210622802 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1528159590 0x7f152806aa70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f151800b4a0 tx=0x7f151800b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f151e7fc700 1 -- 172.21.15.28:0/3210622802 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518011830 con 0x7f1528159590 2023-11-28T11:09:42.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f151e7fc700 1 -- 172.21.15.28:0/3210622802 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1518011990 con 0x7f1528159590 2023-11-28T11:09:42.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f151e7fc700 1 -- 172.21.15.28:0/3210622802 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1518011d40 con 0x7f1528159590 2023-11-28T11:09:42.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.668+0000 7f152fe95700 1 -- 172.21.15.28:0/3210622802 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f152815d370 con 0x7f1528159590 2023-11-28T11:09:42.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.669+0000 7f152fe95700 1 -- 172.21.15.28:0/3210622802 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15280af240 con 0x7f1528159590 2023-11-28T11:09:42.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.669+0000 7f152fe95700 1 -- 172.21.15.28:0/3210622802 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f152815e010 con 0x7f1528159590 2023-11-28T11:09:42.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.670+0000 7f151e7fc700 1 -- 172.21.15.28:0/3210622802 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f151800f050 con 0x7f1528159590 2023-11-28T11:09:42.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.670+0000 7f151e7fc700 1 --2- 172.21.15.28:0/3210622802 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f150c0682f0 0x7f150c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:42.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.670+0000 7f151e7fc700 1 -- 172.21.15.28:0/3210622802 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f1518085bb0 con 0x7f1528159590 2023-11-28T11:09:42.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.670+0000 7f152dc31700 1 --2- 172.21.15.28:0/3210622802 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f150c0682f0 0x7f150c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:42.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.671+0000 7f152dc31700 1 --2- 172.21.15.28:0/3210622802 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f150c0682f0 0x7f150c06a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1524009c40 tx=0x7f1524008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:42.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.675+0000 7f151e7fc700 1 -- 172.21.15.28:0/3210622802 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f151808c080 con 0x7f1528159590 2023-11-28T11:09:42.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.882+0000 7f152fe95700 1 -- 172.21.15.28:0/3210622802 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1528156a90 con 0x7f1528159590 2023-11-28T11:09:42.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.883+0000 7f151e7fc700 1 -- 172.21.15.28:0/3210622802 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f15180520e0 con 0x7f1528159590 2023-11-28T11:09:42.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.885+0000 7f152fe95700 1 -- 172.21.15.28:0/3210622802 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f150c0682f0 msgr2=0x7f150c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.885+0000 7f152fe95700 1 --2- 172.21.15.28:0/3210622802 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f150c0682f0 0x7f150c06a7a0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1524009c40 tx=0x7f1524008030 comp rx=0 tx=0).stop 2023-11-28T11:09:42.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.885+0000 7f152fe95700 1 -- 172.21.15.28:0/3210622802 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1528159590 msgr2=0x7f152806aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:42.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.885+0000 7f152fe95700 1 --2- 172.21.15.28:0/3210622802 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1528159590 0x7f152806aa70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f151800b4a0 tx=0x7f151800b860 comp rx=0 tx=0).stop 2023-11-28T11:09:42.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.885+0000 7f152fe95700 1 -- 172.21.15.28:0/3210622802 shutdown_connections 2023-11-28T11:09:42.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.885+0000 7f152fe95700 1 --2- 172.21.15.28:0/3210622802 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f150c0682f0 0x7f150c06a7a0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.885+0000 7f152fe95700 1 --2- 172.21.15.28:0/3210622802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f152815db70 0x7f1528156db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.885+0000 7f152fe95700 1 --2- 172.21.15.28:0/3210622802 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15280ce890 0x7f1528150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.885+0000 7f152fe95700 1 --2- 172.21.15.28:0/3210622802 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1528159590 0x7f152806aa70 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:42.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.885+0000 7f152fe95700 1 -- 172.21.15.28:0/3210622802 >> 172.21.15.28:0/3210622802 conn(0x7f15280b1650 msgr2=0x7f15280caf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:42.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.885+0000 7f152fe95700 1 -- 172.21.15.28:0/3210622802 shutdown_connections 2023-11-28T11:09:42.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:42.885+0000 7f152fe95700 1 -- 172.21.15.28:0/3210622802 wait complete. 2023-11-28T11:09:42.887 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2023-11-28T11:09:42.897 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2023-11-28T11:09:42.897 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:42.898 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-11-28T11:09:42.949 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:42.949 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:09:42.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:42.949 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T11:09:42.983 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:09:42.983 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:43.001 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:09:43.001 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:43.059 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:43.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.224+0000 7f70128f9700 1 Processor -- start 2023-11-28T11:09:43.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.224+0000 7f70128f9700 1 -- start start 2023-11-28T11:09:43.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.224+0000 7f70128f9700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f700c06aa00 0x7f700c0cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.225+0000 7f70128f9700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c06ae80 con 0x7f700c06a560 2023-11-28T11:09:43.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.225+0000 7f70128f9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c0cdec0 con 0x7f700c15d1e0 2023-11-28T11:09:43.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.225+0000 7f70128f9700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700c0c9cd0 con 0x7f700c06aa00 2023-11-28T11:09:43.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.225+0000 7f7010e96700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f700c06aa00 0x7f700c0cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.225+0000 7f700b7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f700c06a560 0x7f700c0cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36504/0 (socket says 172.21.15.28:36504) 2023-11-28T11:09:43.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.225+0000 7f700b7fe700 1 -- 172.21.15.28:0/2348292993 learned_addr learned my addr 172.21.15.28:0/2348292993 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:43.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.225+0000 7f7010e96700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f700c06aa00 0x7f700c0cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48448/0 (socket says 172.21.15.28:48448) 2023-11-28T11:09:43.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.225+0000 7f7010e96700 1 -- 172.21.15.28:0/2348292993 >> v1:172.21.15.59:6789/0 conn(0x7f700c15d1e0 legacy=0x7f700c15d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:43.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.225+0000 7f7010e96700 1 -- 172.21.15.28:0/2348292993 >> v1:172.21.15.28:6789/0 conn(0x7f700c06a560 legacy=0x7f700c0cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.225+0000 7f700affd700 1 -- 172.21.15.28:0/2348292993 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 220124182 0 0) 0x7f700c06ae80 con 0x7f700c06a560 2023-11-28T11:09:43.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.225+0000 7f7010e96700 1 -- 172.21.15.28:0/2348292993 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c0c9fb0 con 0x7f700c06aa00 2023-11-28T11:09:43.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.225+0000 7f7010e96700 1 --2- 172.21.15.28:0/2348292993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f700c06aa00 0x7f700c0cd5c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7004009d90 tx=0x7f700400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e889fb2ad36d1a72 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.225+0000 7f700affd700 1 -- 172.21.15.28:0/2348292993 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700400e070 con 0x7f700c06aa00 2023-11-28T11:09:43.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.225+0000 7f700affd700 1 -- 172.21.15.28:0/2348292993 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700400bb90 con 0x7f700c06aa00 2023-11-28T11:09:43.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.225+0000 7f700affd700 1 -- 172.21.15.28:0/2348292993 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7004003a20 con 0x7f700c06aa00 2023-11-28T11:09:43.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.226+0000 7f70128f9700 1 -- 172.21.15.28:0/2348292993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f700c06aa00 msgr2=0x7f700c0cd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.226+0000 7f70128f9700 1 --2- 172.21.15.28:0/2348292993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f700c06aa00 0x7f700c0cd5c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f7004009d90 tx=0x7f700400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:43.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.226+0000 7f70128f9700 1 -- 172.21.15.28:0/2348292993 shutdown_connections 2023-11-28T11:09:43.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.226+0000 7f70128f9700 1 --2- 172.21.15.28:0/2348292993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f700c06aa00 0x7f700c0cd5c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.226+0000 7f70128f9700 1 -- 172.21.15.28:0/2348292993 >> 172.21.15.28:0/2348292993 conn(0x7f700c0b0dd0 msgr2=0x7f700c0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:43.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.226+0000 7f70128f9700 1 -- 172.21.15.28:0/2348292993 shutdown_connections 2023-11-28T11:09:43.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.226+0000 7f70128f9700 1 -- 172.21.15.28:0/2348292993 wait complete. 2023-11-28T11:09:43.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.227+0000 7f70128f9700 1 Processor -- start 2023-11-28T11:09:43.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.227+0000 7f70128f9700 1 -- start start 2023-11-28T11:09:43.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.228+0000 7f70128f9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f700c06a560 0x7f700c14f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.228+0000 7f70128f9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f700c06aa00 0x7f700c157ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.228+0000 7f70128f9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f700c15d1e0 0x7f700c154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.228+0000 7f70128f9700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0c6e00 con 0x7f700c06aa00 2023-11-28T11:09:43.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.228+0000 7f70128f9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0c70e0 con 0x7f700c06a560 2023-11-28T11:09:43.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.228+0000 7f70128f9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0c73c0 con 0x7f700c15d1e0 2023-11-28T11:09:43.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.228+0000 7f700b7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f700c06aa00 0x7f700c157ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.228+0000 7f700b7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f700c06aa00 0x7f700c157ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48452/0 (socket says 172.21.15.28:48452) 2023-11-28T11:09:43.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.228+0000 7f700b7fe700 1 -- 172.21.15.28:0/633221313 learned_addr learned my addr 172.21.15.28:0/633221313 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:43.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.228+0000 7f700bfff700 1 --2- 172.21.15.28:0/633221313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f700c06a560 0x7f700c14f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.228+0000 7f7010e96700 1 --2- 172.21.15.28:0/633221313 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f700c15d1e0 0x7f700c154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.229+0000 7f700b7fe700 1 -- 172.21.15.28:0/633221313 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f700c15d1e0 msgr2=0x7f700c154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.229+0000 7f700b7fe700 1 --2- 172.21.15.28:0/633221313 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f700c15d1e0 0x7f700c154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.229+0000 7f700b7fe700 1 -- 172.21.15.28:0/633221313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f700c06a560 msgr2=0x7f700c14f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.229+0000 7f700b7fe700 1 --2- 172.21.15.28:0/633221313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f700c06a560 0x7f700c14f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.229+0000 7f700b7fe700 1 -- 172.21.15.28:0/633221313 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7004009a70 con 0x7f700c06aa00 2023-11-28T11:09:43.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.229+0000 7f700bfff700 1 --2- 172.21.15.28:0/633221313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f700c06a560 0x7f700c14f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:43.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.229+0000 7f700b7fe700 1 --2- 172.21.15.28:0/633221313 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f700c06aa00 0x7f700c157ce0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f700000c550 tx=0x7f700000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.229+0000 7f7008ff9700 1 -- 172.21.15.28:0/633221313 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000004c80 con 0x7f700c06aa00 2023-11-28T11:09:43.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.229+0000 7f7008ff9700 1 -- 172.21.15.28:0/633221313 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7000004e00 con 0x7f700c06aa00 2023-11-28T11:09:43.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.229+0000 7f7008ff9700 1 -- 172.21.15.28:0/633221313 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7000007310 con 0x7f700c06aa00 2023-11-28T11:09:43.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.229+0000 7f70128f9700 1 -- 172.21.15.28:0/633221313 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c0ca570 con 0x7f700c06aa00 2023-11-28T11:09:43.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.229+0000 7f70128f9700 1 -- 172.21.15.28:0/633221313 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c0ca290 con 0x7f700c06aa00 2023-11-28T11:09:43.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.230+0000 7f70128f9700 1 -- 172.21.15.28:0/633221313 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700c0ca850 con 0x7f700c06aa00 2023-11-28T11:09:43.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.230+0000 7f7008ff9700 1 -- 172.21.15.28:0/633221313 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f700001a020 con 0x7f700c06aa00 2023-11-28T11:09:43.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.231+0000 7f7008ff9700 1 --2- 172.21.15.28:0/633221313 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6ff80682f0 0x7f6ff806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.231+0000 7f7008ff9700 1 -- 172.21.15.28:0/633221313 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f7000085b70 con 0x7f700c06aa00 2023-11-28T11:09:43.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.231+0000 7f700bfff700 1 --2- 172.21.15.28:0/633221313 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6ff80682f0 0x7f6ff806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.231+0000 7f700bfff700 1 --2- 172.21.15.28:0/633221313 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6ff80682f0 0x7f6ff806a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0017d0 tx=0x7f6ffc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.236 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.235+0000 7f7008ff9700 1 -- 172.21.15.28:0/633221313 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7000052150 con 0x7f700c06aa00 2023-11-28T11:09:43.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.429+0000 7f70128f9700 1 -- 172.21.15.28:0/633221313 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f700c0c7da0 con 0x7f700c06aa00 2023-11-28T11:09:43.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.430+0000 7f7008ff9700 1 -- 172.21.15.28:0/633221313 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19737 (secure 0 0 0) 0x7f7000055770 con 0x7f700c06aa00 2023-11-28T11:09:43.434 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:43.435 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:09:34.614188+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:43.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.432+0000 7f70128f9700 1 -- 172.21.15.28:0/633221313 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6ff80682f0 msgr2=0x7f6ff806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.432+0000 7f70128f9700 1 --2- 172.21.15.28:0/633221313 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6ff80682f0 0x7f6ff806a7a0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f6ffc0017d0 tx=0x7f6ffc009040 comp rx=0 tx=0).stop 2023-11-28T11:09:43.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.432+0000 7f70128f9700 1 -- 172.21.15.28:0/633221313 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f700c06aa00 msgr2=0x7f700c157ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.432+0000 7f70128f9700 1 --2- 172.21.15.28:0/633221313 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f700c06aa00 0x7f700c157ce0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f700000c550 tx=0x7f700000c910 comp rx=0 tx=0).stop 2023-11-28T11:09:43.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.432+0000 7f70128f9700 1 -- 172.21.15.28:0/633221313 shutdown_connections 2023-11-28T11:09:43.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.432+0000 7f70128f9700 1 --2- 172.21.15.28:0/633221313 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6ff80682f0 0x7f6ff806a7a0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.432+0000 7f70128f9700 1 --2- 172.21.15.28:0/633221313 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f700c15d1e0 0x7f700c154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.432+0000 7f70128f9700 1 --2- 172.21.15.28:0/633221313 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f700c06a560 0x7f700c14f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.432+0000 7f70128f9700 1 --2- 172.21.15.28:0/633221313 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f700c06aa00 0x7f700c157ce0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.432+0000 7f70128f9700 1 -- 172.21.15.28:0/633221313 >> 172.21.15.28:0/633221313 conn(0x7f700c0b0dd0 msgr2=0x7f700c0cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:43.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.432+0000 7f70128f9700 1 -- 172.21.15.28:0/633221313 shutdown_connections 2023-11-28T11:09:43.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.432+0000 7f70128f9700 1 -- 172.21.15.28:0/633221313 wait complete. 2023-11-28T11:09:43.445 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:43.446 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:43.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.574+0000 7f2321ed9700 1 Processor -- start 2023-11-28T11:09:43.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.574+0000 7f2321ed9700 1 -- start start 2023-11-28T11:09:43.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.574+0000 7f2321ed9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f231c0b1630 0x7f231c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.574+0000 7f2321ed9700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f231c06afc0 0x7f231c06b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.574+0000 7f2321ed9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f231c0ce550 con 0x7f231c0b1a00 2023-11-28T11:09:43.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.574+0000 7f2321ed9700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f231c0ce830 con 0x7f231c06afc0 2023-11-28T11:09:43.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.574+0000 7f2321ed9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231c0ceb10 con 0x7f231c0b1630 2023-11-28T11:09:43.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.575+0000 7f231bfff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f231c06afc0 0x7f231c06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.575+0000 7f231bfff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f231c06afc0 0x7f231c06b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48462/0 (socket says 172.21.15.28:48462) 2023-11-28T11:09:43.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.575+0000 7f231bfff700 1 -- 172.21.15.28:0/1502795420 learned_addr learned my addr 172.21.15.28:0/1502795420 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:43.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.575+0000 7f231b7fe700 1 --2- 172.21.15.28:0/1502795420 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f231c0b1630 0x7f231c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.575+0000 7f231bfff700 1 -- 172.21.15.28:0/1502795420 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f231c0b1630 msgr2=0x7f231c0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:43.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.575+0000 7f231bfff700 1 --2- 172.21.15.28:0/1502795420 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f231c0b1630 0x7f231c0cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.575+0000 7f231bfff700 1 -- 172.21.15.28:0/1502795420 >> v1:172.21.15.59:6789/0 conn(0x7f231c0b1a00 legacy=0x7f231c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:43.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.575+0000 7f231bfff700 1 -- 172.21.15.28:0/1502795420 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f231c0cedf0 con 0x7f231c06afc0 2023-11-28T11:09:43.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.575+0000 7f231bfff700 1 --2- 172.21.15.28:0/1502795420 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f231c06afc0 0x7f231c06b440 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2314009e50 tx=0x7f231400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84dcb98fdac465ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.575+0000 7f231a7fc700 1 -- 172.21.15.28:0/1502795420 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231400e070 con 0x7f231c06afc0 2023-11-28T11:09:43.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.575+0000 7f231a7fc700 1 -- 172.21.15.28:0/1502795420 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2314003a20 con 0x7f231c06afc0 2023-11-28T11:09:43.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.575+0000 7f231a7fc700 1 -- 172.21.15.28:0/1502795420 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314003dd0 con 0x7f231c06afc0 2023-11-28T11:09:43.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.576+0000 7f2321ed9700 1 -- 172.21.15.28:0/1502795420 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f231c06afc0 msgr2=0x7f231c06b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.576+0000 7f2321ed9700 1 --2- 172.21.15.28:0/1502795420 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f231c06afc0 0x7f231c06b440 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f2314009e50 tx=0x7f231400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:43.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.576+0000 7f2321ed9700 1 -- 172.21.15.28:0/1502795420 shutdown_connections 2023-11-28T11:09:43.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.576+0000 7f2321ed9700 1 --2- 172.21.15.28:0/1502795420 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f231c06afc0 0x7f231c06b440 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.576+0000 7f2321ed9700 1 --2- 172.21.15.28:0/1502795420 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f231c0b1630 0x7f231c0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.576+0000 7f2321ed9700 1 -- 172.21.15.28:0/1502795420 >> 172.21.15.28:0/1502795420 conn(0x7f231c0cc630 msgr2=0x7f231c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:43.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.576+0000 7f2321ed9700 1 -- 172.21.15.28:0/1502795420 shutdown_connections 2023-11-28T11:09:43.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.576+0000 7f2321ed9700 1 -- 172.21.15.28:0/1502795420 wait complete. 2023-11-28T11:09:43.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.577+0000 7f2321ed9700 1 Processor -- start 2023-11-28T11:09:43.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.577+0000 7f2321ed9700 1 -- start start 2023-11-28T11:09:43.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f2321ed9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f231c06afc0 0x7f231c157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f2321ed9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c0b1630 0x7f231c157c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f2321ed9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f231c0b1a00 0x7f231c0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f2321ed9700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c157690 con 0x7f231c0b1a00 2023-11-28T11:09:43.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f2321ed9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f231c0b3aa0 con 0x7f231c0b1630 2023-11-28T11:09:43.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f2321ed9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f231c0b3d80 con 0x7f231c06afc0 2023-11-28T11:09:43.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f231bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f231c0b1a00 0x7f231c0c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f231bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f231c0b1a00 0x7f231c0c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48468/0 (socket says 172.21.15.28:48468) 2023-11-28T11:09:43.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f231bfff700 1 -- 172.21.15.28:0/1550209143 learned_addr learned my addr 172.21.15.28:0/1550209143 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:43.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f231b7fe700 1 --2- 172.21.15.28:0/1550209143 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f231c06afc0 0x7f231c157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f231affd700 1 --2- 172.21.15.28:0/1550209143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c0b1630 0x7f231c157c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f231bfff700 1 -- 172.21.15.28:0/1550209143 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f231c06afc0 msgr2=0x7f231c157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f231bfff700 1 --2- 172.21.15.28:0/1550209143 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f231c06afc0 0x7f231c157150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f231bfff700 1 -- 172.21.15.28:0/1550209143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c0b1630 msgr2=0x7f231c157c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f231bfff700 1 --2- 172.21.15.28:0/1550209143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c0b1630 0x7f231c157c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f231bfff700 1 -- 172.21.15.28:0/1550209143 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2314009b30 con 0x7f231c0b1a00 2023-11-28T11:09:43.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.578+0000 7f231affd700 1 --2- 172.21.15.28:0/1550209143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c0b1630 0x7f231c157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:43.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.579+0000 7f231bfff700 1 --2- 172.21.15.28:0/1550209143 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f231c0b1a00 0x7f231c0c6c80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f231c066a20 tx=0x7f2314004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.579+0000 7f2303fff700 1 -- 172.21.15.28:0/1550209143 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314004240 con 0x7f231c0b1a00 2023-11-28T11:09:43.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.579+0000 7f2303fff700 1 -- 172.21.15.28:0/1550209143 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23140043a0 con 0x7f231c0b1a00 2023-11-28T11:09:43.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.579+0000 7f2321ed9700 1 -- 172.21.15.28:0/1550209143 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f231c0b1e70 con 0x7f231c0b1a00 2023-11-28T11:09:43.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.579+0000 7f2303fff700 1 -- 172.21.15.28:0/1550209143 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2314004750 con 0x7f231c0b1a00 2023-11-28T11:09:43.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.579+0000 7f2321ed9700 1 -- 172.21.15.28:0/1550209143 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f231c0cf0d0 con 0x7f231c0b1a00 2023-11-28T11:09:43.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.580+0000 7f2321ed9700 1 -- 172.21.15.28:0/1550209143 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f231c0cb5b0 con 0x7f231c0b1a00 2023-11-28T11:09:43.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.580+0000 7f2303fff700 1 -- 172.21.15.28:0/1550209143 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2314012430 con 0x7f231c0b1a00 2023-11-28T11:09:43.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.581+0000 7f2303fff700 1 --2- 172.21.15.28:0/1550209143 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f23080682f0 0x7f230806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.581+0000 7f2303fff700 1 -- 172.21.15.28:0/1550209143 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2314087520 con 0x7f231c0b1a00 2023-11-28T11:09:43.588 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.581+0000 7f231b7fe700 1 --2- 172.21.15.28:0/1550209143 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f23080682f0 0x7f230806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.588 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.581+0000 7f231b7fe700 1 --2- 172.21.15.28:0/1550209143 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f23080682f0 0x7f230806a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f230c008850 tx=0x7f230c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.588 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.585+0000 7f2303fff700 1 -- 172.21.15.28:0/1550209143 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2314053880 con 0x7f231c0b1a00 2023-11-28T11:09:43.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.777+0000 7f2321ed9700 1 -- 172.21.15.28:0/1550209143 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f231c0742e0 con 0x7f231c0b1a00 2023-11-28T11:09:43.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.777+0000 7f2303fff700 1 -- 172.21.15.28:0/1550209143 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2314020020 con 0x7f231c0b1a00 2023-11-28T11:09:43.778 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:09:43.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.779+0000 7f2321ed9700 1 -- 172.21.15.28:0/1550209143 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f23080682f0 msgr2=0x7f230806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.779+0000 7f2321ed9700 1 --2- 172.21.15.28:0/1550209143 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f23080682f0 0x7f230806a7a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f230c008850 tx=0x7f230c008040 comp rx=0 tx=0).stop 2023-11-28T11:09:43.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.779+0000 7f2321ed9700 1 -- 172.21.15.28:0/1550209143 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f231c0b1a00 msgr2=0x7f231c0c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.779+0000 7f2321ed9700 1 --2- 172.21.15.28:0/1550209143 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f231c0b1a00 0x7f231c0c6c80 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f231c066a20 tx=0x7f2314004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.779+0000 7f2321ed9700 1 -- 172.21.15.28:0/1550209143 shutdown_connections 2023-11-28T11:09:43.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.779+0000 7f2321ed9700 1 --2- 172.21.15.28:0/1550209143 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f23080682f0 0x7f230806a7a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.779+0000 7f2321ed9700 1 --2- 172.21.15.28:0/1550209143 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f231c0b1a00 0x7f231c0c6c80 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.779+0000 7f2321ed9700 1 --2- 172.21.15.28:0/1550209143 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f231c06afc0 0x7f231c157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.779+0000 7f2321ed9700 1 --2- 172.21.15.28:0/1550209143 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f231c0b1630 0x7f231c157c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.779+0000 7f2321ed9700 1 -- 172.21.15.28:0/1550209143 >> 172.21.15.28:0/1550209143 conn(0x7f231c0cc630 msgr2=0x7f231c0caea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:43.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.779+0000 7f2321ed9700 1 -- 172.21.15.28:0/1550209143 shutdown_connections 2023-11-28T11:09:43.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.779+0000 7f2321ed9700 1 -- 172.21.15.28:0/1550209143 wait complete. 2023-11-28T11:09:43.792 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.919+0000 7f3a48c68700 1 Processor -- start 2023-11-28T11:09:43.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.919+0000 7f3a48c68700 1 -- start start 2023-11-28T11:09:43.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.920+0000 7f3a48c68700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3a440b1630 0x7f3a440cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.920+0000 7f3a48c68700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3a4406afc0 0x7f3a4406b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.920+0000 7f3a48c68700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a440ce550 con 0x7f3a440b1a00 2023-11-28T11:09:43.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.920+0000 7f3a48c68700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a440ce830 con 0x7f3a4406afc0 2023-11-28T11:09:43.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.920+0000 7f3a48c68700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a440ceb10 con 0x7f3a440b1630 2023-11-28T11:09:43.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.920+0000 7f3a42d9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3a4406afc0 0x7f3a4406b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.920+0000 7f3a42d9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3a4406afc0 0x7f3a4406b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48476/0 (socket says 172.21.15.28:48476) 2023-11-28T11:09:43.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.920+0000 7f3a42d9d700 1 -- 172.21.15.28:0/289917778 learned_addr learned my addr 172.21.15.28:0/289917778 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:43.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.920+0000 7f3a4259c700 1 --2- 172.21.15.28:0/289917778 >> v2:172.21.15.59:3300/0 conn(0x7f3a440b1630 0x7f3a440cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.920+0000 7f3a42d9d700 1 -- 172.21.15.28:0/289917778 >> v2:172.21.15.59:3300/0 conn(0x7f3a440b1630 msgr2=0x7f3a440cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:43.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.920+0000 7f3a42d9d700 1 --2- 172.21.15.28:0/289917778 >> v2:172.21.15.59:3300/0 conn(0x7f3a440b1630 0x7f3a440cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.920+0000 7f3a42d9d700 1 -- 172.21.15.28:0/289917778 >> v1:172.21.15.59:6789/0 conn(0x7f3a440b1a00 legacy=0x7f3a440caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.920+0000 7f3a42d9d700 1 -- 172.21.15.28:0/289917778 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a440cedf0 con 0x7f3a4406afc0 2023-11-28T11:09:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.921+0000 7f3a42d9d700 1 --2- 172.21.15.28:0/289917778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a4406afc0 0x7f3a4406b420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009e50 tx=0x7f3a3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d74dfbdd24953fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.921+0000 7f3a4159a700 1 -- 172.21.15.28:0/289917778 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c00e070 con 0x7f3a4406afc0 2023-11-28T11:09:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.921+0000 7f3a4159a700 1 -- 172.21.15.28:0/289917778 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3c003a20 con 0x7f3a4406afc0 2023-11-28T11:09:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.921+0000 7f3a4159a700 1 -- 172.21.15.28:0/289917778 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3c003dd0 con 0x7f3a4406afc0 2023-11-28T11:09:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.921+0000 7f3a48c68700 1 -- 172.21.15.28:0/289917778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a4406afc0 msgr2=0x7f3a4406b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.921+0000 7f3a48c68700 1 --2- 172.21.15.28:0/289917778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a4406afc0 0x7f3a4406b420 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3a3c009e50 tx=0x7f3a3c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:43.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.921+0000 7f3a48c68700 1 -- 172.21.15.28:0/289917778 shutdown_connections 2023-11-28T11:09:43.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.921+0000 7f3a48c68700 1 --2- 172.21.15.28:0/289917778 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a4406afc0 0x7f3a4406b420 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.921+0000 7f3a48c68700 1 --2- 172.21.15.28:0/289917778 >> v2:172.21.15.59:3300/0 conn(0x7f3a440b1630 0x7f3a440cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.921+0000 7f3a48c68700 1 -- 172.21.15.28:0/289917778 >> 172.21.15.28:0/289917778 conn(0x7f3a440cc630 msgr2=0x7f3a440cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:43.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.921+0000 7f3a48c68700 1 -- 172.21.15.28:0/289917778 shutdown_connections 2023-11-28T11:09:43.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.921+0000 7f3a48c68700 1 -- 172.21.15.28:0/289917778 wait complete. 2023-11-28T11:09:43.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.923+0000 7f3a48c68700 1 Processor -- start 2023-11-28T11:09:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.923+0000 7f3a48c68700 1 -- start start 2023-11-28T11:09:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.923+0000 7f3a48c68700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a4406afc0 0x7f3a440c83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.923+0000 7f3a48c68700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a440b1630 0x7f3a440ad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.923+0000 7f3a48c68700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3a440b1a00 0x7f3a440ad5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.923+0000 7f3a48c68700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a440c8c90 con 0x7f3a4406afc0 2023-11-28T11:09:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.923+0000 7f3a48c68700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a440c8f70 con 0x7f3a440b1630 2023-11-28T11:09:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.923+0000 7f3a48c68700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a440c9250 con 0x7f3a440b1a00 2023-11-28T11:09:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.923+0000 7f3a4259c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a4406afc0 0x7f3a440c83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a4259c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a4406afc0 0x7f3a440c83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48478/0 (socket says 172.21.15.28:48478) 2023-11-28T11:09:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a4259c700 1 -- 172.21.15.28:0/2695758168 learned_addr learned my addr 172.21.15.28:0/2695758168 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:43.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a41d9b700 1 --2- 172.21.15.28:0/2695758168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a440b1630 0x7f3a440ad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a42d9d700 1 --2- 172.21.15.28:0/2695758168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3a440b1a00 0x7f3a440ad5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a4259c700 1 -- 172.21.15.28:0/2695758168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3a440b1a00 msgr2=0x7f3a440ad5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a4259c700 1 --2- 172.21.15.28:0/2695758168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3a440b1a00 0x7f3a440ad5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a4259c700 1 -- 172.21.15.28:0/2695758168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a440b1630 msgr2=0x7f3a440ad080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a4259c700 1 --2- 172.21.15.28:0/2695758168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a440b1630 0x7f3a440ad080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a4259c700 1 -- 172.21.15.28:0/2695758168 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c009b30 con 0x7f3a4406afc0 2023-11-28T11:09:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a42d9d700 1 --2- 172.21.15.28:0/2695758168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3a440b1a00 0x7f3a440ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a4259c700 1 --2- 172.21.15.28:0/2695758168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a4406afc0 0x7f3a440c83b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800c7f0 tx=0x7f3a3800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a32ffd700 1 -- 172.21.15.28:0/2695758168 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a38004220 con 0x7f3a4406afc0 2023-11-28T11:09:43.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a32ffd700 1 -- 172.21.15.28:0/2695758168 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a38004380 con 0x7f3a4406afc0 2023-11-28T11:09:43.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a48c68700 1 -- 172.21.15.28:0/2695758168 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a440b1ea0 con 0x7f3a4406afc0 2023-11-28T11:09:43.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a32ffd700 1 -- 172.21.15.28:0/2695758168 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a380036a0 con 0x7f3a4406afc0 2023-11-28T11:09:43.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.924+0000 7f3a48c68700 1 -- 172.21.15.28:0/2695758168 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a440cf0d0 con 0x7f3a4406afc0 2023-11-28T11:09:43.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.925+0000 7f3a48c68700 1 -- 172.21.15.28:0/2695758168 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a440cb5b0 con 0x7f3a4406afc0 2023-11-28T11:09:43.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.926+0000 7f3a32ffd700 1 -- 172.21.15.28:0/2695758168 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3a38020070 con 0x7f3a4406afc0 2023-11-28T11:09:43.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.926+0000 7f3a32ffd700 1 --2- 172.21.15.28:0/2695758168 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3a240682f0 0x7f3a2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:43.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.926+0000 7f3a32ffd700 1 -- 172.21.15.28:0/2695758168 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3a38085a90 con 0x7f3a4406afc0 2023-11-28T11:09:43.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.926+0000 7f3a41d9b700 1 --2- 172.21.15.28:0/2695758168 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3a240682f0 0x7f3a2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:43.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.926+0000 7f3a41d9b700 1 --2- 172.21.15.28:0/2695758168 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3a240682f0 0x7f3a2406a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a441f5cb0 tx=0x7f3a340090e0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:43.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:43.930+0000 7f3a32ffd700 1 -- 172.21.15.28:0/2695758168 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3a38051f40 con 0x7f3a4406afc0 2023-11-28T11:09:44.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.123+0000 7f3a48c68700 1 -- 172.21.15.28:0/2695758168 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3a440739f0 con 0x7f3a4406afc0 2023-11-28T11:09:44.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.124+0000 7f3a32ffd700 1 -- 172.21.15.28:0/2695758168 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3a38055560 con 0x7f3a4406afc0 2023-11-28T11:09:44.125 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:09:44.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.126+0000 7f3a48c68700 1 -- 172.21.15.28:0/2695758168 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3a240682f0 msgr2=0x7f3a2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.126+0000 7f3a48c68700 1 --2- 172.21.15.28:0/2695758168 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3a240682f0 0x7f3a2406a7a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f3a441f5cb0 tx=0x7f3a340090e0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.126+0000 7f3a48c68700 1 -- 172.21.15.28:0/2695758168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a4406afc0 msgr2=0x7f3a440c83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.126+0000 7f3a48c68700 1 --2- 172.21.15.28:0/2695758168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a4406afc0 0x7f3a440c83b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3a3800c7f0 tx=0x7f3a3800cb00 comp rx=0 tx=0).stop 2023-11-28T11:09:44.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.126+0000 7f3a48c68700 1 -- 172.21.15.28:0/2695758168 shutdown_connections 2023-11-28T11:09:44.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.126+0000 7f3a48c68700 1 --2- 172.21.15.28:0/2695758168 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3a240682f0 0x7f3a2406a7a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.126+0000 7f3a48c68700 1 --2- 172.21.15.28:0/2695758168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3a4406afc0 0x7f3a440c83b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.126+0000 7f3a48c68700 1 --2- 172.21.15.28:0/2695758168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3a440b1a00 0x7f3a440ad5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.126+0000 7f3a48c68700 1 --2- 172.21.15.28:0/2695758168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a440b1630 0x7f3a440ad080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.126+0000 7f3a48c68700 1 -- 172.21.15.28:0/2695758168 >> 172.21.15.28:0/2695758168 conn(0x7f3a440cc630 msgr2=0x7f3a440ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.126+0000 7f3a48c68700 1 -- 172.21.15.28:0/2695758168 shutdown_connections 2023-11-28T11:09:44.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.126+0000 7f3a48c68700 1 -- 172.21.15.28:0/2695758168 wait complete. 2023-11-28T11:09:44.139 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.266+0000 7f868976c700 1 Processor -- start 2023-11-28T11:09:44.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.266+0000 7f868976c700 1 -- start start 2023-11-28T11:09:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.267+0000 7f868976c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f86840cb630 0x7f86840b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.267+0000 7f868976c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86840b2bc0 0x7f86840ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.267+0000 7f868976c700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86840cd2a0 con 0x7f86840b62d0 2023-11-28T11:09:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.267+0000 7f868976c700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86840b3040 con 0x7f86840cb630 2023-11-28T11:09:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.267+0000 7f868976c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86840d0400 con 0x7f86840b2bc0 2023-11-28T11:09:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.267+0000 7f86827fc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f86840cb630 0x7f86840b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.267+0000 7f8682ffd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f86840b62d0 0x7f868406f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36544/0 (socket says 172.21.15.28:36544) 2023-11-28T11:09:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.267+0000 7f8682ffd700 1 -- 172.21.15.28:0/279904163 learned_addr learned my addr 172.21.15.28:0/279904163 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.267+0000 7f86837fe700 1 --2- 172.21.15.28:0/279904163 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86840b2bc0 0x7f86840ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.268+0000 7f86827fc700 1 -- 172.21.15.28:0/279904163 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86840b2bc0 msgr2=0x7f86840ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.268+0000 7f86827fc700 1 --2- 172.21.15.28:0/279904163 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86840b2bc0 0x7f86840ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.268+0000 7f86827fc700 1 -- 172.21.15.28:0/279904163 >> v1:172.21.15.28:6789/0 conn(0x7f86840b62d0 legacy=0x7f868406f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.268+0000 7f86827fc700 1 -- 172.21.15.28:0/279904163 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86840d06e0 con 0x7f86840cb630 2023-11-28T11:09:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.268+0000 7f86827fc700 1 --2- 172.21.15.28:0/279904163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86840cb630 0x7f86840b2680 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8674009840 tx=0x7f8674009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b7d1062118d6b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.268+0000 7f8681ffb700 1 -- 172.21.15.28:0/279904163 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674011830 con 0x7f86840cb630 2023-11-28T11:09:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.268+0000 7f8681ffb700 1 -- 172.21.15.28:0/279904163 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8674011990 con 0x7f86840cb630 2023-11-28T11:09:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.268+0000 7f8681ffb700 1 -- 172.21.15.28:0/279904163 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8674011d20 con 0x7f86840cb630 2023-11-28T11:09:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.268+0000 7f868976c700 1 -- 172.21.15.28:0/279904163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86840cb630 msgr2=0x7f86840b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.268+0000 7f868976c700 1 --2- 172.21.15.28:0/279904163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86840cb630 0x7f86840b2680 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8674009840 tx=0x7f8674009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.268+0000 7f868976c700 1 -- 172.21.15.28:0/279904163 shutdown_connections 2023-11-28T11:09:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.268+0000 7f868976c700 1 --2- 172.21.15.28:0/279904163 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86840b2bc0 0x7f86840ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.268+0000 7f868976c700 1 --2- 172.21.15.28:0/279904163 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86840cb630 0x7f86840b2680 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.268+0000 7f868976c700 1 -- 172.21.15.28:0/279904163 >> 172.21.15.28:0/279904163 conn(0x7f86840cd680 msgr2=0x7f86840cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.269+0000 7f868976c700 1 -- 172.21.15.28:0/279904163 shutdown_connections 2023-11-28T11:09:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.269+0000 7f868976c700 1 -- 172.21.15.28:0/279904163 wait complete. 2023-11-28T11:09:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.270+0000 7f868976c700 1 Processor -- start 2023-11-28T11:09:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.270+0000 7f868976c700 1 -- start start 2023-11-28T11:09:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f868976c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86840b2bc0 0x7f86841ef2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f868976c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86840b62d0 0x7f86840ceb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f868976c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86840cb630 0x7f86840cf090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f868976c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840c9e60 con 0x7f86840b2bc0 2023-11-28T11:09:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f868976c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86840c8dc0 con 0x7f86840b62d0 2023-11-28T11:09:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f868976c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86840c90a0 con 0x7f86840cb630 2023-11-28T11:09:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f8682ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86840b2bc0 0x7f86841ef2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f8682ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86840b2bc0 0x7f86841ef2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48492/0 (socket says 172.21.15.28:48492) 2023-11-28T11:09:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f8682ffd700 1 -- 172.21.15.28:0/1479340482 learned_addr learned my addr 172.21.15.28:0/1479340482 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f86827fc700 1 --2- 172.21.15.28:0/1479340482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86840b62d0 0x7f86840ceb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f86837fe700 1 --2- 172.21.15.28:0/1479340482 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86840cb630 0x7f86840cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f8682ffd700 1 -- 172.21.15.28:0/1479340482 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86840cb630 msgr2=0x7f86840cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f8682ffd700 1 --2- 172.21.15.28:0/1479340482 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86840cb630 0x7f86840cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f8682ffd700 1 -- 172.21.15.28:0/1479340482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86840b62d0 msgr2=0x7f86840ceb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f8682ffd700 1 --2- 172.21.15.28:0/1479340482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86840b62d0 0x7f86840ceb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f8682ffd700 1 -- 172.21.15.28:0/1479340482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86740094a0 con 0x7f86840b2bc0 2023-11-28T11:09:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.271+0000 7f8682ffd700 1 --2- 172.21.15.28:0/1479340482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86840b2bc0 0x7f86841ef2a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f867800c520 tx=0x7f867800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.273+0000 7f866b7fe700 1 -- 172.21.15.28:0/1479340482 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8678005d70 con 0x7f86840b2bc0 2023-11-28T11:09:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.273+0000 7f868976c700 1 -- 172.21.15.28:0/1479340482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86840b66a0 con 0x7f86840b2bc0 2023-11-28T11:09:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.273+0000 7f868976c700 1 -- 172.21.15.28:0/1479340482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86840cffb0 con 0x7f86840b2bc0 2023-11-28T11:09:44.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.273+0000 7f866b7fe700 1 -- 172.21.15.28:0/1479340482 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8678001280 con 0x7f86840b2bc0 2023-11-28T11:09:44.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.273+0000 7f866b7fe700 1 -- 172.21.15.28:0/1479340482 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86780072d0 con 0x7f86840b2bc0 2023-11-28T11:09:44.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.274+0000 7f868976c700 1 -- 172.21.15.28:0/1479340482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86640052f0 con 0x7f86840b2bc0 2023-11-28T11:09:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.274+0000 7f866b7fe700 1 -- 172.21.15.28:0/1479340482 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8678019030 con 0x7f86840b2bc0 2023-11-28T11:09:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.274+0000 7f866b7fe700 1 --2- 172.21.15.28:0/1479340482 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86700682f0 0x7f867006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.275+0000 7f866b7fe700 1 -- 172.21.15.28:0/1479340482 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f86780850f0 con 0x7f86840b2bc0 2023-11-28T11:09:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.275+0000 7f86827fc700 1 --2- 172.21.15.28:0/1479340482 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86700682f0 0x7f867006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.275+0000 7f86827fc700 1 --2- 172.21.15.28:0/1479340482 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86700682f0 0x7f867006a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f86740090b0 tx=0x7f8674008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.279+0000 7f866b7fe700 1 -- 172.21.15.28:0/1479340482 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f867804b5e0 con 0x7f86840b2bc0 2023-11-28T11:09:44.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.472+0000 7f868976c700 1 -- 172.21.15.28:0/1479340482 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8664005b20 con 0x7f86840b2bc0 2023-11-28T11:09:44.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.472+0000 7f866b7fe700 1 -- 172.21.15.28:0/1479340482 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8678051650 con 0x7f86840b2bc0 2023-11-28T11:09:44.473 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:44.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.474+0000 7f868976c700 1 -- 172.21.15.28:0/1479340482 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86700682f0 msgr2=0x7f867006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.474+0000 7f868976c700 1 --2- 172.21.15.28:0/1479340482 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86700682f0 0x7f867006a7a0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f86740090b0 tx=0x7f8674008030 comp rx=0 tx=0).stop 2023-11-28T11:09:44.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.474+0000 7f868976c700 1 -- 172.21.15.28:0/1479340482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86840b2bc0 msgr2=0x7f86841ef2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.474+0000 7f868976c700 1 --2- 172.21.15.28:0/1479340482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86840b2bc0 0x7f86841ef2a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f867800c520 tx=0x7f867800c830 comp rx=0 tx=0).stop 2023-11-28T11:09:44.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.474+0000 7f868976c700 1 -- 172.21.15.28:0/1479340482 shutdown_connections 2023-11-28T11:09:44.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.474+0000 7f868976c700 1 --2- 172.21.15.28:0/1479340482 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86700682f0 0x7f867006a7a0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.474+0000 7f868976c700 1 --2- 172.21.15.28:0/1479340482 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86840b2bc0 0x7f86841ef2a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.474+0000 7f868976c700 1 --2- 172.21.15.28:0/1479340482 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86840cb630 0x7f86840cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.474+0000 7f868976c700 1 --2- 172.21.15.28:0/1479340482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86840b62d0 0x7f86840ceb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.474+0000 7f868976c700 1 -- 172.21.15.28:0/1479340482 >> 172.21.15.28:0/1479340482 conn(0x7f86840cd680 msgr2=0x7f868406f720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:44.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.474+0000 7f868976c700 1 -- 172.21.15.28:0/1479340482 shutdown_connections 2023-11-28T11:09:44.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.474+0000 7f868976c700 1 -- 172.21.15.28:0/1479340482 wait complete. 2023-11-28T11:09:44.487 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:44.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.614+0000 7fba089cf700 1 Processor -- start 2023-11-28T11:09:44.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.614+0000 7fba089cf700 1 -- start start 2023-11-28T11:09:44.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.615+0000 7fba089cf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba0415fc00 0x7fba0406eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.615+0000 7fba089cf700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fba0406f3e0 0x7fba040b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.615+0000 7fba089cf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fba040b2bc0 0x7fba041602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.615+0000 7fba089cf700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba040b3040 con 0x7fba0406f3e0 2023-11-28T11:09:44.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.615+0000 7fba089cf700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba0406f940 con 0x7fba040b2bc0 2023-11-28T11:09:44.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.615+0000 7fba089cf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba0406fc20 con 0x7fba0415fc00 2023-11-28T11:09:44.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.615+0000 7fba01d9b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fba0406f3e0 0x7fba040b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.616+0000 7fba01d9b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fba0406f3e0 0x7fba040b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48500/0 (socket says 172.21.15.28:48500) 2023-11-28T11:09:44.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.616+0000 7fba01d9b700 1 -- 172.21.15.28:0/517299775 learned_addr learned my addr 172.21.15.28:0/517299775 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:44.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.616+0000 7fba02d9d700 1 --2- 172.21.15.28:0/517299775 >> v2:172.21.15.59:3300/0 conn(0x7fba040b2bc0 0x7fba041602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.616+0000 7fba0259c700 1 --2- 172.21.15.28:0/517299775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba0415fc00 0x7fba0406eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.616+0000 7fba01d9b700 1 -- 172.21.15.28:0/517299775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba0415fc00 msgr2=0x7fba0406eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:44.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.616+0000 7fba01d9b700 1 --2- 172.21.15.28:0/517299775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba0415fc00 0x7fba0406eea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.616+0000 7fba01d9b700 1 -- 172.21.15.28:0/517299775 >> v2:172.21.15.59:3300/0 conn(0x7fba040b2bc0 msgr2=0x7fba041602e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.616+0000 7fba01d9b700 1 --2- 172.21.15.28:0/517299775 >> v2:172.21.15.59:3300/0 conn(0x7fba040b2bc0 0x7fba041602e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.616+0000 7fba01d9b700 1 -- 172.21.15.28:0/517299775 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba040cb3f0 con 0x7fba0406f3e0 2023-11-28T11:09:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.616+0000 7fba0259c700 1 --2- 172.21.15.28:0/517299775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba0415fc00 0x7fba0406eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:44.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.616+0000 7fba01d9b700 1 --2- 172.21.15.28:0/517299775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba0406f3e0 0x7fba040b2680 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fba040c7c60 tx=0x7fb9f4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e28d57f7d347b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.616+0000 7fba0159a700 1 -- 172.21.15.28:0/517299775 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4011830 con 0x7fba0406f3e0 2023-11-28T11:09:44.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.616+0000 7fba0159a700 1 -- 172.21.15.28:0/517299775 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f4011990 con 0x7fba0406f3e0 2023-11-28T11:09:44.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.616+0000 7fba0159a700 1 -- 172.21.15.28:0/517299775 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f4011d20 con 0x7fba0406f3e0 2023-11-28T11:09:44.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.617+0000 7fba089cf700 1 -- 172.21.15.28:0/517299775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba0406f3e0 msgr2=0x7fba040b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.617+0000 7fba089cf700 1 --2- 172.21.15.28:0/517299775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba0406f3e0 0x7fba040b2680 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fba040c7c60 tx=0x7fb9f4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.617+0000 7fba089cf700 1 -- 172.21.15.28:0/517299775 shutdown_connections 2023-11-28T11:09:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.617+0000 7fba089cf700 1 --2- 172.21.15.28:0/517299775 >> v2:172.21.15.59:3300/0 conn(0x7fba040b2bc0 0x7fba041602e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.617+0000 7fba089cf700 1 --2- 172.21.15.28:0/517299775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba0415fc00 0x7fba0406eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.617+0000 7fba089cf700 1 --2- 172.21.15.28:0/517299775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba0406f3e0 0x7fba040b2680 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.617+0000 7fba089cf700 1 -- 172.21.15.28:0/517299775 >> 172.21.15.28:0/517299775 conn(0x7fba040cd680 msgr2=0x7fba040cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:44.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.617+0000 7fba089cf700 1 -- 172.21.15.28:0/517299775 shutdown_connections 2023-11-28T11:09:44.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.617+0000 7fba089cf700 1 -- 172.21.15.28:0/517299775 wait complete. 2023-11-28T11:09:44.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.618+0000 7fba089cf700 1 Processor -- start 2023-11-28T11:09:44.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.618+0000 7fba089cf700 1 -- start start 2023-11-28T11:09:44.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba089cf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0406f3e0 0x7fba04153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba089cf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba040b2bc0 0x7fba040c4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba089cf700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba0415fc00 0x7fba040cc2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba089cf700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba04153f90 con 0x7fba0415fc00 2023-11-28T11:09:44.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba089cf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba040c6c70 con 0x7fba0406f3e0 2023-11-28T11:09:44.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba089cf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba040c6f10 con 0x7fba040b2bc0 2023-11-28T11:09:44.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba02d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba0415fc00 0x7fba040cc2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba02d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba0415fc00 0x7fba040cc2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48510/0 (socket says 172.21.15.28:48510) 2023-11-28T11:09:44.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba02d9d700 1 -- 172.21.15.28:0/105609306 learned_addr learned my addr 172.21.15.28:0/105609306 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:44.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba0259c700 1 --2- 172.21.15.28:0/105609306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0406f3e0 0x7fba04153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba01d9b700 1 --2- 172.21.15.28:0/105609306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba040b2bc0 0x7fba040c4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba02d9d700 1 -- 172.21.15.28:0/105609306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba040b2bc0 msgr2=0x7fba040c4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba02d9d700 1 --2- 172.21.15.28:0/105609306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba040b2bc0 0x7fba040c4de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba02d9d700 1 -- 172.21.15.28:0/105609306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0406f3e0 msgr2=0x7fba04153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba02d9d700 1 --2- 172.21.15.28:0/105609306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0406f3e0 0x7fba04153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.619+0000 7fba02d9d700 1 -- 172.21.15.28:0/105609306 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f4009550 con 0x7fba0415fc00 2023-11-28T11:09:44.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.620+0000 7fba02d9d700 1 --2- 172.21.15.28:0/105609306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba0415fc00 0x7fba040cc2b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00eed0 tx=0x7fb9fc00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.620+0000 7fb9f2ffd700 1 -- 172.21.15.28:0/105609306 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc010040 con 0x7fba0415fc00 2023-11-28T11:09:44.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.620+0000 7fb9f2ffd700 1 -- 172.21.15.28:0/105609306 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9fc00bc80 con 0x7fba0415fc00 2023-11-28T11:09:44.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.620+0000 7fba089cf700 1 -- 172.21.15.28:0/105609306 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba0415ffd0 con 0x7fba0415fc00 2023-11-28T11:09:44.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.620+0000 7fb9f2ffd700 1 -- 172.21.15.28:0/105609306 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9fc009e40 con 0x7fba0415fc00 2023-11-28T11:09:44.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.620+0000 7fba089cf700 1 -- 172.21.15.28:0/105609306 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba040cb6d0 con 0x7fba0415fc00 2023-11-28T11:09:44.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.621+0000 7fba089cf700 1 -- 172.21.15.28:0/105609306 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba040ce340 con 0x7fba0415fc00 2023-11-28T11:09:44.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.621+0000 7fb9f2ffd700 1 -- 172.21.15.28:0/105609306 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb9fc014400 con 0x7fba0415fc00 2023-11-28T11:09:44.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.622+0000 7fb9f2ffd700 1 --2- 172.21.15.28:0/105609306 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb9e40682c0 0x7fb9e406a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.622+0000 7fb9f2ffd700 1 -- 172.21.15.28:0/105609306 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb9fc086490 con 0x7fba0415fc00 2023-11-28T11:09:44.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.622+0000 7fba0259c700 1 --2- 172.21.15.28:0/105609306 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb9e40682c0 0x7fb9e406a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.622+0000 7fba0259c700 1 --2- 172.21.15.28:0/105609306 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb9e40682c0 0x7fb9e406a770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8006f10 tx=0x7fb9f8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.626+0000 7fb9f2ffd700 1 -- 172.21.15.28:0/105609306 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb9fc052960 con 0x7fba0415fc00 2023-11-28T11:09:44.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.818+0000 7fba089cf700 1 -- 172.21.15.28:0/105609306 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fba041f2590 con 0x7fba0415fc00 2023-11-28T11:09:44.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.819+0000 7fb9f2ffd700 1 -- 172.21.15.28:0/105609306 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb9fc0180a0 con 0x7fba0415fc00 2023-11-28T11:09:44.819 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:09:44.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.821+0000 7fba089cf700 1 -- 172.21.15.28:0/105609306 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb9e40682c0 msgr2=0x7fb9e406a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.821+0000 7fba089cf700 1 --2- 172.21.15.28:0/105609306 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb9e40682c0 0x7fb9e406a770 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8006f10 tx=0x7fb9f8008040 comp rx=0 tx=0).stop 2023-11-28T11:09:44.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.821+0000 7fba089cf700 1 -- 172.21.15.28:0/105609306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba0415fc00 msgr2=0x7fba040cc2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.821+0000 7fba089cf700 1 --2- 172.21.15.28:0/105609306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba0415fc00 0x7fba040cc2b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fb9fc00eed0 tx=0x7fb9fc00b390 comp rx=0 tx=0).stop 2023-11-28T11:09:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.821+0000 7fba089cf700 1 -- 172.21.15.28:0/105609306 shutdown_connections 2023-11-28T11:09:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.821+0000 7fba089cf700 1 --2- 172.21.15.28:0/105609306 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb9e40682c0 0x7fb9e406a770 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.821+0000 7fba089cf700 1 --2- 172.21.15.28:0/105609306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fba0415fc00 0x7fba040cc2b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.821+0000 7fba089cf700 1 --2- 172.21.15.28:0/105609306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba0406f3e0 0x7fba04153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.821+0000 7fba089cf700 1 --2- 172.21.15.28:0/105609306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fba040b2bc0 0x7fba040c4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.821+0000 7fba089cf700 1 -- 172.21.15.28:0/105609306 >> 172.21.15.28:0/105609306 conn(0x7fba040cd680 msgr2=0x7fba040cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.821+0000 7fba089cf700 1 -- 172.21.15.28:0/105609306 shutdown_connections 2023-11-28T11:09:44.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.821+0000 7fba089cf700 1 -- 172.21.15.28:0/105609306 wait complete. 2023-11-28T11:09:44.834 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:44.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.960+0000 7f8767ace700 1 Processor -- start 2023-11-28T11:09:44.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.960+0000 7f8767ace700 1 -- start start 2023-11-28T11:09:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f8767ace700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f87600b1630 0x7f87600cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f8767ace700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87600b1a00 0x7f87600caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f8767ace700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87600ce550 con 0x7f87600cb3e0 2023-11-28T11:09:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f8767ace700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87600ce830 con 0x7f87600b1630 2023-11-28T11:09:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f8767ace700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87600ceb10 con 0x7f87600b1a00 2023-11-28T11:09:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f876586a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f87600b1630 0x7f87600cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f876586a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f87600b1630 0x7f87600cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48514/0 (socket says 172.21.15.28:48514) 2023-11-28T11:09:44.963 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f876606b700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f87600cb3e0 0x7f8760160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36574/0 (socket says 172.21.15.28:36574) 2023-11-28T11:09:44.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f876586a700 1 -- 172.21.15.28:0/2463009328 learned_addr learned my addr 172.21.15.28:0/2463009328 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:44.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f8765069700 1 --2- 172.21.15.28:0/2463009328 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87600b1a00 0x7f87600caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f876586a700 1 -- 172.21.15.28:0/2463009328 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87600b1a00 msgr2=0x7f87600caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:44.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f876586a700 1 --2- 172.21.15.28:0/2463009328 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87600b1a00 0x7f87600caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f876586a700 1 -- 172.21.15.28:0/2463009328 >> v1:172.21.15.28:6789/0 conn(0x7f87600cb3e0 legacy=0x7f8760160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f876586a700 1 -- 172.21.15.28:0/2463009328 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87600cedf0 con 0x7f87600b1630 2023-11-28T11:09:44.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f8764868700 1 -- 172.21.15.28:0/2463009328 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2690776966 0 0) 0x7f87600ce550 con 0x7f87600cb3e0 2023-11-28T11:09:44.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.961+0000 7f876586a700 1 --2- 172.21.15.28:0/2463009328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87600b1630 0x7f87600cf5f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f875c0098f0 tx=0x7f875c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b91f4a7e062ccd80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.962+0000 7f8764868700 1 -- 172.21.15.28:0/2463009328 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00b8a0 con 0x7f87600b1630 2023-11-28T11:09:44.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.962+0000 7f8764868700 1 -- 172.21.15.28:0/2463009328 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875c00ba00 con 0x7f87600b1630 2023-11-28T11:09:44.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.962+0000 7f8764868700 1 -- 172.21.15.28:0/2463009328 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875c00bd90 con 0x7f87600b1630 2023-11-28T11:09:44.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.962+0000 7f8767ace700 1 -- 172.21.15.28:0/2463009328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87600b1630 msgr2=0x7f87600cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.962+0000 7f8767ace700 1 --2- 172.21.15.28:0/2463009328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87600b1630 0x7f87600cf5f0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f875c0098f0 tx=0x7f875c009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:44.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.962+0000 7f8767ace700 1 -- 172.21.15.28:0/2463009328 shutdown_connections 2023-11-28T11:09:44.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.962+0000 7f8767ace700 1 --2- 172.21.15.28:0/2463009328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87600b1630 0x7f87600cf5f0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.962+0000 7f8767ace700 1 --2- 172.21.15.28:0/2463009328 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87600b1a00 0x7f87600caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.962+0000 7f8767ace700 1 -- 172.21.15.28:0/2463009328 >> 172.21.15.28:0/2463009328 conn(0x7f87600cc630 msgr2=0x7f87600cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:44.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.962+0000 7f8767ace700 1 -- 172.21.15.28:0/2463009328 shutdown_connections 2023-11-28T11:09:44.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.962+0000 7f8767ace700 1 -- 172.21.15.28:0/2463009328 wait complete. 2023-11-28T11:09:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.963+0000 7f8767ace700 1 Processor -- start 2023-11-28T11:09:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.963+0000 7f8767ace700 1 -- start start 2023-11-28T11:09:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.964+0000 7f8767ace700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87600b1630 0x7f87601572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.964+0000 7f8767ace700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87600b1a00 0x7f87600c5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.964+0000 7f8767ace700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87600cb3e0 0x7f87600b4c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.964+0000 7f8767ace700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87600c65c0 con 0x7f87600cb3e0 2023-11-28T11:09:44.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.964+0000 7f8767ace700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8760157800 con 0x7f87600b1a00 2023-11-28T11:09:44.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.964+0000 7f8767ace700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87600c4730 con 0x7f87600b1630 2023-11-28T11:09:44.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.964+0000 7f876606b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87600cb3e0 0x7f87600b4c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.964+0000 7f876606b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87600cb3e0 0x7f87600b4c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48522/0 (socket says 172.21.15.28:48522) 2023-11-28T11:09:44.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.964+0000 7f876606b700 1 -- 172.21.15.28:0/2784768864 learned_addr learned my addr 172.21.15.28:0/2784768864 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:44.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.964+0000 7f8765069700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87600b1a00 0x7f87600c5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.964+0000 7f876586a700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87600b1630 0x7f87601572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.964+0000 7f876606b700 1 -- 172.21.15.28:0/2784768864 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87600b1630 msgr2=0x7f87601572c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.964+0000 7f876606b700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87600b1630 0x7f87601572c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.965+0000 7f876606b700 1 -- 172.21.15.28:0/2784768864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87600b1a00 msgr2=0x7f87600c5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.965+0000 7f876606b700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87600b1a00 0x7f87600c5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.965+0000 7f876606b700 1 -- 172.21.15.28:0/2784768864 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f875c009550 con 0x7f87600cb3e0 2023-11-28T11:09:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.965+0000 7f8765069700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87600b1a00 0x7f87600c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.965+0000 7f876586a700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87600b1630 0x7f87601572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.965+0000 7f876606b700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87600cb3e0 0x7f87600b4c70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f875800cba0 tx=0x7f875800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.965+0000 7f87567fc700 1 -- 172.21.15.28:0/2784768864 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f875800ab80 con 0x7f87600cb3e0 2023-11-28T11:09:44.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.965+0000 7f87567fc700 1 -- 172.21.15.28:0/2784768864 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f875800ad00 con 0x7f87600cb3e0 2023-11-28T11:09:44.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.965+0000 7f8767ace700 1 -- 172.21.15.28:0/2784768864 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87600b1e70 con 0x7f87600cb3e0 2023-11-28T11:09:44.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.965+0000 7f87567fc700 1 -- 172.21.15.28:0/2784768864 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8758007310 con 0x7f87600cb3e0 2023-11-28T11:09:44.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.965+0000 7f8767ace700 1 -- 172.21.15.28:0/2784768864 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87600cf0d0 con 0x7f87600cb3e0 2023-11-28T11:09:44.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.966+0000 7f8767ace700 1 -- 172.21.15.28:0/2784768864 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87600ce140 con 0x7f87600cb3e0 2023-11-28T11:09:44.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.966+0000 7f87567fc700 1 -- 172.21.15.28:0/2784768864 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f875801a030 con 0x7f87600cb3e0 2023-11-28T11:09:44.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.967+0000 7f87567fc700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f87440682f0 0x7f874406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:44.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.967+0000 7f87567fc700 1 -- 172.21.15.28:0/2784768864 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8758085cf0 con 0x7f87600cb3e0 2023-11-28T11:09:44.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.967+0000 7f876586a700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f87440682f0 0x7f874406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:44.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.967+0000 7f876586a700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f87440682f0 0x7f874406a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f875c000c00 tx=0x7f875c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:44.973 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:44.971+0000 7f87567fc700 1 -- 172.21.15.28:0/2784768864 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8758052220 con 0x7f87600cb3e0 2023-11-28T11:09:45.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.164+0000 7f8767ace700 1 -- 172.21.15.28:0/2784768864 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f87600739f0 con 0x7f87600cb3e0 2023-11-28T11:09:45.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.165+0000 7f87567fc700 1 -- 172.21.15.28:0/2784768864 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f87580180a0 con 0x7f87600cb3e0 2023-11-28T11:09:45.165 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:45.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.167+0000 7f8767ace700 1 -- 172.21.15.28:0/2784768864 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f87440682f0 msgr2=0x7f874406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.167+0000 7f8767ace700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f87440682f0 0x7f874406a7a0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f875c000c00 tx=0x7f875c008030 comp rx=0 tx=0).stop 2023-11-28T11:09:45.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.167+0000 7f8767ace700 1 -- 172.21.15.28:0/2784768864 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87600cb3e0 msgr2=0x7f87600b4c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.167+0000 7f8767ace700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87600cb3e0 0x7f87600b4c70 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f875800cba0 tx=0x7f875800cf60 comp rx=0 tx=0).stop 2023-11-28T11:09:45.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.167+0000 7f8767ace700 1 -- 172.21.15.28:0/2784768864 shutdown_connections 2023-11-28T11:09:45.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.167+0000 7f8767ace700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f87440682f0 0x7f874406a7a0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.167+0000 7f8767ace700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87600cb3e0 0x7f87600b4c70 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.167+0000 7f8767ace700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87600b1630 0x7f87601572c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.167+0000 7f8767ace700 1 --2- 172.21.15.28:0/2784768864 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87600b1a00 0x7f87600c5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.167+0000 7f8767ace700 1 -- 172.21.15.28:0/2784768864 >> 172.21.15.28:0/2784768864 conn(0x7f87600cc630 msgr2=0x7f876006bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:45.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.167+0000 7f8767ace700 1 -- 172.21.15.28:0/2784768864 shutdown_connections 2023-11-28T11:09:45.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.167+0000 7f8767ace700 1 -- 172.21.15.28:0/2784768864 wait complete. 2023-11-28T11:09:45.180 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:45.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.306+0000 7ff17daa7700 1 Processor -- start 2023-11-28T11:09:45.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.306+0000 7ff17daa7700 1 -- start start 2023-11-28T11:09:45.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.307+0000 7ff17daa7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff178159590 0x7ff17806e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.307+0000 7ff17daa7700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff1780ce890 0x7ff1780ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.307+0000 7ff17daa7700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff17815d120 con 0x7ff17815db70 2023-11-28T11:09:45.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.307+0000 7ff17daa7700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1780cf250 con 0x7ff1780ce890 2023-11-28T11:09:45.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.307+0000 7ff17daa7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff17815d6e0 con 0x7ff178159590 2023-11-28T11:09:45.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.307+0000 7ff176ffd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff17815db70 0x7ff1780ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36584/0 (socket says 172.21.15.28:36584) 2023-11-28T11:09:45.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.307+0000 7ff176ffd700 1 -- 172.21.15.28:0/3753222809 learned_addr learned my addr 172.21.15.28:0/3753222809 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:45.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.307+0000 7ff1777fe700 1 --2- 172.21.15.28:0/3753222809 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff178159590 0x7ff17806e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.307+0000 7ff177fff700 1 --2- 172.21.15.28:0/3753222809 >> v2:172.21.15.59:3300/0 conn(0x7ff1780ce890 0x7ff1780ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.307+0000 7ff1767fc700 1 -- 172.21.15.28:0/3753222809 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1937546363 0 0) 0x7ff17815d120 con 0x7ff17815db70 2023-11-28T11:09:45.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.307+0000 7ff1767fc700 1 -- 172.21.15.28:0/3753222809 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1640035b0 con 0x7ff17815db70 2023-11-28T11:09:45.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.307+0000 7ff1777fe700 1 -- 172.21.15.28:0/3753222809 >> v2:172.21.15.59:3300/0 conn(0x7ff1780ce890 msgr2=0x7ff1780ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:45.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.307+0000 7ff1777fe700 1 --2- 172.21.15.28:0/3753222809 >> v2:172.21.15.59:3300/0 conn(0x7ff1780ce890 0x7ff1780ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.307+0000 7ff1777fe700 1 -- 172.21.15.28:0/3753222809 >> v1:172.21.15.28:6789/0 conn(0x7ff17815db70 legacy=0x7ff1780ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.307+0000 7ff1777fe700 1 -- 172.21.15.28:0/3753222809 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1780aef60 con 0x7ff178159590 2023-11-28T11:09:45.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.308+0000 7ff1777fe700 1 --2- 172.21.15.28:0/3753222809 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff178159590 0x7ff17806e400 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff160009840 tx=0x7ff160009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=60813b0606908b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:45.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.308+0000 7ff1767fc700 1 -- 172.21.15.28:0/3753222809 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16000f8a0 con 0x7ff178159590 2023-11-28T11:09:45.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.308+0000 7ff1767fc700 1 -- 172.21.15.28:0/3753222809 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16000fa00 con 0x7ff178159590 2023-11-28T11:09:45.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.308+0000 7ff1767fc700 1 -- 172.21.15.28:0/3753222809 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16000fd70 con 0x7ff178159590 2023-11-28T11:09:45.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.308+0000 7ff17daa7700 1 -- 172.21.15.28:0/3753222809 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff178159590 msgr2=0x7ff17806e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.308+0000 7ff17daa7700 1 --2- 172.21.15.28:0/3753222809 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff178159590 0x7ff17806e400 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff160009840 tx=0x7ff160009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:45.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.308+0000 7ff17daa7700 1 -- 172.21.15.28:0/3753222809 shutdown_connections 2023-11-28T11:09:45.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.308+0000 7ff17daa7700 1 --2- 172.21.15.28:0/3753222809 >> v2:172.21.15.59:3300/0 conn(0x7ff1780ce890 0x7ff1780ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.308+0000 7ff17daa7700 1 --2- 172.21.15.28:0/3753222809 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff178159590 0x7ff17806e400 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.308+0000 7ff17daa7700 1 -- 172.21.15.28:0/3753222809 >> 172.21.15.28:0/3753222809 conn(0x7ff1780b1650 msgr2=0x7ff1780b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:45.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.308+0000 7ff17daa7700 1 -- 172.21.15.28:0/3753222809 shutdown_connections 2023-11-28T11:09:45.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.308+0000 7ff17daa7700 1 -- 172.21.15.28:0/3753222809 wait complete. 2023-11-28T11:09:45.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.310+0000 7ff17daa7700 1 Processor -- start 2023-11-28T11:09:45.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.310+0000 7ff17daa7700 1 -- start start 2023-11-28T11:09:45.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.310+0000 7ff17daa7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1780ce890 0x7ff1780b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.310+0000 7ff17daa7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff178159590 0x7ff1780c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.310+0000 7ff17daa7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17815db70 0x7ff178157d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff17daa7700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1780b4670 con 0x7ff1780ce890 2023-11-28T11:09:45.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff17daa7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff178151800 con 0x7ff17815db70 2023-11-28T11:09:45.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff17daa7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1780c7460 con 0x7ff178159590 2023-11-28T11:09:45.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff1777fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1780ce890 0x7ff1780b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff1777fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1780ce890 0x7ff1780b4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48532/0 (socket says 172.21.15.28:48532) 2023-11-28T11:09:45.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff1777fe700 1 -- 172.21.15.28:0/1711391543 learned_addr learned my addr 172.21.15.28:0/1711391543 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:45.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff176ffd700 1 --2- 172.21.15.28:0/1711391543 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff178159590 0x7ff1780c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff177fff700 1 --2- 172.21.15.28:0/1711391543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17815db70 0x7ff178157d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff1777fe700 1 -- 172.21.15.28:0/1711391543 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff178159590 msgr2=0x7ff1780c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff1777fe700 1 --2- 172.21.15.28:0/1711391543 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff178159590 0x7ff1780c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff1777fe700 1 -- 172.21.15.28:0/1711391543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17815db70 msgr2=0x7ff178157d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff1777fe700 1 --2- 172.21.15.28:0/1711391543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17815db70 0x7ff178157d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff1777fe700 1 -- 172.21.15.28:0/1711391543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1600094a0 con 0x7ff1780ce890 2023-11-28T11:09:45.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff177fff700 1 --2- 172.21.15.28:0/1711391543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17815db70 0x7ff178157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:45.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff1777fe700 1 --2- 172.21.15.28:0/1711391543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1780ce890 0x7ff1780b4130 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff160009fd0 tx=0x7ff160004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:45.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff16ffff700 1 -- 172.21.15.28:0/1711391543 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff16001d070 con 0x7ff1780ce890 2023-11-28T11:09:45.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff16ffff700 1 -- 172.21.15.28:0/1711391543 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff16000f6a0 con 0x7ff1780ce890 2023-11-28T11:09:45.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff17daa7700 1 -- 172.21.15.28:0/1711391543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff17815d370 con 0x7ff1780ce890 2023-11-28T11:09:45.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff16ffff700 1 -- 172.21.15.28:0/1711391543 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1600038a0 con 0x7ff1780ce890 2023-11-28T11:09:45.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.311+0000 7ff17daa7700 1 -- 172.21.15.28:0/1711391543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1780af240 con 0x7ff1780ce890 2023-11-28T11:09:45.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.312+0000 7ff17daa7700 1 -- 172.21.15.28:0/1711391543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff17815dfe0 con 0x7ff1780ce890 2023-11-28T11:09:45.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.313+0000 7ff16ffff700 1 -- 172.21.15.28:0/1711391543 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff160021070 con 0x7ff1780ce890 2023-11-28T11:09:45.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.313+0000 7ff16ffff700 1 --2- 172.21.15.28:0/1711391543 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1580682f0 0x7ff15806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.313+0000 7ff16ffff700 1 -- 172.21.15.28:0/1711391543 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff160087190 con 0x7ff1780ce890 2023-11-28T11:09:45.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.313+0000 7ff176ffd700 1 --2- 172.21.15.28:0/1711391543 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1580682f0 0x7ff15806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.314+0000 7ff176ffd700 1 --2- 172.21.15.28:0/1711391543 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1580682f0 0x7ff15806a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff168007fd0 tx=0x7ff168009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:45.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.317+0000 7ff16ffff700 1 -- 172.21.15.28:0/1711391543 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff1600536c0 con 0x7ff1780ce890 2023-11-28T11:09:45.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.527+0000 7ff17daa7700 1 -- 172.21.15.28:0/1711391543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff1780675f0 con 0x7ff1780ce890 2023-11-28T11:09:45.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.527+0000 7ff16ffff700 1 -- 172.21.15.28:0/1711391543 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7ff160056ce0 con 0x7ff1780ce890 2023-11-28T11:09:45.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.529+0000 7ff17daa7700 1 -- 172.21.15.28:0/1711391543 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1580682f0 msgr2=0x7ff15806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.529+0000 7ff17daa7700 1 --2- 172.21.15.28:0/1711391543 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1580682f0 0x7ff15806a7a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff168007fd0 tx=0x7ff168009040 comp rx=0 tx=0).stop 2023-11-28T11:09:45.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.529+0000 7ff17daa7700 1 -- 172.21.15.28:0/1711391543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1780ce890 msgr2=0x7ff1780b4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.529+0000 7ff17daa7700 1 --2- 172.21.15.28:0/1711391543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1780ce890 0x7ff1780b4130 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff160009fd0 tx=0x7ff160004750 comp rx=0 tx=0).stop 2023-11-28T11:09:45.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.529+0000 7ff17daa7700 1 -- 172.21.15.28:0/1711391543 shutdown_connections 2023-11-28T11:09:45.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.529+0000 7ff17daa7700 1 --2- 172.21.15.28:0/1711391543 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1580682f0 0x7ff15806a7a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.529+0000 7ff17daa7700 1 --2- 172.21.15.28:0/1711391543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1780ce890 0x7ff1780b4130 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.529+0000 7ff17daa7700 1 --2- 172.21.15.28:0/1711391543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17815db70 0x7ff178157d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.529+0000 7ff17daa7700 1 --2- 172.21.15.28:0/1711391543 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff178159590 0x7ff1780c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.529+0000 7ff17daa7700 1 -- 172.21.15.28:0/1711391543 >> 172.21.15.28:0/1711391543 conn(0x7ff1780b1650 msgr2=0x7ff17806d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:45.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.529+0000 7ff17daa7700 1 -- 172.21.15.28:0/1711391543 shutdown_connections 2023-11-28T11:09:45.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.529+0000 7ff17daa7700 1 -- 172.21.15.28:0/1711391543 wait complete. 2023-11-28T11:09:45.531 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2023-11-28T11:09:45.542 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:09:45.542 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:45.560 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:45.725 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.724+0000 7f0bafb7e700 1 Processor -- start 2023-11-28T11:09:45.725 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.724+0000 7f0bafb7e700 1 -- start start 2023-11-28T11:09:45.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.725+0000 7f0bafb7e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ba8161090 0x7f0ba80ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.725+0000 7f0bafb7e700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba80ce340 con 0x7f0ba80caeb0 2023-11-28T11:09:45.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.725+0000 7f0bafb7e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba80cebe0 con 0x7f0ba815c990 2023-11-28T11:09:45.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.725+0000 7f0bafb7e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba80cf780 con 0x7f0ba8161090 2023-11-28T11:09:45.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.725+0000 7f0bae11b700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0ba80caeb0 0x7f0ba80cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36602/0 (socket says 172.21.15.28:36602) 2023-11-28T11:09:45.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.725+0000 7f0bae11b700 1 -- 172.21.15.28:0/3663585566 learned_addr learned my addr 172.21.15.28:0/3663585566 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:45.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bac918700 1 -- 172.21.15.28:0/3663585566 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2662020432 0 0) 0x7f0ba80ce340 con 0x7f0ba80caeb0 2023-11-28T11:09:45.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bac918700 1 -- 172.21.15.28:0/3663585566 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b940035b0 con 0x7f0ba80caeb0 2023-11-28T11:09:45.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bad119700 1 --2- 172.21.15.28:0/3663585566 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ba8161090 0x7f0ba80ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bac918700 1 -- 172.21.15.28:0/3663585566 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 471351072 0 0) 0x7f0b940035b0 con 0x7f0ba80caeb0 2023-11-28T11:09:45.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bac918700 1 -- 172.21.15.28:0/3663585566 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ba8161090 msgr2=0x7f0ba80ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:45.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bac918700 1 --2- 172.21.15.28:0/3663585566 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ba8161090 0x7f0ba80ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bac918700 1 -- 172.21.15.28:0/3663585566 >> v1:172.21.15.59:6789/0 conn(0x7f0ba815c990 legacy=0x7f0ba80cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bac918700 1 -- 172.21.15.28:0/3663585566 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba80cb3f0 con 0x7f0ba80caeb0 2023-11-28T11:09:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bac918700 1 -- 172.21.15.28:0/3663585566 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f0ba4001670 con 0x7f0ba80caeb0 2023-11-28T11:09:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bac918700 1 -- 172.21.15.28:0/3663585566 >> v1:172.21.15.28:6789/0 conn(0x7f0ba80caeb0 legacy=0x7f0ba80cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bac918700 1 --2- 172.21.15.28:0/3663585566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b94003a30 0x7f0b94003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bac918700 1 --2- 172.21.15.28:0/3663585566 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ba8161090 0x7f0b94004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bac918700 1 --2- 172.21.15.28:0/3663585566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94004d90 0x7f0b94005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bac918700 1 -- 172.21.15.28:0/3663585566 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba80cf780 con 0x7f0b94003a30 2023-11-28T11:09:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bac918700 1 -- 172.21.15.28:0/3663585566 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9400bbd0 con 0x7f0b94004d90 2023-11-28T11:09:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bac918700 1 -- 172.21.15.28:0/3663585566 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9400bd90 con 0x7f0ba8161090 2023-11-28T11:09:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bad91a700 1 --2- 172.21.15.28:0/3663585566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b94003a30 0x7f0b94003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bad119700 1 --2- 172.21.15.28:0/3663585566 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ba8161090 0x7f0b94004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.726+0000 7f0bae11b700 1 --2- 172.21.15.28:0/3663585566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94004d90 0x7f0b94005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.727+0000 7f0bad91a700 1 -- 172.21.15.28:0/3663585566 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ba8161090 msgr2=0x7f0b94004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.727+0000 7f0bad91a700 1 --2- 172.21.15.28:0/3663585566 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ba8161090 0x7f0b94004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.727+0000 7f0bad91a700 1 -- 172.21.15.28:0/3663585566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94004d90 msgr2=0x7f0b94005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:45.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.727+0000 7f0bad91a700 1 --2- 172.21.15.28:0/3663585566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94004d90 0x7f0b94005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.727+0000 7f0bad91a700 1 -- 172.21.15.28:0/3663585566 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b940035b0 con 0x7f0b94003a30 2023-11-28T11:09:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.727+0000 7f0bae11b700 1 --2- 172.21.15.28:0/3663585566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94004d90 0x7f0b94005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.727+0000 7f0bad91a700 1 --2- 172.21.15.28:0/3663585566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b94003a30 0x7f0b94003e40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000c160 tx=0x7f0b94005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18af1a55bd17f9d5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.727+0000 7f0bac918700 1 -- 172.21.15.28:0/3663585566 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0001db0 con 0x7f0b94003a30 2023-11-28T11:09:45.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.727+0000 7f0bac918700 1 -- 172.21.15.28:0/3663585566 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ba000cca0 con 0x7f0b94003a30 2023-11-28T11:09:45.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.727+0000 7f0bac918700 1 -- 172.21.15.28:0/3663585566 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0003c40 con 0x7f0b94003a30 2023-11-28T11:09:45.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.727+0000 7f0bafb7e700 1 -- 172.21.15.28:0/3663585566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b94003a30 msgr2=0x7f0b94003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.727+0000 7f0bafb7e700 1 --2- 172.21.15.28:0/3663585566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b94003a30 0x7f0b94003e40 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f0ba000c160 tx=0x7f0b94005880 comp rx=0 tx=0).stop 2023-11-28T11:09:45.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.727+0000 7f0bae11b700 1 -- 172.21.15.28:0/3663585566 reap_dead start 2023-11-28T11:09:45.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.727+0000 7f0bafb7e700 1 -- 172.21.15.28:0/3663585566 shutdown_connections 2023-11-28T11:09:45.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.727+0000 7f0bafb7e700 1 -- 172.21.15.28:0/3663585566 >> 172.21.15.28:0/3663585566 conn(0x7f0ba80b1650 msgr2=0x7f0ba80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:45.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.728+0000 7f0bafb7e700 1 -- 172.21.15.28:0/3663585566 shutdown_connections 2023-11-28T11:09:45.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.728+0000 7f0bafb7e700 1 -- 172.21.15.28:0/3663585566 wait complete. 2023-11-28T11:09:45.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.729+0000 7f0bafb7e700 1 Processor -- start 2023-11-28T11:09:45.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.729+0000 7f0bafb7e700 1 -- start start 2023-11-28T11:09:45.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.729+0000 7f0bafb7e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94003a30 0x7f0ba80c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.729+0000 7f0bafb7e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ba80c3350 0x7f0ba81f2870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.729+0000 7f0bafb7e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ba815b710 0x7f0ba815bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.729+0000 7f0bafb7e700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba81f2450 con 0x7f0ba80c3350 2023-11-28T11:09:45.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bafb7e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba81f20e0 con 0x7f0b94003a30 2023-11-28T11:09:45.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bafb7e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba80c3b30 con 0x7f0ba815b710 2023-11-28T11:09:45.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bad119700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ba80c3350 0x7f0ba81f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bad119700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ba80c3350 0x7f0ba81f2870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48554/0 (socket says 172.21.15.28:48554) 2023-11-28T11:09:45.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bad119700 1 -- 172.21.15.28:0/3644850807 learned_addr learned my addr 172.21.15.28:0/3644850807 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:45.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bae11b700 1 --2- 172.21.15.28:0/3644850807 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ba815b710 0x7f0ba815bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bad91a700 1 --2- 172.21.15.28:0/3644850807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94003a30 0x7f0ba80c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bad119700 1 -- 172.21.15.28:0/3644850807 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ba815b710 msgr2=0x7f0ba815bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bad119700 1 --2- 172.21.15.28:0/3644850807 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ba815b710 0x7f0ba815bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bad119700 1 -- 172.21.15.28:0/3644850807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94003a30 msgr2=0x7f0ba80c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bad119700 1 --2- 172.21.15.28:0/3644850807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94003a30 0x7f0ba80c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bad119700 1 -- 172.21.15.28:0/3644850807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba80ce870 con 0x7f0ba80c3350 2023-11-28T11:09:45.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bad91a700 1 --2- 172.21.15.28:0/3644850807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94003a30 0x7f0ba80c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:45.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bad119700 1 --2- 172.21.15.28:0/3644850807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ba80c3350 0x7f0ba81f2870 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0b980119c0 tx=0x7f0b98011cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:45.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0b9e7fc700 1 -- 172.21.15.28:0/3644850807 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b98004240 con 0x7f0ba80c3350 2023-11-28T11:09:45.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0b9e7fc700 1 -- 172.21.15.28:0/3644850807 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b980043a0 con 0x7f0ba80c3350 2023-11-28T11:09:45.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0b9e7fc700 1 -- 172.21.15.28:0/3644850807 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9800f730 con 0x7f0ba80c3350 2023-11-28T11:09:45.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bafb7e700 1 -- 172.21.15.28:0/3644850807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba80ce590 con 0x7f0ba80c3350 2023-11-28T11:09:45.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.730+0000 7f0bafb7e700 1 -- 172.21.15.28:0/3644850807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba80cca20 con 0x7f0ba80c3350 2023-11-28T11:09:45.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.731+0000 7f0bafb7e700 1 -- 172.21.15.28:0/3644850807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba80675a0 con 0x7f0ba80c3350 2023-11-28T11:09:45.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.732+0000 7f0b9e7fc700 1 -- 172.21.15.28:0/3644850807 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0b98025070 con 0x7f0ba80c3350 2023-11-28T11:09:45.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.732+0000 7f0b9e7fc700 1 --2- 172.21.15.28:0/3644850807 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b8c0682f0 0x7f0b8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:45.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.732+0000 7f0b9e7fc700 1 -- 172.21.15.28:0/3644850807 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0b98086260 con 0x7f0ba80c3350 2023-11-28T11:09:45.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.732+0000 7f0bad91a700 1 --2- 172.21.15.28:0/3644850807 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b8c0682f0 0x7f0b8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:45.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.733+0000 7f0bad91a700 1 --2- 172.21.15.28:0/3644850807 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b8c0682f0 0x7f0b8c06a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0005040 tx=0x7f0ba0009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:45.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.736+0000 7f0b9e7fc700 1 -- 172.21.15.28:0/3644850807 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0b98052790 con 0x7f0ba80c3350 2023-11-28T11:09:45.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.930+0000 7f0bafb7e700 1 -- 172.21.15.28:0/3644850807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ba80cdc50 con 0x7f0ba80c3350 2023-11-28T11:09:45.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.932+0000 7f0b9e7fc700 1 -- 172.21.15.28:0/3644850807 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19737 (secure 0 0 0) 0x7f0b98019090 con 0x7f0ba80c3350 2023-11-28T11:09:45.933 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:45.933 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:09:34.614188+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:45.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.934+0000 7f0bafb7e700 1 -- 172.21.15.28:0/3644850807 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b8c0682f0 msgr2=0x7f0b8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.934+0000 7f0bafb7e700 1 --2- 172.21.15.28:0/3644850807 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b8c0682f0 0x7f0b8c06a7a0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0005040 tx=0x7f0ba0009030 comp rx=0 tx=0).stop 2023-11-28T11:09:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.934+0000 7f0bafb7e700 1 -- 172.21.15.28:0/3644850807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ba80c3350 msgr2=0x7f0ba81f2870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.934+0000 7f0bafb7e700 1 --2- 172.21.15.28:0/3644850807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ba80c3350 0x7f0ba81f2870 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f0b980119c0 tx=0x7f0b98011cd0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.934+0000 7f0bafb7e700 1 -- 172.21.15.28:0/3644850807 shutdown_connections 2023-11-28T11:09:45.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.934+0000 7f0bafb7e700 1 --2- 172.21.15.28:0/3644850807 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b8c0682f0 0x7f0b8c06a7a0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.934+0000 7f0bafb7e700 1 --2- 172.21.15.28:0/3644850807 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0ba815b710 0x7f0ba815bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.934+0000 7f0bafb7e700 1 --2- 172.21.15.28:0/3644850807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b94003a30 0x7f0ba80c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.934+0000 7f0bafb7e700 1 --2- 172.21.15.28:0/3644850807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0ba80c3350 0x7f0ba81f2870 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.934+0000 7f0bafb7e700 1 -- 172.21.15.28:0/3644850807 >> 172.21.15.28:0/3644850807 conn(0x7f0ba80b1650 msgr2=0x7f0ba80aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:45.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.934+0000 7f0bafb7e700 1 -- 172.21.15.28:0/3644850807 shutdown_connections 2023-11-28T11:09:45.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:45.934+0000 7f0bafb7e700 1 -- 172.21.15.28:0/3644850807 wait complete. 2023-11-28T11:09:45.948 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:45.949 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:46.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.076+0000 7f8acf4e4700 1 Processor -- start 2023-11-28T11:09:46.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.076+0000 7f8acf4e4700 1 -- start start 2023-11-28T11:09:46.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.077+0000 7f8acf4e4700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8ac80cf5f0 0x7f8ac80caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.077+0000 7f8acf4e4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8ac80cb3e0 0x7f8ac806afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.077+0000 7f8acf4e4700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac80ce550 con 0x7f8ac80b1630 2023-11-28T11:09:46.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.077+0000 7f8acf4e4700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac80ce830 con 0x7f8ac80cf5f0 2023-11-28T11:09:46.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.077+0000 7f8acf4e4700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac80ceb10 con 0x7f8ac80cb3e0 2023-11-28T11:09:46.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.077+0000 7f8acca7f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8ac80cf5f0 0x7f8ac80caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.077+0000 7f8acd280700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8ac80b1630 0x7f8ac80b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36618/0 (socket says 172.21.15.28:36618) 2023-11-28T11:09:46.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.077+0000 7f8acd280700 1 -- 172.21.15.28:0/1447891315 learned_addr learned my addr 172.21.15.28:0/1447891315 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:46.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.077+0000 7f8acda81700 1 --2- 172.21.15.28:0/1447891315 >> v2:172.21.15.59:3300/0 conn(0x7f8ac80cb3e0 0x7f8ac806afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.077+0000 7f8acca7f700 1 -- 172.21.15.28:0/1447891315 >> v2:172.21.15.59:3300/0 conn(0x7f8ac80cb3e0 msgr2=0x7f8ac806afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:46.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.077+0000 7f8acca7f700 1 --2- 172.21.15.28:0/1447891315 >> v2:172.21.15.59:3300/0 conn(0x7f8ac80cb3e0 0x7f8ac806afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.077+0000 7f8acca7f700 1 -- 172.21.15.28:0/1447891315 >> v1:172.21.15.28:6789/0 conn(0x7f8ac80b1630 legacy=0x7f8ac80b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.077+0000 7f8abffff700 1 -- 172.21.15.28:0/1447891315 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334674526 0 0) 0x7f8ac80ce550 con 0x7f8ac80b1630 2023-11-28T11:09:46.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.077+0000 7f8acca7f700 1 -- 172.21.15.28:0/1447891315 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac80cedf0 con 0x7f8ac80cf5f0 2023-11-28T11:09:46.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.078+0000 7f8acca7f700 1 --2- 172.21.15.28:0/1447891315 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ac80cf5f0 0x7f8ac80caea0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009840 tx=0x7f8ac4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f480d2b49b4c5f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.078+0000 7f8abffff700 1 -- 172.21.15.28:0/1447891315 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4011830 con 0x7f8ac80cf5f0 2023-11-28T11:09:46.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.078+0000 7f8abffff700 1 -- 172.21.15.28:0/1447891315 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac4011990 con 0x7f8ac80cf5f0 2023-11-28T11:09:46.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.078+0000 7f8abffff700 1 -- 172.21.15.28:0/1447891315 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4011d20 con 0x7f8ac80cf5f0 2023-11-28T11:09:46.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.078+0000 7f8acf4e4700 1 -- 172.21.15.28:0/1447891315 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ac80cf5f0 msgr2=0x7f8ac80caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.078+0000 7f8acf4e4700 1 --2- 172.21.15.28:0/1447891315 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ac80cf5f0 0x7f8ac80caea0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4009840 tx=0x7f8ac4009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:46.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.078+0000 7f8acf4e4700 1 -- 172.21.15.28:0/1447891315 shutdown_connections 2023-11-28T11:09:46.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.078+0000 7f8acf4e4700 1 --2- 172.21.15.28:0/1447891315 >> v2:172.21.15.59:3300/0 conn(0x7f8ac80cb3e0 0x7f8ac806afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.078+0000 7f8acf4e4700 1 --2- 172.21.15.28:0/1447891315 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ac80cf5f0 0x7f8ac80caea0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.078+0000 7f8acf4e4700 1 -- 172.21.15.28:0/1447891315 >> 172.21.15.28:0/1447891315 conn(0x7f8ac80cc630 msgr2=0x7f8ac80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:46.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.078+0000 7f8acf4e4700 1 -- 172.21.15.28:0/1447891315 shutdown_connections 2023-11-28T11:09:46.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.078+0000 7f8acf4e4700 1 -- 172.21.15.28:0/1447891315 wait complete. 2023-11-28T11:09:46.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.080+0000 7f8acf4e4700 1 Processor -- start 2023-11-28T11:09:46.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.080+0000 7f8acf4e4700 1 -- start start 2023-11-28T11:09:46.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.080+0000 7f8acf4e4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ac80b1630 0x7f8ac80c9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.080+0000 7f8acf4e4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ac80cb3e0 0x7f8ac80ad930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.080+0000 7f8acf4e4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ac80cf5f0 0x7f8ac80ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.080+0000 7f8acf4e4700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac80c7730 con 0x7f8ac80cb3e0 2023-11-28T11:09:46.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.080+0000 7f8acf4e4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac80c7a10 con 0x7f8ac80cf5f0 2023-11-28T11:09:46.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.080+0000 7f8acf4e4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac80c7cf0 con 0x7f8ac80b1630 2023-11-28T11:09:46.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.080+0000 7f8acca7f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ac80cb3e0 0x7f8ac80ad930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.080+0000 7f8acca7f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ac80cb3e0 0x7f8ac80ad930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48568/0 (socket says 172.21.15.28:48568) 2023-11-28T11:09:46.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.080+0000 7f8acca7f700 1 -- 172.21.15.28:0/38465998 learned_addr learned my addr 172.21.15.28:0/38465998 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:46.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.081+0000 7f8acd280700 1 --2- 172.21.15.28:0/38465998 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ac80b1630 0x7f8ac80c9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.081+0000 7f8acda81700 1 --2- 172.21.15.28:0/38465998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ac80cf5f0 0x7f8ac80ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.081+0000 7f8acca7f700 1 -- 172.21.15.28:0/38465998 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ac80b1630 msgr2=0x7f8ac80c9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.081+0000 7f8acca7f700 1 --2- 172.21.15.28:0/38465998 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ac80b1630 0x7f8ac80c9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.081+0000 7f8acca7f700 1 -- 172.21.15.28:0/38465998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ac80cf5f0 msgr2=0x7f8ac80ade70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.081+0000 7f8acca7f700 1 --2- 172.21.15.28:0/38465998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ac80cf5f0 0x7f8ac80ade70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.081+0000 7f8acca7f700 1 -- 172.21.15.28:0/38465998 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac40094a0 con 0x7f8ac80cb3e0 2023-11-28T11:09:46.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.081+0000 7f8acca7f700 1 --2- 172.21.15.28:0/38465998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ac80cb3e0 0x7f8ac80ad930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4014040 tx=0x7f8ac4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.081+0000 7f8abdffb700 1 -- 172.21.15.28:0/38465998 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac401aa60 con 0x7f8ac80cb3e0 2023-11-28T11:09:46.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.081+0000 7f8abdffb700 1 -- 172.21.15.28:0/38465998 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac401abc0 con 0x7f8ac80cb3e0 2023-11-28T11:09:46.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.081+0000 7f8acf4e4700 1 -- 172.21.15.28:0/38465998 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac80ce140 con 0x7f8ac80cb3e0 2023-11-28T11:09:46.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.081+0000 7f8abdffb700 1 -- 172.21.15.28:0/38465998 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40243f0 con 0x7f8ac80cb3e0 2023-11-28T11:09:46.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.081+0000 7f8acf4e4700 1 -- 172.21.15.28:0/38465998 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac80cf0d0 con 0x7f8ac80cb3e0 2023-11-28T11:09:46.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.082+0000 7f8acf4e4700 1 -- 172.21.15.28:0/38465998 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac80ce2d0 con 0x7f8ac80cb3e0 2023-11-28T11:09:46.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.083+0000 7f8abdffb700 1 -- 172.21.15.28:0/38465998 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8ac400f050 con 0x7f8ac80cb3e0 2023-11-28T11:09:46.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.083+0000 7f8abdffb700 1 --2- 172.21.15.28:0/38465998 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8aac0682f0 0x7f8aac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.083+0000 7f8abdffb700 1 -- 172.21.15.28:0/38465998 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8ac4087350 con 0x7f8ac80cb3e0 2023-11-28T11:09:46.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.083+0000 7f8acd280700 1 --2- 172.21.15.28:0/38465998 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8aac0682f0 0x7f8aac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.083+0000 7f8acd280700 1 --2- 172.21.15.28:0/38465998 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8aac0682f0 0x7f8aac06a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8007fd0 tx=0x7f8ab8007de0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.091 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.087+0000 7f8abdffb700 1 -- 172.21.15.28:0/38465998 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8ac4053880 con 0x7f8ac80cb3e0 2023-11-28T11:09:46.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.282+0000 7f8acf4e4700 1 -- 172.21.15.28:0/38465998 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8ac80739f0 con 0x7f8ac80cb3e0 2023-11-28T11:09:46.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.283+0000 7f8abdffb700 1 -- 172.21.15.28:0/38465998 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8ac4056ea0 con 0x7f8ac80cb3e0 2023-11-28T11:09:46.283 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:09:46.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.284+0000 7f8acf4e4700 1 -- 172.21.15.28:0/38465998 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8aac0682f0 msgr2=0x7f8aac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.284+0000 7f8acf4e4700 1 --2- 172.21.15.28:0/38465998 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8aac0682f0 0x7f8aac06a7a0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f8ab8007fd0 tx=0x7f8ab8007de0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.284+0000 7f8acf4e4700 1 -- 172.21.15.28:0/38465998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ac80cb3e0 msgr2=0x7f8ac80ad930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.285+0000 7f8acf4e4700 1 --2- 172.21.15.28:0/38465998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ac80cb3e0 0x7f8ac80ad930 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4014040 tx=0x7f8ac4004750 comp rx=0 tx=0).stop 2023-11-28T11:09:46.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.285+0000 7f8acf4e4700 1 -- 172.21.15.28:0/38465998 shutdown_connections 2023-11-28T11:09:46.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.285+0000 7f8acf4e4700 1 --2- 172.21.15.28:0/38465998 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8aac0682f0 0x7f8aac06a7a0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.285+0000 7f8acf4e4700 1 --2- 172.21.15.28:0/38465998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ac80cf5f0 0x7f8ac80ade70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.285+0000 7f8acf4e4700 1 --2- 172.21.15.28:0/38465998 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ac80b1630 0x7f8ac80c9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.285+0000 7f8acf4e4700 1 --2- 172.21.15.28:0/38465998 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ac80cb3e0 0x7f8ac80ad930 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.285+0000 7f8acf4e4700 1 -- 172.21.15.28:0/38465998 >> 172.21.15.28:0/38465998 conn(0x7f8ac80cc630 msgr2=0x7f8ac806afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:46.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.285+0000 7f8acf4e4700 1 -- 172.21.15.28:0/38465998 shutdown_connections 2023-11-28T11:09:46.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.285+0000 7f8acf4e4700 1 -- 172.21.15.28:0/38465998 wait complete. 2023-11-28T11:09:46.297 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:46.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.425+0000 7f2c3b132700 1 Processor -- start 2023-11-28T11:09:46.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.425+0000 7f2c3b132700 1 -- start start 2023-11-28T11:09:46.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.425+0000 7f2c3b132700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2c3415c950 0x7f2c3406a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.425+0000 7f2c3b132700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c340ca360 0x7f2c3406f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.425+0000 7f2c3b132700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c340caab0 con 0x7f2c340cae50 2023-11-28T11:09:46.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.425+0000 7f2c3b132700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3415cd20 con 0x7f2c3415c950 2023-11-28T11:09:46.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.425+0000 7f2c3b132700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c3406e770 con 0x7f2c340ca360 2023-11-28T11:09:46.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.426+0000 7f2c38ece700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2c3415c950 0x7f2c3406a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.426+0000 7f2c38ece700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2c3415c950 0x7f2c3406a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48574/0 (socket says 172.21.15.28:48574) 2023-11-28T11:09:46.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.426+0000 7f2c38ece700 1 -- 172.21.15.28:0/3223635740 learned_addr learned my addr 172.21.15.28:0/3223635740 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:46.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.426+0000 7f2c33fff700 1 --2- 172.21.15.28:0/3223635740 >> v2:172.21.15.59:3300/0 conn(0x7f2c340ca360 0x7f2c3406f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.426+0000 7f2c38ece700 1 -- 172.21.15.28:0/3223635740 >> v2:172.21.15.59:3300/0 conn(0x7f2c340ca360 msgr2=0x7f2c3406f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:46.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.426+0000 7f2c38ece700 1 --2- 172.21.15.28:0/3223635740 >> v2:172.21.15.59:3300/0 conn(0x7f2c340ca360 0x7f2c3406f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.426+0000 7f2c38ece700 1 -- 172.21.15.28:0/3223635740 >> v1:172.21.15.59:6789/0 conn(0x7f2c340cae50 legacy=0x7f2c340cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.426+0000 7f2c38ece700 1 -- 172.21.15.28:0/3223635740 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c340ca7d0 con 0x7f2c3415c950 2023-11-28T11:09:46.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.426+0000 7f2c38ece700 1 --2- 172.21.15.28:0/3223635740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c3415c950 0x7f2c3406a1e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2c3406d100 tx=0x7f2c28009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af844b4d6eee5d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.426+0000 7f2c337fe700 1 -- 172.21.15.28:0/3223635740 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800b870 con 0x7f2c3415c950 2023-11-28T11:09:46.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.426+0000 7f2c337fe700 1 -- 172.21.15.28:0/3223635740 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2800b9d0 con 0x7f2c3415c950 2023-11-28T11:09:46.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.426+0000 7f2c337fe700 1 -- 172.21.15.28:0/3223635740 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800bd60 con 0x7f2c3415c950 2023-11-28T11:09:46.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.427+0000 7f2c3b132700 1 -- 172.21.15.28:0/3223635740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c3415c950 msgr2=0x7f2c3406a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.427+0000 7f2c3b132700 1 --2- 172.21.15.28:0/3223635740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c3415c950 0x7f2c3406a1e0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f2c3406d100 tx=0x7f2c28009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.427+0000 7f2c3b132700 1 -- 172.21.15.28:0/3223635740 shutdown_connections 2023-11-28T11:09:46.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.427+0000 7f2c3b132700 1 --2- 172.21.15.28:0/3223635740 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c3415c950 0x7f2c3406a1e0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.427+0000 7f2c3b132700 1 --2- 172.21.15.28:0/3223635740 >> v2:172.21.15.59:3300/0 conn(0x7f2c340ca360 0x7f2c3406f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.427+0000 7f2c3b132700 1 -- 172.21.15.28:0/3223635740 >> 172.21.15.28:0/3223635740 conn(0x7f2c340b0df0 msgr2=0x7f2c340b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:46.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.427+0000 7f2c3b132700 1 -- 172.21.15.28:0/3223635740 shutdown_connections 2023-11-28T11:09:46.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.427+0000 7f2c3b132700 1 -- 172.21.15.28:0/3223635740 wait complete. 2023-11-28T11:09:46.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.428+0000 7f2c3b132700 1 Processor -- start 2023-11-28T11:09:46.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.428+0000 7f2c3b132700 1 -- start start 2023-11-28T11:09:46.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.429+0000 7f2c3b132700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c340ca360 0x7f2c340c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.429+0000 7f2c3b132700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c340cae50 0x7f2c341477d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.429+0000 7f2c3b132700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c3415c950 0x7f2c34147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.429+0000 7f2c3b132700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c3414ba80 con 0x7f2c340ca360 2023-11-28T11:09:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.429+0000 7f2c3b132700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c340c4250 con 0x7f2c340cae50 2023-11-28T11:09:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.429+0000 7f2c3b132700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c34148320 con 0x7f2c3415c950 2023-11-28T11:09:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.429+0000 7f2c38ece700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c340ca360 0x7f2c340c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.429+0000 7f2c38ece700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c340ca360 0x7f2c340c4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48580/0 (socket says 172.21.15.28:48580) 2023-11-28T11:09:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.429+0000 7f2c38ece700 1 -- 172.21.15.28:0/1493163414 learned_addr learned my addr 172.21.15.28:0/1493163414 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.429+0000 7f2c38ece700 1 -- 172.21.15.28:0/1493163414 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c3415c950 msgr2=0x7f2c34147d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.429+0000 7f2c38ece700 1 --2- 172.21.15.28:0/1493163414 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c3415c950 0x7f2c34147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.429+0000 7f2c38ece700 1 -- 172.21.15.28:0/1493163414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c340cae50 msgr2=0x7f2c341477d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.429+0000 7f2c38ece700 1 --2- 172.21.15.28:0/1493163414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c340cae50 0x7f2c341477d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.429+0000 7f2c38ece700 1 -- 172.21.15.28:0/1493163414 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c28009550 con 0x7f2c340ca360 2023-11-28T11:09:46.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.430+0000 7f2c38ece700 1 --2- 172.21.15.28:0/1493163414 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c340ca360 0x7f2c340c4cf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2c28000c00 tx=0x7f2c28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.430+0000 7f2c317fa700 1 -- 172.21.15.28:0/1493163414 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c28004520 con 0x7f2c340ca360 2023-11-28T11:09:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.430+0000 7f2c317fa700 1 -- 172.21.15.28:0/1493163414 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c2801e070 con 0x7f2c340ca360 2023-11-28T11:09:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.430+0000 7f2c3b132700 1 -- 172.21.15.28:0/1493163414 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c3406ecf0 con 0x7f2c340ca360 2023-11-28T11:09:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.430+0000 7f2c317fa700 1 -- 172.21.15.28:0/1493163414 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280038a0 con 0x7f2c340ca360 2023-11-28T11:09:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.430+0000 7f2c3b132700 1 -- 172.21.15.28:0/1493163414 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c3406ea10 con 0x7f2c340ca360 2023-11-28T11:09:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.431+0000 7f2c3b132700 1 -- 172.21.15.28:0/1493163414 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3406efd0 con 0x7f2c340ca360 2023-11-28T11:09:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.431+0000 7f2c317fa700 1 -- 172.21.15.28:0/1493163414 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2c2800f050 con 0x7f2c340ca360 2023-11-28T11:09:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.432+0000 7f2c317fa700 1 --2- 172.21.15.28:0/1493163414 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2c180682f0 0x7f2c1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.432+0000 7f2c33fff700 1 --2- 172.21.15.28:0/1493163414 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2c180682f0 0x7f2c1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.432+0000 7f2c317fa700 1 -- 172.21.15.28:0/1493163414 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f2c28020070 con 0x7f2c340ca360 2023-11-28T11:09:46.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.432+0000 7f2c33fff700 1 --2- 172.21.15.28:0/1493163414 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2c180682f0 0x7f2c1806a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2c340c56c0 tx=0x7f2c20008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.436+0000 7f2c317fa700 1 -- 172.21.15.28:0/1493163414 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2c28053b20 con 0x7f2c340ca360 2023-11-28T11:09:46.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.627+0000 7f2c3b132700 1 -- 172.21.15.28:0/1493163414 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2c340739f0 con 0x7f2c340ca360 2023-11-28T11:09:46.629 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.628+0000 7f2c317fa700 1 -- 172.21.15.28:0/1493163414 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c28057140 con 0x7f2c340ca360 2023-11-28T11:09:46.629 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:09:46.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.630+0000 7f2c3b132700 1 -- 172.21.15.28:0/1493163414 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2c180682f0 msgr2=0x7f2c1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.630+0000 7f2c3b132700 1 --2- 172.21.15.28:0/1493163414 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2c180682f0 0x7f2c1806a7a0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2c340c56c0 tx=0x7f2c20008040 comp rx=0 tx=0).stop 2023-11-28T11:09:46.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.630+0000 7f2c3b132700 1 -- 172.21.15.28:0/1493163414 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c340ca360 msgr2=0x7f2c340c4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.630+0000 7f2c3b132700 1 --2- 172.21.15.28:0/1493163414 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c340ca360 0x7f2c340c4cf0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f2c28000c00 tx=0x7f2c28004750 comp rx=0 tx=0).stop 2023-11-28T11:09:46.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.630+0000 7f2c3b132700 1 -- 172.21.15.28:0/1493163414 shutdown_connections 2023-11-28T11:09:46.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.630+0000 7f2c3b132700 1 --2- 172.21.15.28:0/1493163414 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2c180682f0 0x7f2c1806a7a0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.630+0000 7f2c3b132700 1 --2- 172.21.15.28:0/1493163414 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c340ca360 0x7f2c340c4cf0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.630+0000 7f2c3b132700 1 --2- 172.21.15.28:0/1493163414 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c3415c950 0x7f2c34147d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.630+0000 7f2c3b132700 1 --2- 172.21.15.28:0/1493163414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c340cae50 0x7f2c341477d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.630+0000 7f2c3b132700 1 -- 172.21.15.28:0/1493163414 >> 172.21.15.28:0/1493163414 conn(0x7f2c340b0df0 msgr2=0x7f2c340c9930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:46.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.630+0000 7f2c3b132700 1 -- 172.21.15.28:0/1493163414 shutdown_connections 2023-11-28T11:09:46.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.630+0000 7f2c3b132700 1 -- 172.21.15.28:0/1493163414 wait complete. 2023-11-28T11:09:46.643 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:46.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.770+0000 7f5728176700 1 Processor -- start 2023-11-28T11:09:46.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.770+0000 7f5728176700 1 -- start start 2023-11-28T11:09:46.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.771+0000 7f5728176700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57200b1630 0x7f57200cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.771+0000 7f5728176700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f572006afc0 0x7f572006b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.771+0000 7f5728176700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57200ce550 con 0x7f57200b1a00 2023-11-28T11:09:46.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.771+0000 7f5728176700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57200ce830 con 0x7f572006afc0 2023-11-28T11:09:46.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.771+0000 7f5728176700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57200ceb10 con 0x7f57200b1630 2023-11-28T11:09:46.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.771+0000 7f5725711700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f57200b1a00 0x7f57200caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36646/0 (socket says 172.21.15.28:36646) 2023-11-28T11:09:46.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.771+0000 7f5725711700 1 -- 172.21.15.28:0/800145102 learned_addr learned my addr 172.21.15.28:0/800145102 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:46.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.771+0000 7f5725f12700 1 --2- 172.21.15.28:0/800145102 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57200b1630 0x7f57200cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.771+0000 7f5726713700 1 --2- 172.21.15.28:0/800145102 >> v2:172.21.15.59:3300/0 conn(0x7f572006afc0 0x7f572006b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 -- 172.21.15.28:0/800145102 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2962220299 0 0) 0x7f57200ce550 con 0x7f57200b1a00 2023-11-28T11:09:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 -- 172.21.15.28:0/800145102 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f570c0035b0 con 0x7f57200b1a00 2023-11-28T11:09:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 -- 172.21.15.28:0/800145102 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 88785822 0 0) 0x7f570c0035b0 con 0x7f57200b1a00 2023-11-28T11:09:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 -- 172.21.15.28:0/800145102 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57200b1630 msgr2=0x7f57200cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 --2- 172.21.15.28:0/800145102 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57200b1630 0x7f57200cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 -- 172.21.15.28:0/800145102 >> v2:172.21.15.59:3300/0 conn(0x7f572006afc0 msgr2=0x7f572006b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 --2- 172.21.15.28:0/800145102 >> v2:172.21.15.59:3300/0 conn(0x7f572006afc0 0x7f572006b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 -- 172.21.15.28:0/800145102 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57200cedf0 con 0x7f57200b1a00 2023-11-28T11:09:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 -- 172.21.15.28:0/800145102 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f571c001670 con 0x7f57200b1a00 2023-11-28T11:09:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 -- 172.21.15.28:0/800145102 >> v1:172.21.15.28:6789/0 conn(0x7f57200b1a00 legacy=0x7f57200caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 --2- 172.21.15.28:0/800145102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f570c003a00 0x7f570c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 --2- 172.21.15.28:0/800145102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f570c004430 0x7f570c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 --2- 172.21.15.28:0/800145102 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57200b1630 0x7f570c004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 -- 172.21.15.28:0/800145102 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57200ce830 con 0x7f570c003a00 2023-11-28T11:09:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 -- 172.21.15.28:0/800145102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57200ceb10 con 0x7f570c004430 2023-11-28T11:09:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5724f10700 1 -- 172.21.15.28:0/800145102 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570c00ba10 con 0x7f57200b1630 2023-11-28T11:09:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5725f12700 1 --2- 172.21.15.28:0/800145102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f570c003a00 0x7f570c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5726713700 1 --2- 172.21.15.28:0/800145102 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57200b1630 0x7f570c004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5725711700 1 --2- 172.21.15.28:0/800145102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f570c004430 0x7f570c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5725f12700 1 -- 172.21.15.28:0/800145102 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57200b1630 msgr2=0x7f570c004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5725f12700 1 --2- 172.21.15.28:0/800145102 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57200b1630 0x7f570c004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5725f12700 1 -- 172.21.15.28:0/800145102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f570c004430 msgr2=0x7f570c0048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5725f12700 1 --2- 172.21.15.28:0/800145102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f570c004430 0x7f570c0048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5725f12700 1 -- 172.21.15.28:0/800145102 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570c0035b0 con 0x7f570c003a00 2023-11-28T11:09:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.772+0000 7f5725711700 1 --2- 172.21.15.28:0/800145102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f570c004430 0x7f570c0048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.773+0000 7f5725f12700 1 --2- 172.21.15.28:0/800145102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f570c003a00 0x7f570c003e10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5710009b60 tx=0x7f570c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6d51ad666584e3c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.773+0000 7f5724f10700 1 -- 172.21.15.28:0/800145102 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571000e6c0 con 0x7f570c003a00 2023-11-28T11:09:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.773+0000 7f5724f10700 1 -- 172.21.15.28:0/800145102 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571000e820 con 0x7f570c003a00 2023-11-28T11:09:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.773+0000 7f5724f10700 1 -- 172.21.15.28:0/800145102 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571000eb90 con 0x7f570c003a00 2023-11-28T11:09:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.773+0000 7f5728176700 1 -- 172.21.15.28:0/800145102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f570c003a00 msgr2=0x7f570c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.773+0000 7f5728176700 1 --2- 172.21.15.28:0/800145102 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f570c003a00 0x7f570c003e10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5710009b60 tx=0x7f570c005880 comp rx=0 tx=0).stop 2023-11-28T11:09:46.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.773+0000 7f5726713700 1 -- 172.21.15.28:0/800145102 reap_dead start 2023-11-28T11:09:46.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.773+0000 7f5728176700 1 -- 172.21.15.28:0/800145102 shutdown_connections 2023-11-28T11:09:46.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.773+0000 7f5728176700 1 -- 172.21.15.28:0/800145102 >> 172.21.15.28:0/800145102 conn(0x7f57200cc630 msgr2=0x7f57200cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:46.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.773+0000 7f5728176700 1 -- 172.21.15.28:0/800145102 shutdown_connections 2023-11-28T11:09:46.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.773+0000 7f5728176700 1 -- 172.21.15.28:0/800145102 wait complete. 2023-11-28T11:09:46.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.775+0000 7f5728176700 1 Processor -- start 2023-11-28T11:09:46.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.775+0000 7f5728176700 1 -- start start 2023-11-28T11:09:46.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.775+0000 7f5728176700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f570c003a00 0x7f572015e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.775+0000 7f5728176700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57200c8f90 0x7f572015caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.775+0000 7f5728176700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57200c9400 0x7f572015b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.775+0000 7f5728176700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572015d560 con 0x7f570c003a00 2023-11-28T11:09:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.775+0000 7f5728176700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f572015d840 con 0x7f57200c9400 2023-11-28T11:09:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.775+0000 7f5728176700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f572015db20 con 0x7f57200c8f90 2023-11-28T11:09:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.775+0000 7f5725f12700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f570c003a00 0x7f572015e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.776+0000 7f5725f12700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f570c003a00 0x7f572015e600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48600/0 (socket says 172.21.15.28:48600) 2023-11-28T11:09:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.776+0000 7f5725f12700 1 -- 172.21.15.28:0/1841643688 learned_addr learned my addr 172.21.15.28:0/1841643688 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.776+0000 7f5725711700 1 --2- 172.21.15.28:0/1841643688 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57200c8f90 0x7f572015caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.776+0000 7f5726713700 1 --2- 172.21.15.28:0/1841643688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57200c9400 0x7f572015b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.776+0000 7f5725f12700 1 -- 172.21.15.28:0/1841643688 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57200c8f90 msgr2=0x7f572015caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.776+0000 7f5725f12700 1 --2- 172.21.15.28:0/1841643688 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57200c8f90 0x7f572015caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.776+0000 7f5725f12700 1 -- 172.21.15.28:0/1841643688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57200c9400 msgr2=0x7f572015b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.776+0000 7f5725f12700 1 --2- 172.21.15.28:0/1841643688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57200c9400 0x7f572015b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.776+0000 7f5725f12700 1 -- 172.21.15.28:0/1841643688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57200b1e70 con 0x7f570c003a00 2023-11-28T11:09:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.776+0000 7f5725f12700 1 --2- 172.21.15.28:0/1841643688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f570c003a00 0x7f572015e600 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5710000c00 tx=0x7f57100047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.776+0000 7f57167fc700 1 -- 172.21.15.28:0/1841643688 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5710021070 con 0x7f570c003a00 2023-11-28T11:09:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.776+0000 7f57167fc700 1 -- 172.21.15.28:0/1841643688 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57100043f0 con 0x7f570c003a00 2023-11-28T11:09:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.776+0000 7f5728176700 1 -- 172.21.15.28:0/1841643688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57200cf0d0 con 0x7f570c003a00 2023-11-28T11:09:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.776+0000 7f57167fc700 1 -- 172.21.15.28:0/1841643688 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571000e6c0 con 0x7f570c003a00 2023-11-28T11:09:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.776+0000 7f5728176700 1 -- 172.21.15.28:0/1841643688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57200cb5b0 con 0x7f570c003a00 2023-11-28T11:09:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.777+0000 7f5728176700 1 -- 172.21.15.28:0/1841643688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57200739f0 con 0x7f570c003a00 2023-11-28T11:09:46.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.778+0000 7f57167fc700 1 -- 172.21.15.28:0/1841643688 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f571002e030 con 0x7f570c003a00 2023-11-28T11:09:46.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.778+0000 7f57167fc700 1 --2- 172.21.15.28:0/1841643688 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f57040682f0 0x7f570406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:46.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.778+0000 7f57167fc700 1 -- 172.21.15.28:0/1841643688 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f57100871e0 con 0x7f570c003a00 2023-11-28T11:09:46.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.778+0000 7f5725711700 1 --2- 172.21.15.28:0/1841643688 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f57040682f0 0x7f570406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:46.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.778+0000 7f5725711700 1 --2- 172.21.15.28:0/1841643688 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f57040682f0 0x7f570406a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f571c000ff0 tx=0x7f571c007040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:46.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.782+0000 7f57167fc700 1 -- 172.21.15.28:0/1841643688 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5710053710 con 0x7f570c003a00 2023-11-28T11:09:46.974 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.973+0000 7f5728176700 1 -- 172.21.15.28:0/1841643688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f57200cac80 con 0x7f570c003a00 2023-11-28T11:09:46.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.974+0000 7f57167fc700 1 -- 172.21.15.28:0/1841643688 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5710056d30 con 0x7f570c003a00 2023-11-28T11:09:46.975 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:46.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.976+0000 7f5728176700 1 -- 172.21.15.28:0/1841643688 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f57040682f0 msgr2=0x7f570406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.976+0000 7f5728176700 1 --2- 172.21.15.28:0/1841643688 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f57040682f0 0x7f570406a7a0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f571c000ff0 tx=0x7f571c007040 comp rx=0 tx=0).stop 2023-11-28T11:09:46.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.976+0000 7f5728176700 1 -- 172.21.15.28:0/1841643688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f570c003a00 msgr2=0x7f572015e600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:46.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.976+0000 7f5728176700 1 --2- 172.21.15.28:0/1841643688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f570c003a00 0x7f572015e600 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5710000c00 tx=0x7f57100047c0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.976+0000 7f5728176700 1 -- 172.21.15.28:0/1841643688 shutdown_connections 2023-11-28T11:09:46.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.976+0000 7f5728176700 1 --2- 172.21.15.28:0/1841643688 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f57040682f0 0x7f570406a7a0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.976+0000 7f5728176700 1 --2- 172.21.15.28:0/1841643688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f570c003a00 0x7f572015e600 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.976+0000 7f5728176700 1 --2- 172.21.15.28:0/1841643688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f57200c9400 0x7f572015b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.976+0000 7f5728176700 1 --2- 172.21.15.28:0/1841643688 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f57200c8f90 0x7f572015caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:46.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.976+0000 7f5728176700 1 -- 172.21.15.28:0/1841643688 >> 172.21.15.28:0/1841643688 conn(0x7f57200cc630 msgr2=0x7f57200cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:46.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.976+0000 7f5728176700 1 -- 172.21.15.28:0/1841643688 shutdown_connections 2023-11-28T11:09:46.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:46.976+0000 7f5728176700 1 -- 172.21.15.28:0/1841643688 wait complete. 2023-11-28T11:09:46.988 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:47.119 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.116+0000 7f8ca5b98700 1 Processor -- start 2023-11-28T11:09:47.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.117+0000 7f8ca5b98700 1 -- start start 2023-11-28T11:09:47.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.117+0000 7f8ca5b98700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8ca00b1630 0x7f8ca00cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.117+0000 7f8ca5b98700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca00ce550 con 0x7f8ca00b1a00 2023-11-28T11:09:47.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.117+0000 7f8ca5b98700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ca00ce830 con 0x7f8ca006afc0 2023-11-28T11:09:47.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.117+0000 7f8ca5b98700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ca00ceb10 con 0x7f8ca00b1630 2023-11-28T11:09:47.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.117+0000 7f8c9f7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8ca00b1630 0x7f8ca00cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.117+0000 7f8c9effd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8ca00b1a00 0x7f8ca00caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36666/0 (socket says 172.21.15.28:36666) 2023-11-28T11:09:47.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.117+0000 7f8c9effd700 1 -- 172.21.15.28:0/2395688962 learned_addr learned my addr 172.21.15.28:0/2395688962 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:47.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.118+0000 7f8c9f7fe700 1 -- 172.21.15.28:0/2395688962 >> v1:172.21.15.59:6789/0 conn(0x7f8ca006afc0 legacy=0x7f8ca006b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.118+0000 7f8c9f7fe700 1 -- 172.21.15.28:0/2395688962 >> v1:172.21.15.28:6789/0 conn(0x7f8ca00b1a00 legacy=0x7f8ca00caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.118+0000 7f8c9f7fe700 1 -- 172.21.15.28:0/2395688962 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ca00cedf0 con 0x7f8ca00b1630 2023-11-28T11:09:47.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.118+0000 7f8c9f7fe700 1 --2- 172.21.15.28:0/2395688962 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ca00b1630 0x7f8ca00cf5f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009870 tx=0x7f8c90009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4764b234aa4db081 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.118+0000 7f8c9e7fc700 1 -- 172.21.15.28:0/2395688962 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c90004230 con 0x7f8ca00b1630 2023-11-28T11:09:47.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.118+0000 7f8c9e7fc700 1 -- 172.21.15.28:0/2395688962 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c90004390 con 0x7f8ca00b1630 2023-11-28T11:09:47.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.118+0000 7f8c9e7fc700 1 -- 172.21.15.28:0/2395688962 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c900036a0 con 0x7f8ca00b1630 2023-11-28T11:09:47.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.119+0000 7f8ca5b98700 1 -- 172.21.15.28:0/2395688962 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ca00b1630 msgr2=0x7f8ca00cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.119+0000 7f8ca5b98700 1 --2- 172.21.15.28:0/2395688962 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ca00b1630 0x7f8ca00cf5f0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8c90009870 tx=0x7f8c90009b80 comp rx=0 tx=0).stop 2023-11-28T11:09:47.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.119+0000 7f8ca5b98700 1 -- 172.21.15.28:0/2395688962 shutdown_connections 2023-11-28T11:09:47.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.119+0000 7f8ca5b98700 1 --2- 172.21.15.28:0/2395688962 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ca00b1630 0x7f8ca00cf5f0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.119+0000 7f8ca5b98700 1 -- 172.21.15.28:0/2395688962 >> 172.21.15.28:0/2395688962 conn(0x7f8ca00cc630 msgr2=0x7f8ca00cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:47.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.119+0000 7f8ca5b98700 1 -- 172.21.15.28:0/2395688962 shutdown_connections 2023-11-28T11:09:47.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.119+0000 7f8ca5b98700 1 -- 172.21.15.28:0/2395688962 wait complete. 2023-11-28T11:09:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.120+0000 7f8ca5b98700 1 Processor -- start 2023-11-28T11:09:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.121+0000 7f8ca5b98700 1 -- start start 2023-11-28T11:09:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.121+0000 7f8ca5b98700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ca006afc0 0x7f8ca01f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.121+0000 7f8ca5b98700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ca00b1630 0x7f8ca0157c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.121+0000 7f8ca5b98700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ca00b1a00 0x7f8ca0156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.121+0000 7f8ca5b98700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca01f2dd0 con 0x7f8ca00b1a00 2023-11-28T11:09:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.121+0000 7f8ca5b98700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca01581a0 con 0x7f8ca006afc0 2023-11-28T11:09:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.121+0000 7f8ca5b98700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ca00b4490 con 0x7f8ca00b1630 2023-11-28T11:09:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8c9ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ca00b1a00 0x7f8ca0156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8c9ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ca00b1a00 0x7f8ca0156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48616/0 (socket says 172.21.15.28:48616) 2023-11-28T11:09:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8c9ffff700 1 -- 172.21.15.28:0/977194172 learned_addr learned my addr 172.21.15.28:0/977194172 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8c9f7fe700 1 --2- 172.21.15.28:0/977194172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ca006afc0 0x7f8ca01f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8c9effd700 1 --2- 172.21.15.28:0/977194172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ca00b1630 0x7f8ca0157c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8c9ffff700 1 -- 172.21.15.28:0/977194172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ca00b1630 msgr2=0x7f8ca0157c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8c9ffff700 1 --2- 172.21.15.28:0/977194172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ca00b1630 0x7f8ca0157c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8c9ffff700 1 -- 172.21.15.28:0/977194172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ca006afc0 msgr2=0x7f8ca01f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8c9ffff700 1 --2- 172.21.15.28:0/977194172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ca006afc0 0x7f8ca01f3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8c9ffff700 1 -- 172.21.15.28:0/977194172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c900094a0 con 0x7f8ca00b1a00 2023-11-28T11:09:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8c9ffff700 1 --2- 172.21.15.28:0/977194172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ca00b1a00 0x7f8ca0156f90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800cc10 tx=0x7f8c9800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8c97fff700 1 -- 172.21.15.28:0/977194172 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c9800abb0 con 0x7f8ca00b1a00 2023-11-28T11:09:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8c97fff700 1 -- 172.21.15.28:0/977194172 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c9800ad10 con 0x7f8ca00b1a00 2023-11-28T11:09:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8ca5b98700 1 -- 172.21.15.28:0/977194172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ca00b1ea0 con 0x7f8ca00b1a00 2023-11-28T11:09:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8c97fff700 1 -- 172.21.15.28:0/977194172 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c98012660 con 0x7f8ca00b1a00 2023-11-28T11:09:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.122+0000 7f8ca5b98700 1 -- 172.21.15.28:0/977194172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ca00cf0d0 con 0x7f8ca00b1a00 2023-11-28T11:09:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.123+0000 7f8ca5b98700 1 -- 172.21.15.28:0/977194172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ca00cb5b0 con 0x7f8ca00b1a00 2023-11-28T11:09:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.124+0000 7f8c97fff700 1 -- 172.21.15.28:0/977194172 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8c98022070 con 0x7f8ca00b1a00 2023-11-28T11:09:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.124+0000 7f8c97fff700 1 --2- 172.21.15.28:0/977194172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c800682f0 0x7f8c8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.124+0000 7f8c97fff700 1 -- 172.21.15.28:0/977194172 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8c98086690 con 0x7f8ca00b1a00 2023-11-28T11:09:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.124+0000 7f8c9f7fe700 1 --2- 172.21.15.28:0/977194172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c800682f0 0x7f8c8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.125+0000 7f8c9f7fe700 1 --2- 172.21.15.28:0/977194172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c800682f0 0x7f8c8006a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8c90000c00 tx=0x7f8c90008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.129 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.129+0000 7f8c97fff700 1 -- 172.21.15.28:0/977194172 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8c98052bc0 con 0x7f8ca00b1a00 2023-11-28T11:09:47.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.324+0000 7f8ca5b98700 1 -- 172.21.15.28:0/977194172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8ca00739f0 con 0x7f8ca00b1a00 2023-11-28T11:09:47.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.325+0000 7f8c97fff700 1 -- 172.21.15.28:0/977194172 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c980160a0 con 0x7f8ca00b1a00 2023-11-28T11:09:47.326 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:09:47.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.327+0000 7f8ca5b98700 1 -- 172.21.15.28:0/977194172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c800682f0 msgr2=0x7f8c8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.327+0000 7f8ca5b98700 1 --2- 172.21.15.28:0/977194172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c800682f0 0x7f8c8006a7a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f8c90000c00 tx=0x7f8c90008030 comp rx=0 tx=0).stop 2023-11-28T11:09:47.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.327+0000 7f8ca5b98700 1 -- 172.21.15.28:0/977194172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ca00b1a00 msgr2=0x7f8ca0156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.327+0000 7f8ca5b98700 1 --2- 172.21.15.28:0/977194172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ca00b1a00 0x7f8ca0156f90 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8c9800cc10 tx=0x7f8c9800a060 comp rx=0 tx=0).stop 2023-11-28T11:09:47.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.327+0000 7f8ca5b98700 1 -- 172.21.15.28:0/977194172 shutdown_connections 2023-11-28T11:09:47.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.327+0000 7f8ca5b98700 1 --2- 172.21.15.28:0/977194172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8c800682f0 0x7f8c8006a7a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.327+0000 7f8ca5b98700 1 --2- 172.21.15.28:0/977194172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ca00b1a00 0x7f8ca0156f90 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.327+0000 7f8ca5b98700 1 --2- 172.21.15.28:0/977194172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ca006afc0 0x7f8ca01f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.327+0000 7f8ca5b98700 1 --2- 172.21.15.28:0/977194172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ca00b1630 0x7f8ca0157c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.329 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.327+0000 7f8ca5b98700 1 -- 172.21.15.28:0/977194172 >> 172.21.15.28:0/977194172 conn(0x7f8ca00cc630 msgr2=0x7f8ca00cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:47.329 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.327+0000 7f8ca5b98700 1 -- 172.21.15.28:0/977194172 shutdown_connections 2023-11-28T11:09:47.329 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.327+0000 7f8ca5b98700 1 -- 172.21.15.28:0/977194172 wait complete. 2023-11-28T11:09:47.340 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:47.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.468+0000 7f765d704700 1 Processor -- start 2023-11-28T11:09:47.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.468+0000 7f765d704700 1 -- start start 2023-11-28T11:09:47.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.468+0000 7f765d704700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f76580d2820 0x7f76580b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.468+0000 7f765d704700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f765806bf30 0x7f765806c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.468+0000 7f765d704700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f765806c870 con 0x7f765815c640 2023-11-28T11:09:47.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.468+0000 7f765d704700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f765815ca10 con 0x7f76580d2820 2023-11-28T11:09:47.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.468+0000 7f765d704700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76580d1470 con 0x7f765806bf30 2023-11-28T11:09:47.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7656ffd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f765815c640 0x7f76580cf040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36680/0 (socket says 172.21.15.28:36680) 2023-11-28T11:09:47.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7656ffd700 1 -- 172.21.15.28:0/2381759070 learned_addr learned my addr 172.21.15.28:0/2381759070 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:47.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f76577fe700 1 --2- 172.21.15.28:0/2381759070 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f765806bf30 0x7f765806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f76567fc700 1 --2- 172.21.15.28:0/2381759070 >> v2:172.21.15.59:3300/0 conn(0x7f76580d2820 0x7f76580b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 -- 172.21.15.28:0/2381759070 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3085633502 0 0) 0x7f765806c870 con 0x7f765815c640 2023-11-28T11:09:47.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 -- 172.21.15.28:0/2381759070 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76400035b0 con 0x7f765815c640 2023-11-28T11:09:47.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 -- 172.21.15.28:0/2381759070 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3140733147 0 0) 0x7f76400035b0 con 0x7f765815c640 2023-11-28T11:09:47.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 -- 172.21.15.28:0/2381759070 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f765806bf30 msgr2=0x7f765806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 --2- 172.21.15.28:0/2381759070 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f765806bf30 0x7f765806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 -- 172.21.15.28:0/2381759070 >> v2:172.21.15.59:3300/0 conn(0x7f76580d2820 msgr2=0x7f76580b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 --2- 172.21.15.28:0/2381759070 >> v2:172.21.15.59:3300/0 conn(0x7f76580d2820 0x7f76580b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 -- 172.21.15.28:0/2381759070 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76580cd4e0 con 0x7f765815c640 2023-11-28T11:09:47.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 -- 172.21.15.28:0/2381759070 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f7648001670 con 0x7f765815c640 2023-11-28T11:09:47.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 -- 172.21.15.28:0/2381759070 >> v1:172.21.15.28:6789/0 conn(0x7f765815c640 legacy=0x7f76580cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 --2- 172.21.15.28:0/2381759070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7640003a00 0x7f7640003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 --2- 172.21.15.28:0/2381759070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7640004430 0x7f76400048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 --2- 172.21.15.28:0/2381759070 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f765806bf30 0x7f7640004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 -- 172.21.15.28:0/2381759070 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765815ca10 con 0x7f7640003a00 2023-11-28T11:09:47.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 -- 172.21.15.28:0/2381759070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76580d1470 con 0x7f7640004430 2023-11-28T11:09:47.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7655ffb700 1 -- 172.21.15.28:0/2381759070 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764000ba10 con 0x7f765806bf30 2023-11-28T11:09:47.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.469+0000 7f7656ffd700 1 --2- 172.21.15.28:0/2381759070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7640003a00 0x7f7640003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.470+0000 7f76577fe700 1 --2- 172.21.15.28:0/2381759070 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f765806bf30 0x7f7640004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.470+0000 7f76567fc700 1 --2- 172.21.15.28:0/2381759070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7640004430 0x7f76400048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.470+0000 7f7656ffd700 1 -- 172.21.15.28:0/2381759070 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f765806bf30 msgr2=0x7f7640004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.470+0000 7f7656ffd700 1 --2- 172.21.15.28:0/2381759070 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f765806bf30 0x7f7640004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.470+0000 7f7656ffd700 1 -- 172.21.15.28:0/2381759070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7640004430 msgr2=0x7f76400048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.470+0000 7f7656ffd700 1 --2- 172.21.15.28:0/2381759070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7640004430 0x7f76400048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.470+0000 7f7656ffd700 1 -- 172.21.15.28:0/2381759070 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76400035b0 con 0x7f7640003a00 2023-11-28T11:09:47.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.470+0000 7f7656ffd700 1 --2- 172.21.15.28:0/2381759070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7640003a00 0x7f7640003e10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f764800c370 tx=0x7f7640005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2043d59089e551aa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.470+0000 7f7655ffb700 1 -- 172.21.15.28:0/2381759070 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648007c60 con 0x7f7640003a00 2023-11-28T11:09:47.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.470+0000 7f7655ffb700 1 -- 172.21.15.28:0/2381759070 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7648005960 con 0x7f7640003a00 2023-11-28T11:09:47.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.470+0000 7f7655ffb700 1 -- 172.21.15.28:0/2381759070 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7648005d10 con 0x7f7640003a00 2023-11-28T11:09:47.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.471+0000 7f765d704700 1 -- 172.21.15.28:0/2381759070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7640003a00 msgr2=0x7f7640003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.471+0000 7f765d704700 1 --2- 172.21.15.28:0/2381759070 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7640003a00 0x7f7640003e10 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f764800c370 tx=0x7f7640005880 comp rx=0 tx=0).stop 2023-11-28T11:09:47.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.471+0000 7f76577fe700 1 -- 172.21.15.28:0/2381759070 reap_dead start 2023-11-28T11:09:47.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.471+0000 7f765d704700 1 -- 172.21.15.28:0/2381759070 shutdown_connections 2023-11-28T11:09:47.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.471+0000 7f765d704700 1 -- 172.21.15.28:0/2381759070 >> 172.21.15.28:0/2381759070 conn(0x7f76580d2df0 msgr2=0x7f76580d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:47.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.471+0000 7f765d704700 1 -- 172.21.15.28:0/2381759070 shutdown_connections 2023-11-28T11:09:47.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.471+0000 7f765d704700 1 -- 172.21.15.28:0/2381759070 wait complete. 2023-11-28T11:09:47.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.472+0000 7f765d704700 1 Processor -- start 2023-11-28T11:09:47.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.472+0000 7f765d704700 1 -- start start 2023-11-28T11:09:47.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f765d704700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7640003a00 0x7f76580c8d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f765d704700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76580b0c50 0x7f76580cc880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f765d704700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f76580ccdc0 0x7f76580c9ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f765d704700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76580c88c0 con 0x7f76580b0c50 2023-11-28T11:09:47.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f765d704700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f765814f460 con 0x7f7640003a00 2023-11-28T11:09:47.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f765d704700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76580c9290 con 0x7f76580ccdc0 2023-11-28T11:09:47.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f76567fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76580b0c50 0x7f76580cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f76567fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76580b0c50 0x7f76580cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48636/0 (socket says 172.21.15.28:48636) 2023-11-28T11:09:47.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f76567fc700 1 -- 172.21.15.28:0/2457178426 learned_addr learned my addr 172.21.15.28:0/2457178426 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:47.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f7656ffd700 1 --2- 172.21.15.28:0/2457178426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7640003a00 0x7f76580c8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f76577fe700 1 --2- 172.21.15.28:0/2457178426 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f76580ccdc0 0x7f76580c9ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f76567fc700 1 -- 172.21.15.28:0/2457178426 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f76580ccdc0 msgr2=0x7f76580c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f76567fc700 1 --2- 172.21.15.28:0/2457178426 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f76580ccdc0 0x7f76580c9ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f76567fc700 1 -- 172.21.15.28:0/2457178426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7640003a00 msgr2=0x7f76580c8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f76567fc700 1 --2- 172.21.15.28:0/2457178426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7640003a00 0x7f76580c8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f76567fc700 1 -- 172.21.15.28:0/2457178426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76580d1710 con 0x7f76580b0c50 2023-11-28T11:09:47.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f7656ffd700 1 --2- 172.21.15.28:0/2457178426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7640003a00 0x7f76580c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:47.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.473+0000 7f76567fc700 1 --2- 172.21.15.28:0/2457178426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76580b0c50 0x7f76580cc880 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f764c011aa0 tx=0x7f764c011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.474+0000 7f763f7fe700 1 -- 172.21.15.28:0/2457178426 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c00f980 con 0x7f76580b0c50 2023-11-28T11:09:47.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.474+0000 7f763f7fe700 1 -- 172.21.15.28:0/2457178426 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764c00fb00 con 0x7f76580b0c50 2023-11-28T11:09:47.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.474+0000 7f765d704700 1 -- 172.21.15.28:0/2457178426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76580cd7c0 con 0x7f76580b0c50 2023-11-28T11:09:47.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.474+0000 7f763f7fe700 1 -- 172.21.15.28:0/2457178426 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764c00feb0 con 0x7f76580b0c50 2023-11-28T11:09:47.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.474+0000 7f765d704700 1 -- 172.21.15.28:0/2457178426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76580cdaa0 con 0x7f76580b0c50 2023-11-28T11:09:47.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.475+0000 7f765d704700 1 -- 172.21.15.28:0/2457178426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76580c8a30 con 0x7f76580b0c50 2023-11-28T11:09:47.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.475+0000 7f763f7fe700 1 -- 172.21.15.28:0/2457178426 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f764c014440 con 0x7f76580b0c50 2023-11-28T11:09:47.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.475+0000 7f763f7fe700 1 --2- 172.21.15.28:0/2457178426 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f76440682f0 0x7f764406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.476+0000 7f763f7fe700 1 -- 172.21.15.28:0/2457178426 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f764c086850 con 0x7f76580b0c50 2023-11-28T11:09:47.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.476+0000 7f7656ffd700 1 --2- 172.21.15.28:0/2457178426 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f76440682f0 0x7f764406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.476+0000 7f7656ffd700 1 --2- 172.21.15.28:0/2457178426 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f76440682f0 0x7f764406a7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7648015040 tx=0x7f764800a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.480+0000 7f763f7fe700 1 -- 172.21.15.28:0/2457178426 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f764c052de0 con 0x7f76580b0c50 2023-11-28T11:09:47.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.674+0000 7f765d704700 1 -- 172.21.15.28:0/2457178426 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f76580739f0 con 0x7f76580b0c50 2023-11-28T11:09:47.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.675+0000 7f763f7fe700 1 -- 172.21.15.28:0/2457178426 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f764c056400 con 0x7f76580b0c50 2023-11-28T11:09:47.675 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:47.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.676+0000 7f765d704700 1 -- 172.21.15.28:0/2457178426 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f76440682f0 msgr2=0x7f764406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.676+0000 7f765d704700 1 --2- 172.21.15.28:0/2457178426 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f76440682f0 0x7f764406a7a0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7648015040 tx=0x7f764800a000 comp rx=0 tx=0).stop 2023-11-28T11:09:47.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.676+0000 7f765d704700 1 -- 172.21.15.28:0/2457178426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76580b0c50 msgr2=0x7f76580cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.676+0000 7f765d704700 1 --2- 172.21.15.28:0/2457178426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76580b0c50 0x7f76580cc880 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f764c011aa0 tx=0x7f764c011e60 comp rx=0 tx=0).stop 2023-11-28T11:09:47.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.677+0000 7f765d704700 1 -- 172.21.15.28:0/2457178426 shutdown_connections 2023-11-28T11:09:47.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.677+0000 7f765d704700 1 --2- 172.21.15.28:0/2457178426 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f76440682f0 0x7f764406a7a0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.677+0000 7f765d704700 1 --2- 172.21.15.28:0/2457178426 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f76580ccdc0 0x7f76580c9ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.677+0000 7f765d704700 1 --2- 172.21.15.28:0/2457178426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7640003a00 0x7f76580c8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.677+0000 7f765d704700 1 --2- 172.21.15.28:0/2457178426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f76580b0c50 0x7f76580cc880 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.677+0000 7f765d704700 1 -- 172.21.15.28:0/2457178426 >> 172.21.15.28:0/2457178426 conn(0x7f76580d2df0 msgr2=0x7f76580d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:47.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.677+0000 7f765d704700 1 -- 172.21.15.28:0/2457178426 shutdown_connections 2023-11-28T11:09:47.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.677+0000 7f765d704700 1 -- 172.21.15.28:0/2457178426 wait complete. 2023-11-28T11:09:47.689 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:47.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.815+0000 7fdaa24dd700 1 Processor -- start 2023-11-28T11:09:47.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.815+0000 7fdaa24dd700 1 -- start start 2023-11-28T11:09:47.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.815+0000 7fdaa24dd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda9c15a6c0 0x7fda9c0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fdaa24dd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fda9c0d0020 0x7fda9c0ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fdaa24dd700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda9c0cfb90 con 0x7fda9c0b2660 2023-11-28T11:09:47.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fdaa24dd700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda9c0cb930 con 0x7fda9c0d0020 2023-11-28T11:09:47.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fdaa24dd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c0cf260 con 0x7fda9c15a6c0 2023-11-28T11:09:47.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fda9b7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fda9c0d0020 0x7fda9c0ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fdaa0a7a700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fda9c0b2660 0x7fda9c0b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36704/0 (socket says 172.21.15.28:36704) 2023-11-28T11:09:47.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fdaa0a7a700 1 -- 172.21.15.28:0/1214152383 learned_addr learned my addr 172.21.15.28:0/1214152383 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:47.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fda9bfff700 1 --2- 172.21.15.28:0/1214152383 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda9c15a6c0 0x7fda9c0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fda9b7fe700 1 -- 172.21.15.28:0/1214152383 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda9c15a6c0 msgr2=0x7fda9c0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:47.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fda9b7fe700 1 --2- 172.21.15.28:0/1214152383 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda9c15a6c0 0x7fda9c0cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fda9b7fe700 1 -- 172.21.15.28:0/1214152383 >> v1:172.21.15.28:6789/0 conn(0x7fda9c0b2660 legacy=0x7fda9c0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fda9b7fe700 1 -- 172.21.15.28:0/1214152383 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda9c0cf540 con 0x7fda9c0d0020 2023-11-28T11:09:47.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fda9bfff700 1 --2- 172.21.15.28:0/1214152383 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda9c15a6c0 0x7fda9c0cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:47.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fda9affd700 1 -- 172.21.15.28:0/1214152383 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 729384135 0 0) 0x7fda9c0cfb90 con 0x7fda9c0b2660 2023-11-28T11:09:47.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fda9b7fe700 1 --2- 172.21.15.28:0/1214152383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda9c0d0020 0x7fda9c0ccea0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fda90009840 tx=0x7fda90009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b4c826a8adb9688 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fda9affd700 1 -- 172.21.15.28:0/1214152383 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90011830 con 0x7fda9c0d0020 2023-11-28T11:09:47.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fda9affd700 1 -- 172.21.15.28:0/1214152383 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda90011990 con 0x7fda9c0d0020 2023-11-28T11:09:47.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.816+0000 7fda9affd700 1 -- 172.21.15.28:0/1214152383 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda90011d20 con 0x7fda9c0d0020 2023-11-28T11:09:47.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.817+0000 7fdaa24dd700 1 -- 172.21.15.28:0/1214152383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda9c0d0020 msgr2=0x7fda9c0ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.817+0000 7fdaa24dd700 1 --2- 172.21.15.28:0/1214152383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda9c0d0020 0x7fda9c0ccea0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fda90009840 tx=0x7fda90009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:47.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.817+0000 7fdaa24dd700 1 -- 172.21.15.28:0/1214152383 shutdown_connections 2023-11-28T11:09:47.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.817+0000 7fdaa24dd700 1 --2- 172.21.15.28:0/1214152383 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda9c15a6c0 0x7fda9c0cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.817+0000 7fdaa24dd700 1 --2- 172.21.15.28:0/1214152383 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda9c0d0020 0x7fda9c0ccea0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.817+0000 7fdaa24dd700 1 -- 172.21.15.28:0/1214152383 >> 172.21.15.28:0/1214152383 conn(0x7fda9c0cd660 msgr2=0x7fda9c0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:47.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.817+0000 7fdaa24dd700 1 -- 172.21.15.28:0/1214152383 shutdown_connections 2023-11-28T11:09:47.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.817+0000 7fdaa24dd700 1 -- 172.21.15.28:0/1214152383 wait complete. 2023-11-28T11:09:47.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.818+0000 7fdaa24dd700 1 Processor -- start 2023-11-28T11:09:47.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.818+0000 7fdaa24dd700 1 -- start start 2023-11-28T11:09:47.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.819+0000 7fdaa24dd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda9c0b2660 0x7fda9c0abac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.819+0000 7fdaa24dd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda9c0d0020 0x7fda9c15d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.819+0000 7fdaa24dd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda9c15a6c0 0x7fda9c0b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.819+0000 7fdaa24dd700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c15da90 con 0x7fda9c0b2660 2023-11-28T11:09:47.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.819+0000 7fdaa24dd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c0adb10 con 0x7fda9c0d0020 2023-11-28T11:09:47.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.819+0000 7fdaa24dd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda9c1f5950 con 0x7fda9c15a6c0 2023-11-28T11:09:47.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.819+0000 7fda9bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda9c0b2660 0x7fda9c0abac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.819+0000 7fda9bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda9c0b2660 0x7fda9c0abac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48648/0 (socket says 172.21.15.28:48648) 2023-11-28T11:09:47.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.819+0000 7fda9bfff700 1 -- 172.21.15.28:0/1203964464 learned_addr learned my addr 172.21.15.28:0/1203964464 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:47.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.820+0000 7fda9bfff700 1 -- 172.21.15.28:0/1203964464 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda9c15a6c0 msgr2=0x7fda9c0b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.820+0000 7fda9bfff700 1 --2- 172.21.15.28:0/1203964464 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda9c15a6c0 0x7fda9c0b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.820+0000 7fda9bfff700 1 -- 172.21.15.28:0/1203964464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda9c0d0020 msgr2=0x7fda9c15d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:47.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.820+0000 7fda9bfff700 1 --2- 172.21.15.28:0/1203964464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda9c0d0020 0x7fda9c15d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:47.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.820+0000 7fda9bfff700 1 -- 172.21.15.28:0/1203964464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda900094a0 con 0x7fda9c0b2660 2023-11-28T11:09:47.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.820+0000 7fda9bfff700 1 --2- 172.21.15.28:0/1203964464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda9c0b2660 0x7fda9c0abac0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00c7f0 tx=0x7fda8c00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.820+0000 7fda98ff9700 1 -- 172.21.15.28:0/1203964464 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c011800 con 0x7fda9c0b2660 2023-11-28T11:09:47.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.820+0000 7fda98ff9700 1 -- 172.21.15.28:0/1203964464 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda8c011960 con 0x7fda9c0b2660 2023-11-28T11:09:47.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.820+0000 7fdaa24dd700 1 -- 172.21.15.28:0/1203964464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda9c0cd3e0 con 0x7fda9c0b2660 2023-11-28T11:09:47.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.820+0000 7fda98ff9700 1 -- 172.21.15.28:0/1203964464 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda8c011d10 con 0x7fda9c0b2660 2023-11-28T11:09:47.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.820+0000 7fdaa24dd700 1 -- 172.21.15.28:0/1203964464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda9c0cf820 con 0x7fda9c0b2660 2023-11-28T11:09:47.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.821+0000 7fdaa24dd700 1 -- 172.21.15.28:0/1203964464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda9c0d0470 con 0x7fda9c0b2660 2023-11-28T11:09:47.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.821+0000 7fda98ff9700 1 -- 172.21.15.28:0/1203964464 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fda8c021070 con 0x7fda9c0b2660 2023-11-28T11:09:47.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.822+0000 7fda98ff9700 1 --2- 172.21.15.28:0/1203964464 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fda880682f0 0x7fda8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:47.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.822+0000 7fda98ff9700 1 -- 172.21.15.28:0/1203964464 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fda8c085ec0 con 0x7fda9c0b2660 2023-11-28T11:09:47.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.822+0000 7fda9b7fe700 1 --2- 172.21.15.28:0/1203964464 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fda880682f0 0x7fda8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:47.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.822+0000 7fda9b7fe700 1 --2- 172.21.15.28:0/1203964464 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fda880682f0 0x7fda8806a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0b1a60 tx=0x7fda90010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:47.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:47.826+0000 7fda98ff9700 1 -- 172.21.15.28:0/1203964464 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fda8c0523f0 con 0x7fda9c0b2660 2023-11-28T11:09:48.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.035+0000 7fdaa24dd700 1 -- 172.21.15.28:0/1203964464 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fda9c0675a0 con 0x7fda9c0b2660 2023-11-28T11:09:48.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.035+0000 7fda98ff9700 1 -- 172.21.15.28:0/1203964464 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fda8c055a10 con 0x7fda9c0b2660 2023-11-28T11:09:48.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.037+0000 7fdaa24dd700 1 -- 172.21.15.28:0/1203964464 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fda880682f0 msgr2=0x7fda8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.037+0000 7fdaa24dd700 1 --2- 172.21.15.28:0/1203964464 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fda880682f0 0x7fda8806a7a0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0b1a60 tx=0x7fda90010000 comp rx=0 tx=0).stop 2023-11-28T11:09:48.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.037+0000 7fdaa24dd700 1 -- 172.21.15.28:0/1203964464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda9c0b2660 msgr2=0x7fda9c0abac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.037+0000 7fdaa24dd700 1 --2- 172.21.15.28:0/1203964464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda9c0b2660 0x7fda9c0abac0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fda8c00c7f0 tx=0x7fda8c00cbb0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.037+0000 7fdaa24dd700 1 -- 172.21.15.28:0/1203964464 shutdown_connections 2023-11-28T11:09:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.037+0000 7fdaa24dd700 1 --2- 172.21.15.28:0/1203964464 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fda880682f0 0x7fda8806a7a0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.037+0000 7fdaa24dd700 1 --2- 172.21.15.28:0/1203964464 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fda9c0b2660 0x7fda9c0abac0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.037+0000 7fdaa24dd700 1 --2- 172.21.15.28:0/1203964464 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fda9c15a6c0 0x7fda9c0b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.037+0000 7fdaa24dd700 1 --2- 172.21.15.28:0/1203964464 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fda9c0d0020 0x7fda9c15d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.037+0000 7fdaa24dd700 1 -- 172.21.15.28:0/1203964464 >> 172.21.15.28:0/1203964464 conn(0x7fda9c0cd660 msgr2=0x7fda9c0b5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.037+0000 7fdaa24dd700 1 -- 172.21.15.28:0/1203964464 shutdown_connections 2023-11-28T11:09:48.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.037+0000 7fdaa24dd700 1 -- 172.21.15.28:0/1203964464 wait complete. 2023-11-28T11:09:48.041 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2023-11-28T11:09:48.051 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi028.front.sepia.ceph.com... 2023-11-28T11:09:48.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:48.051 DEBUG:teuthology.orchestra.run.smithi028:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-11-28T11:09:48.089 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:48.090 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:09:48.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:48.090 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T11:09:48.148 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:09:48.148 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:48.169 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:09:48.169 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:48.231 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:48.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.355+0000 7fc4b5c9d700 1 Processor -- start 2023-11-28T11:09:48.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.355+0000 7fc4b5c9d700 1 -- start start 2023-11-28T11:09:48.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.355+0000 7fc4b5c9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc4b0158a30 0x7fc4b006aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.355+0000 7fc4b5c9d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc4b00ce150 0x7fc4b00ce5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.355+0000 7fc4b5c9d700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4b006ed80 con 0x7fc4b00ceb10 2023-11-28T11:09:48.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.356+0000 7fc4b5c9d700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b006f060 con 0x7fc4b0158a30 2023-11-28T11:09:48.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.356+0000 7fc4b5c9d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4b00cd950 con 0x7fc4b00ce150 2023-11-28T11:09:48.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.356+0000 7fc4af7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc4b0158a30 0x7fc4b006aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.356+0000 7fc4affff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc4b00ceb10 0x7fc4b00ca010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36718/0 (socket says 172.21.15.28:36718) 2023-11-28T11:09:48.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.356+0000 7fc4affff700 1 -- 172.21.15.28:0/322951983 learned_addr learned my addr 172.21.15.28:0/322951983 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:48.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.356+0000 7fc4aeffd700 1 --2- 172.21.15.28:0/322951983 >> v2:172.21.15.59:3300/0 conn(0x7fc4b00ce150 0x7fc4b00ce5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.356+0000 7fc4af7fe700 1 -- 172.21.15.28:0/322951983 >> v2:172.21.15.59:3300/0 conn(0x7fc4b00ce150 msgr2=0x7fc4b00ce5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:48.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.356+0000 7fc4af7fe700 1 --2- 172.21.15.28:0/322951983 >> v2:172.21.15.59:3300/0 conn(0x7fc4b00ce150 0x7fc4b00ce5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.356+0000 7fc4af7fe700 1 -- 172.21.15.28:0/322951983 >> v1:172.21.15.28:6789/0 conn(0x7fc4b00ceb10 legacy=0x7fc4b00ca010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.356+0000 7fc4ae7fc700 1 -- 172.21.15.28:0/322951983 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3196535119 0 0) 0x7fc4b006ed80 con 0x7fc4b00ceb10 2023-11-28T11:09:48.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.356+0000 7fc4af7fe700 1 -- 172.21.15.28:0/322951983 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b00cdc30 con 0x7fc4b0158a30 2023-11-28T11:09:48.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.356+0000 7fc4af7fe700 1 --2- 172.21.15.28:0/322951983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b0158a30 0x7fc4b006aa70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0071180 tx=0x7fc4a000acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6fe167ceb0d4dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:48.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.356+0000 7fc4ae7fc700 1 -- 172.21.15.28:0/322951983 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a000cb70 con 0x7fc4b0158a30 2023-11-28T11:09:48.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.356+0000 7fc4ae7fc700 1 -- 172.21.15.28:0/322951983 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a000ccd0 con 0x7fc4b0158a30 2023-11-28T11:09:48.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.356+0000 7fc4ae7fc700 1 -- 172.21.15.28:0/322951983 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a0018620 con 0x7fc4b0158a30 2023-11-28T11:09:48.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.357+0000 7fc4b5c9d700 1 -- 172.21.15.28:0/322951983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b0158a30 msgr2=0x7fc4b006aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.357+0000 7fc4b5c9d700 1 --2- 172.21.15.28:0/322951983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b0158a30 0x7fc4b006aa70 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0071180 tx=0x7fc4a000acc0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.357+0000 7fc4b5c9d700 1 -- 172.21.15.28:0/322951983 shutdown_connections 2023-11-28T11:09:48.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.357+0000 7fc4b5c9d700 1 --2- 172.21.15.28:0/322951983 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b0158a30 0x7fc4b006aa70 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.357+0000 7fc4b5c9d700 1 --2- 172.21.15.28:0/322951983 >> v2:172.21.15.59:3300/0 conn(0x7fc4b00ce150 0x7fc4b00ce5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.357+0000 7fc4b5c9d700 1 -- 172.21.15.28:0/322951983 >> 172.21.15.28:0/322951983 conn(0x7fc4b00b0dd0 msgr2=0x7fc4b00b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:48.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.357+0000 7fc4b5c9d700 1 -- 172.21.15.28:0/322951983 shutdown_connections 2023-11-28T11:09:48.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.357+0000 7fc4b5c9d700 1 -- 172.21.15.28:0/322951983 wait complete. 2023-11-28T11:09:48.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.358+0000 7fc4b5c9d700 1 Processor -- start 2023-11-28T11:09:48.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.358+0000 7fc4b5c9d700 1 -- start start 2023-11-28T11:09:48.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.359+0000 7fc4b5c9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4b00ce150 0x7fc4b01f02b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.359+0000 7fc4b5c9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b00ceb10 0x7fc4b01e3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.359+0000 7fc4b5c9d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4b0158a30 0x7fc4b01f4300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.359+0000 7fc4b5c9d700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b01e4000 con 0x7fc4b00ceb10 2023-11-28T11:09:48.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.359+0000 7fc4b5c9d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b014f7b0 con 0x7fc4b00ce150 2023-11-28T11:09:48.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.359+0000 7fc4b5c9d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4b01f4840 con 0x7fc4b0158a30 2023-11-28T11:09:48.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.359+0000 7fc4aeffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b00ceb10 0x7fc4b01e3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.359+0000 7fc4aeffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b00ceb10 0x7fc4b01e3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48664/0 (socket says 172.21.15.28:48664) 2023-11-28T11:09:48.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.359+0000 7fc4aeffd700 1 -- 172.21.15.28:0/2439231446 learned_addr learned my addr 172.21.15.28:0/2439231446 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:48.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.359+0000 7fc4af7fe700 1 --2- 172.21.15.28:0/2439231446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4b00ce150 0x7fc4b01f02b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.359+0000 7fc4aeffd700 1 -- 172.21.15.28:0/2439231446 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4b0158a30 msgr2=0x7fc4b01f4300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.359+0000 7fc4aeffd700 1 --2- 172.21.15.28:0/2439231446 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4b0158a30 0x7fc4b01f4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.359+0000 7fc4aeffd700 1 -- 172.21.15.28:0/2439231446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4b00ce150 msgr2=0x7fc4b01f02b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.359+0000 7fc4aeffd700 1 --2- 172.21.15.28:0/2439231446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4b00ce150 0x7fc4b01f02b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.359+0000 7fc4aeffd700 1 -- 172.21.15.28:0/2439231446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a000a640 con 0x7fc4b00ceb10 2023-11-28T11:09:48.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.360+0000 7fc4aeffd700 1 --2- 172.21.15.28:0/2439231446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b00ceb10 0x7fc4b01e3ac0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400c820 tx=0x7fc4a400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:48.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.360+0000 7fc497fff700 1 -- 172.21.15.28:0/2439231446 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4011800 con 0x7fc4b00ceb10 2023-11-28T11:09:48.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.360+0000 7fc497fff700 1 -- 172.21.15.28:0/2439231446 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4a4011960 con 0x7fc4b00ceb10 2023-11-28T11:09:48.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.360+0000 7fc4b5c9d700 1 -- 172.21.15.28:0/2439231446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4b006e9c0 con 0x7fc4b00ceb10 2023-11-28T11:09:48.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.360+0000 7fc497fff700 1 -- 172.21.15.28:0/2439231446 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4a4011d10 con 0x7fc4b00ceb10 2023-11-28T11:09:48.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.360+0000 7fc4b5c9d700 1 -- 172.21.15.28:0/2439231446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4b006f340 con 0x7fc4b00ceb10 2023-11-28T11:09:48.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.361+0000 7fc4b5c9d700 1 -- 172.21.15.28:0/2439231446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4b006eb50 con 0x7fc4b00ceb10 2023-11-28T11:09:48.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.361+0000 7fc497fff700 1 -- 172.21.15.28:0/2439231446 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc4a4021070 con 0x7fc4b00ceb10 2023-11-28T11:09:48.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.362+0000 7fc497fff700 1 --2- 172.21.15.28:0/2439231446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc49c0682f0 0x7fc49c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.362+0000 7fc4af7fe700 1 --2- 172.21.15.28:0/2439231446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc49c0682f0 0x7fc49c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.362+0000 7fc497fff700 1 -- 172.21.15.28:0/2439231446 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fc4a4085e50 con 0x7fc4b00ceb10 2023-11-28T11:09:48.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.362+0000 7fc4af7fe700 1 --2- 172.21.15.28:0/2439231446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc49c0682f0 0x7fc49c06a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0155bf0 tx=0x7fc4a0012000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:48.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.366+0000 7fc497fff700 1 -- 172.21.15.28:0/2439231446 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc4a4052380 con 0x7fc4b00ceb10 2023-11-28T11:09:48.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.561+0000 7fc4b5c9d700 1 -- 172.21.15.28:0/2439231446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4b014f980 con 0x7fc4b00ceb10 2023-11-28T11:09:48.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.563+0000 7fc497fff700 1 -- 172.21.15.28:0/2439231446 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19737 (secure 0 0 0) 0x7fc4b014f980 con 0x7fc4b00ceb10 2023-11-28T11:09:48.564 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:48.565 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:09:34.614188+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:48.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.565+0000 7fc4b5c9d700 1 -- 172.21.15.28:0/2439231446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc49c0682f0 msgr2=0x7fc49c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.565+0000 7fc4b5c9d700 1 --2- 172.21.15.28:0/2439231446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc49c0682f0 0x7fc49c06a7a0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0155bf0 tx=0x7fc4a0012000 comp rx=0 tx=0).stop 2023-11-28T11:09:48.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.565+0000 7fc4b5c9d700 1 -- 172.21.15.28:0/2439231446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b00ceb10 msgr2=0x7fc4b01e3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.565+0000 7fc4b5c9d700 1 --2- 172.21.15.28:0/2439231446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b00ceb10 0x7fc4b01e3ac0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc4a400c820 tx=0x7fc4a400cbe0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.565+0000 7fc4b5c9d700 1 -- 172.21.15.28:0/2439231446 shutdown_connections 2023-11-28T11:09:48.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.565+0000 7fc4b5c9d700 1 --2- 172.21.15.28:0/2439231446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc49c0682f0 0x7fc49c06a7a0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.565+0000 7fc4b5c9d700 1 --2- 172.21.15.28:0/2439231446 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4b0158a30 0x7fc4b01f4300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.565+0000 7fc4b5c9d700 1 --2- 172.21.15.28:0/2439231446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4b00ce150 0x7fc4b01f02b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.565+0000 7fc4b5c9d700 1 --2- 172.21.15.28:0/2439231446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc4b00ceb10 0x7fc4b01e3ac0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.565+0000 7fc4b5c9d700 1 -- 172.21.15.28:0/2439231446 >> 172.21.15.28:0/2439231446 conn(0x7fc4b00b0dd0 msgr2=0x7fc4b00cab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:48.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.565+0000 7fc4b5c9d700 1 -- 172.21.15.28:0/2439231446 shutdown_connections 2023-11-28T11:09:48.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.565+0000 7fc4b5c9d700 1 -- 172.21.15.28:0/2439231446 wait complete. 2023-11-28T11:09:48.581 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:48.582 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:48.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.708+0000 7efed0cba700 1 Processor -- start 2023-11-28T11:09:48.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.708+0000 7efed0cba700 1 -- start start 2023-11-28T11:09:48.712 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.709+0000 7efed0cba700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efecc06be20 0x7efecc0cf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.712 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.709+0000 7efed0cba700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc06c1f0 0x7efecc160ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.709+0000 7efed0cba700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efecc0cd390 con 0x7efecc06a1e0 2023-11-28T11:09:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.709+0000 7efed0cba700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efecc161010 con 0x7efecc06be20 2023-11-28T11:09:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.709+0000 7efed0cba700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0cd630 con 0x7efecc06c1f0 2023-11-28T11:09:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.709+0000 7efeca59c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efecc06be20 0x7efecc0cf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.709+0000 7efec9d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc06c1f0 0x7efecc160ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.709+0000 7efeca59c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efecc06be20 0x7efecc0cf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:35960/0 (socket says 172.21.15.28:35960) 2023-11-28T11:09:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.709+0000 7efec9d9b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc06c1f0 0x7efecc160ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:54428/0 (socket says 172.21.15.28:54428) 2023-11-28T11:09:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.709+0000 7efecad9d700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7efecc06a1e0 0x7efecc06a660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:47518/0 (socket says 172.21.15.28:47518) 2023-11-28T11:09:48.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.709+0000 7efeca59c700 1 -- 172.21.15.28:0/2286980605 learned_addr learned my addr 172.21.15.28:0/2286980605 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.710+0000 7efeca59c700 1 -- 172.21.15.28:0/2286980605 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc06c1f0 msgr2=0x7efecc160ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.710+0000 7efeca59c700 1 --2- 172.21.15.28:0/2286980605 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc06c1f0 0x7efecc160ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.710+0000 7efeca59c700 1 -- 172.21.15.28:0/2286980605 >> v1:172.21.15.59:6789/0 conn(0x7efecc06a1e0 legacy=0x7efecc06a660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.710+0000 7efec959a700 1 -- 172.21.15.28:0/2286980605 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 939625294 0 0) 0x7efecc0cd390 con 0x7efecc06a1e0 2023-11-28T11:09:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.710+0000 7efeca59c700 1 -- 172.21.15.28:0/2286980605 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0cd910 con 0x7efecc06be20 2023-11-28T11:09:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.710+0000 7efec9d9b700 1 --2- 172.21.15.28:0/2286980605 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc06c1f0 0x7efecc160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.710+0000 7efeca59c700 1 --2- 172.21.15.28:0/2286980605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc06be20 0x7efecc0cf190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efec00098f0 tx=0x7efec0009c00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1802f0a87203bba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.710+0000 7efec959a700 1 -- 172.21.15.28:0/2286980605 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0011830 con 0x7efecc06be20 2023-11-28T11:09:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.710+0000 7efec959a700 1 -- 172.21.15.28:0/2286980605 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec0011990 con 0x7efecc06be20 2023-11-28T11:09:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.710+0000 7efec959a700 1 -- 172.21.15.28:0/2286980605 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0011d20 con 0x7efecc06be20 2023-11-28T11:09:48.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.711+0000 7efed0cba700 1 -- 172.21.15.28:0/2286980605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc06be20 msgr2=0x7efecc0cf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.711+0000 7efed0cba700 1 --2- 172.21.15.28:0/2286980605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc06be20 0x7efecc0cf190 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7efec00098f0 tx=0x7efec0009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:48.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.711+0000 7efed0cba700 1 -- 172.21.15.28:0/2286980605 shutdown_connections 2023-11-28T11:09:48.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.711+0000 7efed0cba700 1 --2- 172.21.15.28:0/2286980605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc06be20 0x7efecc0cf190 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.711+0000 7efed0cba700 1 --2- 172.21.15.28:0/2286980605 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc06c1f0 0x7efecc160ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.711+0000 7efed0cba700 1 -- 172.21.15.28:0/2286980605 >> 172.21.15.28:0/2286980605 conn(0x7efecc0d03b0 msgr2=0x7efecc0d2ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:48.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.711+0000 7efed0cba700 1 -- 172.21.15.28:0/2286980605 shutdown_connections 2023-11-28T11:09:48.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.711+0000 7efed0cba700 1 -- 172.21.15.28:0/2286980605 wait complete. 2023-11-28T11:09:48.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.712+0000 7efed0cba700 1 Processor -- start 2023-11-28T11:09:48.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.712+0000 7efed0cba700 1 -- start start 2023-11-28T11:09:48.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efed0cba700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efecc06a1e0 0x7efecc1512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efed0cba700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc06be20 0x7efecc0c7a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efed0cba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc06c1f0 0x7efecc0b2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efed0cba700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0cadf0 con 0x7efecc06a1e0 2023-11-28T11:09:48.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efed0cba700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0caac0 con 0x7efecc06c1f0 2023-11-28T11:09:48.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efed0cba700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efecc0c8480 con 0x7efecc06be20 2023-11-28T11:09:48.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efeca59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efecc06a1e0 0x7efecc1512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efeca59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efecc06a1e0 0x7efecc1512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48676/0 (socket says 172.21.15.28:48676) 2023-11-28T11:09:48.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efeca59c700 1 -- 172.21.15.28:0/1598711328 learned_addr learned my addr 172.21.15.28:0/1598711328 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:48.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efeca59c700 1 -- 172.21.15.28:0/1598711328 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc06be20 msgr2=0x7efecc0c7a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efeca59c700 1 --2- 172.21.15.28:0/1598711328 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc06be20 0x7efecc0c7a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efeca59c700 1 -- 172.21.15.28:0/1598711328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc06c1f0 msgr2=0x7efecc0b2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efecad9d700 1 --2- 172.21.15.28:0/1598711328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc06c1f0 0x7efecc0b2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efeca59c700 1 --2- 172.21.15.28:0/1598711328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc06c1f0 0x7efecc0b2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efeca59c700 1 -- 172.21.15.28:0/1598711328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efebc0094a0 con 0x7efecc06a1e0 2023-11-28T11:09:48.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.713+0000 7efecad9d700 1 --2- 172.21.15.28:0/1598711328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc06c1f0 0x7efecc0b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:48.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.714+0000 7efeca59c700 1 --2- 172.21.15.28:0/1598711328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efecc06a1e0 0x7efecc1512b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efec0009c40 tx=0x7efec0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:48.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.714+0000 7efebaffd700 1 -- 172.21.15.28:0/1598711328 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec0004480 con 0x7efecc06a1e0 2023-11-28T11:09:48.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.714+0000 7efebaffd700 1 -- 172.21.15.28:0/1598711328 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efec001f070 con 0x7efecc06a1e0 2023-11-28T11:09:48.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.714+0000 7efed0cba700 1 -- 172.21.15.28:0/1598711328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efec0009550 con 0x7efecc06a1e0 2023-11-28T11:09:48.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.714+0000 7efebaffd700 1 -- 172.21.15.28:0/1598711328 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efec00038a0 con 0x7efecc06a1e0 2023-11-28T11:09:48.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.714+0000 7efed0cba700 1 -- 172.21.15.28:0/1598711328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efecc0d0e80 con 0x7efecc06a1e0 2023-11-28T11:09:48.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.715+0000 7efed0cba700 1 -- 172.21.15.28:0/1598711328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efecc0d0ba0 con 0x7efecc06a1e0 2023-11-28T11:09:48.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.715+0000 7efebaffd700 1 -- 172.21.15.28:0/1598711328 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efec002a020 con 0x7efecc06a1e0 2023-11-28T11:09:48.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.716+0000 7efebaffd700 1 --2- 172.21.15.28:0/1598711328 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efeac0682f0 0x7efeac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:48.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.716+0000 7efebaffd700 1 -- 172.21.15.28:0/1598711328 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7efec0087610 con 0x7efecc06a1e0 2023-11-28T11:09:48.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.716+0000 7efec9d9b700 1 --2- 172.21.15.28:0/1598711328 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efeac0682f0 0x7efeac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:48.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.716+0000 7efec9d9b700 1 --2- 172.21.15.28:0/1598711328 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efeac0682f0 0x7efeac06a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efebc006f10 tx=0x7efebc006e60 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:48.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.720+0000 7efebaffd700 1 -- 172.21.15.28:0/1598711328 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efec0053b40 con 0x7efecc06a1e0 2023-11-28T11:09:48.913 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.911+0000 7efed0cba700 1 -- 172.21.15.28:0/1598711328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efecc0b8830 con 0x7efecc06a1e0 2023-11-28T11:09:48.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.911+0000 7efebaffd700 1 -- 172.21.15.28:0/1598711328 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7efec0057160 con 0x7efecc06a1e0 2023-11-28T11:09:48.959 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:09:48.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.913+0000 7efed0cba700 1 -- 172.21.15.28:0/1598711328 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efeac0682f0 msgr2=0x7efeac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.913+0000 7efed0cba700 1 --2- 172.21.15.28:0/1598711328 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efeac0682f0 0x7efeac06a7a0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7efebc006f10 tx=0x7efebc006e60 comp rx=0 tx=0).stop 2023-11-28T11:09:48.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.913+0000 7efed0cba700 1 -- 172.21.15.28:0/1598711328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efecc06a1e0 msgr2=0x7efecc1512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:48.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.913+0000 7efed0cba700 1 --2- 172.21.15.28:0/1598711328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efecc06a1e0 0x7efecc1512b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7efec0009c40 tx=0x7efec0004750 comp rx=0 tx=0).stop 2023-11-28T11:09:48.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.913+0000 7efed0cba700 1 -- 172.21.15.28:0/1598711328 shutdown_connections 2023-11-28T11:09:48.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.913+0000 7efed0cba700 1 --2- 172.21.15.28:0/1598711328 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efeac0682f0 0x7efeac06a7a0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.913+0000 7efed0cba700 1 --2- 172.21.15.28:0/1598711328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efecc06a1e0 0x7efecc1512b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.913+0000 7efed0cba700 1 --2- 172.21.15.28:0/1598711328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efecc06c1f0 0x7efecc0b2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.913+0000 7efed0cba700 1 --2- 172.21.15.28:0/1598711328 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efecc06be20 0x7efecc0c7a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:48.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.914+0000 7efed0cba700 1 -- 172.21.15.28:0/1598711328 >> 172.21.15.28:0/1598711328 conn(0x7efecc0d03b0 msgr2=0x7efecc15c210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:48.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.914+0000 7efed0cba700 1 -- 172.21.15.28:0/1598711328 shutdown_connections 2023-11-28T11:09:48.962 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:48.914+0000 7efed0cba700 1 -- 172.21.15.28:0/1598711328 wait complete. 2023-11-28T11:09:48.962 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:49.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.088+0000 7fe10843b700 1 Processor -- start 2023-11-28T11:09:49.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.088+0000 7fe10843b700 1 -- start start 2023-11-28T11:09:49.092 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.089+0000 7fe10843b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe1000b1630 0x7fe1000cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.092 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.089+0000 7fe10843b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe10006afc0 0x7fe10006b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.092 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.089+0000 7fe10843b700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1000ce550 con 0x7fe1000b1a00 2023-11-28T11:09:49.092 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.089+0000 7fe10843b700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1000ce830 con 0x7fe1000b1630 2023-11-28T11:09:49.092 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.089+0000 7fe10843b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1000ceb10 con 0x7fe10006afc0 2023-11-28T11:09:49.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.089+0000 7fe1061d7700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe1000b1630 0x7fe1000cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.089+0000 7fe1059d6700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe1000b1a00 0x7fe1000caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36742/0 (socket says 172.21.15.28:36742) 2023-11-28T11:09:49.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.089+0000 7fe1059d6700 1 -- 172.21.15.28:0/2182438247 learned_addr learned my addr 172.21.15.28:0/2182438247 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:49.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.089+0000 7fe1069d8700 1 --2- 172.21.15.28:0/2182438247 >> v2:172.21.15.59:3300/0 conn(0x7fe10006afc0 0x7fe10006b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.090+0000 7fe1061d7700 1 -- 172.21.15.28:0/2182438247 >> v2:172.21.15.59:3300/0 conn(0x7fe10006afc0 msgr2=0x7fe10006b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:49.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.090+0000 7fe1061d7700 1 --2- 172.21.15.28:0/2182438247 >> v2:172.21.15.59:3300/0 conn(0x7fe10006afc0 0x7fe10006b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.090+0000 7fe1061d7700 1 -- 172.21.15.28:0/2182438247 >> v1:172.21.15.28:6789/0 conn(0x7fe1000b1a00 legacy=0x7fe1000caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.090+0000 7fe1061d7700 1 -- 172.21.15.28:0/2182438247 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1000cedf0 con 0x7fe1000b1630 2023-11-28T11:09:49.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.090+0000 7fe1061d7700 1 --2- 172.21.15.28:0/2182438247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1000b1630 0x7fe1000cf5f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009840 tx=0x7fe0f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dcb1870aa37544a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:49.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.090+0000 7fe1051d5700 1 -- 172.21.15.28:0/2182438247 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0011830 con 0x7fe1000b1630 2023-11-28T11:09:49.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.090+0000 7fe1051d5700 1 -- 172.21.15.28:0/2182438247 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f0011990 con 0x7fe1000b1630 2023-11-28T11:09:49.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.090+0000 7fe1051d5700 1 -- 172.21.15.28:0/2182438247 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0011d20 con 0x7fe1000b1630 2023-11-28T11:09:49.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.090+0000 7fe10843b700 1 -- 172.21.15.28:0/2182438247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1000b1630 msgr2=0x7fe1000cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.090+0000 7fe10843b700 1 --2- 172.21.15.28:0/2182438247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1000b1630 0x7fe1000cf5f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7fe0f0009840 tx=0x7fe0f0009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:49.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.091+0000 7fe10843b700 1 -- 172.21.15.28:0/2182438247 shutdown_connections 2023-11-28T11:09:49.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.091+0000 7fe10843b700 1 --2- 172.21.15.28:0/2182438247 >> v2:172.21.15.59:3300/0 conn(0x7fe10006afc0 0x7fe10006b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.091+0000 7fe10843b700 1 --2- 172.21.15.28:0/2182438247 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1000b1630 0x7fe1000cf5f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.091+0000 7fe10843b700 1 -- 172.21.15.28:0/2182438247 >> 172.21.15.28:0/2182438247 conn(0x7fe1000cc630 msgr2=0x7fe1000cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:49.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.091+0000 7fe10843b700 1 -- 172.21.15.28:0/2182438247 shutdown_connections 2023-11-28T11:09:49.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.091+0000 7fe10843b700 1 -- 172.21.15.28:0/2182438247 wait complete. 2023-11-28T11:09:49.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.092+0000 7fe10843b700 1 Processor -- start 2023-11-28T11:09:49.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.092+0000 7fe10843b700 1 -- start start 2023-11-28T11:09:49.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe10843b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe10006afc0 0x7fe1000b4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe10843b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1000b1630 0x7fe1000b49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe10843b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1000b1a00 0x7fe1001f2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe10843b700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1001f39c0 con 0x7fe10006afc0 2023-11-28T11:09:49.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe10843b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1000b51e0 con 0x7fe1000b1a00 2023-11-28T11:09:49.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe10843b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1001f3bd0 con 0x7fe1000b1630 2023-11-28T11:09:49.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe1061d7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe10006afc0 0x7fe1000b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe1061d7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe10006afc0 0x7fe1000b4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48690/0 (socket says 172.21.15.28:48690) 2023-11-28T11:09:49.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe1061d7700 1 -- 172.21.15.28:0/990244828 learned_addr learned my addr 172.21.15.28:0/990244828 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:49.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe1061d7700 1 -- 172.21.15.28:0/990244828 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1000b1630 msgr2=0x7fe1000b49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe1061d7700 1 --2- 172.21.15.28:0/990244828 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1000b1630 0x7fe1000b49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe1061d7700 1 -- 172.21.15.28:0/990244828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1000b1a00 msgr2=0x7fe1001f2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe1061d7700 1 --2- 172.21.15.28:0/990244828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1000b1a00 0x7fe1001f2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe1061d7700 1 -- 172.21.15.28:0/990244828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0f00094a0 con 0x7fe10006afc0 2023-11-28T11:09:49.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe1061d7700 1 --2- 172.21.15.28:0/990244828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe10006afc0 0x7fe1000b4460 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00090b0 tx=0x7fe0f0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:49.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe0f6ffd700 1 -- 172.21.15.28:0/990244828 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f00038a0 con 0x7fe10006afc0 2023-11-28T11:09:49.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe0f6ffd700 1 -- 172.21.15.28:0/990244828 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0f001f070 con 0x7fe10006afc0 2023-11-28T11:09:49.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe10843b700 1 -- 172.21.15.28:0/990244828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1000b1ea0 con 0x7fe10006afc0 2023-11-28T11:09:49.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe0f6ffd700 1 -- 172.21.15.28:0/990244828 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0f0003c20 con 0x7fe10006afc0 2023-11-28T11:09:49.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.093+0000 7fe10843b700 1 -- 172.21.15.28:0/990244828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1000cf0d0 con 0x7fe10006afc0 2023-11-28T11:09:49.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.094+0000 7fe10843b700 1 -- 172.21.15.28:0/990244828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1000cb5b0 con 0x7fe10006afc0 2023-11-28T11:09:49.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.095+0000 7fe0f6ffd700 1 -- 172.21.15.28:0/990244828 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe0f000f050 con 0x7fe10006afc0 2023-11-28T11:09:49.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.095+0000 7fe0f6ffd700 1 --2- 172.21.15.28:0/990244828 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0e40682f0 0x7fe0e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.095+0000 7fe0f6ffd700 1 -- 172.21.15.28:0/990244828 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fe0f0021070 con 0x7fe10006afc0 2023-11-28T11:09:49.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.095+0000 7fe1059d6700 1 --2- 172.21.15.28:0/990244828 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0e40682f0 0x7fe0e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.096+0000 7fe1059d6700 1 --2- 172.21.15.28:0/990244828 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0e40682f0 0x7fe0e406a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0015d0 tx=0x7fe0fc007ad0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:49.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.099+0000 7fe0f6ffd700 1 -- 172.21.15.28:0/990244828 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe0f0053f00 con 0x7fe10006afc0 2023-11-28T11:09:49.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.290+0000 7fe10843b700 1 -- 172.21.15.28:0/990244828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1000739f0 con 0x7fe10006afc0 2023-11-28T11:09:49.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.290+0000 7fe0f6ffd700 1 -- 172.21.15.28:0/990244828 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0f0057520 con 0x7fe10006afc0 2023-11-28T11:09:49.292 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:09:49.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.293+0000 7fe10843b700 1 -- 172.21.15.28:0/990244828 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0e40682f0 msgr2=0x7fe0e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.293+0000 7fe10843b700 1 --2- 172.21.15.28:0/990244828 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0e40682f0 0x7fe0e406a7a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe0fc0015d0 tx=0x7fe0fc007ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.293+0000 7fe10843b700 1 -- 172.21.15.28:0/990244828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe10006afc0 msgr2=0x7fe1000b4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.293+0000 7fe10843b700 1 --2- 172.21.15.28:0/990244828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe10006afc0 0x7fe1000b4460 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fe0f00090b0 tx=0x7fe0f0004750 comp rx=0 tx=0).stop 2023-11-28T11:09:49.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.293+0000 7fe10843b700 1 -- 172.21.15.28:0/990244828 shutdown_connections 2023-11-28T11:09:49.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.293+0000 7fe10843b700 1 --2- 172.21.15.28:0/990244828 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0e40682f0 0x7fe0e406a7a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.293+0000 7fe10843b700 1 --2- 172.21.15.28:0/990244828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe10006afc0 0x7fe1000b4460 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.293+0000 7fe10843b700 1 --2- 172.21.15.28:0/990244828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1000b1a00 0x7fe1001f2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.293+0000 7fe10843b700 1 --2- 172.21.15.28:0/990244828 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe1000b1630 0x7fe1000b49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.293+0000 7fe10843b700 1 -- 172.21.15.28:0/990244828 >> 172.21.15.28:0/990244828 conn(0x7fe1000cc630 msgr2=0x7fe1000ab950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:49.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.293+0000 7fe10843b700 1 -- 172.21.15.28:0/990244828 shutdown_connections 2023-11-28T11:09:49.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.293+0000 7fe10843b700 1 -- 172.21.15.28:0/990244828 wait complete. 2023-11-28T11:09:49.307 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:49.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.433+0000 7fed3a579700 1 Processor -- start 2023-11-28T11:09:49.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.434+0000 7fed3a579700 1 -- start start 2023-11-28T11:09:49.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.434+0000 7fed3a579700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fed340b1630 0x7fed340cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.434+0000 7fed3a579700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fed340b1a00 0x7fed340caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.434+0000 7fed3a579700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed340cb3e0 0x7fed34160350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.434+0000 7fed3a579700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed340ce550 con 0x7fed340b1a00 2023-11-28T11:09:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.434+0000 7fed3a579700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed340ce830 con 0x7fed340b1630 2023-11-28T11:09:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.434+0000 7fed3a579700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed340ceb10 con 0x7fed340cb3e0 2023-11-28T11:09:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.434+0000 7fed337fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fed340b1a00 0x7fed340caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.435+0000 7fed337fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fed340b1a00 0x7fed340caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48700/0 (socket says 172.21.15.28:48700) 2023-11-28T11:09:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.435+0000 7fed337fe700 1 -- 172.21.15.28:0/431751607 learned_addr learned my addr 172.21.15.28:0/431751607 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.435+0000 7fed38b16700 1 --2- 172.21.15.28:0/431751607 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed340cb3e0 0x7fed34160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.435+0000 7fed33fff700 1 --2- 172.21.15.28:0/431751607 >> v2:172.21.15.59:3300/0 conn(0x7fed340b1630 0x7fed340cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.435+0000 7fed337fe700 1 -- 172.21.15.28:0/431751607 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed340cb3e0 msgr2=0x7fed34160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.435+0000 7fed337fe700 1 --2- 172.21.15.28:0/431751607 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed340cb3e0 0x7fed34160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.435+0000 7fed337fe700 1 -- 172.21.15.28:0/431751607 >> v2:172.21.15.59:3300/0 conn(0x7fed340b1630 msgr2=0x7fed340cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.435+0000 7fed337fe700 1 --2- 172.21.15.28:0/431751607 >> v2:172.21.15.59:3300/0 conn(0x7fed340b1630 0x7fed340cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.435+0000 7fed337fe700 1 -- 172.21.15.28:0/431751607 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed340cedf0 con 0x7fed340b1a00 2023-11-28T11:09:49.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.435+0000 7fed337fe700 1 --2- 172.21.15.28:0/431751607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed340b1a00 0x7fed340caea0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fed280098f0 tx=0x7fed28009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bbc2a26adf83d81 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:49.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.435+0000 7fed32ffd700 1 -- 172.21.15.28:0/431751607 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2800b860 con 0x7fed340b1a00 2023-11-28T11:09:49.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.435+0000 7fed32ffd700 1 -- 172.21.15.28:0/431751607 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2800b9c0 con 0x7fed340b1a00 2023-11-28T11:09:49.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.435+0000 7fed32ffd700 1 -- 172.21.15.28:0/431751607 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed2800bd50 con 0x7fed340b1a00 2023-11-28T11:09:49.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.436+0000 7fed3a579700 1 -- 172.21.15.28:0/431751607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed340b1a00 msgr2=0x7fed340caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.436+0000 7fed3a579700 1 --2- 172.21.15.28:0/431751607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed340b1a00 0x7fed340caea0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fed280098f0 tx=0x7fed28009c00 comp rx=0 tx=0).stop 2023-11-28T11:09:49.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.436+0000 7fed3a579700 1 -- 172.21.15.28:0/431751607 shutdown_connections 2023-11-28T11:09:49.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.436+0000 7fed3a579700 1 --2- 172.21.15.28:0/431751607 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed340cb3e0 0x7fed34160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.436+0000 7fed3a579700 1 --2- 172.21.15.28:0/431751607 >> v2:172.21.15.59:3300/0 conn(0x7fed340b1630 0x7fed340cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.436+0000 7fed3a579700 1 --2- 172.21.15.28:0/431751607 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed340b1a00 0x7fed340caea0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.436+0000 7fed3a579700 1 -- 172.21.15.28:0/431751607 >> 172.21.15.28:0/431751607 conn(0x7fed340cc630 msgr2=0x7fed340cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:49.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.436+0000 7fed3a579700 1 -- 172.21.15.28:0/431751607 shutdown_connections 2023-11-28T11:09:49.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.436+0000 7fed3a579700 1 -- 172.21.15.28:0/431751607 wait complete. 2023-11-28T11:09:49.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.437+0000 7fed3a579700 1 Processor -- start 2023-11-28T11:09:49.444 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.437+0000 7fed3a579700 1 -- start start 2023-11-28T11:09:49.444 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed3a579700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed340b1630 0x7fed341f1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.444 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed3a579700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed340b1a00 0x7fed340ac2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.444 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed3a579700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed340cb3e0 0x7fed341e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.444 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed3a579700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed341f20e0 con 0x7fed340b1a00 2023-11-28T11:09:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed3a579700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed340b3c40 con 0x7fed340cb3e0 2023-11-28T11:09:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed3a579700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed340b3f20 con 0x7fed340b1630 2023-11-28T11:09:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed337fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed340b1a00 0x7fed340ac2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed337fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed340b1a00 0x7fed340ac2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48706/0 (socket says 172.21.15.28:48706) 2023-11-28T11:09:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed337fe700 1 -- 172.21.15.28:0/2872112821 learned_addr learned my addr 172.21.15.28:0/2872112821 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed38b16700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed340cb3e0 0x7fed341e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed33fff700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed340b1630 0x7fed341f1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed337fe700 1 -- 172.21.15.28:0/2872112821 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed340b1630 msgr2=0x7fed341f1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.445 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed337fe700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed340b1630 0x7fed341f1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed337fe700 1 -- 172.21.15.28:0/2872112821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed340cb3e0 msgr2=0x7fed341e65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed337fe700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed340cb3e0 0x7fed341e65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed337fe700 1 -- 172.21.15.28:0/2872112821 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed28009550 con 0x7fed340b1a00 2023-11-28T11:09:49.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed38b16700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed340cb3e0 0x7fed341e65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:49.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.438+0000 7fed33fff700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed340b1630 0x7fed341f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:49.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.439+0000 7fed337fe700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed340b1a00 0x7fed340ac2d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fed28009c40 tx=0x7fed28004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:49.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.439+0000 7fed30ff9700 1 -- 172.21.15.28:0/2872112821 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28004380 con 0x7fed340b1a00 2023-11-28T11:09:49.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.439+0000 7fed30ff9700 1 -- 172.21.15.28:0/2872112821 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed2801f070 con 0x7fed340b1a00 2023-11-28T11:09:49.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.439+0000 7fed3a579700 1 -- 172.21.15.28:0/2872112821 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed340b1ea0 con 0x7fed340b1a00 2023-11-28T11:09:49.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.439+0000 7fed30ff9700 1 -- 172.21.15.28:0/2872112821 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280036a0 con 0x7fed340b1a00 2023-11-28T11:09:49.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.439+0000 7fed3a579700 1 -- 172.21.15.28:0/2872112821 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed340cf0d0 con 0x7fed340b1a00 2023-11-28T11:09:49.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.440+0000 7fed3a579700 1 -- 172.21.15.28:0/2872112821 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed340ce140 con 0x7fed340b1a00 2023-11-28T11:09:49.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.440+0000 7fed30ff9700 1 -- 172.21.15.28:0/2872112821 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fed2802a020 con 0x7fed340b1a00 2023-11-28T11:09:49.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.440+0000 7fed30ff9700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed200682f0 0x7fed2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.441+0000 7fed30ff9700 1 -- 172.21.15.28:0/2872112821 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fed28087560 con 0x7fed340b1a00 2023-11-28T11:09:49.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.441+0000 7fed33fff700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed200682f0 0x7fed2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.441+0000 7fed33fff700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed200682f0 0x7fed2006a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fed240099e0 tx=0x7fed24008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:49.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.445+0000 7fed30ff9700 1 -- 172.21.15.28:0/2872112821 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fed28053a90 con 0x7fed340b1a00 2023-11-28T11:09:49.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.638+0000 7fed3a579700 1 -- 172.21.15.28:0/2872112821 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fed340739f0 con 0x7fed340b1a00 2023-11-28T11:09:49.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.638+0000 7fed30ff9700 1 -- 172.21.15.28:0/2872112821 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fed280570b0 con 0x7fed340b1a00 2023-11-28T11:09:49.640 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:49.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.640+0000 7fed3a579700 1 -- 172.21.15.28:0/2872112821 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed200682f0 msgr2=0x7fed2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.640+0000 7fed3a579700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed200682f0 0x7fed2006a7a0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fed240099e0 tx=0x7fed24008040 comp rx=0 tx=0).stop 2023-11-28T11:09:49.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.640+0000 7fed3a579700 1 -- 172.21.15.28:0/2872112821 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed340b1a00 msgr2=0x7fed340ac2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.640+0000 7fed3a579700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed340b1a00 0x7fed340ac2d0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fed28009c40 tx=0x7fed28004750 comp rx=0 tx=0).stop 2023-11-28T11:09:49.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.640+0000 7fed3a579700 1 -- 172.21.15.28:0/2872112821 shutdown_connections 2023-11-28T11:09:49.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.640+0000 7fed3a579700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed200682f0 0x7fed2006a7a0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.640+0000 7fed3a579700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed340cb3e0 0x7fed341e65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.640+0000 7fed3a579700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed340b1630 0x7fed341f1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.640+0000 7fed3a579700 1 --2- 172.21.15.28:0/2872112821 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed340b1a00 0x7fed340ac2d0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.640+0000 7fed3a579700 1 -- 172.21.15.28:0/2872112821 >> 172.21.15.28:0/2872112821 conn(0x7fed340cc630 msgr2=0x7fed340cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:49.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.640+0000 7fed3a579700 1 -- 172.21.15.28:0/2872112821 shutdown_connections 2023-11-28T11:09:49.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.640+0000 7fed3a579700 1 -- 172.21.15.28:0/2872112821 wait complete. 2023-11-28T11:09:49.654 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:49.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.781+0000 7f8cc69de700 1 Processor -- start 2023-11-28T11:09:49.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.781+0000 7f8cc69de700 1 -- start start 2023-11-28T11:09:49.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.782+0000 7f8cc69de700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cc00b62d0 0x7f8cc00cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.782+0000 7f8cc69de700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8cc006f8b0 0x7f8cc00b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.782+0000 7f8cc69de700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cc00b3020 con 0x7f8cc00b2bc0 2023-11-28T11:09:49.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.782+0000 7f8cc69de700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cc00cd2a0 con 0x7f8cc006f8b0 2023-11-28T11:09:49.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.782+0000 7f8cc69de700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc00d0400 con 0x7f8cc00b62d0 2023-11-28T11:09:49.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.782+0000 7f8cbf7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8cc006f8b0 0x7f8cc00b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.782+0000 7f8cc4f7b700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8cc00b2bc0 0x7f8cc00ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36774/0 (socket says 172.21.15.28:36774) 2023-11-28T11:09:49.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.782+0000 7f8cc4f7b700 1 -- 172.21.15.28:0/1066582011 learned_addr learned my addr 172.21.15.28:0/1066582011 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:49.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.782+0000 7f8cbffff700 1 --2- 172.21.15.28:0/1066582011 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cc00b62d0 0x7f8cc00cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.782+0000 7f8cbf7fe700 1 -- 172.21.15.28:0/1066582011 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cc00b62d0 msgr2=0x7f8cc00cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:49.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.782+0000 7f8cbf7fe700 1 --2- 172.21.15.28:0/1066582011 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cc00b62d0 0x7f8cc00cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.783+0000 7f8cbf7fe700 1 -- 172.21.15.28:0/1066582011 >> v1:172.21.15.28:6789/0 conn(0x7f8cc00b2bc0 legacy=0x7f8cc00ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.783+0000 7f8cbeffd700 1 -- 172.21.15.28:0/1066582011 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3665640227 0 0) 0x7f8cc00b3020 con 0x7f8cc00b2bc0 2023-11-28T11:09:49.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.783+0000 7f8cbf7fe700 1 -- 172.21.15.28:0/1066582011 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc00d06e0 con 0x7f8cc006f8b0 2023-11-28T11:09:49.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.783+0000 7f8cbf7fe700 1 --2- 172.21.15.28:0/1066582011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cc006f8b0 0x7f8cc00b2680 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009840 tx=0x7f8cb0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a027afa11a5ad902 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:49.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.783+0000 7f8cbeffd700 1 -- 172.21.15.28:0/1066582011 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0011830 con 0x7f8cc006f8b0 2023-11-28T11:09:49.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.783+0000 7f8cbeffd700 1 -- 172.21.15.28:0/1066582011 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb0011990 con 0x7f8cc006f8b0 2023-11-28T11:09:49.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.783+0000 7f8cbeffd700 1 -- 172.21.15.28:0/1066582011 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0011d20 con 0x7f8cc006f8b0 2023-11-28T11:09:49.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.783+0000 7f8cc69de700 1 -- 172.21.15.28:0/1066582011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cc006f8b0 msgr2=0x7f8cc00b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.783+0000 7f8cc69de700 1 --2- 172.21.15.28:0/1066582011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cc006f8b0 0x7f8cc00b2680 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009840 tx=0x7f8cb0009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:49.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.783+0000 7f8cc69de700 1 -- 172.21.15.28:0/1066582011 shutdown_connections 2023-11-28T11:09:49.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.783+0000 7f8cc69de700 1 --2- 172.21.15.28:0/1066582011 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cc00b62d0 0x7f8cc00cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.783+0000 7f8cc69de700 1 --2- 172.21.15.28:0/1066582011 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cc006f8b0 0x7f8cc00b2680 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.783+0000 7f8cc69de700 1 -- 172.21.15.28:0/1066582011 >> 172.21.15.28:0/1066582011 conn(0x7f8cc00cd680 msgr2=0x7f8cc00cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:49.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.784+0000 7f8cc69de700 1 -- 172.21.15.28:0/1066582011 shutdown_connections 2023-11-28T11:09:49.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.784+0000 7f8cc69de700 1 -- 172.21.15.28:0/1066582011 wait complete. 2023-11-28T11:09:49.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.785+0000 7f8cc69de700 1 Processor -- start 2023-11-28T11:09:49.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.785+0000 7f8cc69de700 1 -- start start 2023-11-28T11:09:49.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc69de700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cc006f8b0 0x7f8cc00c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc69de700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cc00b2bc0 0x7f8cc01ef950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc69de700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cc00b62d0 0x7f8cc006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc69de700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc006c250 con 0x7f8cc00b62d0 2023-11-28T11:09:49.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc69de700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc006c4d0 con 0x7f8cc00b2bc0 2023-11-28T11:09:49.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc69de700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc006c7b0 con 0x7f8cc006f8b0 2023-11-28T11:09:49.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc4f7b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cc00b62d0 0x7f8cc006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc4f7b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cc00b62d0 0x7f8cc006bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48722/0 (socket says 172.21.15.28:48722) 2023-11-28T11:09:49.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc4f7b700 1 -- 172.21.15.28:0/2461689521 learned_addr learned my addr 172.21.15.28:0/2461689521 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:49.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cbf7fe700 1 --2- 172.21.15.28:0/2461689521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cc00b2bc0 0x7f8cc01ef950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cbffff700 1 --2- 172.21.15.28:0/2461689521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cc006f8b0 0x7f8cc00c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc4f7b700 1 -- 172.21.15.28:0/2461689521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cc006f8b0 msgr2=0x7f8cc00c7fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc4f7b700 1 --2- 172.21.15.28:0/2461689521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cc006f8b0 0x7f8cc00c7fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc4f7b700 1 -- 172.21.15.28:0/2461689521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cc00b2bc0 msgr2=0x7f8cc01ef950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc4f7b700 1 --2- 172.21.15.28:0/2461689521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cc00b2bc0 0x7f8cc01ef950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc4f7b700 1 -- 172.21.15.28:0/2461689521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb00094a0 con 0x7f8cc00b62d0 2023-11-28T11:09:49.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cbf7fe700 1 --2- 172.21.15.28:0/2461689521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cc00b2bc0 0x7f8cc01ef950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:49.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc4f7b700 1 --2- 172.21.15.28:0/2461689521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cc00b62d0 0x7f8cc006bd10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8002230 tx=0x7f8cb800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:49.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cbcff9700 1 -- 172.21.15.28:0/2461689521 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb800ac20 con 0x7f8cc00b62d0 2023-11-28T11:09:49.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cbcff9700 1 -- 172.21.15.28:0/2461689521 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb800ada0 con 0x7f8cc00b62d0 2023-11-28T11:09:49.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cc69de700 1 -- 172.21.15.28:0/2461689521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cc00b66a0 con 0x7f8cc00b62d0 2023-11-28T11:09:49.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.786+0000 7f8cbcff9700 1 -- 172.21.15.28:0/2461689521 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb8007310 con 0x7f8cc00b62d0 2023-11-28T11:09:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.787+0000 7f8cc69de700 1 -- 172.21.15.28:0/2461689521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cc00cffb0 con 0x7f8cc00b62d0 2023-11-28T11:09:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.787+0000 7f8cc69de700 1 -- 172.21.15.28:0/2461689521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cc006fd20 con 0x7f8cc00b62d0 2023-11-28T11:09:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.788+0000 7f8cbcff9700 1 -- 172.21.15.28:0/2461689521 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8cb801a030 con 0x7f8cc00b62d0 2023-11-28T11:09:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.788+0000 7f8cbcff9700 1 --2- 172.21.15.28:0/2461689521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8cac0682f0 0x7f8cac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:49.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.789+0000 7f8cbcff9700 1 -- 172.21.15.28:0/2461689521 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f8cb8085ea0 con 0x7f8cc00b62d0 2023-11-28T11:09:49.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.789+0000 7f8cbffff700 1 --2- 172.21.15.28:0/2461689521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8cac0682f0 0x7f8cac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:49.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.789+0000 7f8cbffff700 1 --2- 172.21.15.28:0/2461689521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8cac0682f0 0x7f8cac06a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400a950 tx=0x7f8cb4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:49.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.793+0000 7f8cbcff9700 1 -- 172.21.15.28:0/2461689521 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8cb8052280 con 0x7f8cc00b62d0 2023-11-28T11:09:49.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.990+0000 7f8cc69de700 1 -- 172.21.15.28:0/2461689521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8cc00739f0 con 0x7f8cc00b62d0 2023-11-28T11:09:49.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.990+0000 7f8cbcff9700 1 -- 172.21.15.28:0/2461689521 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8cb80180a0 con 0x7f8cc00b62d0 2023-11-28T11:09:49.992 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:09:49.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.992+0000 7f8cc69de700 1 -- 172.21.15.28:0/2461689521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8cac0682f0 msgr2=0x7f8cac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.992+0000 7f8cc69de700 1 --2- 172.21.15.28:0/2461689521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8cac0682f0 0x7f8cac06a7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f8cb400a950 tx=0x7f8cb4008040 comp rx=0 tx=0).stop 2023-11-28T11:09:49.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.992+0000 7f8cc69de700 1 -- 172.21.15.28:0/2461689521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cc00b62d0 msgr2=0x7f8cc006bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:49.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.992+0000 7f8cc69de700 1 --2- 172.21.15.28:0/2461689521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cc00b62d0 0x7f8cc006bd10 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f8cb8002230 tx=0x7f8cb800a060 comp rx=0 tx=0).stop 2023-11-28T11:09:49.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.992+0000 7f8cc69de700 1 -- 172.21.15.28:0/2461689521 shutdown_connections 2023-11-28T11:09:49.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.992+0000 7f8cc69de700 1 --2- 172.21.15.28:0/2461689521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8cac0682f0 0x7f8cac06a7a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.992+0000 7f8cc69de700 1 --2- 172.21.15.28:0/2461689521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8cc00b62d0 0x7f8cc006bd10 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.992+0000 7f8cc69de700 1 --2- 172.21.15.28:0/2461689521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8cc006f8b0 0x7f8cc00c7fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.992+0000 7f8cc69de700 1 --2- 172.21.15.28:0/2461689521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8cc00b2bc0 0x7f8cc01ef950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:49.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.992+0000 7f8cc69de700 1 -- 172.21.15.28:0/2461689521 >> 172.21.15.28:0/2461689521 conn(0x7f8cc00cd680 msgr2=0x7f8cc00cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:49.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.993+0000 7f8cc69de700 1 -- 172.21.15.28:0/2461689521 shutdown_connections 2023-11-28T11:09:49.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:49.993+0000 7f8cc69de700 1 -- 172.21.15.28:0/2461689521 wait complete. 2023-11-28T11:09:50.006 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:50.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.132+0000 7fcfffc36700 1 Processor -- start 2023-11-28T11:09:50.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.132+0000 7fcfffc36700 1 -- start start 2023-11-28T11:09:50.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcfffc36700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcff80b1630 0x7fcff80cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcfffc36700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcff806afc0 0x7fcff806b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcfffc36700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff80ce550 con 0x7fcff80b1a00 2023-11-28T11:09:50.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcfffc36700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcff80ce830 con 0x7fcff80b1630 2023-11-28T11:09:50.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcfffc36700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff80ceb10 con 0x7fcff806afc0 2023-11-28T11:09:50.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcffd1d1700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fcff80b1a00 0x7fcff80caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36784/0 (socket says 172.21.15.28:36784) 2023-11-28T11:09:50.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcffd1d1700 1 -- 172.21.15.28:0/601388357 learned_addr learned my addr 172.21.15.28:0/601388357 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:50.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcffe1d3700 1 --2- 172.21.15.28:0/601388357 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcff806afc0 0x7fcff806b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcffd9d2700 1 --2- 172.21.15.28:0/601388357 >> v2:172.21.15.59:3300/0 conn(0x7fcff80b1630 0x7fcff80cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcffc9d0700 1 -- 172.21.15.28:0/601388357 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3408453433 0 0) 0x7fcff80ce550 con 0x7fcff80b1a00 2023-11-28T11:09:50.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcffc9d0700 1 -- 172.21.15.28:0/601388357 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfe40035b0 con 0x7fcff80b1a00 2023-11-28T11:09:50.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcffc9d0700 1 -- 172.21.15.28:0/601388357 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3753315586 0 0) 0x7fcfe40035b0 con 0x7fcff80b1a00 2023-11-28T11:09:50.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcffe1d3700 1 -- 172.21.15.28:0/601388357 >> v2:172.21.15.59:3300/0 conn(0x7fcff80b1630 msgr2=0x7fcff80cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:50.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcffe1d3700 1 --2- 172.21.15.28:0/601388357 >> v2:172.21.15.59:3300/0 conn(0x7fcff80b1630 0x7fcff80cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcffe1d3700 1 -- 172.21.15.28:0/601388357 >> v1:172.21.15.28:6789/0 conn(0x7fcff80b1a00 legacy=0x7fcff80caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcffe1d3700 1 -- 172.21.15.28:0/601388357 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff80cedf0 con 0x7fcff806afc0 2023-11-28T11:09:50.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.133+0000 7fcffd9d2700 1 --2- 172.21.15.28:0/601388357 >> v2:172.21.15.59:3300/0 conn(0x7fcff80b1630 0x7fcff80cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:50.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.134+0000 7fcffe1d3700 1 --2- 172.21.15.28:0/601388357 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcff806afc0 0x7fcff806b440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009dc0 tx=0x7fcff000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9070f4a10dd7793e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.134+0000 7fcffc9d0700 1 -- 172.21.15.28:0/601388357 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff000e070 con 0x7fcff806afc0 2023-11-28T11:09:50.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.134+0000 7fcffc9d0700 1 -- 172.21.15.28:0/601388357 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff0003a20 con 0x7fcff806afc0 2023-11-28T11:09:50.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.134+0000 7fcffc9d0700 1 -- 172.21.15.28:0/601388357 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8001670 con 0x7fcff806afc0 2023-11-28T11:09:50.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.134+0000 7fcfffc36700 1 -- 172.21.15.28:0/601388357 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcff806afc0 msgr2=0x7fcff806b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.134+0000 7fcfffc36700 1 --2- 172.21.15.28:0/601388357 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcff806afc0 0x7fcff806b440 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fcff0009dc0 tx=0x7fcff000b330 comp rx=0 tx=0).stop 2023-11-28T11:09:50.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.134+0000 7fcfffc36700 1 -- 172.21.15.28:0/601388357 shutdown_connections 2023-11-28T11:09:50.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.134+0000 7fcfffc36700 1 --2- 172.21.15.28:0/601388357 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcff806afc0 0x7fcff806b440 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.134+0000 7fcfffc36700 1 --2- 172.21.15.28:0/601388357 >> v2:172.21.15.59:3300/0 conn(0x7fcff80b1630 0x7fcff80cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.134+0000 7fcfffc36700 1 -- 172.21.15.28:0/601388357 >> 172.21.15.28:0/601388357 conn(0x7fcff80cc630 msgr2=0x7fcff80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.134+0000 7fcfffc36700 1 -- 172.21.15.28:0/601388357 shutdown_connections 2023-11-28T11:09:50.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.134+0000 7fcfffc36700 1 -- 172.21.15.28:0/601388357 wait complete. 2023-11-28T11:09:50.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.136+0000 7fcfffc36700 1 Processor -- start 2023-11-28T11:09:50.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.136+0000 7fcfffc36700 1 -- start start 2023-11-28T11:09:50.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.136+0000 7fcfffc36700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcff806afc0 0x7fcff8145e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.136+0000 7fcfffc36700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcff80b1630 0x7fcff80c7660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.136+0000 7fcfffc36700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcff80b1a00 0x7fcff8158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.136+0000 7fcfffc36700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff8159c70 con 0x7fcff80b1630 2023-11-28T11:09:50.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.136+0000 7fcfffc36700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff8146520 con 0x7fcff80b1a00 2023-11-28T11:09:50.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.136+0000 7fcfffc36700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff8146800 con 0x7fcff806afc0 2023-11-28T11:09:50.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.136+0000 7fcffd1d1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcff80b1630 0x7fcff80c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.136+0000 7fcffd1d1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcff80b1630 0x7fcff80c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48732/0 (socket says 172.21.15.28:48732) 2023-11-28T11:09:50.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcffd1d1700 1 -- 172.21.15.28:0/3220627069 learned_addr learned my addr 172.21.15.28:0/3220627069 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:50.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcffe1d3700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcff80b1a00 0x7fcff8158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcffd9d2700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcff806afc0 0x7fcff8145e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcffd1d1700 1 -- 172.21.15.28:0/3220627069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcff806afc0 msgr2=0x7fcff8145e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcffd1d1700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcff806afc0 0x7fcff8145e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcffd1d1700 1 -- 172.21.15.28:0/3220627069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcff80b1a00 msgr2=0x7fcff8158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcffd1d1700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcff80b1a00 0x7fcff8158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcffd1d1700 1 -- 172.21.15.28:0/3220627069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff40094a0 con 0x7fcff80b1630 2023-11-28T11:09:50.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcffe1d3700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcff80b1a00 0x7fcff8158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:50.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcffd9d2700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcff806afc0 0x7fcff8145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:50.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcffd1d1700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcff80b1630 0x7fcff80c7660 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80017e0 tx=0x7fcfe800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcfee7fc700 1 -- 172.21.15.28:0/3220627069 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8004c80 con 0x7fcff80b1630 2023-11-28T11:09:50.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcfee7fc700 1 -- 172.21.15.28:0/3220627069 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe8004de0 con 0x7fcff80b1630 2023-11-28T11:09:50.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcfffc36700 1 -- 172.21.15.28:0/3220627069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff0009a70 con 0x7fcff80b1630 2023-11-28T11:09:50.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcfee7fc700 1 -- 172.21.15.28:0/3220627069 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe80072f0 con 0x7fcff80b1630 2023-11-28T11:09:50.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.137+0000 7fcfffc36700 1 -- 172.21.15.28:0/3220627069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff80b1ea0 con 0x7fcff80b1630 2023-11-28T11:09:50.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.138+0000 7fcfffc36700 1 -- 172.21.15.28:0/3220627069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff80cf0d0 con 0x7fcff80b1630 2023-11-28T11:09:50.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.139+0000 7fcfee7fc700 1 -- 172.21.15.28:0/3220627069 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcfe8010070 con 0x7fcff80b1630 2023-11-28T11:09:50.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.139+0000 7fcfee7fc700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcfdc0682f0 0x7fcfdc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.139+0000 7fcfee7fc700 1 -- 172.21.15.28:0/3220627069 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fcfe8085020 con 0x7fcff80b1630 2023-11-28T11:09:50.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.139+0000 7fcffd9d2700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcfdc0682f0 0x7fcfdc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.139+0000 7fcffd9d2700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcfdc0682f0 0x7fcfdc06a7a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009cf0 tx=0x7fcff4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.143+0000 7fcfee7fc700 1 -- 172.21.15.28:0/3220627069 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcfe80514d0 con 0x7fcff80b1630 2023-11-28T11:09:50.339 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.337+0000 7fcfffc36700 1 -- 172.21.15.28:0/3220627069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fcff80cb5b0 con 0x7fcff80b1630 2023-11-28T11:09:50.339 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.338+0000 7fcfee7fc700 1 -- 172.21.15.28:0/3220627069 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fcfe8018020 con 0x7fcff80b1630 2023-11-28T11:09:50.340 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.340+0000 7fcfffc36700 1 -- 172.21.15.28:0/3220627069 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcfdc0682f0 msgr2=0x7fcfdc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.340+0000 7fcfffc36700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcfdc0682f0 0x7fcfdc06a7a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcff4009cf0 tx=0x7fcff4008040 comp rx=0 tx=0).stop 2023-11-28T11:09:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.340+0000 7fcfffc36700 1 -- 172.21.15.28:0/3220627069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcff80b1630 msgr2=0x7fcff80c7660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.340+0000 7fcfffc36700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcff80b1630 0x7fcff80c7660 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fcfe80017e0 tx=0x7fcfe800cae0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.340+0000 7fcfffc36700 1 -- 172.21.15.28:0/3220627069 shutdown_connections 2023-11-28T11:09:50.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.340+0000 7fcfffc36700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcfdc0682f0 0x7fcfdc06a7a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.340+0000 7fcfffc36700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcff80b1a00 0x7fcff8158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.340+0000 7fcfffc36700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcff806afc0 0x7fcff8145e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.340+0000 7fcfffc36700 1 --2- 172.21.15.28:0/3220627069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcff80b1630 0x7fcff80c7660 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.340+0000 7fcfffc36700 1 -- 172.21.15.28:0/3220627069 >> 172.21.15.28:0/3220627069 conn(0x7fcff80cc630 msgr2=0x7fcff80b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.340+0000 7fcfffc36700 1 -- 172.21.15.28:0/3220627069 shutdown_connections 2023-11-28T11:09:50.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.340+0000 7fcfffc36700 1 -- 172.21.15.28:0/3220627069 wait complete. 2023-11-28T11:09:50.354 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.478+0000 7f67076b4700 1 Processor -- start 2023-11-28T11:09:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.478+0000 7f67076b4700 1 -- start start 2023-11-28T11:09:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.479+0000 7f67076b4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f670015c990 0x7f67000cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.479+0000 7f67076b4700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f67000cc310 0x7f67000cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.479+0000 7f67076b4700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67000ce340 con 0x7f6700161090 2023-11-28T11:09:50.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.479+0000 7f67076b4700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67000cebe0 con 0x7f67000cc310 2023-11-28T11:09:50.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.479+0000 7f67076b4700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67000cccb0 con 0x7f670015c990 2023-11-28T11:09:50.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.479+0000 7f6705c51700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f67000cc310 0x7f67000cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.479+0000 7f6705c51700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f67000cc310 0x7f67000cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48742/0 (socket says 172.21.15.28:48742) 2023-11-28T11:09:50.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.479+0000 7f6704c4f700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6700161090 0x7f67000ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36796/0 (socket says 172.21.15.28:36796) 2023-11-28T11:09:50.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.479+0000 7f6705c51700 1 -- 172.21.15.28:0/1462759388 learned_addr learned my addr 172.21.15.28:0/1462759388 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:50.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.480+0000 7f6705450700 1 --2- 172.21.15.28:0/1462759388 >> v2:172.21.15.59:3300/0 conn(0x7f670015c990 0x7f67000cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.480+0000 7f6705c51700 1 -- 172.21.15.28:0/1462759388 >> v2:172.21.15.59:3300/0 conn(0x7f670015c990 msgr2=0x7f67000cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:50.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.480+0000 7f6705c51700 1 --2- 172.21.15.28:0/1462759388 >> v2:172.21.15.59:3300/0 conn(0x7f670015c990 0x7f67000cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.480+0000 7f6705c51700 1 -- 172.21.15.28:0/1462759388 >> v1:172.21.15.28:6789/0 conn(0x7f6700161090 legacy=0x7f67000ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.480+0000 7f6705c51700 1 -- 172.21.15.28:0/1462759388 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67000cb3f0 con 0x7f67000cc310 2023-11-28T11:09:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.480+0000 7f66f7fff700 1 -- 172.21.15.28:0/1462759388 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082904888 0 0) 0x7f67000ce340 con 0x7f6700161090 2023-11-28T11:09:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.480+0000 7f6705450700 1 --2- 172.21.15.28:0/1462759388 >> v2:172.21.15.59:3300/0 conn(0x7f670015c990 0x7f67000cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.480+0000 7f6705c51700 1 --2- 172.21.15.28:0/1462759388 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67000cc310 0x7f67000cc770 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f670006ea90 tx=0x7f66f800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b861d1775a7bcc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.480+0000 7f66f7fff700 1 -- 172.21.15.28:0/1462759388 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f800e070 con 0x7f67000cc310 2023-11-28T11:09:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.480+0000 7f66f7fff700 1 -- 172.21.15.28:0/1462759388 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f800b9c0 con 0x7f67000cc310 2023-11-28T11:09:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.480+0000 7f66f7fff700 1 -- 172.21.15.28:0/1462759388 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f800bd70 con 0x7f67000cc310 2023-11-28T11:09:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.480+0000 7f67076b4700 1 -- 172.21.15.28:0/1462759388 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67000cc310 msgr2=0x7f67000cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.480+0000 7f67076b4700 1 --2- 172.21.15.28:0/1462759388 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67000cc310 0x7f67000cc770 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f670006ea90 tx=0x7f66f800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.481+0000 7f67076b4700 1 -- 172.21.15.28:0/1462759388 shutdown_connections 2023-11-28T11:09:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.481+0000 7f67076b4700 1 --2- 172.21.15.28:0/1462759388 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67000cc310 0x7f67000cc770 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.481+0000 7f67076b4700 1 --2- 172.21.15.28:0/1462759388 >> v2:172.21.15.59:3300/0 conn(0x7f670015c990 0x7f67000cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.481+0000 7f67076b4700 1 -- 172.21.15.28:0/1462759388 >> 172.21.15.28:0/1462759388 conn(0x7f67000b1650 msgr2=0x7f67000b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.481+0000 7f67076b4700 1 -- 172.21.15.28:0/1462759388 shutdown_connections 2023-11-28T11:09:50.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.481+0000 7f67076b4700 1 -- 172.21.15.28:0/1462759388 wait complete. 2023-11-28T11:09:50.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.482+0000 7f67076b4700 1 Processor -- start 2023-11-28T11:09:50.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.482+0000 7f67076b4700 1 -- start start 2023-11-28T11:09:50.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f67076b4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67000cc310 0x7f670015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f67076b4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f670015c990 0x7f67000ae2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f67076b4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6700161090 0x7f67000abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f67076b4700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670015efa0 con 0x7f67000cc310 2023-11-28T11:09:50.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f67076b4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670015e390 con 0x7f6700161090 2023-11-28T11:09:50.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f67076b4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f670015e670 con 0x7f670015c990 2023-11-28T11:09:50.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f6705450700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67000cc310 0x7f670015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f6705450700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67000cc310 0x7f670015ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48744/0 (socket says 172.21.15.28:48744) 2023-11-28T11:09:50.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f6705450700 1 -- 172.21.15.28:0/2426479761 learned_addr learned my addr 172.21.15.28:0/2426479761 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:50.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f6705c51700 1 --2- 172.21.15.28:0/2426479761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6700161090 0x7f67000abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f6705450700 1 -- 172.21.15.28:0/2426479761 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f670015c990 msgr2=0x7f67000ae2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f6705450700 1 --2- 172.21.15.28:0/2426479761 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f670015c990 0x7f67000ae2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f6705450700 1 -- 172.21.15.28:0/2426479761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6700161090 msgr2=0x7f67000abb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f6705450700 1 --2- 172.21.15.28:0/2426479761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6700161090 0x7f67000abb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f6705450700 1 -- 172.21.15.28:0/2426479761 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f8009b30 con 0x7f67000cc310 2023-11-28T11:09:50.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f6705450700 1 --2- 172.21.15.28:0/2426479761 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67000cc310 0x7f670015ddb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f66f000c750 tx=0x7f66f000cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f66f5ffb700 1 -- 172.21.15.28:0/2426479761 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0011800 con 0x7f67000cc310 2023-11-28T11:09:50.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f66f5ffb700 1 -- 172.21.15.28:0/2426479761 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66f0011960 con 0x7f67000cc310 2023-11-28T11:09:50.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f66f5ffb700 1 -- 172.21.15.28:0/2426479761 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66f0011cd0 con 0x7f67000cc310 2023-11-28T11:09:50.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f67076b4700 1 -- 172.21.15.28:0/2426479761 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67000ce870 con 0x7f67000cc310 2023-11-28T11:09:50.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.483+0000 7f67076b4700 1 -- 172.21.15.28:0/2426479761 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67000ce590 con 0x7f67000cc310 2023-11-28T11:09:50.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.484+0000 7f67076b4700 1 -- 172.21.15.28:0/2426479761 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67000cb080 con 0x7f67000cc310 2023-11-28T11:09:50.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.485+0000 7f66f5ffb700 1 -- 172.21.15.28:0/2426479761 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f66f0020070 con 0x7f67000cc310 2023-11-28T11:09:50.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.485+0000 7f66f5ffb700 1 --2- 172.21.15.28:0/2426479761 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f66e40682f0 0x7f66e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.485+0000 7f6704c4f700 1 --2- 172.21.15.28:0/2426479761 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f66e40682f0 0x7f66e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.485+0000 7f66f5ffb700 1 -- 172.21.15.28:0/2426479761 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f66f0085e40 con 0x7f67000cc310 2023-11-28T11:09:50.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.486+0000 7f6704c4f700 1 --2- 172.21.15.28:0/2426479761 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f66e40682f0 0x7f66e406a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0011e0 tx=0x7f66fc0079b0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.489+0000 7f66f5ffb700 1 -- 172.21.15.28:0/2426479761 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f66f0052370 con 0x7f67000cc310 2023-11-28T11:09:50.699 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.698+0000 7f67076b4700 1 -- 172.21.15.28:0/2426479761 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f67000675a0 con 0x7f67000cc310 2023-11-28T11:09:50.700 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.698+0000 7f66f5ffb700 1 -- 172.21.15.28:0/2426479761 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7f66f0055990 con 0x7f67000cc310 2023-11-28T11:09:50.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.700+0000 7f67076b4700 1 -- 172.21.15.28:0/2426479761 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f66e40682f0 msgr2=0x7f66e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.700+0000 7f67076b4700 1 --2- 172.21.15.28:0/2426479761 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f66e40682f0 0x7f66e406a7a0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f66fc0011e0 tx=0x7f66fc0079b0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.700+0000 7f67076b4700 1 -- 172.21.15.28:0/2426479761 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67000cc310 msgr2=0x7f670015ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.700+0000 7f67076b4700 1 --2- 172.21.15.28:0/2426479761 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67000cc310 0x7f670015ddb0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f66f000c750 tx=0x7f66f000cb10 comp rx=0 tx=0).stop 2023-11-28T11:09:50.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.700+0000 7f67076b4700 1 -- 172.21.15.28:0/2426479761 shutdown_connections 2023-11-28T11:09:50.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.700+0000 7f67076b4700 1 --2- 172.21.15.28:0/2426479761 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f66e40682f0 0x7f66e406a7a0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.700+0000 7f67076b4700 1 --2- 172.21.15.28:0/2426479761 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f67000cc310 0x7f670015ddb0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.700+0000 7f67076b4700 1 --2- 172.21.15.28:0/2426479761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6700161090 0x7f67000abb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.700+0000 7f67076b4700 1 --2- 172.21.15.28:0/2426479761 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f670015c990 0x7f67000ae2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.700+0000 7f67076b4700 1 -- 172.21.15.28:0/2426479761 >> 172.21.15.28:0/2426479761 conn(0x7f67000b1650 msgr2=0x7f670006ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.700+0000 7f67076b4700 1 -- 172.21.15.28:0/2426479761 shutdown_connections 2023-11-28T11:09:50.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.700+0000 7f67076b4700 1 -- 172.21.15.28:0/2426479761 wait complete. 2023-11-28T11:09:50.704 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2023-11-28T11:09:50.714 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:09:50.714 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:09:50.738 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:50.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.860+0000 7f1272620700 1 Processor -- start 2023-11-28T11:09:50.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.860+0000 7f1272620700 1 -- start start 2023-11-28T11:09:50.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f1272620700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f126c06e400 0x7f126c15db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f1272620700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f126c0ce890 0x7f126c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.866 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f1272620700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f126c15e0b0 con 0x7f126c159590 2023-11-28T11:09:50.866 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f1272620700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f126c0cf250 con 0x7f126c06e400 2023-11-28T11:09:50.866 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f1272620700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126c15d120 con 0x7f126c0ce890 2023-11-28T11:09:50.866 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f126b7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f126c06e400 0x7f126c15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.866 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f126bfff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f126c159590 0x7f126c0ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36810/0 (socket says 172.21.15.28:36810) 2023-11-28T11:09:50.866 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f126bfff700 1 -- 172.21.15.28:0/32717836 learned_addr learned my addr 172.21.15.28:0/32717836 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:50.866 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f1270bbd700 1 --2- 172.21.15.28:0/32717836 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f126c0ce890 0x7f126c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.867 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f126b7fe700 1 -- 172.21.15.28:0/32717836 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f126c0ce890 msgr2=0x7f126c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:50.867 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f126b7fe700 1 --2- 172.21.15.28:0/32717836 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f126c0ce890 0x7f126c0ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.867 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f126b7fe700 1 -- 172.21.15.28:0/32717836 >> v1:172.21.15.28:6789/0 conn(0x7f126c159590 legacy=0x7f126c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.867 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f126b7fe700 1 -- 172.21.15.28:0/32717836 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f126c15d6e0 con 0x7f126c06e400 2023-11-28T11:09:50.867 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f126affd700 1 -- 172.21.15.28:0/32717836 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4205491798 0 0) 0x7f1254001110 con 0x7f126c159590 2023-11-28T11:09:50.867 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f126b7fe700 1 --2- 172.21.15.28:0/32717836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126c06e400 0x7f126c15db70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f125c009840 tx=0x7f125c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2ca261f7ff3d7c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.867 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f126affd700 1 -- 172.21.15.28:0/32717836 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c011830 con 0x7f126c06e400 2023-11-28T11:09:50.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f126affd700 1 -- 172.21.15.28:0/32717836 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125c011990 con 0x7f126c06e400 2023-11-28T11:09:50.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.862+0000 7f126affd700 1 -- 172.21.15.28:0/32717836 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c011d20 con 0x7f126c06e400 2023-11-28T11:09:50.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.863+0000 7f1272620700 1 -- 172.21.15.28:0/32717836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126c06e400 msgr2=0x7f126c15db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.863+0000 7f1272620700 1 --2- 172.21.15.28:0/32717836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126c06e400 0x7f126c15db70 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f125c009840 tx=0x7f125c009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:50.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.863+0000 7f1272620700 1 -- 172.21.15.28:0/32717836 shutdown_connections 2023-11-28T11:09:50.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.863+0000 7f1272620700 1 --2- 172.21.15.28:0/32717836 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f126c0ce890 0x7f126c0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.863+0000 7f1272620700 1 --2- 172.21.15.28:0/32717836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126c06e400 0x7f126c15db70 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.863+0000 7f1272620700 1 -- 172.21.15.28:0/32717836 >> 172.21.15.28:0/32717836 conn(0x7f126c0b1650 msgr2=0x7f126c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:50.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.863+0000 7f1272620700 1 -- 172.21.15.28:0/32717836 shutdown_connections 2023-11-28T11:09:50.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.863+0000 7f1272620700 1 -- 172.21.15.28:0/32717836 wait complete. 2023-11-28T11:09:50.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.864+0000 7f1272620700 1 Processor -- start 2023-11-28T11:09:50.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.864+0000 7f1272620700 1 -- start start 2023-11-28T11:09:50.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.865+0000 7f1272620700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f126c06e400 0x7f126c1f07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.865+0000 7f1272620700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126c0ce890 0x7f126c155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.865+0000 7f1272620700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f126c159590 0x7f126c0c3040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.865+0000 7f1272620700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c0c5ba0 con 0x7f126c0ce890 2023-11-28T11:09:50.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.865+0000 7f1272620700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f126c1f0cf0 con 0x7f126c159590 2023-11-28T11:09:50.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.865+0000 7f1272620700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f126c1555d0 con 0x7f126c06e400 2023-11-28T11:09:50.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.865+0000 7f126b7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126c0ce890 0x7f126c155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.865+0000 7f126b7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126c0ce890 0x7f126c155090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48760/0 (socket says 172.21.15.28:48760) 2023-11-28T11:09:50.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.865+0000 7f126b7fe700 1 -- 172.21.15.28:0/2602441716 learned_addr learned my addr 172.21.15.28:0/2602441716 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:50.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.865+0000 7f126b7fe700 1 -- 172.21.15.28:0/2602441716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f126c06e400 msgr2=0x7f126c1f07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.865+0000 7f126b7fe700 1 --2- 172.21.15.28:0/2602441716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f126c06e400 0x7f126c1f07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.866+0000 7f126b7fe700 1 -- 172.21.15.28:0/2602441716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f126c159590 msgr2=0x7f126c0c3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:50.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.866+0000 7f126b7fe700 1 --2- 172.21.15.28:0/2602441716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f126c159590 0x7f126c0c3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:50.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.866+0000 7f126b7fe700 1 -- 172.21.15.28:0/2602441716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f125c0094a0 con 0x7f126c0ce890 2023-11-28T11:09:50.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.866+0000 7f126b7fe700 1 --2- 172.21.15.28:0/2602441716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126c0ce890 0x7f126c155090 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f126c06eac0 tx=0x7f125c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.866+0000 7f1268ff9700 1 -- 172.21.15.28:0/2602441716 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c0038a0 con 0x7f126c0ce890 2023-11-28T11:09:50.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.866+0000 7f1268ff9700 1 -- 172.21.15.28:0/2602441716 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125c01f070 con 0x7f126c0ce890 2023-11-28T11:09:50.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.866+0000 7f1272620700 1 -- 172.21.15.28:0/2602441716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f126c0af240 con 0x7f126c0ce890 2023-11-28T11:09:50.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.866+0000 7f1268ff9700 1 -- 172.21.15.28:0/2602441716 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c003c20 con 0x7f126c0ce890 2023-11-28T11:09:50.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.866+0000 7f1272620700 1 -- 172.21.15.28:0/2602441716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f126c0aef60 con 0x7f126c0ce890 2023-11-28T11:09:50.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.867+0000 7f1272620700 1 -- 172.21.15.28:0/2602441716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f126c15d370 con 0x7f126c0ce890 2023-11-28T11:09:50.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.867+0000 7f1268ff9700 1 -- 172.21.15.28:0/2602441716 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f125c00f050 con 0x7f126c0ce890 2023-11-28T11:09:50.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.868+0000 7f1268ff9700 1 --2- 172.21.15.28:0/2602441716 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f124c0682f0 0x7f124c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:50.874 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.868+0000 7f1268ff9700 1 -- 172.21.15.28:0/2602441716 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f125c021070 con 0x7f126c0ce890 2023-11-28T11:09:50.874 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.868+0000 7f126bfff700 1 --2- 172.21.15.28:0/2602441716 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f124c0682f0 0x7f124c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:50.874 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.868+0000 7f126bfff700 1 --2- 172.21.15.28:0/2602441716 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f124c0682f0 0x7f124c06a7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1254001610 tx=0x7f12540079f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:50.874 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:50.872+0000 7f1268ff9700 1 -- 172.21.15.28:0/2602441716 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f125c053e60 con 0x7f126c0ce890 2023-11-28T11:09:50.992 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:09:50.995 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:09:51.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.069+0000 7f1272620700 1 -- 172.21.15.28:0/2602441716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f126c0675a0 con 0x7f126c0ce890 2023-11-28T11:09:51.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.070+0000 7f1268ff9700 1 -- 172.21.15.28:0/2602441716 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19737 (secure 0 0 0) 0x7f125c057480 con 0x7f126c0ce890 2023-11-28T11:09:51.072 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:51.072 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:09:34.614188+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:51.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.072+0000 7f1272620700 1 -- 172.21.15.28:0/2602441716 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f124c0682f0 msgr2=0x7f124c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.072+0000 7f1272620700 1 --2- 172.21.15.28:0/2602441716 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f124c0682f0 0x7f124c06a7a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1254001610 tx=0x7f12540079f0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.072+0000 7f1272620700 1 -- 172.21.15.28:0/2602441716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126c0ce890 msgr2=0x7f126c155090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.072+0000 7f1272620700 1 --2- 172.21.15.28:0/2602441716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126c0ce890 0x7f126c155090 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f126c06eac0 tx=0x7f125c004750 comp rx=0 tx=0).stop 2023-11-28T11:09:51.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.072+0000 7f1272620700 1 -- 172.21.15.28:0/2602441716 shutdown_connections 2023-11-28T11:09:51.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.072+0000 7f1272620700 1 --2- 172.21.15.28:0/2602441716 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f124c0682f0 0x7f124c06a7a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.072+0000 7f1272620700 1 --2- 172.21.15.28:0/2602441716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f126c159590 0x7f126c0c3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.072+0000 7f1272620700 1 --2- 172.21.15.28:0/2602441716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f126c06e400 0x7f126c1f07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.072+0000 7f1272620700 1 --2- 172.21.15.28:0/2602441716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126c0ce890 0x7f126c155090 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.072+0000 7f1272620700 1 -- 172.21.15.28:0/2602441716 >> 172.21.15.28:0/2602441716 conn(0x7f126c0b1650 msgr2=0x7f126c0b0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.072+0000 7f1272620700 1 -- 172.21.15.28:0/2602441716 shutdown_connections 2023-11-28T11:09:51.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.072+0000 7f1272620700 1 -- 172.21.15.28:0/2602441716 wait complete. 2023-11-28T11:09:51.086 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:51.087 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:51.254 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.253+0000 7fdee77c5700 1 Processor -- start 2023-11-28T11:09:51.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.253+0000 7fdee77c5700 1 -- start start 2023-11-28T11:09:51.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.254+0000 7fdee77c5700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdee015fc00 0x7fdee006eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.254+0000 7fdee77c5700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee006f940 con 0x7fdee006f3e0 2023-11-28T11:09:51.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.254+0000 7fdee77c5700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdee006fc20 con 0x7fdee00b2d90 2023-11-28T11:09:51.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.254+0000 7fdee77c5700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdee00cb3f0 con 0x7fdee015fc00 2023-11-28T11:09:51.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.254+0000 7fdee5561700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdee015fc00 0x7fdee006eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.254+0000 7fdee4d60700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdee006f3e0 0x7fdee00b2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36824/0 (socket says 172.21.15.28:36824) 2023-11-28T11:09:51.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.254+0000 7fdee4d60700 1 -- 172.21.15.28:0/1094721182 learned_addr learned my addr 172.21.15.28:0/1094721182 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:51.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.254+0000 7fdee5561700 1 -- 172.21.15.28:0/1094721182 >> v1:172.21.15.59:6789/0 conn(0x7fdee00b2d90 legacy=0x7fdee00cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.254+0000 7fdee5561700 1 -- 172.21.15.28:0/1094721182 >> v1:172.21.15.28:6789/0 conn(0x7fdee006f3e0 legacy=0x7fdee00b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.254+0000 7fded7fff700 1 -- 172.21.15.28:0/1094721182 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 634764477 0 0) 0x7fdee006f940 con 0x7fdee006f3e0 2023-11-28T11:09:51.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.254+0000 7fdee5561700 1 -- 172.21.15.28:0/1094721182 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee00cb6d0 con 0x7fdee015fc00 2023-11-28T11:09:51.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.255+0000 7fdee5561700 1 --2- 172.21.15.28:0/1094721182 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee015fc00 0x7fdee006eea0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fded0009840 tx=0x7fded0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7deda58f826c1365 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.255+0000 7fded7fff700 1 -- 172.21.15.28:0/1094721182 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded0011830 con 0x7fdee015fc00 2023-11-28T11:09:51.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.255+0000 7fded7fff700 1 -- 172.21.15.28:0/1094721182 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded0011990 con 0x7fdee015fc00 2023-11-28T11:09:51.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.255+0000 7fded7fff700 1 -- 172.21.15.28:0/1094721182 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded0011d20 con 0x7fdee015fc00 2023-11-28T11:09:51.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.255+0000 7fdee77c5700 1 -- 172.21.15.28:0/1094721182 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee015fc00 msgr2=0x7fdee006eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.255+0000 7fdee77c5700 1 --2- 172.21.15.28:0/1094721182 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee015fc00 0x7fdee006eea0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fded0009840 tx=0x7fded0009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:51.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.255+0000 7fdee77c5700 1 -- 172.21.15.28:0/1094721182 shutdown_connections 2023-11-28T11:09:51.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.255+0000 7fdee77c5700 1 --2- 172.21.15.28:0/1094721182 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee015fc00 0x7fdee006eea0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.255+0000 7fdee77c5700 1 -- 172.21.15.28:0/1094721182 >> 172.21.15.28:0/1094721182 conn(0x7fdee00cd680 msgr2=0x7fdee00cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.255+0000 7fdee77c5700 1 -- 172.21.15.28:0/1094721182 shutdown_connections 2023-11-28T11:09:51.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.255+0000 7fdee77c5700 1 -- 172.21.15.28:0/1094721182 wait complete. 2023-11-28T11:09:51.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.257+0000 7fdee77c5700 1 Processor -- start 2023-11-28T11:09:51.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.257+0000 7fdee77c5700 1 -- start start 2023-11-28T11:09:51.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.257+0000 7fdee77c5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdee006f3e0 0x7fdee0158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.257+0000 7fdee77c5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdee00b2d90 0x7fdee00c9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.257+0000 7fdee77c5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee015fc00 0x7fdee00b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.257+0000 7fdee77c5700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee01591d0 con 0x7fdee015fc00 2023-11-28T11:09:51.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.257+0000 7fdee77c5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00cb1c0 con 0x7fdee006f3e0 2023-11-28T11:09:51.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.257+0000 7fdee77c5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdee00ca920 con 0x7fdee00b2d90 2023-11-28T11:09:51.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.257+0000 7fdee5d62700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee015fc00 0x7fdee00b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.258+0000 7fdee5d62700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee015fc00 0x7fdee00b1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48776/0 (socket says 172.21.15.28:48776) 2023-11-28T11:09:51.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.258+0000 7fdee5d62700 1 -- 172.21.15.28:0/444289300 learned_addr learned my addr 172.21.15.28:0/444289300 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:51.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.258+0000 7fdee5561700 1 --2- 172.21.15.28:0/444289300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdee006f3e0 0x7fdee0158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.258+0000 7fdee4d60700 1 --2- 172.21.15.28:0/444289300 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdee00b2d90 0x7fdee00c9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.258+0000 7fdee5d62700 1 -- 172.21.15.28:0/444289300 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdee00b2d90 msgr2=0x7fdee00c9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.258+0000 7fdee5d62700 1 --2- 172.21.15.28:0/444289300 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdee00b2d90 0x7fdee00c9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.258+0000 7fdee5d62700 1 -- 172.21.15.28:0/444289300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdee006f3e0 msgr2=0x7fdee0158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.258+0000 7fdee5d62700 1 --2- 172.21.15.28:0/444289300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdee006f3e0 0x7fdee0158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.258+0000 7fdee5d62700 1 -- 172.21.15.28:0/444289300 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded00094a0 con 0x7fdee015fc00 2023-11-28T11:09:51.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.258+0000 7fdee5d62700 1 --2- 172.21.15.28:0/444289300 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee015fc00 0x7fdee00b1330 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fded800cc50 tx=0x7fded800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.258+0000 7fded5ffb700 1 -- 172.21.15.28:0/444289300 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded8003c00 con 0x7fdee015fc00 2023-11-28T11:09:51.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.258+0000 7fded5ffb700 1 -- 172.21.15.28:0/444289300 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fded8003d60 con 0x7fdee015fc00 2023-11-28T11:09:51.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.258+0000 7fdee77c5700 1 -- 172.21.15.28:0/444289300 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdee00ce340 con 0x7fdee015fc00 2023-11-28T11:09:51.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.258+0000 7fded5ffb700 1 -- 172.21.15.28:0/444289300 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fded800ab60 con 0x7fdee015fc00 2023-11-28T11:09:51.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.258+0000 7fdee77c5700 1 -- 172.21.15.28:0/444289300 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdee015ffd0 con 0x7fdee015fc00 2023-11-28T11:09:51.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.259+0000 7fdee77c5700 1 -- 172.21.15.28:0/444289300 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdee00cc300 con 0x7fdee015fc00 2023-11-28T11:09:51.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.260+0000 7fded5ffb700 1 -- 172.21.15.28:0/444289300 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fded8010050 con 0x7fdee015fc00 2023-11-28T11:09:51.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.260+0000 7fded5ffb700 1 --2- 172.21.15.28:0/444289300 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdec40682f0 0x7fdec406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.260+0000 7fded5ffb700 1 -- 172.21.15.28:0/444289300 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fded8085ea0 con 0x7fdee015fc00 2023-11-28T11:09:51.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.260+0000 7fdee5561700 1 --2- 172.21.15.28:0/444289300 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdec40682f0 0x7fdec406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.260+0000 7fdee5561700 1 --2- 172.21.15.28:0/444289300 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdec40682f0 0x7fdec406a7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fded00090b0 tx=0x7fded001c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.264+0000 7fded5ffb700 1 -- 172.21.15.28:0/444289300 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fded80523d0 con 0x7fdee015fc00 2023-11-28T11:09:51.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.458+0000 7fdee77c5700 1 -- 172.21.15.28:0/444289300 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdee00739f0 con 0x7fdee015fc00 2023-11-28T11:09:51.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.459+0000 7fded5ffb700 1 -- 172.21.15.28:0/444289300 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7fded80559f0 con 0x7fdee015fc00 2023-11-28T11:09:51.460 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:09:51.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.461+0000 7fdee77c5700 1 -- 172.21.15.28:0/444289300 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdec40682f0 msgr2=0x7fdec406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.461+0000 7fdee77c5700 1 --2- 172.21.15.28:0/444289300 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdec40682f0 0x7fdec406a7a0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fded00090b0 tx=0x7fded001c040 comp rx=0 tx=0).stop 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.461+0000 7fdee77c5700 1 -- 172.21.15.28:0/444289300 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee015fc00 msgr2=0x7fdee00b1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.461+0000 7fdee77c5700 1 --2- 172.21.15.28:0/444289300 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee015fc00 0x7fdee00b1330 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fded800cc50 tx=0x7fded800cf60 comp rx=0 tx=0).stop 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.461+0000 7fdee77c5700 1 -- 172.21.15.28:0/444289300 shutdown_connections 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.461+0000 7fdee77c5700 1 --2- 172.21.15.28:0/444289300 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdec40682f0 0x7fdec406a7a0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.461+0000 7fdee77c5700 1 --2- 172.21.15.28:0/444289300 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdee015fc00 0x7fdee00b1330 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.461+0000 7fdee77c5700 1 --2- 172.21.15.28:0/444289300 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdee006f3e0 0x7fdee0158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.461+0000 7fdee77c5700 1 --2- 172.21.15.28:0/444289300 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdee00b2d90 0x7fdee00c9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.461+0000 7fdee77c5700 1 -- 172.21.15.28:0/444289300 >> 172.21.15.28:0/444289300 conn(0x7fdee00cd680 msgr2=0x7fdee00ce4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.461+0000 7fdee77c5700 1 -- 172.21.15.28:0/444289300 shutdown_connections 2023-11-28T11:09:51.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.461+0000 7fdee77c5700 1 -- 172.21.15.28:0/444289300 wait complete. 2023-11-28T11:09:51.474 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:51.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.601+0000 7f3ccc493700 1 Processor -- start 2023-11-28T11:09:51.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.601+0000 7f3ccc493700 1 -- start start 2023-11-28T11:09:51.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3ccc493700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cc40b1630 0x7f3cc40cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3ccc493700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3cc40b1a00 0x7f3cc40caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3ccc493700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3cc40cb3e0 0x7f3cc4160350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3ccc493700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc40ce550 con 0x7f3cc40b1a00 2023-11-28T11:09:51.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3ccc493700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc40ce830 con 0x7f3cc40cb3e0 2023-11-28T11:09:51.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3ccc493700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40ceb10 con 0x7f3cc40b1630 2023-11-28T11:09:51.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3cc9a2e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3cc40b1a00 0x7f3cc40caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3cc9a2e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3cc40b1a00 0x7f3cc40caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48782/0 (socket says 172.21.15.28:48782) 2023-11-28T11:09:51.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3cc9a2e700 1 -- 172.21.15.28:0/3231403138 learned_addr learned my addr 172.21.15.28:0/3231403138 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:51.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3cca22f700 1 --2- 172.21.15.28:0/3231403138 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cc40b1630 0x7f3cc40cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3ccaa30700 1 --2- 172.21.15.28:0/3231403138 >> v2:172.21.15.59:3300/0 conn(0x7f3cc40cb3e0 0x7f3cc4160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3cc9a2e700 1 -- 172.21.15.28:0/3231403138 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cc40b1630 msgr2=0x7f3cc40cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3cc9a2e700 1 --2- 172.21.15.28:0/3231403138 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cc40b1630 0x7f3cc40cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3cc9a2e700 1 -- 172.21.15.28:0/3231403138 >> v2:172.21.15.59:3300/0 conn(0x7f3cc40cb3e0 msgr2=0x7f3cc4160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3cc9a2e700 1 --2- 172.21.15.28:0/3231403138 >> v2:172.21.15.59:3300/0 conn(0x7f3cc40cb3e0 0x7f3cc4160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3cc9a2e700 1 -- 172.21.15.28:0/3231403138 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc40cedf0 con 0x7f3cc40b1a00 2023-11-28T11:09:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3ccaa30700 1 --2- 172.21.15.28:0/3231403138 >> v2:172.21.15.59:3300/0 conn(0x7f3cc40cb3e0 0x7f3cc4160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.602+0000 7f3cc9a2e700 1 --2- 172.21.15.28:0/3231403138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cc40b1a00 0x7f3cc40caea0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3cc406eb10 tx=0x7f3cb4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac7e46307aa1e85 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.603+0000 7f3cc922d700 1 -- 172.21.15.28:0/3231403138 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb400b870 con 0x7f3cc40b1a00 2023-11-28T11:09:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.603+0000 7f3cc922d700 1 -- 172.21.15.28:0/3231403138 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cb400b9d0 con 0x7f3cc40b1a00 2023-11-28T11:09:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.603+0000 7f3cc922d700 1 -- 172.21.15.28:0/3231403138 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cb400bd60 con 0x7f3cc40b1a00 2023-11-28T11:09:51.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.603+0000 7f3ccc493700 1 -- 172.21.15.28:0/3231403138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cc40b1a00 msgr2=0x7f3cc40caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.603+0000 7f3ccc493700 1 --2- 172.21.15.28:0/3231403138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cc40b1a00 0x7f3cc40caea0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f3cc406eb10 tx=0x7f3cb4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.603+0000 7f3ccc493700 1 -- 172.21.15.28:0/3231403138 shutdown_connections 2023-11-28T11:09:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.603+0000 7f3ccc493700 1 --2- 172.21.15.28:0/3231403138 >> v2:172.21.15.59:3300/0 conn(0x7f3cc40cb3e0 0x7f3cc4160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.603+0000 7f3ccc493700 1 --2- 172.21.15.28:0/3231403138 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cc40b1630 0x7f3cc40cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.603+0000 7f3ccc493700 1 --2- 172.21.15.28:0/3231403138 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cc40b1a00 0x7f3cc40caea0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.603+0000 7f3ccc493700 1 -- 172.21.15.28:0/3231403138 >> 172.21.15.28:0/3231403138 conn(0x7f3cc40cc630 msgr2=0x7f3cc40cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.603+0000 7f3ccc493700 1 -- 172.21.15.28:0/3231403138 shutdown_connections 2023-11-28T11:09:51.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.603+0000 7f3ccc493700 1 -- 172.21.15.28:0/3231403138 wait complete. 2023-11-28T11:09:51.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.605+0000 7f3ccc493700 1 Processor -- start 2023-11-28T11:09:51.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.605+0000 7f3ccc493700 1 -- start start 2023-11-28T11:09:51.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.605+0000 7f3ccc493700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cc40b1630 0x7f3cc40c60e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.605+0000 7f3ccc493700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cc40b1a00 0x7f3cc415b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.605+0000 7f3ccc493700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cc40cb3e0 0x7f3cc40b4e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.605+0000 7f3ccc493700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40c6e20 con 0x7f3cc40cb3e0 2023-11-28T11:09:51.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.605+0000 7f3ccc493700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40c7100 con 0x7f3cc40b1a00 2023-11-28T11:09:51.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.605+0000 7f3ccc493700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc40c73e0 con 0x7f3cc40b1630 2023-11-28T11:09:51.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.605+0000 7f3ccaa30700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cc40cb3e0 0x7f3cc40b4e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.605+0000 7f3ccaa30700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cc40cb3e0 0x7f3cc40b4e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48790/0 (socket says 172.21.15.28:48790) 2023-11-28T11:09:51.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.605+0000 7f3ccaa30700 1 -- 172.21.15.28:0/726994729 learned_addr learned my addr 172.21.15.28:0/726994729 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:51.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.606+0000 7f3cca22f700 1 --2- 172.21.15.28:0/726994729 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cc40b1630 0x7f3cc40c60e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.606+0000 7f3cc9a2e700 1 --2- 172.21.15.28:0/726994729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cc40b1a00 0x7f3cc415b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.606+0000 7f3ccaa30700 1 -- 172.21.15.28:0/726994729 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cc40b1630 msgr2=0x7f3cc40c60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.606+0000 7f3ccaa30700 1 --2- 172.21.15.28:0/726994729 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cc40b1630 0x7f3cc40c60e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.606+0000 7f3ccaa30700 1 -- 172.21.15.28:0/726994729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cc40b1a00 msgr2=0x7f3cc415b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.606+0000 7f3ccaa30700 1 --2- 172.21.15.28:0/726994729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cc40b1a00 0x7f3cc415b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.606+0000 7f3ccaa30700 1 -- 172.21.15.28:0/726994729 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb4009550 con 0x7f3cc40cb3e0 2023-11-28T11:09:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.606+0000 7f3cc9a2e700 1 --2- 172.21.15.28:0/726994729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cc40b1a00 0x7f3cc415b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.606+0000 7f3ccaa30700 1 --2- 172.21.15.28:0/726994729 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cc40cb3e0 0x7f3cc40b4e90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00cd50 tx=0x7f3cbc00a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.606+0000 7f3cbaffd700 1 -- 172.21.15.28:0/726994729 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc00cea0 con 0x7f3cc40cb3e0 2023-11-28T11:09:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.606+0000 7f3cbaffd700 1 -- 172.21.15.28:0/726994729 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cbc00f480 con 0x7f3cc40cb3e0 2023-11-28T11:09:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.606+0000 7f3ccc493700 1 -- 172.21.15.28:0/726994729 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc40b1e70 con 0x7f3cc40cb3e0 2023-11-28T11:09:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.606+0000 7f3cbaffd700 1 -- 172.21.15.28:0/726994729 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cbc00f810 con 0x7f3cc40cb3e0 2023-11-28T11:09:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.606+0000 7f3ccc493700 1 -- 172.21.15.28:0/726994729 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc40cf0d0 con 0x7f3cc40cb3e0 2023-11-28T11:09:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.607+0000 7f3ccc493700 1 -- 172.21.15.28:0/726994729 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc40ce140 con 0x7f3cc40cb3e0 2023-11-28T11:09:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.607+0000 7f3cbaffd700 1 -- 172.21.15.28:0/726994729 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3cbc009030 con 0x7f3cc40cb3e0 2023-11-28T11:09:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.608+0000 7f3cbaffd700 1 --2- 172.21.15.28:0/726994729 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ca80682f0 0x7f3ca806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.608+0000 7f3cbaffd700 1 -- 172.21.15.28:0/726994729 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f3cbc0868e0 con 0x7f3cc40cb3e0 2023-11-28T11:09:51.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.608+0000 7f3cca22f700 1 --2- 172.21.15.28:0/726994729 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ca80682f0 0x7f3ca806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.608+0000 7f3cca22f700 1 --2- 172.21.15.28:0/726994729 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ca80682f0 0x7f3ca806a7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40b4e30 tx=0x7f3cc0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.612+0000 7f3cbaffd700 1 -- 172.21.15.28:0/726994729 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3cbc052e10 con 0x7f3cc40cb3e0 2023-11-28T11:09:51.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.802+0000 7f3ccc493700 1 -- 172.21.15.28:0/726994729 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3cc40739f0 con 0x7f3cc40cb3e0 2023-11-28T11:09:51.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.803+0000 7f3cbaffd700 1 -- 172.21.15.28:0/726994729 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3cbc056430 con 0x7f3cc40cb3e0 2023-11-28T11:09:51.804 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:09:51.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.805+0000 7f3ccc493700 1 -- 172.21.15.28:0/726994729 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ca80682f0 msgr2=0x7f3ca806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.805+0000 7f3ccc493700 1 --2- 172.21.15.28:0/726994729 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ca80682f0 0x7f3ca806a7a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3cc40b4e30 tx=0x7f3cc0008040 comp rx=0 tx=0).stop 2023-11-28T11:09:51.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.805+0000 7f3ccc493700 1 -- 172.21.15.28:0/726994729 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cc40cb3e0 msgr2=0x7f3cc40b4e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.805+0000 7f3ccc493700 1 --2- 172.21.15.28:0/726994729 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cc40cb3e0 0x7f3cc40b4e90 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f3cbc00cd50 tx=0x7f3cbc00a560 comp rx=0 tx=0).stop 2023-11-28T11:09:51.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.805+0000 7f3ccc493700 1 -- 172.21.15.28:0/726994729 shutdown_connections 2023-11-28T11:09:51.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.805+0000 7f3ccc493700 1 --2- 172.21.15.28:0/726994729 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ca80682f0 0x7f3ca806a7a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.805+0000 7f3ccc493700 1 --2- 172.21.15.28:0/726994729 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cc40cb3e0 0x7f3cc40b4e90 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.805+0000 7f3ccc493700 1 --2- 172.21.15.28:0/726994729 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cc40b1630 0x7f3cc40c60e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.805+0000 7f3ccc493700 1 --2- 172.21.15.28:0/726994729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cc40b1a00 0x7f3cc415b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.805+0000 7f3ccc493700 1 -- 172.21.15.28:0/726994729 >> 172.21.15.28:0/726994729 conn(0x7f3cc40cc630 msgr2=0x7f3cc406c130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.805+0000 7f3ccc493700 1 -- 172.21.15.28:0/726994729 shutdown_connections 2023-11-28T11:09:51.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.805+0000 7f3ccc493700 1 -- 172.21.15.28:0/726994729 wait complete. 2023-11-28T11:09:51.819 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:51.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.945+0000 7fb85edaf700 1 Processor -- start 2023-11-28T11:09:51.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.946+0000 7fb85edaf700 1 -- start start 2023-11-28T11:09:51.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.946+0000 7fb85edaf700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb858159500 0x7fb8580cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.946+0000 7fb85edaf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb8580b1d40 0x7fb8580ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.951 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.946+0000 7fb85edaf700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb85815e040 con 0x7fb85815dbf0 2023-11-28T11:09:51.951 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.946+0000 7fb85edaf700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8580cede0 con 0x7fb858159500 2023-11-28T11:09:51.951 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.946+0000 7fb85edaf700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8580ceff0 con 0x7fb8580b1d40 2023-11-28T11:09:51.951 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.947+0000 7fb85cb4b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb858159500 0x7fb8580cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.951 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.947+0000 7fb85cb4b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb858159500 0x7fb8580cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48794/0 (socket says 172.21.15.28:48794) 2023-11-28T11:09:51.951 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.947+0000 7fb85cb4b700 1 -- 172.21.15.28:0/3401845144 learned_addr learned my addr 172.21.15.28:0/3401845144 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:51.951 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.947+0000 7fb85d34c700 1 --2- 172.21.15.28:0/3401845144 >> v2:172.21.15.59:3300/0 conn(0x7fb8580b1d40 0x7fb8580ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.947+0000 7fb85cb4b700 1 -- 172.21.15.28:0/3401845144 >> v2:172.21.15.59:3300/0 conn(0x7fb8580b1d40 msgr2=0x7fb8580ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:51.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.947+0000 7fb85cb4b700 1 --2- 172.21.15.28:0/3401845144 >> v2:172.21.15.59:3300/0 conn(0x7fb8580b1d40 0x7fb8580ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.947+0000 7fb85cb4b700 1 -- 172.21.15.28:0/3401845144 >> v1:172.21.15.59:6789/0 conn(0x7fb85815dbf0 legacy=0x7fb8580b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.947+0000 7fb85cb4b700 1 -- 172.21.15.28:0/3401845144 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8580cf2d0 con 0x7fb858159500 2023-11-28T11:09:51.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.947+0000 7fb85d34c700 1 --2- 172.21.15.28:0/3401845144 >> v2:172.21.15.59:3300/0 conn(0x7fb8580b1d40 0x7fb8580ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:51.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.947+0000 7fb85cb4b700 1 --2- 172.21.15.28:0/3401845144 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb858159500 0x7fb8580cd140 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb8580cfb50 tx=0x7fb84c00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad04b2c3ead8b752 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.947+0000 7fb8577fe700 1 -- 172.21.15.28:0/3401845144 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c00dd30 con 0x7fb858159500 2023-11-28T11:09:51.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.947+0000 7fb8577fe700 1 -- 172.21.15.28:0/3401845144 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb84c00de90 con 0x7fb858159500 2023-11-28T11:09:51.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.947+0000 7fb8577fe700 1 -- 172.21.15.28:0/3401845144 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb84c010690 con 0x7fb858159500 2023-11-28T11:09:51.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.948+0000 7fb85edaf700 1 -- 172.21.15.28:0/3401845144 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb858159500 msgr2=0x7fb8580cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.948+0000 7fb85edaf700 1 --2- 172.21.15.28:0/3401845144 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb858159500 0x7fb8580cd140 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fb8580cfb50 tx=0x7fb84c00be70 comp rx=0 tx=0).stop 2023-11-28T11:09:51.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.948+0000 7fb85edaf700 1 -- 172.21.15.28:0/3401845144 shutdown_connections 2023-11-28T11:09:51.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.948+0000 7fb85edaf700 1 --2- 172.21.15.28:0/3401845144 >> v2:172.21.15.59:3300/0 conn(0x7fb8580b1d40 0x7fb8580ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.948+0000 7fb85edaf700 1 --2- 172.21.15.28:0/3401845144 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb858159500 0x7fb8580cd140 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.948+0000 7fb85edaf700 1 -- 172.21.15.28:0/3401845144 >> 172.21.15.28:0/3401845144 conn(0x7fb8580cc630 msgr2=0x7fb8580cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:51.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.948+0000 7fb85edaf700 1 -- 172.21.15.28:0/3401845144 shutdown_connections 2023-11-28T11:09:51.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.948+0000 7fb85edaf700 1 -- 172.21.15.28:0/3401845144 wait complete. 2023-11-28T11:09:51.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.949+0000 7fb85edaf700 1 Processor -- start 2023-11-28T11:09:51.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.949+0000 7fb85edaf700 1 -- start start 2023-11-28T11:09:51.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb85edaf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8580b1d40 0x7fb8580b4410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb85edaf700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb858159500 0x7fb858146e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb85edaf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb85815dbf0 0x7fb8580c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb85edaf700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb85815a310 con 0x7fb858159500 2023-11-28T11:09:51.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb85edaf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580b4950 con 0x7fb85815dbf0 2023-11-28T11:09:51.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb85edaf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8580b4be0 con 0x7fb8580b1d40 2023-11-28T11:09:51.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb857fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb858159500 0x7fb858146e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb857fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb858159500 0x7fb858146e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48802/0 (socket says 172.21.15.28:48802) 2023-11-28T11:09:51.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb857fff700 1 -- 172.21.15.28:0/3866527033 learned_addr learned my addr 172.21.15.28:0/3866527033 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:51.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb85d34c700 1 --2- 172.21.15.28:0/3866527033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb85815dbf0 0x7fb8580c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb85cb4b700 1 --2- 172.21.15.28:0/3866527033 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8580b1d40 0x7fb8580b4410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb857fff700 1 -- 172.21.15.28:0/3866527033 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8580b1d40 msgr2=0x7fb8580b4410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb857fff700 1 --2- 172.21.15.28:0/3866527033 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8580b1d40 0x7fb8580b4410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb857fff700 1 -- 172.21.15.28:0/3866527033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb85815dbf0 msgr2=0x7fb8580c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:51.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb857fff700 1 --2- 172.21.15.28:0/3866527033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb85815dbf0 0x7fb8580c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:51.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb857fff700 1 -- 172.21.15.28:0/3866527033 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb84c00b7f0 con 0x7fb858159500 2023-11-28T11:09:51.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb85d34c700 1 --2- 172.21.15.28:0/3866527033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb85815dbf0 0x7fb8580c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:51.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.950+0000 7fb857fff700 1 --2- 172.21.15.28:0/3866527033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb858159500 0x7fb858146e30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c420 tx=0x7fb84400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.951+0000 7fb8557fa700 1 -- 172.21.15.28:0/3866527033 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8440052d0 con 0x7fb858159500 2023-11-28T11:09:51.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.951+0000 7fb8557fa700 1 -- 172.21.15.28:0/3866527033 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb844005430 con 0x7fb858159500 2023-11-28T11:09:51.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.951+0000 7fb85edaf700 1 -- 172.21.15.28:0/3866527033 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb85815d3f0 con 0x7fb858159500 2023-11-28T11:09:51.958 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.951+0000 7fb8557fa700 1 -- 172.21.15.28:0/3866527033 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8440038c0 con 0x7fb858159500 2023-11-28T11:09:51.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.951+0000 7fb85edaf700 1 -- 172.21.15.28:0/3866527033 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb85815d110 con 0x7fb858159500 2023-11-28T11:09:51.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.952+0000 7fb85edaf700 1 -- 172.21.15.28:0/3866527033 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb85815d6d0 con 0x7fb858159500 2023-11-28T11:09:51.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.952+0000 7fb8557fa700 1 -- 172.21.15.28:0/3866527033 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb844019030 con 0x7fb858159500 2023-11-28T11:09:51.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.952+0000 7fb8557fa700 1 --2- 172.21.15.28:0/3866527033 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb83c0682f0 0x7fb83c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:51.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.953+0000 7fb8557fa700 1 -- 172.21.15.28:0/3866527033 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb844085250 con 0x7fb858159500 2023-11-28T11:09:51.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.953+0000 7fb85cb4b700 1 --2- 172.21.15.28:0/3866527033 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb83c0682f0 0x7fb83c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:51.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.953+0000 7fb85cb4b700 1 --2- 172.21.15.28:0/3866527033 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb83c0682f0 0x7fb83c06a7a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b6f0 tx=0x7fb84c012000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:51.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:51.957+0000 7fb8557fa700 1 -- 172.21.15.28:0/3866527033 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb844051780 con 0x7fb858159500 2023-11-28T11:09:52.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.152+0000 7fb85edaf700 1 -- 172.21.15.28:0/3866527033 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb8580739f0 con 0x7fb858159500 2023-11-28T11:09:52.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.152+0000 7fb8557fa700 1 -- 172.21.15.28:0/3866527033 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb844054da0 con 0x7fb858159500 2023-11-28T11:09:52.154 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:52.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.154+0000 7fb85edaf700 1 -- 172.21.15.28:0/3866527033 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb83c0682f0 msgr2=0x7fb83c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.154+0000 7fb85edaf700 1 --2- 172.21.15.28:0/3866527033 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb83c0682f0 0x7fb83c06a7a0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb84c00b6f0 tx=0x7fb84c012000 comp rx=0 tx=0).stop 2023-11-28T11:09:52.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.154+0000 7fb85edaf700 1 -- 172.21.15.28:0/3866527033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb858159500 msgr2=0x7fb858146e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.154+0000 7fb85edaf700 1 --2- 172.21.15.28:0/3866527033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb858159500 0x7fb858146e30 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fb84400c420 tx=0x7fb84400c7e0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.154+0000 7fb85edaf700 1 -- 172.21.15.28:0/3866527033 shutdown_connections 2023-11-28T11:09:52.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.154+0000 7fb85edaf700 1 --2- 172.21.15.28:0/3866527033 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb83c0682f0 0x7fb83c06a7a0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.155+0000 7fb85edaf700 1 --2- 172.21.15.28:0/3866527033 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb85815dbf0 0x7fb8580c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.155+0000 7fb85edaf700 1 --2- 172.21.15.28:0/3866527033 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb8580b1d40 0x7fb8580b4410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.155+0000 7fb85edaf700 1 --2- 172.21.15.28:0/3866527033 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb858159500 0x7fb858146e30 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.155+0000 7fb85edaf700 1 -- 172.21.15.28:0/3866527033 >> 172.21.15.28:0/3866527033 conn(0x7fb8580cc630 msgr2=0x7fb85806b610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.155+0000 7fb85edaf700 1 -- 172.21.15.28:0/3866527033 shutdown_connections 2023-11-28T11:09:52.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.155+0000 7fb85edaf700 1 -- 172.21.15.28:0/3866527033 wait complete. 2023-11-28T11:09:52.168 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:52.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.295+0000 7f93d47c9700 1 Processor -- start 2023-11-28T11:09:52.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.295+0000 7f93d47c9700 1 -- start start 2023-11-28T11:09:52.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.296+0000 7f93d47c9700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f93cc0cf5f0 0x7f93cc0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.296+0000 7f93d47c9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93cc0cb3e0 0x7f93cc06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.296+0000 7f93d47c9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93cc0ce550 con 0x7f93cc0b1630 2023-11-28T11:09:52.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.296+0000 7f93d47c9700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93cc0ce830 con 0x7f93cc0cf5f0 2023-11-28T11:09:52.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.296+0000 7f93d47c9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc0ceb10 con 0x7f93cc0cb3e0 2023-11-28T11:09:52.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.296+0000 7f93d1d64700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f93cc0cf5f0 0x7f93cc0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.296+0000 7f93d1d64700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f93cc0cf5f0 0x7f93cc0caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48812/0 (socket says 172.21.15.28:48812) 2023-11-28T11:09:52.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.296+0000 7f93d1d64700 1 -- 172.21.15.28:0/2212982940 learned_addr learned my addr 172.21.15.28:0/2212982940 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:52.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.296+0000 7f93d1d64700 1 -- 172.21.15.28:0/2212982940 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93cc0cb3e0 msgr2=0x7f93cc06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.296+0000 7f93d2d66700 1 --2- 172.21.15.28:0/2212982940 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93cc0cb3e0 0x7f93cc06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.296+0000 7f93d1d64700 1 --2- 172.21.15.28:0/2212982940 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93cc0cb3e0 0x7f93cc06afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.296+0000 7f93d1d64700 1 -- 172.21.15.28:0/2212982940 >> v1:172.21.15.59:6789/0 conn(0x7f93cc0b1630 legacy=0x7f93cc0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.296+0000 7f93d1d64700 1 -- 172.21.15.28:0/2212982940 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93cc0cedf0 con 0x7f93cc0cf5f0 2023-11-28T11:09:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.296+0000 7f93d2d66700 1 --2- 172.21.15.28:0/2212982940 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93cc0cb3e0 0x7f93cc06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.296+0000 7f93d1d64700 1 --2- 172.21.15.28:0/2212982940 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93cc0cf5f0 0x7f93cc0caea0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009920 tx=0x7f93c8009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1418d5da84c6820 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.297+0000 7f93d1563700 1 -- 172.21.15.28:0/2212982940 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c8004230 con 0x7f93cc0cf5f0 2023-11-28T11:09:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.297+0000 7f93d1563700 1 -- 172.21.15.28:0/2212982940 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c8004390 con 0x7f93cc0cf5f0 2023-11-28T11:09:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.297+0000 7f93d1563700 1 -- 172.21.15.28:0/2212982940 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c8009e70 con 0x7f93cc0cf5f0 2023-11-28T11:09:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.297+0000 7f93d47c9700 1 -- 172.21.15.28:0/2212982940 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93cc0cf5f0 msgr2=0x7f93cc0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.297+0000 7f93d47c9700 1 --2- 172.21.15.28:0/2212982940 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93cc0cf5f0 0x7f93cc0caea0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f93c8009920 tx=0x7f93c8009c30 comp rx=0 tx=0).stop 2023-11-28T11:09:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.297+0000 7f93d47c9700 1 -- 172.21.15.28:0/2212982940 shutdown_connections 2023-11-28T11:09:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.297+0000 7f93d47c9700 1 --2- 172.21.15.28:0/2212982940 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93cc0cb3e0 0x7f93cc06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.297+0000 7f93d47c9700 1 --2- 172.21.15.28:0/2212982940 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93cc0cf5f0 0x7f93cc0caea0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.297+0000 7f93d47c9700 1 -- 172.21.15.28:0/2212982940 >> 172.21.15.28:0/2212982940 conn(0x7f93cc0cc630 msgr2=0x7f93cc0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.297+0000 7f93d47c9700 1 -- 172.21.15.28:0/2212982940 shutdown_connections 2023-11-28T11:09:52.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.297+0000 7f93d47c9700 1 -- 172.21.15.28:0/2212982940 wait complete. 2023-11-28T11:09:52.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.298+0000 7f93d47c9700 1 Processor -- start 2023-11-28T11:09:52.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.299+0000 7f93d47c9700 1 -- start start 2023-11-28T11:09:52.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.299+0000 7f93d47c9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93cc0b1630 0x7f93cc157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.299+0000 7f93d47c9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93cc0cb3e0 0x7f93cc0c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.299+0000 7f93d47c9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93cc0cf5f0 0x7f93cc0ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.299+0000 7f93d47c9700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc157ca0 con 0x7f93cc0cb3e0 2023-11-28T11:09:52.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.299+0000 7f93d47c9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc0c8050 con 0x7f93cc0cf5f0 2023-11-28T11:09:52.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.299+0000 7f93d47c9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93cc0c7d70 con 0x7f93cc0b1630 2023-11-28T11:09:52.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.299+0000 7f93d1d64700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93cc0cb3e0 0x7f93cc0c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.299+0000 7f93d1d64700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93cc0cb3e0 0x7f93cc0c7490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48816/0 (socket says 172.21.15.28:48816) 2023-11-28T11:09:52.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.299+0000 7f93d1d64700 1 -- 172.21.15.28:0/1888929290 learned_addr learned my addr 172.21.15.28:0/1888929290 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:52.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.299+0000 7f93d2d66700 1 --2- 172.21.15.28:0/1888929290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93cc0cf5f0 0x7f93cc0ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.299+0000 7f93d2565700 1 --2- 172.21.15.28:0/1888929290 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93cc0b1630 0x7f93cc157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.300+0000 7f93d1d64700 1 -- 172.21.15.28:0/1888929290 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93cc0b1630 msgr2=0x7f93cc157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.300+0000 7f93d1d64700 1 --2- 172.21.15.28:0/1888929290 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93cc0b1630 0x7f93cc157450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.300+0000 7f93d1d64700 1 -- 172.21.15.28:0/1888929290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93cc0cf5f0 msgr2=0x7f93cc0ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.300+0000 7f93d1d64700 1 --2- 172.21.15.28:0/1888929290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93cc0cf5f0 0x7f93cc0ac1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.300+0000 7f93d1d64700 1 -- 172.21.15.28:0/1888929290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c8009550 con 0x7f93cc0cb3e0 2023-11-28T11:09:52.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.300+0000 7f93d1d64700 1 --2- 172.21.15.28:0/1888929290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93cc0cb3e0 0x7f93cc0c7490 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f93c80098f0 tx=0x7f93c8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.300+0000 7f93c2ffd700 1 -- 172.21.15.28:0/1888929290 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c80036a0 con 0x7f93cc0cb3e0 2023-11-28T11:09:52.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.300+0000 7f93c2ffd700 1 -- 172.21.15.28:0/1888929290 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93c8003820 con 0x7f93cc0cb3e0 2023-11-28T11:09:52.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.300+0000 7f93d47c9700 1 -- 172.21.15.28:0/1888929290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93cc0ce140 con 0x7f93cc0cb3e0 2023-11-28T11:09:52.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.300+0000 7f93c2ffd700 1 -- 172.21.15.28:0/1888929290 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93c8003bd0 con 0x7f93cc0cb3e0 2023-11-28T11:09:52.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.300+0000 7f93d47c9700 1 -- 172.21.15.28:0/1888929290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93cc0cf0d0 con 0x7f93cc0cb3e0 2023-11-28T11:09:52.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.301+0000 7f93d47c9700 1 -- 172.21.15.28:0/1888929290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93cc0ce2d0 con 0x7f93cc0cb3e0 2023-11-28T11:09:52.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.301+0000 7f93c2ffd700 1 -- 172.21.15.28:0/1888929290 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f93c8017020 con 0x7f93cc0cb3e0 2023-11-28T11:09:52.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.302+0000 7f93c2ffd700 1 --2- 172.21.15.28:0/1888929290 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f93b00682f0 0x7f93b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.302+0000 7f93d2565700 1 --2- 172.21.15.28:0/1888929290 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f93b00682f0 0x7f93b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.302+0000 7f93c2ffd700 1 -- 172.21.15.28:0/1888929290 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f93c8087870 con 0x7f93cc0cb3e0 2023-11-28T11:09:52.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.302+0000 7f93d2565700 1 --2- 172.21.15.28:0/1888929290 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f93b00682f0 0x7f93b006a7a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f93bc004f30 tx=0x7f93bc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.306+0000 7f93c2ffd700 1 -- 172.21.15.28:0/1888929290 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f93c8053da0 con 0x7f93cc0cb3e0 2023-11-28T11:09:52.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.497+0000 7f93d47c9700 1 -- 172.21.15.28:0/1888929290 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f93cc0b3c70 con 0x7f93cc0cb3e0 2023-11-28T11:09:52.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.497+0000 7f93c2ffd700 1 -- 172.21.15.28:0/1888929290 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7f93c80573c0 con 0x7f93cc0cb3e0 2023-11-28T11:09:52.499 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:09:52.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.499+0000 7f93d47c9700 1 -- 172.21.15.28:0/1888929290 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f93b00682f0 msgr2=0x7f93b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.499+0000 7f93d47c9700 1 --2- 172.21.15.28:0/1888929290 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f93b00682f0 0x7f93b006a7a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f93bc004f30 tx=0x7f93bc009040 comp rx=0 tx=0).stop 2023-11-28T11:09:52.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.499+0000 7f93d47c9700 1 -- 172.21.15.28:0/1888929290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93cc0cb3e0 msgr2=0x7f93cc0c7490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.499+0000 7f93d47c9700 1 --2- 172.21.15.28:0/1888929290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93cc0cb3e0 0x7f93cc0c7490 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f93c80098f0 tx=0x7f93c8004750 comp rx=0 tx=0).stop 2023-11-28T11:09:52.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.499+0000 7f93d47c9700 1 -- 172.21.15.28:0/1888929290 shutdown_connections 2023-11-28T11:09:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.499+0000 7f93d47c9700 1 --2- 172.21.15.28:0/1888929290 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f93b00682f0 0x7f93b006a7a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.499+0000 7f93d47c9700 1 --2- 172.21.15.28:0/1888929290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f93cc0cf5f0 0x7f93cc0ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.499+0000 7f93d47c9700 1 --2- 172.21.15.28:0/1888929290 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f93cc0b1630 0x7f93cc157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.499+0000 7f93d47c9700 1 --2- 172.21.15.28:0/1888929290 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f93cc0cb3e0 0x7f93cc0c7490 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.499+0000 7f93d47c9700 1 -- 172.21.15.28:0/1888929290 >> 172.21.15.28:0/1888929290 conn(0x7f93cc0cc630 msgr2=0x7f93cc0b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.499+0000 7f93d47c9700 1 -- 172.21.15.28:0/1888929290 shutdown_connections 2023-11-28T11:09:52.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.499+0000 7f93d47c9700 1 -- 172.21.15.28:0/1888929290 wait complete. 2023-11-28T11:09:52.513 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:52.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.639+0000 7f4d238f0700 1 Processor -- start 2023-11-28T11:09:52.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.639+0000 7f4d238f0700 1 -- start start 2023-11-28T11:09:52.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.640+0000 7f4d238f0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d1c0cf5f0 0x7f4d1c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.640+0000 7f4d238f0700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4d1c0cb3e0 0x7f4d1c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.640+0000 7f4d238f0700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d1c0ce550 con 0x7f4d1c0b1630 2023-11-28T11:09:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.640+0000 7f4d238f0700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d1c0cfa60 con 0x7f4d1c0cb3e0 2023-11-28T11:09:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.640+0000 7f4d238f0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0ce830 con 0x7f4d1c0cf5f0 2023-11-28T11:09:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.640+0000 7f4d21e8d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4d1c0cb3e0 0x7f4d1c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.640+0000 7f4d2168c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4d1c0b1630 0x7f4d1c0b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36880/0 (socket says 172.21.15.28:36880) 2023-11-28T11:09:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.640+0000 7f4d2168c700 1 -- 172.21.15.28:0/1410038348 learned_addr learned my addr 172.21.15.28:0/1410038348 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.640+0000 7f4d20e8b700 1 --2- 172.21.15.28:0/1410038348 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d1c0cf5f0 0x7f4d1c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.640+0000 7f4d21e8d700 1 -- 172.21.15.28:0/1410038348 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d1c0cf5f0 msgr2=0x7f4d1c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:52.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.640+0000 7f4d21e8d700 1 --2- 172.21.15.28:0/1410038348 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d1c0cf5f0 0x7f4d1c0caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.640+0000 7f4d21e8d700 1 -- 172.21.15.28:0/1410038348 >> v1:172.21.15.28:6789/0 conn(0x7f4d1c0b1630 legacy=0x7f4d1c0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.640+0000 7f4d13fff700 1 -- 172.21.15.28:0/1410038348 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616171821 0 0) 0x7f4d1c0ce550 con 0x7f4d1c0b1630 2023-11-28T11:09:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.640+0000 7f4d21e8d700 1 -- 172.21.15.28:0/1410038348 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d1c0ceb10 con 0x7f4d1c0cb3e0 2023-11-28T11:09:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.640+0000 7f4d20e8b700 1 --2- 172.21.15.28:0/1410038348 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d1c0cf5f0 0x7f4d1c0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.641+0000 7f4d21e8d700 1 --2- 172.21.15.28:0/1410038348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d1c0cb3e0 0x7f4d1c06afc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009d90 tx=0x7f4d1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4db225c19ed244ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.641+0000 7f4d13fff700 1 -- 172.21.15.28:0/1410038348 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d1400e070 con 0x7f4d1c0cb3e0 2023-11-28T11:09:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.641+0000 7f4d13fff700 1 -- 172.21.15.28:0/1410038348 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d1400bbd0 con 0x7f4d1c0cb3e0 2023-11-28T11:09:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.641+0000 7f4d13fff700 1 -- 172.21.15.28:0/1410038348 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14012430 con 0x7f4d1c0cb3e0 2023-11-28T11:09:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.641+0000 7f4d238f0700 1 -- 172.21.15.28:0/1410038348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d1c0cb3e0 msgr2=0x7f4d1c06afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.641+0000 7f4d238f0700 1 --2- 172.21.15.28:0/1410038348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d1c0cb3e0 0x7f4d1c06afc0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f4d14009d90 tx=0x7f4d1400b330 comp rx=0 tx=0).stop 2023-11-28T11:09:52.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.641+0000 7f4d238f0700 1 -- 172.21.15.28:0/1410038348 shutdown_connections 2023-11-28T11:09:52.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.641+0000 7f4d238f0700 1 --2- 172.21.15.28:0/1410038348 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d1c0cb3e0 0x7f4d1c06afc0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.641+0000 7f4d238f0700 1 --2- 172.21.15.28:0/1410038348 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d1c0cf5f0 0x7f4d1c0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.641+0000 7f4d238f0700 1 -- 172.21.15.28:0/1410038348 >> 172.21.15.28:0/1410038348 conn(0x7f4d1c0cc630 msgr2=0x7f4d1c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.641+0000 7f4d238f0700 1 -- 172.21.15.28:0/1410038348 shutdown_connections 2023-11-28T11:09:52.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.641+0000 7f4d238f0700 1 -- 172.21.15.28:0/1410038348 wait complete. 2023-11-28T11:09:52.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.643+0000 7f4d238f0700 1 Processor -- start 2023-11-28T11:09:52.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.643+0000 7f4d238f0700 1 -- start start 2023-11-28T11:09:52.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.644+0000 7f4d238f0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d1c0b1630 0x7f4d1c0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.644+0000 7f4d238f0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d1c0cb3e0 0x7f4d1c155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.644+0000 7f4d238f0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d1c0cf5f0 0x7f4d1c0b4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.644+0000 7f4d238f0700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0b3ed0 con 0x7f4d1c0cf5f0 2023-11-28T11:09:52.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.644+0000 7f4d238f0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c1567f0 con 0x7f4d1c0b1630 2023-11-28T11:09:52.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.644+0000 7f4d238f0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d1c0c9c20 con 0x7f4d1c0cb3e0 2023-11-28T11:09:52.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.644+0000 7f4d21e8d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d1c0cf5f0 0x7f4d1c0b4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.644+0000 7f4d21e8d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d1c0cf5f0 0x7f4d1c0b4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48830/0 (socket says 172.21.15.28:48830) 2023-11-28T11:09:52.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.644+0000 7f4d21e8d700 1 -- 172.21.15.28:0/1437563942 learned_addr learned my addr 172.21.15.28:0/1437563942 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:52.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.644+0000 7f4d20e8b700 1 --2- 172.21.15.28:0/1437563942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d1c0cb3e0 0x7f4d1c155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.644+0000 7f4d2168c700 1 --2- 172.21.15.28:0/1437563942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d1c0b1630 0x7f4d1c0c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.644+0000 7f4d21e8d700 1 -- 172.21.15.28:0/1437563942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d1c0cb3e0 msgr2=0x7f4d1c155c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.644+0000 7f4d21e8d700 1 --2- 172.21.15.28:0/1437563942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d1c0cb3e0 0x7f4d1c155c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.644+0000 7f4d21e8d700 1 -- 172.21.15.28:0/1437563942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d1c0b1630 msgr2=0x7f4d1c0c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.645+0000 7f4d21e8d700 1 --2- 172.21.15.28:0/1437563942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d1c0b1630 0x7f4d1c0c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.645+0000 7f4d21e8d700 1 -- 172.21.15.28:0/1437563942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d14009a70 con 0x7f4d1c0cf5f0 2023-11-28T11:09:52.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.645+0000 7f4d2168c700 1 --2- 172.21.15.28:0/1437563942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d1c0b1630 0x7f4d1c0c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:52.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.645+0000 7f4d21e8d700 1 --2- 172.21.15.28:0/1437563942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d1c0cf5f0 0x7f4d1c0b4d80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c066a20 tx=0x7f4d14004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.645+0000 7f4d11ffb700 1 -- 172.21.15.28:0/1437563942 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14012430 con 0x7f4d1c0cf5f0 2023-11-28T11:09:52.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.645+0000 7f4d11ffb700 1 -- 172.21.15.28:0/1437563942 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d14020070 con 0x7f4d1c0cf5f0 2023-11-28T11:09:52.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.645+0000 7f4d11ffb700 1 -- 172.21.15.28:0/1437563942 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d14004280 con 0x7f4d1c0cf5f0 2023-11-28T11:09:52.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.645+0000 7f4d238f0700 1 -- 172.21.15.28:0/1437563942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d1c0cf0d0 con 0x7f4d1c0cf5f0 2023-11-28T11:09:52.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.645+0000 7f4d238f0700 1 -- 172.21.15.28:0/1437563942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d1c0cedf0 con 0x7f4d1c0cf5f0 2023-11-28T11:09:52.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.646+0000 7f4d238f0700 1 -- 172.21.15.28:0/1437563942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d1c0ce140 con 0x7f4d1c0cf5f0 2023-11-28T11:09:52.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.646+0000 7f4d11ffb700 1 -- 172.21.15.28:0/1437563942 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4d1402e070 con 0x7f4d1c0cf5f0 2023-11-28T11:09:52.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.647+0000 7f4d11ffb700 1 --2- 172.21.15.28:0/1437563942 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4d000682f0 0x7f4d0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.647+0000 7f4d11ffb700 1 -- 172.21.15.28:0/1437563942 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f4d14022070 con 0x7f4d1c0cf5f0 2023-11-28T11:09:52.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.647+0000 7f4d2168c700 1 --2- 172.21.15.28:0/1437563942 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4d000682f0 0x7f4d0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.647+0000 7f4d2168c700 1 --2- 172.21.15.28:0/1437563942 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4d000682f0 0x7f4d0006a7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c000f60 tx=0x7f4d0c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.651+0000 7f4d11ffb700 1 -- 172.21.15.28:0/1437563942 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4d14054350 con 0x7f4d1c0cf5f0 2023-11-28T11:09:52.844 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.843+0000 7f4d238f0700 1 -- 172.21.15.28:0/1437563942 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f4d1c0c6630 con 0x7f4d1c0cf5f0 2023-11-28T11:09:52.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.843+0000 7f4d11ffb700 1 -- 172.21.15.28:0/1437563942 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4d1402c020 con 0x7f4d1c0cf5f0 2023-11-28T11:09:52.845 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.845+0000 7f4d238f0700 1 -- 172.21.15.28:0/1437563942 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4d000682f0 msgr2=0x7f4d0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.845+0000 7f4d238f0700 1 --2- 172.21.15.28:0/1437563942 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4d000682f0 0x7f4d0006a7a0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f4d0c000f60 tx=0x7f4d0c009040 comp rx=0 tx=0).stop 2023-11-28T11:09:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.845+0000 7f4d238f0700 1 -- 172.21.15.28:0/1437563942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d1c0cf5f0 msgr2=0x7f4d1c0b4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.845+0000 7f4d238f0700 1 --2- 172.21.15.28:0/1437563942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d1c0cf5f0 0x7f4d1c0b4d80 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f4d1c066a20 tx=0x7f4d14004ad0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.845+0000 7f4d238f0700 1 -- 172.21.15.28:0/1437563942 shutdown_connections 2023-11-28T11:09:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.845+0000 7f4d238f0700 1 --2- 172.21.15.28:0/1437563942 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4d000682f0 0x7f4d0006a7a0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.845+0000 7f4d238f0700 1 --2- 172.21.15.28:0/1437563942 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d1c0cf5f0 0x7f4d1c0b4d80 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.845+0000 7f4d238f0700 1 --2- 172.21.15.28:0/1437563942 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d1c0b1630 0x7f4d1c0c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.845+0000 7f4d238f0700 1 --2- 172.21.15.28:0/1437563942 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d1c0cb3e0 0x7f4d1c155c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.845+0000 7f4d238f0700 1 -- 172.21.15.28:0/1437563942 >> 172.21.15.28:0/1437563942 conn(0x7f4d1c0cc630 msgr2=0x7f4d1c0cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.845+0000 7f4d238f0700 1 -- 172.21.15.28:0/1437563942 shutdown_connections 2023-11-28T11:09:52.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.845+0000 7f4d238f0700 1 -- 172.21.15.28:0/1437563942 wait complete. 2023-11-28T11:09:52.860 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:09:52.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.983+0000 7fb403ae9700 1 Processor -- start 2023-11-28T11:09:52.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.984+0000 7fb403ae9700 1 -- start start 2023-11-28T11:09:52.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.984+0000 7fb403ae9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3fc15c990 0x7fb3fc0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.984+0000 7fb403ae9700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3fc0ce340 con 0x7fb3fc161090 2023-11-28T11:09:52.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.984+0000 7fb403ae9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3fc0cebe0 con 0x7fb3fc0cc310 2023-11-28T11:09:52.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.984+0000 7fb403ae9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc0cb3f0 con 0x7fb3fc15c990 2023-11-28T11:09:52.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.985+0000 7fb401084700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb3fc161090 0x7fb3fc0ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36892/0 (socket says 172.21.15.28:36892) 2023-11-28T11:09:52.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.985+0000 7fb401084700 1 -- 172.21.15.28:0/2714450637 learned_addr learned my addr 172.21.15.28:0/2714450637 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:52.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.985+0000 7fb401885700 1 --2- 172.21.15.28:0/2714450637 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3fc15c990 0x7fb3fc0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.985+0000 7fb400883700 1 -- 172.21.15.28:0/2714450637 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3419307267 0 0) 0x7fb3fc0ce340 con 0x7fb3fc161090 2023-11-28T11:09:52.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.985+0000 7fb400883700 1 -- 172.21.15.28:0/2714450637 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3e80035b0 con 0x7fb3fc161090 2023-11-28T11:09:52.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.985+0000 7fb400883700 1 -- 172.21.15.28:0/2714450637 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780727843 0 0) 0x7fb3fc0cebe0 con 0x7fb3fc0cc310 2023-11-28T11:09:52.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.985+0000 7fb400883700 1 -- 172.21.15.28:0/2714450637 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3fc0ce340 con 0x7fb3fc0cc310 2023-11-28T11:09:52.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.985+0000 7fb401885700 1 -- 172.21.15.28:0/2714450637 >> v1:172.21.15.59:6789/0 conn(0x7fb3fc0cc310 legacy=0x7fb3fc0cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.985+0000 7fb401885700 1 -- 172.21.15.28:0/2714450637 >> v1:172.21.15.28:6789/0 conn(0x7fb3fc161090 legacy=0x7fb3fc0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.985+0000 7fb401885700 1 -- 172.21.15.28:0/2714450637 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3fc0ce590 con 0x7fb3fc15c990 2023-11-28T11:09:52.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.985+0000 7fb401885700 1 --2- 172.21.15.28:0/2714450637 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3fc15c990 0x7fb3fc0cf070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009870 tx=0x7fb3f8009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f79fdef26356815 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.985+0000 7fb400883700 1 -- 172.21.15.28:0/2714450637 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8011830 con 0x7fb3fc15c990 2023-11-28T11:09:52.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.985+0000 7fb400883700 1 -- 172.21.15.28:0/2714450637 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f8011990 con 0x7fb3fc15c990 2023-11-28T11:09:52.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.986+0000 7fb400883700 1 -- 172.21.15.28:0/2714450637 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f8011d20 con 0x7fb3fc15c990 2023-11-28T11:09:52.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.986+0000 7fb403ae9700 1 -- 172.21.15.28:0/2714450637 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3fc15c990 msgr2=0x7fb3fc0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.986+0000 7fb403ae9700 1 --2- 172.21.15.28:0/2714450637 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3fc15c990 0x7fb3fc0cf070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb3f8009870 tx=0x7fb3f8009b80 comp rx=0 tx=0).stop 2023-11-28T11:09:52.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.986+0000 7fb403ae9700 1 -- 172.21.15.28:0/2714450637 shutdown_connections 2023-11-28T11:09:52.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.986+0000 7fb403ae9700 1 --2- 172.21.15.28:0/2714450637 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3fc15c990 0x7fb3fc0cf070 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.986+0000 7fb403ae9700 1 -- 172.21.15.28:0/2714450637 >> 172.21.15.28:0/2714450637 conn(0x7fb3fc0b1650 msgr2=0x7fb3fc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:52.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.986+0000 7fb403ae9700 1 -- 172.21.15.28:0/2714450637 shutdown_connections 2023-11-28T11:09:52.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.986+0000 7fb403ae9700 1 -- 172.21.15.28:0/2714450637 wait complete. 2023-11-28T11:09:52.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.987+0000 7fb403ae9700 1 Processor -- start 2023-11-28T11:09:52.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.987+0000 7fb403ae9700 1 -- start start 2023-11-28T11:09:52.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.988+0000 7fb403ae9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3fc0cc310 0x7fb3fc160570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.988+0000 7fb403ae9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3fc15c990 0x7fb3fc0ae370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.988+0000 7fb403ae9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3fc161090 0x7fb3fc0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.988+0000 7fb403ae9700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc0ad2b0 con 0x7fb3fc0cc310 2023-11-28T11:09:52.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.988+0000 7fb403ae9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc160ab0 con 0x7fb3fc15c990 2023-11-28T11:09:52.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.988+0000 7fb403ae9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3fc0c4760 con 0x7fb3fc161090 2023-11-28T11:09:52.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.988+0000 7fb401885700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3fc0cc310 0x7fb3fc160570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.988+0000 7fb401885700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3fc0cc310 0x7fb3fc160570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48840/0 (socket says 172.21.15.28:48840) 2023-11-28T11:09:52.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.988+0000 7fb401885700 1 -- 172.21.15.28:0/3381049089 learned_addr learned my addr 172.21.15.28:0/3381049089 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:52.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.988+0000 7fb401885700 1 -- 172.21.15.28:0/3381049089 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3fc161090 msgr2=0x7fb3fc0ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.988+0000 7fb401885700 1 --2- 172.21.15.28:0/3381049089 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3fc161090 0x7fb3fc0ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.988+0000 7fb401885700 1 -- 172.21.15.28:0/3381049089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3fc15c990 msgr2=0x7fb3fc0ae370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:52.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.988+0000 7fb401084700 1 --2- 172.21.15.28:0/3381049089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3fc15c990 0x7fb3fc0ae370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.989+0000 7fb401885700 1 --2- 172.21.15.28:0/3381049089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3fc15c990 0x7fb3fc0ae370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:52.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.989+0000 7fb401885700 1 -- 172.21.15.28:0/3381049089 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f80094a0 con 0x7fb3fc0cc310 2023-11-28T11:09:52.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.989+0000 7fb401084700 1 --2- 172.21.15.28:0/3381049089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3fc15c990 0x7fb3fc0ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:52.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.989+0000 7fb401885700 1 --2- 172.21.15.28:0/3381049089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3fc0cc310 0x7fb3fc160570 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80097c0 tx=0x7fb3f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.989+0000 7fb3f27fc700 1 -- 172.21.15.28:0/3381049089 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f801aa70 con 0x7fb3fc0cc310 2023-11-28T11:09:52.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.989+0000 7fb3f27fc700 1 -- 172.21.15.28:0/3381049089 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f801abd0 con 0x7fb3fc0cc310 2023-11-28T11:09:52.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.989+0000 7fb403ae9700 1 -- 172.21.15.28:0/3381049089 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3fc0cb080 con 0x7fb3fc0cc310 2023-11-28T11:09:52.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.989+0000 7fb3f27fc700 1 -- 172.21.15.28:0/3381049089 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f80243f0 con 0x7fb3fc0cc310 2023-11-28T11:09:52.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.989+0000 7fb403ae9700 1 -- 172.21.15.28:0/3381049089 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3fc0ce870 con 0x7fb3fc0cc310 2023-11-28T11:09:52.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.990+0000 7fb403ae9700 1 -- 172.21.15.28:0/3381049089 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3fc0cf5b0 con 0x7fb3fc0cc310 2023-11-28T11:09:52.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.990+0000 7fb3f27fc700 1 -- 172.21.15.28:0/3381049089 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb3f800f050 con 0x7fb3fc0cc310 2023-11-28T11:09:52.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.991+0000 7fb3f27fc700 1 --2- 172.21.15.28:0/3381049089 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb3e00682f0 0x7fb3e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:52.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.991+0000 7fb401084700 1 --2- 172.21.15.28:0/3381049089 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb3e00682f0 0x7fb3e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:52.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.991+0000 7fb3f27fc700 1 -- 172.21.15.28:0/3381049089 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fb3f8087360 con 0x7fb3fc0cc310 2023-11-28T11:09:52.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.991+0000 7fb401084700 1 --2- 172.21.15.28:0/3381049089 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb3e00682f0 0x7fb3e006a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0c5cb0 tx=0x7fb3ec009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:52.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:52.995+0000 7fb3f27fc700 1 -- 172.21.15.28:0/3381049089 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb3f8053890 con 0x7fb3fc0cc310 2023-11-28T11:09:53.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.204+0000 7fb403ae9700 1 -- 172.21.15.28:0/3381049089 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb3fc0675a0 con 0x7fb3fc0cc310 2023-11-28T11:09:53.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.205+0000 7fb3f27fc700 1 -- 172.21.15.28:0/3381049089 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v27) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb3f8015080 con 0x7fb3fc0cc310 2023-11-28T11:09:53.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.207+0000 7fb403ae9700 1 -- 172.21.15.28:0/3381049089 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb3e00682f0 msgr2=0x7fb3e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.207+0000 7fb403ae9700 1 --2- 172.21.15.28:0/3381049089 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb3e00682f0 0x7fb3e006a7a0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc0c5cb0 tx=0x7fb3ec009040 comp rx=0 tx=0).stop 2023-11-28T11:09:53.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.207+0000 7fb403ae9700 1 -- 172.21.15.28:0/3381049089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3fc0cc310 msgr2=0x7fb3fc160570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.207+0000 7fb403ae9700 1 --2- 172.21.15.28:0/3381049089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3fc0cc310 0x7fb3fc160570 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb3f80097c0 tx=0x7fb3f8004750 comp rx=0 tx=0).stop 2023-11-28T11:09:53.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.207+0000 7fb403ae9700 1 -- 172.21.15.28:0/3381049089 shutdown_connections 2023-11-28T11:09:53.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.207+0000 7fb403ae9700 1 --2- 172.21.15.28:0/3381049089 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb3e00682f0 0x7fb3e006a7a0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.207+0000 7fb403ae9700 1 --2- 172.21.15.28:0/3381049089 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb3fc0cc310 0x7fb3fc160570 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.207+0000 7fb403ae9700 1 --2- 172.21.15.28:0/3381049089 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb3fc161090 0x7fb3fc0ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.207+0000 7fb403ae9700 1 --2- 172.21.15.28:0/3381049089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb3fc15c990 0x7fb3fc0ae370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.207+0000 7fb403ae9700 1 -- 172.21.15.28:0/3381049089 >> 172.21.15.28:0/3381049089 conn(0x7fb3fc0b1650 msgr2=0x7fb3fc15bcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:53.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.207+0000 7fb403ae9700 1 -- 172.21.15.28:0/3381049089 shutdown_connections 2023-11-28T11:09:53.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.207+0000 7fb403ae9700 1 -- 172.21.15.28:0/3381049089 wait complete. 2023-11-28T11:09:53.211 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2023-11-28T11:09:53.220 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2023-11-28T11:09:53.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:53.221 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-11-28T11:09:53.267 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:09:53.267 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:09:53.267 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:09:53.267 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T11:09:53.289 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:09:53.290 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:53.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.414+0000 7f245951b700 1 Processor -- start 2023-11-28T11:09:53.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.414+0000 7f245951b700 1 -- start start 2023-11-28T11:09:53.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.415+0000 7f245951b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f245406bf30 0x7f245406c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.415+0000 7f245951b700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24540b6350 con 0x7f24540d2820 2023-11-28T11:09:53.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.415+0000 7f245951b700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24540d1470 con 0x7f245415c640 2023-11-28T11:09:53.420 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.415+0000 7f245951b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f245415ca10 con 0x7f245406bf30 2023-11-28T11:09:53.420 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.415+0000 7f24527fc700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f24540d2820 0x7f24540b5c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36906/0 (socket says 172.21.15.28:36906) 2023-11-28T11:09:53.420 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.415+0000 7f24527fc700 1 -- 172.21.15.28:0/4239080630 learned_addr learned my addr 172.21.15.28:0/4239080630 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:53.420 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.415+0000 7f2451ffb700 1 -- 172.21.15.28:0/4239080630 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3195162890 0 0) 0x7f24540b6350 con 0x7f24540d2820 2023-11-28T11:09:53.420 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.415+0000 7f2451ffb700 1 -- 172.21.15.28:0/4239080630 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f243c0035b0 con 0x7f24540d2820 2023-11-28T11:09:53.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.415+0000 7f24537fe700 1 --2- 172.21.15.28:0/4239080630 >> v2:172.21.15.59:3300/0 conn(0x7f245406bf30 0x7f245406c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.415+0000 7f2451ffb700 1 -- 172.21.15.28:0/4239080630 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3122994244 0 0) 0x7f243c0035b0 con 0x7f24540d2820 2023-11-28T11:09:53.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.415+0000 7f2451ffb700 1 -- 172.21.15.28:0/4239080630 >> v2:172.21.15.59:3300/0 conn(0x7f245406bf30 msgr2=0x7f245406c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:53.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.415+0000 7f2451ffb700 1 --2- 172.21.15.28:0/4239080630 >> v2:172.21.15.59:3300/0 conn(0x7f245406bf30 0x7f245406c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.415+0000 7f2451ffb700 1 -- 172.21.15.28:0/4239080630 >> v1:172.21.15.59:6789/0 conn(0x7f245415c640 legacy=0x7f24540cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.415+0000 7f2451ffb700 1 -- 172.21.15.28:0/4239080630 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f245406c840 con 0x7f24540d2820 2023-11-28T11:09:53.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.415+0000 7f2451ffb700 1 -- 172.21.15.28:0/4239080630 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f2444001670 con 0x7f24540d2820 2023-11-28T11:09:53.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f2451ffb700 1 -- 172.21.15.28:0/4239080630 >> v1:172.21.15.28:6789/0 conn(0x7f24540d2820 legacy=0x7f24540b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f2451ffb700 1 --2- 172.21.15.28:0/4239080630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c003a00 0x7f243c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f2451ffb700 1 --2- 172.21.15.28:0/4239080630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f243c0043f0 0x7f243c0048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f2451ffb700 1 --2- 172.21.15.28:0/4239080630 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f243c0050b0 0x7f243c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f2451ffb700 1 -- 172.21.15.28:0/4239080630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245415ca10 con 0x7f243c0043f0 2023-11-28T11:09:53.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f2451ffb700 1 -- 172.21.15.28:0/4239080630 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c00bf50 con 0x7f243c003a00 2023-11-28T11:09:53.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f2451ffb700 1 -- 172.21.15.28:0/4239080630 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c00c110 con 0x7f243c0050b0 2023-11-28T11:09:53.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f24527fc700 1 --2- 172.21.15.28:0/4239080630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f243c0043f0 0x7f243c0048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f2452ffd700 1 --2- 172.21.15.28:0/4239080630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c003a00 0x7f243c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f24537fe700 1 --2- 172.21.15.28:0/4239080630 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f243c0050b0 0x7f243c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f24527fc700 1 -- 172.21.15.28:0/4239080630 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f243c0050b0 msgr2=0x7f243c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:53.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f24527fc700 1 --2- 172.21.15.28:0/4239080630 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f243c0050b0 0x7f243c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f24527fc700 1 -- 172.21.15.28:0/4239080630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c003a00 msgr2=0x7f243c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:53.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f24527fc700 1 --2- 172.21.15.28:0/4239080630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c003a00 0x7f243c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f24537fe700 1 -- 172.21.15.28:0/4239080630 reap_dead start 2023-11-28T11:09:53.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f24527fc700 1 -- 172.21.15.28:0/4239080630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c0035b0 con 0x7f243c0043f0 2023-11-28T11:09:53.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f24527fc700 1 --2- 172.21.15.28:0/4239080630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f243c0043f0 0x7f243c0048b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f244400c2f0 tx=0x7f243c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf1beb9287a1a61 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:53.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f2451ffb700 1 -- 172.21.15.28:0/4239080630 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244400cdb0 con 0x7f243c0043f0 2023-11-28T11:09:53.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f2451ffb700 1 -- 172.21.15.28:0/4239080630 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2444005960 con 0x7f243c0043f0 2023-11-28T11:09:53.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.416+0000 7f2451ffb700 1 -- 172.21.15.28:0/4239080630 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2444005d10 con 0x7f243c0043f0 2023-11-28T11:09:53.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.417+0000 7f245951b700 1 -- 172.21.15.28:0/4239080630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f243c0043f0 msgr2=0x7f243c0048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.417+0000 7f245951b700 1 --2- 172.21.15.28:0/4239080630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f243c0043f0 0x7f243c0048b0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f244400c2f0 tx=0x7f243c005880 comp rx=0 tx=0).stop 2023-11-28T11:09:53.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.417+0000 7f245951b700 1 -- 172.21.15.28:0/4239080630 shutdown_connections 2023-11-28T11:09:53.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.417+0000 7f245951b700 1 --2- 172.21.15.28:0/4239080630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f243c0043f0 0x7f243c0048b0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.417+0000 7f245951b700 1 -- 172.21.15.28:0/4239080630 >> 172.21.15.28:0/4239080630 conn(0x7f24540d2df0 msgr2=0x7f24540d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:53.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.417+0000 7f245951b700 1 -- 172.21.15.28:0/4239080630 shutdown_connections 2023-11-28T11:09:53.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.417+0000 7f245951b700 1 -- 172.21.15.28:0/4239080630 wait complete. 2023-11-28T11:09:53.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.419+0000 7f245951b700 1 Processor -- start 2023-11-28T11:09:53.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.419+0000 7f245951b700 1 -- start start 2023-11-28T11:09:53.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f245951b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c0043f0 0x7f245414e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f245951b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f245415f420 0x7f24540c7330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f245951b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24540ca6d0 0x7f24540c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f245951b700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245415d620 con 0x7f24540ca6d0 2023-11-28T11:09:53.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f245951b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f245415ccf0 con 0x7f243c0043f0 2023-11-28T11:09:53.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f245951b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f245415cfd0 con 0x7f245415f420 2023-11-28T11:09:53.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f24537fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24540ca6d0 0x7f24540c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f24537fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24540ca6d0 0x7f24540c86c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48864/0 (socket says 172.21.15.28:48864) 2023-11-28T11:09:53.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f2452ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c0043f0 0x7f245414e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f24537fe700 1 -- 172.21.15.28:0/374973351 learned_addr learned my addr 172.21.15.28:0/374973351 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:53.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f24527fc700 1 --2- 172.21.15.28:0/374973351 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f245415f420 0x7f24540c7330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f24537fe700 1 -- 172.21.15.28:0/374973351 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f245415f420 msgr2=0x7f24540c7330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f24537fe700 1 --2- 172.21.15.28:0/374973351 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f245415f420 0x7f24540c7330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f24537fe700 1 -- 172.21.15.28:0/374973351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c0043f0 msgr2=0x7f245414e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f24537fe700 1 --2- 172.21.15.28:0/374973351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c0043f0 0x7f245414e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f24537fe700 1 -- 172.21.15.28:0/374973351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24540cd7c0 con 0x7f24540ca6d0 2023-11-28T11:09:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.420+0000 7f24537fe700 1 --2- 172.21.15.28:0/374973351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24540ca6d0 0x7f24540c86c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f245415ffd0 tx=0x7f244c011ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.421+0000 7f243b7fe700 1 -- 172.21.15.28:0/374973351 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c00e040 con 0x7f24540ca6d0 2023-11-28T11:09:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.421+0000 7f243b7fe700 1 -- 172.21.15.28:0/374973351 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f244c00f040 con 0x7f24540ca6d0 2023-11-28T11:09:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.421+0000 7f245951b700 1 -- 172.21.15.28:0/374973351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24540cd4e0 con 0x7f24540ca6d0 2023-11-28T11:09:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.421+0000 7f243b7fe700 1 -- 172.21.15.28:0/374973351 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f244c00fc40 con 0x7f24540ca6d0 2023-11-28T11:09:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.421+0000 7f245951b700 1 -- 172.21.15.28:0/374973351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24540d1710 con 0x7f24540ca6d0 2023-11-28T11:09:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.422+0000 7f245951b700 1 -- 172.21.15.28:0/374973351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2454068570 con 0x7f24540ca6d0 2023-11-28T11:09:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.422+0000 7f243b7fe700 1 -- 172.21.15.28:0/374973351 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f244c00fda0 con 0x7f24540ca6d0 2023-11-28T11:09:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.422+0000 7f243b7fe700 1 --2- 172.21.15.28:0/374973351 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24400682f0 0x7f244006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.423+0000 7f243b7fe700 1 -- 172.21.15.28:0/374973351 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f244c086640 con 0x7f24540ca6d0 2023-11-28T11:09:53.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.423+0000 7f2452ffd700 1 --2- 172.21.15.28:0/374973351 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24400682f0 0x7f244006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.423+0000 7f2452ffd700 1 --2- 172.21.15.28:0/374973351 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24400682f0 0x7f244006a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2454069660 tx=0x7f2448009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:53.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.427+0000 7f243b7fe700 1 -- 172.21.15.28:0/374973351 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f244c052b70 con 0x7f24540ca6d0 2023-11-28T11:09:53.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.667+0000 7f245951b700 1 -- 172.21.15.28:0/374973351 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24540d10e0 con 0x7f24540ca6d0 2023-11-28T11:09:53.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.667+0000 7f243b7fe700 1 -- 172.21.15.28:0/374973351 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4644 (secure 0 0 0) 0x7f244c056190 con 0x7f24540ca6d0 2023-11-28T11:09:53.669 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:53.669 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/3112877467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3112877467},{"type":"v1","addr":"172.21.15.59:6834","nonce":3112877467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3448236489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3448236489},{"type":"v1","addr":"172.21.15.28:6836","nonce":3448236489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4232269940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4232269940},{"type":"v1","addr":"172.21.15.28:6837","nonce":4232269940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:45.924816+0000","modified":"2023-11-28T11:09:03.234785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/3293053261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3293053261},{"type":"v1","addr":"172.21.15.59:6835","nonce":3293053261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:09:53.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.669+0000 7f245951b700 1 -- 172.21.15.28:0/374973351 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24400682f0 msgr2=0x7f244006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.669+0000 7f245951b700 1 --2- 172.21.15.28:0/374973351 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24400682f0 0x7f244006a7a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f2454069660 tx=0x7f2448009040 comp rx=0 tx=0).stop 2023-11-28T11:09:53.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.669+0000 7f245951b700 1 -- 172.21.15.28:0/374973351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24540ca6d0 msgr2=0x7f24540c86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.669+0000 7f245951b700 1 --2- 172.21.15.28:0/374973351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24540ca6d0 0x7f24540c86c0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f245415ffd0 tx=0x7f244c011ed0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.669+0000 7f245951b700 1 -- 172.21.15.28:0/374973351 shutdown_connections 2023-11-28T11:09:53.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.669+0000 7f245951b700 1 --2- 172.21.15.28:0/374973351 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24400682f0 0x7f244006a7a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.669+0000 7f245951b700 1 --2- 172.21.15.28:0/374973351 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24540ca6d0 0x7f24540c86c0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.669+0000 7f245951b700 1 --2- 172.21.15.28:0/374973351 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c0043f0 0x7f245414e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.669+0000 7f245951b700 1 --2- 172.21.15.28:0/374973351 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f245415f420 0x7f24540c7330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.669+0000 7f245951b700 1 -- 172.21.15.28:0/374973351 >> 172.21.15.28:0/374973351 conn(0x7f24540d2df0 msgr2=0x7f245406a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:53.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.669+0000 7f245951b700 1 -- 172.21.15.28:0/374973351 shutdown_connections 2023-11-28T11:09:53.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.669+0000 7f245951b700 1 -- 172.21.15.28:0/374973351 wait complete. 2023-11-28T11:09:53.674 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 9 2023-11-28T11:09:53.683 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:53.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.807+0000 7fbe9a319700 1 Processor -- start 2023-11-28T11:09:53.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.807+0000 7fbe9a319700 1 -- start start 2023-11-28T11:09:53.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe9a319700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbe94159590 0x7fbe9406e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe9a319700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbe9415db70 0x7fbe940ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe9a319700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe9415d120 con 0x7fbe940ce890 2023-11-28T11:09:53.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe9a319700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9415d6e0 con 0x7fbe9415db70 2023-11-28T11:09:53.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe9a319700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe940aef60 con 0x7fbe94159590 2023-11-28T11:09:53.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe937fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbe9415db70 0x7fbe940ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe988b6700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbe940ce890 0x7fbe940caf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36930/0 (socket says 172.21.15.28:36930) 2023-11-28T11:09:53.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe988b6700 1 -- 172.21.15.28:0/2428976107 learned_addr learned my addr 172.21.15.28:0/2428976107 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:53.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe93fff700 1 --2- 172.21.15.28:0/2428976107 >> v2:172.21.15.59:3300/0 conn(0x7fbe94159590 0x7fbe9406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe937fe700 1 -- 172.21.15.28:0/2428976107 >> v2:172.21.15.59:3300/0 conn(0x7fbe94159590 msgr2=0x7fbe9406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:53.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe937fe700 1 --2- 172.21.15.28:0/2428976107 >> v2:172.21.15.59:3300/0 conn(0x7fbe94159590 0x7fbe9406e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe937fe700 1 -- 172.21.15.28:0/2428976107 >> v1:172.21.15.28:6789/0 conn(0x7fbe940ce890 legacy=0x7fbe940caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe937fe700 1 -- 172.21.15.28:0/2428976107 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe940af240 con 0x7fbe9415db70 2023-11-28T11:09:53.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe92ffd700 1 -- 172.21.15.28:0/2428976107 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 966925047 0 0) 0x7fbe9415d120 con 0x7fbe940ce890 2023-11-28T11:09:53.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe937fe700 1 --2- 172.21.15.28:0/2428976107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe9415db70 0x7fbe940ae730 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbe9406bd10 tx=0x7fbe88009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1e4c311fc1d9fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:53.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe92ffd700 1 -- 172.21.15.28:0/2428976107 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88011830 con 0x7fbe9415db70 2023-11-28T11:09:53.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe92ffd700 1 -- 172.21.15.28:0/2428976107 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe88011990 con 0x7fbe9415db70 2023-11-28T11:09:53.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.808+0000 7fbe92ffd700 1 -- 172.21.15.28:0/2428976107 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe88011d20 con 0x7fbe9415db70 2023-11-28T11:09:53.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.809+0000 7fbe9a319700 1 -- 172.21.15.28:0/2428976107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe9415db70 msgr2=0x7fbe940ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.809+0000 7fbe9a319700 1 --2- 172.21.15.28:0/2428976107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe9415db70 0x7fbe940ae730 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbe9406bd10 tx=0x7fbe88009b20 comp rx=0 tx=0).stop 2023-11-28T11:09:53.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.809+0000 7fbe9a319700 1 -- 172.21.15.28:0/2428976107 shutdown_connections 2023-11-28T11:09:53.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.809+0000 7fbe9a319700 1 --2- 172.21.15.28:0/2428976107 >> v2:172.21.15.59:3300/0 conn(0x7fbe94159590 0x7fbe9406e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.809+0000 7fbe9a319700 1 --2- 172.21.15.28:0/2428976107 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe9415db70 0x7fbe940ae730 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.809+0000 7fbe9a319700 1 -- 172.21.15.28:0/2428976107 >> 172.21.15.28:0/2428976107 conn(0x7fbe940b1650 msgr2=0x7fbe940b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:53.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.809+0000 7fbe9a319700 1 -- 172.21.15.28:0/2428976107 shutdown_connections 2023-11-28T11:09:53.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.809+0000 7fbe9a319700 1 -- 172.21.15.28:0/2428976107 wait complete. 2023-11-28T11:09:53.817 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.810+0000 7fbe9a319700 1 Processor -- start 2023-11-28T11:09:53.817 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.810+0000 7fbe9a319700 1 -- start start 2023-11-28T11:09:53.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.811+0000 7fbe9a319700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe940ce890 0x7fbe9414a200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.811+0000 7fbe9a319700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe94159590 0x7fbe9415c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.811+0000 7fbe9a319700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe9415db70 0x7fbe940b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.811+0000 7fbe9a319700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe940b2f80 con 0x7fbe940ce890 2023-11-28T11:09:53.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.811+0000 7fbe9a319700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe940b2950 con 0x7fbe94159590 2023-11-28T11:09:53.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.811+0000 7fbe9a319700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9415b990 con 0x7fbe9415db70 2023-11-28T11:09:53.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.811+0000 7fbe93fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe940ce890 0x7fbe9414a200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.811+0000 7fbe93fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe940ce890 0x7fbe9414a200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48874/0 (socket says 172.21.15.28:48874) 2023-11-28T11:09:53.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.811+0000 7fbe93fff700 1 -- 172.21.15.28:0/1335979567 learned_addr learned my addr 172.21.15.28:0/1335979567 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:53.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.812+0000 7fbe93fff700 1 -- 172.21.15.28:0/1335979567 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe9415db70 msgr2=0x7fbe940b0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.812+0000 7fbe988b6700 1 --2- 172.21.15.28:0/1335979567 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe9415db70 0x7fbe940b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.812+0000 7fbe937fe700 1 --2- 172.21.15.28:0/1335979567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe94159590 0x7fbe9415c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.812+0000 7fbe93fff700 1 --2- 172.21.15.28:0/1335979567 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe9415db70 0x7fbe940b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.812+0000 7fbe93fff700 1 -- 172.21.15.28:0/1335979567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe94159590 msgr2=0x7fbe9415c2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:53.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.812+0000 7fbe93fff700 1 --2- 172.21.15.28:0/1335979567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe94159590 0x7fbe9415c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:53.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.812+0000 7fbe93fff700 1 -- 172.21.15.28:0/1335979567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe880094a0 con 0x7fbe940ce890 2023-11-28T11:09:53.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.812+0000 7fbe93fff700 1 --2- 172.21.15.28:0/1335979567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe940ce890 0x7fbe9414a200 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400e810 tx=0x7fbe8400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:53.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.812+0000 7fbe90ff9700 1 -- 172.21.15.28:0/1335979567 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84013610 con 0x7fbe940ce890 2023-11-28T11:09:53.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.812+0000 7fbe90ff9700 1 -- 172.21.15.28:0/1335979567 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe84013770 con 0x7fbe940ce890 2023-11-28T11:09:53.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.812+0000 7fbe9a319700 1 -- 172.21.15.28:0/1335979567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9415e010 con 0x7fbe940ce890 2023-11-28T11:09:53.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.812+0000 7fbe90ff9700 1 -- 172.21.15.28:0/1335979567 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe84013b20 con 0x7fbe940ce890 2023-11-28T11:09:53.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.812+0000 7fbe9a319700 1 -- 172.21.15.28:0/1335979567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9415d370 con 0x7fbe940ce890 2023-11-28T11:09:53.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.813+0000 7fbe9a319700 1 -- 172.21.15.28:0/1335979567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe940aec70 con 0x7fbe940ce890 2023-11-28T11:09:53.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.813+0000 7fbe90ff9700 1 -- 172.21.15.28:0/1335979567 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbe84023070 con 0x7fbe940ce890 2023-11-28T11:09:53.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.814+0000 7fbe90ff9700 1 --2- 172.21.15.28:0/1335979567 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbe800682f0 0x7fbe8006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:53.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.814+0000 7fbe90ff9700 1 -- 172.21.15.28:0/1335979567 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbe84085dc0 con 0x7fbe940ce890 2023-11-28T11:09:53.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.814+0000 7fbe937fe700 1 --2- 172.21.15.28:0/1335979567 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbe800682f0 0x7fbe8006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:53.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.814+0000 7fbe937fe700 1 --2- 172.21.15.28:0/1335979567 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbe800682f0 0x7fbe8006a7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbe940b5a70 tx=0x7fbe88008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:53.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:53.818+0000 7fbe90ff9700 1 -- 172.21.15.28:0/1335979567 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbe84052180 con 0x7fbe940ce890 2023-11-28T11:09:54.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.013+0000 7fbe9a319700 1 -- 172.21.15.28:0/1335979567 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbe940675a0 con 0x7fbe940ce890 2023-11-28T11:09:54.016 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.014+0000 7fbe90ff9700 1 -- 172.21.15.28:0/1335979567 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19737 (secure 0 0 0) 0x7fbe840557a0 con 0x7fbe940ce890 2023-11-28T11:09:54.017 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:54.017 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:09:34.614188+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:54.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.016+0000 7fbe9a319700 1 -- 172.21.15.28:0/1335979567 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbe800682f0 msgr2=0x7fbe8006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.016+0000 7fbe9a319700 1 --2- 172.21.15.28:0/1335979567 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbe800682f0 0x7fbe8006a7a0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbe940b5a70 tx=0x7fbe88008030 comp rx=0 tx=0).stop 2023-11-28T11:09:54.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.016+0000 7fbe9a319700 1 -- 172.21.15.28:0/1335979567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe940ce890 msgr2=0x7fbe9414a200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.016+0000 7fbe9a319700 1 --2- 172.21.15.28:0/1335979567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe940ce890 0x7fbe9414a200 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbe8400e810 tx=0x7fbe8400ebd0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.016+0000 7fbe9a319700 1 -- 172.21.15.28:0/1335979567 shutdown_connections 2023-11-28T11:09:54.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.016+0000 7fbe9a319700 1 --2- 172.21.15.28:0/1335979567 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbe800682f0 0x7fbe8006a7a0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.021 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.016+0000 7fbe9a319700 1 --2- 172.21.15.28:0/1335979567 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe940ce890 0x7fbe9414a200 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.021 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.016+0000 7fbe9a319700 1 --2- 172.21.15.28:0/1335979567 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe9415db70 0x7fbe940b0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.021 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.016+0000 7fbe9a319700 1 --2- 172.21.15.28:0/1335979567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe94159590 0x7fbe9415c2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.021 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.016+0000 7fbe9a319700 1 -- 172.21.15.28:0/1335979567 >> 172.21.15.28:0/1335979567 conn(0x7fbe940b1650 msgr2=0x7fbe940cd900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:54.021 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.016+0000 7fbe9a319700 1 -- 172.21.15.28:0/1335979567 shutdown_connections 2023-11-28T11:09:54.021 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.016+0000 7fbe9a319700 1 -- 172.21.15.28:0/1335979567 wait complete. 2023-11-28T11:09:54.035 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:08:43.661652+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:08:44.889002+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-11-28T11:08:48.125959+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T11:09:54.035 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:09:54.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.164+0000 7ff7975b3700 1 Processor -- start 2023-11-28T11:09:54.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.164+0000 7ff7975b3700 1 -- start start 2023-11-28T11:09:54.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff7975b3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff7900b1630 0x7ff7900cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff7975b3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7900b1a00 0x7ff7900caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff7975b3700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff7900cb3e0 0x7ff790160350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff7975b3700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7900ce550 con 0x7ff7900cb3e0 2023-11-28T11:09:54.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff7975b3700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7900ce830 con 0x7ff7900b1630 2023-11-28T11:09:54.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff7975b3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900ceb10 con 0x7ff7900b1a00 2023-11-28T11:09:54.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff795b50700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff7900cb3e0 0x7ff790160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff795b50700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff7900cb3e0 0x7ff790160350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48886/0 (socket says 172.21.15.28:48886) 2023-11-28T11:09:54.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff795b50700 1 -- 172.21.15.28:0/2138333780 learned_addr learned my addr 172.21.15.28:0/2138333780 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:54.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff794b4e700 1 --2- 172.21.15.28:0/2138333780 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7900b1a00 0x7ff7900caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff79534f700 1 --2- 172.21.15.28:0/2138333780 >> v2:172.21.15.59:3300/0 conn(0x7ff7900b1630 0x7ff7900cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff795b50700 1 -- 172.21.15.28:0/2138333780 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7900b1a00 msgr2=0x7ff7900caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:54.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff795b50700 1 --2- 172.21.15.28:0/2138333780 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7900b1a00 0x7ff7900caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff795b50700 1 -- 172.21.15.28:0/2138333780 >> v2:172.21.15.59:3300/0 conn(0x7ff7900b1630 msgr2=0x7ff7900cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:54.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff795b50700 1 --2- 172.21.15.28:0/2138333780 >> v2:172.21.15.59:3300/0 conn(0x7ff7900b1630 0x7ff7900cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.165+0000 7ff795b50700 1 -- 172.21.15.28:0/2138333780 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7900cedf0 con 0x7ff7900cb3e0 2023-11-28T11:09:54.172 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.166+0000 7ff795b50700 1 --2- 172.21.15.28:0/2138333780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7900cb3e0 0x7ff790160350 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff7900d0bc0 tx=0x7ff78800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ab09eb1f86d507 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.172 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.166+0000 7ff787fff700 1 -- 172.21.15.28:0/2138333780 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800e070 con 0x7ff7900cb3e0 2023-11-28T11:09:54.172 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.166+0000 7ff787fff700 1 -- 172.21.15.28:0/2138333780 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78800b9c0 con 0x7ff7900cb3e0 2023-11-28T11:09:54.172 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.166+0000 7ff787fff700 1 -- 172.21.15.28:0/2138333780 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800bd70 con 0x7ff7900cb3e0 2023-11-28T11:09:54.172 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.166+0000 7ff7975b3700 1 -- 172.21.15.28:0/2138333780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7900cb3e0 msgr2=0x7ff790160350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.173 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.166+0000 7ff7975b3700 1 --2- 172.21.15.28:0/2138333780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7900cb3e0 0x7ff790160350 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7ff7900d0bc0 tx=0x7ff78800b330 comp rx=0 tx=0).stop 2023-11-28T11:09:54.173 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.166+0000 7ff7975b3700 1 -- 172.21.15.28:0/2138333780 shutdown_connections 2023-11-28T11:09:54.173 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.166+0000 7ff7975b3700 1 --2- 172.21.15.28:0/2138333780 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7900cb3e0 0x7ff790160350 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.173 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.166+0000 7ff7975b3700 1 --2- 172.21.15.28:0/2138333780 >> v2:172.21.15.59:3300/0 conn(0x7ff7900b1630 0x7ff7900cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.173 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.166+0000 7ff7975b3700 1 --2- 172.21.15.28:0/2138333780 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7900b1a00 0x7ff7900caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.173 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.166+0000 7ff7975b3700 1 -- 172.21.15.28:0/2138333780 >> 172.21.15.28:0/2138333780 conn(0x7ff7900cc630 msgr2=0x7ff7900cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:54.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.167+0000 7ff7975b3700 1 -- 172.21.15.28:0/2138333780 shutdown_connections 2023-11-28T11:09:54.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.167+0000 7ff7975b3700 1 -- 172.21.15.28:0/2138333780 wait complete. 2023-11-28T11:09:54.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.168+0000 7ff7975b3700 1 Processor -- start 2023-11-28T11:09:54.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.168+0000 7ff7975b3700 1 -- start start 2023-11-28T11:09:54.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff7975b3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7900b1630 0x7ff79015ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff7975b3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7900b1a00 0x7ff7900c8960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff7975b3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7900cb3e0 0x7ff79015a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff7975b3700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900c4dc0 con 0x7ff7900b1a00 2023-11-28T11:09:54.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff7975b3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900c50a0 con 0x7ff7900cb3e0 2023-11-28T11:09:54.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff7975b3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7900c3810 con 0x7ff7900b1630 2023-11-28T11:09:54.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff794b4e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7900b1a00 0x7ff7900c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff794b4e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7900b1a00 0x7ff7900c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48890/0 (socket says 172.21.15.28:48890) 2023-11-28T11:09:54.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff794b4e700 1 -- 172.21.15.28:0/470188943 learned_addr learned my addr 172.21.15.28:0/470188943 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:54.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff79534f700 1 --2- 172.21.15.28:0/470188943 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7900b1630 0x7ff79015ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff795b50700 1 --2- 172.21.15.28:0/470188943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7900cb3e0 0x7ff79015a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff794b4e700 1 -- 172.21.15.28:0/470188943 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7900b1630 msgr2=0x7ff79015ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff794b4e700 1 --2- 172.21.15.28:0/470188943 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7900b1630 0x7ff79015ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff794b4e700 1 -- 172.21.15.28:0/470188943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7900cb3e0 msgr2=0x7ff79015a1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff794b4e700 1 --2- 172.21.15.28:0/470188943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7900cb3e0 0x7ff79015a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff794b4e700 1 -- 172.21.15.28:0/470188943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff788009b30 con 0x7ff7900b1a00 2023-11-28T11:09:54.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff794b4e700 1 --2- 172.21.15.28:0/470188943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7900b1a00 0x7ff7900c8960 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff78c00e810 tx=0x7ff78c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff785ffb700 1 -- 172.21.15.28:0/470188943 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c013610 con 0x7ff7900b1a00 2023-11-28T11:09:54.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff785ffb700 1 -- 172.21.15.28:0/470188943 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c013770 con 0x7ff7900b1a00 2023-11-28T11:09:54.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff7975b3700 1 -- 172.21.15.28:0/470188943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7900b1e70 con 0x7ff7900b1a00 2023-11-28T11:09:54.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.169+0000 7ff785ffb700 1 -- 172.21.15.28:0/470188943 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c013b20 con 0x7ff7900b1a00 2023-11-28T11:09:54.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.170+0000 7ff7975b3700 1 -- 172.21.15.28:0/470188943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7900cf0d0 con 0x7ff7900b1a00 2023-11-28T11:09:54.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.170+0000 7ff7975b3700 1 -- 172.21.15.28:0/470188943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7900ce140 con 0x7ff7900b1a00 2023-11-28T11:09:54.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.171+0000 7ff785ffb700 1 -- 172.21.15.28:0/470188943 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff78c023070 con 0x7ff7900b1a00 2023-11-28T11:09:54.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.171+0000 7ff785ffb700 1 --2- 172.21.15.28:0/470188943 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7740682f0 0x7ff77406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.172+0000 7ff785ffb700 1 -- 172.21.15.28:0/470188943 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7ff78c085cf0 con 0x7ff7900b1a00 2023-11-28T11:09:54.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.172+0000 7ff79534f700 1 --2- 172.21.15.28:0/470188943 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7740682f0 0x7ff77406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.172+0000 7ff79534f700 1 --2- 172.21.15.28:0/470188943 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7740682f0 0x7ff77406a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff7900ade90 tx=0x7ff780008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.176+0000 7ff785ffb700 1 -- 172.21.15.28:0/470188943 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff78c0521a0 con 0x7ff7900b1a00 2023-11-28T11:09:54.370 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.368+0000 7ff7975b3700 1 -- 172.21.15.28:0/470188943 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff7900b5b10 con 0x7ff7900b1a00 2023-11-28T11:09:54.370 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.368+0000 7ff785ffb700 1 -- 172.21.15.28:0/470188943 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v27) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff78c0557c0 con 0x7ff7900b1a00 2023-11-28T11:09:54.370 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:09:54.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.370+0000 7ff7975b3700 1 -- 172.21.15.28:0/470188943 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7740682f0 msgr2=0x7ff77406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.370+0000 7ff7975b3700 1 --2- 172.21.15.28:0/470188943 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7740682f0 0x7ff77406a7a0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7ff7900ade90 tx=0x7ff780008040 comp rx=0 tx=0).stop 2023-11-28T11:09:54.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.370+0000 7ff7975b3700 1 -- 172.21.15.28:0/470188943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7900b1a00 msgr2=0x7ff7900c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.370+0000 7ff7975b3700 1 --2- 172.21.15.28:0/470188943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7900b1a00 0x7ff7900c8960 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff78c00e810 tx=0x7ff78c00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.371+0000 7ff7975b3700 1 -- 172.21.15.28:0/470188943 shutdown_connections 2023-11-28T11:09:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.371+0000 7ff7975b3700 1 --2- 172.21.15.28:0/470188943 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7740682f0 0x7ff77406a7a0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.371+0000 7ff7975b3700 1 --2- 172.21.15.28:0/470188943 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7900cb3e0 0x7ff79015a1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.371+0000 7ff7975b3700 1 --2- 172.21.15.28:0/470188943 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7900b1630 0x7ff79015ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.371+0000 7ff7975b3700 1 --2- 172.21.15.28:0/470188943 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7900b1a00 0x7ff7900c8960 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.371+0000 7ff7975b3700 1 -- 172.21.15.28:0/470188943 >> 172.21.15.28:0/470188943 conn(0x7ff7900cc630 msgr2=0x7ff7900cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.371+0000 7ff7975b3700 1 -- 172.21.15.28:0/470188943 shutdown_connections 2023-11-28T11:09:54.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.371+0000 7ff7975b3700 1 -- 172.21.15.28:0/470188943 wait complete. 2023-11-28T11:09:54.385 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:09:54.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.511+0000 7f70681ea700 1 Processor -- start 2023-11-28T11:09:54.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.511+0000 7f70681ea700 1 -- start start 2023-11-28T11:09:54.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.512+0000 7f70681ea700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f70600b62d0 0x7f70600cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.512+0000 7f70681ea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f706006f8b0 0x7f70600b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.512+0000 7f70681ea700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70600b3020 con 0x7f70600b2bc0 2023-11-28T11:09:54.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.512+0000 7f70681ea700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70600cd2a0 con 0x7f70600b62d0 2023-11-28T11:09:54.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.512+0000 7f70681ea700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70600d0400 con 0x7f706006f8b0 2023-11-28T11:09:54.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.512+0000 7f7065785700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f706006f8b0 0x7f70600b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.512+0000 7f7065f86700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f70600b62d0 0x7f70600cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.512+0000 7f7066787700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f70600b2bc0 0x7f70600ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:47744/0 (socket says 172.21.15.28:47744) 2023-11-28T11:09:54.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.512+0000 7f7066787700 1 -- 172.21.15.28:0/2556139122 learned_addr learned my addr 172.21.15.28:0/2556139122 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:54.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.512+0000 7f7065785700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f706006f8b0 0x7f70600b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:54654/0 (socket says 172.21.15.28:54654) 2023-11-28T11:09:54.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.512+0000 7f7065f86700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f70600b62d0 0x7f70600cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:36186/0 (socket says 172.21.15.28:36186) 2023-11-28T11:09:54.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.513+0000 7f7064f84700 1 -- 172.21.15.28:0/2556139122 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 970143250 0 0) 0x7f70600b3020 con 0x7f70600b2bc0 2023-11-28T11:09:54.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.513+0000 7f7064f84700 1 -- 172.21.15.28:0/2556139122 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f704c0035b0 con 0x7f70600b2bc0 2023-11-28T11:09:54.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.513+0000 7f7065785700 1 -- 172.21.15.28:0/2556139122 >> v2:172.21.15.59:3300/0 conn(0x7f70600b62d0 msgr2=0x7f70600cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:54.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.513+0000 7f7065785700 1 --2- 172.21.15.28:0/2556139122 >> v2:172.21.15.59:3300/0 conn(0x7f70600b62d0 0x7f70600cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.513+0000 7f7065785700 1 -- 172.21.15.28:0/2556139122 >> v1:172.21.15.59:6789/0 conn(0x7f70600b2bc0 legacy=0x7f70600ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.513+0000 7f7065785700 1 -- 172.21.15.28:0/2556139122 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70600d06e0 con 0x7f706006f8b0 2023-11-28T11:09:54.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.513+0000 7f7065785700 1 --2- 172.21.15.28:0/2556139122 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f706006f8b0 0x7f70600b2680 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7050009840 tx=0x7f7050009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8561e435a615ac53 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.513+0000 7f7064f84700 1 -- 172.21.15.28:0/2556139122 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050011830 con 0x7f706006f8b0 2023-11-28T11:09:54.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.513+0000 7f7064f84700 1 -- 172.21.15.28:0/2556139122 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7050011990 con 0x7f706006f8b0 2023-11-28T11:09:54.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.513+0000 7f7064f84700 1 -- 172.21.15.28:0/2556139122 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7050011d20 con 0x7f706006f8b0 2023-11-28T11:09:54.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.514+0000 7f70681ea700 1 -- 172.21.15.28:0/2556139122 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f706006f8b0 msgr2=0x7f70600b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.514+0000 7f70681ea700 1 --2- 172.21.15.28:0/2556139122 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f706006f8b0 0x7f70600b2680 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f7050009840 tx=0x7f7050009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:54.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.514+0000 7f70681ea700 1 -- 172.21.15.28:0/2556139122 shutdown_connections 2023-11-28T11:09:54.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.514+0000 7f70681ea700 1 --2- 172.21.15.28:0/2556139122 >> v2:172.21.15.59:3300/0 conn(0x7f70600b62d0 0x7f70600cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.514+0000 7f70681ea700 1 --2- 172.21.15.28:0/2556139122 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f706006f8b0 0x7f70600b2680 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.514+0000 7f70681ea700 1 -- 172.21.15.28:0/2556139122 >> 172.21.15.28:0/2556139122 conn(0x7f70600cd680 msgr2=0x7f70600cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:54.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.514+0000 7f70681ea700 1 -- 172.21.15.28:0/2556139122 shutdown_connections 2023-11-28T11:09:54.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.514+0000 7f70681ea700 1 -- 172.21.15.28:0/2556139122 wait complete. 2023-11-28T11:09:54.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.515+0000 7f70681ea700 1 Processor -- start 2023-11-28T11:09:54.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.515+0000 7f70681ea700 1 -- start start 2023-11-28T11:09:54.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.516+0000 7f70681ea700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f706006f8b0 0x7f70601f1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.516+0000 7f70681ea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70600b2bc0 0x7f70601e3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.516+0000 7f70681ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70600b62d0 0x7f706006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.516+0000 7f70681ea700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706006bb80 con 0x7f706006f8b0 2023-11-28T11:09:54.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.516+0000 7f70681ea700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70601f2160 con 0x7f70600b62d0 2023-11-28T11:09:54.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.516+0000 7f70681ea700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70601f2440 con 0x7f70600b2bc0 2023-11-28T11:09:54.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.516+0000 7f7065f86700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f706006f8b0 0x7f70601f1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.516+0000 7f7065f86700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f706006f8b0 0x7f70601f1b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48902/0 (socket says 172.21.15.28:48902) 2023-11-28T11:09:54.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.516+0000 7f7065f86700 1 -- 172.21.15.28:0/688284441 learned_addr learned my addr 172.21.15.28:0/688284441 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:54.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.516+0000 7f7065785700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70600b2bc0 0x7f70601e3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.516+0000 7f7066787700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70600b62d0 0x7f706006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.517+0000 7f7065f86700 1 -- 172.21.15.28:0/688284441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70600b2bc0 msgr2=0x7f70601e3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.517+0000 7f7065f86700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70600b2bc0 0x7f70601e3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.517+0000 7f7065f86700 1 -- 172.21.15.28:0/688284441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70600b62d0 msgr2=0x7f706006d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.517+0000 7f7065f86700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70600b62d0 0x7f706006d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.517+0000 7f7065f86700 1 -- 172.21.15.28:0/688284441 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70500094a0 con 0x7f706006f8b0 2023-11-28T11:09:54.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.517+0000 7f7065785700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70600b2bc0 0x7f70601e3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:54.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.517+0000 7f7066787700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70600b62d0 0x7f706006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:54.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.517+0000 7f7065f86700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f706006f8b0 0x7f70601f1b80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f705c00e870 tx=0x7f705c00ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.517+0000 7f70567fc700 1 -- 172.21.15.28:0/688284441 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c009590 con 0x7f706006f8b0 2023-11-28T11:09:54.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.517+0000 7f70567fc700 1 -- 172.21.15.28:0/688284441 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f705c00ee70 con 0x7f706006f8b0 2023-11-28T11:09:54.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.517+0000 7f70681ea700 1 -- 172.21.15.28:0/688284441 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70600b66a0 con 0x7f706006f8b0 2023-11-28T11:09:54.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.517+0000 7f70567fc700 1 -- 172.21.15.28:0/688284441 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f705c00ca50 con 0x7f706006f8b0 2023-11-28T11:09:54.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.517+0000 7f70681ea700 1 -- 172.21.15.28:0/688284441 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70600cffb0 con 0x7f706006f8b0 2023-11-28T11:09:54.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.518+0000 7f70681ea700 1 -- 172.21.15.28:0/688284441 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706006fd00 con 0x7f706006f8b0 2023-11-28T11:09:54.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.518+0000 7f70567fc700 1 -- 172.21.15.28:0/688284441 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f705c019030 con 0x7f706006f8b0 2023-11-28T11:09:54.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.519+0000 7f70567fc700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70440682f0 0x7f704406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.519+0000 7f70567fc700 1 -- 172.21.15.28:0/688284441 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f705c085ed0 con 0x7f706006f8b0 2023-11-28T11:09:54.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.519+0000 7f7065785700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70440682f0 0x7f704406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.519+0000 7f7065785700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70440682f0 0x7f704406a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7050000c00 tx=0x7f7050008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.523+0000 7f70567fc700 1 -- 172.21.15.28:0/688284441 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f705c052380 con 0x7f706006f8b0 2023-11-28T11:09:54.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.714+0000 7f70681ea700 1 -- 172.21.15.28:0/688284441 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f706006beb0 con 0x7f706006f8b0 2023-11-28T11:09:54.716 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.714+0000 7f70567fc700 1 -- 172.21.15.28:0/688284441 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v27) v1 ==== 93+0+10 (secure 0 0 0) 0x7f705c0559a0 con 0x7f706006f8b0 2023-11-28T11:09:54.716 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:09:54.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.717+0000 7f70681ea700 1 -- 172.21.15.28:0/688284441 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70440682f0 msgr2=0x7f704406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.717+0000 7f70681ea700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70440682f0 0x7f704406a7a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f7050000c00 tx=0x7f7050008030 comp rx=0 tx=0).stop 2023-11-28T11:09:54.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.717+0000 7f70681ea700 1 -- 172.21.15.28:0/688284441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f706006f8b0 msgr2=0x7f70601f1b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.717+0000 7f70681ea700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f706006f8b0 0x7f70601f1b80 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f705c00e870 tx=0x7f705c00ec30 comp rx=0 tx=0).stop 2023-11-28T11:09:54.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.717+0000 7f70681ea700 1 -- 172.21.15.28:0/688284441 shutdown_connections 2023-11-28T11:09:54.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.717+0000 7f70681ea700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70440682f0 0x7f704406a7a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.717+0000 7f70681ea700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f706006f8b0 0x7f70601f1b80 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.717+0000 7f70681ea700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70600b62d0 0x7f706006d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.717+0000 7f70681ea700 1 --2- 172.21.15.28:0/688284441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70600b2bc0 0x7f70601e3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.717+0000 7f70681ea700 1 -- 172.21.15.28:0/688284441 >> 172.21.15.28:0/688284441 conn(0x7f70600cd680 msgr2=0x7f70600b3a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:54.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.717+0000 7f70681ea700 1 -- 172.21.15.28:0/688284441 shutdown_connections 2023-11-28T11:09:54.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.717+0000 7f70681ea700 1 -- 172.21.15.28:0/688284441 wait complete. 2023-11-28T11:09:54.730 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:09:54.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.856+0000 7f96028ba700 1 Processor -- start 2023-11-28T11:09:54.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.856+0000 7f96028ba700 1 -- start start 2023-11-28T11:09:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.857+0000 7f96028ba700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f95fc159500 0x7f95fc0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.857+0000 7f96028ba700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f95fc15dbf0 0x7f95fc0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.857+0000 7f96028ba700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95fc15e040 con 0x7f95fc0b1630 2023-11-28T11:09:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.857+0000 7f96028ba700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95fc0cede0 con 0x7f95fc15dbf0 2023-11-28T11:09:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.857+0000 7f96028ba700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95fc0ceff0 con 0x7f95fc159500 2023-11-28T11:09:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.857+0000 7f95fb7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f95fc15dbf0 0x7f95fc0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.857+0000 7f95fb7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f95fc15dbf0 0x7f95fc0ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48912/0 (socket says 172.21.15.28:48912) 2023-11-28T11:09:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.857+0000 7f95fb7fe700 1 -- 172.21.15.28:0/2849323591 learned_addr learned my addr 172.21.15.28:0/2849323591 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:54.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.857+0000 7f95fbfff700 1 --2- 172.21.15.28:0/2849323591 >> v2:172.21.15.59:3300/0 conn(0x7f95fc159500 0x7f95fc0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.857+0000 7f95fb7fe700 1 -- 172.21.15.28:0/2849323591 >> v2:172.21.15.59:3300/0 conn(0x7f95fc159500 msgr2=0x7f95fc0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.857+0000 7f95fb7fe700 1 --2- 172.21.15.28:0/2849323591 >> v2:172.21.15.59:3300/0 conn(0x7f95fc159500 0x7f95fc0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.857+0000 7f95fb7fe700 1 -- 172.21.15.28:0/2849323591 >> v1:172.21.15.59:6789/0 conn(0x7f95fc0b1630 legacy=0x7f95fc06bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.857+0000 7f95fb7fe700 1 -- 172.21.15.28:0/2849323591 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95fc0cf2d0 con 0x7f95fc15dbf0 2023-11-28T11:09:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.857+0000 7f95fb7fe700 1 --2- 172.21.15.28:0/2849323591 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95fc15dbf0 0x7f95fc0ce8a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0d0bc0 tx=0x7f95e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d06e33c0a4d016 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.858+0000 7f95faffd700 1 -- 172.21.15.28:0/2849323591 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e400b870 con 0x7f95fc15dbf0 2023-11-28T11:09:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.858+0000 7f95faffd700 1 -- 172.21.15.28:0/2849323591 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e400b9d0 con 0x7f95fc15dbf0 2023-11-28T11:09:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.858+0000 7f95faffd700 1 -- 172.21.15.28:0/2849323591 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e400bd60 con 0x7f95fc15dbf0 2023-11-28T11:09:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.858+0000 7f96028ba700 1 -- 172.21.15.28:0/2849323591 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95fc15dbf0 msgr2=0x7f95fc0ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.858+0000 7f96028ba700 1 --2- 172.21.15.28:0/2849323591 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95fc15dbf0 0x7f95fc0ce8a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0d0bc0 tx=0x7f95e4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.858+0000 7f96028ba700 1 -- 172.21.15.28:0/2849323591 shutdown_connections 2023-11-28T11:09:54.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.858+0000 7f96028ba700 1 --2- 172.21.15.28:0/2849323591 >> v2:172.21.15.59:3300/0 conn(0x7f95fc159500 0x7f95fc0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.858+0000 7f96028ba700 1 --2- 172.21.15.28:0/2849323591 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95fc15dbf0 0x7f95fc0ce8a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.858+0000 7f96028ba700 1 -- 172.21.15.28:0/2849323591 >> 172.21.15.28:0/2849323591 conn(0x7f95fc0cc630 msgr2=0x7f95fc0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:54.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.858+0000 7f96028ba700 1 -- 172.21.15.28:0/2849323591 shutdown_connections 2023-11-28T11:09:54.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.858+0000 7f96028ba700 1 -- 172.21.15.28:0/2849323591 wait complete. 2023-11-28T11:09:54.867 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.860+0000 7f96028ba700 1 Processor -- start 2023-11-28T11:09:54.867 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.860+0000 7f96028ba700 1 -- start start 2023-11-28T11:09:54.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f96028ba700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95fc0b1630 0x7f95fc0c2aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f96028ba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95fc159500 0x7f95fc157f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f96028ba700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95fc15dbf0 0x7f95fc146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f96028ba700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc157ae0 con 0x7f95fc15dbf0 2023-11-28T11:09:54.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f96028ba700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc156ed0 con 0x7f95fc159500 2023-11-28T11:09:54.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f96028ba700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95fc1571b0 con 0x7f95fc0b1630 2023-11-28T11:09:54.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f9600e57700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95fc15dbf0 0x7f95fc146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f9600e57700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95fc15dbf0 0x7f95fc146e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48920/0 (socket says 172.21.15.28:48920) 2023-11-28T11:09:54.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f9600e57700 1 -- 172.21.15.28:0/450868359 learned_addr learned my addr 172.21.15.28:0/450868359 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:54.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f95fbfff700 1 --2- 172.21.15.28:0/450868359 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95fc0b1630 0x7f95fc0c2aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f95fb7fe700 1 --2- 172.21.15.28:0/450868359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95fc159500 0x7f95fc157f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f9600e57700 1 -- 172.21.15.28:0/450868359 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95fc0b1630 msgr2=0x7f95fc0c2aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f9600e57700 1 --2- 172.21.15.28:0/450868359 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95fc0b1630 0x7f95fc0c2aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f9600e57700 1 -- 172.21.15.28:0/450868359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95fc159500 msgr2=0x7f95fc157f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:54.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f9600e57700 1 --2- 172.21.15.28:0/450868359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95fc159500 0x7f95fc157f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:54.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.861+0000 7f9600e57700 1 -- 172.21.15.28:0/450868359 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e4009550 con 0x7f95fc15dbf0 2023-11-28T11:09:54.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.862+0000 7f9600e57700 1 --2- 172.21.15.28:0/450868359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95fc15dbf0 0x7f95fc146e20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f95f400cc50 tx=0x7f95f400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.862+0000 7f95f8ff9700 1 -- 172.21.15.28:0/450868359 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f4003c00 con 0x7f95fc15dbf0 2023-11-28T11:09:54.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.862+0000 7f95f8ff9700 1 -- 172.21.15.28:0/450868359 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f4003d60 con 0x7f95fc15dbf0 2023-11-28T11:09:54.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.862+0000 7f96028ba700 1 -- 172.21.15.28:0/450868359 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95fc15d3f0 con 0x7f95fc15dbf0 2023-11-28T11:09:54.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.862+0000 7f95f8ff9700 1 -- 172.21.15.28:0/450868359 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f400aab0 con 0x7f95fc15dbf0 2023-11-28T11:09:54.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.862+0000 7f96028ba700 1 -- 172.21.15.28:0/450868359 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95fc15d110 con 0x7f95fc15dbf0 2023-11-28T11:09:54.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.863+0000 7f96028ba700 1 -- 172.21.15.28:0/450868359 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95fc15d6d0 con 0x7f95fc15dbf0 2023-11-28T11:09:54.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.863+0000 7f95f8ff9700 1 -- 172.21.15.28:0/450868359 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f95f4010050 con 0x7f95fc15dbf0 2023-11-28T11:09:54.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.864+0000 7f95f8ff9700 1 --2- 172.21.15.28:0/450868359 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f95dc0682f0 0x7f95dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:54.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.864+0000 7f95f8ff9700 1 -- 172.21.15.28:0/450868359 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f95f4085eb0 con 0x7f95fc15dbf0 2023-11-28T11:09:54.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.864+0000 7f95fbfff700 1 --2- 172.21.15.28:0/450868359 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f95dc0682f0 0x7f95dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:54.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.864+0000 7f95fbfff700 1 --2- 172.21.15.28:0/450868359 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f95dc0682f0 0x7f95dc06a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0cd270 tx=0x7f95ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:54.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:54.868+0000 7f95f8ff9700 1 -- 172.21.15.28:0/450868359 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f95f4052380 con 0x7f95fc15dbf0 2023-11-28T11:09:55.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.060+0000 7f96028ba700 1 -- 172.21.15.28:0/450868359 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f95fc0739f0 con 0x7f95fc15dbf0 2023-11-28T11:09:55.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.061+0000 7f95f8ff9700 1 -- 172.21.15.28:0/450868359 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v27) v1 ==== 104+0+11 (secure 0 0 0) 0x7f95f40559a0 con 0x7f95fc15dbf0 2023-11-28T11:09:55.062 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:55.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.062+0000 7f96028ba700 1 -- 172.21.15.28:0/450868359 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f95dc0682f0 msgr2=0x7f95dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.062+0000 7f96028ba700 1 --2- 172.21.15.28:0/450868359 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f95dc0682f0 0x7f95dc06a7a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0cd270 tx=0x7f95ec008040 comp rx=0 tx=0).stop 2023-11-28T11:09:55.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.062+0000 7f96028ba700 1 -- 172.21.15.28:0/450868359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95fc15dbf0 msgr2=0x7f95fc146e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.062+0000 7f96028ba700 1 --2- 172.21.15.28:0/450868359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95fc15dbf0 0x7f95fc146e20 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f95f400cc50 tx=0x7f95f400cf60 comp rx=0 tx=0).stop 2023-11-28T11:09:55.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.062+0000 7f96028ba700 1 -- 172.21.15.28:0/450868359 shutdown_connections 2023-11-28T11:09:55.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.062+0000 7f96028ba700 1 --2- 172.21.15.28:0/450868359 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f95dc0682f0 0x7f95dc06a7a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.062+0000 7f96028ba700 1 --2- 172.21.15.28:0/450868359 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f95fc15dbf0 0x7f95fc146e20 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.062+0000 7f96028ba700 1 --2- 172.21.15.28:0/450868359 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f95fc0b1630 0x7f95fc0c2aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.062+0000 7f96028ba700 1 --2- 172.21.15.28:0/450868359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95fc159500 0x7f95fc157f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.063+0000 7f96028ba700 1 -- 172.21.15.28:0/450868359 >> 172.21.15.28:0/450868359 conn(0x7f95fc0cc630 msgr2=0x7f95fc0b0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:55.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.063+0000 7f96028ba700 1 -- 172.21.15.28:0/450868359 shutdown_connections 2023-11-28T11:09:55.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.063+0000 7f96028ba700 1 -- 172.21.15.28:0/450868359 wait complete. 2023-11-28T11:09:55.077 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:09:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.203+0000 7fa395efc700 1 Processor -- start 2023-11-28T11:09:55.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.203+0000 7fa395efc700 1 -- start start 2023-11-28T11:09:55.214 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa395efc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3900b1d40 0x7fa3900cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.215 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa395efc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa39015dbf0 0x7fa3900ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.215 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa395efc700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3900ceff0 con 0x7fa390159500 2023-11-28T11:09:55.215 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa395efc700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa39015e090 con 0x7fa3900b1d40 2023-11-28T11:09:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa395efc700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900cede0 con 0x7fa39015dbf0 2023-11-28T11:09:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa38effd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3900b1d40 0x7fa3900cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa38effd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3900b1d40 0x7fa3900cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48926/0 (socket says 172.21.15.28:48926) 2023-11-28T11:09:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa38effd700 1 -- 172.21.15.28:0/1664616029 learned_addr learned my addr 172.21.15.28:0/1664616029 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:55.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa38ffff700 1 --2- 172.21.15.28:0/1664616029 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa39015dbf0 0x7fa3900ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa38effd700 1 -- 172.21.15.28:0/1664616029 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa39015dbf0 msgr2=0x7fa3900ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:55.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa38effd700 1 --2- 172.21.15.28:0/1664616029 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa39015dbf0 0x7fa3900ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa38effd700 1 -- 172.21.15.28:0/1664616029 >> v1:172.21.15.59:6789/0 conn(0x7fa390159500 legacy=0x7fa3900b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa38effd700 1 -- 172.21.15.28:0/1664616029 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3900cf2d0 con 0x7fa3900b1d40 2023-11-28T11:09:55.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa38ffff700 1 --2- 172.21.15.28:0/1664616029 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa39015dbf0 0x7fa3900ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:55.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa38effd700 1 --2- 172.21.15.28:0/1664616029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3900b1d40 0x7fa3900cd140 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa38000bb90 tx=0x7fa38000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d25567c6835cac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:55.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa38e7fc700 1 -- 172.21.15.28:0/1664616029 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa38000dd50 con 0x7fa3900b1d40 2023-11-28T11:09:55.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa38e7fc700 1 -- 172.21.15.28:0/1664616029 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa380010470 con 0x7fa3900b1d40 2023-11-28T11:09:55.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.204+0000 7fa38e7fc700 1 -- 172.21.15.28:0/1664616029 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa380010800 con 0x7fa3900b1d40 2023-11-28T11:09:55.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.205+0000 7fa395efc700 1 -- 172.21.15.28:0/1664616029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3900b1d40 msgr2=0x7fa3900cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.205+0000 7fa395efc700 1 --2- 172.21.15.28:0/1664616029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3900b1d40 0x7fa3900cd140 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fa38000bb90 tx=0x7fa38000bea0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.205+0000 7fa395efc700 1 -- 172.21.15.28:0/1664616029 shutdown_connections 2023-11-28T11:09:55.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.205+0000 7fa395efc700 1 --2- 172.21.15.28:0/1664616029 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa39015dbf0 0x7fa3900ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.205+0000 7fa395efc700 1 --2- 172.21.15.28:0/1664616029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3900b1d40 0x7fa3900cd140 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.205+0000 7fa395efc700 1 -- 172.21.15.28:0/1664616029 >> 172.21.15.28:0/1664616029 conn(0x7fa3900cc630 msgr2=0x7fa3900cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:55.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.205+0000 7fa395efc700 1 -- 172.21.15.28:0/1664616029 shutdown_connections 2023-11-28T11:09:55.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.205+0000 7fa395efc700 1 -- 172.21.15.28:0/1664616029 wait complete. 2023-11-28T11:09:55.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.206+0000 7fa395efc700 1 Processor -- start 2023-11-28T11:09:55.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.206+0000 7fa395efc700 1 -- start start 2023-11-28T11:09:55.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa395efc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3900b1d40 0x7fa390152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa395efc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa390159500 0x7fa3901528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa395efc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa39015dbf0 0x7fa3900c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.218 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa395efc700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900c3fc0 con 0x7fa3900b1d40 2023-11-28T11:09:55.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa395efc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa390153700 con 0x7fa390159500 2023-11-28T11:09:55.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa395efc700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3900c3620 con 0x7fa39015dbf0 2023-11-28T11:09:55.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa38f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3900b1d40 0x7fa390152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa38f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3900b1d40 0x7fa390152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48932/0 (socket says 172.21.15.28:48932) 2023-11-28T11:09:55.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa38f7fe700 1 -- 172.21.15.28:0/789218615 learned_addr learned my addr 172.21.15.28:0/789218615 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:55.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa38ffff700 1 --2- 172.21.15.28:0/789218615 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa39015dbf0 0x7fa3900c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa38effd700 1 --2- 172.21.15.28:0/789218615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa390159500 0x7fa3901528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa38f7fe700 1 -- 172.21.15.28:0/789218615 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa39015dbf0 msgr2=0x7fa3900c43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa38f7fe700 1 --2- 172.21.15.28:0/789218615 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa39015dbf0 0x7fa3900c43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa38f7fe700 1 -- 172.21.15.28:0/789218615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa390159500 msgr2=0x7fa3901528e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa38f7fe700 1 --2- 172.21.15.28:0/789218615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa390159500 0x7fa3901528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.219 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.207+0000 7fa38f7fe700 1 -- 172.21.15.28:0/789218615 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38000b7f0 con 0x7fa3900b1d40 2023-11-28T11:09:55.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.208+0000 7fa38ffff700 1 --2- 172.21.15.28:0/789218615 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa39015dbf0 0x7fa3900c43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:55.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.208+0000 7fa38f7fe700 1 --2- 172.21.15.28:0/789218615 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3900b1d40 0x7fa390152e70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa38400c510 tx=0x7fa38400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:55.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.208+0000 7fa377fff700 1 -- 172.21.15.28:0/789218615 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3840052d0 con 0x7fa3900b1d40 2023-11-28T11:09:55.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.208+0000 7fa377fff700 1 -- 172.21.15.28:0/789218615 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa384005430 con 0x7fa3900b1d40 2023-11-28T11:09:55.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.208+0000 7fa395efc700 1 -- 172.21.15.28:0/789218615 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa39015d3f0 con 0x7fa3900b1d40 2023-11-28T11:09:55.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.208+0000 7fa377fff700 1 -- 172.21.15.28:0/789218615 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3840038c0 con 0x7fa3900b1d40 2023-11-28T11:09:55.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.208+0000 7fa395efc700 1 -- 172.21.15.28:0/789218615 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa39015d110 con 0x7fa3900b1d40 2023-11-28T11:09:55.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.209+0000 7fa395efc700 1 -- 172.21.15.28:0/789218615 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa39015d6d0 con 0x7fa3900b1d40 2023-11-28T11:09:55.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.209+0000 7fa377fff700 1 -- 172.21.15.28:0/789218615 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa384019030 con 0x7fa3900b1d40 2023-11-28T11:09:55.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.210+0000 7fa377fff700 1 --2- 172.21.15.28:0/789218615 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa37c0682f0 0x7fa37c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.210+0000 7fa377fff700 1 -- 172.21.15.28:0/789218615 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa384085370 con 0x7fa3900b1d40 2023-11-28T11:09:55.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.210+0000 7fa38effd700 1 --2- 172.21.15.28:0/789218615 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa37c0682f0 0x7fa37c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.210+0000 7fa38effd700 1 --2- 172.21.15.28:0/789218615 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa37c0682f0 0x7fa37c06a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa38000bb90 tx=0x7fa380013000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:55.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.214+0000 7fa377fff700 1 -- 172.21.15.28:0/789218615 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa3840518a0 con 0x7fa3900b1d40 2023-11-28T11:09:55.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.408+0000 7fa395efc700 1 -- 172.21.15.28:0/789218615 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa3900739f0 con 0x7fa3900b1d40 2023-11-28T11:09:55.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.408+0000 7fa377fff700 1 -- 172.21.15.28:0/789218615 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v27) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa384054ec0 con 0x7fa3900b1d40 2023-11-28T11:09:55.410 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:09:55.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.410+0000 7fa395efc700 1 -- 172.21.15.28:0/789218615 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa37c0682f0 msgr2=0x7fa37c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.410+0000 7fa395efc700 1 --2- 172.21.15.28:0/789218615 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa37c0682f0 0x7fa37c06a7a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fa38000bb90 tx=0x7fa380013000 comp rx=0 tx=0).stop 2023-11-28T11:09:55.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.410+0000 7fa395efc700 1 -- 172.21.15.28:0/789218615 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3900b1d40 msgr2=0x7fa390152e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.410+0000 7fa395efc700 1 --2- 172.21.15.28:0/789218615 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3900b1d40 0x7fa390152e70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fa38400c510 tx=0x7fa38400c8d0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.410+0000 7fa395efc700 1 -- 172.21.15.28:0/789218615 shutdown_connections 2023-11-28T11:09:55.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.410+0000 7fa395efc700 1 --2- 172.21.15.28:0/789218615 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa37c0682f0 0x7fa37c06a7a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.410+0000 7fa395efc700 1 --2- 172.21.15.28:0/789218615 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3900b1d40 0x7fa390152e70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.410+0000 7fa395efc700 1 --2- 172.21.15.28:0/789218615 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa39015dbf0 0x7fa3900c43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.410+0000 7fa395efc700 1 --2- 172.21.15.28:0/789218615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa390159500 0x7fa3901528e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.410+0000 7fa395efc700 1 -- 172.21.15.28:0/789218615 >> 172.21.15.28:0/789218615 conn(0x7fa3900cc630 msgr2=0x7fa3901539c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:55.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.410+0000 7fa395efc700 1 -- 172.21.15.28:0/789218615 shutdown_connections 2023-11-28T11:09:55.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.410+0000 7fa395efc700 1 -- 172.21.15.28:0/789218615 wait complete. 2023-11-28T11:09:55.424 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-11-28T11:09:55.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.550+0000 7fdd9ad0a700 1 Processor -- start 2023-11-28T11:09:55.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.550+0000 7fdd9ad0a700 1 -- start start 2023-11-28T11:09:55.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.550+0000 7fdd9ad0a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdd9406b900 0x7fdd940cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.550+0000 7fdd9ad0a700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd940ae730 con 0x7fdd940cd140 2023-11-28T11:09:55.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd9ad0a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd940c3e10 con 0x7fdd9415a640 2023-11-28T11:09:55.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd9ad0a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd940cb210 con 0x7fdd9406b900 2023-11-28T11:09:55.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd93fff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdd940cd140 0x7fdd9406b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:36998/0 (socket says 172.21.15.28:36998) 2023-11-28T11:09:55.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd93fff700 1 -- 172.21.15.28:0/3325609000 learned_addr learned my addr 172.21.15.28:0/3325609000 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:55.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd992a7700 1 --2- 172.21.15.28:0/3325609000 >> v2:172.21.15.59:3300/0 conn(0x7fdd9406b900 0x7fdd940cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd937fe700 1 -- 172.21.15.28:0/3325609000 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406602238 0 0) 0x7fdd940ae730 con 0x7fdd940cd140 2023-11-28T11:09:55.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd937fe700 1 -- 172.21.15.28:0/3325609000 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd840035b0 con 0x7fdd940cd140 2023-11-28T11:09:55.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd937fe700 1 -- 172.21.15.28:0/3325609000 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1190014410 0 0) 0x7fdd840035b0 con 0x7fdd940cd140 2023-11-28T11:09:55.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd937fe700 1 -- 172.21.15.28:0/3325609000 >> v2:172.21.15.59:3300/0 conn(0x7fdd9406b900 msgr2=0x7fdd940cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:55.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd937fe700 1 --2- 172.21.15.28:0/3325609000 >> v2:172.21.15.59:3300/0 conn(0x7fdd9406b900 0x7fdd940cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd937fe700 1 -- 172.21.15.28:0/3325609000 >> v1:172.21.15.59:6789/0 conn(0x7fdd9415a640 legacy=0x7fdd940c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd937fe700 1 -- 172.21.15.28:0/3325609000 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd9406e9e0 con 0x7fdd940cd140 2023-11-28T11:09:55.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd937fe700 1 -- 172.21.15.28:0/3325609000 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fdd80001670 con 0x7fdd940cd140 2023-11-28T11:09:55.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd937fe700 1 -- 172.21.15.28:0/3325609000 >> v1:172.21.15.28:6789/0 conn(0x7fdd940cd140 legacy=0x7fdd9406b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd937fe700 1 --2- 172.21.15.28:0/3325609000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd84003a00 0x7fdd84003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd937fe700 1 --2- 172.21.15.28:0/3325609000 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdd840043f0 0x7fdd840048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd937fe700 1 --2- 172.21.15.28:0/3325609000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd840050b0 0x7fdd8400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd937fe700 1 -- 172.21.15.28:0/3325609000 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd940cb210 con 0x7fdd84003a00 2023-11-28T11:09:55.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd937fe700 1 -- 172.21.15.28:0/3325609000 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8400c000 con 0x7fdd840050b0 2023-11-28T11:09:55.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd937fe700 1 -- 172.21.15.28:0/3325609000 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8400c1c0 con 0x7fdd840043f0 2023-11-28T11:09:55.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.551+0000 7fdd98aa6700 1 --2- 172.21.15.28:0/3325609000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd84003a00 0x7fdd84003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd992a7700 1 --2- 172.21.15.28:0/3325609000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd840050b0 0x7fdd8400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd93fff700 1 --2- 172.21.15.28:0/3325609000 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdd840043f0 0x7fdd840048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd98aa6700 1 -- 172.21.15.28:0/3325609000 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdd840043f0 msgr2=0x7fdd840048b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:55.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd98aa6700 1 --2- 172.21.15.28:0/3325609000 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdd840043f0 0x7fdd840048b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd98aa6700 1 -- 172.21.15.28:0/3325609000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd840050b0 msgr2=0x7fdd8400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:55.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd98aa6700 1 --2- 172.21.15.28:0/3325609000 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd840050b0 0x7fdd8400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd98aa6700 1 -- 172.21.15.28:0/3325609000 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd840035b0 con 0x7fdd84003a00 2023-11-28T11:09:55.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd992a7700 1 -- 172.21.15.28:0/3325609000 reap_dead start 2023-11-28T11:09:55.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd98aa6700 1 --2- 172.21.15.28:0/3325609000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd84003a00 0x7fdd84003dd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdd88001980 tx=0x7fdd84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78fb53d0948bb01c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:55.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd937fe700 1 -- 172.21.15.28:0/3325609000 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88001420 con 0x7fdd84003a00 2023-11-28T11:09:55.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd937fe700 1 -- 172.21.15.28:0/3325609000 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd880052d0 con 0x7fdd84003a00 2023-11-28T11:09:55.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd937fe700 1 -- 172.21.15.28:0/3325609000 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd88003680 con 0x7fdd84003a00 2023-11-28T11:09:55.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd9ad0a700 1 -- 172.21.15.28:0/3325609000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd84003a00 msgr2=0x7fdd84003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd9ad0a700 1 --2- 172.21.15.28:0/3325609000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd84003a00 0x7fdd84003dd0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fdd88001980 tx=0x7fdd84005880 comp rx=0 tx=0).stop 2023-11-28T11:09:55.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd9ad0a700 1 -- 172.21.15.28:0/3325609000 shutdown_connections 2023-11-28T11:09:55.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd9ad0a700 1 --2- 172.21.15.28:0/3325609000 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd84003a00 0x7fdd84003dd0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.552+0000 7fdd9ad0a700 1 -- 172.21.15.28:0/3325609000 >> 172.21.15.28:0/3325609000 conn(0x7fdd940cc630 msgr2=0x7fdd940cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:55.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.553+0000 7fdd9ad0a700 1 -- 172.21.15.28:0/3325609000 shutdown_connections 2023-11-28T11:09:55.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.553+0000 7fdd9ad0a700 1 -- 172.21.15.28:0/3325609000 wait complete. 2023-11-28T11:09:55.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.554+0000 7fdd9ad0a700 1 Processor -- start 2023-11-28T11:09:55.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.554+0000 7fdd9ad0a700 1 -- start start 2023-11-28T11:09:55.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd9ad0a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdd84003a00 0x7fdd941515b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd9ad0a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd94152320 0x7fdd9415d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd9ad0a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd94151d60 0x7fdd94155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd9ad0a700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9415ce50 con 0x7fdd94152320 2023-11-28T11:09:55.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd9ad0a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9415c1d0 con 0x7fdd94151d60 2023-11-28T11:09:55.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd9ad0a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd9415c4b0 con 0x7fdd84003a00 2023-11-28T11:09:55.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd93fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd94152320 0x7fdd9415d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd93fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd94152320 0x7fdd9415d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48952/0 (socket says 172.21.15.28:48952) 2023-11-28T11:09:55.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd93fff700 1 -- 172.21.15.28:0/3592495956 learned_addr learned my addr 172.21.15.28:0/3592495956 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:55.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd98aa6700 1 --2- 172.21.15.28:0/3592495956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdd84003a00 0x7fdd941515b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd992a7700 1 --2- 172.21.15.28:0/3592495956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd94151d60 0x7fdd94155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd93fff700 1 -- 172.21.15.28:0/3592495956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdd84003a00 msgr2=0x7fdd941515b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd93fff700 1 --2- 172.21.15.28:0/3592495956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdd84003a00 0x7fdd941515b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd93fff700 1 -- 172.21.15.28:0/3592495956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd94151d60 msgr2=0x7fdd94155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd93fff700 1 --2- 172.21.15.28:0/3592495956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd94151d60 0x7fdd94155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd93fff700 1 -- 172.21.15.28:0/3592495956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd940aec60 con 0x7fdd94152320 2023-11-28T11:09:55.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd992a7700 1 --2- 172.21.15.28:0/3592495956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd94151d60 0x7fdd94155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:55.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd93fff700 1 --2- 172.21.15.28:0/3592495956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd94152320 0x7fdd9415d270 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdd80000f60 tx=0x7fdd8000ea60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:55.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.555+0000 7fdd917fa700 1 -- 172.21.15.28:0/3592495956 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80012590 con 0x7fdd94152320 2023-11-28T11:09:55.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.556+0000 7fdd917fa700 1 -- 172.21.15.28:0/3592495956 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd80012710 con 0x7fdd94152320 2023-11-28T11:09:55.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.556+0000 7fdd9ad0a700 1 -- 172.21.15.28:0/3592495956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd940ae980 con 0x7fdd94152320 2023-11-28T11:09:55.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.556+0000 7fdd917fa700 1 -- 172.21.15.28:0/3592495956 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd80012ac0 con 0x7fdd94152320 2023-11-28T11:09:55.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.556+0000 7fdd9ad0a700 1 -- 172.21.15.28:0/3592495956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd940aef40 con 0x7fdd94152320 2023-11-28T11:09:55.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.556+0000 7fdd9ad0a700 1 -- 172.21.15.28:0/3592495956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd940739f0 con 0x7fdd94152320 2023-11-28T11:09:55.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.558+0000 7fdd917fa700 1 -- 172.21.15.28:0/3592495956 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdd80012c20 con 0x7fdd94152320 2023-11-28T11:09:55.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.558+0000 7fdd917fa700 1 --2- 172.21.15.28:0/3592495956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdd780682f0 0x7fdd7806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.558+0000 7fdd917fa700 1 -- 172.21.15.28:0/3592495956 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fdd800867e0 con 0x7fdd94152320 2023-11-28T11:09:55.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.558+0000 7fdd98aa6700 1 --2- 172.21.15.28:0/3592495956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdd780682f0 0x7fdd7806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.559+0000 7fdd98aa6700 1 --2- 172.21.15.28:0/3592495956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdd780682f0 0x7fdd7806a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdd88000c00 tx=0x7fdd8800a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:55.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.562+0000 7fdd917fa700 1 -- 172.21.15.28:0/3592495956 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdd80052db0 con 0x7fdd94152320 2023-11-28T11:09:55.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.756+0000 7fdd9ad0a700 1 -- 172.21.15.28:0/3592495956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fdd940af210 con 0x7fdd94152320 2023-11-28T11:09:55.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.756+0000 7fdd917fa700 1 -- 172.21.15.28:0/3592495956 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v27) v1 ==== 103+0+11 (secure 0 0 0) 0x7fdd800563d0 con 0x7fdd94152320 2023-11-28T11:09:55.758 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:09:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.759+0000 7fdd9ad0a700 1 -- 172.21.15.28:0/3592495956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdd780682f0 msgr2=0x7fdd7806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.759+0000 7fdd9ad0a700 1 --2- 172.21.15.28:0/3592495956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdd780682f0 0x7fdd7806a7a0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdd88000c00 tx=0x7fdd8800a000 comp rx=0 tx=0).stop 2023-11-28T11:09:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.759+0000 7fdd9ad0a700 1 -- 172.21.15.28:0/3592495956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd94152320 msgr2=0x7fdd9415d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.759+0000 7fdd9ad0a700 1 --2- 172.21.15.28:0/3592495956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd94152320 0x7fdd9415d270 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fdd80000f60 tx=0x7fdd8000ea60 comp rx=0 tx=0).stop 2023-11-28T11:09:55.763 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.759+0000 7fdd9ad0a700 1 -- 172.21.15.28:0/3592495956 shutdown_connections 2023-11-28T11:09:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.759+0000 7fdd9ad0a700 1 --2- 172.21.15.28:0/3592495956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdd780682f0 0x7fdd7806a7a0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.759+0000 7fdd9ad0a700 1 --2- 172.21.15.28:0/3592495956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd94151d60 0x7fdd94155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.759+0000 7fdd9ad0a700 1 --2- 172.21.15.28:0/3592495956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdd84003a00 0x7fdd941515b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.759+0000 7fdd9ad0a700 1 --2- 172.21.15.28:0/3592495956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdd94152320 0x7fdd9415d270 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.759+0000 7fdd9ad0a700 1 -- 172.21.15.28:0/3592495956 >> 172.21.15.28:0/3592495956 conn(0x7fdd940cc630 msgr2=0x7fdd94156c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:55.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.759+0000 7fdd9ad0a700 1 -- 172.21.15.28:0/3592495956 shutdown_connections 2023-11-28T11:09:55.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.759+0000 7fdd9ad0a700 1 -- 172.21.15.28:0/3592495956 wait complete. 2023-11-28T11:09:55.772 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:55.897 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.895+0000 7f0b94cbb700 1 Processor -- start 2023-11-28T11:09:55.897 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.895+0000 7f0b94cbb700 1 -- start start 2023-11-28T11:09:55.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.896+0000 7f0b94cbb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0b9015a6c0 0x7f0b900cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.896+0000 7f0b94cbb700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b900cfb90 con 0x7f0b900b2660 2023-11-28T11:09:55.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.896+0000 7f0b94cbb700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b900cb930 con 0x7f0b900d0020 2023-11-28T11:09:55.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.896+0000 7f0b94cbb700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b900cf260 con 0x7f0b9015a6c0 2023-11-28T11:09:55.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.896+0000 7f0b8ed9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0b900b2660 0x7f0b900b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37020/0 (socket says 172.21.15.28:37020) 2023-11-28T11:09:55.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.896+0000 7f0b8ed9d700 1 -- 172.21.15.28:0/3044965291 learned_addr learned my addr 172.21.15.28:0/3044965291 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:55.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.896+0000 7f0b8e59c700 1 --2- 172.21.15.28:0/3044965291 >> v2:172.21.15.59:3300/0 conn(0x7f0b9015a6c0 0x7f0b900cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.896+0000 7f0b8d59a700 1 -- 172.21.15.28:0/3044965291 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3220559948 0 0) 0x7f0b900cfb90 con 0x7f0b900b2660 2023-11-28T11:09:55.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.896+0000 7f0b8d59a700 1 -- 172.21.15.28:0/3044965291 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b780035b0 con 0x7f0b900b2660 2023-11-28T11:09:55.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8d59a700 1 -- 172.21.15.28:0/3044965291 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3686601873 0 0) 0x7f0b780035b0 con 0x7f0b900b2660 2023-11-28T11:09:55.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8d59a700 1 -- 172.21.15.28:0/3044965291 >> v2:172.21.15.59:3300/0 conn(0x7f0b9015a6c0 msgr2=0x7f0b900cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:55.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8d59a700 1 --2- 172.21.15.28:0/3044965291 >> v2:172.21.15.59:3300/0 conn(0x7f0b9015a6c0 0x7f0b900cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8d59a700 1 -- 172.21.15.28:0/3044965291 >> v1:172.21.15.59:6789/0 conn(0x7f0b900d0020 legacy=0x7f0b900ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8d59a700 1 -- 172.21.15.28:0/3044965291 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b900cf540 con 0x7f0b900b2660 2023-11-28T11:09:55.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8d59a700 1 -- 172.21.15.28:0/3044965291 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f0b880019f0 con 0x7f0b900b2660 2023-11-28T11:09:55.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8d59a700 1 -- 172.21.15.28:0/3044965291 >> v1:172.21.15.28:6789/0 conn(0x7f0b900b2660 legacy=0x7f0b900b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8d59a700 1 --2- 172.21.15.28:0/3044965291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b78003a00 0x7f0b78003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8d59a700 1 --2- 172.21.15.28:0/3044965291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b780043f0 0x7f0b780048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8d59a700 1 --2- 172.21.15.28:0/3044965291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b780050b0 0x7f0b7800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8d59a700 1 -- 172.21.15.28:0/3044965291 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b900cf260 con 0x7f0b780043f0 2023-11-28T11:09:55.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8d59a700 1 -- 172.21.15.28:0/3044965291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7800bf50 con 0x7f0b78003a00 2023-11-28T11:09:55.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8d59a700 1 -- 172.21.15.28:0/3044965291 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7800c110 con 0x7f0b780050b0 2023-11-28T11:09:55.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8dd9b700 1 --2- 172.21.15.28:0/3044965291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b780043f0 0x7f0b780048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8e59c700 1 --2- 172.21.15.28:0/3044965291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b78003a00 0x7f0b78003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8ed9d700 1 --2- 172.21.15.28:0/3044965291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b780050b0 0x7f0b7800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8dd9b700 1 -- 172.21.15.28:0/3044965291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b780050b0 msgr2=0x7f0b7800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:55.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8dd9b700 1 --2- 172.21.15.28:0/3044965291 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b780050b0 0x7f0b7800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8dd9b700 1 -- 172.21.15.28:0/3044965291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b78003a00 msgr2=0x7f0b78003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:55.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8dd9b700 1 --2- 172.21.15.28:0/3044965291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b78003a00 0x7f0b78003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8ed9d700 1 -- 172.21.15.28:0/3044965291 reap_dead start 2023-11-28T11:09:55.903 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8dd9b700 1 -- 172.21.15.28:0/3044965291 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b780035b0 con 0x7f0b780043f0 2023-11-28T11:09:55.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.897+0000 7f0b8dd9b700 1 --2- 172.21.15.28:0/3044965291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b780043f0 0x7f0b780048b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0b800018d0 tx=0x7f0b78005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d817ad2e147a0771 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:55.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.898+0000 7f0b8d59a700 1 -- 172.21.15.28:0/3044965291 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b800052d0 con 0x7f0b780043f0 2023-11-28T11:09:55.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.898+0000 7f0b8d59a700 1 -- 172.21.15.28:0/3044965291 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b80005430 con 0x7f0b780043f0 2023-11-28T11:09:55.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.898+0000 7f0b8d59a700 1 -- 172.21.15.28:0/3044965291 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b800038a0 con 0x7f0b780043f0 2023-11-28T11:09:55.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.898+0000 7f0b94cbb700 1 -- 172.21.15.28:0/3044965291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b780043f0 msgr2=0x7f0b780048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.898+0000 7f0b94cbb700 1 --2- 172.21.15.28:0/3044965291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b780043f0 0x7f0b780048b0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f0b800018d0 tx=0x7f0b78005880 comp rx=0 tx=0).stop 2023-11-28T11:09:55.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.898+0000 7f0b94cbb700 1 -- 172.21.15.28:0/3044965291 shutdown_connections 2023-11-28T11:09:55.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.898+0000 7f0b94cbb700 1 --2- 172.21.15.28:0/3044965291 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b780043f0 0x7f0b780048b0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.898+0000 7f0b94cbb700 1 -- 172.21.15.28:0/3044965291 >> 172.21.15.28:0/3044965291 conn(0x7f0b900cd660 msgr2=0x7f0b900cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:55.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.898+0000 7f0b94cbb700 1 -- 172.21.15.28:0/3044965291 shutdown_connections 2023-11-28T11:09:55.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.898+0000 7f0b94cbb700 1 -- 172.21.15.28:0/3044965291 wait complete. 2023-11-28T11:09:55.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.900+0000 7f0b94cbb700 1 Processor -- start 2023-11-28T11:09:55.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.900+0000 7f0b94cbb700 1 -- start start 2023-11-28T11:09:55.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.900+0000 7f0b94cbb700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b780043f0 0x7f0b9015aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.900+0000 7f0b94cbb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b901f07b0 0x7f0b900c9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.900+0000 7f0b94cbb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b9015cd00 0x7f0b9015d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.900+0000 7f0b94cbb700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9015d6c0 con 0x7f0b780043f0 2023-11-28T11:09:55.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.900+0000 7f0b94cbb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9015b120 con 0x7f0b9015cd00 2023-11-28T11:09:55.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.900+0000 7f0b94cbb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b9015b400 con 0x7f0b901f07b0 2023-11-28T11:09:55.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.900+0000 7f0b8e59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b780043f0 0x7f0b9015aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.900+0000 7f0b8e59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b780043f0 0x7f0b9015aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48970/0 (socket says 172.21.15.28:48970) 2023-11-28T11:09:55.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.900+0000 7f0b8e59c700 1 -- 172.21.15.28:0/1662881553 learned_addr learned my addr 172.21.15.28:0/1662881553 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:55.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.901+0000 7f0b8ed9d700 1 --2- 172.21.15.28:0/1662881553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b9015cd00 0x7f0b9015d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.901+0000 7f0b8dd9b700 1 --2- 172.21.15.28:0/1662881553 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b901f07b0 0x7f0b900c9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.901+0000 7f0b8e59c700 1 -- 172.21.15.28:0/1662881553 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b901f07b0 msgr2=0x7f0b900c9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.901+0000 7f0b8e59c700 1 --2- 172.21.15.28:0/1662881553 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b901f07b0 0x7f0b900c9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.901+0000 7f0b8e59c700 1 -- 172.21.15.28:0/1662881553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b9015cd00 msgr2=0x7f0b9015d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:55.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.901+0000 7f0b8e59c700 1 --2- 172.21.15.28:0/1662881553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b9015cd00 0x7f0b9015d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:55.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.901+0000 7f0b8e59c700 1 -- 172.21.15.28:0/1662881553 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b900d0470 con 0x7f0b780043f0 2023-11-28T11:09:55.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.901+0000 7f0b8e59c700 1 --2- 172.21.15.28:0/1662881553 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b780043f0 0x7f0b9015aa90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0b840119d0 tx=0x7f0b84011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:55.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.901+0000 7f0b7effd700 1 -- 172.21.15.28:0/1662881553 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b84009630 con 0x7f0b780043f0 2023-11-28T11:09:55.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.901+0000 7f0b7effd700 1 -- 172.21.15.28:0/1662881553 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b84004240 con 0x7f0b780043f0 2023-11-28T11:09:55.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.901+0000 7f0b94cbb700 1 -- 172.21.15.28:0/1662881553 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b900cf820 con 0x7f0b780043f0 2023-11-28T11:09:55.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.901+0000 7f0b7effd700 1 -- 172.21.15.28:0/1662881553 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8400f730 con 0x7f0b780043f0 2023-11-28T11:09:55.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.901+0000 7f0b94cbb700 1 -- 172.21.15.28:0/1662881553 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b900d0600 con 0x7f0b780043f0 2023-11-28T11:09:55.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.902+0000 7f0b94cbb700 1 -- 172.21.15.28:0/1662881553 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b900675a0 con 0x7f0b780043f0 2023-11-28T11:09:55.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.903+0000 7f0b7effd700 1 -- 172.21.15.28:0/1662881553 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0b84025070 con 0x7f0b780043f0 2023-11-28T11:09:55.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.903+0000 7f0b7effd700 1 --2- 172.21.15.28:0/1662881553 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b700682f0 0x7f0b7006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:55.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.903+0000 7f0b7effd700 1 -- 172.21.15.28:0/1662881553 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f0b840864c0 con 0x7f0b780043f0 2023-11-28T11:09:55.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.903+0000 7f0b8dd9b700 1 --2- 172.21.15.28:0/1662881553 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b700682f0 0x7f0b7006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:55.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.903+0000 7f0b8dd9b700 1 --2- 172.21.15.28:0/1662881553 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b700682f0 0x7f0b7006a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0b900b2280 tx=0x7f0b80005000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:55.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:55.907+0000 7f0b7effd700 1 -- 172.21.15.28:0/1662881553 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0b840529f0 con 0x7f0b780043f0 2023-11-28T11:09:56.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.146+0000 7f0b94cbb700 1 -- 172.21.15.28:0/1662881553 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0b9006fb30 con 0x7f0b780043f0 2023-11-28T11:09:56.148 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.147+0000 7f0b7effd700 1 -- 172.21.15.28:0/1662881553 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4644 (secure 0 0 0) 0x7f0b84056010 con 0x7f0b780043f0 2023-11-28T11:09:56.148 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:56.148 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/3112877467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3112877467},{"type":"v1","addr":"172.21.15.59:6834","nonce":3112877467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3448236489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3448236489},{"type":"v1","addr":"172.21.15.28:6836","nonce":3448236489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4232269940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4232269940},{"type":"v1","addr":"172.21.15.28:6837","nonce":4232269940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:45.924816+0000","modified":"2023-11-28T11:09:03.234785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/3293053261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3293053261},{"type":"v1","addr":"172.21.15.59:6835","nonce":3293053261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:09:56.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.148+0000 7f0b94cbb700 1 -- 172.21.15.28:0/1662881553 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b700682f0 msgr2=0x7f0b7006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.148+0000 7f0b94cbb700 1 --2- 172.21.15.28:0/1662881553 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b700682f0 0x7f0b7006a7a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0b900b2280 tx=0x7f0b80005000 comp rx=0 tx=0).stop 2023-11-28T11:09:56.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.148+0000 7f0b94cbb700 1 -- 172.21.15.28:0/1662881553 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b780043f0 msgr2=0x7f0b9015aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.148+0000 7f0b94cbb700 1 --2- 172.21.15.28:0/1662881553 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b780043f0 0x7f0b9015aa90 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f0b840119d0 tx=0x7f0b84011d90 comp rx=0 tx=0).stop 2023-11-28T11:09:56.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.148+0000 7f0b94cbb700 1 -- 172.21.15.28:0/1662881553 shutdown_connections 2023-11-28T11:09:56.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.148+0000 7f0b94cbb700 1 --2- 172.21.15.28:0/1662881553 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b700682f0 0x7f0b7006a7a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.148+0000 7f0b94cbb700 1 --2- 172.21.15.28:0/1662881553 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b780043f0 0x7f0b9015aa90 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.148+0000 7f0b94cbb700 1 --2- 172.21.15.28:0/1662881553 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b9015cd00 0x7f0b9015d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.148+0000 7f0b94cbb700 1 --2- 172.21.15.28:0/1662881553 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b901f07b0 0x7f0b900c9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.148+0000 7f0b94cbb700 1 -- 172.21.15.28:0/1662881553 >> 172.21.15.28:0/1662881553 conn(0x7f0b900cd660 msgr2=0x7f0b90065ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:56.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.149+0000 7f0b94cbb700 1 -- 172.21.15.28:0/1662881553 shutdown_connections 2023-11-28T11:09:56.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.149+0000 7f0b94cbb700 1 -- 172.21.15.28:0/1662881553 wait complete. 2023-11-28T11:09:56.152 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 9 2023-11-28T11:09:56.163 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:56.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.286+0000 7f94fec82700 1 Processor -- start 2023-11-28T11:09:56.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.286+0000 7f94fec82700 1 -- start start 2023-11-28T11:09:56.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94fec82700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f94f815a650 0x7f94f80b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94fec82700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f80ca990 con 0x7f94f80b5500 2023-11-28T11:09:56.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94fec82700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94f80caf50 con 0x7f94f815aa20 2023-11-28T11:09:56.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94fec82700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94f80aef10 con 0x7f94f815a650 2023-11-28T11:09:56.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94fd21f700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f94f80b5500 0x7f94f80cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37038/0 (socket says 172.21.15.28:37038) 2023-11-28T11:09:56.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94fd21f700 1 -- 172.21.15.28:0/300337355 learned_addr learned my addr 172.21.15.28:0/300337355 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:56.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94f77fe700 1 -- 172.21.15.28:0/300337355 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546344693 0 0) 0x7f94f80ca990 con 0x7f94f80b5500 2023-11-28T11:09:56.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94fca1e700 1 --2- 172.21.15.28:0/300337355 >> v2:172.21.15.59:3300/0 conn(0x7f94f815a650 0x7f94f80b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94f77fe700 1 -- 172.21.15.28:0/300337355 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94e80035b0 con 0x7f94f80b5500 2023-11-28T11:09:56.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94f77fe700 1 -- 172.21.15.28:0/300337355 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3609661187 0 0) 0x7f94e80035b0 con 0x7f94f80b5500 2023-11-28T11:09:56.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94f77fe700 1 -- 172.21.15.28:0/300337355 >> v2:172.21.15.59:3300/0 conn(0x7f94f815a650 msgr2=0x7f94f80b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:56.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94f77fe700 1 --2- 172.21.15.28:0/300337355 >> v2:172.21.15.59:3300/0 conn(0x7f94f815a650 0x7f94f80b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94f77fe700 1 -- 172.21.15.28:0/300337355 >> v1:172.21.15.59:6789/0 conn(0x7f94f815aa20 legacy=0x7f94f80ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94f77fe700 1 -- 172.21.15.28:0/300337355 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f80af1f0 con 0x7f94f80b5500 2023-11-28T11:09:56.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94f77fe700 1 -- 172.21.15.28:0/300337355 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f94f00019f0 con 0x7f94f80b5500 2023-11-28T11:09:56.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94f77fe700 1 -- 172.21.15.28:0/300337355 >> v1:172.21.15.28:6789/0 conn(0x7f94f80b5500 legacy=0x7f94f80cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94f77fe700 1 --2- 172.21.15.28:0/300337355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94e8003a00 0x7f94e8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.287+0000 7f94f77fe700 1 --2- 172.21.15.28:0/300337355 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94e8004850 0x7f94e8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94f77fe700 1 --2- 172.21.15.28:0/300337355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e8005200 0x7f94e800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94f77fe700 1 -- 172.21.15.28:0/300337355 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f80aef10 con 0x7f94e8003a00 2023-11-28T11:09:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94f77fe700 1 -- 172.21.15.28:0/300337355 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94e800c080 con 0x7f94e8005200 2023-11-28T11:09:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94f77fe700 1 -- 172.21.15.28:0/300337355 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94e800c240 con 0x7f94e8004850 2023-11-28T11:09:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94fca1e700 1 --2- 172.21.15.28:0/300337355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94e8003a00 0x7f94e8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94fd21f700 1 --2- 172.21.15.28:0/300337355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e8005200 0x7f94e800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94f7fff700 1 --2- 172.21.15.28:0/300337355 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94e8004850 0x7f94e8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94fca1e700 1 -- 172.21.15.28:0/300337355 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94e8004850 msgr2=0x7f94e8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94fca1e700 1 --2- 172.21.15.28:0/300337355 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94e8004850 0x7f94e8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94fca1e700 1 -- 172.21.15.28:0/300337355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e8005200 msgr2=0x7f94e800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:56.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94fca1e700 1 --2- 172.21.15.28:0/300337355 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94e8005200 0x7f94e800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94fca1e700 1 -- 172.21.15.28:0/300337355 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94e80035b0 con 0x7f94e8003a00 2023-11-28T11:09:56.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94fd21f700 1 -- 172.21.15.28:0/300337355 reap_dead start 2023-11-28T11:09:56.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94fca1e700 1 --2- 172.21.15.28:0/300337355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94e8003a00 0x7f94e8003e10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009b60 tx=0x7f94e8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c8aa631933adc1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94f77fe700 1 -- 172.21.15.28:0/300337355 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec00e6c0 con 0x7f94e8003a00 2023-11-28T11:09:56.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94f77fe700 1 -- 172.21.15.28:0/300337355 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ec00e820 con 0x7f94e8003a00 2023-11-28T11:09:56.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.288+0000 7f94f77fe700 1 -- 172.21.15.28:0/300337355 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec00eb90 con 0x7f94e8003a00 2023-11-28T11:09:56.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.289+0000 7f94fec82700 1 -- 172.21.15.28:0/300337355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94e8003a00 msgr2=0x7f94e8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.289+0000 7f94fec82700 1 --2- 172.21.15.28:0/300337355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94e8003a00 0x7f94e8003e10 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f94ec009b60 tx=0x7f94e8005880 comp rx=0 tx=0).stop 2023-11-28T11:09:56.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.289+0000 7f94fec82700 1 -- 172.21.15.28:0/300337355 shutdown_connections 2023-11-28T11:09:56.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.289+0000 7f94fec82700 1 --2- 172.21.15.28:0/300337355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94e8003a00 0x7f94e8003e10 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.289+0000 7f94fec82700 1 -- 172.21.15.28:0/300337355 >> 172.21.15.28:0/300337355 conn(0x7f94f80b1650 msgr2=0x7f94f80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:56.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.289+0000 7f94fec82700 1 -- 172.21.15.28:0/300337355 shutdown_connections 2023-11-28T11:09:56.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.289+0000 7f94fec82700 1 -- 172.21.15.28:0/300337355 wait complete. 2023-11-28T11:09:56.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.290+0000 7f94fec82700 1 Processor -- start 2023-11-28T11:09:56.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.290+0000 7f94fec82700 1 -- start start 2023-11-28T11:09:56.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fec82700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94e8003a00 0x7f94f815ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fec82700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94f80ac060 0x7f94f8158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fec82700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94f8159220 0x7f94f814cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fec82700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f80cd670 con 0x7f94e8003a00 2023-11-28T11:09:56.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fec82700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94f815b390 con 0x7f94f8159220 2023-11-28T11:09:56.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fec82700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94f81596c0 con 0x7f94f80ac060 2023-11-28T11:09:56.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fca1e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94e8003a00 0x7f94f815ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fca1e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94e8003a00 0x7f94f815ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:48988/0 (socket says 172.21.15.28:48988) 2023-11-28T11:09:56.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fca1e700 1 -- 172.21.15.28:0/3971757001 learned_addr learned my addr 172.21.15.28:0/3971757001 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:56.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94f7fff700 1 --2- 172.21.15.28:0/3971757001 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94f80ac060 0x7f94f8158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fd21f700 1 --2- 172.21.15.28:0/3971757001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94f8159220 0x7f94f814cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fca1e700 1 -- 172.21.15.28:0/3971757001 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94f80ac060 msgr2=0x7f94f8158ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fca1e700 1 --2- 172.21.15.28:0/3971757001 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94f80ac060 0x7f94f8158ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fca1e700 1 -- 172.21.15.28:0/3971757001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94f8159220 msgr2=0x7f94f814cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fca1e700 1 --2- 172.21.15.28:0/3971757001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94f8159220 0x7f94f814cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fca1e700 1 -- 172.21.15.28:0/3971757001 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f80b2950 con 0x7f94e8003a00 2023-11-28T11:09:56.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.291+0000 7f94fca1e700 1 --2- 172.21.15.28:0/3971757001 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94e8003a00 0x7f94f815ae50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f94e800c050 tx=0x7f94ec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.292+0000 7f94f57fa700 1 -- 172.21.15.28:0/3971757001 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec00e6c0 con 0x7f94e8003a00 2023-11-28T11:09:56.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.292+0000 7f94f57fa700 1 -- 172.21.15.28:0/3971757001 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94ec003ad0 con 0x7f94e8003a00 2023-11-28T11:09:56.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.292+0000 7f94fec82700 1 -- 172.21.15.28:0/3971757001 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94f80cabe0 con 0x7f94e8003a00 2023-11-28T11:09:56.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.292+0000 7f94f57fa700 1 -- 172.21.15.28:0/3971757001 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94ec003e80 con 0x7f94e8003a00 2023-11-28T11:09:56.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.292+0000 7f94fec82700 1 -- 172.21.15.28:0/3971757001 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94f80b2ae0 con 0x7f94e8003a00 2023-11-28T11:09:56.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.292+0000 7f94fec82700 1 -- 172.21.15.28:0/3971757001 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94f80ce1f0 con 0x7f94e8003a00 2023-11-28T11:09:56.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.293+0000 7f94f57fa700 1 -- 172.21.15.28:0/3971757001 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f94ec025070 con 0x7f94e8003a00 2023-11-28T11:09:56.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.293+0000 7f94f57fa700 1 --2- 172.21.15.28:0/3971757001 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94dc0682f0 0x7f94dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.294+0000 7f94f57fa700 1 -- 172.21.15.28:0/3971757001 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f94ec087850 con 0x7f94e8003a00 2023-11-28T11:09:56.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.294+0000 7f94f7fff700 1 --2- 172.21.15.28:0/3971757001 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94dc0682f0 0x7f94dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.294+0000 7f94f7fff700 1 --2- 172.21.15.28:0/3971757001 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94dc0682f0 0x7f94dc06a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f94f80ae4f0 tx=0x7f94e4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.298+0000 7f94f57fa700 1 -- 172.21.15.28:0/3971757001 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f94ec053d80 con 0x7f94e8003a00 2023-11-28T11:09:56.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.495+0000 7f94fec82700 1 -- 172.21.15.28:0/3971757001 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f94f80675a0 con 0x7f94e8003a00 2023-11-28T11:09:56.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.496+0000 7f94f57fa700 1 -- 172.21.15.28:0/3971757001 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19737 (secure 0 0 0) 0x7f94ec0573a0 con 0x7f94e8003a00 2023-11-28T11:09:56.498 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:56.498 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:09:34.614188+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:56.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.498+0000 7f94fec82700 1 -- 172.21.15.28:0/3971757001 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94dc0682f0 msgr2=0x7f94dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.498+0000 7f94fec82700 1 --2- 172.21.15.28:0/3971757001 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94dc0682f0 0x7f94dc06a7a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f94f80ae4f0 tx=0x7f94e4009040 comp rx=0 tx=0).stop 2023-11-28T11:09:56.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.498+0000 7f94fec82700 1 -- 172.21.15.28:0/3971757001 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94e8003a00 msgr2=0x7f94f815ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.498+0000 7f94fec82700 1 --2- 172.21.15.28:0/3971757001 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94e8003a00 0x7f94f815ae50 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f94e800c050 tx=0x7f94ec0047c0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.498+0000 7f94fec82700 1 -- 172.21.15.28:0/3971757001 shutdown_connections 2023-11-28T11:09:56.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.498+0000 7f94fec82700 1 --2- 172.21.15.28:0/3971757001 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94dc0682f0 0x7f94dc06a7a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.498+0000 7f94fec82700 1 --2- 172.21.15.28:0/3971757001 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94e8003a00 0x7f94f815ae50 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.498+0000 7f94fec82700 1 --2- 172.21.15.28:0/3971757001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94f8159220 0x7f94f814cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.498+0000 7f94fec82700 1 --2- 172.21.15.28:0/3971757001 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94f80ac060 0x7f94f8158ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.498+0000 7f94fec82700 1 -- 172.21.15.28:0/3971757001 >> 172.21.15.28:0/3971757001 conn(0x7f94f80b1650 msgr2=0x7f94f80cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:56.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.499+0000 7f94fec82700 1 -- 172.21.15.28:0/3971757001 shutdown_connections 2023-11-28T11:09:56.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.499+0000 7f94fec82700 1 -- 172.21.15.28:0/3971757001 wait complete. 2023-11-28T11:09:56.513 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:09:56.513 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:56.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.635+0000 7fbbec0bb700 1 Processor -- start 2023-11-28T11:09:56.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.636+0000 7fbbec0bb700 1 -- start start 2023-11-28T11:09:56.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.636+0000 7fbbec0bb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbbe4161090 0x7fbbe40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.636+0000 7fbbec0bb700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbbe40caeb0 0x7fbbe40cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.636+0000 7fbbec0bb700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe40ce340 con 0x7fbbe415c990 2023-11-28T11:09:56.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.636+0000 7fbbec0bb700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe40cebe0 con 0x7fbbe40caeb0 2023-11-28T11:09:56.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.636+0000 7fbbec0bb700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe40cf780 con 0x7fbbe4161090 2023-11-28T11:09:56.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.637+0000 7fbbea658700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbbe40caeb0 0x7fbbe40cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.637+0000 7fbbe9e57700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbbe415c990 0x7fbbe40cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37054/0 (socket says 172.21.15.28:37054) 2023-11-28T11:09:56.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.637+0000 7fbbe9e57700 1 -- 172.21.15.28:0/4250309098 learned_addr learned my addr 172.21.15.28:0/4250309098 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:56.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.637+0000 7fbbea658700 1 -- 172.21.15.28:0/4250309098 >> v2:172.21.15.59:3300/0 conn(0x7fbbe4161090 msgr2=0x7fbbe40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:56.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.637+0000 7fbbea658700 1 --2- 172.21.15.28:0/4250309098 >> v2:172.21.15.59:3300/0 conn(0x7fbbe4161090 0x7fbbe40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.637+0000 7fbbea658700 1 -- 172.21.15.28:0/4250309098 >> v1:172.21.15.28:6789/0 conn(0x7fbbe415c990 legacy=0x7fbbe40cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.637+0000 7fbbea658700 1 -- 172.21.15.28:0/4250309098 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe40cb3f0 con 0x7fbbe40caeb0 2023-11-28T11:09:56.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.637+0000 7fbbe8e55700 1 -- 172.21.15.28:0/4250309098 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4135836308 0 0) 0x7fbbe40ce340 con 0x7fbbe415c990 2023-11-28T11:09:56.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.637+0000 7fbbea658700 1 --2- 172.21.15.28:0/4250309098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbe40caeb0 0x7fbbe40cc310 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbbe406ab20 tx=0x7fbbdc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=698704e5188a4daf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.637+0000 7fbbe8e55700 1 -- 172.21.15.28:0/4250309098 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc00e070 con 0x7fbbe40caeb0 2023-11-28T11:09:56.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.637+0000 7fbbe8e55700 1 -- 172.21.15.28:0/4250309098 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbdc00bbd0 con 0x7fbbe40caeb0 2023-11-28T11:09:56.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.637+0000 7fbbe8e55700 1 -- 172.21.15.28:0/4250309098 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc012430 con 0x7fbbe40caeb0 2023-11-28T11:09:56.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.638+0000 7fbbec0bb700 1 -- 172.21.15.28:0/4250309098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbe40caeb0 msgr2=0x7fbbe40cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.638+0000 7fbbec0bb700 1 --2- 172.21.15.28:0/4250309098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbe40caeb0 0x7fbbe40cc310 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fbbe406ab20 tx=0x7fbbdc00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:56.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.638+0000 7fbbec0bb700 1 -- 172.21.15.28:0/4250309098 shutdown_connections 2023-11-28T11:09:56.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.638+0000 7fbbec0bb700 1 --2- 172.21.15.28:0/4250309098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbe40caeb0 0x7fbbe40cc310 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.638+0000 7fbbec0bb700 1 --2- 172.21.15.28:0/4250309098 >> v2:172.21.15.59:3300/0 conn(0x7fbbe4161090 0x7fbbe40ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.638+0000 7fbbec0bb700 1 -- 172.21.15.28:0/4250309098 >> 172.21.15.28:0/4250309098 conn(0x7fbbe40b1650 msgr2=0x7fbbe40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:56.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.638+0000 7fbbec0bb700 1 -- 172.21.15.28:0/4250309098 shutdown_connections 2023-11-28T11:09:56.645 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.638+0000 7fbbec0bb700 1 -- 172.21.15.28:0/4250309098 wait complete. 2023-11-28T11:09:56.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.639+0000 7fbbec0bb700 1 Processor -- start 2023-11-28T11:09:56.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.639+0000 7fbbec0bb700 1 -- start start 2023-11-28T11:09:56.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbec0bb700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbe40caeb0 0x7fbbe414e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbec0bb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbe415c990 0x7fbbe415efb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbec0bb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbbe4161090 0x7fbbe415f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbec0bb700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe40b4220 con 0x7fbbe40caeb0 2023-11-28T11:09:56.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbec0bb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe40b5820 con 0x7fbbe415c990 2023-11-28T11:09:56.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbec0bb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe40b35a0 con 0x7fbbe4161090 2023-11-28T11:09:56.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbe9e57700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbe40caeb0 0x7fbbe414e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbe9e57700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbe40caeb0 0x7fbbe414e020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49002/0 (socket says 172.21.15.28:49002) 2023-11-28T11:09:56.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbe9e57700 1 -- 172.21.15.28:0/2862780188 learned_addr learned my addr 172.21.15.28:0/2862780188 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:56.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbea658700 1 --2- 172.21.15.28:0/2862780188 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbbe4161090 0x7fbbe415f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbe9656700 1 --2- 172.21.15.28:0/2862780188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbe415c990 0x7fbbe415efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbe9e57700 1 -- 172.21.15.28:0/2862780188 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbbe4161090 msgr2=0x7fbbe415f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbe9e57700 1 --2- 172.21.15.28:0/2862780188 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbbe4161090 0x7fbbe415f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbe9e57700 1 -- 172.21.15.28:0/2862780188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbe415c990 msgr2=0x7fbbe415efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbe9e57700 1 --2- 172.21.15.28:0/2862780188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbe415c990 0x7fbbe415efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbe9e57700 1 -- 172.21.15.28:0/2862780188 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc009a70 con 0x7fbbe40caeb0 2023-11-28T11:09:56.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbe9656700 1 --2- 172.21.15.28:0/2862780188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbe415c990 0x7fbbe415efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:56.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbe9e57700 1 --2- 172.21.15.28:0/2862780188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbe40caeb0 0x7fbbe414e020 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00011e0 tx=0x7fbbe000cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbd67fc700 1 -- 172.21.15.28:0/2862780188 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0004c80 con 0x7fbbe40caeb0 2023-11-28T11:09:56.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbd67fc700 1 -- 172.21.15.28:0/2862780188 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbe0004e00 con 0x7fbbe40caeb0 2023-11-28T11:09:56.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbec0bb700 1 -- 172.21.15.28:0/2862780188 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe40ce870 con 0x7fbbe40caeb0 2023-11-28T11:09:56.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.640+0000 7fbbd67fc700 1 -- 172.21.15.28:0/2862780188 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0007310 con 0x7fbbe40caeb0 2023-11-28T11:09:56.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.641+0000 7fbbec0bb700 1 -- 172.21.15.28:0/2862780188 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe40ce590 con 0x7fbbe40caeb0 2023-11-28T11:09:56.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.641+0000 7fbbec0bb700 1 -- 172.21.15.28:0/2862780188 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe40cc850 con 0x7fbbe40caeb0 2023-11-28T11:09:56.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.642+0000 7fbbd67fc700 1 -- 172.21.15.28:0/2862780188 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbbe001a020 con 0x7fbbe40caeb0 2023-11-28T11:09:56.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.642+0000 7fbbd67fc700 1 --2- 172.21.15.28:0/2862780188 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbbc80682f0 0x7fbbc806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:56.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.642+0000 7fbbd67fc700 1 -- 172.21.15.28:0/2862780188 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fbbe0085c90 con 0x7fbbe40caeb0 2023-11-28T11:09:56.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.643+0000 7fbbe9656700 1 --2- 172.21.15.28:0/2862780188 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbbc80682f0 0x7fbbc806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:56.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.643+0000 7fbbe9656700 1 --2- 172.21.15.28:0/2862780188 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbbc80682f0 0x7fbbc806a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbbe415cfe0 tx=0x7fbbd8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:56.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.647+0000 7fbbd67fc700 1 -- 172.21.15.28:0/2862780188 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbbe00521c0 con 0x7fbbe40caeb0 2023-11-28T11:09:56.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.884+0000 7fbbec0bb700 1 -- 172.21.15.28:0/2862780188 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbbe40675a0 con 0x7fbbe40caeb0 2023-11-28T11:09:56.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.885+0000 7fbbd67fc700 1 -- 172.21.15.28:0/2862780188 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4644 (secure 0 0 0) 0x7fbbe00557e0 con 0x7fbbe40caeb0 2023-11-28T11:09:56.886 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:56.887 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/3112877467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3112877467},{"type":"v1","addr":"172.21.15.59:6834","nonce":3112877467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3448236489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3448236489},{"type":"v1","addr":"172.21.15.28:6836","nonce":3448236489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4232269940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4232269940},{"type":"v1","addr":"172.21.15.28:6837","nonce":4232269940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:45.924816+0000","modified":"2023-11-28T11:09:03.234785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/3293053261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3293053261},{"type":"v1","addr":"172.21.15.59:6835","nonce":3293053261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:09:56.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.886+0000 7fbbec0bb700 1 -- 172.21.15.28:0/2862780188 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbbc80682f0 msgr2=0x7fbbc806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.886+0000 7fbbec0bb700 1 --2- 172.21.15.28:0/2862780188 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbbc80682f0 0x7fbbc806a7a0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fbbe415cfe0 tx=0x7fbbd8008040 comp rx=0 tx=0).stop 2023-11-28T11:09:56.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.886+0000 7fbbec0bb700 1 -- 172.21.15.28:0/2862780188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbe40caeb0 msgr2=0x7fbbe414e020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:56.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.886+0000 7fbbec0bb700 1 --2- 172.21.15.28:0/2862780188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbe40caeb0 0x7fbbe414e020 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fbbe00011e0 tx=0x7fbbe000cae0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.886+0000 7fbbec0bb700 1 -- 172.21.15.28:0/2862780188 shutdown_connections 2023-11-28T11:09:56.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.886+0000 7fbbec0bb700 1 --2- 172.21.15.28:0/2862780188 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbbc80682f0 0x7fbbc806a7a0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.886+0000 7fbbec0bb700 1 --2- 172.21.15.28:0/2862780188 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbbe40caeb0 0x7fbbe414e020 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.886+0000 7fbbec0bb700 1 --2- 172.21.15.28:0/2862780188 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbbe4161090 0x7fbbe415f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.886+0000 7fbbec0bb700 1 --2- 172.21.15.28:0/2862780188 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbbe415c990 0x7fbbe415efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:56.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.886+0000 7fbbec0bb700 1 -- 172.21.15.28:0/2862780188 >> 172.21.15.28:0/2862780188 conn(0x7fbbe40b1650 msgr2=0x7fbbe40ccc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:56.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.887+0000 7fbbec0bb700 1 -- 172.21.15.28:0/2862780188 shutdown_connections 2023-11-28T11:09:56.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:56.887+0000 7fbbec0bb700 1 -- 172.21.15.28:0/2862780188 wait complete. 2023-11-28T11:09:56.890 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 9 2023-11-28T11:09:56.900 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:09:57.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.023+0000 7f22aa1d6700 1 Processor -- start 2023-11-28T11:09:57.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.024+0000 7f22aa1d6700 1 -- start start 2023-11-28T11:09:57.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.024+0000 7f22aa1d6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22a415c990 0x7f22a40cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.024+0000 7f22aa1d6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f22a4161090 0x7f22a40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.024+0000 7f22aa1d6700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f22a40caeb0 0x7f22a40cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.024+0000 7f22aa1d6700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a40ce340 con 0x7f22a40caeb0 2023-11-28T11:09:57.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.024+0000 7f22aa1d6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a40cebe0 con 0x7f22a4161090 2023-11-28T11:09:57.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.024+0000 7f22aa1d6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40cb3f0 con 0x7f22a415c990 2023-11-28T11:09:57.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.024+0000 7f22a3fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f22a40caeb0 0x7f22a40cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.025+0000 7f22a3fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f22a40caeb0 0x7f22a40cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49014/0 (socket says 172.21.15.28:49014) 2023-11-28T11:09:57.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.025+0000 7f22a3fff700 1 -- 172.21.15.28:0/3075768151 learned_addr learned my addr 172.21.15.28:0/3075768151 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:57.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.025+0000 7f22a2ffd700 1 --2- 172.21.15.28:0/3075768151 >> v2:172.21.15.59:3300/0 conn(0x7f22a4161090 0x7f22a40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.025+0000 7f22a37fe700 1 --2- 172.21.15.28:0/3075768151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22a415c990 0x7f22a40cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.025+0000 7f22a3fff700 1 -- 172.21.15.28:0/3075768151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22a415c990 msgr2=0x7f22a40cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:57.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.025+0000 7f22a3fff700 1 --2- 172.21.15.28:0/3075768151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22a415c990 0x7f22a40cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.025+0000 7f22a3fff700 1 -- 172.21.15.28:0/3075768151 >> v2:172.21.15.59:3300/0 conn(0x7f22a4161090 msgr2=0x7f22a40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:57.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.025+0000 7f22a3fff700 1 --2- 172.21.15.28:0/3075768151 >> v2:172.21.15.59:3300/0 conn(0x7f22a4161090 0x7f22a40ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.025+0000 7f22a3fff700 1 -- 172.21.15.28:0/3075768151 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a40ce590 con 0x7f22a40caeb0 2023-11-28T11:09:57.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.025+0000 7f22a2ffd700 1 --2- 172.21.15.28:0/3075768151 >> v2:172.21.15.59:3300/0 conn(0x7f22a4161090 0x7f22a40ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:57.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.025+0000 7f22a37fe700 1 --2- 172.21.15.28:0/3075768151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22a415c990 0x7f22a40cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:57.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.025+0000 7f22a3fff700 1 --2- 172.21.15.28:0/3075768151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22a40caeb0 0x7f22a40cc310 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f22a406f850 tx=0x7f229c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed1796099ab20e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.025+0000 7f22a27fc700 1 -- 172.21.15.28:0/3075768151 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c00e070 con 0x7f22a40caeb0 2023-11-28T11:09:57.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.025+0000 7f22a27fc700 1 -- 172.21.15.28:0/3075768151 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229c00b9c0 con 0x7f22a40caeb0 2023-11-28T11:09:57.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.025+0000 7f22a27fc700 1 -- 172.21.15.28:0/3075768151 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229c00bd70 con 0x7f22a40caeb0 2023-11-28T11:09:57.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.026+0000 7f22aa1d6700 1 -- 172.21.15.28:0/3075768151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22a40caeb0 msgr2=0x7f22a40cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.030 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.026+0000 7f22aa1d6700 1 --2- 172.21.15.28:0/3075768151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22a40caeb0 0x7f22a40cc310 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f22a406f850 tx=0x7f229c00b330 comp rx=0 tx=0).stop 2023-11-28T11:09:57.030 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.026+0000 7f22aa1d6700 1 -- 172.21.15.28:0/3075768151 shutdown_connections 2023-11-28T11:09:57.030 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.026+0000 7f22aa1d6700 1 --2- 172.21.15.28:0/3075768151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22a40caeb0 0x7f22a40cc310 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.030 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.026+0000 7f22aa1d6700 1 --2- 172.21.15.28:0/3075768151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22a415c990 0x7f22a40cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.030 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.026+0000 7f22aa1d6700 1 --2- 172.21.15.28:0/3075768151 >> v2:172.21.15.59:3300/0 conn(0x7f22a4161090 0x7f22a40ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.030 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.026+0000 7f22aa1d6700 1 -- 172.21.15.28:0/3075768151 >> 172.21.15.28:0/3075768151 conn(0x7f22a40b1650 msgr2=0x7f22a40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:57.030 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.026+0000 7f22aa1d6700 1 -- 172.21.15.28:0/3075768151 shutdown_connections 2023-11-28T11:09:57.030 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.026+0000 7f22aa1d6700 1 -- 172.21.15.28:0/3075768151 wait complete. 2023-11-28T11:09:57.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.027+0000 7f22aa1d6700 1 Processor -- start 2023-11-28T11:09:57.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.027+0000 7f22aa1d6700 1 -- start start 2023-11-28T11:09:57.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22aa1d6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22a40caeb0 0x7f22a415d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22aa1d6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22a415c990 0x7f22a415a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22aa1d6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a4161090 0x7f22a415ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22aa1d6700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40c48c0 con 0x7f22a415c990 2023-11-28T11:09:57.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22aa1d6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40c4ba0 con 0x7f22a4161090 2023-11-28T11:09:57.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22aa1d6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a40c4e80 con 0x7f22a40caeb0 2023-11-28T11:09:57.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22a2ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22a415c990 0x7f22a415a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22a2ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22a415c990 0x7f22a415a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49018/0 (socket says 172.21.15.28:49018) 2023-11-28T11:09:57.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22a2ffd700 1 -- 172.21.15.28:0/2576861575 learned_addr learned my addr 172.21.15.28:0/2576861575 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:57.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22a3fff700 1 --2- 172.21.15.28:0/2576861575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a4161090 0x7f22a415ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22a37fe700 1 --2- 172.21.15.28:0/2576861575 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22a40caeb0 0x7f22a415d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22a2ffd700 1 -- 172.21.15.28:0/2576861575 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22a40caeb0 msgr2=0x7f22a415d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22a2ffd700 1 --2- 172.21.15.28:0/2576861575 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22a40caeb0 0x7f22a415d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22a2ffd700 1 -- 172.21.15.28:0/2576861575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a4161090 msgr2=0x7f22a415ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22a2ffd700 1 --2- 172.21.15.28:0/2576861575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a4161090 0x7f22a415ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22a2ffd700 1 -- 172.21.15.28:0/2576861575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f229c009b30 con 0x7f22a415c990 2023-11-28T11:09:57.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.028+0000 7f22a2ffd700 1 --2- 172.21.15.28:0/2576861575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22a415c990 0x7f22a415a650 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f228c00c7f0 tx=0x7f228c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.029+0000 7f229bfff700 1 -- 172.21.15.28:0/2576861575 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c011800 con 0x7f22a415c990 2023-11-28T11:09:57.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.029+0000 7f229bfff700 1 -- 172.21.15.28:0/2576861575 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f228c011960 con 0x7f22a415c990 2023-11-28T11:09:57.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.029+0000 7f22aa1d6700 1 -- 172.21.15.28:0/2576861575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a40cf5b0 con 0x7f22a415c990 2023-11-28T11:09:57.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.029+0000 7f229bfff700 1 -- 172.21.15.28:0/2576861575 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f228c011d10 con 0x7f22a415c990 2023-11-28T11:09:57.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.029+0000 7f22aa1d6700 1 -- 172.21.15.28:0/2576861575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a40ce870 con 0x7f22a415c990 2023-11-28T11:09:57.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.030+0000 7f22aa1d6700 1 -- 172.21.15.28:0/2576861575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a40cf740 con 0x7f22a415c990 2023-11-28T11:09:57.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.030+0000 7f229bfff700 1 -- 172.21.15.28:0/2576861575 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f228c021070 con 0x7f22a415c990 2023-11-28T11:09:57.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.031+0000 7f229bfff700 1 --2- 172.21.15.28:0/2576861575 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f22840682f0 0x7f228406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.031+0000 7f229bfff700 1 -- 172.21.15.28:0/2576861575 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f228c085d70 con 0x7f22a415c990 2023-11-28T11:09:57.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.031+0000 7f22a37fe700 1 --2- 172.21.15.28:0/2576861575 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f22840682f0 0x7f228406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.031+0000 7f22a37fe700 1 --2- 172.21.15.28:0/2576861575 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f22840682f0 0x7f228406a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2294006ba0 tx=0x7f2294008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.035+0000 7f229bfff700 1 -- 172.21.15.28:0/2576861575 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f228c0521b0 con 0x7f22a415c990 2023-11-28T11:09:57.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.276+0000 7f22aa1d6700 1 -- 172.21.15.28:0/2576861575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f22a40c54d0 con 0x7f22a415c990 2023-11-28T11:09:57.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.277+0000 7f229bfff700 1 -- 172.21.15.28:0/2576861575 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+4644 (secure 0 0 0) 0x7f228c0557d0 con 0x7f22a415c990 2023-11-28T11:09:57.278 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:57.279 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":9,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4255,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/3112877467","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3112877467},{"type":"v1","addr":"172.21.15.59:6834","nonce":3112877467}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4287,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3448236489","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3448236489},{"type":"v1","addr":"172.21.15.28:6836","nonce":3448236489}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4290,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/4232269940","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":4232269940},{"type":"v1","addr":"172.21.15.28:6837","nonce":4232269940}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":9,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:08:45.924816+0000","modified":"2023-11-28T11:09:03.234785+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4296},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4296":{"gid":4296,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6835/3293053261","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":3293053261},{"type":"v1","addr":"172.21.15.59:6835","nonce":3293053261}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4,5],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-11-28T11:09:57.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.279+0000 7f22aa1d6700 1 -- 172.21.15.28:0/2576861575 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f22840682f0 msgr2=0x7f228406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.279+0000 7f22aa1d6700 1 --2- 172.21.15.28:0/2576861575 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f22840682f0 0x7f228406a7a0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2294006ba0 tx=0x7f2294008040 comp rx=0 tx=0).stop 2023-11-28T11:09:57.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.279+0000 7f22aa1d6700 1 -- 172.21.15.28:0/2576861575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22a415c990 msgr2=0x7f22a415a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.279+0000 7f22aa1d6700 1 --2- 172.21.15.28:0/2576861575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22a415c990 0x7f22a415a650 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f228c00c7f0 tx=0x7f228c00cb00 comp rx=0 tx=0).stop 2023-11-28T11:09:57.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.279+0000 7f22aa1d6700 1 -- 172.21.15.28:0/2576861575 shutdown_connections 2023-11-28T11:09:57.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.279+0000 7f22aa1d6700 1 --2- 172.21.15.28:0/2576861575 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f22840682f0 0x7f228406a7a0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.279+0000 7f22aa1d6700 1 --2- 172.21.15.28:0/2576861575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22a4161090 0x7f22a415ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.279+0000 7f22aa1d6700 1 --2- 172.21.15.28:0/2576861575 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22a40caeb0 0x7f22a415d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.279+0000 7f22aa1d6700 1 --2- 172.21.15.28:0/2576861575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22a415c990 0x7f22a415a650 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.279+0000 7f22aa1d6700 1 -- 172.21.15.28:0/2576861575 >> 172.21.15.28:0/2576861575 conn(0x7f22a40b1650 msgr2=0x7f22a40cce90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:57.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.279+0000 7f22aa1d6700 1 -- 172.21.15.28:0/2576861575 shutdown_connections 2023-11-28T11:09:57.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.279+0000 7f22aa1d6700 1 -- 172.21.15.28:0/2576861575 wait complete. 2023-11-28T11:09:57.282 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 9 2023-11-28T11:09:57.293 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:09:57.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.417+0000 7f905c688700 1 Processor -- start 2023-11-28T11:09:57.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.417+0000 7f905c688700 1 -- start start 2023-11-28T11:09:57.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905c688700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f905415a6c0 0x7f90540cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905c688700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90540d0020 0x7f90540ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905c688700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f90540b2660 0x7f90540b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905c688700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90540b3020 con 0x7f905415a6c0 2023-11-28T11:09:57.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905c688700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90540cfb90 con 0x7f90540b2660 2023-11-28T11:09:57.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905c688700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90540cb930 con 0x7f90540d0020 2023-11-28T11:09:57.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905a424700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f905415a6c0 0x7f90540cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905a424700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f905415a6c0 0x7f90540cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49024/0 (socket says 172.21.15.28:49024) 2023-11-28T11:09:57.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905a424700 1 -- 172.21.15.28:0/2170279751 learned_addr learned my addr 172.21.15.28:0/2170279751 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:57.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905ac25700 1 --2- 172.21.15.28:0/2170279751 >> v2:172.21.15.59:3300/0 conn(0x7f90540b2660 0x7f90540b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f9059c23700 1 --2- 172.21.15.28:0/2170279751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90540d0020 0x7f90540ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905a424700 1 -- 172.21.15.28:0/2170279751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90540d0020 msgr2=0x7f90540ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:57.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905a424700 1 --2- 172.21.15.28:0/2170279751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90540d0020 0x7f90540ccea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905a424700 1 -- 172.21.15.28:0/2170279751 >> v2:172.21.15.59:3300/0 conn(0x7f90540b2660 msgr2=0x7f90540b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:57.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905a424700 1 --2- 172.21.15.28:0/2170279751 >> v2:172.21.15.59:3300/0 conn(0x7f90540b2660 0x7f90540b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905a424700 1 -- 172.21.15.28:0/2170279751 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90540cf260 con 0x7f905415a6c0 2023-11-28T11:09:57.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f905a424700 1 --2- 172.21.15.28:0/2170279751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f905415a6c0 0x7f90540cb230 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9054072920 tx=0x7f9050009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc29d3d5cb808c0f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f9059422700 1 -- 172.21.15.28:0/2170279751 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050011830 con 0x7f905415a6c0 2023-11-28T11:09:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f9059422700 1 -- 172.21.15.28:0/2170279751 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9050011990 con 0x7f905415a6c0 2023-11-28T11:09:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.418+0000 7f9059422700 1 -- 172.21.15.28:0/2170279751 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9050011d20 con 0x7f905415a6c0 2023-11-28T11:09:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.419+0000 7f905c688700 1 -- 172.21.15.28:0/2170279751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f905415a6c0 msgr2=0x7f90540cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.419+0000 7f905c688700 1 --2- 172.21.15.28:0/2170279751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f905415a6c0 0x7f90540cb230 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f9054072920 tx=0x7f9050009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.419+0000 7f905c688700 1 -- 172.21.15.28:0/2170279751 shutdown_connections 2023-11-28T11:09:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.419+0000 7f905c688700 1 --2- 172.21.15.28:0/2170279751 >> v2:172.21.15.59:3300/0 conn(0x7f90540b2660 0x7f90540b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.419+0000 7f905c688700 1 --2- 172.21.15.28:0/2170279751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f905415a6c0 0x7f90540cb230 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.419+0000 7f905c688700 1 --2- 172.21.15.28:0/2170279751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f90540d0020 0x7f90540ccea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.419+0000 7f905c688700 1 -- 172.21.15.28:0/2170279751 >> 172.21.15.28:0/2170279751 conn(0x7f90540cd660 msgr2=0x7f90540cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:57.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.419+0000 7f905c688700 1 -- 172.21.15.28:0/2170279751 shutdown_connections 2023-11-28T11:09:57.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.419+0000 7f905c688700 1 -- 172.21.15.28:0/2170279751 wait complete. 2023-11-28T11:09:57.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.420+0000 7f905c688700 1 Processor -- start 2023-11-28T11:09:57.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.420+0000 7f905c688700 1 -- start start 2023-11-28T11:09:57.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.421+0000 7f905c688700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90540b2660 0x7f905415cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.421+0000 7f905c688700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90540d0020 0x7f90541f4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.421+0000 7f905c688700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f905415a6c0 0x7f9054157d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.421+0000 7f905c688700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905406cb20 con 0x7f90540d0020 2023-11-28T11:09:57.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.421+0000 7f905c688700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905415ca80 con 0x7f90540b2660 2023-11-28T11:09:57.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.421+0000 7f905c688700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f905406c840 con 0x7f905415a6c0 2023-11-28T11:09:57.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.421+0000 7f9059c23700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90540d0020 0x7f90541f4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.421+0000 7f9059c23700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90540d0020 0x7f90541f4c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49030/0 (socket says 172.21.15.28:49030) 2023-11-28T11:09:57.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.421+0000 7f9059c23700 1 -- 172.21.15.28:0/4191154221 learned_addr learned my addr 172.21.15.28:0/4191154221 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:57.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.422+0000 7f9059c23700 1 -- 172.21.15.28:0/4191154221 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f905415a6c0 msgr2=0x7f9054157d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.422+0000 7f905ac25700 1 --2- 172.21.15.28:0/4191154221 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f905415a6c0 0x7f9054157d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.422+0000 7f9059c23700 1 --2- 172.21.15.28:0/4191154221 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f905415a6c0 0x7f9054157d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.422+0000 7f905a424700 1 --2- 172.21.15.28:0/4191154221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90540b2660 0x7f905415cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.422+0000 7f9059c23700 1 -- 172.21.15.28:0/4191154221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90540b2660 msgr2=0x7f905415cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.422+0000 7f9059c23700 1 --2- 172.21.15.28:0/4191154221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90540b2660 0x7f905415cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.422+0000 7f9059c23700 1 -- 172.21.15.28:0/4191154221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9050009550 con 0x7f90540d0020 2023-11-28T11:09:57.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.422+0000 7f9059c23700 1 --2- 172.21.15.28:0/4191154221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90540d0020 0x7f90541f4c90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f904400e9b0 tx=0x7f904400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.422+0000 7f904affd700 1 -- 172.21.15.28:0/4191154221 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9044009710 con 0x7f90540d0020 2023-11-28T11:09:57.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.422+0000 7f904affd700 1 -- 172.21.15.28:0/4191154221 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f904400c850 con 0x7f90540d0020 2023-11-28T11:09:57.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.422+0000 7f904affd700 1 -- 172.21.15.28:0/4191154221 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904400cbc0 con 0x7f90540d0020 2023-11-28T11:09:57.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.422+0000 7f905c688700 1 -- 172.21.15.28:0/4191154221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90540cf820 con 0x7f90540d0020 2023-11-28T11:09:57.428 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.422+0000 7f905c688700 1 -- 172.21.15.28:0/4191154221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90540cf540 con 0x7f90540d0020 2023-11-28T11:09:57.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.423+0000 7f905c688700 1 -- 172.21.15.28:0/4191154221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90540cd3e0 con 0x7f90540d0020 2023-11-28T11:09:57.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.424+0000 7f904affd700 1 -- 172.21.15.28:0/4191154221 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9044019030 con 0x7f90540d0020 2023-11-28T11:09:57.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.424+0000 7f904affd700 1 --2- 172.21.15.28:0/4191154221 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f90380682f0 0x7f903806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.424+0000 7f904affd700 1 -- 172.21.15.28:0/4191154221 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7f9044086420 con 0x7f90540d0020 2023-11-28T11:09:57.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.424+0000 7f905a424700 1 --2- 172.21.15.28:0/4191154221 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f90380682f0 0x7f903806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.425+0000 7f905a424700 1 --2- 172.21.15.28:0/4191154221 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f90380682f0 0x7f903806a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9050009c10 tx=0x7f9050008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.428+0000 7f904affd700 1 -- 172.21.15.28:0/4191154221 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f90440528e0 con 0x7f90540d0020 2023-11-28T11:09:57.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.622+0000 7f905c688700 1 -- 172.21.15.28:0/4191154221 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f905415c290 con 0x7f90540d0020 2023-11-28T11:09:57.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.623+0000 7f904affd700 1 -- 172.21.15.28:0/4191154221 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) v1 ==== 74+0+19737 (secure 0 0 0) 0x7f905415c290 con 0x7f90540d0020 2023-11-28T11:09:57.625 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:09:57.625 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:09:34.614188+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:08:43.661652+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T11:08:44.889002+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data_ec","create_time":"2023-11-28T11:08:48.125959+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"4.0","mappings":[{"from":3,"to":2}]},{"pgid":"4.31","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:09:57.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.625+0000 7f905c688700 1 -- 172.21.15.28:0/4191154221 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f90380682f0 msgr2=0x7f903806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.625+0000 7f905c688700 1 --2- 172.21.15.28:0/4191154221 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f90380682f0 0x7f903806a7a0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f9050009c10 tx=0x7f9050008030 comp rx=0 tx=0).stop 2023-11-28T11:09:57.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.625+0000 7f905c688700 1 -- 172.21.15.28:0/4191154221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90540d0020 msgr2=0x7f90541f4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.625+0000 7f905c688700 1 --2- 172.21.15.28:0/4191154221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90540d0020 0x7f90541f4c90 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f904400e9b0 tx=0x7f904400ed70 comp rx=0 tx=0).stop 2023-11-28T11:09:57.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.625+0000 7f905c688700 1 -- 172.21.15.28:0/4191154221 shutdown_connections 2023-11-28T11:09:57.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.625+0000 7f905c688700 1 --2- 172.21.15.28:0/4191154221 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f90380682f0 0x7f903806a7a0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.625+0000 7f905c688700 1 --2- 172.21.15.28:0/4191154221 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f905415a6c0 0x7f9054157d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.625+0000 7f905c688700 1 --2- 172.21.15.28:0/4191154221 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90540b2660 0x7f905415cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.626+0000 7f905c688700 1 --2- 172.21.15.28:0/4191154221 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f90540d0020 0x7f90541f4c90 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.626+0000 7f905c688700 1 -- 172.21.15.28:0/4191154221 >> 172.21.15.28:0/4191154221 conn(0x7f90540cd660 msgr2=0x7f90540b0640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:57.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.626+0000 7f905c688700 1 -- 172.21.15.28:0/4191154221 shutdown_connections 2023-11-28T11:09:57.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.626+0000 7f905c688700 1 -- 172.21.15.28:0/4191154221 wait complete. 2023-11-28T11:09:57.640 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:09:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.764+0000 7fa32a559700 1 Processor -- start 2023-11-28T11:09:57.766 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.765+0000 7fa32a559700 1 -- start start 2023-11-28T11:09:57.768 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.765+0000 7fa32a559700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32415c560 0x7fa32406f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.765+0000 7fa32a559700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3240b2660 0x7fa3240ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.765+0000 7fa32a559700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa3240cecf0 0x7fa324160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.765+0000 7fa32a559700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3240b30b0 con 0x7fa3240b2660 2023-11-28T11:09:57.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.765+0000 7fa32a559700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32406c020 con 0x7fa3240cecf0 2023-11-28T11:09:57.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.765+0000 7fa32a559700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240cf110 con 0x7fa32415c560 2023-11-28T11:09:57.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.765+0000 7fa3237fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3240b2660 0x7fa3240ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.765+0000 7fa3237fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3240b2660 0x7fa3240ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49036/0 (socket says 172.21.15.28:49036) 2023-11-28T11:09:57.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.765+0000 7fa3237fe700 1 -- 172.21.15.28:0/3123967259 learned_addr learned my addr 172.21.15.28:0/3123967259 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:57.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.766+0000 7fa328af6700 1 --2- 172.21.15.28:0/3123967259 >> v2:172.21.15.59:3300/0 conn(0x7fa3240cecf0 0x7fa324160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.766+0000 7fa323fff700 1 --2- 172.21.15.28:0/3123967259 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32415c560 0x7fa32406f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.766+0000 7fa3237fe700 1 -- 172.21.15.28:0/3123967259 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32415c560 msgr2=0x7fa32406f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.766+0000 7fa3237fe700 1 --2- 172.21.15.28:0/3123967259 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32415c560 0x7fa32406f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.766+0000 7fa3237fe700 1 -- 172.21.15.28:0/3123967259 >> v2:172.21.15.59:3300/0 conn(0x7fa3240cecf0 msgr2=0x7fa324160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.766+0000 7fa3237fe700 1 --2- 172.21.15.28:0/3123967259 >> v2:172.21.15.59:3300/0 conn(0x7fa3240cecf0 0x7fa324160200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.766+0000 7fa3237fe700 1 -- 172.21.15.28:0/3123967259 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3240d09c0 con 0x7fa3240b2660 2023-11-28T11:09:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.766+0000 7fa3237fe700 1 --2- 172.21.15.28:0/3123967259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240b2660 0x7fa3240ce7b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa3240c7c40 tx=0x7fa30c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96fe5042c897e2cb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.766+0000 7fa322ffd700 1 -- 172.21.15.28:0/3123967259 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c00b870 con 0x7fa3240b2660 2023-11-28T11:09:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.766+0000 7fa322ffd700 1 -- 172.21.15.28:0/3123967259 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa30c00b9d0 con 0x7fa3240b2660 2023-11-28T11:09:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.766+0000 7fa322ffd700 1 -- 172.21.15.28:0/3123967259 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa30c00bd60 con 0x7fa3240b2660 2023-11-28T11:09:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.767+0000 7fa32a559700 1 -- 172.21.15.28:0/3123967259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240b2660 msgr2=0x7fa3240ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.767+0000 7fa32a559700 1 --2- 172.21.15.28:0/3123967259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240b2660 0x7fa3240ce7b0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fa3240c7c40 tx=0x7fa30c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.770 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.767+0000 7fa32a559700 1 -- 172.21.15.28:0/3123967259 shutdown_connections 2023-11-28T11:09:57.771 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.767+0000 7fa32a559700 1 --2- 172.21.15.28:0/3123967259 >> v2:172.21.15.59:3300/0 conn(0x7fa3240cecf0 0x7fa324160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.771 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.767+0000 7fa32a559700 1 --2- 172.21.15.28:0/3123967259 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32415c560 0x7fa32406f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.771 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.767+0000 7fa32a559700 1 --2- 172.21.15.28:0/3123967259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240b2660 0x7fa3240ce7b0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.771 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.767+0000 7fa32a559700 1 -- 172.21.15.28:0/3123967259 >> 172.21.15.28:0/3123967259 conn(0x7fa3240cd660 msgr2=0x7fa3240cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:57.771 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.767+0000 7fa32a559700 1 -- 172.21.15.28:0/3123967259 shutdown_connections 2023-11-28T11:09:57.771 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.767+0000 7fa32a559700 1 -- 172.21.15.28:0/3123967259 wait complete. 2023-11-28T11:09:57.773 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.768+0000 7fa32a559700 1 Processor -- start 2023-11-28T11:09:57.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.768+0000 7fa32a559700 1 -- start start 2023-11-28T11:09:57.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa32a559700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3240b2660 0x7fa324151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa32a559700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3240cecf0 0x7fa3240ac110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa32a559700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa32415c560 0x7fa3240af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa32a559700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3241f2e00 con 0x7fa32415c560 2023-11-28T11:09:57.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa32a559700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3241f30e0 con 0x7fa3240b2660 2023-11-28T11:09:57.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa32a559700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa32415c930 con 0x7fa3240cecf0 2023-11-28T11:09:57.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa328af6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa32415c560 0x7fa3240af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa328af6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa32415c560 0x7fa3240af040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49046/0 (socket says 172.21.15.28:49046) 2023-11-28T11:09:57.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa328af6700 1 -- 172.21.15.28:0/4191396896 learned_addr learned my addr 172.21.15.28:0/4191396896 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:57.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa323fff700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3240b2660 0x7fa324151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.775 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa3237fe700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3240cecf0 0x7fa3240ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.775 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa328af6700 1 -- 172.21.15.28:0/4191396896 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3240cecf0 msgr2=0x7fa3240ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.775 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa328af6700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3240cecf0 0x7fa3240ac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.775 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa328af6700 1 -- 172.21.15.28:0/4191396896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3240b2660 msgr2=0x7fa324151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:57.775 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa328af6700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3240b2660 0x7fa324151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:57.775 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa328af6700 1 -- 172.21.15.28:0/4191396896 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c009550 con 0x7fa32415c560 2023-11-28T11:09:57.775 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa3237fe700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3240cecf0 0x7fa3240ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:57.775 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa323fff700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3240b2660 0x7fa324151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:09:57.775 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.769+0000 7fa328af6700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa32415c560 0x7fa3240af040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa31c007d20 tx=0x7fa31c00e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.775 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.770+0000 7fa320ff9700 1 -- 172.21.15.28:0/4191396896 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c00ede0 con 0x7fa32415c560 2023-11-28T11:09:57.775 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.770+0000 7fa320ff9700 1 -- 172.21.15.28:0/4191396896 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31c014420 con 0x7fa32415c560 2023-11-28T11:09:57.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.770+0000 7fa320ff9700 1 -- 172.21.15.28:0/4191396896 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c0147d0 con 0x7fa32415c560 2023-11-28T11:09:57.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.770+0000 7fa32a559700 1 -- 172.21.15.28:0/4191396896 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3240b2af0 con 0x7fa32415c560 2023-11-28T11:09:57.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.770+0000 7fa32a559700 1 -- 172.21.15.28:0/4191396896 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa32406fdf0 con 0x7fa32415c560 2023-11-28T11:09:57.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.771+0000 7fa32a559700 1 -- 172.21.15.28:0/4191396896 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3240b2dd0 con 0x7fa32415c560 2023-11-28T11:09:57.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.771+0000 7fa320ff9700 1 -- 172.21.15.28:0/4191396896 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa31c012050 con 0x7fa32415c560 2023-11-28T11:09:57.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.771+0000 7fa320ff9700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3040682f0 0x7fa30406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:57.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.772+0000 7fa320ff9700 1 -- 172.21.15.28:0/4191396896 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(27..27 src has 1..27) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fa31c086580 con 0x7fa32415c560 2023-11-28T11:09:57.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.772+0000 7fa323fff700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3040682f0 0x7fa30406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:57.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.772+0000 7fa323fff700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3040682f0 0x7fa30406a7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa3240addd0 tx=0x7fa314008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:57.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:57.776+0000 7fa320ff9700 1 -- 172.21.15.28:0/4191396896 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa31c052a50 con 0x7fa32415c560 2023-11-28T11:09:58.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.016+0000 7fa32a559700 1 -- 172.21.15.28:0/4191396896 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa3240675a0 con 0x7fa32415c560 2023-11-28T11:09:58.304 INFO:tasks.ceph.mon.a.smithi028.stderr:2023-11-28T11:09:58.303+0000 7fde5abc8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:09:58.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.306+0000 7fa320ff9700 1 -- 172.21.15.28:0/4191396896 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v10) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa31c056070 con 0x7fa32415c560 2023-11-28T11:09:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.308+0000 7fa32a559700 1 -- 172.21.15.28:0/4191396896 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3040682f0 msgr2=0x7fa30406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.308+0000 7fa32a559700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3040682f0 0x7fa30406a7a0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa3240addd0 tx=0x7fa314008040 comp rx=0 tx=0).stop 2023-11-28T11:09:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.308+0000 7fa32a559700 1 -- 172.21.15.28:0/4191396896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa32415c560 msgr2=0x7fa3240af040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.308+0000 7fa32a559700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa32415c560 0x7fa3240af040 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fa31c007d20 tx=0x7fa31c00e3c0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.308+0000 7fa32a559700 1 -- 172.21.15.28:0/4191396896 shutdown_connections 2023-11-28T11:09:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.309+0000 7fa32a559700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3040682f0 0x7fa30406a7a0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.309+0000 7fa32a559700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa32415c560 0x7fa3240af040 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.309+0000 7fa32a559700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa3240b2660 0x7fa324151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.309+0000 7fa32a559700 1 --2- 172.21.15.28:0/4191396896 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3240cecf0 0x7fa3240ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.309+0000 7fa32a559700 1 -- 172.21.15.28:0/4191396896 >> 172.21.15.28:0/4191396896 conn(0x7fa3240cd660 msgr2=0x7fa3240b0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.309+0000 7fa32a559700 1 -- 172.21.15.28:0/4191396896 shutdown_connections 2023-11-28T11:09:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.309+0000 7fa32a559700 1 -- 172.21.15.28:0/4191396896 wait complete. 2023-11-28T11:09:58.311 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:09:58.312 INFO:tasks.ceph.mds.d.smithi059.stderr: -5094> 2023-11-28T11:08:51.584+0000 7f532f296700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:09:58.323 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:09:58.345 INFO:tasks.ceph.mds.d.smithi059.stdout:starting mds.d at 2023-11-28T11:09:58.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.452+0000 7f56563f8700 1 Processor -- start 2023-11-28T11:09:58.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.452+0000 7f56563f8700 1 -- start start 2023-11-28T11:09:58.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f56563f8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56500b1630 0x7f56500cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f56563f8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f565006afc0 0x7f565006b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f56563f8700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56500ce550 con 0x7f56500b1a00 2023-11-28T11:09:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f56563f8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56500ce830 con 0x7f565006afc0 2023-11-28T11:09:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f56563f8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56500ceb10 con 0x7f56500b1630 2023-11-28T11:09:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f564f7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f56500b1a00 0x7f56500caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37110/0 (socket says 172.21.15.28:37110) 2023-11-28T11:09:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f564f7fe700 1 -- 172.21.15.28:0/3797777450 learned_addr learned my addr 172.21.15.28:0/3797777450 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:09:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f564ffff700 1 --2- 172.21.15.28:0/3797777450 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56500b1630 0x7f56500cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f5654995700 1 --2- 172.21.15.28:0/3797777450 >> v2:172.21.15.59:3300/0 conn(0x7f565006afc0 0x7f565006b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f564effd700 1 -- 172.21.15.28:0/3797777450 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232181896 0 0) 0x7f56500ce550 con 0x7f56500b1a00 2023-11-28T11:09:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f564effd700 1 -- 172.21.15.28:0/3797777450 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f563c0035b0 con 0x7f56500b1a00 2023-11-28T11:09:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f564ffff700 1 -- 172.21.15.28:0/3797777450 >> v2:172.21.15.59:3300/0 conn(0x7f565006afc0 msgr2=0x7f565006b440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f564ffff700 1 --2- 172.21.15.28:0/3797777450 >> v2:172.21.15.59:3300/0 conn(0x7f565006afc0 0x7f565006b440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f564ffff700 1 -- 172.21.15.28:0/3797777450 >> v1:172.21.15.28:6789/0 conn(0x7f56500b1a00 legacy=0x7f56500caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f564effd700 1 -- 172.21.15.28:0/3797777450 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2362922000 0 0) 0x7f563c0035b0 con 0x7f56500b1a00 2023-11-28T11:09:58.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f564ffff700 1 -- 172.21.15.28:0/3797777450 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56500cedf0 con 0x7f56500b1630 2023-11-28T11:09:58.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.453+0000 7f5654995700 1 --2- 172.21.15.28:0/3797777450 >> v2:172.21.15.59:3300/0 conn(0x7f565006afc0 0x7f565006b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:58.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.454+0000 7f564ffff700 1 --2- 172.21.15.28:0/3797777450 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56500b1630 0x7f56500cf5f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5638009840 tx=0x7f5638009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=962b0c566fc43a15 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.454+0000 7f564effd700 1 -- 172.21.15.28:0/3797777450 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640001670 con 0x7f56500b1630 2023-11-28T11:09:58.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.454+0000 7f564effd700 1 -- 172.21.15.28:0/3797777450 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5638004030 con 0x7f56500b1630 2023-11-28T11:09:58.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.454+0000 7f564effd700 1 -- 172.21.15.28:0/3797777450 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56380043c0 con 0x7f56500b1630 2023-11-28T11:09:58.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.454+0000 7f56563f8700 1 -- 172.21.15.28:0/3797777450 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56500b1630 msgr2=0x7f56500cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.454+0000 7f56563f8700 1 --2- 172.21.15.28:0/3797777450 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56500b1630 0x7f56500cf5f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f5638009840 tx=0x7f5638009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:58.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.454+0000 7f56563f8700 1 -- 172.21.15.28:0/3797777450 shutdown_connections 2023-11-28T11:09:58.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.454+0000 7f56563f8700 1 --2- 172.21.15.28:0/3797777450 >> v2:172.21.15.59:3300/0 conn(0x7f565006afc0 0x7f565006b440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.454+0000 7f56563f8700 1 --2- 172.21.15.28:0/3797777450 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56500b1630 0x7f56500cf5f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.454+0000 7f56563f8700 1 -- 172.21.15.28:0/3797777450 >> 172.21.15.28:0/3797777450 conn(0x7f56500cc630 msgr2=0x7f56500cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:58.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.454+0000 7f56563f8700 1 -- 172.21.15.28:0/3797777450 shutdown_connections 2023-11-28T11:09:58.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.454+0000 7f56563f8700 1 -- 172.21.15.28:0/3797777450 wait complete. 2023-11-28T11:09:58.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.456+0000 7f56563f8700 1 Processor -- start 2023-11-28T11:09:58.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.456+0000 7f56563f8700 1 -- start start 2023-11-28T11:09:58.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.456+0000 7f56563f8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f565006afc0 0x7f56501f2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f56563f8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56500b1630 0x7f56500b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f56563f8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56500b1a00 0x7f565015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f56563f8700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56500b2f20 con 0x7f56500b1630 2023-11-28T11:09:58.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f56563f8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56500b3130 con 0x7f565006afc0 2023-11-28T11:09:58.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f56563f8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56500b3410 con 0x7f56500b1a00 2023-11-28T11:09:58.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f564f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56500b1630 0x7f56500b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f564f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56500b1630 0x7f56500b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49058/0 (socket says 172.21.15.28:49058) 2023-11-28T11:09:58.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f564f7fe700 1 -- 172.21.15.28:0/999580039 learned_addr learned my addr 172.21.15.28:0/999580039 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:58.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f564ffff700 1 --2- 172.21.15.28:0/999580039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f565006afc0 0x7f56501f2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f5654995700 1 --2- 172.21.15.28:0/999580039 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56500b1a00 0x7f565015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f564f7fe700 1 -- 172.21.15.28:0/999580039 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56500b1a00 msgr2=0x7f565015bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f564f7fe700 1 --2- 172.21.15.28:0/999580039 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56500b1a00 0x7f565015bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f564f7fe700 1 -- 172.21.15.28:0/999580039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f565006afc0 msgr2=0x7f56501f2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:58.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f564f7fe700 1 --2- 172.21.15.28:0/999580039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f565006afc0 0x7f56501f2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:58.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f564f7fe700 1 -- 172.21.15.28:0/999580039 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5648009a70 con 0x7f56500b1630 2023-11-28T11:09:58.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f564f7fe700 1 --2- 172.21.15.28:0/999580039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56500b1630 0x7f56500b29e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f564000c750 tx=0x7f564000ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f564cff9700 1 -- 172.21.15.28:0/999580039 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5640004c80 con 0x7f56500b1630 2023-11-28T11:09:58.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f564cff9700 1 -- 172.21.15.28:0/999580039 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5640004de0 con 0x7f56500b1630 2023-11-28T11:09:58.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f56563f8700 1 -- 172.21.15.28:0/999580039 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56380094a0 con 0x7f56500b1630 2023-11-28T11:09:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f564cff9700 1 -- 172.21.15.28:0/999580039 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56400072d0 con 0x7f56500b1630 2023-11-28T11:09:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.457+0000 7f56563f8700 1 -- 172.21.15.28:0/999580039 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56500b1e70 con 0x7f56500b1630 2023-11-28T11:09:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.458+0000 7f56563f8700 1 -- 172.21.15.28:0/999580039 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56500cf0d0 con 0x7f56500b1630 2023-11-28T11:09:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.459+0000 7f564cff9700 1 -- 172.21.15.28:0/999580039 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f564001a020 con 0x7f56500b1630 2023-11-28T11:09:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.459+0000 7f564cff9700 1 --2- 172.21.15.28:0/999580039 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f56300682f0 0x7f563006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:58.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.459+0000 7f564ffff700 1 --2- 172.21.15.28:0/999580039 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f56300682f0 0x7f563006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.459+0000 7f564cff9700 1 -- 172.21.15.28:0/999580039 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f5640085ba0 con 0x7f56500b1630 2023-11-28T11:09:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.460+0000 7f564ffff700 1 --2- 172.21.15.28:0/999580039 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f56300682f0 0x7f563006a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f56380090b0 tx=0x7f5638008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:58.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.464+0000 7f564cff9700 1 -- 172.21.15.28:0/999580039 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5640052000 con 0x7f56500b1630 2023-11-28T11:09:58.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:58.707+0000 7f56563f8700 1 -- 172.21.15.28:0/999580039 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f56500cb5b0 con 0x7f56500b1630 2023-11-28T11:09:59.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.316+0000 7f564cff9700 1 -- 172.21.15.28:0/999580039 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5640055620 con 0x7f56500b1630 2023-11-28T11:09:59.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.319+0000 7f56563f8700 1 -- 172.21.15.28:0/999580039 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f56300682f0 msgr2=0x7f563006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.319+0000 7f56563f8700 1 --2- 172.21.15.28:0/999580039 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f56300682f0 0x7f563006a7a0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f56380090b0 tx=0x7f5638008030 comp rx=0 tx=0).stop 2023-11-28T11:09:59.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.319+0000 7f56563f8700 1 -- 172.21.15.28:0/999580039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56500b1630 msgr2=0x7f56500b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.319+0000 7f56563f8700 1 --2- 172.21.15.28:0/999580039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56500b1630 0x7f56500b29e0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f564000c750 tx=0x7f564000ca60 comp rx=0 tx=0).stop 2023-11-28T11:09:59.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.319+0000 7f56563f8700 1 -- 172.21.15.28:0/999580039 shutdown_connections 2023-11-28T11:09:59.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.319+0000 7f56563f8700 1 --2- 172.21.15.28:0/999580039 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f56300682f0 0x7f563006a7a0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.319+0000 7f56563f8700 1 --2- 172.21.15.28:0/999580039 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f56500b1a00 0x7f565015bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.319+0000 7f56563f8700 1 --2- 172.21.15.28:0/999580039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f565006afc0 0x7f56501f2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.319+0000 7f56563f8700 1 --2- 172.21.15.28:0/999580039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f56500b1630 0x7f56500b29e0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.319+0000 7f56563f8700 1 -- 172.21.15.28:0/999580039 >> 172.21.15.28:0/999580039 conn(0x7f56500cc630 msgr2=0x7f56500cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:59.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.320+0000 7f56563f8700 1 -- 172.21.15.28:0/999580039 shutdown_connections 2023-11-28T11:09:59.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.320+0000 7f56563f8700 1 -- 172.21.15.28:0/999580039 wait complete. 2023-11-28T11:09:59.336 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:09:59.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.471+0000 7f94454a4700 1 Processor -- start 2023-11-28T11:09:59.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.471+0000 7f94454a4700 1 -- start start 2023-11-28T11:09:59.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.472+0000 7f94454a4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f944015c740 0x7f944006bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:59.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.472+0000 7f94454a4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f944006c470 0x7f944015d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:59.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.472+0000 7f94454a4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9440160ad0 con 0x7f9440160f60 2023-11-28T11:09:59.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.472+0000 7f94454a4700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f944006c8f0 con 0x7f944015c740 2023-11-28T11:09:59.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.472+0000 7f94454a4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f944015d760 con 0x7f944006c470 2023-11-28T11:09:59.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.472+0000 7f943f7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f944006c470 0x7f944015d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:59.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.472+0000 7f943effd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f944015c740 0x7f944006bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:59.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.472+0000 7f943f7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f944006c470 0x7f944015d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:54826/0 (socket says 172.21.15.28:54826) 2023-11-28T11:09:59.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.472+0000 7f943f7fe700 1 -- 172.21.15.28:0/73461806 learned_addr learned my addr 172.21.15.28:0/73461806 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:59.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.472+0000 7f943e7fc700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f9440160f60 0x7f944015cb10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:47908/0 (socket says 172.21.15.28:47908) 2023-11-28T11:09:59.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.473+0000 7f943dffb700 1 -- 172.21.15.28:0/73461806 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 300434705 0 0) 0x7f9440160ad0 con 0x7f9440160f60 2023-11-28T11:09:59.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.473+0000 7f943dffb700 1 -- 172.21.15.28:0/73461806 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94280035b0 con 0x7f9440160f60 2023-11-28T11:09:59.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.473+0000 7f943effd700 1 -- 172.21.15.28:0/73461806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f944006c470 msgr2=0x7f944015d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:09:59.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.473+0000 7f943effd700 1 --2- 172.21.15.28:0/73461806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f944006c470 0x7f944015d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.473+0000 7f943effd700 1 -- 172.21.15.28:0/73461806 >> v1:172.21.15.59:6789/0 conn(0x7f9440160f60 legacy=0x7f944015cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.473+0000 7f943effd700 1 -- 172.21.15.28:0/73461806 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944015d9d0 con 0x7f944015c740 2023-11-28T11:09:59.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.473+0000 7f943f7fe700 1 --2- 172.21.15.28:0/73461806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f944006c470 0x7f944015d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:09:59.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.473+0000 7f943effd700 1 --2- 172.21.15.28:0/73461806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944015c740 0x7f944006bf30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9434009840 tx=0x7f9434009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6d15ee0741261b29 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:59.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.473+0000 7f943dffb700 1 -- 172.21.15.28:0/73461806 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434011830 con 0x7f944015c740 2023-11-28T11:09:59.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.473+0000 7f943dffb700 1 -- 172.21.15.28:0/73461806 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9434011990 con 0x7f944015c740 2023-11-28T11:09:59.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.473+0000 7f943dffb700 1 -- 172.21.15.28:0/73461806 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9434011d00 con 0x7f944015c740 2023-11-28T11:09:59.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.474+0000 7f94454a4700 1 -- 172.21.15.28:0/73461806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944015c740 msgr2=0x7f944006bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.474+0000 7f94454a4700 1 --2- 172.21.15.28:0/73461806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944015c740 0x7f944006bf30 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f9434009840 tx=0x7f9434009b50 comp rx=0 tx=0).stop 2023-11-28T11:09:59.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.474+0000 7f94454a4700 1 -- 172.21.15.28:0/73461806 shutdown_connections 2023-11-28T11:09:59.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.474+0000 7f94454a4700 1 --2- 172.21.15.28:0/73461806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f944015c740 0x7f944006bf30 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.474+0000 7f94454a4700 1 --2- 172.21.15.28:0/73461806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f944006c470 0x7f944015d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.474+0000 7f94454a4700 1 -- 172.21.15.28:0/73461806 >> 172.21.15.28:0/73461806 conn(0x7f94400d2e70 msgr2=0x7f94400d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:09:59.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.474+0000 7f94454a4700 1 -- 172.21.15.28:0/73461806 shutdown_connections 2023-11-28T11:09:59.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.474+0000 7f94454a4700 1 -- 172.21.15.28:0/73461806 wait complete. 2023-11-28T11:09:59.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.475+0000 7f94454a4700 1 Processor -- start 2023-11-28T11:09:59.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.475+0000 7f94454a4700 1 -- start start 2023-11-28T11:09:59.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f94454a4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f944006c470 0x7f94400b75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:59.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f94454a4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f944015c740 0x7f944015f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:59.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f94454a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9440160f60 0x7f944015a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:59.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f94454a4700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94400c9f40 con 0x7f944006c470 2023-11-28T11:09:59.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f94454a4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94400b7e10 con 0x7f9440160f60 2023-11-28T11:09:59.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f94454a4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94400ca150 con 0x7f944015c740 2023-11-28T11:09:59.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f943effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f944006c470 0x7f94400b75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f943effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f944006c470 0x7f94400b75c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49070/0 (socket says 172.21.15.28:49070) 2023-11-28T11:09:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f943effd700 1 -- 172.21.15.28:0/3798447263 learned_addr learned my addr 172.21.15.28:0/3798447263 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:09:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f943f7fe700 1 --2- 172.21.15.28:0/3798447263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9440160f60 0x7f944015a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f943e7fc700 1 --2- 172.21.15.28:0/3798447263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f944015c740 0x7f944015f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f943effd700 1 -- 172.21.15.28:0/3798447263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f944015c740 msgr2=0x7f944015f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f943effd700 1 --2- 172.21.15.28:0/3798447263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f944015c740 0x7f944015f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f943effd700 1 -- 172.21.15.28:0/3798447263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9440160f60 msgr2=0x7f944015a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:09:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f943effd700 1 --2- 172.21.15.28:0/3798447263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9440160f60 0x7f944015a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:09:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f943effd700 1 -- 172.21.15.28:0/3798447263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9438009b30 con 0x7f944006c470 2023-11-28T11:09:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.476+0000 7f943f7fe700 1 --2- 172.21.15.28:0/3798447263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9440160f60 0x7f944015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:09:59.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.477+0000 7f943effd700 1 --2- 172.21.15.28:0/3798447263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f944006c470 0x7f94400b75c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f94340090b0 tx=0x7f9434004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:59.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.477+0000 7f94277fe700 1 -- 172.21.15.28:0/3798447263 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94340045d0 con 0x7f944006c470 2023-11-28T11:09:59.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.477+0000 7f94277fe700 1 -- 172.21.15.28:0/3798447263 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f943401f070 con 0x7f944006c470 2023-11-28T11:09:59.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.477+0000 7f94277fe700 1 -- 172.21.15.28:0/3798447263 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94340038a0 con 0x7f944006c470 2023-11-28T11:09:59.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.477+0000 7f94454a4700 1 -- 172.21.15.28:0/3798447263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94340094a0 con 0x7f944006c470 2023-11-28T11:09:59.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.477+0000 7f94454a4700 1 -- 172.21.15.28:0/3798447263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9440160760 con 0x7f944006c470 2023-11-28T11:09:59.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.478+0000 7f94454a4700 1 -- 172.21.15.28:0/3798447263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9440160480 con 0x7f944006c470 2023-11-28T11:09:59.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.478+0000 7f94277fe700 1 -- 172.21.15.28:0/3798447263 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f943400f050 con 0x7f944006c470 2023-11-28T11:09:59.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.479+0000 7f94277fe700 1 --2- 172.21.15.28:0/3798447263 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f942c0682f0 0x7f942c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:09:59.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.479+0000 7f94277fe700 1 -- 172.21.15.28:0/3798447263 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(28..28 src has 1..28) v4 ==== 6957+0+0 (secure 0 0 0) 0x7f9434021070 con 0x7f944006c470 2023-11-28T11:09:59.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.479+0000 7f943e7fc700 1 --2- 172.21.15.28:0/3798447263 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f942c0682f0 0x7f942c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:09:59.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.479+0000 7f943e7fc700 1 --2- 172.21.15.28:0/3798447263 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f942c0682f0 0x7f942c06a7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f94400b8990 tx=0x7f9430009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:09:59.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.483+0000 7f94277fe700 1 -- 172.21.15.28:0/3798447263 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9434053e30 con 0x7f944006c470 2023-11-28T11:09:59.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:09:59.674+0000 7f94454a4700 1 -- 172.21.15.28:0/3798447263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f94400739f0 con 0x7f944006c470 2023-11-28T11:10:00.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.366+0000 7f94277fe700 1 -- 172.21.15.28:0/3798447263 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v29) v1 ==== 181+0+0 (secure 0 0 0) 0x7f9434057450 con 0x7f944006c470 2023-11-28T11:10:00.371 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.370+0000 7f94454a4700 1 -- 172.21.15.28:0/3798447263 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f942c0682f0 msgr2=0x7f942c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.371 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.370+0000 7f94454a4700 1 --2- 172.21.15.28:0/3798447263 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f942c0682f0 0x7f942c06a7a0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f94400b8990 tx=0x7f9430009040 comp rx=0 tx=0).stop 2023-11-28T11:10:00.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.370+0000 7f94454a4700 1 -- 172.21.15.28:0/3798447263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f944006c470 msgr2=0x7f94400b75c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.370+0000 7f94454a4700 1 --2- 172.21.15.28:0/3798447263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f944006c470 0x7f94400b75c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f94340090b0 tx=0x7f9434004750 comp rx=0 tx=0).stop 2023-11-28T11:10:00.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.370+0000 7f94454a4700 1 -- 172.21.15.28:0/3798447263 shutdown_connections 2023-11-28T11:10:00.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.370+0000 7f94454a4700 1 --2- 172.21.15.28:0/3798447263 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f942c0682f0 0x7f942c06a7a0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.370+0000 7f94454a4700 1 --2- 172.21.15.28:0/3798447263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f944006c470 0x7f94400b75c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.370+0000 7f94454a4700 1 --2- 172.21.15.28:0/3798447263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9440160f60 0x7f944015a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.370+0000 7f94454a4700 1 --2- 172.21.15.28:0/3798447263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f944015c740 0x7f944015f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.370+0000 7f94454a4700 1 -- 172.21.15.28:0/3798447263 >> 172.21.15.28:0/3798447263 conn(0x7f94400d2e70 msgr2=0x7f94400b3e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:00.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.370+0000 7f94454a4700 1 -- 172.21.15.28:0/3798447263 shutdown_connections 2023-11-28T11:10:00.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.370+0000 7f94454a4700 1 -- 172.21.15.28:0/3798447263 wait complete. 2023-11-28T11:10:00.373 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:10:00.387 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:10:00.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.524+0000 7fd77bb6c700 1 Processor -- start 2023-11-28T11:10:00.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.524+0000 7fd77bb6c700 1 -- start start 2023-11-28T11:10:00.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd77bb6c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd77415c2e0 0x7fd7740cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd77bb6c700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7740b5870 con 0x7fd77415c6b0 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd77bb6c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7740b5b50 con 0x7fd7740b6630 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd77bb6c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740b5e30 con 0x7fd77415c2e0 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd779107700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd77415c6b0 0x7fd7740b3930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37136/0 (socket says 172.21.15.28:37136) 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd779107700 1 -- 172.21.15.28:0/2588353789 learned_addr learned my addr 172.21.15.28:0/2588353789 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd779908700 1 --2- 172.21.15.28:0/2588353789 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd77415c2e0 0x7fd7740cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3623493837 0 0) 0x7fd7740b5870 con 0x7fd77415c6b0 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7600035b0 con 0x7fd77415c6b0 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1690182083 0 0) 0x7fd7740b5b50 con 0x7fd7740b6630 2023-11-28T11:10:00.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7740b5870 con 0x7fd7740b6630 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2189750125 0 0) 0x7fd7600035b0 con 0x7fd77415c6b0 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd77415c2e0 msgr2=0x7fd7740cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd778906700 1 --2- 172.21.15.28:0/2588353789 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd77415c2e0 0x7fd7740cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 >> v1:172.21.15.59:6789/0 conn(0x7fd7740b6630 legacy=0x7fd7740704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7740b6110 con 0x7fd77415c6b0 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fd764001670 con 0x7fd77415c6b0 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd779908700 1 --2- 172.21.15.28:0/2588353789 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd77415c2e0 0x7fd7740cf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 >> v1:172.21.15.28:6789/0 conn(0x7fd77415c6b0 legacy=0x7fd7740b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd778906700 1 --2- 172.21.15.28:0/2588353789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd760003bc0 0x7fd760003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd778906700 1 --2- 172.21.15.28:0/2588353789 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd77415c2e0 0x7fd760004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.525+0000 7fd778906700 1 --2- 172.21.15.28:0/2588353789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd760004d90 0x7fd760005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7740b5e30 con 0x7fd760003bc0 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd76000bac0 con 0x7fd760004d90 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd76000bc80 con 0x7fd77415c2e0 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd779908700 1 --2- 172.21.15.28:0/2588353789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd760003bc0 0x7fd760003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd779107700 1 --2- 172.21.15.28:0/2588353789 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd77415c2e0 0x7fd760004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd77a109700 1 --2- 172.21.15.28:0/2588353789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd760004d90 0x7fd760005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd779908700 1 -- 172.21.15.28:0/2588353789 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd77415c2e0 msgr2=0x7fd760004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd779908700 1 --2- 172.21.15.28:0/2588353789 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd77415c2e0 0x7fd760004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd779908700 1 -- 172.21.15.28:0/2588353789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd760004d90 msgr2=0x7fd760005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd779908700 1 --2- 172.21.15.28:0/2588353789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd760004d90 0x7fd760005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd779908700 1 -- 172.21.15.28:0/2588353789 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7700094a0 con 0x7fd760003bc0 2023-11-28T11:10:00.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd77a109700 1 --2- 172.21.15.28:0/2588353789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd760004d90 0x7fd760005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:00.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd779908700 1 --2- 172.21.15.28:0/2588353789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd760003bc0 0x7fd760003fb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd770009d30 tx=0x7fd760005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b998b6d14bb4d8a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:00.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd770009e80 con 0x7fd760003bc0 2023-11-28T11:10:00.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77000e9a0 con 0x7fd760003bc0 2023-11-28T11:10:00.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.526+0000 7fd778906700 1 -- 172.21.15.28:0/2588353789 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77000ed10 con 0x7fd760003bc0 2023-11-28T11:10:00.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.527+0000 7fd77bb6c700 1 -- 172.21.15.28:0/2588353789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd760003bc0 msgr2=0x7fd760003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.527+0000 7fd77bb6c700 1 --2- 172.21.15.28:0/2588353789 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd760003bc0 0x7fd760003fb0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fd770009d30 tx=0x7fd760005880 comp rx=0 tx=0).stop 2023-11-28T11:10:00.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.527+0000 7fd77a109700 1 -- 172.21.15.28:0/2588353789 reap_dead start 2023-11-28T11:10:00.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.527+0000 7fd77bb6c700 1 -- 172.21.15.28:0/2588353789 shutdown_connections 2023-11-28T11:10:00.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.527+0000 7fd77bb6c700 1 -- 172.21.15.28:0/2588353789 >> 172.21.15.28:0/2588353789 conn(0x7fd7740d0ae0 msgr2=0x7fd7740d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:00.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.527+0000 7fd77bb6c700 1 -- 172.21.15.28:0/2588353789 shutdown_connections 2023-11-28T11:10:00.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.527+0000 7fd77bb6c700 1 -- 172.21.15.28:0/2588353789 wait complete. 2023-11-28T11:10:00.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.528+0000 7fd77bb6c700 1 Processor -- start 2023-11-28T11:10:00.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.528+0000 7fd77bb6c700 1 -- start start 2023-11-28T11:10:00.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd77bb6c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd760003bc0 0x7fd774150f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd77bb6c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd774159370 0x7fd7741597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd77bb6c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd77406f020 0x7fd77406f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd77bb6c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77406e030 con 0x7fd774159370 2023-11-28T11:10:00.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd77bb6c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd77406e260 con 0x7fd760003bc0 2023-11-28T11:10:00.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd77bb6c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77406e540 con 0x7fd77406f020 2023-11-28T11:10:00.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd779107700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd774159370 0x7fd7741597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd779107700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd774159370 0x7fd7741597e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49092/0 (socket says 172.21.15.28:49092) 2023-11-28T11:10:00.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd779107700 1 -- 172.21.15.28:0/2964432618 learned_addr learned my addr 172.21.15.28:0/2964432618 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:00.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd779908700 1 --2- 172.21.15.28:0/2964432618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd760003bc0 0x7fd774150f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd77a109700 1 --2- 172.21.15.28:0/2964432618 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd77406f020 0x7fd77406f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd779107700 1 -- 172.21.15.28:0/2964432618 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd77406f020 msgr2=0x7fd77406f490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd779107700 1 --2- 172.21.15.28:0/2964432618 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd77406f020 0x7fd77406f490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd779107700 1 -- 172.21.15.28:0/2964432618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd760003bc0 msgr2=0x7fd774150f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:00.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd779107700 1 --2- 172.21.15.28:0/2964432618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd760003bc0 0x7fd774150f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:00.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.529+0000 7fd779107700 1 -- 172.21.15.28:0/2964432618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd774070bc0 con 0x7fd774159370 2023-11-28T11:10:00.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.530+0000 7fd779107700 1 --2- 172.21.15.28:0/2964432618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd774159370 0x7fd7741597e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd764001c30 tx=0x7fd7640075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:00.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.530+0000 7fd76a7fc700 1 -- 172.21.15.28:0/2964432618 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76400f6c0 con 0x7fd774159370 2023-11-28T11:10:00.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.530+0000 7fd76a7fc700 1 -- 172.21.15.28:0/2964432618 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd76400f840 con 0x7fd774159370 2023-11-28T11:10:00.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.530+0000 7fd76a7fc700 1 -- 172.21.15.28:0/2964432618 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd76400fbf0 con 0x7fd774159370 2023-11-28T11:10:00.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.530+0000 7fd77bb6c700 1 -- 172.21.15.28:0/2964432618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7740b5520 con 0x7fd774159370 2023-11-28T11:10:00.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.530+0000 7fd77bb6c700 1 -- 172.21.15.28:0/2964432618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd774070d50 con 0x7fd774159370 2023-11-28T11:10:00.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.531+0000 7fd77bb6c700 1 -- 172.21.15.28:0/2964432618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7740739f0 con 0x7fd774159370 2023-11-28T11:10:00.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.531+0000 7fd76a7fc700 1 -- 172.21.15.28:0/2964432618 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd76400fd50 con 0x7fd774159370 2023-11-28T11:10:00.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.531+0000 7fd76a7fc700 1 --2- 172.21.15.28:0/2964432618 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7580682f0 0x7fd75806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:00.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.532+0000 7fd76a7fc700 1 -- 172.21.15.28:0/2964432618 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 6519+0+0 (secure 0 0 0) 0x7fd764085f30 con 0x7fd774159370 2023-11-28T11:10:00.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.532+0000 7fd779908700 1 --2- 172.21.15.28:0/2964432618 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7580682f0 0x7fd75806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:00.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.532+0000 7fd779908700 1 --2- 172.21.15.28:0/2964432618 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7580682f0 0x7fd75806a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd7700090b0 tx=0x7fd77001f040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:00.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.536+0000 7fd76a7fc700 1 -- 172.21.15.28:0/2964432618 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd7640525c0 con 0x7fd774159370 2023-11-28T11:10:00.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:00.741+0000 7fd77bb6c700 1 -- 172.21.15.28:0/2964432618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd7740d0180 con 0x7fd774159370 2023-11-28T11:10:01.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.373+0000 7fd76a7fc700 1 -- 172.21.15.28:0/2964432618 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v30) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd764055be0 con 0x7fd774159370 2023-11-28T11:10:01.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.378+0000 7fd77bb6c700 1 -- 172.21.15.28:0/2964432618 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7580682f0 msgr2=0x7fd75806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:01.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.378+0000 7fd77bb6c700 1 --2- 172.21.15.28:0/2964432618 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7580682f0 0x7fd75806a7a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fd7700090b0 tx=0x7fd77001f040 comp rx=0 tx=0).stop 2023-11-28T11:10:01.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.378+0000 7fd77bb6c700 1 -- 172.21.15.28:0/2964432618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd774159370 msgr2=0x7fd7741597e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:01.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.378+0000 7fd77bb6c700 1 --2- 172.21.15.28:0/2964432618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd774159370 0x7fd7741597e0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fd764001c30 tx=0x7fd7640075f0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.378+0000 7fd77bb6c700 1 -- 172.21.15.28:0/2964432618 shutdown_connections 2023-11-28T11:10:01.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.378+0000 7fd77bb6c700 1 --2- 172.21.15.28:0/2964432618 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7580682f0 0x7fd75806a7a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.378+0000 7fd77bb6c700 1 --2- 172.21.15.28:0/2964432618 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd77406f020 0x7fd77406f490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.378+0000 7fd77bb6c700 1 --2- 172.21.15.28:0/2964432618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd760003bc0 0x7fd774150f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.378+0000 7fd77bb6c700 1 --2- 172.21.15.28:0/2964432618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd774159370 0x7fd7741597e0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.378+0000 7fd77bb6c700 1 -- 172.21.15.28:0/2964432618 >> 172.21.15.28:0/2964432618 conn(0x7fd7740d0ae0 msgr2=0x7fd7740b3910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:01.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.378+0000 7fd77bb6c700 1 -- 172.21.15.28:0/2964432618 shutdown_connections 2023-11-28T11:10:01.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.378+0000 7fd77bb6c700 1 -- 172.21.15.28:0/2964432618 wait complete. 2023-11-28T11:10:01.381 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' removed 2023-11-28T11:10:01.395 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-11-28T11:10:01.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.533+0000 7f2c8b124700 1 Processor -- start 2023-11-28T11:10:01.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.533+0000 7f2c8b124700 1 -- start start 2023-11-28T11:10:01.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.534+0000 7f2c8b124700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2c841593d0 0x7f2c840c3820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:01.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.534+0000 7f2c8b124700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c840ce0d0 0x7f2c84070d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:01.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.534+0000 7f2c8b124700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c8415ce80 con 0x7f2c840708f0 2023-11-28T11:10:01.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.534+0000 7f2c8b124700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c841597a0 con 0x7f2c841593d0 2023-11-28T11:10:01.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.534+0000 7f2c8b124700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c840712a0 con 0x7f2c840ce0d0 2023-11-28T11:10:01.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.534+0000 7f2c83fff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2c840708f0 0x7f2c840cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37158/0 (socket says 172.21.15.28:37158) 2023-11-28T11:10:01.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.534+0000 7f2c83fff700 1 -- 172.21.15.28:0/2136413828 learned_addr learned my addr 172.21.15.28:0/2136413828 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:01.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c896c1700 1 --2- 172.21.15.28:0/2136413828 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c840ce0d0 0x7f2c84070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:01.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 -- 172.21.15.28:0/2136413828 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2109065073 0 0) 0x7f2c8415ce80 con 0x7f2c840708f0 2023-11-28T11:10:01.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 -- 172.21.15.28:0/2136413828 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c740035b0 con 0x7f2c840708f0 2023-11-28T11:10:01.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c88ec0700 1 --2- 172.21.15.28:0/2136413828 >> v2:172.21.15.59:3300/0 conn(0x7f2c841593d0 0x7f2c840c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:01.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 -- 172.21.15.28:0/2136413828 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1281210929 0 0) 0x7f2c740035b0 con 0x7f2c840708f0 2023-11-28T11:10:01.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 -- 172.21.15.28:0/2136413828 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c840ce0d0 msgr2=0x7f2c84070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:01.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 --2- 172.21.15.28:0/2136413828 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c840ce0d0 0x7f2c84070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 -- 172.21.15.28:0/2136413828 >> v2:172.21.15.59:3300/0 conn(0x7f2c841593d0 msgr2=0x7f2c840c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:01.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 --2- 172.21.15.28:0/2136413828 >> v2:172.21.15.59:3300/0 conn(0x7f2c841593d0 0x7f2c840c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 -- 172.21.15.28:0/2136413828 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8415d0d0 con 0x7f2c840708f0 2023-11-28T11:10:01.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 -- 172.21.15.28:0/2136413828 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f2c70001670 con 0x7f2c840708f0 2023-11-28T11:10:01.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 -- 172.21.15.28:0/2136413828 >> v1:172.21.15.28:6789/0 conn(0x7f2c840708f0 legacy=0x7f2c840cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:01.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 --2- 172.21.15.28:0/2136413828 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c840ce0d0 0x7f2c74003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:01.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 --2- 172.21.15.28:0/2136413828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c74003fb0 0x7f2c74004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:01.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 --2- 172.21.15.28:0/2136413828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c74004d90 0x7f2c74005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:01.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 -- 172.21.15.28:0/2136413828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c841597a0 con 0x7f2c74004d90 2023-11-28T11:10:01.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 -- 172.21.15.28:0/2136413828 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c840712a0 con 0x7f2c74003fb0 2023-11-28T11:10:01.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c837fe700 1 -- 172.21.15.28:0/2136413828 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c7400dc20 con 0x7f2c840ce0d0 2023-11-28T11:10:01.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c896c1700 1 --2- 172.21.15.28:0/2136413828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c74004d90 0x7f2c74005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:01.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c88ec0700 1 --2- 172.21.15.28:0/2136413828 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c840ce0d0 0x7f2c74003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:01.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c83fff700 1 --2- 172.21.15.28:0/2136413828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c74003fb0 0x7f2c74004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:01.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.535+0000 7f2c896c1700 1 -- 172.21.15.28:0/2136413828 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c840ce0d0 msgr2=0x7f2c74003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:01.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.536+0000 7f2c896c1700 1 --2- 172.21.15.28:0/2136413828 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c840ce0d0 0x7f2c74003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.536+0000 7f2c896c1700 1 -- 172.21.15.28:0/2136413828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c74003fb0 msgr2=0x7f2c74004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:01.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.536+0000 7f2c896c1700 1 --2- 172.21.15.28:0/2136413828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c74003fb0 0x7f2c74004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.536+0000 7f2c896c1700 1 -- 172.21.15.28:0/2136413828 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c740035b0 con 0x7f2c74004d90 2023-11-28T11:10:01.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.536+0000 7f2c896c1700 1 --2- 172.21.15.28:0/2136413828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c74004d90 0x7f2c74005200 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009fc0 tx=0x7f2c74005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=405e05c13820b569 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:01.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.536+0000 7f2c837fe700 1 -- 172.21.15.28:0/2136413828 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c00ec30 con 0x7f2c74004d90 2023-11-28T11:10:01.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.536+0000 7f2c837fe700 1 -- 172.21.15.28:0/2136413828 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7c00f040 con 0x7f2c74004d90 2023-11-28T11:10:01.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.536+0000 7f2c837fe700 1 -- 172.21.15.28:0/2136413828 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7c014480 con 0x7f2c74004d90 2023-11-28T11:10:01.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.536+0000 7f2c8b124700 1 -- 172.21.15.28:0/2136413828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c74004d90 msgr2=0x7f2c74005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:01.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.536+0000 7f2c8b124700 1 --2- 172.21.15.28:0/2136413828 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c74004d90 0x7f2c74005200 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f2c7c009fc0 tx=0x7f2c74005880 comp rx=0 tx=0).stop 2023-11-28T11:10:01.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.536+0000 7f2c896c1700 1 -- 172.21.15.28:0/2136413828 reap_dead start 2023-11-28T11:10:01.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.536+0000 7f2c8b124700 1 -- 172.21.15.28:0/2136413828 shutdown_connections 2023-11-28T11:10:01.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.536+0000 7f2c8b124700 1 -- 172.21.15.28:0/2136413828 >> 172.21.15.28:0/2136413828 conn(0x7f2c840ce9a0 msgr2=0x7f2c840ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:01.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.536+0000 7f2c8b124700 1 -- 172.21.15.28:0/2136413828 shutdown_connections 2023-11-28T11:10:01.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.536+0000 7f2c8b124700 1 -- 172.21.15.28:0/2136413828 wait complete. 2023-11-28T11:10:01.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.538+0000 7f2c8b124700 1 Processor -- start 2023-11-28T11:10:01.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.538+0000 7f2c8b124700 1 -- start start 2023-11-28T11:10:01.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c8b124700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c8406a450 0x7f2c840c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:01.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c8b124700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c840c5fb0 0x7f2c8413efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:01.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c8b124700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c8406a820 0x7f2c8413f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:01.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c8b124700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c840c8320 con 0x7f2c840c5fb0 2023-11-28T11:10:01.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c8b124700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c840c8600 con 0x7f2c8406a450 2023-11-28T11:10:01.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c8b124700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c841536d0 con 0x7f2c8406a820 2023-11-28T11:10:01.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c83fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c840c5fb0 0x7f2c8413efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:01.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c83fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c840c5fb0 0x7f2c8413efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49112/0 (socket says 172.21.15.28:49112) 2023-11-28T11:10:01.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c83fff700 1 -- 172.21.15.28:0/2725993819 learned_addr learned my addr 172.21.15.28:0/2725993819 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:01.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c88ec0700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c8406a450 0x7f2c840c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:01.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c896c1700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c8406a820 0x7f2c8413f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:01.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c83fff700 1 -- 172.21.15.28:0/2725993819 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c8406a820 msgr2=0x7f2c8413f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:01.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c83fff700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c8406a820 0x7f2c8413f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c83fff700 1 -- 172.21.15.28:0/2725993819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c8406a450 msgr2=0x7f2c840c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:01.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c83fff700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c8406a450 0x7f2c840c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:01.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c83fff700 1 -- 172.21.15.28:0/2725993819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c8415d690 con 0x7f2c840c5fb0 2023-11-28T11:10:01.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c88ec0700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c8406a450 0x7f2c840c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:01.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c896c1700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c8406a820 0x7f2c8413f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:01.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c83fff700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c840c5fb0 0x7f2c8413efc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000eb20 tx=0x7f2c7000eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:01.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c817fa700 1 -- 172.21.15.28:0/2725993819 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c70001610 con 0x7f2c840c5fb0 2023-11-28T11:10:01.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c817fa700 1 -- 172.21.15.28:0/2725993819 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7000c7e0 con 0x7f2c840c5fb0 2023-11-28T11:10:01.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c8b124700 1 -- 172.21.15.28:0/2725993819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c8415d3b0 con 0x7f2c840c5fb0 2023-11-28T11:10:01.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c817fa700 1 -- 172.21.15.28:0/2725993819 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7000cb90 con 0x7f2c840c5fb0 2023-11-28T11:10:01.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.539+0000 7f2c8b124700 1 -- 172.21.15.28:0/2725993819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c8415d970 con 0x7f2c840c5fb0 2023-11-28T11:10:01.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.540+0000 7f2c8b124700 1 -- 172.21.15.28:0/2725993819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c84153bf0 con 0x7f2c840c5fb0 2023-11-28T11:10:01.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.541+0000 7f2c817fa700 1 -- 172.21.15.28:0/2725993819 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2c7000ccf0 con 0x7f2c840c5fb0 2023-11-28T11:10:01.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.541+0000 7f2c817fa700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2c680682f0 0x7f2c6806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:01.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.541+0000 7f2c817fa700 1 -- 172.21.15.28:0/2725993819 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 6047+0+0 (secure 0 0 0) 0x7f2c70085cc0 con 0x7f2c840c5fb0 2023-11-28T11:10:01.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.541+0000 7f2c88ec0700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2c680682f0 0x7f2c6806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:01.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.542+0000 7f2c88ec0700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2c680682f0 0x7f2c6806a7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009ce0 tx=0x7f2c78008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:01.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.545+0000 7f2c817fa700 1 -- 172.21.15.28:0/2725993819 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2c84153bf0 con 0x7f2c840c5fb0 2023-11-28T11:10:01.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:01.738+0000 7f2c8b124700 1 -- 172.21.15.28:0/2725993819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2c84153bf0 con 0x7f2c840c5fb0 2023-11-28T11:10:02.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.381+0000 7f2c817fa700 1 -- 172.21.15.28:0/2725993819 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v31) v1 ==== 178+0+0 (secure 0 0 0) 0x7f2c84153bf0 con 0x7f2c840c5fb0 2023-11-28T11:10:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.387+0000 7f2c8b124700 1 -- 172.21.15.28:0/2725993819 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2c680682f0 msgr2=0x7f2c6806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.387+0000 7f2c8b124700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2c680682f0 0x7f2c6806a7a0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009ce0 tx=0x7f2c78008040 comp rx=0 tx=0).stop 2023-11-28T11:10:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.387+0000 7f2c8b124700 1 -- 172.21.15.28:0/2725993819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c840c5fb0 msgr2=0x7f2c8413efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.387+0000 7f2c8b124700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c840c5fb0 0x7f2c8413efc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000eb20 tx=0x7f2c7000eee0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.387+0000 7f2c8b124700 1 -- 172.21.15.28:0/2725993819 shutdown_connections 2023-11-28T11:10:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.387+0000 7f2c8b124700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2c680682f0 0x7f2c6806a7a0 secure :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f2c78009ce0 tx=0x7f2c78008040 comp rx=0 tx=0).stop 2023-11-28T11:10:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.387+0000 7f2c8b124700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2c8406a820 0x7f2c8413f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.387+0000 7f2c8b124700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c8406a450 0x7f2c840c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.387+0000 7f2c8b124700 1 --2- 172.21.15.28:0/2725993819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2c840c5fb0 0x7f2c8413efc0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.387+0000 7f2c8b124700 1 -- 172.21.15.28:0/2725993819 >> 172.21.15.28:0/2725993819 conn(0x7f2c840ce9a0 msgr2=0x7f2c841577d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.387+0000 7f2c8b124700 1 -- 172.21.15.28:0/2725993819 shutdown_connections 2023-11-28T11:10:02.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.387+0000 7f2c8b124700 1 -- 172.21.15.28:0/2725993819 wait complete. 2023-11-28T11:10:02.390 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data_ec' removed 2023-11-28T11:10:02.405 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:10:02.405 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:10:02.405 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:10:02.405 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:02.405 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:10:02.405 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:10:02.406 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:10:02.406 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:02.406 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:10:02.406 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:10:02.406 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:10:02.406 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:02.407 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:10:02.407 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:10:02.407 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:10:02.407 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:02.418 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:10:02.418 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:10:02.419 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:10:02.419 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:10:02.420 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:10:02.420 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:10:02.421 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:10:02.421 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:10:02.461 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:10:02.461 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:02.462 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:10:02.462 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:02.463 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:10:02.463 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:02.464 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:10:02.464 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:02.465 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:10:02.549 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2023-11-28T11:10:02.550 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2023-11-28T11:10:02.555 INFO:tasks.ceph.mds.b.smithi059.stdout:starting mds.b at 2023-11-28T11:10:02.557 INFO:tasks.ceph.mds.d.smithi059.stdout:starting mds.d at 2023-11-28T11:10:02.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.603+0000 7f55888c6700 1 Processor -- start 2023-11-28T11:10:02.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.603+0000 7f55888c6700 1 -- start start 2023-11-28T11:10:02.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.604+0000 7f55888c6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f558415c890 0x7f55840b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.604+0000 7f55888c6700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f558415d2d0 con 0x7f5584160d30 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.604+0000 7f55888c6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55841611a0 con 0x7f55840ceec0 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.604+0000 7f55888c6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55840b2ba0 con 0x7f558415c890 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.604+0000 7f5581d9b700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5584160d30 0x7f55840ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37202/0 (socket says 172.21.15.28:37202) 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.604+0000 7f5581d9b700 1 -- 172.21.15.28:0/732089872 learned_addr learned my addr 172.21.15.28:0/732089872 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.604+0000 7f558259c700 1 --2- 172.21.15.28:0/732089872 >> v2:172.21.15.59:3300/0 conn(0x7f558415c890 0x7f55840b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.604+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 339589316 0 0) 0x7f558415d2d0 con 0x7f5584160d30 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.604+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f556c0035b0 con 0x7f5584160d30 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1716255772 0 0) 0x7f55841611a0 con 0x7f55840ceec0 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f558415d2d0 con 0x7f55840ceec0 2023-11-28T11:10:02.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2045501799 0 0) 0x7f556c0035b0 con 0x7f5584160d30 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 >> v2:172.21.15.59:3300/0 conn(0x7f558415c890 msgr2=0x7f55840b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558159a700 1 --2- 172.21.15.28:0/732089872 >> v2:172.21.15.59:3300/0 conn(0x7f558415c890 0x7f55840b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 >> v1:172.21.15.59:6789/0 conn(0x7f55840ceec0 legacy=0x7f55840cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55840b2d90 con 0x7f5584160d30 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f5574001670 con 0x7f5584160d30 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 >> v1:172.21.15.28:6789/0 conn(0x7f5584160d30 legacy=0x7f55840ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558159a700 1 --2- 172.21.15.28:0/732089872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f556c003b90 0x7f556c003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558159a700 1 --2- 172.21.15.28:0/732089872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f556c004850 0x7f556c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558159a700 1 --2- 172.21.15.28:0/732089872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f556c005200 0x7f556c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c00c030 con 0x7f556c005200 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c00c1a0 con 0x7f556c004850 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f556c00c360 con 0x7f556c003b90 2023-11-28T11:10:02.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558259c700 1 --2- 172.21.15.28:0/732089872 >> v2:172.21.15.59:3300/0 conn(0x7f558415c890 0x7f55840b2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f5582d9d700 1 --2- 172.21.15.28:0/732089872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f556c005200 0x7f556c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f5581d9b700 1 --2- 172.21.15.28:0/732089872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f556c004850 0x7f556c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558259c700 1 --2- 172.21.15.28:0/732089872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f556c003b90 0x7f556c003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f5582d9d700 1 -- 172.21.15.28:0/732089872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f556c003b90 msgr2=0x7f556c003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f5582d9d700 1 --2- 172.21.15.28:0/732089872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f556c003b90 0x7f556c003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f5582d9d700 1 -- 172.21.15.28:0/732089872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f556c004850 msgr2=0x7f556c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f5582d9d700 1 --2- 172.21.15.28:0/732089872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f556c004850 0x7f556c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f5582d9d700 1 -- 172.21.15.28:0/732089872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f556c0035b0 con 0x7f556c005200 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f558259c700 1 --2- 172.21.15.28:0/732089872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f556c003b90 0x7f556c003f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f5582d9d700 1 -- 172.21.15.28:0/732089872 reap_dead start 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.605+0000 7f5582d9d700 1 --2- 172.21.15.28:0/732089872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f556c005200 0x7f556c00ba10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f557c00c2f0 tx=0x7f556c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77278c4e2e3c7ca9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.606+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c001f20 con 0x7f556c005200 2023-11-28T11:10:02.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.606+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557c00ce70 con 0x7f556c005200 2023-11-28T11:10:02.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.606+0000 7f558159a700 1 -- 172.21.15.28:0/732089872 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c003c20 con 0x7f556c005200 2023-11-28T11:10:02.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.606+0000 7f55888c6700 1 -- 172.21.15.28:0/732089872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f556c005200 msgr2=0x7f556c00ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.606+0000 7f55888c6700 1 --2- 172.21.15.28:0/732089872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f556c005200 0x7f556c00ba10 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f557c00c2f0 tx=0x7f556c005880 comp rx=0 tx=0).stop 2023-11-28T11:10:02.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.606+0000 7f55888c6700 1 -- 172.21.15.28:0/732089872 shutdown_connections 2023-11-28T11:10:02.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.606+0000 7f55888c6700 1 --2- 172.21.15.28:0/732089872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f556c005200 0x7f556c00ba10 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.606+0000 7f55888c6700 1 -- 172.21.15.28:0/732089872 >> 172.21.15.28:0/732089872 conn(0x7f558406e9b0 msgr2=0x7f55840cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:02.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.606+0000 7f55888c6700 1 -- 172.21.15.28:0/732089872 shutdown_connections 2023-11-28T11:10:02.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.606+0000 7f55888c6700 1 -- 172.21.15.28:0/732089872 wait complete. 2023-11-28T11:10:02.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.608+0000 7f55888c6700 1 Processor -- start 2023-11-28T11:10:02.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.608+0000 7f55888c6700 1 -- start start 2023-11-28T11:10:02.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f55888c6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f556c005200 0x7f55841f22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f55888c6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5584150790 0x7f558415fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f55888c6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5584150c00 0x7f5584160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f55888c6700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55841f27f0 con 0x7f5584150c00 2023-11-28T11:10:02.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f55888c6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55841f6720 con 0x7f556c005200 2023-11-28T11:10:02.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f55888c6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f558415d660 con 0x7f5584150790 2023-11-28T11:10:02.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f5582d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5584150c00 0x7f5584160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f5582d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5584150c00 0x7f5584160140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49160/0 (socket says 172.21.15.28:49160) 2023-11-28T11:10:02.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f5582d9d700 1 -- 172.21.15.28:0/2307790514 learned_addr learned my addr 172.21.15.28:0/2307790514 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:02.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f558259c700 1 --2- 172.21.15.28:0/2307790514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f556c005200 0x7f55841f22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f5581d9b700 1 --2- 172.21.15.28:0/2307790514 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5584150790 0x7f558415fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f5582d9d700 1 -- 172.21.15.28:0/2307790514 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5584150790 msgr2=0x7f558415fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f5582d9d700 1 --2- 172.21.15.28:0/2307790514 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5584150790 0x7f558415fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f5582d9d700 1 -- 172.21.15.28:0/2307790514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f556c005200 msgr2=0x7f55841f22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:02.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f5582d9d700 1 --2- 172.21.15.28:0/2307790514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f556c005200 0x7f55841f22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:02.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.609+0000 7f5582d9d700 1 -- 172.21.15.28:0/2307790514 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558415cf60 con 0x7f5584150c00 2023-11-28T11:10:02.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.610+0000 7f5582d9d700 1 --2- 172.21.15.28:0/2307790514 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5584150c00 0x7f5584160140 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f55840d03e0 tx=0x7f557c001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.610+0000 7f5572ffd700 1 -- 172.21.15.28:0/2307790514 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c005270 con 0x7f5584150c00 2023-11-28T11:10:02.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.610+0000 7f5572ffd700 1 -- 172.21.15.28:0/2307790514 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f557c0053d0 con 0x7f5584150c00 2023-11-28T11:10:02.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.610+0000 7f55888c6700 1 -- 172.21.15.28:0/2307790514 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55840b3070 con 0x7f5584150c00 2023-11-28T11:10:02.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.610+0000 7f5572ffd700 1 -- 172.21.15.28:0/2307790514 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f557c005780 con 0x7f5584150c00 2023-11-28T11:10:02.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.610+0000 7f55888c6700 1 -- 172.21.15.28:0/2307790514 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f558415cc60 con 0x7f5584150c00 2023-11-28T11:10:02.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.611+0000 7f55888c6700 1 -- 172.21.15.28:0/2307790514 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5584068570 con 0x7f5584150c00 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.612+0000 7f5572ffd700 1 -- 172.21.15.28:0/2307790514 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f557c019040 con 0x7f5584150c00 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.612+0000 7f5572ffd700 1 --2- 172.21.15.28:0/2307790514 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f55640682f0 0x7f556406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.612+0000 7f5572ffd700 1 -- 172.21.15.28:0/2307790514 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f557c086f50 con 0x7f5584150c00 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.612+0000 7f558259c700 1 --2- 172.21.15.28:0/2307790514 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f55640682f0 0x7f556406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.612+0000 7f558259c700 1 --2- 172.21.15.28:0/2307790514 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f55640682f0 0x7f556406a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5578006f90 tx=0x7f5578008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:02.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.616+0000 7f5572ffd700 1 -- 172.21.15.28:0/2307790514 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f557c053840 con 0x7f5584150c00 2023-11-28T11:10:02.835 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:02.833+0000 7f55888c6700 1 -- 172.21.15.28:0/2307790514 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5584068570 con 0x7f5584150c00 2023-11-28T11:10:03.593 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.591+0000 7f5572ffd700 1 -- 172.21.15.28:0/2307790514 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v36) v1 ==== 97+0+0 (secure 0 0 0) 0x7f557c002080 con 0x7f5584150c00 2023-11-28T11:10:03.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.594+0000 7f55888c6700 1 -- 172.21.15.28:0/2307790514 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f55640682f0 msgr2=0x7f556406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:03.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.594+0000 7f55888c6700 1 --2- 172.21.15.28:0/2307790514 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f55640682f0 0x7f556406a7a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5578006f90 tx=0x7f5578008040 comp rx=0 tx=0).stop 2023-11-28T11:10:03.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.594+0000 7f55888c6700 1 -- 172.21.15.28:0/2307790514 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5584150c00 msgr2=0x7f5584160140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:03.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.594+0000 7f55888c6700 1 --2- 172.21.15.28:0/2307790514 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5584150c00 0x7f5584160140 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f55840d03e0 tx=0x7f557c001650 comp rx=0 tx=0).stop 2023-11-28T11:10:03.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.594+0000 7f55888c6700 1 -- 172.21.15.28:0/2307790514 shutdown_connections 2023-11-28T11:10:03.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.594+0000 7f55888c6700 1 --2- 172.21.15.28:0/2307790514 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f55640682f0 0x7f556406a7a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:03.597 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.594+0000 7f55888c6700 1 --2- 172.21.15.28:0/2307790514 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5584150c00 0x7f5584160140 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:03.597 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.594+0000 7f55888c6700 1 --2- 172.21.15.28:0/2307790514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f556c005200 0x7f55841f22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:03.597 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.594+0000 7f55888c6700 1 --2- 172.21.15.28:0/2307790514 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5584150790 0x7f558415fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:03.597 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.594+0000 7f55888c6700 1 -- 172.21.15.28:0/2307790514 >> 172.21.15.28:0/2307790514 conn(0x7f558406e9b0 msgr2=0x7f55840b6150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:03.597 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.594+0000 7f55888c6700 1 -- 172.21.15.28:0/2307790514 shutdown_connections 2023-11-28T11:10:03.597 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.594+0000 7f55888c6700 1 -- 172.21.15.28:0/2307790514 wait complete. 2023-11-28T11:10:03.597 INFO:teuthology.orchestra.run.smithi028.stderr: removed all blocklist entries 2023-11-28T11:10:03.611 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:10:03.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.741+0000 7f90022d6700 1 Processor -- start 2023-11-28T11:10:03.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.741+0000 7f90022d6700 1 -- start start 2023-11-28T11:10:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f90022d6700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8ffc15a5a0 0x7f8ffc06b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f90022d6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8ffc0ca280 0x7f8ffc0cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f90022d6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ffc0cbf40 0x7f8ffc0cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f90022d6700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ffc0cb570 con 0x7f8ffc15a5a0 2023-11-28T11:10:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f90022d6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ffc0ca6f0 con 0x7f8ffc0ca280 2023-11-28T11:10:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f90022d6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc0ca960 con 0x7f8ffc0cbf40 2023-11-28T11:10:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f8ffbfff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8ffc15a5a0 0x7f8ffc06b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f8ffbfff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8ffc15a5a0 0x7f8ffc06b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49168/0 (socket says 172.21.15.28:49168) 2023-11-28T11:10:03.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f8ffbfff700 1 -- 172.21.15.28:0/2715528167 learned_addr learned my addr 172.21.15.28:0/2715528167 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:03.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f9000873700 1 --2- 172.21.15.28:0/2715528167 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ffc0cbf40 0x7f8ffc0cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:03.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f8ffb7fe700 1 --2- 172.21.15.28:0/2715528167 >> v2:172.21.15.59:3300/0 conn(0x7f8ffc0ca280 0x7f8ffc0cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:03.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f8ffbfff700 1 -- 172.21.15.28:0/2715528167 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ffc0cbf40 msgr2=0x7f8ffc0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:03.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f8ffbfff700 1 --2- 172.21.15.28:0/2715528167 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ffc0cbf40 0x7f8ffc0cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:03.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f8ffbfff700 1 -- 172.21.15.28:0/2715528167 >> v2:172.21.15.59:3300/0 conn(0x7f8ffc0ca280 msgr2=0x7f8ffc0cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:03.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f8ffbfff700 1 --2- 172.21.15.28:0/2715528167 >> v2:172.21.15.59:3300/0 conn(0x7f8ffc0ca280 0x7f8ffc0cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:03.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f8ffbfff700 1 -- 172.21.15.28:0/2715528167 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ffc0cac40 con 0x7f8ffc15a5a0 2023-11-28T11:10:03.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f9000873700 1 --2- 172.21.15.28:0/2715528167 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ffc0cbf40 0x7f8ffc0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:03.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f8ffb7fe700 1 --2- 172.21.15.28:0/2715528167 >> v2:172.21.15.59:3300/0 conn(0x7f8ffc0ca280 0x7f8ffc0cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:03.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.742+0000 7f8ffbfff700 1 --2- 172.21.15.28:0/2715528167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ffc15a5a0 0x7f8ffc06b930 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc0c6b20 tx=0x7f8fec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=819f1dfbe458adb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:03.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.743+0000 7f8ffaffd700 1 -- 172.21.15.28:0/2715528167 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00b870 con 0x7f8ffc15a5a0 2023-11-28T11:10:03.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.743+0000 7f8ffaffd700 1 -- 172.21.15.28:0/2715528167 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fec00b9d0 con 0x7f8ffc15a5a0 2023-11-28T11:10:03.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.743+0000 7f8ffaffd700 1 -- 172.21.15.28:0/2715528167 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fec00bd60 con 0x7f8ffc15a5a0 2023-11-28T11:10:03.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.743+0000 7f90022d6700 1 -- 172.21.15.28:0/2715528167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ffc15a5a0 msgr2=0x7f8ffc06b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:03.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.743+0000 7f90022d6700 1 --2- 172.21.15.28:0/2715528167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ffc15a5a0 0x7f8ffc06b930 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc0c6b20 tx=0x7f8fec009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:03.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.743+0000 7f90022d6700 1 -- 172.21.15.28:0/2715528167 shutdown_connections 2023-11-28T11:10:03.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.743+0000 7f90022d6700 1 --2- 172.21.15.28:0/2715528167 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ffc0cbf40 0x7f8ffc0cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:03.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.743+0000 7f90022d6700 1 --2- 172.21.15.28:0/2715528167 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ffc15a5a0 0x7f8ffc06b930 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:03.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.743+0000 7f90022d6700 1 --2- 172.21.15.28:0/2715528167 >> v2:172.21.15.59:3300/0 conn(0x7f8ffc0ca280 0x7f8ffc0cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:03.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.743+0000 7f90022d6700 1 -- 172.21.15.28:0/2715528167 >> 172.21.15.28:0/2715528167 conn(0x7f8ffc0b1650 msgr2=0x7f8ffc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:03.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.743+0000 7f90022d6700 1 -- 172.21.15.28:0/2715528167 shutdown_connections 2023-11-28T11:10:03.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.743+0000 7f90022d6700 1 -- 172.21.15.28:0/2715528167 wait complete. 2023-11-28T11:10:03.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.744+0000 7f90022d6700 1 Processor -- start 2023-11-28T11:10:03.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.745+0000 7f90022d6700 1 -- start start 2023-11-28T11:10:03.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.745+0000 7f90022d6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ffc0ca280 0x7f8ffc1f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:03.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.745+0000 7f90022d6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ffc0cbf40 0x7f8ffc0ceab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:03.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.745+0000 7f90022d6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ffc15a5a0 0x7f8ffc0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:03.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.745+0000 7f90022d6700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc0ce620 con 0x7f8ffc15a5a0 2023-11-28T11:10:03.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.745+0000 7f90022d6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc1f69e0 con 0x7f8ffc0ca280 2023-11-28T11:10:03.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.745+0000 7f90022d6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ffc0ceff0 con 0x7f8ffc0cbf40 2023-11-28T11:10:03.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.745+0000 7f9000873700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ffc15a5a0 0x7f8ffc0ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:03.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.745+0000 7f9000873700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ffc15a5a0 0x7f8ffc0ab8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49178/0 (socket says 172.21.15.28:49178) 2023-11-28T11:10:03.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.745+0000 7f9000873700 1 -- 172.21.15.28:0/3244590995 learned_addr learned my addr 172.21.15.28:0/3244590995 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:03.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.746+0000 7f8ffbfff700 1 --2- 172.21.15.28:0/3244590995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ffc0ca280 0x7f8ffc1f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:03.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.746+0000 7f8ffb7fe700 1 --2- 172.21.15.28:0/3244590995 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ffc0cbf40 0x7f8ffc0ceab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:03.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.746+0000 7f9000873700 1 -- 172.21.15.28:0/3244590995 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ffc0cbf40 msgr2=0x7f8ffc0ceab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:03.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.746+0000 7f9000873700 1 --2- 172.21.15.28:0/3244590995 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ffc0cbf40 0x7f8ffc0ceab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:03.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.746+0000 7f9000873700 1 -- 172.21.15.28:0/3244590995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ffc0ca280 msgr2=0x7f8ffc1f64a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:03.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.746+0000 7f9000873700 1 --2- 172.21.15.28:0/3244590995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ffc0ca280 0x7f8ffc1f64a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:03.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.746+0000 7f9000873700 1 -- 172.21.15.28:0/3244590995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fec009550 con 0x7f8ffc15a5a0 2023-11-28T11:10:03.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.746+0000 7f8ffbfff700 1 --2- 172.21.15.28:0/3244590995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ffc0ca280 0x7f8ffc1f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:03.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.746+0000 7f9000873700 1 --2- 172.21.15.28:0/3244590995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ffc15a5a0 0x7f8ffc0ab8c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4007ef0 tx=0x7f8ff400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:03.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.746+0000 7f8ff8ff9700 1 -- 172.21.15.28:0/3244590995 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff4015400 con 0x7f8ffc15a5a0 2023-11-28T11:10:03.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.746+0000 7f8ff8ff9700 1 -- 172.21.15.28:0/3244590995 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff400f040 con 0x7f8ffc15a5a0 2023-11-28T11:10:03.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.746+0000 7f90022d6700 1 -- 172.21.15.28:0/3244590995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ffc0cb200 con 0x7f8ffc15a5a0 2023-11-28T11:10:03.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.746+0000 7f8ff8ff9700 1 -- 172.21.15.28:0/3244590995 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff40157a0 con 0x7f8ffc15a5a0 2023-11-28T11:10:03.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.746+0000 7f90022d6700 1 -- 172.21.15.28:0/3244590995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ffc0caf20 con 0x7f8ffc15a5a0 2023-11-28T11:10:03.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.747+0000 7f90022d6700 1 -- 172.21.15.28:0/3244590995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ffc15a970 con 0x7f8ffc15a5a0 2023-11-28T11:10:03.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.748+0000 7f8ff8ff9700 1 -- 172.21.15.28:0/3244590995 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8ff401c030 con 0x7f8ffc15a5a0 2023-11-28T11:10:03.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.748+0000 7f8ff8ff9700 1 --2- 172.21.15.28:0/3244590995 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8fe80682f0 0x7f8fe806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:03.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.748+0000 7f8ff8ff9700 1 -- 172.21.15.28:0/3244590995 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f8ff4085ac0 con 0x7f8ffc15a5a0 2023-11-28T11:10:03.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.748+0000 7f8ffbfff700 1 --2- 172.21.15.28:0/3244590995 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8fe80682f0 0x7f8fe806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:03.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.749+0000 7f8ffbfff700 1 --2- 172.21.15.28:0/3244590995 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8fe80682f0 0x7f8fe806a7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009130 tx=0x7f8fec008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:03.756 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:03.752+0000 7f8ff8ff9700 1 -- 172.21.15.28:0/3244590995 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8ff4052530 con 0x7f8ffc15a5a0 2023-11-28T11:10:04.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.023+0000 7f90022d6700 1 -- 172.21.15.28:0/3244590995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f8ffc0675a0 con 0x7f8ffc15a5a0 2023-11-28T11:10:04.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.024+0000 7f8ff8ff9700 1 -- 172.21.15.28:0/3244590995 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v8) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f8ff401a090 con 0x7f8ffc15a5a0 2023-11-28T11:10:04.026 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:04.026 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:10:04.026 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_dump": [ 2023-11-28T11:10:04.026 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.026 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.a", 2023-11-28T11:10:04.026 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlx1F8HRAAqlv0qwBqCCSPbuXOK00uLQ==", 2023-11-28T11:10:04.026 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.026 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:04.026 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2023-11-28T11:10:04.027 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2023-11-28T11:10:04.027 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:04.027 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.027 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.027 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.027 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.b", 2023-11-28T11:10:04.027 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlqQd7KhAASvfFg9HJbZLZ48lOZ8rfdg==", 2023-11-28T11:10:04.027 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.027 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:04.027 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2023-11-28T11:10:04.027 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2023-11-28T11:10:04.027 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:04.028 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.028 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.028 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.028 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.c", 2023-11-28T11:10:04.028 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlz/TyJBAAug4qbJp/xfFm3aHFNMSlBA==", 2023-11-28T11:10:04.028 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.028 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:04.028 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2023-11-28T11:10:04.028 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2023-11-28T11:10:04.028 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:04.028 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.028 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.029 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.029 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.d", 2023-11-28T11:10:04.029 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlJA0pMhAAgmUFSzlTLDsYT7SPRaS4hg==", 2023-11-28T11:10:04.029 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.029 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:04.029 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2023-11-28T11:10:04.029 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2023-11-28T11:10:04.029 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:04.029 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.029 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.029 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.029 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.0", 2023-11-28T11:10:04.029 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQALymVlGI9xJBAAFY51VVoRxuhhNHGi8tlG5g==", 2023-11-28T11:10:04.030 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.030 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:04.030 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:04.030 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:04.030 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.030 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.030 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.030 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.1", 2023-11-28T11:10:04.030 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAOymVl6bn5IxAAE02o425v2koATSJg/CKx2Q==", 2023-11-28T11:10:04.030 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.030 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:04.030 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:04.031 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:04.031 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.031 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.031 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.031 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.2", 2023-11-28T11:10:04.031 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQARymVl0JCmIxAASw1qp+1l8/G+7uP2t2Gfcg==", 2023-11-28T11:10:04.031 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.031 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:04.031 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:04.031 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:04.031 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.031 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.032 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.032 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.3", 2023-11-28T11:10:04.032 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAUymVlHpKYIxAAeEFLqUE4cWvE29wkIdb4xQ==", 2023-11-28T11:10:04.032 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.032 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:04.032 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:04.032 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:04.032 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.032 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.032 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.032 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.4", 2023-11-28T11:10:04.032 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAcymVlRw7IDBAACbu1rTbZMEVtwM33xdVPSA==", 2023-11-28T11:10:04.032 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.032 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:04.033 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:04.033 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:04.033 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.033 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.033 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.033 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.5", 2023-11-28T11:10:04.033 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAfymVlDGXHDBAA5lGng1DglyCM+DnWCQ0iUg==", 2023-11-28T11:10:04.033 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.033 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:04.033 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:04.033 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:04.033 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.033 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.034 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.034 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.6", 2023-11-28T11:10:04.034 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAiymVlfw+IDBAAX5cXoXUhrY8BX3tZFVyqCA==", 2023-11-28T11:10:04.034 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.034 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:04.034 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:04.034 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:04.034 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.034 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.034 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.034 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.7", 2023-11-28T11:10:04.034 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAlymVliBSsDBAACyShS6zNLoUAlVFbB5XEag==", 2023-11-28T11:10:04.034 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.034 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:04.035 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:04.035 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:04.035 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.035 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.035 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.035 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.0", 2023-11-28T11:10:04.035 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVluSHRNhAAmBrJ4shI0iFZ+s0TazdTrw==", 2023-11-28T11:10:04.035 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.035 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:04.035 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2023-11-28T11:10:04.035 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2023-11-28T11:10:04.035 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2023-11-28T11:10:04.035 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.036 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.036 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.036 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.1", 2023-11-28T11:10:04.036 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAKymVl2+otAhAATGAAuzHz3lKdV0enZTr6qA==", 2023-11-28T11:10:04.036 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.036 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:04.036 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2023-11-28T11:10:04.036 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2023-11-28T11:10:04.036 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2023-11-28T11:10:04.036 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.036 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.036 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.036 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.2", 2023-11-28T11:10:04.036 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlxqlTOhAAU7U9b//feat8r6WDQNx/Gg==", 2023-11-28T11:10:04.037 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.037 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:04.037 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2023-11-28T11:10:04.037 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2023-11-28T11:10:04.037 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2023-11-28T11:10:04.037 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.037 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.037 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.037 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.3", 2023-11-28T11:10:04.037 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAKymVl592XBRAASQZMNs5hbHPm2CYq2ktYdg==", 2023-11-28T11:10:04.037 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.037 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:04.037 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2023-11-28T11:10:04.038 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2023-11-28T11:10:04.038 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2023-11-28T11:10:04.038 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.038 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.038 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.038 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.admin", 2023-11-28T11:10:04.038 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAIymVl5ZzXOhAAt2N6VJgsHFGvT1U6Uee9VA==", 2023-11-28T11:10:04.038 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.038 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2023-11-28T11:10:04.038 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2023-11-28T11:10:04.038 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2023-11-28T11:10:04.038 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:04.039 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.039 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.039 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.039 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-mds", 2023-11-28T11:10:04.039 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQArymVluiH/EhAANby2k2jNz+EEQbyH9Gm73g==", 2023-11-28T11:10:04.039 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.039 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-mds" 2023-11-28T11:10:04.039 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.039 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.039 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.039 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-mgr", 2023-11-28T11:10:04.039 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQArymVl8Sz/EhAAO6ngPiRfmgQPmZlcID3VxA==", 2023-11-28T11:10:04.039 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.039 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-28T11:10:04.040 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.040 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.040 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.040 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-osd", 2023-11-28T11:10:04.040 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQArymVlTDb/EhAA6pb/qPjKsz2ZCFIXXaNT+Q==", 2023-11-28T11:10:04.040 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.040 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-osd" 2023-11-28T11:10:04.040 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.040 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.040 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.040 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-rbd", 2023-11-28T11:10:04.040 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQArymVlfD7/EhAA0BUP6XRZlTRM8E4Iy8EmRg==", 2023-11-28T11:10:04.040 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.040 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-28T11:10:04.041 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.041 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.041 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.041 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-28T11:10:04.041 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQArymVlc0f/EhAAMSubKPCn3lBgGtNbWVcncA==", 2023-11-28T11:10:04.041 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.041 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-28T11:10:04.041 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.041 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.041 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.041 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.bootstrap-rgw", 2023-11-28T11:10:04.041 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQArymVlP1D/EhAA1T3+/26NTtS3DMW/QMpR8Q==", 2023-11-28T11:10:04.041 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.042 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-28T11:10:04.042 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.042 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.042 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.042 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.x", 2023-11-28T11:10:04.042 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlZrj9GRAAXu1z37jFMZo3YXUJLRB2+w==", 2023-11-28T11:10:04.042 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.042 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2023-11-28T11:10:04.042 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2023-11-28T11:10:04.042 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:04.042 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.042 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:04.042 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:04.043 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.y", 2023-11-28T11:10:04.043 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlyBOJFRAA6cjReBkYUD/3YOUHw7DHpg==", 2023-11-28T11:10:04.043 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:04.043 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2023-11-28T11:10:04.043 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2023-11-28T11:10:04.043 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:04.043 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.043 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:04.043 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2023-11-28T11:10:04.043 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:10:04.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.026+0000 7f90022d6700 1 -- 172.21.15.28:0/3244590995 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8fe80682f0 msgr2=0x7f8fe806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:04.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.026+0000 7f90022d6700 1 --2- 172.21.15.28:0/3244590995 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8fe80682f0 0x7f8fe806a7a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f8fec009130 tx=0x7f8fec008030 comp rx=0 tx=0).stop 2023-11-28T11:10:04.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.026+0000 7f90022d6700 1 -- 172.21.15.28:0/3244590995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ffc15a5a0 msgr2=0x7f8ffc0ab8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:04.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.026+0000 7f90022d6700 1 --2- 172.21.15.28:0/3244590995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ffc15a5a0 0x7f8ffc0ab8c0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8ff4007ef0 tx=0x7f8ff400d3b0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.026+0000 7f90022d6700 1 -- 172.21.15.28:0/3244590995 shutdown_connections 2023-11-28T11:10:04.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.026+0000 7f90022d6700 1 --2- 172.21.15.28:0/3244590995 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8fe80682f0 0x7f8fe806a7a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.026+0000 7f90022d6700 1 --2- 172.21.15.28:0/3244590995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ffc15a5a0 0x7f8ffc0ab8c0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.026+0000 7f90022d6700 1 --2- 172.21.15.28:0/3244590995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ffc0ca280 0x7f8ffc1f64a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.026+0000 7f90022d6700 1 --2- 172.21.15.28:0/3244590995 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ffc0cbf40 0x7f8ffc0ceab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.026+0000 7f90022d6700 1 -- 172.21.15.28:0/3244590995 >> 172.21.15.28:0/3244590995 conn(0x7f8ffc0b1650 msgr2=0x7f8ffc15c9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:04.046 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.026+0000 7f90022d6700 1 -- 172.21.15.28:0/3244590995 shutdown_connections 2023-11-28T11:10:04.046 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.026+0000 7f90022d6700 1 -- 172.21.15.28:0/3244590995 wait complete. 2023-11-28T11:10:04.046 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-28T11:10:04.172 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.171+0000 7f86ea891700 1 Processor -- start 2023-11-28T11:10:04.173 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.171+0000 7f86ea891700 1 -- start start 2023-11-28T11:10:04.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86ea891700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f86e40cb230 0x7f86e40ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:04.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86ea891700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f86e40b2660 0x7f86e40b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:04.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86ea891700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86e40cfb90 con 0x7f86e415a6c0 2023-11-28T11:10:04.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86ea891700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e40b3020 con 0x7f86e40cb230 2023-11-28T11:10:04.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86ea891700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86e40cb680 con 0x7f86e40b2660 2023-11-28T11:10:04.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86e37fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f86e40cb230 0x7f86e40ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:04.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86e3fff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f86e415a6c0 0x7f86e40d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37240/0 (socket says 172.21.15.28:37240) 2023-11-28T11:10:04.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86e37fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f86e40cb230 0x7f86e40ccea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49184/0 (socket says 172.21.15.28:49184) 2023-11-28T11:10:04.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86e3fff700 1 -- 172.21.15.28:0/1307332719 learned_addr learned my addr 172.21.15.28:0/1307332719 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:04.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86e37fe700 1 -- 172.21.15.28:0/1307332719 >> v2:172.21.15.59:3300/0 conn(0x7f86e40b2660 msgr2=0x7f86e40b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:04.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86e8e2e700 1 --2- 172.21.15.28:0/1307332719 >> v2:172.21.15.59:3300/0 conn(0x7f86e40b2660 0x7f86e40b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:04.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86e37fe700 1 --2- 172.21.15.28:0/1307332719 >> v2:172.21.15.59:3300/0 conn(0x7f86e40b2660 0x7f86e40b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86e37fe700 1 -- 172.21.15.28:0/1307332719 >> v1:172.21.15.28:6789/0 conn(0x7f86e415a6c0 legacy=0x7f86e40d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:04.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86e2ffd700 1 -- 172.21.15.28:0/1307332719 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3574571904 0 0) 0x7f86e40cfb90 con 0x7f86e415a6c0 2023-11-28T11:10:04.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86e37fe700 1 -- 172.21.15.28:0/1307332719 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86e40cb930 con 0x7f86e40cb230 2023-11-28T11:10:04.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86e8e2e700 1 --2- 172.21.15.28:0/1307332719 >> v2:172.21.15.59:3300/0 conn(0x7f86e40b2660 0x7f86e40b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:04.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86e37fe700 1 --2- 172.21.15.28:0/1307332719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e40cb230 0x7f86e40ccea0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f86e40cb1c0 tx=0x7f86d8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa31e6873b6673ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:04.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.172+0000 7f86e2ffd700 1 -- 172.21.15.28:0/1307332719 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8011830 con 0x7f86e40cb230 2023-11-28T11:10:04.176 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.173+0000 7f86e2ffd700 1 -- 172.21.15.28:0/1307332719 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86d8011990 con 0x7f86e40cb230 2023-11-28T11:10:04.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.173+0000 7f86e2ffd700 1 -- 172.21.15.28:0/1307332719 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86d8011d20 con 0x7f86e40cb230 2023-11-28T11:10:04.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.173+0000 7f86ea891700 1 -- 172.21.15.28:0/1307332719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e40cb230 msgr2=0x7f86e40ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:04.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.173+0000 7f86ea891700 1 --2- 172.21.15.28:0/1307332719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e40cb230 0x7f86e40ccea0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f86e40cb1c0 tx=0x7f86d8009b20 comp rx=0 tx=0).stop 2023-11-28T11:10:04.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.173+0000 7f86ea891700 1 -- 172.21.15.28:0/1307332719 shutdown_connections 2023-11-28T11:10:04.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.173+0000 7f86ea891700 1 --2- 172.21.15.28:0/1307332719 >> v2:172.21.15.59:3300/0 conn(0x7f86e40b2660 0x7f86e40b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.173+0000 7f86ea891700 1 --2- 172.21.15.28:0/1307332719 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e40cb230 0x7f86e40ccea0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.173+0000 7f86ea891700 1 -- 172.21.15.28:0/1307332719 >> 172.21.15.28:0/1307332719 conn(0x7f86e40cd660 msgr2=0x7f86e40cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:04.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.173+0000 7f86ea891700 1 -- 172.21.15.28:0/1307332719 shutdown_connections 2023-11-28T11:10:04.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.173+0000 7f86ea891700 1 -- 172.21.15.28:0/1307332719 wait complete. 2023-11-28T11:10:04.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.174+0000 7f86ea891700 1 Processor -- start 2023-11-28T11:10:04.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.175+0000 7f86ea891700 1 -- start start 2023-11-28T11:10:04.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.175+0000 7f86ea891700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86e40b2660 0x7f86e406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:04.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.175+0000 7f86ea891700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86e40cb230 0x7f86e406cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:04.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.175+0000 7f86ea891700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e415a6c0 0x7f86e41f1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:04.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.175+0000 7f86ea891700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e406d530 con 0x7f86e415a6c0 2023-11-28T11:10:04.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.175+0000 7f86ea891700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86e40ad760 con 0x7f86e40b2660 2023-11-28T11:10:04.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.175+0000 7f86ea891700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86e40ada40 con 0x7f86e40cb230 2023-11-28T11:10:04.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.175+0000 7f86e8e2e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e415a6c0 0x7f86e41f1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:04.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.175+0000 7f86e8e2e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e415a6c0 0x7f86e41f1c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49192/0 (socket says 172.21.15.28:49192) 2023-11-28T11:10:04.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.175+0000 7f86e8e2e700 1 -- 172.21.15.28:0/1766244979 learned_addr learned my addr 172.21.15.28:0/1766244979 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:04.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.176+0000 7f86e37fe700 1 --2- 172.21.15.28:0/1766244979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86e40cb230 0x7f86e406cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:04.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.176+0000 7f86e3fff700 1 --2- 172.21.15.28:0/1766244979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86e40b2660 0x7f86e406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:04.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.176+0000 7f86e8e2e700 1 -- 172.21.15.28:0/1766244979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86e40cb230 msgr2=0x7f86e406cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:04.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.176+0000 7f86e8e2e700 1 --2- 172.21.15.28:0/1766244979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86e40cb230 0x7f86e406cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.176+0000 7f86e8e2e700 1 -- 172.21.15.28:0/1766244979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86e40b2660 msgr2=0x7f86e406c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:04.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.176+0000 7f86e8e2e700 1 --2- 172.21.15.28:0/1766244979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86e40b2660 0x7f86e406c7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.176+0000 7f86e8e2e700 1 -- 172.21.15.28:0/1766244979 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d80094a0 con 0x7f86e415a6c0 2023-11-28T11:10:04.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.176+0000 7f86e3fff700 1 --2- 172.21.15.28:0/1766244979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86e40b2660 0x7f86e406c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.176+0000 7f86e8e2e700 1 --2- 172.21.15.28:0/1766244979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e415a6c0 0x7f86e41f1c90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00cd50 tx=0x7f86dc00a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.176+0000 7f86e0ff9700 1 -- 172.21.15.28:0/1766244979 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc00f480 con 0x7f86e415a6c0 2023-11-28T11:10:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.176+0000 7f86e0ff9700 1 -- 172.21.15.28:0/1766244979 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86dc00f600 con 0x7f86e415a6c0 2023-11-28T11:10:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.176+0000 7f86ea891700 1 -- 172.21.15.28:0/1766244979 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86e40cf540 con 0x7f86e415a6c0 2023-11-28T11:10:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.176+0000 7f86e0ff9700 1 -- 172.21.15.28:0/1766244979 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86dc00f9b0 con 0x7f86e415a6c0 2023-11-28T11:10:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.176+0000 7f86ea891700 1 -- 172.21.15.28:0/1766244979 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86e40cf260 con 0x7f86e415a6c0 2023-11-28T11:10:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.177+0000 7f86ea891700 1 -- 172.21.15.28:0/1766244979 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86e40cf820 con 0x7f86e415a6c0 2023-11-28T11:10:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.178+0000 7f86e0ff9700 1 -- 172.21.15.28:0/1766244979 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f86dc009030 con 0x7f86e415a6c0 2023-11-28T11:10:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.178+0000 7f86e0ff9700 1 --2- 172.21.15.28:0/1766244979 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86d00682f0 0x7f86d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.178+0000 7f86e0ff9700 1 -- 172.21.15.28:0/1766244979 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f86dc086480 con 0x7f86e415a6c0 2023-11-28T11:10:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.178+0000 7f86e3fff700 1 --2- 172.21.15.28:0/1766244979 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86d00682f0 0x7f86d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:04.182 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.178+0000 7f86e3fff700 1 --2- 172.21.15.28:0/1766244979 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86d00682f0 0x7f86d006a7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f86d4000f60 tx=0x7f86d4007ca0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:04.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.182+0000 7f86e0ff9700 1 -- 172.21.15.28:0/1766244979 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f86dc052ef0 con 0x7f86e415a6c0 2023-11-28T11:10:04.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.455+0000 7f86ea891700 1 -- 172.21.15.28:0/1766244979 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f86e40675a0 con 0x7f86e415a6c0 2023-11-28T11:10:04.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.458+0000 7f86e0ff9700 1 -- 172.21.15.28:0/1766244979 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f86dc056510 con 0x7f86e415a6c0 2023-11-28T11:10:04.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.460+0000 7f86ea891700 1 -- 172.21.15.28:0/1766244979 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86d00682f0 msgr2=0x7f86d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:04.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.460+0000 7f86ea891700 1 --2- 172.21.15.28:0/1766244979 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86d00682f0 0x7f86d006a7a0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f86d4000f60 tx=0x7f86d4007ca0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.460+0000 7f86ea891700 1 -- 172.21.15.28:0/1766244979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e415a6c0 msgr2=0x7f86e41f1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:04.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.460+0000 7f86ea891700 1 --2- 172.21.15.28:0/1766244979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e415a6c0 0x7f86e41f1c90 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f86dc00cd50 tx=0x7f86dc00a560 comp rx=0 tx=0).stop 2023-11-28T11:10:04.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.460+0000 7f86ea891700 1 -- 172.21.15.28:0/1766244979 shutdown_connections 2023-11-28T11:10:04.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.460+0000 7f86ea891700 1 --2- 172.21.15.28:0/1766244979 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86d00682f0 0x7f86d006a7a0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.460+0000 7f86ea891700 1 --2- 172.21.15.28:0/1766244979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f86e415a6c0 0x7f86e41f1c90 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.460+0000 7f86ea891700 1 --2- 172.21.15.28:0/1766244979 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86e40b2660 0x7f86e406c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.460+0000 7f86ea891700 1 --2- 172.21.15.28:0/1766244979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f86e40cb230 0x7f86e406cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.460+0000 7f86ea891700 1 -- 172.21.15.28:0/1766244979 >> 172.21.15.28:0/1766244979 conn(0x7f86e40cd660 msgr2=0x7f86e40b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:04.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.460+0000 7f86ea891700 1 -- 172.21.15.28:0/1766244979 shutdown_connections 2023-11-28T11:10:04.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.460+0000 7f86ea891700 1 -- 172.21.15.28:0/1766244979 wait complete. 2023-11-28T11:10:04.474 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-28T11:10:04.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.598+0000 7fe005763700 1 Processor -- start 2023-11-28T11:10:04.600 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.598+0000 7fe005763700 1 -- start start 2023-11-28T11:10:04.601 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.599+0000 7fe005763700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe00006bf30 0x7fe00006c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:04.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.599+0000 7fe005763700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0000b6350 con 0x7fe00015c640 2023-11-28T11:10:04.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.599+0000 7fe005763700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0000d1470 con 0x7fe0000d2820 2023-11-28T11:10:04.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.599+0000 7fe005763700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe00015ca10 con 0x7fe00006bf30 2023-11-28T11:10:04.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.599+0000 7fdfff7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe00006bf30 0x7fe00006c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:04.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.599+0000 7fdffeffd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe00015c640 0x7fe0000cf040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37254/0 (socket says 172.21.15.28:37254) 2023-11-28T11:10:04.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.599+0000 7fdffeffd700 1 -- 172.21.15.28:0/4043577075 learned_addr learned my addr 172.21.15.28:0/4043577075 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:04.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.599+0000 7fdfff7fe700 1 -- 172.21.15.28:0/4043577075 >> v1:172.21.15.59:6789/0 conn(0x7fe0000d2820 legacy=0x7fe0000b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:04.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.599+0000 7fdfff7fe700 1 -- 172.21.15.28:0/4043577075 >> v1:172.21.15.28:6789/0 conn(0x7fe00015c640 legacy=0x7fe0000cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:04.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.599+0000 7fdffdffb700 1 -- 172.21.15.28:0/4043577075 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3662170104 0 0) 0x7fe0000b6350 con 0x7fe00015c640 2023-11-28T11:10:04.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.599+0000 7fdfff7fe700 1 -- 172.21.15.28:0/4043577075 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00006c840 con 0x7fe00006bf30 2023-11-28T11:10:04.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.599+0000 7fdfff7fe700 1 --2- 172.21.15.28:0/4043577075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe00006bf30 0x7fe00006c300 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009d90 tx=0x7fdff800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=144557b9d7efe036 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:04.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.600+0000 7fdffdffb700 1 -- 172.21.15.28:0/4043577075 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff800e070 con 0x7fe00006bf30 2023-11-28T11:10:04.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.600+0000 7fdffdffb700 1 -- 172.21.15.28:0/4043577075 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff800bb90 con 0x7fe00006bf30 2023-11-28T11:10:04.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.600+0000 7fdffdffb700 1 -- 172.21.15.28:0/4043577075 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8003a20 con 0x7fe00006bf30 2023-11-28T11:10:04.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.600+0000 7fe005763700 1 -- 172.21.15.28:0/4043577075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe00006bf30 msgr2=0x7fe00006c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:04.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.600+0000 7fe005763700 1 --2- 172.21.15.28:0/4043577075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe00006bf30 0x7fe00006c300 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fdff8009d90 tx=0x7fdff800b330 comp rx=0 tx=0).stop 2023-11-28T11:10:04.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.600+0000 7fe005763700 1 -- 172.21.15.28:0/4043577075 shutdown_connections 2023-11-28T11:10:04.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.600+0000 7fe005763700 1 --2- 172.21.15.28:0/4043577075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe00006bf30 0x7fe00006c300 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.600+0000 7fe005763700 1 -- 172.21.15.28:0/4043577075 >> 172.21.15.28:0/4043577075 conn(0x7fe0000d2df0 msgr2=0x7fe0000d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:04.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.600+0000 7fe005763700 1 -- 172.21.15.28:0/4043577075 shutdown_connections 2023-11-28T11:10:04.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.600+0000 7fe005763700 1 -- 172.21.15.28:0/4043577075 wait complete. 2023-11-28T11:10:04.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.601+0000 7fe005763700 1 Processor -- start 2023-11-28T11:10:04.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.602+0000 7fe005763700 1 -- start start 2023-11-28T11:10:04.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.602+0000 7fe005763700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe00006bf30 0x7fe00015fa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:04.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.602+0000 7fe005763700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0000d2820 0x7fe0000c6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:04.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.602+0000 7fe005763700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe00015c640 0x7fe00014f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:04.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.602+0000 7fe005763700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0000c8bf0 con 0x7fe00015c640 2023-11-28T11:10:04.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.602+0000 7fe005763700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe000158400 con 0x7fe00006bf30 2023-11-28T11:10:04.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.602+0000 7fe005763700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0000c6b50 con 0x7fe0000d2820 2023-11-28T11:10:04.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.602+0000 7fdfff7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe00015c640 0x7fe00014f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:04.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.602+0000 7fdfff7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe00015c640 0x7fe00014f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49202/0 (socket says 172.21.15.28:49202) 2023-11-28T11:10:04.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.602+0000 7fdfff7fe700 1 -- 172.21.15.28:0/2714776879 learned_addr learned my addr 172.21.15.28:0/2714776879 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:04.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.603+0000 7fdffeffd700 1 --2- 172.21.15.28:0/2714776879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe00006bf30 0x7fe00015fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:04.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.603+0000 7fdffe7fc700 1 --2- 172.21.15.28:0/2714776879 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0000d2820 0x7fe0000c6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:04.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.603+0000 7fdfff7fe700 1 -- 172.21.15.28:0/2714776879 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0000d2820 msgr2=0x7fe0000c6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:04.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.603+0000 7fdfff7fe700 1 --2- 172.21.15.28:0/2714776879 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0000d2820 0x7fe0000c6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.603+0000 7fdfff7fe700 1 -- 172.21.15.28:0/2714776879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe00006bf30 msgr2=0x7fe00015fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:04.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.603+0000 7fdfff7fe700 1 --2- 172.21.15.28:0/2714776879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe00006bf30 0x7fe00015fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.603+0000 7fdfff7fe700 1 -- 172.21.15.28:0/2714776879 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdff8009a70 con 0x7fe00015c640 2023-11-28T11:10:04.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.603+0000 7fdffe7fc700 1 --2- 172.21.15.28:0/2714776879 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0000d2820 0x7fe0000c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:04.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.603+0000 7fdfff7fe700 1 --2- 172.21.15.28:0/2714776879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe00015c640 0x7fe00014f250 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdff8004560 tx=0x7fdff8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:04.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.603+0000 7fdfe77fe700 1 -- 172.21.15.28:0/2714776879 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8012630 con 0x7fe00015c640 2023-11-28T11:10:04.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.603+0000 7fdfe77fe700 1 -- 172.21.15.28:0/2714776879 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdff80127b0 con 0x7fe00015c640 2023-11-28T11:10:04.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.603+0000 7fdfe77fe700 1 -- 172.21.15.28:0/2714776879 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdff8012b60 con 0x7fe00015c640 2023-11-28T11:10:04.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.603+0000 7fe005763700 1 -- 172.21.15.28:0/2714776879 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0000cd7c0 con 0x7fe00015c640 2023-11-28T11:10:04.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.603+0000 7fe005763700 1 -- 172.21.15.28:0/2714776879 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0000cd4e0 con 0x7fe00015c640 2023-11-28T11:10:04.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.604+0000 7fe005763700 1 -- 172.21.15.28:0/2714776879 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0000d1710 con 0x7fe00015c640 2023-11-28T11:10:04.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.604+0000 7fdfe77fe700 1 -- 172.21.15.28:0/2714776879 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdff8012cc0 con 0x7fe00015c640 2023-11-28T11:10:04.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.605+0000 7fdfe77fe700 1 --2- 172.21.15.28:0/2714776879 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdfec0682f0 0x7fdfec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:04.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.605+0000 7fdfe77fe700 1 -- 172.21.15.28:0/2714776879 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fdff8087b40 con 0x7fe00015c640 2023-11-28T11:10:04.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.605+0000 7fdffeffd700 1 --2- 172.21.15.28:0/2714776879 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdfec0682f0 0x7fdfec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:04.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.605+0000 7fdffeffd700 1 --2- 172.21.15.28:0/2714776879 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdfec0682f0 0x7fdfec06a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdff4000f60 tx=0x7fdff4007e70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:04.610 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.609+0000 7fdfe77fe700 1 -- 172.21.15.28:0/2714776879 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdff80544c0 con 0x7fe00015c640 2023-11-28T11:10:04.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.880+0000 7fe005763700 1 -- 172.21.15.28:0/2714776879 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7fe0000c8d30 con 0x7fe00015c640 2023-11-28T11:10:04.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.884+0000 7fdfe77fe700 1 -- 172.21.15.28:0/2714776879 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7fe0000c8d30 con 0x7fe00015c640 2023-11-28T11:10:04.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.886+0000 7fe005763700 1 -- 172.21.15.28:0/2714776879 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdfec0682f0 msgr2=0x7fdfec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:04.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.886+0000 7fe005763700 1 --2- 172.21.15.28:0/2714776879 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdfec0682f0 0x7fdfec06a7a0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdff4000f60 tx=0x7fdff4007e70 comp rx=0 tx=0).stop 2023-11-28T11:10:04.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.886+0000 7fe005763700 1 -- 172.21.15.28:0/2714776879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe00015c640 msgr2=0x7fe00014f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:04.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.886+0000 7fe005763700 1 --2- 172.21.15.28:0/2714776879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe00015c640 0x7fe00014f250 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fdff8004560 tx=0x7fdff8004ad0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.886+0000 7fe005763700 1 -- 172.21.15.28:0/2714776879 shutdown_connections 2023-11-28T11:10:04.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.886+0000 7fe005763700 1 --2- 172.21.15.28:0/2714776879 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdfec0682f0 0x7fdfec06a7a0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.886+0000 7fe005763700 1 --2- 172.21.15.28:0/2714776879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe00015c640 0x7fe00014f250 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.886+0000 7fe005763700 1 --2- 172.21.15.28:0/2714776879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe00006bf30 0x7fe00015fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.886+0000 7fe005763700 1 --2- 172.21.15.28:0/2714776879 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0000d2820 0x7fe0000c6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:04.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.886+0000 7fe005763700 1 -- 172.21.15.28:0/2714776879 >> 172.21.15.28:0/2714776879 conn(0x7fe0000d2df0 msgr2=0x7fe00015a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:04.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.886+0000 7fe005763700 1 -- 172.21.15.28:0/2714776879 shutdown_connections 2023-11-28T11:10:04.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:04.886+0000 7fe005763700 1 -- 172.21.15.28:0/2714776879 wait complete. 2023-11-28T11:10:04.899 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-28T11:10:05.024 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.023+0000 7fcf2afb8700 1 Processor -- start 2023-11-28T11:10:05.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.023+0000 7fcf2afb8700 1 -- start start 2023-11-28T11:10:05.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.024+0000 7fcf2afb8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf2406e400 0x7fcf2415db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.024+0000 7fcf2afb8700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2415e0b0 con 0x7fcf24159590 2023-11-28T11:10:05.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.024+0000 7fcf2afb8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf2415d120 con 0x7fcf240ce890 2023-11-28T11:10:05.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.024+0000 7fcf2afb8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf2415d6e0 con 0x7fcf2406e400 2023-11-28T11:10:05.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.024+0000 7fcf28d54700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fcf24159590 0x7fcf240ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37266/0 (socket says 172.21.15.28:37266) 2023-11-28T11:10:05.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.024+0000 7fcf28d54700 1 -- 172.21.15.28:0/349205661 learned_addr learned my addr 172.21.15.28:0/349205661 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:05.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.024+0000 7fcf23fff700 1 --2- 172.21.15.28:0/349205661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf2406e400 0x7fcf2415db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.024+0000 7fcf237fe700 1 -- 172.21.15.28:0/349205661 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2897666986 0 0) 0x7fcf18001110 con 0x7fcf24159590 2023-11-28T11:10:05.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.024+0000 7fcf237fe700 1 -- 172.21.15.28:0/349205661 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf140035b0 con 0x7fcf24159590 2023-11-28T11:10:05.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.024+0000 7fcf237fe700 1 -- 172.21.15.28:0/349205661 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1601022119 0 0) 0x7fcf2415d120 con 0x7fcf240ce890 2023-11-28T11:10:05.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.024+0000 7fcf237fe700 1 -- 172.21.15.28:0/349205661 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf18001110 con 0x7fcf240ce890 2023-11-28T11:10:05.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.024+0000 7fcf23fff700 1 -- 172.21.15.28:0/349205661 >> v1:172.21.15.59:6789/0 conn(0x7fcf240ce890 legacy=0x7fcf240caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.024+0000 7fcf23fff700 1 -- 172.21.15.28:0/349205661 >> v1:172.21.15.28:6789/0 conn(0x7fcf24159590 legacy=0x7fcf240ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.024+0000 7fcf23fff700 1 -- 172.21.15.28:0/349205661 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf240aef60 con 0x7fcf2406e400 2023-11-28T11:10:05.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.025+0000 7fcf23fff700 1 --2- 172.21.15.28:0/349205661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf2406e400 0x7fcf2415db70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009870 tx=0x7fcf10009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c5caa4a35ed65b7d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:05.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.025+0000 7fcf237fe700 1 -- 172.21.15.28:0/349205661 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10011830 con 0x7fcf2406e400 2023-11-28T11:10:05.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.025+0000 7fcf237fe700 1 -- 172.21.15.28:0/349205661 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf10011990 con 0x7fcf2406e400 2023-11-28T11:10:05.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.025+0000 7fcf237fe700 1 -- 172.21.15.28:0/349205661 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf10011d20 con 0x7fcf2406e400 2023-11-28T11:10:05.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.025+0000 7fcf2afb8700 1 -- 172.21.15.28:0/349205661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf2406e400 msgr2=0x7fcf2415db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.025+0000 7fcf2afb8700 1 --2- 172.21.15.28:0/349205661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf2406e400 0x7fcf2415db70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcf10009870 tx=0x7fcf10009b80 comp rx=0 tx=0).stop 2023-11-28T11:10:05.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.025+0000 7fcf2afb8700 1 -- 172.21.15.28:0/349205661 shutdown_connections 2023-11-28T11:10:05.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.025+0000 7fcf2afb8700 1 --2- 172.21.15.28:0/349205661 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf2406e400 0x7fcf2415db70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.025+0000 7fcf2afb8700 1 -- 172.21.15.28:0/349205661 >> 172.21.15.28:0/349205661 conn(0x7fcf240b1650 msgr2=0x7fcf240b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:05.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.025+0000 7fcf2afb8700 1 -- 172.21.15.28:0/349205661 shutdown_connections 2023-11-28T11:10:05.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.025+0000 7fcf2afb8700 1 -- 172.21.15.28:0/349205661 wait complete. 2023-11-28T11:10:05.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.026+0000 7fcf2afb8700 1 Processor -- start 2023-11-28T11:10:05.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.027+0000 7fcf2afb8700 1 -- start start 2023-11-28T11:10:05.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.027+0000 7fcf2afb8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf2406e400 0x7fcf240ad450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.027+0000 7fcf2afb8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf240ce890 0x7fcf241559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.027+0000 7fcf2afb8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24159590 0x7fcf240af480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.027+0000 7fcf2afb8700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf240ada30 con 0x7fcf2406e400 2023-11-28T11:10:05.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.027+0000 7fcf2afb8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf240c8320 con 0x7fcf24159590 2023-11-28T11:10:05.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.027+0000 7fcf2afb8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf241581d0 con 0x7fcf240ce890 2023-11-28T11:10:05.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.027+0000 7fcf28d54700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf2406e400 0x7fcf240ad450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf28d54700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf2406e400 0x7fcf240ad450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49214/0 (socket says 172.21.15.28:49214) 2023-11-28T11:10:05.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf28d54700 1 -- 172.21.15.28:0/2607491694 learned_addr learned my addr 172.21.15.28:0/2607491694 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:05.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf23fff700 1 --2- 172.21.15.28:0/2607491694 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf240ce890 0x7fcf241559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf29555700 1 --2- 172.21.15.28:0/2607491694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24159590 0x7fcf240af480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf28d54700 1 -- 172.21.15.28:0/2607491694 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf240ce890 msgr2=0x7fcf241559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf28d54700 1 --2- 172.21.15.28:0/2607491694 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf240ce890 0x7fcf241559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf28d54700 1 -- 172.21.15.28:0/2607491694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24159590 msgr2=0x7fcf240af480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf28d54700 1 --2- 172.21.15.28:0/2607491694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24159590 0x7fcf240af480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf28d54700 1 -- 172.21.15.28:0/2607491694 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf100094a0 con 0x7fcf2406e400 2023-11-28T11:10:05.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf29555700 1 --2- 172.21.15.28:0/2607491694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24159590 0x7fcf240af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:05.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf28d54700 1 --2- 172.21.15.28:0/2607491694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf2406e400 0x7fcf240ad450 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcf18001eb0 tx=0x7fcf1800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:05.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf217fa700 1 -- 172.21.15.28:0/2607491694 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18005d70 con 0x7fcf2406e400 2023-11-28T11:10:05.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf217fa700 1 -- 172.21.15.28:0/2607491694 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf18007070 con 0x7fcf2406e400 2023-11-28T11:10:05.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf2afb8700 1 -- 172.21.15.28:0/2607491694 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf2415d370 con 0x7fcf2406e400 2023-11-28T11:10:05.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf217fa700 1 -- 172.21.15.28:0/2607491694 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf18007400 con 0x7fcf2406e400 2023-11-28T11:10:05.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.028+0000 7fcf2afb8700 1 -- 172.21.15.28:0/2607491694 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf240af240 con 0x7fcf2406e400 2023-11-28T11:10:05.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.029+0000 7fcf2afb8700 1 -- 172.21.15.28:0/2607491694 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf24159960 con 0x7fcf2406e400 2023-11-28T11:10:05.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.030+0000 7fcf217fa700 1 -- 172.21.15.28:0/2607491694 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcf18019030 con 0x7fcf2406e400 2023-11-28T11:10:05.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.030+0000 7fcf217fa700 1 --2- 172.21.15.28:0/2607491694 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcf080682f0 0x7fcf0806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.030+0000 7fcf23fff700 1 --2- 172.21.15.28:0/2607491694 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcf080682f0 0x7fcf0806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.030+0000 7fcf217fa700 1 -- 172.21.15.28:0/2607491694 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fcf18084d40 con 0x7fcf2406e400 2023-11-28T11:10:05.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.031+0000 7fcf23fff700 1 --2- 172.21.15.28:0/2607491694 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcf080682f0 0x7fcf0806a7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcf100097c0 tx=0x7fcf1001c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:05.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.035+0000 7fcf217fa700 1 -- 172.21.15.28:0/2607491694 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcf180517b0 con 0x7fcf2406e400 2023-11-28T11:10:05.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.307+0000 7fcf2afb8700 1 -- 172.21.15.28:0/2607491694 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7fcf240c8180 con 0x7fcf2406e400 2023-11-28T11:10:05.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.311+0000 7fcf217fa700 1 -- 172.21.15.28:0/2607491694 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v11) v1 ==== 90+0+0 (secure 0 0 0) 0x7fcf240c8180 con 0x7fcf2406e400 2023-11-28T11:10:05.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.313+0000 7fcf2afb8700 1 -- 172.21.15.28:0/2607491694 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcf080682f0 msgr2=0x7fcf0806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.313+0000 7fcf2afb8700 1 --2- 172.21.15.28:0/2607491694 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcf080682f0 0x7fcf0806a7a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fcf100097c0 tx=0x7fcf1001c040 comp rx=0 tx=0).stop 2023-11-28T11:10:05.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.313+0000 7fcf2afb8700 1 -- 172.21.15.28:0/2607491694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf2406e400 msgr2=0x7fcf240ad450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.313+0000 7fcf2afb8700 1 --2- 172.21.15.28:0/2607491694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf2406e400 0x7fcf240ad450 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcf18001eb0 tx=0x7fcf1800c9e0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.313+0000 7fcf2afb8700 1 -- 172.21.15.28:0/2607491694 shutdown_connections 2023-11-28T11:10:05.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.313+0000 7fcf2afb8700 1 --2- 172.21.15.28:0/2607491694 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcf080682f0 0x7fcf0806a7a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.313+0000 7fcf2afb8700 1 --2- 172.21.15.28:0/2607491694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf2406e400 0x7fcf240ad450 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.313+0000 7fcf2afb8700 1 --2- 172.21.15.28:0/2607491694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24159590 0x7fcf240af480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.313+0000 7fcf2afb8700 1 --2- 172.21.15.28:0/2607491694 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf240ce890 0x7fcf241559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.313+0000 7fcf2afb8700 1 -- 172.21.15.28:0/2607491694 >> 172.21.15.28:0/2607491694 conn(0x7fcf240b1650 msgr2=0x7fcf2415bbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:05.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.313+0000 7fcf2afb8700 1 -- 172.21.15.28:0/2607491694 shutdown_connections 2023-11-28T11:10:05.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.313+0000 7fcf2afb8700 1 -- 172.21.15.28:0/2607491694 wait complete. 2023-11-28T11:10:05.326 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-28T11:10:05.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.451+0000 7f01a3138700 1 Processor -- start 2023-11-28T11:10:05.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.451+0000 7f01a3138700 1 -- start start 2023-11-28T11:10:05.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.452+0000 7f01a3138700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f019c0cb470 0x7f019c0ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.452+0000 7f01a3138700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019c06c140 con 0x7f019c06bb80 2023-11-28T11:10:05.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.452+0000 7f01a3138700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f019c0c9ab0 con 0x7f019c0cb0a0 2023-11-28T11:10:05.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.452+0000 7f01a3138700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019c0caad0 con 0x7f019c0cb470 2023-11-28T11:10:05.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.452+0000 7f01a16d5700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f019c06bb80 0x7f019c14fea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37284/0 (socket says 172.21.15.28:37284) 2023-11-28T11:10:05.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.452+0000 7f01a16d5700 1 -- 172.21.15.28:0/1389479963 learned_addr learned my addr 172.21.15.28:0/1389479963 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:05.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.452+0000 7f019bfff700 1 --2- 172.21.15.28:0/1389479963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f019c0cb470 0x7f019c0ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.452+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1328195420 0 0) 0x7f019c06c140 con 0x7f019c06bb80 2023-11-28T11:10:05.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.452+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f018c0035b0 con 0x7f019c06bb80 2023-11-28T11:10:05.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2451793122 0 0) 0x7f019c0c9ab0 con 0x7f019c0cb0a0 2023-11-28T11:10:05.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f019c06c140 con 0x7f019c0cb0a0 2023-11-28T11:10:05.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1239456355 0 0) 0x7f018c0035b0 con 0x7f019c06bb80 2023-11-28T11:10:05.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f019c0cb470 msgr2=0x7f019c0ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:05.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019b7fe700 1 --2- 172.21.15.28:0/1389479963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f019c0cb470 0x7f019c0ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 >> v1:172.21.15.59:6789/0 conn(0x7f019c0cb0a0 legacy=0x7f019c15a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c0c9d90 con 0x7f019c06bb80 2023-11-28T11:10:05.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f01940019f0 con 0x7f019c06bb80 2023-11-28T11:10:05.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 >> v1:172.21.15.28:6789/0 conn(0x7f019c06bb80 legacy=0x7f019c14fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019b7fe700 1 --2- 172.21.15.28:0/1389479963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f018c003bc0 0x7f018c003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019b7fe700 1 --2- 172.21.15.28:0/1389479963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f018c004850 0x7f018c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019b7fe700 1 --2- 172.21.15.28:0/1389479963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f019c0cb470 0x7f018c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c00bac0 con 0x7f018c003bc0 2023-11-28T11:10:05.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018c00bc30 con 0x7f018c004850 2023-11-28T11:10:05.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018c00bdf0 con 0x7f019c0cb470 2023-11-28T11:10:05.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f01a0ed4700 1 --2- 172.21.15.28:0/1389479963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f018c003bc0 0x7f018c003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f019bfff700 1 --2- 172.21.15.28:0/1389479963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f018c004850 0x7f018c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f01a16d5700 1 --2- 172.21.15.28:0/1389479963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f019c0cb470 0x7f018c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f01a0ed4700 1 -- 172.21.15.28:0/1389479963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f019c0cb470 msgr2=0x7f018c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:05.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f01a0ed4700 1 --2- 172.21.15.28:0/1389479963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f019c0cb470 0x7f018c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f01a0ed4700 1 -- 172.21.15.28:0/1389479963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f018c004850 msgr2=0x7f018c004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:05.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f01a0ed4700 1 --2- 172.21.15.28:0/1389479963 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f018c004850 0x7f018c004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f01a0ed4700 1 -- 172.21.15.28:0/1389479963 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f018c0035b0 con 0x7f018c003bc0 2023-11-28T11:10:05.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f01a16d5700 1 --2- 172.21.15.28:0/1389479963 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f019c0cb470 0x7f018c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:05.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.453+0000 7f01a0ed4700 1 --2- 172.21.15.28:0/1389479963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f018c003bc0 0x7f018c003fb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0190001940 tx=0x7f018c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d72be1d427427221 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:05.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.454+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190001430 con 0x7f018c003bc0 2023-11-28T11:10:05.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.454+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0190001d70 con 0x7f018c003bc0 2023-11-28T11:10:05.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.454+0000 7f019b7fe700 1 -- 172.21.15.28:0/1389479963 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190005510 con 0x7f018c003bc0 2023-11-28T11:10:05.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.454+0000 7f01a3138700 1 -- 172.21.15.28:0/1389479963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f018c003bc0 msgr2=0x7f018c003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.454+0000 7f01a3138700 1 --2- 172.21.15.28:0/1389479963 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f018c003bc0 0x7f018c003fb0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0190001940 tx=0x7f018c005880 comp rx=0 tx=0).stop 2023-11-28T11:10:05.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.454+0000 7f01a16d5700 1 -- 172.21.15.28:0/1389479963 reap_dead start 2023-11-28T11:10:05.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.454+0000 7f01a3138700 1 -- 172.21.15.28:0/1389479963 shutdown_connections 2023-11-28T11:10:05.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.454+0000 7f01a3138700 1 -- 172.21.15.28:0/1389479963 >> 172.21.15.28:0/1389479963 conn(0x7f019c0b0dd0 msgr2=0x7f019c0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:05.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.454+0000 7f01a3138700 1 -- 172.21.15.28:0/1389479963 shutdown_connections 2023-11-28T11:10:05.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.454+0000 7f01a3138700 1 -- 172.21.15.28:0/1389479963 wait complete. 2023-11-28T11:10:05.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.455+0000 7f01a3138700 1 Processor -- start 2023-11-28T11:10:05.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.455+0000 7f01a3138700 1 -- start start 2023-11-28T11:10:05.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.456+0000 7f01a3138700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f018c003bc0 0x7f019c140380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.456+0000 7f01a3138700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f019c1ef700 0x7f019c0b4320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.456+0000 7f01a3138700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f019c0b4860 0x7f019c1423d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.456+0000 7f01a3138700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019c143400 con 0x7f018c003bc0 2023-11-28T11:10:05.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.456+0000 7f01a3138700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f019c0b4ce0 con 0x7f019c1ef700 2023-11-28T11:10:05.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.456+0000 7f01a3138700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f019c0c7b70 con 0x7f019c0b4860 2023-11-28T11:10:05.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.456+0000 7f01a0ed4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f018c003bc0 0x7f019c140380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.456+0000 7f01a0ed4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f018c003bc0 0x7f019c140380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49234/0 (socket says 172.21.15.28:49234) 2023-11-28T11:10:05.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.456+0000 7f01a0ed4700 1 -- 172.21.15.28:0/1700974594 learned_addr learned my addr 172.21.15.28:0/1700974594 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:05.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.456+0000 7f019bfff700 1 --2- 172.21.15.28:0/1700974594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f019c1ef700 0x7f019c0b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.457+0000 7f01a16d5700 1 --2- 172.21.15.28:0/1700974594 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f019c0b4860 0x7f019c1423d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.457+0000 7f01a0ed4700 1 -- 172.21.15.28:0/1700974594 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f019c0b4860 msgr2=0x7f019c1423d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.457+0000 7f01a0ed4700 1 --2- 172.21.15.28:0/1700974594 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f019c0b4860 0x7f019c1423d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.457+0000 7f01a0ed4700 1 -- 172.21.15.28:0/1700974594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f019c1ef700 msgr2=0x7f019c0b4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.457+0000 7f01a0ed4700 1 --2- 172.21.15.28:0/1700974594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f019c1ef700 0x7f019c0b4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.457+0000 7f01a0ed4700 1 -- 172.21.15.28:0/1700974594 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f019c1505b0 con 0x7f018c003bc0 2023-11-28T11:10:05.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.457+0000 7f01a16d5700 1 --2- 172.21.15.28:0/1700974594 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f019c0b4860 0x7f019c1423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:05.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.457+0000 7f01a0ed4700 1 --2- 172.21.15.28:0/1700974594 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f018c003bc0 0x7f019c140380 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0190004000 tx=0x7f01900012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:05.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.457+0000 7f01997fa700 1 -- 172.21.15.28:0/1700974594 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190006c70 con 0x7f018c003bc0 2023-11-28T11:10:05.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.457+0000 7f01997fa700 1 -- 172.21.15.28:0/1700974594 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0190006dd0 con 0x7f018c003bc0 2023-11-28T11:10:05.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.457+0000 7f01997fa700 1 -- 172.21.15.28:0/1700974594 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0190007240 con 0x7f018c003bc0 2023-11-28T11:10:05.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.457+0000 7f01a3138700 1 -- 172.21.15.28:0/1700974594 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f019c0ca070 con 0x7f018c003bc0 2023-11-28T11:10:05.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.457+0000 7f01a3138700 1 -- 172.21.15.28:0/1700974594 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f019c150740 con 0x7f018c003bc0 2023-11-28T11:10:05.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.458+0000 7f01a3138700 1 -- 172.21.15.28:0/1700974594 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f019c0c8540 con 0x7f018c003bc0 2023-11-28T11:10:05.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.458+0000 7f01997fa700 1 -- 172.21.15.28:0/1700974594 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f019002d070 con 0x7f018c003bc0 2023-11-28T11:10:05.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.459+0000 7f01997fa700 1 --2- 172.21.15.28:0/1700974594 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01800682f0 0x7f018006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.459+0000 7f01997fa700 1 -- 172.21.15.28:0/1700974594 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f0190086720 con 0x7f018c003bc0 2023-11-28T11:10:05.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.459+0000 7f019bfff700 1 --2- 172.21.15.28:0/1700974594 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01800682f0 0x7f018006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.459+0000 7f019bfff700 1 --2- 172.21.15.28:0/1700974594 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01800682f0 0x7f018006a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0188009c60 tx=0x7f0188009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:05.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.463+0000 7f01997fa700 1 -- 172.21.15.28:0/1700974594 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0190053190 con 0x7f018c003bc0 2023-11-28T11:10:05.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.737+0000 7f01a3138700 1 -- 172.21.15.28:0/1700974594 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f019c0675a0 con 0x7f018c003bc0 2023-11-28T11:10:05.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.741+0000 7f01997fa700 1 -- 172.21.15.28:0/1700974594 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f01900567b0 con 0x7f018c003bc0 2023-11-28T11:10:05.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.743+0000 7f01a3138700 1 -- 172.21.15.28:0/1700974594 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01800682f0 msgr2=0x7f018006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.743+0000 7f01a3138700 1 --2- 172.21.15.28:0/1700974594 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01800682f0 0x7f018006a7a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f0188009c60 tx=0x7f0188009210 comp rx=0 tx=0).stop 2023-11-28T11:10:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.743+0000 7f01a3138700 1 -- 172.21.15.28:0/1700974594 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f018c003bc0 msgr2=0x7f019c140380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.743+0000 7f01a3138700 1 --2- 172.21.15.28:0/1700974594 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f018c003bc0 0x7f019c140380 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0190004000 tx=0x7f01900012d0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.743+0000 7f01a3138700 1 -- 172.21.15.28:0/1700974594 shutdown_connections 2023-11-28T11:10:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.743+0000 7f01a3138700 1 --2- 172.21.15.28:0/1700974594 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01800682f0 0x7f018006a7a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.743+0000 7f01a3138700 1 --2- 172.21.15.28:0/1700974594 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f018c003bc0 0x7f019c140380 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.743+0000 7f01a3138700 1 --2- 172.21.15.28:0/1700974594 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f019c0b4860 0x7f019c1423d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.743+0000 7f01a3138700 1 --2- 172.21.15.28:0/1700974594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f019c1ef700 0x7f019c0b4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.743+0000 7f01a3138700 1 -- 172.21.15.28:0/1700974594 >> 172.21.15.28:0/1700974594 conn(0x7f019c0b0dd0 msgr2=0x7f019c0ce8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.743+0000 7f01a3138700 1 -- 172.21.15.28:0/1700974594 shutdown_connections 2023-11-28T11:10:05.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.743+0000 7f01a3138700 1 -- 172.21.15.28:0/1700974594 wait complete. 2023-11-28T11:10:05.757 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-28T11:10:05.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.881+0000 7f9478df8700 1 Processor -- start 2023-11-28T11:10:05.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.881+0000 7f9478df8700 1 -- start start 2023-11-28T11:10:05.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.881+0000 7f9478df8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f947415c990 0x7f94740cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.882+0000 7f9478df8700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94740ce340 con 0x7f94740cc310 2023-11-28T11:10:05.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.882+0000 7f9478df8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94740cebe0 con 0x7f9474161090 2023-11-28T11:10:05.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.882+0000 7f9478df8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94740cb3f0 con 0x7f947415c990 2023-11-28T11:10:05.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.882+0000 7f9472d9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f94740cc310 0x7f94740cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37304/0 (socket says 172.21.15.28:37304) 2023-11-28T11:10:05.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.882+0000 7f9472d9d700 1 -- 172.21.15.28:0/2235786764 learned_addr learned my addr 172.21.15.28:0/2235786764 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:05.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.882+0000 7f947259c700 1 --2- 172.21.15.28:0/2235786764 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f947415c990 0x7f94740cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.882+0000 7f947159a700 1 -- 172.21.15.28:0/2235786764 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 370762672 0 0) 0x7f94740ce340 con 0x7f94740cc310 2023-11-28T11:10:05.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.882+0000 7f947159a700 1 -- 172.21.15.28:0/2235786764 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f945c0035b0 con 0x7f94740cc310 2023-11-28T11:10:05.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.882+0000 7f947159a700 1 -- 172.21.15.28:0/2235786764 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1060146947 0 0) 0x7f94740cebe0 con 0x7f9474161090 2023-11-28T11:10:05.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.882+0000 7f947159a700 1 -- 172.21.15.28:0/2235786764 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94740ce340 con 0x7f9474161090 2023-11-28T11:10:05.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.882+0000 7f947259c700 1 -- 172.21.15.28:0/2235786764 >> v1:172.21.15.59:6789/0 conn(0x7f9474161090 legacy=0x7f94740ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.882+0000 7f947259c700 1 -- 172.21.15.28:0/2235786764 >> v1:172.21.15.28:6789/0 conn(0x7f94740cc310 legacy=0x7f94740cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.882+0000 7f947259c700 1 -- 172.21.15.28:0/2235786764 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94740ce590 con 0x7f947415c990 2023-11-28T11:10:05.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.883+0000 7f947259c700 1 --2- 172.21.15.28:0/2235786764 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f947415c990 0x7f94740cf070 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9464009870 tx=0x7f9464009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2aca3e025c31e36a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:05.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.883+0000 7f947159a700 1 -- 172.21.15.28:0/2235786764 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464011830 con 0x7f947415c990 2023-11-28T11:10:05.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.883+0000 7f947159a700 1 -- 172.21.15.28:0/2235786764 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9464011990 con 0x7f947415c990 2023-11-28T11:10:05.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.883+0000 7f947159a700 1 -- 172.21.15.28:0/2235786764 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9464011d20 con 0x7f947415c990 2023-11-28T11:10:05.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.883+0000 7f9478df8700 1 -- 172.21.15.28:0/2235786764 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f947415c990 msgr2=0x7f94740cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.883+0000 7f9478df8700 1 --2- 172.21.15.28:0/2235786764 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f947415c990 0x7f94740cf070 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9464009870 tx=0x7f9464009b80 comp rx=0 tx=0).stop 2023-11-28T11:10:05.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.883+0000 7f9478df8700 1 -- 172.21.15.28:0/2235786764 shutdown_connections 2023-11-28T11:10:05.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.883+0000 7f9478df8700 1 --2- 172.21.15.28:0/2235786764 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f947415c990 0x7f94740cf070 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.883+0000 7f9478df8700 1 -- 172.21.15.28:0/2235786764 >> 172.21.15.28:0/2235786764 conn(0x7f94740b1650 msgr2=0x7f94740b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:05.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.883+0000 7f9478df8700 1 -- 172.21.15.28:0/2235786764 shutdown_connections 2023-11-28T11:10:05.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.884+0000 7f9478df8700 1 -- 172.21.15.28:0/2235786764 wait complete. 2023-11-28T11:10:05.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.885+0000 7f9478df8700 1 Processor -- start 2023-11-28T11:10:05.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.885+0000 7f9478df8700 1 -- start start 2023-11-28T11:10:05.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f9478df8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94740cc310 0x7f947415a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f9478df8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f947415c990 0x7f94740b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f9478df8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9474161090 0x7f94740b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f9478df8700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94740abc10 con 0x7f94740cc310 2023-11-28T11:10:05.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f9478df8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94740ab8c0 con 0x7f9474161090 2023-11-28T11:10:05.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f9478df8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947415a590 con 0x7f947415c990 2023-11-28T11:10:05.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f947259c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94740cc310 0x7f947415a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f947259c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94740cc310 0x7f947415a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49248/0 (socket says 172.21.15.28:49248) 2023-11-28T11:10:05.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f947259c700 1 -- 172.21.15.28:0/4229690013 learned_addr learned my addr 172.21.15.28:0/4229690013 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:05.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f9471d9b700 1 --2- 172.21.15.28:0/4229690013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f947415c990 0x7f94740b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f9472d9d700 1 --2- 172.21.15.28:0/4229690013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9474161090 0x7f94740b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f947259c700 1 -- 172.21.15.28:0/4229690013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f947415c990 msgr2=0x7f94740b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f947259c700 1 --2- 172.21.15.28:0/4229690013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f947415c990 0x7f94740b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f947259c700 1 -- 172.21.15.28:0/4229690013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9474161090 msgr2=0x7f94740b4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:05.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f947259c700 1 --2- 172.21.15.28:0/4229690013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9474161090 0x7f94740b4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:05.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f947259c700 1 -- 172.21.15.28:0/4229690013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94640094a0 con 0x7f94740cc310 2023-11-28T11:10:05.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f947259c700 1 --2- 172.21.15.28:0/4229690013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94740cc310 0x7f947415a050 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f94640090b0 tx=0x7f9464004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:05.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.886+0000 7f9462ffd700 1 -- 172.21.15.28:0/4229690013 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946401a6e0 con 0x7f94740cc310 2023-11-28T11:10:05.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.887+0000 7f9462ffd700 1 -- 172.21.15.28:0/4229690013 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946401a840 con 0x7f94740cc310 2023-11-28T11:10:05.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.887+0000 7f9478df8700 1 -- 172.21.15.28:0/4229690013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94740cb080 con 0x7f94740cc310 2023-11-28T11:10:05.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.887+0000 7f9462ffd700 1 -- 172.21.15.28:0/4229690013 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946401abb0 con 0x7f94740cc310 2023-11-28T11:10:05.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.887+0000 7f9478df8700 1 -- 172.21.15.28:0/4229690013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94740ce870 con 0x7f94740cc310 2023-11-28T11:10:05.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.887+0000 7f9478df8700 1 -- 172.21.15.28:0/4229690013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94740cf5b0 con 0x7f94740cc310 2023-11-28T11:10:05.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.888+0000 7f9462ffd700 1 -- 172.21.15.28:0/4229690013 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f946402b030 con 0x7f94740cc310 2023-11-28T11:10:05.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.888+0000 7f9462ffd700 1 --2- 172.21.15.28:0/4229690013 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94540682f0 0x7f945406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:05.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.888+0000 7f9462ffd700 1 -- 172.21.15.28:0/4229690013 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f9464086b60 con 0x7f94740cc310 2023-11-28T11:10:05.897 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.888+0000 7f9471d9b700 1 --2- 172.21.15.28:0/4229690013 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94540682f0 0x7f945406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:05.897 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.889+0000 7f9471d9b700 1 --2- 172.21.15.28:0/4229690013 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94540682f0 0x7f945406a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f947415bc30 tx=0x7f9468009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:05.897 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:05.893+0000 7f9462ffd700 1 -- 172.21.15.28:0/4229690013 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f94640535d0 con 0x7f94740cc310 2023-11-28T11:10:06.168 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.166+0000 7f9478df8700 1 -- 172.21.15.28:0/4229690013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f94740675a0 con 0x7f94740cc310 2023-11-28T11:10:06.171 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.170+0000 7f9462ffd700 1 -- 172.21.15.28:0/4229690013 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v13) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9464029020 con 0x7f94740cc310 2023-11-28T11:10:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.172+0000 7f9478df8700 1 -- 172.21.15.28:0/4229690013 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94540682f0 msgr2=0x7f945406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.172+0000 7f9478df8700 1 --2- 172.21.15.28:0/4229690013 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94540682f0 0x7f945406a7a0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f947415bc30 tx=0x7f9468009040 comp rx=0 tx=0).stop 2023-11-28T11:10:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.172+0000 7f9478df8700 1 -- 172.21.15.28:0/4229690013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94740cc310 msgr2=0x7f947415a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.172+0000 7f9478df8700 1 --2- 172.21.15.28:0/4229690013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94740cc310 0x7f947415a050 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f94640090b0 tx=0x7f9464004750 comp rx=0 tx=0).stop 2023-11-28T11:10:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.172+0000 7f9478df8700 1 -- 172.21.15.28:0/4229690013 shutdown_connections 2023-11-28T11:10:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.172+0000 7f9478df8700 1 --2- 172.21.15.28:0/4229690013 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94540682f0 0x7f945406a7a0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.172+0000 7f9478df8700 1 --2- 172.21.15.28:0/4229690013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f94740cc310 0x7f947415a050 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.174 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.172+0000 7f9478df8700 1 --2- 172.21.15.28:0/4229690013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9474161090 0x7f94740b4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.172+0000 7f9478df8700 1 --2- 172.21.15.28:0/4229690013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f947415c990 0x7f94740b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.172+0000 7f9478df8700 1 -- 172.21.15.28:0/4229690013 >> 172.21.15.28:0/4229690013 conn(0x7f94740b1650 msgr2=0x7f94740cc7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:06.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.172+0000 7f9478df8700 1 -- 172.21.15.28:0/4229690013 shutdown_connections 2023-11-28T11:10:06.175 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.172+0000 7f9478df8700 1 -- 172.21.15.28:0/4229690013 wait complete. 2023-11-28T11:10:06.186 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-28T11:10:06.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.310+0000 7f83ec802700 1 Processor -- start 2023-11-28T11:10:06.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.310+0000 7f83ec802700 1 -- start start 2023-11-28T11:10:06.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.311+0000 7f83ec802700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f83e815c560 0x7f83e806f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:06.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.311+0000 7f83ec802700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f83e80ceec0 0x7f83e806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:06.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.311+0000 7f83ec802700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83e80b30b0 con 0x7f83e80b2660 2023-11-28T11:10:06.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.311+0000 7f83ec802700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e806c020 con 0x7f83e80ceec0 2023-11-28T11:10:06.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.311+0000 7f83ec802700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83e80d09c0 con 0x7f83e815c560 2023-11-28T11:10:06.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.311+0000 7f83e6d9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f83e80ceec0 0x7f83e806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:06.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.311+0000 7f83e5d9b700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f83e80b2660 0x7f83e80ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37316/0 (socket says 172.21.15.28:37316) 2023-11-28T11:10:06.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.311+0000 7f83e5d9b700 1 -- 172.21.15.28:0/2921748004 learned_addr learned my addr 172.21.15.28:0/2921748004 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:06.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.311+0000 7f83e659c700 1 --2- 172.21.15.28:0/2921748004 >> v2:172.21.15.59:3300/0 conn(0x7f83e815c560 0x7f83e806f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:06.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.311+0000 7f83e6d9d700 1 -- 172.21.15.28:0/2921748004 >> v2:172.21.15.59:3300/0 conn(0x7f83e815c560 msgr2=0x7f83e806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:06.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.311+0000 7f83e6d9d700 1 --2- 172.21.15.28:0/2921748004 >> v2:172.21.15.59:3300/0 conn(0x7f83e815c560 0x7f83e806f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.311+0000 7f83e6d9d700 1 -- 172.21.15.28:0/2921748004 >> v1:172.21.15.28:6789/0 conn(0x7f83e80b2660 legacy=0x7f83e80ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:06.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.311+0000 7f83e559a700 1 -- 172.21.15.28:0/2921748004 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750942462 0 0) 0x7f83e80b30b0 con 0x7f83e80b2660 2023-11-28T11:10:06.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.311+0000 7f83e6d9d700 1 -- 172.21.15.28:0/2921748004 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e806fdf0 con 0x7f83e80ceec0 2023-11-28T11:10:06.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.311+0000 7f83e6d9d700 1 --2- 172.21.15.28:0/2921748004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83e80ceec0 0x7f83e806c4b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f83e8066920 tx=0x7f83e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdc4b56583b22faf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:06.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.312+0000 7f83e559a700 1 -- 172.21.15.28:0/2921748004 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e000e070 con 0x7f83e80ceec0 2023-11-28T11:10:06.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.312+0000 7f83e559a700 1 -- 172.21.15.28:0/2921748004 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e000bb20 con 0x7f83e80ceec0 2023-11-28T11:10:06.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.312+0000 7f83e559a700 1 -- 172.21.15.28:0/2921748004 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0012430 con 0x7f83e80ceec0 2023-11-28T11:10:06.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.312+0000 7f83ec802700 1 -- 172.21.15.28:0/2921748004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83e80ceec0 msgr2=0x7f83e806c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:06.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.312+0000 7f83ec802700 1 --2- 172.21.15.28:0/2921748004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83e80ceec0 0x7f83e806c4b0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f83e8066920 tx=0x7f83e000b330 comp rx=0 tx=0).stop 2023-11-28T11:10:06.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.312+0000 7f83ec802700 1 -- 172.21.15.28:0/2921748004 shutdown_connections 2023-11-28T11:10:06.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.312+0000 7f83ec802700 1 --2- 172.21.15.28:0/2921748004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83e80ceec0 0x7f83e806c4b0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.312+0000 7f83ec802700 1 --2- 172.21.15.28:0/2921748004 >> v2:172.21.15.59:3300/0 conn(0x7f83e815c560 0x7f83e806f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.312+0000 7f83ec802700 1 -- 172.21.15.28:0/2921748004 >> 172.21.15.28:0/2921748004 conn(0x7f83e80cd660 msgr2=0x7f83e80cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:06.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.312+0000 7f83ec802700 1 -- 172.21.15.28:0/2921748004 shutdown_connections 2023-11-28T11:10:06.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.312+0000 7f83ec802700 1 -- 172.21.15.28:0/2921748004 wait complete. 2023-11-28T11:10:06.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.314+0000 7f83ec802700 1 Processor -- start 2023-11-28T11:10:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.314+0000 7f83ec802700 1 -- start start 2023-11-28T11:10:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.314+0000 7f83ec802700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e80b2660 0x7f83e81488f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.314+0000 7f83ec802700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83e80ceec0 0x7f83e8148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.314+0000 7f83ec802700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83e815c560 0x7f83e8149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.314+0000 7f83ec802700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e8160790 con 0x7f83e815c560 2023-11-28T11:10:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.314+0000 7f83ec802700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e815fb80 con 0x7f83e80b2660 2023-11-28T11:10:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.314+0000 7f83ec802700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e815fe60 con 0x7f83e80ceec0 2023-11-28T11:10:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.314+0000 7f83e6d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83e815c560 0x7f83e8149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.314+0000 7f83e6d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83e815c560 0x7f83e8149370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49266/0 (socket says 172.21.15.28:49266) 2023-11-28T11:10:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.314+0000 7f83e6d9d700 1 -- 172.21.15.28:0/1290713688 learned_addr learned my addr 172.21.15.28:0/1290713688 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.315+0000 7f83e659c700 1 --2- 172.21.15.28:0/1290713688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e80b2660 0x7f83e81488f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.315+0000 7f83e6d9d700 1 -- 172.21.15.28:0/1290713688 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83e80ceec0 msgr2=0x7f83e8148e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:06.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.315+0000 7f83e6d9d700 1 --2- 172.21.15.28:0/1290713688 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83e80ceec0 0x7f83e8148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.315+0000 7f83e6d9d700 1 -- 172.21.15.28:0/1290713688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e80b2660 msgr2=0x7f83e81488f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.315+0000 7f83e6d9d700 1 --2- 172.21.15.28:0/1290713688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e80b2660 0x7f83e81488f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.315+0000 7f83e6d9d700 1 -- 172.21.15.28:0/1290713688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e0009a70 con 0x7f83e815c560 2023-11-28T11:10:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.315+0000 7f83e6d9d700 1 --2- 172.21.15.28:0/1290713688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83e815c560 0x7f83e8149370 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f83e8072370 tx=0x7f83e0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.315+0000 7f83d6ffd700 1 -- 172.21.15.28:0/1290713688 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0012430 con 0x7f83e815c560 2023-11-28T11:10:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.315+0000 7f83d6ffd700 1 -- 172.21.15.28:0/1290713688 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e0020070 con 0x7f83e815c560 2023-11-28T11:10:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.315+0000 7f83d6ffd700 1 -- 172.21.15.28:0/1290713688 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e0004280 con 0x7f83e815c560 2023-11-28T11:10:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.315+0000 7f83ec802700 1 -- 172.21.15.28:0/1290713688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e80b2dd0 con 0x7f83e815c560 2023-11-28T11:10:06.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.315+0000 7f83ec802700 1 -- 172.21.15.28:0/1290713688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83e80b2af0 con 0x7f83e815c560 2023-11-28T11:10:06.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.316+0000 7f83ec802700 1 -- 172.21.15.28:0/1290713688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83e806bcb0 con 0x7f83e815c560 2023-11-28T11:10:06.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.316+0000 7f83d6ffd700 1 -- 172.21.15.28:0/1290713688 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f83e002e070 con 0x7f83e815c560 2023-11-28T11:10:06.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.317+0000 7f83d6ffd700 1 --2- 172.21.15.28:0/1290713688 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83c80682f0 0x7f83c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:06.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.317+0000 7f83d6ffd700 1 -- 172.21.15.28:0/1290713688 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f83e0022070 con 0x7f83e815c560 2023-11-28T11:10:06.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.317+0000 7f83e659c700 1 --2- 172.21.15.28:0/1290713688 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83c80682f0 0x7f83c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:06.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.317+0000 7f83e659c700 1 --2- 172.21.15.28:0/1290713688 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83c80682f0 0x7f83c806a7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009a10 tx=0x7f83d8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:06.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.321+0000 7f83d6ffd700 1 -- 172.21.15.28:0/1290713688 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f83e0054240 con 0x7f83e815c560 2023-11-28T11:10:06.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.591+0000 7f83ec802700 1 -- 172.21.15.28:0/1290713688 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f83e80675a0 con 0x7f83e815c560 2023-11-28T11:10:06.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.595+0000 7f83d6ffd700 1 -- 172.21.15.28:0/1290713688 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v14) v1 ==== 90+0+0 (secure 0 0 0) 0x7f83e005be60 con 0x7f83e815c560 2023-11-28T11:10:06.598 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.597+0000 7f83ec802700 1 -- 172.21.15.28:0/1290713688 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83c80682f0 msgr2=0x7f83c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:06.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.597+0000 7f83ec802700 1 --2- 172.21.15.28:0/1290713688 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83c80682f0 0x7f83c806a7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f83d8009a10 tx=0x7f83d8008040 comp rx=0 tx=0).stop 2023-11-28T11:10:06.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.597+0000 7f83ec802700 1 -- 172.21.15.28:0/1290713688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83e815c560 msgr2=0x7f83e8149370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:06.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.597+0000 7f83ec802700 1 --2- 172.21.15.28:0/1290713688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83e815c560 0x7f83e8149370 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f83e8072370 tx=0x7f83e0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.597+0000 7f83ec802700 1 -- 172.21.15.28:0/1290713688 shutdown_connections 2023-11-28T11:10:06.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.597+0000 7f83ec802700 1 --2- 172.21.15.28:0/1290713688 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f83c80682f0 0x7f83c806a7a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.597+0000 7f83ec802700 1 --2- 172.21.15.28:0/1290713688 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f83e815c560 0x7f83e8149370 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.597+0000 7f83ec802700 1 --2- 172.21.15.28:0/1290713688 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f83e80b2660 0x7f83e81488f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.597+0000 7f83ec802700 1 --2- 172.21.15.28:0/1290713688 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f83e80ceec0 0x7f83e8148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.597+0000 7f83ec802700 1 -- 172.21.15.28:0/1290713688 >> 172.21.15.28:0/1290713688 conn(0x7f83e80cd660 msgr2=0x7f83e806c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:06.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.597+0000 7f83ec802700 1 -- 172.21.15.28:0/1290713688 shutdown_connections 2023-11-28T11:10:06.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.597+0000 7f83ec802700 1 -- 172.21.15.28:0/1290713688 wait complete. 2023-11-28T11:10:06.611 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:06.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.734+0000 7f357bd22700 1 Processor -- start 2023-11-28T11:10:06.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.734+0000 7f357bd22700 1 -- start start 2023-11-28T11:10:06.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.734+0000 7f357bd22700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3574161090 0x7f35740ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:06.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f357bd22700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f35740caeb0 0x7f35740cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:06.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f357bd22700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35740ce340 con 0x7f357415c990 2023-11-28T11:10:06.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f357bd22700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35740cebe0 con 0x7f35740caeb0 2023-11-28T11:10:06.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f357bd22700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35740cf780 con 0x7f3574161090 2023-11-28T11:10:06.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f357a2bf700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f35740caeb0 0x7f35740cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:06.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f357a2bf700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f35740caeb0 0x7f35740cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49274/0 (socket says 172.21.15.28:49274) 2023-11-28T11:10:06.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f357a2bf700 1 -- 172.21.15.28:0/2545438831 learned_addr learned my addr 172.21.15.28:0/2545438831 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:06.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f35792bd700 1 --2- 172.21.15.28:0/2545438831 >> v2:172.21.15.59:3300/0 conn(0x7f3574161090 0x7f35740ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:06.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f357a2bf700 1 -- 172.21.15.28:0/2545438831 >> v2:172.21.15.59:3300/0 conn(0x7f3574161090 msgr2=0x7f35740ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:06.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f357a2bf700 1 --2- 172.21.15.28:0/2545438831 >> v2:172.21.15.59:3300/0 conn(0x7f3574161090 0x7f35740ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f357a2bf700 1 -- 172.21.15.28:0/2545438831 >> v1:172.21.15.59:6789/0 conn(0x7f357415c990 legacy=0x7f35740cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:06.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f357a2bf700 1 -- 172.21.15.28:0/2545438831 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35740cb3f0 con 0x7f35740caeb0 2023-11-28T11:10:06.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f357a2bf700 1 --2- 172.21.15.28:0/2545438831 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35740caeb0 0x7f35740cc310 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f357406ab20 tx=0x7f356c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755f66a43f1b99b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:06.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f3578abc700 1 -- 172.21.15.28:0/2545438831 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00e070 con 0x7f35740caeb0 2023-11-28T11:10:06.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f3578abc700 1 -- 172.21.15.28:0/2545438831 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f356c00b910 con 0x7f35740caeb0 2023-11-28T11:10:06.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.735+0000 7f3578abc700 1 -- 172.21.15.28:0/2545438831 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f356c00bcc0 con 0x7f35740caeb0 2023-11-28T11:10:06.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.736+0000 7f357bd22700 1 -- 172.21.15.28:0/2545438831 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35740caeb0 msgr2=0x7f35740cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:06.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.736+0000 7f357bd22700 1 --2- 172.21.15.28:0/2545438831 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35740caeb0 0x7f35740cc310 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f357406ab20 tx=0x7f356c00b330 comp rx=0 tx=0).stop 2023-11-28T11:10:06.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.736+0000 7f357bd22700 1 -- 172.21.15.28:0/2545438831 shutdown_connections 2023-11-28T11:10:06.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.736+0000 7f357bd22700 1 --2- 172.21.15.28:0/2545438831 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35740caeb0 0x7f35740cc310 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.736+0000 7f357bd22700 1 --2- 172.21.15.28:0/2545438831 >> v2:172.21.15.59:3300/0 conn(0x7f3574161090 0x7f35740ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.736+0000 7f357bd22700 1 -- 172.21.15.28:0/2545438831 >> 172.21.15.28:0/2545438831 conn(0x7f35740b1650 msgr2=0x7f35740b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:06.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.736+0000 7f357bd22700 1 -- 172.21.15.28:0/2545438831 shutdown_connections 2023-11-28T11:10:06.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.736+0000 7f357bd22700 1 -- 172.21.15.28:0/2545438831 wait complete. 2023-11-28T11:10:06.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.737+0000 7f357bd22700 1 Processor -- start 2023-11-28T11:10:06.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.737+0000 7f357bd22700 1 -- start start 2023-11-28T11:10:06.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f357bd22700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35740caeb0 0x7f3574159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:06.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f357bd22700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f357415c990 0x7f35740c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:06.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f357bd22700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3574161090 0x7f35741f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:06.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f357bd22700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357415a220 con 0x7f35740caeb0 2023-11-28T11:10:06.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f357bd22700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35740c61c0 con 0x7f3574161090 2023-11-28T11:10:06.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f357bd22700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35740c64a0 con 0x7f357415c990 2023-11-28T11:10:06.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f3579abe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35740caeb0 0x7f3574159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:06.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f3579abe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35740caeb0 0x7f3574159980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49276/0 (socket says 172.21.15.28:49276) 2023-11-28T11:10:06.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f3579abe700 1 -- 172.21.15.28:0/3728603879 learned_addr learned my addr 172.21.15.28:0/3728603879 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:06.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f35792bd700 1 --2- 172.21.15.28:0/3728603879 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f357415c990 0x7f35740c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:06.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f357a2bf700 1 --2- 172.21.15.28:0/3728603879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3574161090 0x7f35741f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:06.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f3579abe700 1 -- 172.21.15.28:0/3728603879 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f357415c990 msgr2=0x7f35740c4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:06.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f3579abe700 1 --2- 172.21.15.28:0/3728603879 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f357415c990 0x7f35740c4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f3579abe700 1 -- 172.21.15.28:0/3728603879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3574161090 msgr2=0x7f35741f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:06.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f3579abe700 1 --2- 172.21.15.28:0/3728603879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3574161090 0x7f35741f6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f3579abe700 1 -- 172.21.15.28:0/3728603879 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f356c009b30 con 0x7f35740caeb0 2023-11-28T11:10:06.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.738+0000 7f35792bd700 1 --2- 172.21.15.28:0/3728603879 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f357415c990 0x7f35740c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:06.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.739+0000 7f3579abe700 1 --2- 172.21.15.28:0/3728603879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35740caeb0 0x7f3574159980 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f357000c490 tx=0x7f357000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:06.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.739+0000 7f356a7fc700 1 -- 172.21.15.28:0/3728603879 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35700052d0 con 0x7f35740caeb0 2023-11-28T11:10:06.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.739+0000 7f356a7fc700 1 -- 172.21.15.28:0/3728603879 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3570005450 con 0x7f35740caeb0 2023-11-28T11:10:06.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.739+0000 7f356a7fc700 1 -- 172.21.15.28:0/3728603879 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3570006aa0 con 0x7f35740caeb0 2023-11-28T11:10:06.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.739+0000 7f357bd22700 1 -- 172.21.15.28:0/3728603879 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35740ce870 con 0x7f35740caeb0 2023-11-28T11:10:06.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.739+0000 7f357bd22700 1 -- 172.21.15.28:0/3728603879 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35740ce590 con 0x7f35740caeb0 2023-11-28T11:10:06.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.740+0000 7f357bd22700 1 -- 172.21.15.28:0/3728603879 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35740cc850 con 0x7f35740caeb0 2023-11-28T11:10:06.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.740+0000 7f356a7fc700 1 -- 172.21.15.28:0/3728603879 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3570010050 con 0x7f35740caeb0 2023-11-28T11:10:06.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.740+0000 7f356a7fc700 1 --2- 172.21.15.28:0/3728603879 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f35580682f0 0x7f355806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:06.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.741+0000 7f356a7fc700 1 -- 172.21.15.28:0/3728603879 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3570084c20 con 0x7f35740caeb0 2023-11-28T11:10:06.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.741+0000 7f35792bd700 1 --2- 172.21.15.28:0/3728603879 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f35580682f0 0x7f355806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:06.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.741+0000 7f35792bd700 1 --2- 172.21.15.28:0/3728603879 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f35580682f0 0x7f355806a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f35740c3980 tx=0x7f3564008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:06.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.745+0000 7f356a7fc700 1 -- 172.21.15.28:0/3728603879 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f357004a5e0 con 0x7f35740caeb0 2023-11-28T11:10:06.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.937+0000 7f357bd22700 1 -- 172.21.15.28:0/3728603879 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f35740675a0 con 0x7f35740caeb0 2023-11-28T11:10:06.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.938+0000 7f356a7fc700 1 -- 172.21.15.28:0/3728603879 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v36) v1 ==== 74+0+14366 (secure 0 0 0) 0x7f3570051690 con 0x7f35740caeb0 2023-11-28T11:10:06.940 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:06.940 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":36,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:10:03.589158+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:06.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.940+0000 7f357bd22700 1 -- 172.21.15.28:0/3728603879 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f35580682f0 msgr2=0x7f355806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:06.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.940+0000 7f357bd22700 1 --2- 172.21.15.28:0/3728603879 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f35580682f0 0x7f355806a7a0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f35740c3980 tx=0x7f3564008040 comp rx=0 tx=0).stop 2023-11-28T11:10:06.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.940+0000 7f357bd22700 1 -- 172.21.15.28:0/3728603879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35740caeb0 msgr2=0x7f3574159980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:06.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.940+0000 7f357bd22700 1 --2- 172.21.15.28:0/3728603879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35740caeb0 0x7f3574159980 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f357000c490 tx=0x7f357000c850 comp rx=0 tx=0).stop 2023-11-28T11:10:06.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.940+0000 7f357bd22700 1 -- 172.21.15.28:0/3728603879 shutdown_connections 2023-11-28T11:10:06.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.940+0000 7f357bd22700 1 --2- 172.21.15.28:0/3728603879 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f35580682f0 0x7f355806a7a0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.940+0000 7f357bd22700 1 --2- 172.21.15.28:0/3728603879 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f35740caeb0 0x7f3574159980 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.940+0000 7f357bd22700 1 --2- 172.21.15.28:0/3728603879 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3574161090 0x7f35741f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.940+0000 7f357bd22700 1 --2- 172.21.15.28:0/3728603879 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f357415c990 0x7f35740c4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:06.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.940+0000 7f357bd22700 1 -- 172.21.15.28:0/3728603879 >> 172.21.15.28:0/3728603879 conn(0x7f35740b1650 msgr2=0x7f35740c6dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:06.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.940+0000 7f357bd22700 1 -- 172.21.15.28:0/3728603879 shutdown_connections 2023-11-28T11:10:06.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:06.940+0000 7f357bd22700 1 -- 172.21.15.28:0/3728603879 wait complete. 2023-11-28T11:10:06.954 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:10:06.955 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:10:07.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.081+0000 7fcced539700 1 Processor -- start 2023-11-28T11:10:07.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.081+0000 7fcced539700 1 -- start start 2023-11-28T11:10:07.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.082+0000 7fcced539700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcce806a1e0 0x7fcce80c9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.082+0000 7fcced539700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcce8089580 0x7fcce806cd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.082+0000 7fcced539700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcce806d290 0x7fcce806d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.082+0000 7fcced539700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce80ca0d0 con 0x7fcce806a1e0 2023-11-28T11:10:07.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.082+0000 7fcced539700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce8169750 con 0x7fcce806d290 2023-11-28T11:10:07.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.082+0000 7fcced539700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce80cd9d0 con 0x7fcce8089580 2023-11-28T11:10:07.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.082+0000 7fcce6ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcce806a1e0 0x7fcce80c9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.082+0000 7fcce6ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcce806a1e0 0x7fcce80c9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49284/0 (socket says 172.21.15.28:49284) 2023-11-28T11:10:07.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.082+0000 7fcce6ffd700 1 -- 172.21.15.28:0/3419139316 learned_addr learned my addr 172.21.15.28:0/3419139316 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:07.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.082+0000 7fcce77fe700 1 --2- 172.21.15.28:0/3419139316 >> v2:172.21.15.59:3300/0 conn(0x7fcce806d290 0x7fcce806d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.082+0000 7fcce67fc700 1 --2- 172.21.15.28:0/3419139316 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcce8089580 0x7fcce806cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.149+0000 7fcce6ffd700 1 -- 172.21.15.28:0/3419139316 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcce8089580 msgr2=0x7fcce806cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:07.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.149+0000 7fcce6ffd700 1 --2- 172.21.15.28:0/3419139316 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcce8089580 0x7fcce806cd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.149+0000 7fcce6ffd700 1 -- 172.21.15.28:0/3419139316 >> v2:172.21.15.59:3300/0 conn(0x7fcce806d290 msgr2=0x7fcce806d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:07.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.149+0000 7fcce6ffd700 1 --2- 172.21.15.28:0/3419139316 >> v2:172.21.15.59:3300/0 conn(0x7fcce806d290 0x7fcce806d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.149+0000 7fcce6ffd700 1 -- 172.21.15.28:0/3419139316 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce80cdcb0 con 0x7fcce806a1e0 2023-11-28T11:10:07.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.149+0000 7fcce6ffd700 1 --2- 172.21.15.28:0/3419139316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcce806a1e0 0x7fcce80c9a40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcce8089510 tx=0x7fccdc00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c742f0655a608913 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:07.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.149+0000 7fcce5ffb700 1 -- 172.21.15.28:0/3419139316 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00f070 con 0x7fcce806a1e0 2023-11-28T11:10:07.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.149+0000 7fcce5ffb700 1 -- 172.21.15.28:0/3419139316 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccdc00b660 con 0x7fcce806a1e0 2023-11-28T11:10:07.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.149+0000 7fcce5ffb700 1 -- 172.21.15.28:0/3419139316 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc00b9f0 con 0x7fcce806a1e0 2023-11-28T11:10:07.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.150+0000 7fcced539700 1 -- 172.21.15.28:0/3419139316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcce806a1e0 msgr2=0x7fcce80c9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:07.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.150+0000 7fcced539700 1 --2- 172.21.15.28:0/3419139316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcce806a1e0 0x7fcce80c9a40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fcce8089510 tx=0x7fccdc00cb40 comp rx=0 tx=0).stop 2023-11-28T11:10:07.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.150+0000 7fcced539700 1 -- 172.21.15.28:0/3419139316 shutdown_connections 2023-11-28T11:10:07.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.150+0000 7fcced539700 1 --2- 172.21.15.28:0/3419139316 >> v2:172.21.15.59:3300/0 conn(0x7fcce806d290 0x7fcce806d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.150+0000 7fcced539700 1 --2- 172.21.15.28:0/3419139316 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcce806a1e0 0x7fcce80c9a40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.150+0000 7fcced539700 1 --2- 172.21.15.28:0/3419139316 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcce8089580 0x7fcce806cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.150+0000 7fcced539700 1 -- 172.21.15.28:0/3419139316 >> 172.21.15.28:0/3419139316 conn(0x7fcce80cf670 msgr2=0x7fcce80cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:07.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.150+0000 7fcced539700 1 -- 172.21.15.28:0/3419139316 shutdown_connections 2023-11-28T11:10:07.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.150+0000 7fcced539700 1 -- 172.21.15.28:0/3419139316 wait complete. 2023-11-28T11:10:07.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.151+0000 7fcced539700 1 Processor -- start 2023-11-28T11:10:07.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.152+0000 7fcced539700 1 -- start start 2023-11-28T11:10:07.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.152+0000 7fcced539700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce806a1e0 0x7fcce8201880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.152+0000 7fcced539700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcce806d290 0x7fcce80885c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.152+0000 7fcced539700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcce8089580 0x7fcce8202100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.152+0000 7fcced539700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce816c800 con 0x7fcce806d290 2023-11-28T11:10:07.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.152+0000 7fcced539700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce8203070 con 0x7fcce806a1e0 2023-11-28T11:10:07.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.152+0000 7fcced539700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce8202dd0 con 0x7fcce8089580 2023-11-28T11:10:07.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.152+0000 7fcce67fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcce806d290 0x7fcce80885c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.152+0000 7fcce67fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcce806d290 0x7fcce80885c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49290/0 (socket says 172.21.15.28:49290) 2023-11-28T11:10:07.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.152+0000 7fcce67fc700 1 -- 172.21.15.28:0/3537161988 learned_addr learned my addr 172.21.15.28:0/3537161988 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:07.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.153+0000 7fcce6ffd700 1 --2- 172.21.15.28:0/3537161988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce806a1e0 0x7fcce8201880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.153+0000 7fcce67fc700 1 -- 172.21.15.28:0/3537161988 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcce8089580 msgr2=0x7fcce8202100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:07.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.153+0000 7fcce77fe700 1 --2- 172.21.15.28:0/3537161988 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcce8089580 0x7fcce8202100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.153+0000 7fcce67fc700 1 --2- 172.21.15.28:0/3537161988 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcce8089580 0x7fcce8202100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.153+0000 7fcce67fc700 1 -- 172.21.15.28:0/3537161988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce806a1e0 msgr2=0x7fcce8201880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:07.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.153+0000 7fcce67fc700 1 --2- 172.21.15.28:0/3537161988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce806a1e0 0x7fcce8201880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.153+0000 7fcce67fc700 1 -- 172.21.15.28:0/3537161988 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc00c550 con 0x7fcce806d290 2023-11-28T11:10:07.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.153+0000 7fcce67fc700 1 --2- 172.21.15.28:0/3537161988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcce806d290 0x7fcce80885c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fccd8011ab0 tx=0x7fccd8011e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:07.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.153+0000 7fcccf7fe700 1 -- 172.21.15.28:0/3537161988 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd800fa10 con 0x7fcce806d290 2023-11-28T11:10:07.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.153+0000 7fcccf7fe700 1 -- 172.21.15.28:0/3537161988 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd8013040 con 0x7fcce806d290 2023-11-28T11:10:07.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.153+0000 7fcced539700 1 -- 172.21.15.28:0/3537161988 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce80ce270 con 0x7fcce806d290 2023-11-28T11:10:07.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.153+0000 7fcccf7fe700 1 -- 172.21.15.28:0/3537161988 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd800fdb0 con 0x7fcce806d290 2023-11-28T11:10:07.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.153+0000 7fcced539700 1 -- 172.21.15.28:0/3537161988 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce80cdf90 con 0x7fcce806d290 2023-11-28T11:10:07.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.154+0000 7fcced539700 1 -- 172.21.15.28:0/3537161988 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce80ce790 con 0x7fcce806d290 2023-11-28T11:10:07.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.154+0000 7fcccf7fe700 1 -- 172.21.15.28:0/3537161988 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fccd8020030 con 0x7fcce806d290 2023-11-28T11:10:07.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.155+0000 7fcccf7fe700 1 --2- 172.21.15.28:0/3537161988 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fccd40682f0 0x7fccd406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.155+0000 7fcccf7fe700 1 -- 172.21.15.28:0/3537161988 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fccd8089520 con 0x7fcce806d290 2023-11-28T11:10:07.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.155+0000 7fcce6ffd700 1 --2- 172.21.15.28:0/3537161988 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fccd40682f0 0x7fccd406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.156+0000 7fcce6ffd700 1 --2- 172.21.15.28:0/3537161988 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fccd40682f0 0x7fccd406a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcce816b490 tx=0x7fccdc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:07.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.159+0000 7fcccf7fe700 1 -- 172.21.15.28:0/3537161988 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fccd80525e0 con 0x7fcce806d290 2023-11-28T11:10:07.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.352+0000 7fcced539700 1 -- 172.21.15.28:0/3537161988 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fcce8169890 con 0x7fcce806d290 2023-11-28T11:10:07.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.352+0000 7fcccf7fe700 1 -- 172.21.15.28:0/3537161988 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v36) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcce8169890 con 0x7fcce806d290 2023-11-28T11:10:07.354 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:10:07.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.355+0000 7fcced539700 1 -- 172.21.15.28:0/3537161988 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fccd40682f0 msgr2=0x7fccd406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:07.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.355+0000 7fcced539700 1 --2- 172.21.15.28:0/3537161988 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fccd40682f0 0x7fccd406a7a0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fcce816b490 tx=0x7fccdc008030 comp rx=0 tx=0).stop 2023-11-28T11:10:07.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.355+0000 7fcced539700 1 -- 172.21.15.28:0/3537161988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcce806d290 msgr2=0x7fcce80885c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:07.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.355+0000 7fcced539700 1 --2- 172.21.15.28:0/3537161988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcce806d290 0x7fcce80885c0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fccd8011ab0 tx=0x7fccd8011e70 comp rx=0 tx=0).stop 2023-11-28T11:10:07.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.355+0000 7fcced539700 1 -- 172.21.15.28:0/3537161988 shutdown_connections 2023-11-28T11:10:07.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.355+0000 7fcced539700 1 --2- 172.21.15.28:0/3537161988 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fccd40682f0 0x7fccd406a7a0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.355+0000 7fcced539700 1 --2- 172.21.15.28:0/3537161988 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcce8089580 0x7fcce8202100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.355+0000 7fcced539700 1 --2- 172.21.15.28:0/3537161988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce806a1e0 0x7fcce8201880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.355+0000 7fcced539700 1 --2- 172.21.15.28:0/3537161988 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcce806d290 0x7fcce80885c0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.355+0000 7fcced539700 1 -- 172.21.15.28:0/3537161988 >> 172.21.15.28:0/3537161988 conn(0x7fcce80cf670 msgr2=0x7fcce80c8cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:07.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.355+0000 7fcced539700 1 -- 172.21.15.28:0/3537161988 shutdown_connections 2023-11-28T11:10:07.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.355+0000 7fcced539700 1 -- 172.21.15.28:0/3537161988 wait complete. 2023-11-28T11:10:07.368 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:10:07.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.495+0000 7fe08947c700 1 Processor -- start 2023-11-28T11:10:07.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.495+0000 7fe08947c700 1 -- start start 2023-11-28T11:10:07.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe08947c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe0840d2820 0x7fe0840b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe08947c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe08406bf30 0x7fe08406c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe08947c700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe08406c870 con 0x7fe08415c640 2023-11-28T11:10:07.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe08947c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe08415ca10 con 0x7fe0840d2820 2023-11-28T11:10:07.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe08947c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840d1470 con 0x7fe08406bf30 2023-11-28T11:10:07.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe082ffd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe08415c640 0x7fe0840cf040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37356/0 (socket says 172.21.15.28:37356) 2023-11-28T11:10:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe082ffd700 1 -- 172.21.15.28:0/224157655 learned_addr learned my addr 172.21.15.28:0/224157655 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe0837fe700 1 --2- 172.21.15.28:0/224157655 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe08406bf30 0x7fe08406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe081ffb700 1 -- 172.21.15.28:0/224157655 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2388816552 0 0) 0x7fe08406c870 con 0x7fe08415c640 2023-11-28T11:10:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe0827fc700 1 --2- 172.21.15.28:0/224157655 >> v2:172.21.15.59:3300/0 conn(0x7fe0840d2820 0x7fe0840b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe081ffb700 1 -- 172.21.15.28:0/224157655 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe06c0035b0 con 0x7fe08415c640 2023-11-28T11:10:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe081ffb700 1 -- 172.21.15.28:0/224157655 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2067219709 0 0) 0x7fe06c0035b0 con 0x7fe08415c640 2023-11-28T11:10:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe081ffb700 1 -- 172.21.15.28:0/224157655 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe08406bf30 msgr2=0x7fe08406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe081ffb700 1 --2- 172.21.15.28:0/224157655 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe08406bf30 0x7fe08406c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe081ffb700 1 -- 172.21.15.28:0/224157655 >> v2:172.21.15.59:3300/0 conn(0x7fe0840d2820 msgr2=0x7fe0840b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe081ffb700 1 --2- 172.21.15.28:0/224157655 >> v2:172.21.15.59:3300/0 conn(0x7fe0840d2820 0x7fe0840b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe081ffb700 1 -- 172.21.15.28:0/224157655 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0840cd4e0 con 0x7fe08415c640 2023-11-28T11:10:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe081ffb700 1 -- 172.21.15.28:0/224157655 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fe078001670 con 0x7fe08415c640 2023-11-28T11:10:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe081ffb700 1 -- 172.21.15.28:0/224157655 >> v1:172.21.15.28:6789/0 conn(0x7fe08415c640 legacy=0x7fe0840cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.496+0000 7fe081ffb700 1 --2- 172.21.15.28:0/224157655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe06c003a00 0x7fe06c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe081ffb700 1 --2- 172.21.15.28:0/224157655 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe08406bf30 0x7fe06c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe081ffb700 1 --2- 172.21.15.28:0/224157655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe06c004d90 0x7fe06c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe081ffb700 1 -- 172.21.15.28:0/224157655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08415ca10 con 0x7fe06c004d90 2023-11-28T11:10:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe081ffb700 1 -- 172.21.15.28:0/224157655 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0840d1470 con 0x7fe06c003a00 2023-11-28T11:10:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe081ffb700 1 -- 172.21.15.28:0/224157655 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe06c00ba10 con 0x7fe08406bf30 2023-11-28T11:10:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe0837fe700 1 --2- 172.21.15.28:0/224157655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe06c004d90 0x7fe06c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe0827fc700 1 --2- 172.21.15.28:0/224157655 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe08406bf30 0x7fe06c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe082ffd700 1 --2- 172.21.15.28:0/224157655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe06c003a00 0x7fe06c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe0837fe700 1 -- 172.21.15.28:0/224157655 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe08406bf30 msgr2=0x7fe06c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe0837fe700 1 --2- 172.21.15.28:0/224157655 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe08406bf30 0x7fe06c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe0837fe700 1 -- 172.21.15.28:0/224157655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe06c003a00 msgr2=0x7fe06c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe0837fe700 1 --2- 172.21.15.28:0/224157655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe06c003a00 0x7fe06c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe0837fe700 1 -- 172.21.15.28:0/224157655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe06c0035b0 con 0x7fe06c004d90 2023-11-28T11:10:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe0837fe700 1 --2- 172.21.15.28:0/224157655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe06c004d90 0x7fe06c005200 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00e170 tx=0x7fe06c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401069d06ae54e6f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe081ffb700 1 -- 172.21.15.28:0/224157655 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c00ec30 con 0x7fe06c004d90 2023-11-28T11:10:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe081ffb700 1 -- 172.21.15.28:0/224157655 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c00f040 con 0x7fe06c004d90 2023-11-28T11:10:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.497+0000 7fe081ffb700 1 -- 172.21.15.28:0/224157655 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c014480 con 0x7fe06c004d90 2023-11-28T11:10:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.498+0000 7fe08947c700 1 -- 172.21.15.28:0/224157655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe06c004d90 msgr2=0x7fe06c005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.498+0000 7fe08947c700 1 --2- 172.21.15.28:0/224157655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe06c004d90 0x7fe06c005200 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00e170 tx=0x7fe06c005880 comp rx=0 tx=0).stop 2023-11-28T11:10:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.498+0000 7fe0837fe700 1 -- 172.21.15.28:0/224157655 reap_dead start 2023-11-28T11:10:07.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.498+0000 7fe08947c700 1 -- 172.21.15.28:0/224157655 shutdown_connections 2023-11-28T11:10:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.498+0000 7fe08947c700 1 -- 172.21.15.28:0/224157655 >> 172.21.15.28:0/224157655 conn(0x7fe0840d2df0 msgr2=0x7fe0840d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.498+0000 7fe08947c700 1 -- 172.21.15.28:0/224157655 shutdown_connections 2023-11-28T11:10:07.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.498+0000 7fe08947c700 1 -- 172.21.15.28:0/224157655 wait complete. 2023-11-28T11:10:07.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.499+0000 7fe08947c700 1 Processor -- start 2023-11-28T11:10:07.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.499+0000 7fe08947c700 1 -- start start 2023-11-28T11:10:07.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe08947c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe06c004d90 0x7fe08414c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe08947c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe084148da0 0x7fe0840a9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe08947c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe08414b630 0x7fe08414bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe08947c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08415f360 con 0x7fe06c004d90 2023-11-28T11:10:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe08947c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08414bff0 con 0x7fe08414b630 2023-11-28T11:10:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe08947c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08415e720 con 0x7fe084148da0 2023-11-28T11:10:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe082ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe06c004d90 0x7fe08414c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe082ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe06c004d90 0x7fe08414c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49310/0 (socket says 172.21.15.28:49310) 2023-11-28T11:10:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe082ffd700 1 -- 172.21.15.28:0/656891446 learned_addr learned my addr 172.21.15.28:0/656891446 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe0827fc700 1 --2- 172.21.15.28:0/656891446 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe084148da0 0x7fe0840a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe0837fe700 1 --2- 172.21.15.28:0/656891446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe08414b630 0x7fe08414bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe082ffd700 1 -- 172.21.15.28:0/656891446 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe084148da0 msgr2=0x7fe0840a9740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe082ffd700 1 --2- 172.21.15.28:0/656891446 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe084148da0 0x7fe0840a9740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe082ffd700 1 -- 172.21.15.28:0/656891446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe08414b630 msgr2=0x7fe08414bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe082ffd700 1 --2- 172.21.15.28:0/656891446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe08414b630 0x7fe08414bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.500+0000 7fe082ffd700 1 -- 172.21.15.28:0/656891446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0840d1710 con 0x7fe06c004d90 2023-11-28T11:10:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.501+0000 7fe082ffd700 1 --2- 172.21.15.28:0/656891446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe06c004d90 0x7fe08414c1d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe078007310 tx=0x7fe078007620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.501+0000 7fe06b7fe700 1 -- 172.21.15.28:0/656891446 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe078007800 con 0x7fe06c004d90 2023-11-28T11:10:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.501+0000 7fe06b7fe700 1 -- 172.21.15.28:0/656891446 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07800f8d0 con 0x7fe06c004d90 2023-11-28T11:10:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.501+0000 7fe08947c700 1 -- 172.21.15.28:0/656891446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0840cd7c0 con 0x7fe06c004d90 2023-11-28T11:10:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.501+0000 7fe06b7fe700 1 -- 172.21.15.28:0/656891446 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07800fc60 con 0x7fe06c004d90 2023-11-28T11:10:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.501+0000 7fe08947c700 1 -- 172.21.15.28:0/656891446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0840cdaa0 con 0x7fe06c004d90 2023-11-28T11:10:07.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.502+0000 7fe08947c700 1 -- 172.21.15.28:0/656891446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe08415f4a0 con 0x7fe06c004d90 2023-11-28T11:10:07.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.502+0000 7fe06b7fe700 1 -- 172.21.15.28:0/656891446 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe07801d460 con 0x7fe06c004d90 2023-11-28T11:10:07.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.502+0000 7fe06b7fe700 1 --2- 172.21.15.28:0/656891446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0700682f0 0x7fe07006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.503+0000 7fe06b7fe700 1 -- 172.21.15.28:0/656891446 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fe078085d00 con 0x7fe06c004d90 2023-11-28T11:10:07.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.503+0000 7fe0827fc700 1 --2- 172.21.15.28:0/656891446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0700682f0 0x7fe07006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.503+0000 7fe0827fc700 1 --2- 172.21.15.28:0/656891446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0700682f0 0x7fe07006a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe074009c20 tx=0x7fe074008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:07.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.507+0000 7fe06b7fe700 1 -- 172.21.15.28:0/656891446 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe08415f4a0 con 0x7fe06c004d90 2023-11-28T11:10:07.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.699+0000 7fe08947c700 1 -- 172.21.15.28:0/656891446 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe08415f4a0 con 0x7fe06c004d90 2023-11-28T11:10:07.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.700+0000 7fe06b7fe700 1 -- 172.21.15.28:0/656891446 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v36) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe08415f4a0 con 0x7fe06c004d90 2023-11-28T11:10:07.702 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:10:07.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.701+0000 7fe08947c700 1 -- 172.21.15.28:0/656891446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0700682f0 msgr2=0x7fe07006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:07.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.701+0000 7fe08947c700 1 --2- 172.21.15.28:0/656891446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0700682f0 0x7fe07006a7a0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe074009c20 tx=0x7fe074008040 comp rx=0 tx=0).stop 2023-11-28T11:10:07.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.701+0000 7fe08947c700 1 -- 172.21.15.28:0/656891446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe06c004d90 msgr2=0x7fe08414c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:07.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.701+0000 7fe08947c700 1 --2- 172.21.15.28:0/656891446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe06c004d90 0x7fe08414c1d0 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fe078007310 tx=0x7fe078007620 comp rx=0 tx=0).stop 2023-11-28T11:10:07.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.701+0000 7fe08947c700 1 -- 172.21.15.28:0/656891446 shutdown_connections 2023-11-28T11:10:07.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.702+0000 7fe08947c700 1 --2- 172.21.15.28:0/656891446 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0700682f0 0x7fe07006a7a0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.702+0000 7fe08947c700 1 --2- 172.21.15.28:0/656891446 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe06c004d90 0x7fe08414c1d0 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.702+0000 7fe08947c700 1 --2- 172.21.15.28:0/656891446 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe08414b630 0x7fe08414bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.702+0000 7fe08947c700 1 --2- 172.21.15.28:0/656891446 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe084148da0 0x7fe0840a9740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.702+0000 7fe08947c700 1 -- 172.21.15.28:0/656891446 >> 172.21.15.28:0/656891446 conn(0x7fe0840d2df0 msgr2=0x7fe0840d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:07.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.702+0000 7fe08947c700 1 -- 172.21.15.28:0/656891446 shutdown_connections 2023-11-28T11:10:07.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.702+0000 7fe08947c700 1 -- 172.21.15.28:0/656891446 wait complete. 2023-11-28T11:10:07.715 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:10:07.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.839+0000 7f03da6f7700 1 Processor -- start 2023-11-28T11:10:07.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.839+0000 7f03da6f7700 1 -- start start 2023-11-28T11:10:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.839+0000 7f03da6f7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03d406e400 0x7f03d415db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03da6f7700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f03d40ce890 0x7f03d40ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.844 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03da6f7700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03d415e0b0 con 0x7f03d4159590 2023-11-28T11:10:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03da6f7700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03d40cf250 con 0x7f03d40ce890 2023-11-28T11:10:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03da6f7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d415d120 con 0x7f03d406e400 2023-11-28T11:10:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03d8c94700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f03d40ce890 0x7f03d40ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03d3fff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f03d4159590 0x7f03d40ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37376/0 (socket says 172.21.15.28:37376) 2023-11-28T11:10:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03d3fff700 1 -- 172.21.15.28:0/832341770 learned_addr learned my addr 172.21.15.28:0/832341770 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03d37fe700 1 --2- 172.21.15.28:0/832341770 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03d406e400 0x7f03d415db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03d8c94700 1 -- 172.21.15.28:0/832341770 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03d406e400 msgr2=0x7f03d415db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:07.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03d8c94700 1 --2- 172.21.15.28:0/832341770 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03d406e400 0x7f03d415db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03d8c94700 1 -- 172.21.15.28:0/832341770 >> v1:172.21.15.28:6789/0 conn(0x7f03d4159590 legacy=0x7f03d40ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:07.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03d8c94700 1 -- 172.21.15.28:0/832341770 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03d415d6e0 con 0x7f03d40ce890 2023-11-28T11:10:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03d2ffd700 1 -- 172.21.15.28:0/832341770 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2624866972 0 0) 0x7f03c4001110 con 0x7f03d4159590 2023-11-28T11:10:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03d8c94700 1 --2- 172.21.15.28:0/832341770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d40ce890 0x7f03d40ced10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009d90 tx=0x7f03cc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=634f04b9a443f83b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03d2ffd700 1 -- 172.21.15.28:0/832341770 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc00e070 con 0x7f03d40ce890 2023-11-28T11:10:07.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03d2ffd700 1 -- 172.21.15.28:0/832341770 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03cc00bbd0 con 0x7f03d40ce890 2023-11-28T11:10:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.840+0000 7f03d2ffd700 1 -- 172.21.15.28:0/832341770 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03cc012430 con 0x7f03d40ce890 2023-11-28T11:10:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.841+0000 7f03da6f7700 1 -- 172.21.15.28:0/832341770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d40ce890 msgr2=0x7f03d40ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:07.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.841+0000 7f03da6f7700 1 --2- 172.21.15.28:0/832341770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d40ce890 0x7f03d40ced10 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f03cc009d90 tx=0x7f03cc00b330 comp rx=0 tx=0).stop 2023-11-28T11:10:07.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.841+0000 7f03da6f7700 1 -- 172.21.15.28:0/832341770 shutdown_connections 2023-11-28T11:10:07.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.841+0000 7f03da6f7700 1 --2- 172.21.15.28:0/832341770 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d40ce890 0x7f03d40ced10 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.841+0000 7f03da6f7700 1 --2- 172.21.15.28:0/832341770 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03d406e400 0x7f03d415db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.841+0000 7f03da6f7700 1 -- 172.21.15.28:0/832341770 >> 172.21.15.28:0/832341770 conn(0x7f03d40b1650 msgr2=0x7f03d40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.841+0000 7f03da6f7700 1 -- 172.21.15.28:0/832341770 shutdown_connections 2023-11-28T11:10:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.841+0000 7f03da6f7700 1 -- 172.21.15.28:0/832341770 wait complete. 2023-11-28T11:10:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.842+0000 7f03da6f7700 1 Processor -- start 2023-11-28T11:10:07.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.842+0000 7f03da6f7700 1 -- start start 2023-11-28T11:10:07.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.843+0000 7f03da6f7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03d406e400 0x7f03d40c6da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.843+0000 7f03da6f7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d40ce890 0x7f03d415b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.843+0000 7f03da6f7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d4159590 0x7f03d415beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.843+0000 7f03da6f7700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4146d90 con 0x7f03d40ce890 2023-11-28T11:10:07.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.843+0000 7f03da6f7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03d4143d30 con 0x7f03d4159590 2023-11-28T11:10:07.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.843+0000 7f03da6f7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03d40c89e0 con 0x7f03d406e400 2023-11-28T11:10:07.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.843+0000 7f03d37fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d40ce890 0x7f03d415b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.843+0000 7f03d37fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d40ce890 0x7f03d415b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49326/0 (socket says 172.21.15.28:49326) 2023-11-28T11:10:07.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.843+0000 7f03d37fe700 1 -- 172.21.15.28:0/3835456475 learned_addr learned my addr 172.21.15.28:0/3835456475 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:07.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.843+0000 7f03d8c94700 1 --2- 172.21.15.28:0/3835456475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d4159590 0x7f03d415beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.843+0000 7f03d3fff700 1 --2- 172.21.15.28:0/3835456475 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03d406e400 0x7f03d40c6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.844+0000 7f03d37fe700 1 -- 172.21.15.28:0/3835456475 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03d406e400 msgr2=0x7f03d40c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:07.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.844+0000 7f03d37fe700 1 --2- 172.21.15.28:0/3835456475 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03d406e400 0x7f03d40c6da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.844+0000 7f03d37fe700 1 -- 172.21.15.28:0/3835456475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d4159590 msgr2=0x7f03d415beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:07.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.844+0000 7f03d37fe700 1 --2- 172.21.15.28:0/3835456475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d4159590 0x7f03d415beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:07.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.844+0000 7f03d37fe700 1 -- 172.21.15.28:0/3835456475 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03cc009a70 con 0x7f03d40ce890 2023-11-28T11:10:07.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.844+0000 7f03d37fe700 1 --2- 172.21.15.28:0/3835456475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d40ce890 0x7f03d415b970 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f03c800e810 tx=0x7f03c800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:07.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.844+0000 7f03d0ff9700 1 -- 172.21.15.28:0/3835456475 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8013610 con 0x7f03d40ce890 2023-11-28T11:10:07.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.844+0000 7f03d0ff9700 1 -- 172.21.15.28:0/3835456475 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03c8013770 con 0x7f03d40ce890 2023-11-28T11:10:07.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.844+0000 7f03da6f7700 1 -- 172.21.15.28:0/3835456475 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03d40af240 con 0x7f03d40ce890 2023-11-28T11:10:07.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.844+0000 7f03d0ff9700 1 -- 172.21.15.28:0/3835456475 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03c8013b20 con 0x7f03d40ce890 2023-11-28T11:10:07.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.844+0000 7f03da6f7700 1 -- 172.21.15.28:0/3835456475 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03d40aef60 con 0x7f03d40ce890 2023-11-28T11:10:07.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.845+0000 7f03da6f7700 1 -- 172.21.15.28:0/3835456475 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03d415d370 con 0x7f03d40ce890 2023-11-28T11:10:07.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.845+0000 7f03d0ff9700 1 -- 172.21.15.28:0/3835456475 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f03c8023070 con 0x7f03d40ce890 2023-11-28T11:10:07.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.846+0000 7f03d0ff9700 1 --2- 172.21.15.28:0/3835456475 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f03c0068320 0x7f03c006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:07.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.846+0000 7f03d0ff9700 1 -- 172.21.15.28:0/3835456475 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f03c80858c0 con 0x7f03d40ce890 2023-11-28T11:10:07.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.846+0000 7f03d3fff700 1 --2- 172.21.15.28:0/3835456475 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f03c0068320 0x7f03c006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:07.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.846+0000 7f03d3fff700 1 --2- 172.21.15.28:0/3835456475 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f03c0068320 0x7f03c006a7d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000f60 tx=0x7f03c4007c90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:07.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:07.850+0000 7f03d0ff9700 1 -- 172.21.15.28:0/3835456475 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f03c80521c0 con 0x7f03d40ce890 2023-11-28T11:10:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.048+0000 7f03da6f7700 1 -- 172.21.15.28:0/3835456475 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f03d40c8ba0 con 0x7f03d40ce890 2023-11-28T11:10:08.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.048+0000 7f03d0ff9700 1 -- 172.21.15.28:0/3835456475 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v36) v1 ==== 84+0+126 (secure 0 0 0) 0x7f03c80557e0 con 0x7f03d40ce890 2023-11-28T11:10:08.051 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:08.051 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2023-11-28T11:10:08.051 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:08.051 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 1, 2023-11-28T11:10:08.051 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": "rbd" 2023-11-28T11:10:08.051 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:08.051 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:08.051 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 2, 2023-11-28T11:10:08.052 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": ".mgr" 2023-11-28T11:10:08.052 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:08.052 INFO:teuthology.orchestra.run.smithi028.stdout:] 2023-11-28T11:10:08.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.050+0000 7f03da6f7700 1 -- 172.21.15.28:0/3835456475 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f03c0068320 msgr2=0x7f03c006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:08.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.050+0000 7f03da6f7700 1 --2- 172.21.15.28:0/3835456475 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f03c0068320 0x7f03c006a7d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000f60 tx=0x7f03c4007c90 comp rx=0 tx=0).stop 2023-11-28T11:10:08.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.050+0000 7f03da6f7700 1 -- 172.21.15.28:0/3835456475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d40ce890 msgr2=0x7f03d415b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.050+0000 7f03da6f7700 1 --2- 172.21.15.28:0/3835456475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d40ce890 0x7f03d415b970 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f03c800e810 tx=0x7f03c800ebd0 comp rx=0 tx=0).stop 2023-11-28T11:10:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.050+0000 7f03da6f7700 1 -- 172.21.15.28:0/3835456475 shutdown_connections 2023-11-28T11:10:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.050+0000 7f03da6f7700 1 --2- 172.21.15.28:0/3835456475 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f03c0068320 0x7f03c006a7d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.050+0000 7f03da6f7700 1 --2- 172.21.15.28:0/3835456475 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03d4159590 0x7f03d415beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.050+0000 7f03da6f7700 1 --2- 172.21.15.28:0/3835456475 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03d406e400 0x7f03d40c6da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.050+0000 7f03da6f7700 1 --2- 172.21.15.28:0/3835456475 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03d40ce890 0x7f03d415b970 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.050+0000 7f03da6f7700 1 -- 172.21.15.28:0/3835456475 >> 172.21.15.28:0/3835456475 conn(0x7f03d40b1650 msgr2=0x7f03d40caf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.051+0000 7f03da6f7700 1 -- 172.21.15.28:0/3835456475 shutdown_connections 2023-11-28T11:10:08.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.051+0000 7f03da6f7700 1 -- 172.21.15.28:0/3835456475 wait complete. 2023-11-28T11:10:08.064 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:10:08.065 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:10:08.193 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.192+0000 7efe7bb3a700 1 Processor -- start 2023-11-28T11:10:08.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.192+0000 7efe7bb3a700 1 -- start start 2023-11-28T11:10:08.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.193+0000 7efe7bb3a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe740cb0a0 0x7efe740ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:08.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.193+0000 7efe7bb3a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7efe7406bb80 0x7efe7414fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:08.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.193+0000 7efe7bb3a700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe7406c140 con 0x7efe740cb470 2023-11-28T11:10:08.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.193+0000 7efe7bb3a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe740caad0 con 0x7efe7406bb80 2023-11-28T11:10:08.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.193+0000 7efe7bb3a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe740c9ab0 con 0x7efe740cb0a0 2023-11-28T11:10:08.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.193+0000 7efe790d5700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7efe740cb470 0x7efe7415a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37390/0 (socket says 172.21.15.28:37390) 2023-11-28T11:10:08.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.193+0000 7efe790d5700 1 -- 172.21.15.28:0/3185280387 learned_addr learned my addr 172.21.15.28:0/3185280387 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:08.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.193+0000 7efe798d6700 1 --2- 172.21.15.28:0/3185280387 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe740cb0a0 0x7efe740ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:08.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.193+0000 7efe7a0d7700 1 --2- 172.21.15.28:0/3185280387 >> v2:172.21.15.59:3300/0 conn(0x7efe7406bb80 0x7efe7414fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:08.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.193+0000 7efe788d4700 1 -- 172.21.15.28:0/3185280387 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008338543 0 0) 0x7efe7406c140 con 0x7efe740cb470 2023-11-28T11:10:08.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.193+0000 7efe788d4700 1 -- 172.21.15.28:0/3185280387 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe600035b0 con 0x7efe740cb470 2023-11-28T11:10:08.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 -- 172.21.15.28:0/3185280387 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3572192597 0 0) 0x7efe600035b0 con 0x7efe740cb470 2023-11-28T11:10:08.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 -- 172.21.15.28:0/3185280387 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe740cb0a0 msgr2=0x7efe740ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:08.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 --2- 172.21.15.28:0/3185280387 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe740cb0a0 0x7efe740ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:08.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 -- 172.21.15.28:0/3185280387 >> v2:172.21.15.59:3300/0 conn(0x7efe7406bb80 msgr2=0x7efe7414fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:08.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 --2- 172.21.15.28:0/3185280387 >> v2:172.21.15.59:3300/0 conn(0x7efe7406bb80 0x7efe7414fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:08.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 -- 172.21.15.28:0/3185280387 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe740c9d90 con 0x7efe740cb470 2023-11-28T11:10:08.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 -- 172.21.15.28:0/3185280387 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7efe70001670 con 0x7efe740cb470 2023-11-28T11:10:08.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 -- 172.21.15.28:0/3185280387 >> v1:172.21.15.28:6789/0 conn(0x7efe740cb470 legacy=0x7efe7415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:08.200 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 --2- 172.21.15.28:0/3185280387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe60003a00 0x7efe60003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:08.201 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 --2- 172.21.15.28:0/3185280387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe60004430 0x7efe600048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:08.201 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 --2- 172.21.15.28:0/3185280387 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe740cb0a0 0x7efe60005060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:08.201 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 -- 172.21.15.28:0/3185280387 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe740c9ab0 con 0x7efe60003a00 2023-11-28T11:10:08.201 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 -- 172.21.15.28:0/3185280387 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6000ba10 con 0x7efe60004430 2023-11-28T11:10:08.201 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 -- 172.21.15.28:0/3185280387 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6000bb50 con 0x7efe740cb0a0 2023-11-28T11:10:08.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe798d6700 1 --2- 172.21.15.28:0/3185280387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe60003a00 0x7efe60003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:08.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe7a0d7700 1 --2- 172.21.15.28:0/3185280387 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe740cb0a0 0x7efe60005060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:08.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe790d5700 1 --2- 172.21.15.28:0/3185280387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe60004430 0x7efe600048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:08.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe798d6700 1 -- 172.21.15.28:0/3185280387 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe740cb0a0 msgr2=0x7efe60005060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:08.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe798d6700 1 --2- 172.21.15.28:0/3185280387 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe740cb0a0 0x7efe60005060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:08.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe798d6700 1 -- 172.21.15.28:0/3185280387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe60004430 msgr2=0x7efe600048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:08.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe798d6700 1 --2- 172.21.15.28:0/3185280387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe60004430 0x7efe600048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:08.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe798d6700 1 -- 172.21.15.28:0/3185280387 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe600035b0 con 0x7efe60003a00 2023-11-28T11:10:08.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe790d5700 1 --2- 172.21.15.28:0/3185280387 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe60004430 0x7efe600048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:08.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe7a0d7700 1 --2- 172.21.15.28:0/3185280387 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe740cb0a0 0x7efe60005060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:08.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe798d6700 1 --2- 172.21.15.28:0/3185280387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe60003a00 0x7efe60003e10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efe64009b90 tx=0x7efe60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62536423185a8838 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:08.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 -- 172.21.15.28:0/3185280387 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6400e6c0 con 0x7efe60003a00 2023-11-28T11:10:08.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 -- 172.21.15.28:0/3185280387 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe6400e820 con 0x7efe60003a00 2023-11-28T11:10:08.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.194+0000 7efe788d4700 1 -- 172.21.15.28:0/3185280387 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe6400eb90 con 0x7efe60003a00 2023-11-28T11:10:08.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.195+0000 7efe7bb3a700 1 -- 172.21.15.28:0/3185280387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe60003a00 msgr2=0x7efe60003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:08.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.195+0000 7efe7bb3a700 1 --2- 172.21.15.28:0/3185280387 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe60003a00 0x7efe60003e10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7efe64009b90 tx=0x7efe60005880 comp rx=0 tx=0).stop 2023-11-28T11:10:08.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.195+0000 7efe7a0d7700 1 -- 172.21.15.28:0/3185280387 reap_dead start 2023-11-28T11:10:08.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.195+0000 7efe7bb3a700 1 -- 172.21.15.28:0/3185280387 shutdown_connections 2023-11-28T11:10:08.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.195+0000 7efe7bb3a700 1 -- 172.21.15.28:0/3185280387 >> 172.21.15.28:0/3185280387 conn(0x7efe740b0dd0 msgr2=0x7efe740b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:08.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.195+0000 7efe7bb3a700 1 -- 172.21.15.28:0/3185280387 shutdown_connections 2023-11-28T11:10:08.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.195+0000 7efe7bb3a700 1 -- 172.21.15.28:0/3185280387 wait complete. 2023-11-28T11:10:08.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.196+0000 7efe7bb3a700 1 Processor -- start 2023-11-28T11:10:08.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.196+0000 7efe7bb3a700 1 -- start start 2023-11-28T11:10:08.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe7bb3a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe60003a00 0x7efe740ab970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:08.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe7bb3a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe740adf40 0x7efe740c8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:08.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe7bb3a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe740c7bc0 0x7efe7406aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:08.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe7bb3a700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe740ac9f0 con 0x7efe740adf40 2023-11-28T11:10:08.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe7bb3a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe740acc60 con 0x7efe60003a00 2023-11-28T11:10:08.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe7bb3a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe740acf40 con 0x7efe740c7bc0 2023-11-28T11:10:08.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe790d5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe740adf40 0x7efe740c8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:08.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe790d5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe740adf40 0x7efe740c8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49346/0 (socket says 172.21.15.28:49346) 2023-11-28T11:10:08.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe790d5700 1 -- 172.21.15.28:0/1265822398 learned_addr learned my addr 172.21.15.28:0/1265822398 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:08.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe7a0d7700 1 --2- 172.21.15.28:0/1265822398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe740c7bc0 0x7efe7406aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:08.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe798d6700 1 --2- 172.21.15.28:0/1265822398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe60003a00 0x7efe740ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:08.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe790d5700 1 -- 172.21.15.28:0/1265822398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe740c7bc0 msgr2=0x7efe7406aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:08.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe790d5700 1 --2- 172.21.15.28:0/1265822398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe740c7bc0 0x7efe7406aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:08.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe790d5700 1 -- 172.21.15.28:0/1265822398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe60003a00 msgr2=0x7efe740ab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:08.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe790d5700 1 --2- 172.21.15.28:0/1265822398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe60003a00 0x7efe740ab970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:08.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.197+0000 7efe790d5700 1 -- 172.21.15.28:0/1265822398 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe740ca590 con 0x7efe740adf40 2023-11-28T11:10:08.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.198+0000 7efe790d5700 1 --2- 172.21.15.28:0/1265822398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe740adf40 0x7efe740c8ca0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efe7000eaf0 tx=0x7efe7000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:08.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.198+0000 7efe6a7fc700 1 -- 172.21.15.28:0/1265822398 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe70005e00 con 0x7efe740adf40 2023-11-28T11:10:08.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.198+0000 7efe6a7fc700 1 -- 172.21.15.28:0/1265822398 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7000c7b0 con 0x7efe740adf40 2023-11-28T11:10:08.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.198+0000 7efe6a7fc700 1 -- 172.21.15.28:0/1265822398 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7000cb60 con 0x7efe740adf40 2023-11-28T11:10:08.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.198+0000 7efe7bb3a700 1 -- 172.21.15.28:0/1265822398 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe740ca070 con 0x7efe740adf40 2023-11-28T11:10:08.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.198+0000 7efe7bb3a700 1 -- 172.21.15.28:0/1265822398 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe740ca720 con 0x7efe740adf40 2023-11-28T11:10:08.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.199+0000 7efe7bb3a700 1 -- 172.21.15.28:0/1265822398 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe740739f0 con 0x7efe740adf40 2023-11-28T11:10:08.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.199+0000 7efe6a7fc700 1 -- 172.21.15.28:0/1265822398 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7efe7000ccc0 con 0x7efe740adf40 2023-11-28T11:10:08.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.200+0000 7efe6a7fc700 1 --2- 172.21.15.28:0/1265822398 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efe580682f0 0x7efe5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:08.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.200+0000 7efe6a7fc700 1 -- 172.21.15.28:0/1265822398 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5530+0+0 (secure 0 0 0) 0x7efe70085b20 con 0x7efe740adf40 2023-11-28T11:10:08.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.200+0000 7efe798d6700 1 --2- 172.21.15.28:0/1265822398 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efe580682f0 0x7efe5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:08.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.200+0000 7efe798d6700 1 --2- 172.21.15.28:0/1265822398 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efe580682f0 0x7efe5806a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efe640090b0 tx=0x7efe64008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:08.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.204+0000 7efe6a7fc700 1 -- 172.21.15.28:0/1265822398 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7efe700524a0 con 0x7efe740adf40 2023-11-28T11:10:08.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:08.398+0000 7efe7bb3a700 1 -- 172.21.15.28:0/1265822398 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7efe740b5260 con 0x7efe740adf40 2023-11-28T11:10:09.401 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.399+0000 7efe6a7fc700 1 -- 172.21.15.28:0/1265822398 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v37) v1 ==== 138+0+0 (secure 0 0 0) 0x7efe70055ac0 con 0x7efe740adf40 2023-11-28T11:10:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.404+0000 7efe7bb3a700 1 -- 172.21.15.28:0/1265822398 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efe580682f0 msgr2=0x7efe5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.404+0000 7efe7bb3a700 1 --2- 172.21.15.28:0/1265822398 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efe580682f0 0x7efe5806a7a0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7efe640090b0 tx=0x7efe64008030 comp rx=0 tx=0).stop 2023-11-28T11:10:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.404+0000 7efe7bb3a700 1 -- 172.21.15.28:0/1265822398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe740adf40 msgr2=0x7efe740c8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.404+0000 7efe7bb3a700 1 --2- 172.21.15.28:0/1265822398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe740adf40 0x7efe740c8ca0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7efe7000eaf0 tx=0x7efe7000eeb0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.405+0000 7efe7bb3a700 1 -- 172.21.15.28:0/1265822398 shutdown_connections 2023-11-28T11:10:09.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.405+0000 7efe7bb3a700 1 --2- 172.21.15.28:0/1265822398 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7efe580682f0 0x7efe5806a7a0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.405+0000 7efe7bb3a700 1 --2- 172.21.15.28:0/1265822398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7efe740c7bc0 0x7efe7406aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.405+0000 7efe7bb3a700 1 --2- 172.21.15.28:0/1265822398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efe60003a00 0x7efe740ab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.405+0000 7efe7bb3a700 1 --2- 172.21.15.28:0/1265822398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7efe740adf40 0x7efe740c8ca0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.405+0000 7efe7bb3a700 1 -- 172.21.15.28:0/1265822398 >> 172.21.15.28:0/1265822398 conn(0x7efe740b0dd0 msgr2=0x7efe74157730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:09.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.405+0000 7efe7bb3a700 1 -- 172.21.15.28:0/1265822398 shutdown_connections 2023-11-28T11:10:09.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.405+0000 7efe7bb3a700 1 -- 172.21.15.28:0/1265822398 wait complete. 2023-11-28T11:10:09.409 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' created 2023-11-28T11:10:09.423 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:10:09.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.560+0000 7f54a1135700 1 Processor -- start 2023-11-28T11:10:09.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.560+0000 7f54a1135700 1 -- start start 2023-11-28T11:10:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.561+0000 7f54a1135700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f549c15a490 0x7f549c06f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.561+0000 7f54a1135700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f549c0b2150 0x7f549c06d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.561+0000 7f54a1135700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f549c06bcd0 0x7f549c06db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.561+0000 7f54a1135700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549c15a860 con 0x7f549c0b2150 2023-11-28T11:10:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.561+0000 7f54a1135700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549c06b510 con 0x7f549c15a490 2023-11-28T11:10:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.561+0000 7f54a1135700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0c9a80 con 0x7f549c06bcd0 2023-11-28T11:10:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.561+0000 7f549a59c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f549c0b2150 0x7f549c06d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.561+0000 7f549a59c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f549c0b2150 0x7f549c06d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49352/0 (socket says 172.21.15.28:49352) 2023-11-28T11:10:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.561+0000 7f549a59c700 1 -- 172.21.15.28:0/1117044939 learned_addr learned my addr 172.21.15.28:0/1117044939 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:09.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.561+0000 7f549ad9d700 1 --2- 172.21.15.28:0/1117044939 >> v2:172.21.15.59:3300/0 conn(0x7f549c15a490 0x7f549c06f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.561+0000 7f549b59e700 1 --2- 172.21.15.28:0/1117044939 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f549c06bcd0 0x7f549c06db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f549a59c700 1 -- 172.21.15.28:0/1117044939 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f549c06bcd0 msgr2=0x7f549c06db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f549a59c700 1 --2- 172.21.15.28:0/1117044939 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f549c06bcd0 0x7f549c06db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f549a59c700 1 -- 172.21.15.28:0/1117044939 >> v2:172.21.15.59:3300/0 conn(0x7f549c15a490 msgr2=0x7f549c06f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f549a59c700 1 --2- 172.21.15.28:0/1117044939 >> v2:172.21.15.59:3300/0 conn(0x7f549c15a490 0x7f549c06f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f549a59c700 1 -- 172.21.15.28:0/1117044939 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c06b230 con 0x7f549c0b2150 2023-11-28T11:10:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f549ad9d700 1 --2- 172.21.15.28:0/1117044939 >> v2:172.21.15.59:3300/0 conn(0x7f549c15a490 0x7f549c06f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f549a59c700 1 --2- 172.21.15.28:0/1117044939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f549c0b2150 0x7f549c06d630 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f549c0cf1b0 tx=0x7f548c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f0c6fcbb6cfd199 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f5499d9b700 1 -- 172.21.15.28:0/1117044939 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c011830 con 0x7f549c0b2150 2023-11-28T11:10:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f5499d9b700 1 -- 172.21.15.28:0/1117044939 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c011990 con 0x7f549c0b2150 2023-11-28T11:10:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f5499d9b700 1 -- 172.21.15.28:0/1117044939 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c011d20 con 0x7f549c0b2150 2023-11-28T11:10:09.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f54a1135700 1 -- 172.21.15.28:0/1117044939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f549c0b2150 msgr2=0x7f549c06d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f54a1135700 1 --2- 172.21.15.28:0/1117044939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f549c0b2150 0x7f549c06d630 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f549c0cf1b0 tx=0x7f548c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f54a1135700 1 -- 172.21.15.28:0/1117044939 shutdown_connections 2023-11-28T11:10:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f54a1135700 1 --2- 172.21.15.28:0/1117044939 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f549c06bcd0 0x7f549c06db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f54a1135700 1 --2- 172.21.15.28:0/1117044939 >> v2:172.21.15.59:3300/0 conn(0x7f549c15a490 0x7f549c06f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f54a1135700 1 --2- 172.21.15.28:0/1117044939 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f549c0b2150 0x7f549c06d630 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f54a1135700 1 -- 172.21.15.28:0/1117044939 >> 172.21.15.28:0/1117044939 conn(0x7f549c0b0df0 msgr2=0x7f549c0b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.562+0000 7f54a1135700 1 -- 172.21.15.28:0/1117044939 shutdown_connections 2023-11-28T11:10:09.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.563+0000 7f54a1135700 1 -- 172.21.15.28:0/1117044939 wait complete. 2023-11-28T11:10:09.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.564+0000 7f54a1135700 1 Processor -- start 2023-11-28T11:10:09.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.564+0000 7f54a1135700 1 -- start start 2023-11-28T11:10:09.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f54a1135700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f549c06bcd0 0x7f549c158ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f54a1135700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f549c0b2150 0x7f549c0a8180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f54a1135700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f549c15a490 0x7f549c0aa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f54a1135700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0ac800 con 0x7f549c0b2150 2023-11-28T11:10:09.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f54a1135700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0ac220 con 0x7f549c15a490 2023-11-28T11:10:09.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f54a1135700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c159000 con 0x7f549c06bcd0 2023-11-28T11:10:09.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f549a59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f549c0b2150 0x7f549c0a8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f549a59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f549c0b2150 0x7f549c0a8180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49360/0 (socket says 172.21.15.28:49360) 2023-11-28T11:10:09.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f549a59c700 1 -- 172.21.15.28:0/2708835082 learned_addr learned my addr 172.21.15.28:0/2708835082 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:09.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f549ad9d700 1 --2- 172.21.15.28:0/2708835082 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f549c06bcd0 0x7f549c158ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f549b59e700 1 --2- 172.21.15.28:0/2708835082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f549c15a490 0x7f549c0aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f549a59c700 1 -- 172.21.15.28:0/2708835082 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f549c06bcd0 msgr2=0x7f549c158ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f549a59c700 1 --2- 172.21.15.28:0/2708835082 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f549c06bcd0 0x7f549c158ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f549a59c700 1 -- 172.21.15.28:0/2708835082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f549c15a490 msgr2=0x7f549c0aa150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:09.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f549a59c700 1 --2- 172.21.15.28:0/2708835082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f549c15a490 0x7f549c0aa150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:09.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.565+0000 7f549a59c700 1 -- 172.21.15.28:0/2708835082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c009550 con 0x7f549c0b2150 2023-11-28T11:10:09.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.566+0000 7f549a59c700 1 --2- 172.21.15.28:0/2708835082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f549c0b2150 0x7f549c0a8180 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f549c06d180 tx=0x7f548c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.566+0000 7f54837fe700 1 -- 172.21.15.28:0/2708835082 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c004450 con 0x7f549c0b2150 2023-11-28T11:10:09.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.566+0000 7f54837fe700 1 -- 172.21.15.28:0/2708835082 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c01e070 con 0x7f549c0b2150 2023-11-28T11:10:09.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.566+0000 7f54a1135700 1 -- 172.21.15.28:0/2708835082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c0ca000 con 0x7f549c0b2150 2023-11-28T11:10:09.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.566+0000 7f54837fe700 1 -- 172.21.15.28:0/2708835082 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c0038e0 con 0x7f549c0b2150 2023-11-28T11:10:09.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.566+0000 7f54a1135700 1 -- 172.21.15.28:0/2708835082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c0c9d20 con 0x7f549c0b2150 2023-11-28T11:10:09.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.567+0000 7f54a1135700 1 -- 172.21.15.28:0/2708835082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c06ae90 con 0x7f549c0b2150 2023-11-28T11:10:09.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.568+0000 7f54837fe700 1 -- 172.21.15.28:0/2708835082 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f548c00f050 con 0x7f549c0b2150 2023-11-28T11:10:09.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.568+0000 7f54837fe700 1 --2- 172.21.15.28:0/2708835082 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f54880682f0 0x7f548806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:09.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.568+0000 7f54837fe700 1 -- 172.21.15.28:0/2708835082 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f548c087270 con 0x7f549c0b2150 2023-11-28T11:10:09.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.568+0000 7f549ad9d700 1 --2- 172.21.15.28:0/2708835082 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f54880682f0 0x7f548806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:09.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.569+0000 7f549ad9d700 1 --2- 172.21.15.28:0/2708835082 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f54880682f0 0x7f548806a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f549c0b00e0 tx=0x7f5490008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:09.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.572+0000 7f54837fe700 1 -- 172.21.15.28:0/2708835082 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f548c053b00 con 0x7f549c0b2150 2023-11-28T11:10:09.767 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:09.765+0000 7f54a1135700 1 -- 172.21.15.28:0/2708835082 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f549c0ac9b0 con 0x7f549c0b2150 2023-11-28T11:10:10.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.405+0000 7f54837fe700 1 -- 172.21.15.28:0/2708835082 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v38) v1 ==== 170+0+0 (secure 0 0 0) 0x7f549c0ac9b0 con 0x7f549c0b2150 2023-11-28T11:10:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.412+0000 7f54a1135700 1 -- 172.21.15.28:0/2708835082 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f54880682f0 msgr2=0x7f548806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.412+0000 7f54a1135700 1 --2- 172.21.15.28:0/2708835082 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f54880682f0 0x7f548806a7a0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f549c0b00e0 tx=0x7f5490008040 comp rx=0 tx=0).stop 2023-11-28T11:10:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.412+0000 7f54a1135700 1 -- 172.21.15.28:0/2708835082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f549c0b2150 msgr2=0x7f549c0a8180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.412+0000 7f54a1135700 1 --2- 172.21.15.28:0/2708835082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f549c0b2150 0x7f549c0a8180 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f549c06d180 tx=0x7f548c004750 comp rx=0 tx=0).stop 2023-11-28T11:10:10.414 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.413+0000 7f54a1135700 1 -- 172.21.15.28:0/2708835082 shutdown_connections 2023-11-28T11:10:10.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.413+0000 7f54a1135700 1 --2- 172.21.15.28:0/2708835082 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f54880682f0 0x7f548806a7a0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.413+0000 7f54a1135700 1 --2- 172.21.15.28:0/2708835082 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f549c15a490 0x7f549c0aa150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.413+0000 7f54a1135700 1 --2- 172.21.15.28:0/2708835082 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f549c06bcd0 0x7f549c158ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.413+0000 7f54a1135700 1 --2- 172.21.15.28:0/2708835082 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f549c0b2150 0x7f549c0a8180 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.413+0000 7f54a1135700 1 -- 172.21.15.28:0/2708835082 >> 172.21.15.28:0/2708835082 conn(0x7f549c0b0df0 msgr2=0x7f549c15ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:10.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.414+0000 7f54a1135700 1 -- 172.21.15.28:0/2708835082 shutdown_connections 2023-11-28T11:10:10.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.414+0000 7f54a1135700 1 -- 172.21.15.28:0/2708835082 wait complete. 2023-11-28T11:10:10.426 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' created 2023-11-28T11:10:10.437 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:10:10.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.566+0000 7f7395eee700 1 Processor -- start 2023-11-28T11:10:10.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.567+0000 7f7395eee700 1 -- start start 2023-11-28T11:10:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.567+0000 7f7395eee700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f73900cd620 0x7f739015cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.567+0000 7f7395eee700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73900cc860 con 0x7f7390071900 2023-11-28T11:10:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.567+0000 7f7395eee700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73900ccb40 con 0x7f7390158ad0 2023-11-28T11:10:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.567+0000 7f7395eee700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73900cce20 con 0x7f73900cd620 2023-11-28T11:10:10.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.567+0000 7f738ffff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f73900cd620 0x7f739015cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.567+0000 7f738ffff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f73900cd620 0x7f739015cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49370/0 (socket says 172.21.15.28:49370) 2023-11-28T11:10:10.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.567+0000 7f738effd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7390071900 0x7f739015c8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37426/0 (socket says 172.21.15.28:37426) 2023-11-28T11:10:10.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.567+0000 7f738ffff700 1 -- 172.21.15.28:0/2240748331 learned_addr learned my addr 172.21.15.28:0/2240748331 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:10.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.568+0000 7f738ffff700 1 -- 172.21.15.28:0/2240748331 >> v1:172.21.15.59:6789/0 conn(0x7f7390158ad0 legacy=0x7f73900711f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:10.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.568+0000 7f738ffff700 1 -- 172.21.15.28:0/2240748331 >> v1:172.21.15.28:6789/0 conn(0x7f7390071900 legacy=0x7f739015c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.568+0000 7f738ffff700 1 -- 172.21.15.28:0/2240748331 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73900cd100 con 0x7f73900cd620 2023-11-28T11:10:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.568+0000 7f738e7fc700 1 -- 172.21.15.28:0/2240748331 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1934958020 0 0) 0x7f73900cc860 con 0x7f7390071900 2023-11-28T11:10:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.568+0000 7f738ffff700 1 --2- 172.21.15.28:0/2240748331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73900cd620 0x7f739015cfc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f738800d150 tx=0x7f738800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e419e75130eb1a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.568+0000 7f738e7fc700 1 -- 172.21.15.28:0/2240748331 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7388010070 con 0x7f73900cd620 2023-11-28T11:10:10.580 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.568+0000 7f738e7fc700 1 -- 172.21.15.28:0/2240748331 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f738800ddc0 con 0x7f73900cd620 2023-11-28T11:10:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.568+0000 7f738e7fc700 1 -- 172.21.15.28:0/2240748331 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f738800b310 con 0x7f73900cd620 2023-11-28T11:10:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.569+0000 7f7395eee700 1 -- 172.21.15.28:0/2240748331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73900cd620 msgr2=0x7f739015cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.569+0000 7f7395eee700 1 --2- 172.21.15.28:0/2240748331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73900cd620 0x7f739015cfc0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f738800d150 tx=0x7f738800d460 comp rx=0 tx=0).stop 2023-11-28T11:10:10.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.569+0000 7f7395eee700 1 -- 172.21.15.28:0/2240748331 shutdown_connections 2023-11-28T11:10:10.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.569+0000 7f7395eee700 1 --2- 172.21.15.28:0/2240748331 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73900cd620 0x7f739015cfc0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.569+0000 7f7395eee700 1 -- 172.21.15.28:0/2240748331 >> 172.21.15.28:0/2240748331 conn(0x7f73900ce9e0 msgr2=0x7f73900cedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:10.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.569+0000 7f7395eee700 1 -- 172.21.15.28:0/2240748331 shutdown_connections 2023-11-28T11:10:10.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.569+0000 7f7395eee700 1 -- 172.21.15.28:0/2240748331 wait complete. 2023-11-28T11:10:10.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.570+0000 7f7395eee700 1 Processor -- start 2023-11-28T11:10:10.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.570+0000 7f7395eee700 1 -- start start 2023-11-28T11:10:10.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f7395eee700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7390071900 0x7f73900c6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f7395eee700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73900cd620 0x7f73900b1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f7395eee700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7390158ad0 0x7f739013ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f7395eee700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73900c7040 con 0x7f7390071900 2023-11-28T11:10:10.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f7395eee700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73900c72e0 con 0x7f7390158ad0 2023-11-28T11:10:10.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f7395eee700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73900c75c0 con 0x7f73900cd620 2023-11-28T11:10:10.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f738f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7390071900 0x7f73900c6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f738f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7390071900 0x7f73900c6b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49372/0 (socket says 172.21.15.28:49372) 2023-11-28T11:10:10.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f738f7fe700 1 -- 172.21.15.28:0/3683206263 learned_addr learned my addr 172.21.15.28:0/3683206263 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:10.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f738ffff700 1 --2- 172.21.15.28:0/3683206263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7390158ad0 0x7f739013ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f738effd700 1 --2- 172.21.15.28:0/3683206263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73900cd620 0x7f73900b1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f738f7fe700 1 -- 172.21.15.28:0/3683206263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73900cd620 msgr2=0x7f73900b1b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f738f7fe700 1 --2- 172.21.15.28:0/3683206263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73900cd620 0x7f73900b1b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f738f7fe700 1 -- 172.21.15.28:0/3683206263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7390158ad0 msgr2=0x7f739013ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f738f7fe700 1 --2- 172.21.15.28:0/3683206263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7390158ad0 0x7f739013ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f738f7fe700 1 -- 172.21.15.28:0/3683206263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738800bd40 con 0x7f7390071900 2023-11-28T11:10:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f738effd700 1 --2- 172.21.15.28:0/3683206263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73900cd620 0x7f73900b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.571+0000 7f738f7fe700 1 --2- 172.21.15.28:0/3683206263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7390071900 0x7f73900c6b00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f737800c4e0 tx=0x7f737800c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.572+0000 7f7387fff700 1 -- 172.21.15.28:0/3683206263 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73780052d0 con 0x7f7390071900 2023-11-28T11:10:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.572+0000 7f7387fff700 1 -- 172.21.15.28:0/3683206263 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7378005430 con 0x7f7390071900 2023-11-28T11:10:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.572+0000 7f7395eee700 1 -- 172.21.15.28:0/3683206263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7390158ea0 con 0x7f7390071900 2023-11-28T11:10:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.572+0000 7f7387fff700 1 -- 172.21.15.28:0/3683206263 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73780038c0 con 0x7f7390071900 2023-11-28T11:10:10.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.572+0000 7f7395eee700 1 -- 172.21.15.28:0/3683206263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f739015d610 con 0x7f7390071900 2023-11-28T11:10:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.573+0000 7f7395eee700 1 -- 172.21.15.28:0/3683206263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7390159030 con 0x7f7390071900 2023-11-28T11:10:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.573+0000 7f7387fff700 1 -- 172.21.15.28:0/3683206263 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7378019030 con 0x7f7390071900 2023-11-28T11:10:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.573+0000 7f7387fff700 1 --2- 172.21.15.28:0/3683206263 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f73700682f0 0x7f737006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.574+0000 7f7387fff700 1 -- 172.21.15.28:0/3683206263 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f73780851a0 con 0x7f7390071900 2023-11-28T11:10:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.574+0000 7f738effd700 1 --2- 172.21.15.28:0/3683206263 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f73700682f0 0x7f737006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.574+0000 7f738effd700 1 --2- 172.21.15.28:0/3683206263 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f73700682f0 0x7f737006a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f73901571f0 tx=0x7f7380009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:10.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.578+0000 7f7387fff700 1 -- 172.21.15.28:0/3683206263 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7378051890 con 0x7f7390071900 2023-11-28T11:10:10.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.818+0000 7f7395eee700 1 -- 172.21.15.28:0/3683206263 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f739006cef0 con 0x7f7390071900 2023-11-28T11:10:10.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.825+0000 7f7387fff700 1 -- 172.21.15.28:0/3683206263 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7f73780170c0 con 0x7f7390071900 2023-11-28T11:10:10.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.829+0000 7f7395eee700 1 -- 172.21.15.28:0/3683206263 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f73700682f0 msgr2=0x7f737006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.829+0000 7f7395eee700 1 --2- 172.21.15.28:0/3683206263 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f73700682f0 0x7f737006a7a0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f73901571f0 tx=0x7f7380009040 comp rx=0 tx=0).stop 2023-11-28T11:10:10.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.829+0000 7f7395eee700 1 -- 172.21.15.28:0/3683206263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7390071900 msgr2=0x7f73900c6b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.829+0000 7f7395eee700 1 --2- 172.21.15.28:0/3683206263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7390071900 0x7f73900c6b00 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f737800c4e0 tx=0x7f737800c8a0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.829+0000 7f7395eee700 1 -- 172.21.15.28:0/3683206263 shutdown_connections 2023-11-28T11:10:10.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.829+0000 7f7395eee700 1 --2- 172.21.15.28:0/3683206263 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f73700682f0 0x7f737006a7a0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.829+0000 7f7395eee700 1 --2- 172.21.15.28:0/3683206263 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7390071900 0x7f73900c6b00 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.829+0000 7f7395eee700 1 --2- 172.21.15.28:0/3683206263 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7390158ad0 0x7f739013ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.829+0000 7f7395eee700 1 --2- 172.21.15.28:0/3683206263 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73900cd620 0x7f73900b1b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.829+0000 7f7395eee700 1 -- 172.21.15.28:0/3683206263 >> 172.21.15.28:0/3683206263 conn(0x7f73900ce9e0 msgr2=0x7f739014ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:10.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.830+0000 7f7395eee700 1 -- 172.21.15.28:0/3683206263 shutdown_connections 2023-11-28T11:10:10.834 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.830+0000 7f7395eee700 1 -- 172.21.15.28:0/3683206263 wait complete. 2023-11-28T11:10:10.838 INFO:teuthology.orchestra.run.smithi028.stderr:new fs with metadata pool 6 and data pool 7 2023-11-28T11:10:10.858 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:10.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.981+0000 7f1b142a9700 1 Processor -- start 2023-11-28T11:10:10.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.981+0000 7f1b142a9700 1 -- start start 2023-11-28T11:10:10.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.982+0000 7f1b142a9700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1b0c161090 0x7f1b0c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.982+0000 7f1b142a9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1b0c0caeb0 0x7f1b0c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.982+0000 7f1b142a9700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b0c0ce340 con 0x7f1b0c15c990 2023-11-28T11:10:10.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.982+0000 7f1b142a9700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0c0cebe0 con 0x7f1b0c161090 2023-11-28T11:10:10.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.982+0000 7f1b142a9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b0c0cf780 con 0x7f1b0c0caeb0 2023-11-28T11:10:10.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.982+0000 7f1b11844700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f1b0c161090 0x7f1b0c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.982+0000 7f1b12045700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1b0c15c990 0x7f1b0c0cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37438/0 (socket says 172.21.15.28:37438) 2023-11-28T11:10:10.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.982+0000 7f1b12045700 1 -- 172.21.15.28:0/2415538274 learned_addr learned my addr 172.21.15.28:0/2415538274 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:10.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.982+0000 7f1b12846700 1 --2- 172.21.15.28:0/2415538274 >> v2:172.21.15.59:3300/0 conn(0x7f1b0c0caeb0 0x7f1b0c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.982+0000 7f1b11844700 1 -- 172.21.15.28:0/2415538274 >> v2:172.21.15.59:3300/0 conn(0x7f1b0c0caeb0 msgr2=0x7f1b0c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:10.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.982+0000 7f1b11844700 1 --2- 172.21.15.28:0/2415538274 >> v2:172.21.15.59:3300/0 conn(0x7f1b0c0caeb0 0x7f1b0c0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.982+0000 7f1b11844700 1 -- 172.21.15.28:0/2415538274 >> v1:172.21.15.28:6789/0 conn(0x7f1b0c15c990 legacy=0x7f1b0c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.982+0000 7f1b11844700 1 -- 172.21.15.28:0/2415538274 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b0c0cb3f0 con 0x7f1b0c161090 2023-11-28T11:10:10.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.982+0000 7f1b12846700 1 --2- 172.21.15.28:0/2415538274 >> v2:172.21.15.59:3300/0 conn(0x7f1b0c0caeb0 0x7f1b0c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:10.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.983+0000 7f1b11844700 1 --2- 172.21.15.28:0/2415538274 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b0c161090 0x7f1b0c0ca970 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c071a50 tx=0x7f1b08009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=351cabac86239f24 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:10.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.983+0000 7f1b11043700 1 -- 172.21.15.28:0/2415538274 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08011830 con 0x7f1b0c161090 2023-11-28T11:10:10.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.983+0000 7f1b11043700 1 -- 172.21.15.28:0/2415538274 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b08011990 con 0x7f1b0c161090 2023-11-28T11:10:10.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.983+0000 7f1b11043700 1 -- 172.21.15.28:0/2415538274 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b08011d20 con 0x7f1b0c161090 2023-11-28T11:10:10.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.983+0000 7f1b142a9700 1 -- 172.21.15.28:0/2415538274 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b0c161090 msgr2=0x7f1b0c0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.983+0000 7f1b142a9700 1 --2- 172.21.15.28:0/2415538274 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b0c161090 0x7f1b0c0ca970 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c071a50 tx=0x7f1b08009b20 comp rx=0 tx=0).stop 2023-11-28T11:10:10.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.983+0000 7f1b142a9700 1 -- 172.21.15.28:0/2415538274 shutdown_connections 2023-11-28T11:10:10.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.983+0000 7f1b142a9700 1 --2- 172.21.15.28:0/2415538274 >> v2:172.21.15.59:3300/0 conn(0x7f1b0c0caeb0 0x7f1b0c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.983+0000 7f1b142a9700 1 --2- 172.21.15.28:0/2415538274 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b0c161090 0x7f1b0c0ca970 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.983+0000 7f1b142a9700 1 -- 172.21.15.28:0/2415538274 >> 172.21.15.28:0/2415538274 conn(0x7f1b0c0b1650 msgr2=0x7f1b0c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:10.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.983+0000 7f1b142a9700 1 -- 172.21.15.28:0/2415538274 shutdown_connections 2023-11-28T11:10:10.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.983+0000 7f1b142a9700 1 -- 172.21.15.28:0/2415538274 wait complete. 2023-11-28T11:10:10.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.985+0000 7f1b142a9700 1 Processor -- start 2023-11-28T11:10:10.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.985+0000 7f1b142a9700 1 -- start start 2023-11-28T11:10:10.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.985+0000 7f1b142a9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b0c0caeb0 0x7f1b0c157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.985+0000 7f1b142a9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b0c15c990 0x7f1b0c0ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.985+0000 7f1b142a9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c161090 0x7f1b0c15f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.985+0000 7f1b142a9700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c157a40 con 0x7f1b0c0caeb0 2023-11-28T11:10:10.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.985+0000 7f1b142a9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0c77f0 con 0x7f1b0c161090 2023-11-28T11:10:10.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.985+0000 7f1b142a9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b0c0ac0f0 con 0x7f1b0c15c990 2023-11-28T11:10:10.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.985+0000 7f1b12045700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b0c0caeb0 0x7f1b0c157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b12045700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b0c0caeb0 0x7f1b0c157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49384/0 (socket says 172.21.15.28:49384) 2023-11-28T11:10:10.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b12045700 1 -- 172.21.15.28:0/3292926707 learned_addr learned my addr 172.21.15.28:0/3292926707 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:10.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b11844700 1 --2- 172.21.15.28:0/3292926707 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b0c15c990 0x7f1b0c0ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b12846700 1 --2- 172.21.15.28:0/3292926707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c161090 0x7f1b0c15f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b12045700 1 -- 172.21.15.28:0/3292926707 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b0c15c990 msgr2=0x7f1b0c0ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b12045700 1 --2- 172.21.15.28:0/3292926707 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b0c15c990 0x7f1b0c0ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b12045700 1 -- 172.21.15.28:0/3292926707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c161090 msgr2=0x7f1b0c15f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:10.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b12045700 1 --2- 172.21.15.28:0/3292926707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c161090 0x7f1b0c15f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:10.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b12045700 1 -- 172.21.15.28:0/3292926707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b080094a0 con 0x7f1b0c0caeb0 2023-11-28T11:10:10.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b11844700 1 --2- 172.21.15.28:0/3292926707 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b0c15c990 0x7f1b0c0ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:10.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b12045700 1 --2- 172.21.15.28:0/3292926707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b0c0caeb0 0x7f1b0c157500 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00c520 tx=0x7f1afc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:10.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b02ffd700 1 -- 172.21.15.28:0/3292926707 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc005d70 con 0x7f1b0c0caeb0 2023-11-28T11:10:10.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b02ffd700 1 -- 172.21.15.28:0/3292926707 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1afc0070d0 con 0x7f1b0c0caeb0 2023-11-28T11:10:10.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b142a9700 1 -- 172.21.15.28:0/3292926707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b0c0ce870 con 0x7f1b0c0caeb0 2023-11-28T11:10:10.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b02ffd700 1 -- 172.21.15.28:0/3292926707 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1afc007440 con 0x7f1b0c0caeb0 2023-11-28T11:10:10.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.986+0000 7f1b142a9700 1 -- 172.21.15.28:0/3292926707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b0c0ce590 con 0x7f1b0c0caeb0 2023-11-28T11:10:10.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.987+0000 7f1b142a9700 1 -- 172.21.15.28:0/3292926707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b0c0cc850 con 0x7f1b0c0caeb0 2023-11-28T11:10:10.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.988+0000 7f1b02ffd700 1 -- 172.21.15.28:0/3292926707 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1afc019030 con 0x7f1b0c0caeb0 2023-11-28T11:10:10.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.988+0000 7f1b02ffd700 1 --2- 172.21.15.28:0/3292926707 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1af00682f0 0x7f1af006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:10.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.988+0000 7f1b02ffd700 1 -- 172.21.15.28:0/3292926707 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1afc085040 con 0x7f1b0c0caeb0 2023-11-28T11:10:10.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.988+0000 7f1b11844700 1 --2- 172.21.15.28:0/3292926707 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1af00682f0 0x7f1af006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:10.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.988+0000 7f1b11844700 1 --2- 172.21.15.28:0/3292926707 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1af00682f0 0x7f1af006a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1b080090b0 tx=0x7f1b08008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:10.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:10.992+0000 7f1b02ffd700 1 -- 172.21.15.28:0/3292926707 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1afc04b5e0 con 0x7f1b0c0caeb0 2023-11-28T11:10:11.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.186+0000 7f1b142a9700 1 -- 172.21.15.28:0/3292926707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1b0c0c79b0 con 0x7f1b0c0caeb0 2023-11-28T11:10:11.189 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.187+0000 7f1b02ffd700 1 -- 172.21.15.28:0/3292926707 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18035 (secure 0 0 0) 0x7f1b0c0c79b0 con 0x7f1b0c0caeb0 2023-11-28T11:10:11.189 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:11.189 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":39,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:10:10.820625+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:08.429278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:09.797038+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:11.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.189+0000 7f1b142a9700 1 -- 172.21.15.28:0/3292926707 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1af00682f0 msgr2=0x7f1af006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.189+0000 7f1b142a9700 1 --2- 172.21.15.28:0/3292926707 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1af00682f0 0x7f1af006a7a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1b080090b0 tx=0x7f1b08008030 comp rx=0 tx=0).stop 2023-11-28T11:10:11.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.189+0000 7f1b142a9700 1 -- 172.21.15.28:0/3292926707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b0c0caeb0 msgr2=0x7f1b0c157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.189+0000 7f1b142a9700 1 --2- 172.21.15.28:0/3292926707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b0c0caeb0 0x7f1b0c157500 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1afc00c520 tx=0x7f1afc00c8e0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.189+0000 7f1b142a9700 1 -- 172.21.15.28:0/3292926707 shutdown_connections 2023-11-28T11:10:11.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.189+0000 7f1b142a9700 1 --2- 172.21.15.28:0/3292926707 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1af00682f0 0x7f1af006a7a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.189+0000 7f1b142a9700 1 --2- 172.21.15.28:0/3292926707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1b0c0caeb0 0x7f1b0c157500 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.189+0000 7f1b142a9700 1 --2- 172.21.15.28:0/3292926707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1b0c161090 0x7f1b0c15f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.189+0000 7f1b142a9700 1 --2- 172.21.15.28:0/3292926707 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1b0c15c990 0x7f1b0c0ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.189+0000 7f1b142a9700 1 -- 172.21.15.28:0/3292926707 >> 172.21.15.28:0/3292926707 conn(0x7f1b0c0b1650 msgr2=0x7f1b0c06c8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:11.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.189+0000 7f1b142a9700 1 -- 172.21.15.28:0/3292926707 shutdown_connections 2023-11-28T11:10:11.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.189+0000 7f1b142a9700 1 -- 172.21.15.28:0/3292926707 wait complete. 2023-11-28T11:10:11.203 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:11.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.326+0000 7ff81a64e700 1 Processor -- start 2023-11-28T11:10:11.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.326+0000 7ff81a64e700 1 -- start start 2023-11-28T11:10:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.327+0000 7ff81a64e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff814161090 0x7ff8140ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.327+0000 7ff81a64e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8140caeb0 0x7ff8140cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.327+0000 7ff81a64e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8140ce340 con 0x7ff81415c990 2023-11-28T11:10:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.327+0000 7ff81a64e700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8140cebe0 con 0x7ff814161090 2023-11-28T11:10:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.327+0000 7ff81a64e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140cf780 con 0x7ff8140caeb0 2023-11-28T11:10:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.327+0000 7ff8137fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff814161090 0x7ff8140ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.327+0000 7ff8137fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff814161090 0x7ff8140ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49396/0 (socket says 172.21.15.28:49396) 2023-11-28T11:10:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.327+0000 7ff8137fe700 1 -- 172.21.15.28:0/787139361 learned_addr learned my addr 172.21.15.28:0/787139361 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:11.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.327+0000 7ff818beb700 1 --2- 172.21.15.28:0/787139361 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8140caeb0 0x7ff8140cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.327+0000 7ff8137fe700 1 -- 172.21.15.28:0/787139361 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8140caeb0 msgr2=0x7ff8140cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.327+0000 7ff8137fe700 1 --2- 172.21.15.28:0/787139361 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8140caeb0 0x7ff8140cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.327+0000 7ff8137fe700 1 -- 172.21.15.28:0/787139361 >> v1:172.21.15.59:6789/0 conn(0x7ff81415c990 legacy=0x7ff8140cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.327+0000 7ff8137fe700 1 -- 172.21.15.28:0/787139361 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8140cb3f0 con 0x7ff814161090 2023-11-28T11:10:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.327+0000 7ff8137fe700 1 --2- 172.21.15.28:0/787139361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff814161090 0x7ff8140ca970 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff8040098f0 tx=0x7ff804009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab59989c6138122 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.328+0000 7ff812ffd700 1 -- 172.21.15.28:0/787139361 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff804004230 con 0x7ff814161090 2023-11-28T11:10:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.328+0000 7ff812ffd700 1 -- 172.21.15.28:0/787139361 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff804004390 con 0x7ff814161090 2023-11-28T11:10:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.328+0000 7ff812ffd700 1 -- 172.21.15.28:0/787139361 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8040036a0 con 0x7ff814161090 2023-11-28T11:10:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.328+0000 7ff81a64e700 1 -- 172.21.15.28:0/787139361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff814161090 msgr2=0x7ff8140ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.328+0000 7ff81a64e700 1 --2- 172.21.15.28:0/787139361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff814161090 0x7ff8140ca970 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7ff8040098f0 tx=0x7ff804009c00 comp rx=0 tx=0).stop 2023-11-28T11:10:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.328+0000 7ff81a64e700 1 -- 172.21.15.28:0/787139361 shutdown_connections 2023-11-28T11:10:11.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.328+0000 7ff81a64e700 1 --2- 172.21.15.28:0/787139361 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8140caeb0 0x7ff8140cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.328+0000 7ff81a64e700 1 --2- 172.21.15.28:0/787139361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff814161090 0x7ff8140ca970 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.328+0000 7ff81a64e700 1 -- 172.21.15.28:0/787139361 >> 172.21.15.28:0/787139361 conn(0x7ff8140b1650 msgr2=0x7ff8140b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:11.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.328+0000 7ff81a64e700 1 -- 172.21.15.28:0/787139361 shutdown_connections 2023-11-28T11:10:11.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.328+0000 7ff81a64e700 1 -- 172.21.15.28:0/787139361 wait complete. 2023-11-28T11:10:11.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.329+0000 7ff81a64e700 1 Processor -- start 2023-11-28T11:10:11.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.330+0000 7ff81a64e700 1 -- start start 2023-11-28T11:10:11.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.330+0000 7ff81a64e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8140caeb0 0x7ff8141f5790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.330+0000 7ff81a64e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff81415c990 0x7ff8140b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.330+0000 7ff81a64e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff814161090 0x7ff8140b5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.330+0000 7ff81a64e700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140c8a80 con 0x7ff814161090 2023-11-28T11:10:11.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.330+0000 7ff81a64e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140c8d60 con 0x7ff81415c990 2023-11-28T11:10:11.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.330+0000 7ff81a64e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8140c9040 con 0x7ff8140caeb0 2023-11-28T11:10:11.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.330+0000 7ff818beb700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff814161090 0x7ff8140b5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.330+0000 7ff818beb700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff814161090 0x7ff8140b5370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49402/0 (socket says 172.21.15.28:49402) 2023-11-28T11:10:11.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.330+0000 7ff818beb700 1 -- 172.21.15.28:0/2607005660 learned_addr learned my addr 172.21.15.28:0/2607005660 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:11.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.331+0000 7ff8137fe700 1 --2- 172.21.15.28:0/2607005660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff81415c990 0x7ff8140b4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.331+0000 7ff813fff700 1 --2- 172.21.15.28:0/2607005660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8140caeb0 0x7ff8141f5790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.331+0000 7ff818beb700 1 -- 172.21.15.28:0/2607005660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8140caeb0 msgr2=0x7ff8141f5790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.331+0000 7ff818beb700 1 --2- 172.21.15.28:0/2607005660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8140caeb0 0x7ff8141f5790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.331+0000 7ff818beb700 1 -- 172.21.15.28:0/2607005660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff81415c990 msgr2=0x7ff8140b4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.331+0000 7ff818beb700 1 --2- 172.21.15.28:0/2607005660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff81415c990 0x7ff8140b4e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.331+0000 7ff818beb700 1 -- 172.21.15.28:0/2607005660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff804009550 con 0x7ff814161090 2023-11-28T11:10:11.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.331+0000 7ff818beb700 1 --2- 172.21.15.28:0/2607005660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff814161090 0x7ff8140b5370 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00cd70 tx=0x7ff80c00a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:11.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.331+0000 7ff810ff9700 1 -- 172.21.15.28:0/2607005660 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c003a20 con 0x7ff814161090 2023-11-28T11:10:11.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.331+0000 7ff810ff9700 1 -- 172.21.15.28:0/2607005660 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80c003b80 con 0x7ff814161090 2023-11-28T11:10:11.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.331+0000 7ff810ff9700 1 -- 172.21.15.28:0/2607005660 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80c004020 con 0x7ff814161090 2023-11-28T11:10:11.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.331+0000 7ff81a64e700 1 -- 172.21.15.28:0/2607005660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8140ce870 con 0x7ff814161090 2023-11-28T11:10:11.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.331+0000 7ff81a64e700 1 -- 172.21.15.28:0/2607005660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8140ce590 con 0x7ff814161090 2023-11-28T11:10:11.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.332+0000 7ff81a64e700 1 -- 172.21.15.28:0/2607005660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8140cc850 con 0x7ff814161090 2023-11-28T11:10:11.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.333+0000 7ff810ff9700 1 -- 172.21.15.28:0/2607005660 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff80c01c020 con 0x7ff814161090 2023-11-28T11:10:11.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.333+0000 7ff810ff9700 1 --2- 172.21.15.28:0/2607005660 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7f40682f0 0x7ff7f406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.333+0000 7ff810ff9700 1 -- 172.21.15.28:0/2607005660 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff80c012070 con 0x7ff814161090 2023-11-28T11:10:11.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.333+0000 7ff813fff700 1 --2- 172.21.15.28:0/2607005660 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7f40682f0 0x7ff7f406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.333+0000 7ff813fff700 1 --2- 172.21.15.28:0/2607005660 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7f40682f0 0x7ff7f406a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc003e60 tx=0x7ff7fc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:11.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.337+0000 7ff810ff9700 1 -- 172.21.15.28:0/2607005660 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff80c052d30 con 0x7ff814161090 2023-11-28T11:10:11.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.532+0000 7ff81a64e700 1 -- 172.21.15.28:0/2607005660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8140675a0 con 0x7ff814161090 2023-11-28T11:10:11.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.533+0000 7ff810ff9700 1 -- 172.21.15.28:0/2607005660 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+18035 (secure 0 0 0) 0x7ff80c056350 con 0x7ff814161090 2023-11-28T11:10:11.535 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:11.535 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":39,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:10:10.820625+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:08.429278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:09.797038+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:11.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.536+0000 7ff81a64e700 1 -- 172.21.15.28:0/2607005660 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7f40682f0 msgr2=0x7ff7f406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.536+0000 7ff81a64e700 1 --2- 172.21.15.28:0/2607005660 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7f40682f0 0x7ff7f406a7a0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc003e60 tx=0x7ff7fc009040 comp rx=0 tx=0).stop 2023-11-28T11:10:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.536+0000 7ff81a64e700 1 -- 172.21.15.28:0/2607005660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff814161090 msgr2=0x7ff8140b5370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.536+0000 7ff81a64e700 1 --2- 172.21.15.28:0/2607005660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff814161090 0x7ff8140b5370 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7ff80c00cd70 tx=0x7ff80c00a570 comp rx=0 tx=0).stop 2023-11-28T11:10:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.536+0000 7ff81a64e700 1 -- 172.21.15.28:0/2607005660 shutdown_connections 2023-11-28T11:10:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.536+0000 7ff81a64e700 1 --2- 172.21.15.28:0/2607005660 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7f40682f0 0x7ff7f406a7a0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.536+0000 7ff81a64e700 1 --2- 172.21.15.28:0/2607005660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff814161090 0x7ff8140b5370 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.536+0000 7ff81a64e700 1 --2- 172.21.15.28:0/2607005660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff8140caeb0 0x7ff8141f5790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.536+0000 7ff81a64e700 1 --2- 172.21.15.28:0/2607005660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff81415c990 0x7ff8140b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.536+0000 7ff81a64e700 1 -- 172.21.15.28:0/2607005660 >> 172.21.15.28:0/2607005660 conn(0x7ff8140b1650 msgr2=0x7ff8140af060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:11.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.536+0000 7ff81a64e700 1 -- 172.21.15.28:0/2607005660 shutdown_connections 2023-11-28T11:10:11.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.536+0000 7ff81a64e700 1 -- 172.21.15.28:0/2607005660 wait complete. 2023-11-28T11:10:11.550 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:10:11.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.676+0000 7f0b86936700 1 Processor -- start 2023-11-28T11:10:11.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.676+0000 7f0b86936700 1 -- start start 2023-11-28T11:10:11.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.676+0000 7f0b86936700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0b800b1630 0x7f0b800cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.676+0000 7f0b86936700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f0b8006afc0 0x7f0b8006b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.676+0000 7f0b86936700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b800ce550 con 0x7f0b800b1a00 2023-11-28T11:10:11.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.676+0000 7f0b86936700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b800ce830 con 0x7f0b800b1630 2023-11-28T11:10:11.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.676+0000 7f0b86936700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b800ceb10 con 0x7f0b8006afc0 2023-11-28T11:10:11.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.677+0000 7f0b7ffff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0b800b1630 0x7f0b800cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.677+0000 7f0b7f7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f0b800b1a00 0x7f0b800caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37464/0 (socket says 172.21.15.28:37464) 2023-11-28T11:10:11.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.677+0000 7f0b7f7fe700 1 -- 172.21.15.28:0/1370179583 learned_addr learned my addr 172.21.15.28:0/1370179583 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:11.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.677+0000 7f0b84ed3700 1 --2- 172.21.15.28:0/1370179583 >> v2:172.21.15.59:3300/0 conn(0x7f0b8006afc0 0x7f0b8006b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.677+0000 7f0b7ffff700 1 -- 172.21.15.28:0/1370179583 >> v2:172.21.15.59:3300/0 conn(0x7f0b8006afc0 msgr2=0x7f0b8006b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:11.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.677+0000 7f0b7ffff700 1 --2- 172.21.15.28:0/1370179583 >> v2:172.21.15.59:3300/0 conn(0x7f0b8006afc0 0x7f0b8006b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.677+0000 7f0b7ffff700 1 -- 172.21.15.28:0/1370179583 >> v1:172.21.15.28:6789/0 conn(0x7f0b800b1a00 legacy=0x7f0b800caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.677+0000 7f0b7effd700 1 -- 172.21.15.28:0/1370179583 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331663886 0 0) 0x7f0b800ce550 con 0x7f0b800b1a00 2023-11-28T11:10:11.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.677+0000 7f0b7ffff700 1 -- 172.21.15.28:0/1370179583 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b800cedf0 con 0x7f0b800b1630 2023-11-28T11:10:11.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.677+0000 7f0b7ffff700 1 --2- 172.21.15.28:0/1370179583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b800b1630 0x7f0b800cf5f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009840 tx=0x7f0b70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d116e2a8269ce386 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:11.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.677+0000 7f0b7effd700 1 -- 172.21.15.28:0/1370179583 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70011830 con 0x7f0b800b1630 2023-11-28T11:10:11.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.677+0000 7f0b7effd700 1 -- 172.21.15.28:0/1370179583 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b70011990 con 0x7f0b800b1630 2023-11-28T11:10:11.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.677+0000 7f0b7effd700 1 -- 172.21.15.28:0/1370179583 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b70011d20 con 0x7f0b800b1630 2023-11-28T11:10:11.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.678+0000 7f0b86936700 1 -- 172.21.15.28:0/1370179583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b800b1630 msgr2=0x7f0b800cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.678+0000 7f0b86936700 1 --2- 172.21.15.28:0/1370179583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b800b1630 0x7f0b800cf5f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f0b70009840 tx=0x7f0b70009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:11.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.678+0000 7f0b86936700 1 -- 172.21.15.28:0/1370179583 shutdown_connections 2023-11-28T11:10:11.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.678+0000 7f0b86936700 1 --2- 172.21.15.28:0/1370179583 >> v2:172.21.15.59:3300/0 conn(0x7f0b8006afc0 0x7f0b8006b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.678+0000 7f0b86936700 1 --2- 172.21.15.28:0/1370179583 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b800b1630 0x7f0b800cf5f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.678+0000 7f0b86936700 1 -- 172.21.15.28:0/1370179583 >> 172.21.15.28:0/1370179583 conn(0x7f0b800cc630 msgr2=0x7f0b800cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:11.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.678+0000 7f0b86936700 1 -- 172.21.15.28:0/1370179583 shutdown_connections 2023-11-28T11:10:11.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.678+0000 7f0b86936700 1 -- 172.21.15.28:0/1370179583 wait complete. 2023-11-28T11:10:11.684 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.679+0000 7f0b86936700 1 Processor -- start 2023-11-28T11:10:11.685 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.679+0000 7f0b86936700 1 -- start start 2023-11-28T11:10:11.685 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.680+0000 7f0b86936700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b8006afc0 0x7f0b800c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.685 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.680+0000 7f0b86936700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b800b1630 0x7f0b80149ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.685 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.680+0000 7f0b86936700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b800b1a00 0x7f0b800c7910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.685 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.680+0000 7f0b86936700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8014a400 con 0x7f0b800b1a00 2023-11-28T11:10:11.685 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.680+0000 7f0b86936700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b800c8390 con 0x7f0b8006afc0 2023-11-28T11:10:11.685 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.680+0000 7f0b86936700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b800c8670 con 0x7f0b800b1630 2023-11-28T11:10:11.685 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.680+0000 7f0b84ed3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b800b1a00 0x7f0b800c7910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.686 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.680+0000 7f0b84ed3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b800b1a00 0x7f0b800c7910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49416/0 (socket says 172.21.15.28:49416) 2023-11-28T11:10:11.686 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.680+0000 7f0b84ed3700 1 -- 172.21.15.28:0/715173075 learned_addr learned my addr 172.21.15.28:0/715173075 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.680+0000 7f0b7ffff700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b8006afc0 0x7f0b800c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.680+0000 7f0b7f7fe700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b800b1630 0x7f0b80149ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.680+0000 7f0b84ed3700 1 -- 172.21.15.28:0/715173075 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b800b1630 msgr2=0x7f0b80149ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.680+0000 7f0b84ed3700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b800b1630 0x7f0b80149ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.681+0000 7f0b84ed3700 1 -- 172.21.15.28:0/715173075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b8006afc0 msgr2=0x7f0b800c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.681+0000 7f0b84ed3700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b8006afc0 0x7f0b800c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.681+0000 7f0b84ed3700 1 -- 172.21.15.28:0/715173075 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b700094a0 con 0x7f0b800b1a00 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.681+0000 7f0b7f7fe700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b800b1630 0x7f0b80149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.681+0000 7f0b7ffff700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b8006afc0 0x7f0b800c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.681+0000 7f0b84ed3700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b800b1a00 0x7f0b800c7910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800efc0 tx=0x7f0b7800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:11.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.681+0000 7f0b7cff9700 1 -- 172.21.15.28:0/715173075 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7800cd40 con 0x7f0b800b1a00 2023-11-28T11:10:11.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.681+0000 7f0b7cff9700 1 -- 172.21.15.28:0/715173075 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7800f040 con 0x7f0b800b1a00 2023-11-28T11:10:11.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.681+0000 7f0b86936700 1 -- 172.21.15.28:0/715173075 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b800b1ea0 con 0x7f0b800b1a00 2023-11-28T11:10:11.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.681+0000 7f0b7cff9700 1 -- 172.21.15.28:0/715173075 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b780146c0 con 0x7f0b800b1a00 2023-11-28T11:10:11.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.681+0000 7f0b86936700 1 -- 172.21.15.28:0/715173075 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b800cf0d0 con 0x7f0b800b1a00 2023-11-28T11:10:11.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.682+0000 7f0b86936700 1 -- 172.21.15.28:0/715173075 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b800cb5b0 con 0x7f0b800b1a00 2023-11-28T11:10:11.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.682+0000 7f0b7cff9700 1 -- 172.21.15.28:0/715173075 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0b78014820 con 0x7f0b800b1a00 2023-11-28T11:10:11.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.683+0000 7f0b7cff9700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b600682f0 0x7f0b6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:11.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.683+0000 7f0b7ffff700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b600682f0 0x7f0b6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:11.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.683+0000 7f0b7cff9700 1 -- 172.21.15.28:0/715173075 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0b78086930 con 0x7f0b800b1a00 2023-11-28T11:10:11.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.683+0000 7f0b7ffff700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b600682f0 0x7f0b6006a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0b70000c00 tx=0x7f0b70010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:11.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.688+0000 7f0b7cff9700 1 -- 172.21.15.28:0/715173075 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0b78052f60 con 0x7f0b800b1a00 2023-11-28T11:10:11.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:11.930+0000 7f0b86936700 1 -- 172.21.15.28:0/715173075 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f0b800739f0 con 0x7f0b800b1a00 2023-11-28T11:10:12.835 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.835+0000 7f0b7cff9700 1 -- 172.21.15.28:0/715173075 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7f0b7800cea0 con 0x7f0b800b1a00 2023-11-28T11:10:12.839 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.837+0000 7f0b86936700 1 -- 172.21.15.28:0/715173075 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b600682f0 msgr2=0x7f0b6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.839 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.837+0000 7f0b86936700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b600682f0 0x7f0b6006a7a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0b70000c00 tx=0x7f0b70010000 comp rx=0 tx=0).stop 2023-11-28T11:10:12.839 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.837+0000 7f0b86936700 1 -- 172.21.15.28:0/715173075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b800b1a00 msgr2=0x7f0b800c7910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.839 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.837+0000 7f0b86936700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b800b1a00 0x7f0b800c7910 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f0b7800efc0 tx=0x7f0b7800c3a0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.839 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.838+0000 7f0b86936700 1 -- 172.21.15.28:0/715173075 shutdown_connections 2023-11-28T11:10:12.839 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.838+0000 7f0b86936700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0b600682f0 0x7f0b6006a7a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.839 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.838+0000 7f0b86936700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0b800b1a00 0x7f0b800c7910 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.839 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.838+0000 7f0b86936700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0b8006afc0 0x7f0b800c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.839 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.838+0000 7f0b86936700 1 --2- 172.21.15.28:0/715173075 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0b800b1630 0x7f0b80149ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.838+0000 7f0b86936700 1 -- 172.21.15.28:0/715173075 >> 172.21.15.28:0/715173075 conn(0x7f0b800cc630 msgr2=0x7f0b8006c160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:12.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.838+0000 7f0b86936700 1 -- 172.21.15.28:0/715173075 shutdown_connections 2023-11-28T11:10:12.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.838+0000 7f0b86936700 1 -- 172.21.15.28:0/715173075 wait complete. 2023-11-28T11:10:12.853 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:10:12.854 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:10:12.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.988+0000 7f87d68d8700 1 Processor -- start 2023-11-28T11:10:12.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.988+0000 7f87d68d8700 1 -- start start 2023-11-28T11:10:12.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.989+0000 7f87d68d8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87d00b2af0 0x7f87d00cc300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.989+0000 7f87d68d8700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d006f940 con 0x7f87d015fc00 2023-11-28T11:10:12.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.989+0000 7f87d68d8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87d006fc20 con 0x7f87d00b2680 2023-11-28T11:10:12.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.989+0000 7f87d68d8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00cb3f0 con 0x7f87d00b2af0 2023-11-28T11:10:12.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.989+0000 7f87cffff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f87d015fc00 0x7f87d006eea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37478/0 (socket says 172.21.15.28:37478) 2023-11-28T11:10:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.989+0000 7f87cffff700 1 -- 172.21.15.28:0/618628541 learned_addr learned my addr 172.21.15.28:0/618628541 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.989+0000 7f87d4e75700 1 --2- 172.21.15.28:0/618628541 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87d00b2af0 0x7f87d00cc300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.989+0000 7f87ceffd700 1 -- 172.21.15.28:0/618628541 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1021888177 0 0) 0x7f87d006f940 con 0x7f87d015fc00 2023-11-28T11:10:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.989+0000 7f87ceffd700 1 -- 172.21.15.28:0/618628541 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87bc0035b0 con 0x7f87d015fc00 2023-11-28T11:10:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.989+0000 7f87ceffd700 1 -- 172.21.15.28:0/618628541 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3263717558 0 0) 0x7f87bc0035b0 con 0x7f87d015fc00 2023-11-28T11:10:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87ceffd700 1 -- 172.21.15.28:0/618628541 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87d00b2af0 msgr2=0x7f87d00cc300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87ceffd700 1 --2- 172.21.15.28:0/618628541 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87d00b2af0 0x7f87d00cc300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87ceffd700 1 -- 172.21.15.28:0/618628541 >> v1:172.21.15.59:6789/0 conn(0x7f87d00b2680 legacy=0x7f87d00cbbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87ceffd700 1 -- 172.21.15.28:0/618628541 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d00cb6d0 con 0x7f87d015fc00 2023-11-28T11:10:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87ceffd700 1 -- 172.21.15.28:0/618628541 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f87b8001670 con 0x7f87d015fc00 2023-11-28T11:10:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87ceffd700 1 -- 172.21.15.28:0/618628541 >> v1:172.21.15.28:6789/0 conn(0x7f87d015fc00 legacy=0x7f87d006eea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87ceffd700 1 --2- 172.21.15.28:0/618628541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87bc003a30 0x7f87bc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87ceffd700 1 --2- 172.21.15.28:0/618628541 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87d00b2af0 0x7f87bc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87ceffd700 1 --2- 172.21.15.28:0/618628541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87bc004d90 0x7f87bc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87ceffd700 1 -- 172.21.15.28:0/618628541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00cb3f0 con 0x7f87bc004d90 2023-11-28T11:10:12.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87ceffd700 1 -- 172.21.15.28:0/618628541 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc00ba10 con 0x7f87bc003a30 2023-11-28T11:10:12.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87ceffd700 1 -- 172.21.15.28:0/618628541 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87bc00bba0 con 0x7f87d00b2af0 2023-11-28T11:10:12.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87d4e75700 1 --2- 172.21.15.28:0/618628541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87bc004d90 0x7f87bc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87cffff700 1 --2- 172.21.15.28:0/618628541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87bc003a30 0x7f87bc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87cf7fe700 1 --2- 172.21.15.28:0/618628541 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87d00b2af0 0x7f87bc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87d4e75700 1 -- 172.21.15.28:0/618628541 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87d00b2af0 msgr2=0x7f87bc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:12.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87d4e75700 1 --2- 172.21.15.28:0/618628541 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87d00b2af0 0x7f87bc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87d4e75700 1 -- 172.21.15.28:0/618628541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87bc003a30 msgr2=0x7f87bc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:12.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87d4e75700 1 --2- 172.21.15.28:0/618628541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87bc003a30 0x7f87bc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:12.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87d4e75700 1 -- 172.21.15.28:0/618628541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87bc0035b0 con 0x7f87bc004d90 2023-11-28T11:10:12.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87d4e75700 1 --2- 172.21.15.28:0/618628541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87bc004d90 0x7f87bc005200 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009fc0 tx=0x7f87bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd0daf4926e1fe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:12.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87ceffd700 1 -- 172.21.15.28:0/618628541 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c800ec30 con 0x7f87bc004d90 2023-11-28T11:10:12.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87ceffd700 1 -- 172.21.15.28:0/618628541 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c800f040 con 0x7f87bc004d90 2023-11-28T11:10:12.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.990+0000 7f87ceffd700 1 -- 172.21.15.28:0/618628541 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8014480 con 0x7f87bc004d90 2023-11-28T11:10:12.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.991+0000 7f87d68d8700 1 -- 172.21.15.28:0/618628541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87bc004d90 msgr2=0x7f87bc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.991+0000 7f87d68d8700 1 --2- 172.21.15.28:0/618628541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87bc004d90 0x7f87bc005200 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f87c8009fc0 tx=0x7f87bc005880 comp rx=0 tx=0).stop 2023-11-28T11:10:12.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.991+0000 7f87d4e75700 1 -- 172.21.15.28:0/618628541 reap_dead start 2023-11-28T11:10:12.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.991+0000 7f87d68d8700 1 -- 172.21.15.28:0/618628541 shutdown_connections 2023-11-28T11:10:12.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.991+0000 7f87d68d8700 1 -- 172.21.15.28:0/618628541 >> 172.21.15.28:0/618628541 conn(0x7f87d00cd680 msgr2=0x7f87d00cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:12.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.991+0000 7f87d68d8700 1 -- 172.21.15.28:0/618628541 shutdown_connections 2023-11-28T11:10:12.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.991+0000 7f87d68d8700 1 -- 172.21.15.28:0/618628541 wait complete. 2023-11-28T11:10:12.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.992+0000 7f87d68d8700 1 Processor -- start 2023-11-28T11:10:12.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.992+0000 7f87d68d8700 1 -- start start 2023-11-28T11:10:12.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.993+0000 7f87d68d8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87d0150000 0x7f87d00c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.993+0000 7f87d68d8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87d01503d0 0x7f87d00ca700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.993+0000 7f87d68d8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87d00c5790 0x7f87d006c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:12.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.993+0000 7f87d68d8700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00c9c40 con 0x7f87d00c5790 2023-11-28T11:10:12.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.993+0000 7f87d68d8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00cac40 con 0x7f87d0150000 2023-11-28T11:10:12.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.993+0000 7f87d68d8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87d00c46f0 con 0x7f87d01503d0 2023-11-28T11:10:12.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.993+0000 7f87d4e75700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87d00c5790 0x7f87d006c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.993+0000 7f87d4e75700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87d00c5790 0x7f87d006c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49436/0 (socket says 172.21.15.28:49436) 2023-11-28T11:10:12.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.993+0000 7f87d4e75700 1 -- 172.21.15.28:0/2712962727 learned_addr learned my addr 172.21.15.28:0/2712962727 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:12.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.993+0000 7f87cf7fe700 1 --2- 172.21.15.28:0/2712962727 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87d01503d0 0x7f87d00ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.993+0000 7f87cffff700 1 --2- 172.21.15.28:0/2712962727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87d0150000 0x7f87d00c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:12.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.994+0000 7f87d4e75700 1 -- 172.21.15.28:0/2712962727 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87d01503d0 msgr2=0x7f87d00ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:12.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.994+0000 7f87d4e75700 1 --2- 172.21.15.28:0/2712962727 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87d01503d0 0x7f87d00ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.994+0000 7f87d4e75700 1 -- 172.21.15.28:0/2712962727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87d0150000 msgr2=0x7f87d00c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.994+0000 7f87d4e75700 1 --2- 172.21.15.28:0/2712962727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87d0150000 0x7f87d00c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.994+0000 7f87d4e75700 1 -- 172.21.15.28:0/2712962727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d015ffd0 con 0x7f87d00c5790 2023-11-28T11:10:13.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.994+0000 7f87d4e75700 1 --2- 172.21.15.28:0/2712962727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87d00c5790 0x7f87d006c260 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f87c8016000 tx=0x7f87c8009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:13.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.994+0000 7f87ccff9700 1 -- 172.21.15.28:0/2712962727 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8007000 con 0x7f87d00c5790 2023-11-28T11:10:13.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.994+0000 7f87ccff9700 1 -- 172.21.15.28:0/2712962727 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87c8022070 con 0x7f87d00c5790 2023-11-28T11:10:13.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.994+0000 7f87d68d8700 1 -- 172.21.15.28:0/2712962727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d00b2f50 con 0x7f87d00c5790 2023-11-28T11:10:13.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.994+0000 7f87ccff9700 1 -- 172.21.15.28:0/2712962727 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87c8015e70 con 0x7f87d00c5790 2023-11-28T11:10:13.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.994+0000 7f87d68d8700 1 -- 172.21.15.28:0/2712962727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87d006f5b0 con 0x7f87d00c5790 2023-11-28T11:10:13.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.995+0000 7f87d68d8700 1 -- 172.21.15.28:0/2712962727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87d00739f0 con 0x7f87d00c5790 2023-11-28T11:10:13.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.996+0000 7f87ccff9700 1 -- 172.21.15.28:0/2712962727 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f87c801e430 con 0x7f87d00c5790 2023-11-28T11:10:13.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.996+0000 7f87ccff9700 1 --2- 172.21.15.28:0/2712962727 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f87b00682f0 0x7f87b006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.996+0000 7f87ccff9700 1 -- 172.21.15.28:0/2712962727 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f87c8024070 con 0x7f87d00c5790 2023-11-28T11:10:13.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.996+0000 7f87cffff700 1 --2- 172.21.15.28:0/2712962727 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f87b00682f0 0x7f87b006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:12.997+0000 7f87cffff700 1 --2- 172.21.15.28:0/2712962727 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f87b00682f0 0x7f87b006a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f87b8000c00 tx=0x7f87b8001bf0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:13.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.000+0000 7f87ccff9700 1 -- 172.21.15.28:0/2712962727 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f87c8054930 con 0x7f87d00c5790 2023-11-28T11:10:13.241 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.241+0000 7f87d68d8700 1 -- 172.21.15.28:0/2712962727 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f87d00b6b80 con 0x7f87d00c5790 2023-11-28T11:10:13.394 INFO:tasks.ceph.mds.d.smithi059.stderr:2023-11-28T11:10:13.392+0000 7f08d132c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:10:13.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.836+0000 7f87ccff9700 1 -- 172.21.15.28:0/2712962727 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f87c8057f50 con 0x7f87d00c5790 2023-11-28T11:10:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.839+0000 7f87d68d8700 1 -- 172.21.15.28:0/2712962727 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f87b00682f0 msgr2=0x7f87b006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.839+0000 7f87d68d8700 1 --2- 172.21.15.28:0/2712962727 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f87b00682f0 0x7f87b006a7a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f87b8000c00 tx=0x7f87b8001bf0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.839+0000 7f87d68d8700 1 -- 172.21.15.28:0/2712962727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87d00c5790 msgr2=0x7f87d006c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.839+0000 7f87d68d8700 1 --2- 172.21.15.28:0/2712962727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87d00c5790 0x7f87d006c260 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f87c8016000 tx=0x7f87c8009750 comp rx=0 tx=0).stop 2023-11-28T11:10:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.839+0000 7f87d68d8700 1 -- 172.21.15.28:0/2712962727 shutdown_connections 2023-11-28T11:10:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.839+0000 7f87d68d8700 1 --2- 172.21.15.28:0/2712962727 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f87b00682f0 0x7f87b006a7a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.839+0000 7f87d68d8700 1 --2- 172.21.15.28:0/2712962727 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87d00c5790 0x7f87d006c260 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.839+0000 7f87d68d8700 1 --2- 172.21.15.28:0/2712962727 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87d0150000 0x7f87d00c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.840 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.839+0000 7f87d68d8700 1 --2- 172.21.15.28:0/2712962727 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87d01503d0 0x7f87d00ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.839+0000 7f87d68d8700 1 -- 172.21.15.28:0/2712962727 >> 172.21.15.28:0/2712962727 conn(0x7f87d00cd680 msgr2=0x7f87d006d510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:13.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.839+0000 7f87d68d8700 1 -- 172.21.15.28:0/2712962727 shutdown_connections 2023-11-28T11:10:13.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.839+0000 7f87d68d8700 1 -- 172.21.15.28:0/2712962727 wait complete. 2023-11-28T11:10:13.855 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:10:13.855 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:13.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.986+0000 7fe4cba35700 1 Processor -- start 2023-11-28T11:10:13.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.987+0000 7fe4cba35700 1 -- start start 2023-11-28T11:10:13.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.987+0000 7fe4cba35700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe4c406e400 0x7fe4c415db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.987+0000 7fe4cba35700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4c40ce890 0x7fe4c40ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.987+0000 7fe4cba35700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4c415e0b0 con 0x7fe4c4159590 2023-11-28T11:10:13.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.987+0000 7fe4cba35700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4c40cf250 con 0x7fe4c406e400 2023-11-28T11:10:13.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.987+0000 7fe4cba35700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c415d120 con 0x7fe4c40ce890 2023-11-28T11:10:13.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.987+0000 7fe4c8fd0700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe4c406e400 0x7fe4c415db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.987+0000 7fe4c8fd0700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe4c406e400 0x7fe4c415db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49444/0 (socket says 172.21.15.28:49444) 2023-11-28T11:10:13.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.987+0000 7fe4c97d1700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe4c4159590 0x7fe4c40ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37498/0 (socket says 172.21.15.28:37498) 2023-11-28T11:10:13.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.987+0000 7fe4c8fd0700 1 -- 172.21.15.28:0/1406056216 learned_addr learned my addr 172.21.15.28:0/1406056216 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:13.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.988+0000 7fe4c9fd2700 1 --2- 172.21.15.28:0/1406056216 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4c40ce890 0x7fe4c40ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.988+0000 7fe4c8fd0700 1 -- 172.21.15.28:0/1406056216 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4c40ce890 msgr2=0x7fe4c40ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:13.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.988+0000 7fe4c8fd0700 1 --2- 172.21.15.28:0/1406056216 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4c40ce890 0x7fe4c40ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.988+0000 7fe4c8fd0700 1 -- 172.21.15.28:0/1406056216 >> v1:172.21.15.28:6789/0 conn(0x7fe4c4159590 legacy=0x7fe4c40ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.988+0000 7fe4c8fd0700 1 -- 172.21.15.28:0/1406056216 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4c415d6e0 con 0x7fe4c406e400 2023-11-28T11:10:13.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.988+0000 7fe4c8fd0700 1 --2- 172.21.15.28:0/1406056216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c406e400 0x7fe4c415db70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40098f0 tx=0x7fe4b4009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed453d2104825d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:13.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.988+0000 7fe4bbfff700 1 -- 172.21.15.28:0/1406056216 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400b8a0 con 0x7fe4c406e400 2023-11-28T11:10:13.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.988+0000 7fe4bbfff700 1 -- 172.21.15.28:0/1406056216 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4b400ba00 con 0x7fe4c406e400 2023-11-28T11:10:13.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.988+0000 7fe4bbfff700 1 -- 172.21.15.28:0/1406056216 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4b400bd90 con 0x7fe4c406e400 2023-11-28T11:10:13.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.989+0000 7fe4cba35700 1 -- 172.21.15.28:0/1406056216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c406e400 msgr2=0x7fe4c415db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.989+0000 7fe4cba35700 1 --2- 172.21.15.28:0/1406056216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c406e400 0x7fe4c415db70 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fe4b40098f0 tx=0x7fe4b4009c00 comp rx=0 tx=0).stop 2023-11-28T11:10:13.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.989+0000 7fe4cba35700 1 -- 172.21.15.28:0/1406056216 shutdown_connections 2023-11-28T11:10:13.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.989+0000 7fe4cba35700 1 --2- 172.21.15.28:0/1406056216 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4c40ce890 0x7fe4c40ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.989+0000 7fe4cba35700 1 --2- 172.21.15.28:0/1406056216 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c406e400 0x7fe4c415db70 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.989+0000 7fe4cba35700 1 -- 172.21.15.28:0/1406056216 >> 172.21.15.28:0/1406056216 conn(0x7fe4c40b1650 msgr2=0x7fe4c40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:13.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.989+0000 7fe4cba35700 1 -- 172.21.15.28:0/1406056216 shutdown_connections 2023-11-28T11:10:13.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.989+0000 7fe4cba35700 1 -- 172.21.15.28:0/1406056216 wait complete. 2023-11-28T11:10:13.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.990+0000 7fe4cba35700 1 Processor -- start 2023-11-28T11:10:13.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.990+0000 7fe4cba35700 1 -- start start 2023-11-28T11:10:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4cba35700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c406e400 0x7fe4c41efcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4cba35700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4c40ce890 0x7fe4c40c4960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4cba35700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c4159590 0x7fe4c415bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4cba35700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c40c5320 con 0x7fe4c406e400 2023-11-28T11:10:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4cba35700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c40c5040 con 0x7fe4c4159590 2023-11-28T11:10:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4cba35700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4c40b2af0 con 0x7fe4c40ce890 2023-11-28T11:10:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4c97d1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c406e400 0x7fe4c41efcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4c97d1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c406e400 0x7fe4c41efcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49446/0 (socket says 172.21.15.28:49446) 2023-11-28T11:10:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4c97d1700 1 -- 172.21.15.28:0/1296647598 learned_addr learned my addr 172.21.15.28:0/1296647598 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4c8fd0700 1 --2- 172.21.15.28:0/1296647598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4c40ce890 0x7fe4c40c4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4c9fd2700 1 --2- 172.21.15.28:0/1296647598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c4159590 0x7fe4c415bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4c97d1700 1 -- 172.21.15.28:0/1296647598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4c40ce890 msgr2=0x7fe4c40c4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4c97d1700 1 --2- 172.21.15.28:0/1296647598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4c40ce890 0x7fe4c40c4960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4c97d1700 1 -- 172.21.15.28:0/1296647598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c4159590 msgr2=0x7fe4c415bb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4c97d1700 1 --2- 172.21.15.28:0/1296647598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c4159590 0x7fe4c415bb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4c97d1700 1 -- 172.21.15.28:0/1296647598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4b4009550 con 0x7fe4c406e400 2023-11-28T11:10:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4c9fd2700 1 --2- 172.21.15.28:0/1296647598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c4159590 0x7fe4c415bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4c97d1700 1 --2- 172.21.15.28:0/1296647598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c406e400 0x7fe4c41efcb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000c500 tx=0x7fe4c000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4b9ffb700 1 -- 172.21.15.28:0/1296647598 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c0005cb0 con 0x7fe4c406e400 2023-11-28T11:10:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4b9ffb700 1 -- 172.21.15.28:0/1296647598 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4c0005e10 con 0x7fe4c406e400 2023-11-28T11:10:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4cba35700 1 -- 172.21.15.28:0/1296647598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4c40af240 con 0x7fe4c406e400 2023-11-28T11:10:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.991+0000 7fe4b9ffb700 1 -- 172.21.15.28:0/1296647598 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c00072d0 con 0x7fe4c406e400 2023-11-28T11:10:13.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.992+0000 7fe4cba35700 1 -- 172.21.15.28:0/1296647598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4c40aef60 con 0x7fe4c406e400 2023-11-28T11:10:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.993+0000 7fe4cba35700 1 -- 172.21.15.28:0/1296647598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4c415d370 con 0x7fe4c406e400 2023-11-28T11:10:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.993+0000 7fe4b9ffb700 1 -- 172.21.15.28:0/1296647598 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe4c0019030 con 0x7fe4c406e400 2023-11-28T11:10:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.993+0000 7fe4b9ffb700 1 --2- 172.21.15.28:0/1296647598 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe4a80682f0 0x7fe4a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.993+0000 7fe4b9ffb700 1 -- 172.21.15.28:0/1296647598 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe4c0084e00 con 0x7fe4c406e400 2023-11-28T11:10:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.993+0000 7fe4c8fd0700 1 --2- 172.21.15.28:0/1296647598 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe4a80682f0 0x7fe4a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:13.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.994+0000 7fe4c8fd0700 1 --2- 172.21.15.28:0/1296647598 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe4a80682f0 0x7fe4a806a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009c40 tx=0x7fe4b4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:13.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:13.998+0000 7fe4b9ffb700 1 -- 172.21.15.28:0/1296647598 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe4c004b5e0 con 0x7fe4c406e400 2023-11-28T11:10:14.239 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.239+0000 7fe4cba35700 1 -- 172.21.15.28:0/1296647598 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe4c40b2950 con 0x7fe4c406e400 2023-11-28T11:10:14.240 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.239+0000 7fe4b9ffb700 1 -- 172.21.15.28:0/1296647598 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7fe4c40b2950 con 0x7fe4c406e400 2023-11-28T11:10:14.240 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:14.240 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2867100265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2867100265},{"type":"v1","addr":"172.21.15.59:6833","nonce":2867100265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3808190953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3808190953},{"type":"v1","addr":"172.21.15.28:6837","nonce":3808190953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3920280453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3920280453},{"type":"v1","addr":"172.21.15.28:6836","nonce":3920280453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:10.820923+0000","modified":"2023-11-28T11:10:13.833216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6835/707419743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":707419743},{"type":"v1","addr":"172.21.15.59:6835","nonce":707419743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:10:14.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.241+0000 7fe4cba35700 1 -- 172.21.15.28:0/1296647598 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe4a80682f0 msgr2=0x7fe4a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.241+0000 7fe4cba35700 1 --2- 172.21.15.28:0/1296647598 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe4a80682f0 0x7fe4a806a7a0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fe4b4009c40 tx=0x7fe4b4008030 comp rx=0 tx=0).stop 2023-11-28T11:10:14.242 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.241+0000 7fe4cba35700 1 -- 172.21.15.28:0/1296647598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c406e400 msgr2=0x7fe4c41efcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.241+0000 7fe4cba35700 1 --2- 172.21.15.28:0/1296647598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c406e400 0x7fe4c41efcb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fe4c000c500 tx=0x7fe4c000c8c0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.241+0000 7fe4cba35700 1 -- 172.21.15.28:0/1296647598 shutdown_connections 2023-11-28T11:10:14.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.241+0000 7fe4cba35700 1 --2- 172.21.15.28:0/1296647598 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe4a80682f0 0x7fe4a806a7a0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.241+0000 7fe4cba35700 1 --2- 172.21.15.28:0/1296647598 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe4c406e400 0x7fe4c41efcb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.241+0000 7fe4cba35700 1 --2- 172.21.15.28:0/1296647598 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe4c4159590 0x7fe4c415bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.241+0000 7fe4cba35700 1 --2- 172.21.15.28:0/1296647598 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe4c40ce890 0x7fe4c40c4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.241+0000 7fe4cba35700 1 -- 172.21.15.28:0/1296647598 >> 172.21.15.28:0/1296647598 conn(0x7fe4c40b1650 msgr2=0x7fe4c40cc510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:14.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.241+0000 7fe4cba35700 1 -- 172.21.15.28:0/1296647598 shutdown_connections 2023-11-28T11:10:14.243 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.241+0000 7fe4cba35700 1 -- 172.21.15.28:0/1296647598 wait complete. 2023-11-28T11:10:14.243 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2023-11-28T11:10:14.254 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:14.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.379+0000 7fdb0e0c9700 1 Processor -- start 2023-11-28T11:10:14.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.379+0000 7fdb0e0c9700 1 -- start start 2023-11-28T11:10:14.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.379+0000 7fdb0e0c9700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdb0815c990 0x7fdb080cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:14.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.379+0000 7fdb0e0c9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb08161090 0x7fdb080ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:14.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.379+0000 7fdb0e0c9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb080ce340 con 0x7fdb080caeb0 2023-11-28T11:10:14.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.379+0000 7fdb0e0c9700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb080cebe0 con 0x7fdb0815c990 2023-11-28T11:10:14.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.379+0000 7fdb0e0c9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb080cb3f0 con 0x7fdb08161090 2023-11-28T11:10:14.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.380+0000 7fdb06ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb08161090 0x7fdb080ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:14.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.380+0000 7fdb06ffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb08161090 0x7fdb080ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:55212/0 (socket says 172.21.15.28:55212) 2023-11-28T11:10:14.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.380+0000 7fdb06ffd700 1 -- 172.21.15.28:0/79516750 learned_addr learned my addr 172.21.15.28:0/79516750 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:14.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.380+0000 7fdb077fe700 1 --2- 172.21.15.28:0/79516750 >> v2:172.21.15.59:3300/0 conn(0x7fdb0815c990 0x7fdb080cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:14.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.380+0000 7fdb06ffd700 1 -- 172.21.15.28:0/79516750 >> v2:172.21.15.59:3300/0 conn(0x7fdb0815c990 msgr2=0x7fdb080cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:14.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.380+0000 7fdb06ffd700 1 --2- 172.21.15.28:0/79516750 >> v2:172.21.15.59:3300/0 conn(0x7fdb0815c990 0x7fdb080cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.380+0000 7fdb06ffd700 1 -- 172.21.15.28:0/79516750 >> v1:172.21.15.59:6789/0 conn(0x7fdb080caeb0 legacy=0x7fdb080cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.380+0000 7fdb06ffd700 1 -- 172.21.15.28:0/79516750 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb080ce590 con 0x7fdb08161090 2023-11-28T11:10:14.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.380+0000 7fdb067fc700 1 -- 172.21.15.28:0/79516750 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2090370690 0 0) 0x7fdb080ce340 con 0x7fdb080caeb0 2023-11-28T11:10:14.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.380+0000 7fdb06ffd700 1 --2- 172.21.15.28:0/79516750 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb08161090 0x7fdb080ca970 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0098f0 tx=0x7fdafc009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdb847a023eea5b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.381+0000 7fdb067fc700 1 -- 172.21.15.28:0/79516750 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc011830 con 0x7fdb08161090 2023-11-28T11:10:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.381+0000 7fdb067fc700 1 -- 172.21.15.28:0/79516750 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc011990 con 0x7fdb08161090 2023-11-28T11:10:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.381+0000 7fdb067fc700 1 -- 172.21.15.28:0/79516750 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc011d20 con 0x7fdb08161090 2023-11-28T11:10:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.381+0000 7fdb0e0c9700 1 -- 172.21.15.28:0/79516750 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb08161090 msgr2=0x7fdb080ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.385 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.381+0000 7fdb0e0c9700 1 --2- 172.21.15.28:0/79516750 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb08161090 0x7fdb080ca970 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fdafc0098f0 tx=0x7fdafc009c00 comp rx=0 tx=0).stop 2023-11-28T11:10:14.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.381+0000 7fdb0e0c9700 1 -- 172.21.15.28:0/79516750 shutdown_connections 2023-11-28T11:10:14.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.381+0000 7fdb0e0c9700 1 --2- 172.21.15.28:0/79516750 >> v2:172.21.15.59:3300/0 conn(0x7fdb0815c990 0x7fdb080cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.381+0000 7fdb0e0c9700 1 --2- 172.21.15.28:0/79516750 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb08161090 0x7fdb080ca970 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.381+0000 7fdb0e0c9700 1 -- 172.21.15.28:0/79516750 >> 172.21.15.28:0/79516750 conn(0x7fdb080b1650 msgr2=0x7fdb080b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:14.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.381+0000 7fdb0e0c9700 1 -- 172.21.15.28:0/79516750 shutdown_connections 2023-11-28T11:10:14.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.381+0000 7fdb0e0c9700 1 -- 172.21.15.28:0/79516750 wait complete. 2023-11-28T11:10:14.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.382+0000 7fdb0e0c9700 1 Processor -- start 2023-11-28T11:10:14.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.383+0000 7fdb0e0c9700 1 -- start start 2023-11-28T11:10:14.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.383+0000 7fdb0e0c9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb080caeb0 0x7fdb081f1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:14.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.383+0000 7fdb0e0c9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb0815c990 0x7fdb080b5440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:14.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.383+0000 7fdb0e0c9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb08161090 0x7fdb08147a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:14.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.383+0000 7fdb0e0c9700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb08147760 con 0x7fdb0815c990 2023-11-28T11:10:14.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.383+0000 7fdb0e0c9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb080b4fb0 con 0x7fdb080caeb0 2023-11-28T11:10:14.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.383+0000 7fdb0e0c9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb080b43a0 con 0x7fdb08161090 2023-11-28T11:10:14.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.383+0000 7fdb06ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb0815c990 0x7fdb080b5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:14.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.383+0000 7fdb06ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb0815c990 0x7fdb080b5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49460/0 (socket says 172.21.15.28:49460) 2023-11-28T11:10:14.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.383+0000 7fdb06ffd700 1 -- 172.21.15.28:0/1558268466 learned_addr learned my addr 172.21.15.28:0/1558268466 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:14.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.384+0000 7fdb077fe700 1 --2- 172.21.15.28:0/1558268466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb080caeb0 0x7fdb081f1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:14.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.384+0000 7fdb07fff700 1 --2- 172.21.15.28:0/1558268466 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb08161090 0x7fdb08147a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:14.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.384+0000 7fdb06ffd700 1 -- 172.21.15.28:0/1558268466 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb08161090 msgr2=0x7fdb08147a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.384+0000 7fdb06ffd700 1 --2- 172.21.15.28:0/1558268466 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb08161090 0x7fdb08147a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.384+0000 7fdb06ffd700 1 -- 172.21.15.28:0/1558268466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb080caeb0 msgr2=0x7fdb081f1ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.384+0000 7fdb06ffd700 1 --2- 172.21.15.28:0/1558268466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb080caeb0 0x7fdb081f1ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.384+0000 7fdb06ffd700 1 -- 172.21.15.28:0/1558268466 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdafc009550 con 0x7fdb0815c990 2023-11-28T11:10:14.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.384+0000 7fdb06ffd700 1 --2- 172.21.15.28:0/1558268466 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb0815c990 0x7fdb080b5440 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009c40 tx=0x7fdafc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:14.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.384+0000 7fdaeffff700 1 -- 172.21.15.28:0/1558268466 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc0036a0 con 0x7fdb0815c990 2023-11-28T11:10:14.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.384+0000 7fdaeffff700 1 -- 172.21.15.28:0/1558268466 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdafc01f070 con 0x7fdb0815c990 2023-11-28T11:10:14.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.384+0000 7fdb0e0c9700 1 -- 172.21.15.28:0/1558268466 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb080cf5b0 con 0x7fdb0815c990 2023-11-28T11:10:14.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.384+0000 7fdaeffff700 1 -- 172.21.15.28:0/1558268466 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdafc003a60 con 0x7fdb0815c990 2023-11-28T11:10:14.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.384+0000 7fdb0e0c9700 1 -- 172.21.15.28:0/1558268466 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb080ce870 con 0x7fdb0815c990 2023-11-28T11:10:14.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.385+0000 7fdb0e0c9700 1 -- 172.21.15.28:0/1558268466 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb080cf740 con 0x7fdb0815c990 2023-11-28T11:10:14.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.386+0000 7fdaeffff700 1 -- 172.21.15.28:0/1558268466 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdafc00f050 con 0x7fdb0815c990 2023-11-28T11:10:14.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.386+0000 7fdaeffff700 1 --2- 172.21.15.28:0/1558268466 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdaf40682f0 0x7fdaf406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:14.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.386+0000 7fdaeffff700 1 -- 172.21.15.28:0/1558268466 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fdafc021070 con 0x7fdb0815c990 2023-11-28T11:10:14.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.386+0000 7fdb077fe700 1 --2- 172.21.15.28:0/1558268466 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdaf40682f0 0x7fdaf406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:14.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.386+0000 7fdb077fe700 1 --2- 172.21.15.28:0/1558268466 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdaf40682f0 0x7fdaf406a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009b70 tx=0x7fdaf8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:14.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.390+0000 7fdaeffff700 1 -- 172.21.15.28:0/1558268466 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdafc053dc0 con 0x7fdb0815c990 2023-11-28T11:10:14.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.584+0000 7fdb0e0c9700 1 -- 172.21.15.28:0/1558268466 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb080675a0 con 0x7fdb0815c990 2023-11-28T11:10:14.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.585+0000 7fdaeffff700 1 -- 172.21.15.28:0/1558268466 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+18022 (secure 0 0 0) 0x7fdafc0253e0 con 0x7fdb0815c990 2023-11-28T11:10:14.586 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:14.586 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":40,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:10:11.822678+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:08.429278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:09.797038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:14.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.587+0000 7fdb0e0c9700 1 -- 172.21.15.28:0/1558268466 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdaf40682f0 msgr2=0x7fdaf406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.587+0000 7fdb0e0c9700 1 --2- 172.21.15.28:0/1558268466 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdaf40682f0 0x7fdaf406a7a0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fdaf8009b70 tx=0x7fdaf8008040 comp rx=0 tx=0).stop 2023-11-28T11:10:14.587 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.587+0000 7fdb0e0c9700 1 -- 172.21.15.28:0/1558268466 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb0815c990 msgr2=0x7fdb080b5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.588 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.587+0000 7fdb0e0c9700 1 --2- 172.21.15.28:0/1558268466 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb0815c990 0x7fdb080b5440 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009c40 tx=0x7fdafc004750 comp rx=0 tx=0).stop 2023-11-28T11:10:14.588 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.587+0000 7fdb0e0c9700 1 -- 172.21.15.28:0/1558268466 shutdown_connections 2023-11-28T11:10:14.588 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.587+0000 7fdb0e0c9700 1 --2- 172.21.15.28:0/1558268466 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdaf40682f0 0x7fdaf406a7a0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.588 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.587+0000 7fdb0e0c9700 1 --2- 172.21.15.28:0/1558268466 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb08161090 0x7fdb08147a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.588 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.587+0000 7fdb0e0c9700 1 --2- 172.21.15.28:0/1558268466 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb080caeb0 0x7fdb081f1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.588 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.587+0000 7fdb0e0c9700 1 --2- 172.21.15.28:0/1558268466 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb0815c990 0x7fdb080b5440 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.588 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.587+0000 7fdb0e0c9700 1 -- 172.21.15.28:0/1558268466 >> 172.21.15.28:0/1558268466 conn(0x7fdb080b1650 msgr2=0x7fdb080c8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:14.588 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.587+0000 7fdb0e0c9700 1 -- 172.21.15.28:0/1558268466 shutdown_connections 2023-11-28T11:10:14.588 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.587+0000 7fdb0e0c9700 1 -- 172.21.15.28:0/1558268466 wait complete. 2023-11-28T11:10:14.600 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:10:14.600 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:10:14.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.725+0000 7f77baca7700 1 Processor -- start 2023-11-28T11:10:14.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.725+0000 7f77baca7700 1 -- start start 2023-11-28T11:10:14.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.726+0000 7f77baca7700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f77b406aa00 0x7f77b40cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:14.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.726+0000 7f77baca7700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b406ae80 con 0x7f77b415d1e0 2023-11-28T11:10:14.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.726+0000 7f77baca7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77b40cdec0 con 0x7f77b406a560 2023-11-28T11:10:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.726+0000 7f77baca7700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77b40c9cd0 con 0x7f77b406aa00 2023-11-28T11:10:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.726+0000 7f77b8a43700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f77b415d1e0 0x7f77b415d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37524/0 (socket says 172.21.15.28:37524) 2023-11-28T11:10:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.726+0000 7f77b8a43700 1 -- 172.21.15.28:0/3852236180 learned_addr learned my addr 172.21.15.28:0/3852236180 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b9244700 1 --2- 172.21.15.28:0/3852236180 >> v2:172.21.15.59:3300/0 conn(0x7f77b406aa00 0x7f77b40cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b37fe700 1 -- 172.21.15.28:0/3852236180 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916046610 0 0) 0x7f77b406ae80 con 0x7f77b415d1e0 2023-11-28T11:10:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b37fe700 1 -- 172.21.15.28:0/3852236180 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77a40035b0 con 0x7f77b415d1e0 2023-11-28T11:10:14.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b37fe700 1 -- 172.21.15.28:0/3852236180 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 314627707 0 0) 0x7f77a40035b0 con 0x7f77b415d1e0 2023-11-28T11:10:14.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b37fe700 1 -- 172.21.15.28:0/3852236180 >> v2:172.21.15.59:3300/0 conn(0x7f77b406aa00 msgr2=0x7f77b40cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:14.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b37fe700 1 --2- 172.21.15.28:0/3852236180 >> v2:172.21.15.59:3300/0 conn(0x7f77b406aa00 0x7f77b40cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b37fe700 1 -- 172.21.15.28:0/3852236180 >> v1:172.21.15.59:6789/0 conn(0x7f77b406a560 legacy=0x7f77b40cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b37fe700 1 -- 172.21.15.28:0/3852236180 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b40c9fb0 con 0x7f77b415d1e0 2023-11-28T11:10:14.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b37fe700 1 -- 172.21.15.28:0/3852236180 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f77a8001670 con 0x7f77b415d1e0 2023-11-28T11:10:14.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b37fe700 1 -- 172.21.15.28:0/3852236180 >> v1:172.21.15.28:6789/0 conn(0x7f77b415d1e0 legacy=0x7f77b415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b37fe700 1 --2- 172.21.15.28:0/3852236180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77a4003a00 0x7f77a4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:14.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b37fe700 1 --2- 172.21.15.28:0/3852236180 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77a4004850 0x7f77a4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:14.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b37fe700 1 --2- 172.21.15.28:0/3852236180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77a4005200 0x7f77a400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:14.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b37fe700 1 -- 172.21.15.28:0/3852236180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40c9cd0 con 0x7f77a4003a00 2023-11-28T11:10:14.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b37fe700 1 -- 172.21.15.28:0/3852236180 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77a400c080 con 0x7f77a4005200 2023-11-28T11:10:14.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b37fe700 1 -- 172.21.15.28:0/3852236180 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77a400c240 con 0x7f77a4004850 2023-11-28T11:10:14.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b8a43700 1 --2- 172.21.15.28:0/3852236180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77a4003a00 0x7f77a4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:14.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b9244700 1 --2- 172.21.15.28:0/3852236180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77a4005200 0x7f77a400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:14.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.727+0000 7f77b3fff700 1 --2- 172.21.15.28:0/3852236180 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77a4004850 0x7f77a4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:14.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.728+0000 7f77b8a43700 1 -- 172.21.15.28:0/3852236180 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77a4004850 msgr2=0x7f77a4004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:14.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.728+0000 7f77b8a43700 1 --2- 172.21.15.28:0/3852236180 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77a4004850 0x7f77a4004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.728+0000 7f77b8a43700 1 -- 172.21.15.28:0/3852236180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77a4005200 msgr2=0x7f77a400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:14.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.728+0000 7f77b8a43700 1 --2- 172.21.15.28:0/3852236180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77a4005200 0x7f77a400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.728+0000 7f77b8a43700 1 -- 172.21.15.28:0/3852236180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77a40035b0 con 0x7f77a4003a00 2023-11-28T11:10:14.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.728+0000 7f77b9244700 1 -- 172.21.15.28:0/3852236180 reap_dead start 2023-11-28T11:10:14.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.728+0000 7f77b8a43700 1 --2- 172.21.15.28:0/3852236180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77a4003a00 0x7f77a4003e10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f77a800c240 tx=0x7f77a4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663beb4a36ba9280 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:14.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.728+0000 7f77b37fe700 1 -- 172.21.15.28:0/3852236180 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a8007c10 con 0x7f77a4003a00 2023-11-28T11:10:14.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.728+0000 7f77b37fe700 1 -- 172.21.15.28:0/3852236180 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a800cd60 con 0x7f77a4003a00 2023-11-28T11:10:14.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.728+0000 7f77b37fe700 1 -- 172.21.15.28:0/3852236180 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a8005ba0 con 0x7f77a4003a00 2023-11-28T11:10:14.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.728+0000 7f77baca7700 1 -- 172.21.15.28:0/3852236180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77a4003a00 msgr2=0x7f77a4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.728+0000 7f77baca7700 1 --2- 172.21.15.28:0/3852236180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77a4003a00 0x7f77a4003e10 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f77a800c240 tx=0x7f77a4005880 comp rx=0 tx=0).stop 2023-11-28T11:10:14.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.728+0000 7f77baca7700 1 -- 172.21.15.28:0/3852236180 shutdown_connections 2023-11-28T11:10:14.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.728+0000 7f77baca7700 1 --2- 172.21.15.28:0/3852236180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77a4003a00 0x7f77a4003e10 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.728+0000 7f77baca7700 1 -- 172.21.15.28:0/3852236180 >> 172.21.15.28:0/3852236180 conn(0x7f77b40b0dd0 msgr2=0x7f77b40b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.729+0000 7f77baca7700 1 -- 172.21.15.28:0/3852236180 shutdown_connections 2023-11-28T11:10:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.729+0000 7f77baca7700 1 -- 172.21.15.28:0/3852236180 wait complete. 2023-11-28T11:10:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.730+0000 7f77baca7700 1 Processor -- start 2023-11-28T11:10:14.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.730+0000 7f77baca7700 1 -- start start 2023-11-28T11:10:14.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77baca7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77a4003a00 0x7f77b40c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:14.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77baca7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b40b3790 0x7f77b40b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:14.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77baca7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b41503f0 0x7f77b4150860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:14.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77baca7700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b4150da0 con 0x7f77a4003a00 2023-11-28T11:10:14.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77baca7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40ab8c0 con 0x7f77b41503f0 2023-11-28T11:10:14.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77baca7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77b40b3ee0 con 0x7f77b40b3790 2023-11-28T11:10:14.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77b8a43700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77a4003a00 0x7f77b40c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:14.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77b8a43700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77a4003a00 0x7f77b40c5a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49478/0 (socket says 172.21.15.28:49478) 2023-11-28T11:10:14.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77b8a43700 1 -- 172.21.15.28:0/2967370376 learned_addr learned my addr 172.21.15.28:0/2967370376 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:14.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77b3fff700 1 --2- 172.21.15.28:0/2967370376 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b40b3790 0x7f77b40b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77b9244700 1 --2- 172.21.15.28:0/2967370376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b41503f0 0x7f77b4150860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77b8a43700 1 -- 172.21.15.28:0/2967370376 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b40b3790 msgr2=0x7f77b40b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77b8a43700 1 --2- 172.21.15.28:0/2967370376 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b40b3790 0x7f77b40b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77b8a43700 1 -- 172.21.15.28:0/2967370376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b41503f0 msgr2=0x7f77b4150860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77b8a43700 1 --2- 172.21.15.28:0/2967370376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b41503f0 0x7f77b4150860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77b8a43700 1 -- 172.21.15.28:0/2967370376 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77b40ca570 con 0x7f77a4003a00 2023-11-28T11:10:14.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77b3fff700 1 --2- 172.21.15.28:0/2967370376 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b40b3790 0x7f77b40b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:14.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77b8a43700 1 --2- 172.21.15.28:0/2967370376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77a4003a00 0x7f77b40c5a50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f77a800c830 tx=0x7f77a8001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:14.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77b17fa700 1 -- 172.21.15.28:0/2967370376 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a8000ed0 con 0x7f77a4003a00 2023-11-28T11:10:14.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77b17fa700 1 -- 172.21.15.28:0/2967370376 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77a80070d0 con 0x7f77a4003a00 2023-11-28T11:10:14.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77baca7700 1 -- 172.21.15.28:0/2967370376 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77b40ca290 con 0x7f77a4003a00 2023-11-28T11:10:14.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.731+0000 7f77b17fa700 1 -- 172.21.15.28:0/2967370376 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77a8007480 con 0x7f77a4003a00 2023-11-28T11:10:14.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.732+0000 7f77baca7700 1 -- 172.21.15.28:0/2967370376 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77b40ca850 con 0x7f77a4003a00 2023-11-28T11:10:14.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.732+0000 7f77baca7700 1 -- 172.21.15.28:0/2967370376 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77b40cb140 con 0x7f77a4003a00 2023-11-28T11:10:14.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.733+0000 7f77b17fa700 1 -- 172.21.15.28:0/2967370376 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f77a8024030 con 0x7f77a4003a00 2023-11-28T11:10:14.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.733+0000 7f77b17fa700 1 --2- 172.21.15.28:0/2967370376 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f77980682f0 0x7f779806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:14.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.733+0000 7f77b17fa700 1 -- 172.21.15.28:0/2967370376 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f77a8086f10 con 0x7f77a4003a00 2023-11-28T11:10:14.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.733+0000 7f77b3fff700 1 --2- 172.21.15.28:0/2967370376 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f77980682f0 0x7f779806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:14.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.734+0000 7f77b3fff700 1 --2- 172.21.15.28:0/2967370376 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f77980682f0 0x7f779806a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f77a0000c00 tx=0x7f77a0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:14.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.738+0000 7f77b17fa700 1 -- 172.21.15.28:0/2967370376 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f77a8053630 con 0x7f77a4003a00 2023-11-28T11:10:14.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.927+0000 7f77baca7700 1 -- 172.21.15.28:0/2967370376 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f77b40675a0 con 0x7f77980682f0 2023-11-28T11:10:14.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.936+0000 7f77b17fa700 1 -- 172.21.15.28:0/2967370376 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296267 (secure 0 0 0) 0x7f77b40675a0 con 0x7f77980682f0 2023-11-28T11:10:14.940 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:14.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.939+0000 7f77baca7700 1 -- 172.21.15.28:0/2967370376 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f77980682f0 msgr2=0x7f779806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.939+0000 7f77baca7700 1 --2- 172.21.15.28:0/2967370376 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f77980682f0 0x7f779806a7a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f77a0000c00 tx=0x7f77a0009040 comp rx=0 tx=0).stop 2023-11-28T11:10:14.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.939+0000 7f77baca7700 1 -- 172.21.15.28:0/2967370376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77a4003a00 msgr2=0x7f77b40c5a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:14.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.939+0000 7f77baca7700 1 --2- 172.21.15.28:0/2967370376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77a4003a00 0x7f77b40c5a50 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f77a800c830 tx=0x7f77a8001a00 comp rx=0 tx=0).stop 2023-11-28T11:10:14.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.939+0000 7f77baca7700 1 -- 172.21.15.28:0/2967370376 shutdown_connections 2023-11-28T11:10:14.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.939+0000 7f77baca7700 1 --2- 172.21.15.28:0/2967370376 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f77980682f0 0x7f779806a7a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.939+0000 7f77baca7700 1 --2- 172.21.15.28:0/2967370376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77a4003a00 0x7f77b40c5a50 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.939+0000 7f77baca7700 1 --2- 172.21.15.28:0/2967370376 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77b41503f0 0x7f77b4150860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.939+0000 7f77baca7700 1 --2- 172.21.15.28:0/2967370376 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77b40b3790 0x7f77b40b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:14.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.939+0000 7f77baca7700 1 -- 172.21.15.28:0/2967370376 >> 172.21.15.28:0/2967370376 conn(0x7f77b40b0dd0 msgr2=0x7f77b40b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:14.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.939+0000 7f77baca7700 1 -- 172.21.15.28:0/2967370376 shutdown_connections 2023-11-28T11:10:14.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:14.939+0000 7f77baca7700 1 -- 172.21.15.28:0/2967370376 wait complete. 2023-11-28T11:10:14.943 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:10:14.952 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":81,"stamp":"2023-11-28T11:10:13.181646+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":69732,"kb_used_data":4388,"kb_used_omap":0,"kb_used_meta":65280,"kb_avail":754904988,"statfs":{"total":773094113280,"available":773022707712,"internally_reserved":0,"allocated":4493312,"data_stored":2623175,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":66846720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000686"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:10:02.590446+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:10:02.590446+0000","last_peered":"2023-11-28T11:10:02.590446+0000","last_clean":"2023-11-28T11:10:02.590446+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:10:02.590446+0000","last_undegraded":"2023-11-28T11:10:02.590446+0000","last_fullsized":"2023-11-28T11:10:02.590446+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:10:02.888610+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:10:02.888610+0000","last_peered":"2023-11-28T11:10:02.888610+0000","last_clean":"2023-11-28T11:10:02.888610+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:10:02.888610+0000","last_undegraded":"2023-11-28T11:10:02.888610+0000","last_fullsized":"2023-11-28T11:10:02.888610+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":27,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:10:02.888869+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:10:02.888869+0000","last_peered":"2023-11-28T11:10:02.888869+0000","last_clean":"2023-11-28T11:10:02.888869+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:10:02.888869+0000","last_undegraded":"2023-11-28T11:10:02.888869+0000","last_fullsized":"2023-11-28T11:10:02.888869+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":176,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:10:02.888152+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:10:02.888152+0000","last_peered":"2023-11-28T11:10:02.888152+0000","last_clean":"2023-11-28T11:10:02.888152+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:10:02.888152+0000","last_undegraded":"2023-11-28T11:10:02.888152+0000","last_fullsized":"2023-11-28T11:10:02.888152+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:10:02.888157+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:10:02.888157+0000","last_peered":"2023-11-28T11:10:02.888157+0000","last_clean":"2023-11-28T11:10:02.888157+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:10:02.888157+0000","last_undegraded":"2023-11-28T11:10:02.888157+0000","last_fullsized":"2023-11-28T11:10:02.888157+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":26,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:10:02.888156+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:10:02.888156+0000","last_peered":"2023-11-28T11:10:02.888156+0000","last_clean":"2023-11-28T11:10:02.888156+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:10:02.888156+0000","last_undegraded":"2023-11-28T11:10:02.888156+0000","last_fullsized":"2023-11-28T11:10:02.888156+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":28,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:10:02.588598+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:10:02.588598+0000","last_peered":"2023-11-28T11:10:02.588598+0000","last_clean":"2023-11-28T11:10:02.588598+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:10:02.588598+0000","last_undegraded":"2023-11-28T11:10:02.588598+0000","last_fullsized":"2023-11-28T11:10:02.588598+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:10:02.890434+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:10:02.890434+0000","last_peered":"2023-11-28T11:10:02.890434+0000","last_clean":"2023-11-28T11:10:02.890434+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:10:02.890434+0000","last_undegraded":"2023-11-28T11:10:02.890434+0000","last_fullsized":"2023-11-28T11:10:02.890434+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:10.399114+0000","last_change":"2023-11-28T11:10:10.399114+0000","last_active":"2023-11-28T11:10:10.399114+0000","last_peered":"2023-11-28T11:10:10.399114+0000","last_clean":"2023-11-28T11:10:10.399114+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:10.399114+0000","last_undegraded":"2023-11-28T11:10:10.399114+0000","last_fullsized":"2023-11-28T11:10:10.399114+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:09.392014+0000","last_change":"2023-11-28T11:10:09.392014+0000","last_active":"2023-11-28T11:10:09.392014+0000","last_peered":"2023-11-28T11:10:09.392014+0000","last_clean":"2023-11-28T11:10:09.392014+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:09.392014+0000","last_undegraded":"2023-11-28T11:10:09.392014+0000","last_fullsized":"2023-11-28T11:10:09.392014+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":24,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-11-28T11:10:02.891909+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:10:02.891909+0000","last_peered":"2023-11-28T11:10:02.891909+0000","last_clean":"2023-11-28T11:10:02.891909+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:10:02.891909+0000","last_undegraded":"2023-11-28T11:10:02.891909+0000","last_fullsized":"2023-11-28T11:10:02.891909+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9568,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362272,"statfs":{"total":96636764160,"available":96626966528,"internally_reserved":0,"allocated":1007616,"data_stored":770793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.2000000000000002,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.48899999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":3.1360000000000001,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.71999999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.3969999999999998,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.66400000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.3069999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.294,"5min":3.294,"15min":3.294},"last":0.70799999999999996}]},{"osd":4,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.64300000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.86399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574869,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8348,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363492,"statfs":{"total":96636764160,"available":96628215808,"internally_reserved":0,"allocated":413696,"data_stored":180425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.374},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.34999999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.32300000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":3.7429999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.46700000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":3.7749999999999999,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.44700000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57099999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.55700000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.61299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574869,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9568,"kb_used_data":984,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362272,"statfs":{"total":96636764160,"available":96626966528,"internally_reserved":0,"allocated":1007616,"data_stored":770793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.252},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.39100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.29199999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.438}]},{"osd":4,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.73499999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.62}]},{"osd":6,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.77300000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.57199999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8476,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363364,"statfs":{"total":96636764160,"available":96628084736,"internally_reserved":0,"allocated":413696,"data_stored":180425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.44400000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.30299999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.36799999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.41199999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.504}]},{"osd":6,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.47899999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.54500000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8344,"kb_used_data":400,"kb_used_omap":0,"kb_used_meta":7936,"kb_avail":94363496,"statfs":{"total":96636764160,"available":96628219904,"internally_reserved":0,"allocated":409600,"data_stored":179464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8126464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.51900000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.58299999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.47299999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.70999999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.72999999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.68200000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.621}]}]},{"osd":3,"up_from":13,"seq":55834574869,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8476,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363364,"statfs":{"total":96636764160,"available":96628084736,"internally_reserved":0,"allocated":413696,"data_stored":180425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":11.965,"5min":11.965,"15min":11.965},"last":0.224},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":12.888999999999999,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.29599999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.26200000000000001},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":16.988,"5min":16.988,"15min":16.988},"last":0.27900000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":1.895,"5min":1.895,"15min":1.895},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":27.887,"5min":27.887,"15min":27.887},"last":0.39400000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.753},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.53200000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":0.65500000000000003},{"interface":"front","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.933999999999999,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":11.888,"5min":11.888,"15min":11.888},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.69599999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.73199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574870,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8540,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363300,"statfs":{"total":96636764160,"available":96628019200,"internally_reserved":0,"allocated":413696,"data_stored":180425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":10.045999999999999,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.61799999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.60799999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52900000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.59399999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.50600000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.439},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.58099999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.67800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574870,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8412,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":8000,"kb_avail":94363428,"statfs":{"total":96636764160,"available":96628150272,"internally_reserved":0,"allocated":413696,"data_stored":180425,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8192000},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.8849999999999998,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.64600000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.497},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58299999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.8999999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.42999999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.629},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.66800000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.68000000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.70999999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:10:14.958 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:10:15.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.116+0000 7fec5d780700 1 Processor -- start 2023-11-28T11:10:15.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.116+0000 7fec5d780700 1 -- start start 2023-11-28T11:10:15.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.117+0000 7fec5d780700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fec580cb230 0x7fec580ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.117+0000 7fec5d780700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fec580b2660 0x7fec580b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.117+0000 7fec5d780700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec580cfb90 con 0x7fec5815a6c0 2023-11-28T11:10:15.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.117+0000 7fec5d780700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec580b3020 con 0x7fec580cb230 2023-11-28T11:10:15.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.117+0000 7fec5d780700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec580cb680 con 0x7fec580b2660 2023-11-28T11:10:15.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.117+0000 7fec567fc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fec580cb230 0x7fec580ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.117+0000 7fec56ffd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fec5815a6c0 0x7fec580d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37544/0 (socket says 172.21.15.28:37544) 2023-11-28T11:10:15.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.117+0000 7fec56ffd700 1 -- 172.21.15.28:0/1522579362 learned_addr learned my addr 172.21.15.28:0/1522579362 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:15.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.118+0000 7fec577fe700 1 --2- 172.21.15.28:0/1522579362 >> v2:172.21.15.59:3300/0 conn(0x7fec580b2660 0x7fec580b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.118+0000 7fec567fc700 1 -- 172.21.15.28:0/1522579362 >> v2:172.21.15.59:3300/0 conn(0x7fec580b2660 msgr2=0x7fec580b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:15.120 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.118+0000 7fec567fc700 1 --2- 172.21.15.28:0/1522579362 >> v2:172.21.15.59:3300/0 conn(0x7fec580b2660 0x7fec580b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.118+0000 7fec567fc700 1 -- 172.21.15.28:0/1522579362 >> v1:172.21.15.28:6789/0 conn(0x7fec5815a6c0 legacy=0x7fec580d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.118+0000 7fec567fc700 1 -- 172.21.15.28:0/1522579362 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec580cb930 con 0x7fec580cb230 2023-11-28T11:10:15.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.118+0000 7fec55ffb700 1 -- 172.21.15.28:0/1522579362 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131628574 0 0) 0x7fec580cfb90 con 0x7fec5815a6c0 2023-11-28T11:10:15.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.118+0000 7fec577fe700 1 --2- 172.21.15.28:0/1522579362 >> v2:172.21.15.59:3300/0 conn(0x7fec580b2660 0x7fec580b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:15.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.118+0000 7fec567fc700 1 --2- 172.21.15.28:0/1522579362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec580cb230 0x7fec580ccea0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fec580cb1c0 tx=0x7fec48009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79ec22c91438f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:15.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.118+0000 7fec55ffb700 1 -- 172.21.15.28:0/1522579362 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48011830 con 0x7fec580cb230 2023-11-28T11:10:15.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.118+0000 7fec55ffb700 1 -- 172.21.15.28:0/1522579362 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec48011990 con 0x7fec580cb230 2023-11-28T11:10:15.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.118+0000 7fec55ffb700 1 -- 172.21.15.28:0/1522579362 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec48011d20 con 0x7fec580cb230 2023-11-28T11:10:15.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.119+0000 7fec5d780700 1 -- 172.21.15.28:0/1522579362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec580cb230 msgr2=0x7fec580ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.119+0000 7fec5d780700 1 --2- 172.21.15.28:0/1522579362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec580cb230 0x7fec580ccea0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fec580cb1c0 tx=0x7fec48009b20 comp rx=0 tx=0).stop 2023-11-28T11:10:15.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.119+0000 7fec5d780700 1 -- 172.21.15.28:0/1522579362 shutdown_connections 2023-11-28T11:10:15.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.119+0000 7fec5d780700 1 --2- 172.21.15.28:0/1522579362 >> v2:172.21.15.59:3300/0 conn(0x7fec580b2660 0x7fec580b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.119+0000 7fec5d780700 1 --2- 172.21.15.28:0/1522579362 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec580cb230 0x7fec580ccea0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.119+0000 7fec5d780700 1 -- 172.21.15.28:0/1522579362 >> 172.21.15.28:0/1522579362 conn(0x7fec580cd660 msgr2=0x7fec580cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:15.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.119+0000 7fec5d780700 1 -- 172.21.15.28:0/1522579362 shutdown_connections 2023-11-28T11:10:15.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.119+0000 7fec5d780700 1 -- 172.21.15.28:0/1522579362 wait complete. 2023-11-28T11:10:15.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.120+0000 7fec5d780700 1 Processor -- start 2023-11-28T11:10:15.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.120+0000 7fec5d780700 1 -- start start 2023-11-28T11:10:15.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec5d780700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fec580b2660 0x7fec58158490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec5d780700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec580cb230 0x7fec5815aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec5d780700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec5815a6c0 0x7fec5815d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec5d780700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5815c510 con 0x7fec5815a6c0 2023-11-28T11:10:15.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec5d780700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5815b470 con 0x7fec580cb230 2023-11-28T11:10:15.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec5d780700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec5815b750 con 0x7fec580b2660 2023-11-28T11:10:15.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec577fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec5815a6c0 0x7fec5815d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec577fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec5815a6c0 0x7fec5815d030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49496/0 (socket says 172.21.15.28:49496) 2023-11-28T11:10:15.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec577fe700 1 -- 172.21.15.28:0/3537545332 learned_addr learned my addr 172.21.15.28:0/3537545332 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:15.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec567fc700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec580cb230 0x7fec5815aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec56ffd700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fec580b2660 0x7fec58158490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec577fe700 1 -- 172.21.15.28:0/3537545332 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fec580b2660 msgr2=0x7fec58158490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec577fe700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fec580b2660 0x7fec58158490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec577fe700 1 -- 172.21.15.28:0/3537545332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec580cb230 msgr2=0x7fec5815aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec577fe700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec580cb230 0x7fec5815aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec577fe700 1 -- 172.21.15.28:0/3537545332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec480094a0 con 0x7fec5815a6c0 2023-11-28T11:10:15.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec567fc700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec580cb230 0x7fec5815aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:15.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec56ffd700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fec580b2660 0x7fec58158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:15.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.121+0000 7fec577fe700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec5815a6c0 0x7fec5815d030 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fec5000cd50 tx=0x7fec5000a560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:15.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.122+0000 7fec3f7fe700 1 -- 172.21.15.28:0/3537545332 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5000f480 con 0x7fec5815a6c0 2023-11-28T11:10:15.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.122+0000 7fec3f7fe700 1 -- 172.21.15.28:0/3537545332 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5000f600 con 0x7fec5815a6c0 2023-11-28T11:10:15.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.122+0000 7fec5d780700 1 -- 172.21.15.28:0/3537545332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec580cf540 con 0x7fec5815a6c0 2023-11-28T11:10:15.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.122+0000 7fec3f7fe700 1 -- 172.21.15.28:0/3537545332 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5000f9b0 con 0x7fec5815a6c0 2023-11-28T11:10:15.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.122+0000 7fec5d780700 1 -- 172.21.15.28:0/3537545332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec580cf260 con 0x7fec5815a6c0 2023-11-28T11:10:15.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.123+0000 7fec5d780700 1 -- 172.21.15.28:0/3537545332 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec580cf820 con 0x7fec5815a6c0 2023-11-28T11:10:15.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.123+0000 7fec3f7fe700 1 -- 172.21.15.28:0/3537545332 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fec50009030 con 0x7fec5815a6c0 2023-11-28T11:10:15.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.123+0000 7fec3f7fe700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fec440682f0 0x7fec4406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.124+0000 7fec56ffd700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fec440682f0 0x7fec4406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.124+0000 7fec3f7fe700 1 -- 172.21.15.28:0/3537545332 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fec500867a0 con 0x7fec5815a6c0 2023-11-28T11:10:15.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.124+0000 7fec56ffd700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fec440682f0 0x7fec4406a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fec4c000f60 tx=0x7fec4c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:15.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.128+0000 7fec3f7fe700 1 -- 172.21.15.28:0/3537545332 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fec50052ec0 con 0x7fec5815a6c0 2023-11-28T11:10:15.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.316+0000 7fec5d780700 1 -- 172.21.15.28:0/3537545332 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec580675a0 con 0x7fec440682f0 2023-11-28T11:10:15.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.325+0000 7fec3f7fe700 1 -- 172.21.15.28:0/3537545332 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311087 (secure 0 0 0) 0x7fec580675a0 con 0x7fec440682f0 2023-11-28T11:10:15.333 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:15.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.328+0000 7fec5d780700 1 -- 172.21.15.28:0/3537545332 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fec440682f0 msgr2=0x7fec4406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.328+0000 7fec5d780700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fec440682f0 0x7fec4406a7a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fec4c000f60 tx=0x7fec4c009040 comp rx=0 tx=0).stop 2023-11-28T11:10:15.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.328+0000 7fec5d780700 1 -- 172.21.15.28:0/3537545332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec5815a6c0 msgr2=0x7fec5815d030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.328+0000 7fec5d780700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec5815a6c0 0x7fec5815d030 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fec5000cd50 tx=0x7fec5000a560 comp rx=0 tx=0).stop 2023-11-28T11:10:15.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.328+0000 7fec5d780700 1 -- 172.21.15.28:0/3537545332 shutdown_connections 2023-11-28T11:10:15.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.328+0000 7fec5d780700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fec440682f0 0x7fec4406a7a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.328+0000 7fec5d780700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fec5815a6c0 0x7fec5815d030 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.328+0000 7fec5d780700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fec580b2660 0x7fec58158490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.339 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.328+0000 7fec5d780700 1 --2- 172.21.15.28:0/3537545332 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec580cb230 0x7fec5815aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.339 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.328+0000 7fec5d780700 1 -- 172.21.15.28:0/3537545332 >> 172.21.15.28:0/3537545332 conn(0x7fec580cd660 msgr2=0x7fec580b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:15.339 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.328+0000 7fec5d780700 1 -- 172.21.15.28:0/3537545332 shutdown_connections 2023-11-28T11:10:15.339 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.328+0000 7fec5d780700 1 -- 172.21.15.28:0/3537545332 wait complete. 2023-11-28T11:10:15.339 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:10:15.341 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":82,"stamp":"2023-11-28T11:10:15.181960+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":71460,"kb_used_data":4836,"kb_used_omap":0,"kb_used_meta":66560,"kb_avail":754903260,"statfs":{"total":773094113280,"available":773020938240,"internally_reserved":0,"allocated":4952064,"data_stored":2850643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":68157440},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.782785"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.322837+0000","last_change":"2023-11-28T11:10:10.843189+0000","last_active":"2023-11-28T11:10:12.322837+0000","last_peered":"2023-11-28T11:10:12.322837+0000","last_clean":"2023-11-28T11:10:12.322837+0000","last_became_active":"2023-11-28T11:10:10.842919+0000","last_became_peered":"2023-11-28T11:10:10.842919+0000","last_unstale":"2023-11-28T11:10:12.322837+0000","last_undegraded":"2023-11-28T11:10:12.322837+0000","last_fullsized":"2023-11-28T11:10:12.322837+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:13.147185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.830918+0000","last_change":"2023-11-28T11:10:10.854947+0000","last_active":"2023-11-28T11:10:11.830918+0000","last_peered":"2023-11-28T11:10:11.830918+0000","last_clean":"2023-11-28T11:10:11.830918+0000","last_became_active":"2023-11-28T11:10:10.854662+0000","last_became_peered":"2023-11-28T11:10:10.854662+0000","last_unstale":"2023-11-28T11:10:11.830918+0000","last_undegraded":"2023-11-28T11:10:11.830918+0000","last_fullsized":"2023-11-28T11:10:11.830918+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:49:24.782807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.382994+0000","last_change":"2023-11-28T11:10:10.848449+0000","last_active":"2023-11-28T11:10:12.382994+0000","last_peered":"2023-11-28T11:10:12.382994+0000","last_clean":"2023-11-28T11:10:12.382994+0000","last_became_active":"2023-11-28T11:10:10.848215+0000","last_became_peered":"2023-11-28T11:10:10.848215+0000","last_unstale":"2023-11-28T11:10:12.382994+0000","last_undegraded":"2023-11-28T11:10:12.382994+0000","last_fullsized":"2023-11-28T11:10:12.382994+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:47:47.621814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948372+0000","last_change":"2023-11-28T11:10:10.849325+0000","last_active":"2023-11-28T11:10:11.948372+0000","last_peered":"2023-11-28T11:10:11.948372+0000","last_clean":"2023-11-28T11:10:11.948372+0000","last_became_active":"2023-11-28T11:10:10.848951+0000","last_became_peered":"2023-11-28T11:10:10.848951+0000","last_unstale":"2023-11-28T11:10:11.948372+0000","last_undegraded":"2023-11-28T11:10:11.948372+0000","last_fullsized":"2023-11-28T11:10:11.948372+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:29:40.483759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.830713+0000","last_change":"2023-11-28T11:10:10.855052+0000","last_active":"2023-11-28T11:10:11.830713+0000","last_peered":"2023-11-28T11:10:11.830713+0000","last_clean":"2023-11-28T11:10:11.830713+0000","last_became_active":"2023-11-28T11:10:10.854715+0000","last_became_peered":"2023-11-28T11:10:10.854715+0000","last_unstale":"2023-11-28T11:10:11.830713+0000","last_undegraded":"2023-11-28T11:10:11.830713+0000","last_fullsized":"2023-11-28T11:10:11.830713+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:50:33.336389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.233396+0000","last_change":"2023-11-28T11:10:10.845737+0000","last_active":"2023-11-28T11:10:12.233396+0000","last_peered":"2023-11-28T11:10:12.233396+0000","last_clean":"2023-11-28T11:10:12.233396+0000","last_became_active":"2023-11-28T11:10:10.845482+0000","last_became_peered":"2023-11-28T11:10:10.845482+0000","last_unstale":"2023-11-28T11:10:12.233396+0000","last_undegraded":"2023-11-28T11:10:12.233396+0000","last_fullsized":"2023-11-28T11:10:12.233396+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:05.083157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.322470+0000","last_change":"2023-11-28T11:10:10.842639+0000","last_active":"2023-11-28T11:10:12.322470+0000","last_peered":"2023-11-28T11:10:12.322470+0000","last_clean":"2023-11-28T11:10:12.322470+0000","last_became_active":"2023-11-28T11:10:10.842356+0000","last_became_peered":"2023-11-28T11:10:10.842356+0000","last_unstale":"2023-11-28T11:10:12.322470+0000","last_undegraded":"2023-11-28T11:10:12.322470+0000","last_fullsized":"2023-11-28T11:10:12.322470+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:59.832830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948044+0000","last_change":"2023-11-28T11:10:10.854319+0000","last_active":"2023-11-28T11:10:11.948044+0000","last_peered":"2023-11-28T11:10:11.948044+0000","last_clean":"2023-11-28T11:10:11.948044+0000","last_became_active":"2023-11-28T11:10:10.853864+0000","last_became_peered":"2023-11-28T11:10:10.853864+0000","last_unstale":"2023-11-28T11:10:11.948044+0000","last_undegraded":"2023-11-28T11:10:11.948044+0000","last_fullsized":"2023-11-28T11:10:11.948044+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:08.823912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948385+0000","last_change":"2023-11-28T11:10:10.844173+0000","last_active":"2023-11-28T11:10:11.948385+0000","last_peered":"2023-11-28T11:10:11.948385+0000","last_clean":"2023-11-28T11:10:11.948385+0000","last_became_active":"2023-11-28T11:10:10.843918+0000","last_became_peered":"2023-11-28T11:10:10.843918+0000","last_unstale":"2023-11-28T11:10:11.948385+0000","last_undegraded":"2023-11-28T11:10:11.948385+0000","last_fullsized":"2023-11-28T11:10:11.948385+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:38.162304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948425+0000","last_change":"2023-11-28T11:10:10.852077+0000","last_active":"2023-11-28T11:10:11.948425+0000","last_peered":"2023-11-28T11:10:11.948425+0000","last_clean":"2023-11-28T11:10:11.948425+0000","last_became_active":"2023-11-28T11:10:10.851611+0000","last_became_peered":"2023-11-28T11:10:10.851611+0000","last_unstale":"2023-11-28T11:10:11.948425+0000","last_undegraded":"2023-11-28T11:10:11.948425+0000","last_fullsized":"2023-11-28T11:10:11.948425+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:43:52.436092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948591+0000","last_change":"2023-11-28T11:10:10.849321+0000","last_active":"2023-11-28T11:10:11.948591+0000","last_peered":"2023-11-28T11:10:11.948591+0000","last_clean":"2023-11-28T11:10:11.948591+0000","last_became_active":"2023-11-28T11:10:10.848982+0000","last_became_peered":"2023-11-28T11:10:10.848982+0000","last_unstale":"2023-11-28T11:10:11.948591+0000","last_undegraded":"2023-11-28T11:10:11.948591+0000","last_fullsized":"2023-11-28T11:10:11.948591+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:21:53.405812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.831245+0000","last_change":"2023-11-28T11:10:10.850411+0000","last_active":"2023-11-28T11:10:11.831245+0000","last_peered":"2023-11-28T11:10:11.831245+0000","last_clean":"2023-11-28T11:10:11.831245+0000","last_became_active":"2023-11-28T11:10:10.849958+0000","last_became_peered":"2023-11-28T11:10:10.849958+0000","last_unstale":"2023-11-28T11:10:11.831245+0000","last_undegraded":"2023-11-28T11:10:11.831245+0000","last_fullsized":"2023-11-28T11:10:11.831245+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:13.147185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.828913+0000","last_change":"2023-11-28T11:10:10.840451+0000","last_active":"2023-11-28T11:10:11.828913+0000","last_peered":"2023-11-28T11:10:11.828913+0000","last_clean":"2023-11-28T11:10:11.828913+0000","last_became_active":"2023-11-28T11:10:10.840187+0000","last_became_peered":"2023-11-28T11:10:10.840187+0000","last_unstale":"2023-11-28T11:10:11.828913+0000","last_undegraded":"2023-11-28T11:10:11.828913+0000","last_fullsized":"2023-11-28T11:10:11.828913+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:52:03.261038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948117+0000","last_change":"2023-11-28T11:10:10.847134+0000","last_active":"2023-11-28T11:10:11.948117+0000","last_peered":"2023-11-28T11:10:11.948117+0000","last_clean":"2023-11-28T11:10:11.948117+0000","last_became_active":"2023-11-28T11:10:10.846864+0000","last_became_peered":"2023-11-28T11:10:10.846864+0000","last_unstale":"2023-11-28T11:10:11.948117+0000","last_undegraded":"2023-11-28T11:10:11.948117+0000","last_fullsized":"2023-11-28T11:10:11.948117+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:05.083157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.233245+0000","last_change":"2023-11-28T11:10:10.839920+0000","last_active":"2023-11-28T11:10:12.233245+0000","last_peered":"2023-11-28T11:10:12.233245+0000","last_clean":"2023-11-28T11:10:12.233245+0000","last_became_active":"2023-11-28T11:10:10.839464+0000","last_became_peered":"2023-11-28T11:10:10.839464+0000","last_unstale":"2023-11-28T11:10:12.233245+0000","last_undegraded":"2023-11-28T11:10:12.233245+0000","last_fullsized":"2023-11-28T11:10:12.233245+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:47:47.621814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.383693+0000","last_change":"2023-11-28T11:10:10.855349+0000","last_active":"2023-11-28T11:10:12.383693+0000","last_peered":"2023-11-28T11:10:12.383693+0000","last_clean":"2023-11-28T11:10:12.383693+0000","last_became_active":"2023-11-28T11:10:10.854819+0000","last_became_peered":"2023-11-28T11:10:10.854819+0000","last_unstale":"2023-11-28T11:10:12.383693+0000","last_undegraded":"2023-11-28T11:10:12.383693+0000","last_fullsized":"2023-11-28T11:10:12.383693+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:24.085214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.383245+0000","last_change":"2023-11-28T11:10:10.851009+0000","last_active":"2023-11-28T11:10:12.383245+0000","last_peered":"2023-11-28T11:10:12.383245+0000","last_clean":"2023-11-28T11:10:12.383245+0000","last_became_active":"2023-11-28T11:10:10.850737+0000","last_became_peered":"2023-11-28T11:10:10.850737+0000","last_unstale":"2023-11-28T11:10:12.383245+0000","last_undegraded":"2023-11-28T11:10:12.383245+0000","last_fullsized":"2023-11-28T11:10:12.383245+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:51:41.393541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.383097+0000","last_change":"2023-11-28T11:10:10.852055+0000","last_active":"2023-11-28T11:10:12.383097+0000","last_peered":"2023-11-28T11:10:12.383097+0000","last_clean":"2023-11-28T11:10:12.383097+0000","last_became_active":"2023-11-28T11:10:10.851810+0000","last_became_peered":"2023-11-28T11:10:10.851810+0000","last_unstale":"2023-11-28T11:10:12.383097+0000","last_undegraded":"2023-11-28T11:10:12.383097+0000","last_fullsized":"2023-11-28T11:10:12.383097+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:28.068248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.384105+0000","last_change":"2023-11-28T11:10:10.852383+0000","last_active":"2023-11-28T11:10:12.384105+0000","last_peered":"2023-11-28T11:10:12.384105+0000","last_clean":"2023-11-28T11:10:12.384105+0000","last_became_active":"2023-11-28T11:10:10.852067+0000","last_became_peered":"2023-11-28T11:10:10.852067+0000","last_unstale":"2023-11-28T11:10:12.384105+0000","last_undegraded":"2023-11-28T11:10:12.384105+0000","last_fullsized":"2023-11-28T11:10:12.384105+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:33.384225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.829034+0000","last_change":"2023-11-28T11:10:10.851826+0000","last_active":"2023-11-28T11:10:11.829034+0000","last_peered":"2023-11-28T11:10:11.829034+0000","last_clean":"2023-11-28T11:10:11.829034+0000","last_became_active":"2023-11-28T11:10:10.851518+0000","last_became_peered":"2023-11-28T11:10:10.851518+0000","last_unstale":"2023-11-28T11:10:11.829034+0000","last_undegraded":"2023-11-28T11:10:11.829034+0000","last_fullsized":"2023-11-28T11:10:11.829034+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:27:03.338069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.233418+0000","last_change":"2023-11-28T11:10:10.845291+0000","last_active":"2023-11-28T11:10:12.233418+0000","last_peered":"2023-11-28T11:10:12.233418+0000","last_clean":"2023-11-28T11:10:12.233418+0000","last_became_active":"2023-11-28T11:10:10.845023+0000","last_became_peered":"2023-11-28T11:10:10.845023+0000","last_unstale":"2023-11-28T11:10:12.233418+0000","last_undegraded":"2023-11-28T11:10:12.233418+0000","last_fullsized":"2023-11-28T11:10:12.233418+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:28.068248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948213+0000","last_change":"2023-11-28T11:10:10.847158+0000","last_active":"2023-11-28T11:10:11.948213+0000","last_peered":"2023-11-28T11:10:11.948213+0000","last_clean":"2023-11-28T11:10:11.948213+0000","last_became_active":"2023-11-28T11:10:10.846853+0000","last_became_peered":"2023-11-28T11:10:10.846853+0000","last_unstale":"2023-11-28T11:10:11.948213+0000","last_undegraded":"2023-11-28T11:10:11.948213+0000","last_fullsized":"2023-11-28T11:10:11.948213+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:28.068248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.384107+0000","last_change":"2023-11-28T11:10:10.852629+0000","last_active":"2023-11-28T11:10:12.384107+0000","last_peered":"2023-11-28T11:10:12.384107+0000","last_clean":"2023-11-28T11:10:12.384107+0000","last_became_active":"2023-11-28T11:10:10.852325+0000","last_became_peered":"2023-11-28T11:10:10.852325+0000","last_unstale":"2023-11-28T11:10:12.384107+0000","last_undegraded":"2023-11-28T11:10:12.384107+0000","last_fullsized":"2023-11-28T11:10:12.384107+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:51.611826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.383706+0000","last_change":"2023-11-28T11:10:10.847630+0000","last_active":"2023-11-28T11:10:12.383706+0000","last_peered":"2023-11-28T11:10:12.383706+0000","last_clean":"2023-11-28T11:10:12.383706+0000","last_became_active":"2023-11-28T11:10:10.847238+0000","last_became_peered":"2023-11-28T11:10:10.847238+0000","last_unstale":"2023-11-28T11:10:12.383706+0000","last_undegraded":"2023-11-28T11:10:12.383706+0000","last_fullsized":"2023-11-28T11:10:12.383706+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:52:03.261038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.384287+0000","last_change":"2023-11-28T11:10:10.844548+0000","last_active":"2023-11-28T11:10:12.384287+0000","last_peered":"2023-11-28T11:10:12.384287+0000","last_clean":"2023-11-28T11:10:12.384287+0000","last_became_active":"2023-11-28T11:10:10.844202+0000","last_became_peered":"2023-11-28T11:10:10.844202+0000","last_unstale":"2023-11-28T11:10:12.384287+0000","last_undegraded":"2023-11-28T11:10:12.384287+0000","last_fullsized":"2023-11-28T11:10:12.384287+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:13.818458+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.829085+0000","last_change":"2023-11-28T11:10:10.840741+0000","last_active":"2023-11-28T11:10:11.829085+0000","last_peered":"2023-11-28T11:10:11.829085+0000","last_clean":"2023-11-28T11:10:11.829085+0000","last_became_active":"2023-11-28T11:10:10.839929+0000","last_became_peered":"2023-11-28T11:10:10.839929+0000","last_unstale":"2023-11-28T11:10:11.829085+0000","last_undegraded":"2023-11-28T11:10:11.829085+0000","last_fullsized":"2023-11-28T11:10:11.829085+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:50.230908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.383437+0000","last_change":"2023-11-28T11:10:10.851299+0000","last_active":"2023-11-28T11:10:12.383437+0000","last_peered":"2023-11-28T11:10:12.383437+0000","last_clean":"2023-11-28T11:10:12.383437+0000","last_became_active":"2023-11-28T11:10:10.849939+0000","last_became_peered":"2023-11-28T11:10:10.849939+0000","last_unstale":"2023-11-28T11:10:12.383437+0000","last_undegraded":"2023-11-28T11:10:12.383437+0000","last_fullsized":"2023-11-28T11:10:12.383437+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:59:43.903083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.831019+0000","last_change":"2023-11-28T11:10:10.850549+0000","last_active":"2023-11-28T11:10:11.831019+0000","last_peered":"2023-11-28T11:10:11.831019+0000","last_clean":"2023-11-28T11:10:11.831019+0000","last_became_active":"2023-11-28T11:10:10.850309+0000","last_became_peered":"2023-11-28T11:10:10.850309+0000","last_unstale":"2023-11-28T11:10:11.831019+0000","last_undegraded":"2023-11-28T11:10:11.831019+0000","last_fullsized":"2023-11-28T11:10:11.831019+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:26:03.513433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.383880+0000","last_change":"2023-11-28T11:10:10.848888+0000","last_active":"2023-11-28T11:10:12.383880+0000","last_peered":"2023-11-28T11:10:12.383880+0000","last_clean":"2023-11-28T11:10:12.383880+0000","last_became_active":"2023-11-28T11:10:10.848605+0000","last_became_peered":"2023-11-28T11:10:10.848605+0000","last_unstale":"2023-11-28T11:10:12.383880+0000","last_undegraded":"2023-11-28T11:10:12.383880+0000","last_fullsized":"2023-11-28T11:10:12.383880+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:50.230908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.382864+0000","last_change":"2023-11-28T11:10:10.843077+0000","last_active":"2023-11-28T11:10:12.382864+0000","last_peered":"2023-11-28T11:10:12.382864+0000","last_clean":"2023-11-28T11:10:12.382864+0000","last_became_active":"2023-11-28T11:10:10.842807+0000","last_became_peered":"2023-11-28T11:10:10.842807+0000","last_unstale":"2023-11-28T11:10:12.382864+0000","last_undegraded":"2023-11-28T11:10:12.382864+0000","last_fullsized":"2023-11-28T11:10:12.382864+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:38.162304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.233253+0000","last_change":"2023-11-28T11:10:10.843032+0000","last_active":"2023-11-28T11:10:12.233253+0000","last_peered":"2023-11-28T11:10:12.233253+0000","last_clean":"2023-11-28T11:10:12.233253+0000","last_became_active":"2023-11-28T11:10:10.842829+0000","last_became_peered":"2023-11-28T11:10:10.842829+0000","last_unstale":"2023-11-28T11:10:12.233253+0000","last_undegraded":"2023-11-28T11:10:12.233253+0000","last_fullsized":"2023-11-28T11:10:12.233253+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:42:38.162304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.828930+0000","last_change":"2023-11-28T11:10:10.838945+0000","last_active":"2023-11-28T11:10:11.828930+0000","last_peered":"2023-11-28T11:10:11.828930+0000","last_clean":"2023-11-28T11:10:11.828930+0000","last_became_active":"2023-11-28T11:10:10.838668+0000","last_became_peered":"2023-11-28T11:10:10.838668+0000","last_unstale":"2023-11-28T11:10:11.828930+0000","last_undegraded":"2023-11-28T11:10:11.828930+0000","last_fullsized":"2023-11-28T11:10:11.828930+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:57:29.137503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.397203+0000","last_change":"2023-11-28T11:10:10.837772+0000","last_active":"2023-11-28T11:10:12.397203+0000","last_peered":"2023-11-28T11:10:12.397203+0000","last_clean":"2023-11-28T11:10:12.397203+0000","last_became_active":"2023-11-28T11:10:10.837329+0000","last_became_peered":"2023-11-28T11:10:10.837329+0000","last_unstale":"2023-11-28T11:10:12.397203+0000","last_undegraded":"2023-11-28T11:10:12.397203+0000","last_fullsized":"2023-11-28T11:10:12.397203+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:24.085214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"39'4","reported_seq":15,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.233721+0000","last_change":"2023-11-28T11:10:10.423837+0000","last_active":"2023-11-28T11:10:12.233721+0000","last_peered":"2023-11-28T11:10:12.233721+0000","last_clean":"2023-11-28T11:10:12.233721+0000","last_became_active":"2023-11-28T11:10:10.423538+0000","last_became_peered":"2023-11-28T11:10:10.423538+0000","last_unstale":"2023-11-28T11:10:12.233721+0000","last_undegraded":"2023-11-28T11:10:12.233721+0000","last_fullsized":"2023-11-28T11:10:12.233721+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:18:00.047037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.384188+0000","last_change":"2023-11-28T11:10:10.848039+0000","last_active":"2023-11-28T11:10:12.384188+0000","last_peered":"2023-11-28T11:10:12.384188+0000","last_clean":"2023-11-28T11:10:12.384188+0000","last_became_active":"2023-11-28T11:10:10.847780+0000","last_became_peered":"2023-11-28T11:10:10.847780+0000","last_unstale":"2023-11-28T11:10:12.384188+0000","last_undegraded":"2023-11-28T11:10:12.384188+0000","last_fullsized":"2023-11-28T11:10:12.384188+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:17:59.356563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.383225+0000","last_change":"2023-11-28T11:10:10.418300+0000","last_active":"2023-11-28T11:10:12.383225+0000","last_peered":"2023-11-28T11:10:12.383225+0000","last_clean":"2023-11-28T11:10:12.383225+0000","last_became_active":"2023-11-28T11:10:10.418006+0000","last_became_peered":"2023-11-28T11:10:10.418006+0000","last_unstale":"2023-11-28T11:10:12.383225+0000","last_undegraded":"2023-11-28T11:10:12.383225+0000","last_fullsized":"2023-11-28T11:10:12.383225+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:55:38.925000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.831132+0000","last_change":"2023-11-28T11:10:10.845293+0000","last_active":"2023-11-28T11:10:11.831132+0000","last_peered":"2023-11-28T11:10:11.831132+0000","last_clean":"2023-11-28T11:10:11.831132+0000","last_became_active":"2023-11-28T11:10:10.845008+0000","last_became_peered":"2023-11-28T11:10:10.845008+0000","last_unstale":"2023-11-28T11:10:11.831132+0000","last_undegraded":"2023-11-28T11:10:11.831132+0000","last_fullsized":"2023-11-28T11:10:11.831132+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:51:22.578162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.397222+0000","last_change":"2023-11-28T11:10:10.421209+0000","last_active":"2023-11-28T11:10:12.397222+0000","last_peered":"2023-11-28T11:10:12.397222+0000","last_clean":"2023-11-28T11:10:12.397222+0000","last_became_active":"2023-11-28T11:10:10.420942+0000","last_became_peered":"2023-11-28T11:10:10.420942+0000","last_unstale":"2023-11-28T11:10:12.397222+0000","last_undegraded":"2023-11-28T11:10:12.397222+0000","last_fullsized":"2023-11-28T11:10:12.397222+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:27:02.330969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.397214+0000","last_change":"2023-11-28T11:10:10.847219+0000","last_active":"2023-11-28T11:10:12.397214+0000","last_peered":"2023-11-28T11:10:12.397214+0000","last_clean":"2023-11-28T11:10:12.397214+0000","last_became_active":"2023-11-28T11:10:10.846833+0000","last_became_peered":"2023-11-28T11:10:10.846833+0000","last_unstale":"2023-11-28T11:10:12.397214+0000","last_undegraded":"2023-11-28T11:10:12.397214+0000","last_fullsized":"2023-11-28T11:10:12.397214+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:22.625890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.322651+0000","last_change":"2023-11-28T11:10:10.428661+0000","last_active":"2023-11-28T11:10:12.322651+0000","last_peered":"2023-11-28T11:10:12.322651+0000","last_clean":"2023-11-28T11:10:12.322651+0000","last_became_active":"2023-11-28T11:10:10.428330+0000","last_became_peered":"2023-11-28T11:10:10.428330+0000","last_unstale":"2023-11-28T11:10:12.322651+0000","last_undegraded":"2023-11-28T11:10:12.322651+0000","last_fullsized":"2023-11-28T11:10:12.322651+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:12.811358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.382929+0000","last_change":"2023-11-28T11:10:10.848017+0000","last_active":"2023-11-28T11:10:12.382929+0000","last_peered":"2023-11-28T11:10:12.382929+0000","last_clean":"2023-11-28T11:10:12.382929+0000","last_became_active":"2023-11-28T11:10:10.847776+0000","last_became_peered":"2023-11-28T11:10:10.847776+0000","last_unstale":"2023-11-28T11:10:12.382929+0000","last_undegraded":"2023-11-28T11:10:12.382929+0000","last_fullsized":"2023-11-28T11:10:12.382929+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:26:05.083157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.233461+0000","last_change":"2023-11-28T11:10:10.424873+0000","last_active":"2023-11-28T11:10:12.233461+0000","last_peered":"2023-11-28T11:10:12.233461+0000","last_clean":"2023-11-28T11:10:12.233461+0000","last_became_active":"2023-11-28T11:10:10.424575+0000","last_became_peered":"2023-11-28T11:10:10.424575+0000","last_unstale":"2023-11-28T11:10:12.233461+0000","last_undegraded":"2023-11-28T11:10:12.233461+0000","last_fullsized":"2023-11-28T11:10:12.233461+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:50:32.329288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948339+0000","last_change":"2023-11-28T11:10:10.841926+0000","last_active":"2023-11-28T11:10:11.948339+0000","last_peered":"2023-11-28T11:10:11.948339+0000","last_clean":"2023-11-28T11:10:11.948339+0000","last_became_active":"2023-11-28T11:10:10.841613+0000","last_became_peered":"2023-11-28T11:10:10.841613+0000","last_unstale":"2023-11-28T11:10:11.948339+0000","last_undegraded":"2023-11-28T11:10:11.948339+0000","last_fullsized":"2023-11-28T11:10:11.948339+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:24:12.960072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.383073+0000","last_change":"2023-11-28T11:10:10.421719+0000","last_active":"2023-11-28T11:10:12.383073+0000","last_peered":"2023-11-28T11:10:12.383073+0000","last_clean":"2023-11-28T11:10:12.383073+0000","last_became_active":"2023-11-28T11:10:10.421431+0000","last_became_peered":"2023-11-28T11:10:10.421431+0000","last_unstale":"2023-11-28T11:10:12.383073+0000","last_undegraded":"2023-11-28T11:10:12.383073+0000","last_fullsized":"2023-11-28T11:10:12.383073+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:13.484601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948139+0000","last_change":"2023-11-28T11:10:10.854178+0000","last_active":"2023-11-28T11:10:11.948139+0000","last_peered":"2023-11-28T11:10:11.948139+0000","last_clean":"2023-11-28T11:10:11.948139+0000","last_became_active":"2023-11-28T11:10:10.849218+0000","last_became_peered":"2023-11-28T11:10:10.849218+0000","last_unstale":"2023-11-28T11:10:11.948139+0000","last_undegraded":"2023-11-28T11:10:11.948139+0000","last_fullsized":"2023-11-28T11:10:11.948139+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:59.832830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.830519+0000","last_change":"2023-11-28T11:10:10.424988+0000","last_active":"2023-11-28T11:10:11.830519+0000","last_peered":"2023-11-28T11:10:11.830519+0000","last_clean":"2023-11-28T11:10:11.830519+0000","last_became_active":"2023-11-28T11:10:10.424692+0000","last_became_peered":"2023-11-28T11:10:10.424692+0000","last_unstale":"2023-11-28T11:10:11.830519+0000","last_undegraded":"2023-11-28T11:10:11.830519+0000","last_fullsized":"2023-11-28T11:10:11.830519+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:17:58.349463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.383748+0000","last_change":"2023-11-28T11:10:10.853176+0000","last_active":"2023-11-28T11:10:12.383748+0000","last_peered":"2023-11-28T11:10:12.383748+0000","last_clean":"2023-11-28T11:10:12.383748+0000","last_became_active":"2023-11-28T11:10:10.852896+0000","last_became_peered":"2023-11-28T11:10:10.852896+0000","last_unstale":"2023-11-28T11:10:12.383748+0000","last_undegraded":"2023-11-28T11:10:12.383748+0000","last_fullsized":"2023-11-28T11:10:12.383748+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:22.625890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948234+0000","last_change":"2023-11-28T11:10:10.420224+0000","last_active":"2023-11-28T11:10:11.948234+0000","last_peered":"2023-11-28T11:10:11.948234+0000","last_clean":"2023-11-28T11:10:11.948234+0000","last_became_active":"2023-11-28T11:10:10.419830+0000","last_became_peered":"2023-11-28T11:10:10.419830+0000","last_unstale":"2023-11-28T11:10:11.948234+0000","last_undegraded":"2023-11-28T11:10:11.948234+0000","last_fullsized":"2023-11-28T11:10:11.948234+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:32.377125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.383227+0000","last_change":"2023-11-28T11:10:10.852787+0000","last_active":"2023-11-28T11:10:12.383227+0000","last_peered":"2023-11-28T11:10:12.383227+0000","last_clean":"2023-11-28T11:10:12.383227+0000","last_became_active":"2023-11-28T11:10:10.852526+0000","last_became_peered":"2023-11-28T11:10:10.852526+0000","last_unstale":"2023-11-28T11:10:12.383227+0000","last_undegraded":"2023-11-28T11:10:12.383227+0000","last_fullsized":"2023-11-28T11:10:12.383227+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:43:52.436092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.383328+0000","last_change":"2023-11-28T11:10:10.419964+0000","last_active":"2023-11-28T11:10:12.383328+0000","last_peered":"2023-11-28T11:10:12.383328+0000","last_clean":"2023-11-28T11:10:12.383328+0000","last_became_active":"2023-11-28T11:10:10.419607+0000","last_became_peered":"2023-11-28T11:10:10.419607+0000","last_unstale":"2023-11-28T11:10:12.383328+0000","last_undegraded":"2023-11-28T11:10:12.383328+0000","last_fullsized":"2023-11-28T11:10:12.383328+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:51:21.571062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.322905+0000","last_change":"2023-11-28T11:10:10.851805+0000","last_active":"2023-11-28T11:10:12.322905+0000","last_peered":"2023-11-28T11:10:12.322905+0000","last_clean":"2023-11-28T11:10:12.322905+0000","last_became_active":"2023-11-28T11:10:10.851511+0000","last_became_peered":"2023-11-28T11:10:10.851511+0000","last_unstale":"2023-11-28T11:10:12.322905+0000","last_undegraded":"2023-11-28T11:10:12.322905+0000","last_fullsized":"2023-11-28T11:10:12.322905+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:08.823912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948524+0000","last_change":"2023-11-28T11:10:10.423687+0000","last_active":"2023-11-28T11:10:11.948524+0000","last_peered":"2023-11-28T11:10:11.948524+0000","last_clean":"2023-11-28T11:10:11.948524+0000","last_became_active":"2023-11-28T11:10:10.423416+0000","last_became_peered":"2023-11-28T11:10:10.423416+0000","last_unstale":"2023-11-28T11:10:11.948524+0000","last_undegraded":"2023-11-28T11:10:11.948524+0000","last_fullsized":"2023-11-28T11:10:11.948524+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:11:23.665801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.233516+0000","last_change":"2023-11-28T11:10:10.836300+0000","last_active":"2023-11-28T11:10:12.233516+0000","last_peered":"2023-11-28T11:10:12.233516+0000","last_clean":"2023-11-28T11:10:12.233516+0000","last_became_active":"2023-11-28T11:10:10.836056+0000","last_became_peered":"2023-11-28T11:10:10.836056+0000","last_unstale":"2023-11-28T11:10:12.233516+0000","last_undegraded":"2023-11-28T11:10:12.233516+0000","last_fullsized":"2023-11-28T11:10:12.233516+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:43:52.436092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"39'3","reported_seq":14,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.322822+0000","last_change":"2023-11-28T11:10:10.423362+0000","last_active":"2023-11-28T11:10:12.322822+0000","last_peered":"2023-11-28T11:10:12.322822+0000","last_clean":"2023-11-28T11:10:12.322822+0000","last_became_active":"2023-11-28T11:10:10.423067+0000","last_became_peered":"2023-11-28T11:10:10.423067+0000","last_unstale":"2023-11-28T11:10:12.322822+0000","last_undegraded":"2023-11-28T11:10:12.322822+0000","last_fullsized":"2023-11-28T11:10:12.322822+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:21.618790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.322700+0000","last_change":"2023-11-28T11:10:10.845283+0000","last_active":"2023-11-28T11:10:12.322700+0000","last_peered":"2023-11-28T11:10:12.322700+0000","last_clean":"2023-11-28T11:10:12.322700+0000","last_became_active":"2023-11-28T11:10:10.845014+0000","last_became_peered":"2023-11-28T11:10:10.845014+0000","last_unstale":"2023-11-28T11:10:12.322700+0000","last_undegraded":"2023-11-28T11:10:12.322700+0000","last_fullsized":"2023-11-28T11:10:12.322700+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:24:12.960072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"39'6","reported_seq":17,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.829207+0000","last_change":"2023-11-28T11:10:10.416466+0000","last_active":"2023-11-28T11:10:11.829207+0000","last_peered":"2023-11-28T11:10:11.829207+0000","last_clean":"2023-11-28T11:10:11.829207+0000","last_became_active":"2023-11-28T11:10:10.416154+0000","last_became_peered":"2023-11-28T11:10:10.416154+0000","last_unstale":"2023-11-28T11:10:11.829207+0000","last_undegraded":"2023-11-28T11:10:11.829207+0000","last_fullsized":"2023-11-28T11:10:11.829207+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:27:58.413745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.383906+0000","last_change":"2023-11-28T11:10:10.851929+0000","last_active":"2023-11-28T11:10:12.383906+0000","last_peered":"2023-11-28T11:10:12.383906+0000","last_clean":"2023-11-28T11:10:12.383906+0000","last_became_active":"2023-11-28T11:10:10.851448+0000","last_became_peered":"2023-11-28T11:10:10.851448+0000","last_unstale":"2023-11-28T11:10:12.383906+0000","last_undegraded":"2023-11-28T11:10:12.383906+0000","last_fullsized":"2023-11-28T11:10:12.383906+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:50:33.336389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.233568+0000","last_change":"2023-11-28T11:10:10.425339+0000","last_active":"2023-11-28T11:10:12.233568+0000","last_peered":"2023-11-28T11:10:12.233568+0000","last_clean":"2023-11-28T11:10:12.233568+0000","last_became_active":"2023-11-28T11:10:10.425027+0000","last_became_peered":"2023-11-28T11:10:10.425027+0000","last_unstale":"2023-11-28T11:10:12.233568+0000","last_undegraded":"2023-11-28T11:10:12.233568+0000","last_fullsized":"2023-11-28T11:10:12.233568+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:49:23.775707+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948265+0000","last_change":"2023-11-28T11:10:10.846016+0000","last_active":"2023-11-28T11:10:11.948265+0000","last_peered":"2023-11-28T11:10:11.948265+0000","last_clean":"2023-11-28T11:10:11.948265+0000","last_became_active":"2023-11-28T11:10:10.845738+0000","last_became_peered":"2023-11-28T11:10:10.845738+0000","last_unstale":"2023-11-28T11:10:11.948265+0000","last_undegraded":"2023-11-28T11:10:11.948265+0000","last_fullsized":"2023-11-28T11:10:11.948265+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:12.144201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.322569+0000","last_change":"2023-11-28T11:10:10.421585+0000","last_active":"2023-11-28T11:10:12.322569+0000","last_peered":"2023-11-28T11:10:12.322569+0000","last_clean":"2023-11-28T11:10:12.322569+0000","last_became_active":"2023-11-28T11:10:10.421295+0000","last_became_peered":"2023-11-28T11:10:10.421295+0000","last_unstale":"2023-11-28T11:10:12.322569+0000","last_undegraded":"2023-11-28T11:10:12.322569+0000","last_fullsized":"2023-11-28T11:10:12.322569+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:23.078114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.233253+0000","last_change":"2023-11-28T11:10:10.835140+0000","last_active":"2023-11-28T11:10:12.233253+0000","last_peered":"2023-11-28T11:10:12.233253+0000","last_clean":"2023-11-28T11:10:12.233253+0000","last_became_active":"2023-11-28T11:10:10.834926+0000","last_became_peered":"2023-11-28T11:10:10.834926+0000","last_unstale":"2023-11-28T11:10:12.233253+0000","last_undegraded":"2023-11-28T11:10:12.233253+0000","last_fullsized":"2023-11-28T11:10:12.233253+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:09:12.144201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.233426+0000","last_change":"2023-11-28T11:10:10.420650+0000","last_active":"2023-11-28T11:10:12.233426+0000","last_peered":"2023-11-28T11:10:12.233426+0000","last_clean":"2023-11-28T11:10:12.233426+0000","last_became_active":"2023-11-28T11:10:10.420375+0000","last_became_peered":"2023-11-28T11:10:10.420375+0000","last_unstale":"2023-11-28T11:10:12.233426+0000","last_undegraded":"2023-11-28T11:10:12.233426+0000","last_fullsized":"2023-11-28T11:10:12.233426+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:50.604726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.322533+0000","last_change":"2023-11-28T11:10:10.846643+0000","last_active":"2023-11-28T11:10:12.322533+0000","last_peered":"2023-11-28T11:10:12.322533+0000","last_clean":"2023-11-28T11:10:12.322533+0000","last_became_active":"2023-11-28T11:10:10.846352+0000","last_became_peered":"2023-11-28T11:10:10.846352+0000","last_unstale":"2023-11-28T11:10:12.322533+0000","last_undegraded":"2023-11-28T11:10:12.322533+0000","last_fullsized":"2023-11-28T11:10:12.322533+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:05:21.229712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"39'8","reported_seq":19,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948339+0000","last_change":"2023-11-28T11:10:10.428637+0000","last_active":"2023-11-28T11:10:11.948339+0000","last_peered":"2023-11-28T11:10:11.948339+0000","last_clean":"2023-11-28T11:10:11.948339+0000","last_became_active":"2023-11-28T11:10:10.428324+0000","last_became_peered":"2023-11-28T11:10:10.428324+0000","last_unstale":"2023-11-28T11:10:11.948339+0000","last_undegraded":"2023-11-28T11:10:11.948339+0000","last_fullsized":"2023-11-28T11:10:11.948339+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:27:02.330969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.829153+0000","last_change":"2023-11-28T11:10:10.850274+0000","last_active":"2023-11-28T11:10:11.829153+0000","last_peered":"2023-11-28T11:10:11.829153+0000","last_clean":"2023-11-28T11:10:11.829153+0000","last_became_active":"2023-11-28T11:10:10.849983+0000","last_became_peered":"2023-11-28T11:10:10.849983+0000","last_unstale":"2023-11-28T11:10:11.829153+0000","last_undegraded":"2023-11-28T11:10:11.829153+0000","last_fullsized":"2023-11-28T11:10:11.829153+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:11:24.672901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.829242+0000","last_change":"2023-11-28T11:10:10.421394+0000","last_active":"2023-11-28T11:10:11.829242+0000","last_peered":"2023-11-28T11:10:11.829242+0000","last_clean":"2023-11-28T11:10:11.829242+0000","last_became_active":"2023-11-28T11:10:10.421015+0000","last_became_peered":"2023-11-28T11:10:10.421015+0000","last_unstale":"2023-11-28T11:10:11.829242+0000","last_undegraded":"2023-11-28T11:10:11.829242+0000","last_fullsized":"2023-11-28T11:10:11.829242+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:24:08.043544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948508+0000","last_change":"2023-11-28T11:10:10.846237+0000","last_active":"2023-11-28T11:10:11.948508+0000","last_peered":"2023-11-28T11:10:11.948508+0000","last_clean":"2023-11-28T11:10:11.948508+0000","last_became_active":"2023-11-28T11:10:10.845979+0000","last_became_peered":"2023-11-28T11:10:10.845979+0000","last_unstale":"2023-11-28T11:10:11.948508+0000","last_undegraded":"2023-11-28T11:10:11.948508+0000","last_fullsized":"2023-11-28T11:10:11.948508+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:04.163924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.383109+0000","last_change":"2023-11-28T11:10:10.415986+0000","last_active":"2023-11-28T11:10:12.383109+0000","last_peered":"2023-11-28T11:10:12.383109+0000","last_clean":"2023-11-28T11:10:12.383109+0000","last_became_active":"2023-11-28T11:10:10.415696+0000","last_became_peered":"2023-11-28T11:10:10.415696+0000","last_unstale":"2023-11-28T11:10:12.383109+0000","last_undegraded":"2023-11-28T11:10:12.383109+0000","last_fullsized":"2023-11-28T11:10:12.383109+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:12.140085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.384043+0000","last_change":"2023-11-28T11:10:10.848816+0000","last_active":"2023-11-28T11:10:12.384043+0000","last_peered":"2023-11-28T11:10:12.384043+0000","last_clean":"2023-11-28T11:10:12.384043+0000","last_became_active":"2023-11-28T11:10:10.848494+0000","last_became_peered":"2023-11-28T11:10:10.848494+0000","last_unstale":"2023-11-28T11:10:12.384043+0000","last_undegraded":"2023-11-28T11:10:12.384043+0000","last_fullsized":"2023-11-28T11:10:12.384043+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:49:24.782807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"39'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.830752+0000","last_change":"2023-11-28T11:10:10.425030+0000","last_active":"2023-11-28T11:10:11.830752+0000","last_peered":"2023-11-28T11:10:11.830752+0000","last_clean":"2023-11-28T11:10:11.830752+0000","last_became_active":"2023-11-28T11:10:10.424682+0000","last_became_peered":"2023-11-28T11:10:10.424682+0000","last_unstale":"2023-11-28T11:10:11.830752+0000","last_undegraded":"2023-11-28T11:10:11.830752+0000","last_fullsized":"2023-11-28T11:10:11.830752+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:12.811358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.384231+0000","last_change":"2023-11-28T11:10:10.846757+0000","last_active":"2023-11-28T11:10:12.384231+0000","last_peered":"2023-11-28T11:10:12.384231+0000","last_clean":"2023-11-28T11:10:12.384231+0000","last_became_active":"2023-11-28T11:10:10.846462+0000","last_became_peered":"2023-11-28T11:10:10.846462+0000","last_unstale":"2023-11-28T11:10:12.384231+0000","last_undegraded":"2023-11-28T11:10:12.384231+0000","last_fullsized":"2023-11-28T11:10:12.384231+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:33:03.152743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.233628+0000","last_change":"2023-11-28T11:10:10.423823+0000","last_active":"2023-11-28T11:10:12.233628+0000","last_peered":"2023-11-28T11:10:12.233628+0000","last_clean":"2023-11-28T11:10:12.233628+0000","last_became_active":"2023-11-28T11:10:10.423610+0000","last_became_peered":"2023-11-28T11:10:10.423610+0000","last_unstale":"2023-11-28T11:10:12.233628+0000","last_undegraded":"2023-11-28T11:10:12.233628+0000","last_fullsized":"2023-11-28T11:10:12.233628+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:58:05.087320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.233345+0000","last_change":"2023-11-28T11:10:10.835152+0000","last_active":"2023-11-28T11:10:12.233345+0000","last_peered":"2023-11-28T11:10:12.233345+0000","last_clean":"2023-11-28T11:10:12.233345+0000","last_became_active":"2023-11-28T11:10:10.834867+0000","last_became_peered":"2023-11-28T11:10:10.834867+0000","last_unstale":"2023-11-28T11:10:12.233345+0000","last_undegraded":"2023-11-28T11:10:12.233345+0000","last_fullsized":"2023-11-28T11:10:12.233345+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:46.488470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.233655+0000","last_change":"2023-11-28T11:10:10.424775+0000","last_active":"2023-11-28T11:10:12.233655+0000","last_peered":"2023-11-28T11:10:12.233655+0000","last_clean":"2023-11-28T11:10:12.233655+0000","last_became_active":"2023-11-28T11:10:10.424500+0000","last_became_peered":"2023-11-28T11:10:10.424500+0000","last_unstale":"2023-11-28T11:10:12.233655+0000","last_undegraded":"2023-11-28T11:10:12.233655+0000","last_fullsized":"2023-11-28T11:10:12.233655+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:53.614405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.396994+0000","last_change":"2023-11-28T11:10:10.853830+0000","last_active":"2023-11-28T11:10:12.396994+0000","last_peered":"2023-11-28T11:10:12.396994+0000","last_clean":"2023-11-28T11:10:12.396994+0000","last_became_active":"2023-11-28T11:10:10.853492+0000","last_became_peered":"2023-11-28T11:10:10.853492+0000","last_unstale":"2023-11-28T11:10:12.396994+0000","last_undegraded":"2023-11-28T11:10:12.396994+0000","last_fullsized":"2023-11-28T11:10:12.396994+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:54.621505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.830737+0000","last_change":"2023-11-28T11:10:10.422785+0000","last_active":"2023-11-28T11:10:11.830737+0000","last_peered":"2023-11-28T11:10:11.830737+0000","last_clean":"2023-11-28T11:10:11.830737+0000","last_became_active":"2023-11-28T11:10:10.422518+0000","last_became_peered":"2023-11-28T11:10:10.422518+0000","last_unstale":"2023-11-28T11:10:11.830737+0000","last_undegraded":"2023-11-28T11:10:11.830737+0000","last_fullsized":"2023-11-28T11:10:11.830737+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:33:02.145643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.828848+0000","last_change":"2023-11-28T11:10:10.845912+0000","last_active":"2023-11-28T11:10:11.828848+0000","last_peered":"2023-11-28T11:10:11.828848+0000","last_clean":"2023-11-28T11:10:11.828848+0000","last_became_active":"2023-11-28T11:10:10.845598+0000","last_became_peered":"2023-11-28T11:10:10.845598+0000","last_unstale":"2023-11-28T11:10:11.828848+0000","last_undegraded":"2023-11-28T11:10:11.828848+0000","last_fullsized":"2023-11-28T11:10:11.828848+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:44:53.857024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948091+0000","last_change":"2023-11-28T11:10:10.425934+0000","last_active":"2023-11-28T11:10:11.948091+0000","last_peered":"2023-11-28T11:10:11.948091+0000","last_clean":"2023-11-28T11:10:11.948091+0000","last_became_active":"2023-11-28T11:10:10.425676+0000","last_became_peered":"2023-11-28T11:10:10.425676+0000","last_unstale":"2023-11-28T11:10:11.948091+0000","last_undegraded":"2023-11-28T11:10:11.948091+0000","last_fullsized":"2023-11-28T11:10:11.948091+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:40:12.811358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948201+0000","last_change":"2023-11-28T11:10:10.848940+0000","last_active":"2023-11-28T11:10:11.948201+0000","last_peered":"2023-11-28T11:10:11.948201+0000","last_clean":"2023-11-28T11:10:11.948201+0000","last_became_active":"2023-11-28T11:10:10.848646+0000","last_became_peered":"2023-11-28T11:10:10.848646+0000","last_unstale":"2023-11-28T11:10:11.948201+0000","last_undegraded":"2023-11-28T11:10:11.948201+0000","last_fullsized":"2023-11-28T11:10:11.948201+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:13:46.488470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.322227+0000","last_change":"2023-11-28T11:10:10.421628+0000","last_active":"2023-11-28T11:10:12.322227+0000","last_peered":"2023-11-28T11:10:12.322227+0000","last_clean":"2023-11-28T11:10:12.322227+0000","last_became_active":"2023-11-28T11:10:10.421357+0000","last_became_peered":"2023-11-28T11:10:10.421357+0000","last_unstale":"2023-11-28T11:10:12.322227+0000","last_undegraded":"2023-11-28T11:10:12.322227+0000","last_fullsized":"2023-11-28T11:10:12.322227+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:17:58.349463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.233545+0000","last_change":"2023-11-28T11:10:10.832351+0000","last_active":"2023-11-28T11:10:12.233545+0000","last_peered":"2023-11-28T11:10:12.233545+0000","last_clean":"2023-11-28T11:10:12.233545+0000","last_became_active":"2023-11-28T11:10:10.832118+0000","last_became_peered":"2023-11-28T11:10:10.832118+0000","last_unstale":"2023-11-28T11:10:12.233545+0000","last_undegraded":"2023-11-28T11:10:12.233545+0000","last_fullsized":"2023-11-28T11:10:12.233545+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:58.330510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.382992+0000","last_change":"2023-11-28T11:10:10.420043+0000","last_active":"2023-11-28T11:10:12.382992+0000","last_peered":"2023-11-28T11:10:12.382992+0000","last_clean":"2023-11-28T11:10:12.382992+0000","last_became_active":"2023-11-28T11:10:10.419763+0000","last_became_peered":"2023-11-28T11:10:10.419763+0000","last_unstale":"2023-11-28T11:10:12.382992+0000","last_undegraded":"2023-11-28T11:10:12.382992+0000","last_fullsized":"2023-11-28T11:10:12.382992+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:26:02.506332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":28,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:10:10.830821+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:10:10.830821+0000","last_peered":"2023-11-28T11:10:10.830821+0000","last_clean":"2023-11-28T11:10:10.830821+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:10:10.830821+0000","last_undegraded":"2023-11-28T11:10:10.830821+0000","last_fullsized":"2023-11-28T11:10:10.830821+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.233471+0000","last_change":"2023-11-28T11:10:10.832854+0000","last_active":"2023-11-28T11:10:12.233471+0000","last_peered":"2023-11-28T11:10:12.233471+0000","last_clean":"2023-11-28T11:10:12.233471+0000","last_became_active":"2023-11-28T11:10:10.832693+0000","last_became_peered":"2023-11-28T11:10:10.832693+0000","last_unstale":"2023-11-28T11:10:12.233471+0000","last_undegraded":"2023-11-28T11:10:12.233471+0000","last_fullsized":"2023-11-28T11:10:12.233471+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:29:40.483759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.322686+0000","last_change":"2023-11-28T11:10:10.423893+0000","last_active":"2023-11-28T11:10:12.322686+0000","last_peered":"2023-11-28T11:10:12.322686+0000","last_clean":"2023-11-28T11:10:12.322686+0000","last_became_active":"2023-11-28T11:10:10.423610+0000","last_became_peered":"2023-11-28T11:10:10.423610+0000","last_unstale":"2023-11-28T11:10:12.322686+0000","last_undegraded":"2023-11-28T11:10:12.322686+0000","last_fullsized":"2023-11-28T11:10:12.322686+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:33:02.145643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":30,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:10:10.825020+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:10:10.825020+0000","last_peered":"2023-11-28T11:10:10.825020+0000","last_clean":"2023-11-28T11:10:10.825020+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:10:10.825020+0000","last_undegraded":"2023-11-28T11:10:10.825020+0000","last_fullsized":"2023-11-28T11:10:10.825020+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.322732+0000","last_change":"2023-11-28T11:10:10.844771+0000","last_active":"2023-11-28T11:10:12.322732+0000","last_peered":"2023-11-28T11:10:12.322732+0000","last_clean":"2023-11-28T11:10:12.322732+0000","last_became_active":"2023-11-28T11:10:10.844495+0000","last_became_peered":"2023-11-28T11:10:10.844495+0000","last_unstale":"2023-11-28T11:10:12.322732+0000","last_undegraded":"2023-11-28T11:10:12.322732+0000","last_fullsized":"2023-11-28T11:10:12.322732+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:26:03.513433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.396994+0000","last_change":"2023-11-28T11:10:10.425829+0000","last_active":"2023-11-28T11:10:12.396994+0000","last_peered":"2023-11-28T11:10:12.396994+0000","last_clean":"2023-11-28T11:10:12.396994+0000","last_became_active":"2023-11-28T11:10:10.425532+0000","last_became_peered":"2023-11-28T11:10:10.425532+0000","last_unstale":"2023-11-28T11:10:12.396994+0000","last_undegraded":"2023-11-28T11:10:12.396994+0000","last_fullsized":"2023-11-28T11:10:12.396994+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:44:49.223808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":31,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:10:10.828685+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:10:10.828685+0000","last_peered":"2023-11-28T11:10:10.828685+0000","last_clean":"2023-11-28T11:10:10.828685+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:10:10.828685+0000","last_undegraded":"2023-11-28T11:10:10.828685+0000","last_fullsized":"2023-11-28T11:10:10.828685+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":180,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:10:10.835892+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:10:10.835892+0000","last_peered":"2023-11-28T11:10:10.835892+0000","last_clean":"2023-11-28T11:10:10.835892+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:10:10.835892+0000","last_undegraded":"2023-11-28T11:10:10.835892+0000","last_fullsized":"2023-11-28T11:10:10.835892+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948441+0000","last_change":"2023-11-28T11:10:10.848896+0000","last_active":"2023-11-28T11:10:11.948441+0000","last_peered":"2023-11-28T11:10:11.948441+0000","last_clean":"2023-11-28T11:10:11.948441+0000","last_became_active":"2023-11-28T11:10:10.848607+0000","last_became_peered":"2023-11-28T11:10:10.848607+0000","last_unstale":"2023-11-28T11:10:11.948441+0000","last_undegraded":"2023-11-28T11:10:11.948441+0000","last_fullsized":"2023-11-28T11:10:11.948441+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:07:58.330510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.322821+0000","last_change":"2023-11-28T11:10:10.423390+0000","last_active":"2023-11-28T11:10:12.322821+0000","last_peered":"2023-11-28T11:10:12.322821+0000","last_clean":"2023-11-28T11:10:12.322821+0000","last_became_active":"2023-11-28T11:10:10.423107+0000","last_became_peered":"2023-11-28T11:10:10.423107+0000","last_unstale":"2023-11-28T11:10:12.322821+0000","last_undegraded":"2023-11-28T11:10:12.322821+0000","last_fullsized":"2023-11-28T11:10:12.322821+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:50:50.604726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":30,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:10:10.837035+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:10:10.837035+0000","last_peered":"2023-11-28T11:10:10.837035+0000","last_clean":"2023-11-28T11:10:10.837035+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:10:10.837035+0000","last_undegraded":"2023-11-28T11:10:10.837035+0000","last_fullsized":"2023-11-28T11:10:10.837035+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948489+0000","last_change":"2023-11-28T11:10:10.851921+0000","last_active":"2023-11-28T11:10:11.948489+0000","last_peered":"2023-11-28T11:10:11.948489+0000","last_clean":"2023-11-28T11:10:11.948489+0000","last_became_active":"2023-11-28T11:10:10.851334+0000","last_became_peered":"2023-11-28T11:10:10.851334+0000","last_unstale":"2023-11-28T11:10:11.948489+0000","last_undegraded":"2023-11-28T11:10:11.948489+0000","last_fullsized":"2023-11-28T11:10:11.948489+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:47:47.621814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"39'2","reported_seq":13,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.831054+0000","last_change":"2023-11-28T11:10:10.422283+0000","last_active":"2023-11-28T11:10:11.831054+0000","last_peered":"2023-11-28T11:10:11.831054+0000","last_clean":"2023-11-28T11:10:11.831054+0000","last_became_active":"2023-11-28T11:10:10.422014+0000","last_became_peered":"2023-11-28T11:10:10.422014+0000","last_unstale":"2023-11-28T11:10:11.831054+0000","last_undegraded":"2023-11-28T11:10:11.831054+0000","last_fullsized":"2023-11-28T11:10:11.831054+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:37:32.377125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":30,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:10:10.835335+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:10:10.835335+0000","last_peered":"2023-11-28T11:10:10.835335+0000","last_clean":"2023-11-28T11:10:10.835335+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:10:10.835335+0000","last_undegraded":"2023-11-28T11:10:10.835335+0000","last_fullsized":"2023-11-28T11:10:10.835335+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.384186+0000","last_change":"2023-11-28T11:10:10.852055+0000","last_active":"2023-11-28T11:10:12.384186+0000","last_peered":"2023-11-28T11:10:12.384186+0000","last_clean":"2023-11-28T11:10:12.384186+0000","last_became_active":"2023-11-28T11:10:10.851746+0000","last_became_peered":"2023-11-28T11:10:10.851746+0000","last_unstale":"2023-11-28T11:10:12.384186+0000","last_undegraded":"2023-11-28T11:10:12.384186+0000","last_fullsized":"2023-11-28T11:10:12.384186+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:28:54.621505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"40'1","reported_seq":12,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.564193+0000","last_change":"2023-11-28T11:10:10.423446+0000","last_active":"2023-11-28T11:10:12.564193+0000","last_peered":"2023-11-28T11:10:12.564193+0000","last_clean":"2023-11-28T11:10:12.564193+0000","last_became_active":"2023-11-28T11:10:10.423138+0000","last_became_peered":"2023-11-28T11:10:10.423138+0000","last_unstale":"2023-11-28T11:10:12.564193+0000","last_undegraded":"2023-11-28T11:10:12.564193+0000","last_fullsized":"2023-11-28T11:10:12.564193+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:55:38.925000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":32,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:10:10.838335+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:10:10.838335+0000","last_peered":"2023-11-28T11:10:10.838335+0000","last_clean":"2023-11-28T11:10:10.838335+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:10:10.838335+0000","last_undegraded":"2023-11-28T11:10:10.838335+0000","last_fullsized":"2023-11-28T11:10:10.838335+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.830710+0000","last_change":"2023-11-28T11:10:10.850410+0000","last_active":"2023-11-28T11:10:11.830710+0000","last_peered":"2023-11-28T11:10:11.830710+0000","last_clean":"2023-11-28T11:10:11.830710+0000","last_became_active":"2023-11-28T11:10:10.850136+0000","last_became_peered":"2023-11-28T11:10:10.850136+0000","last_unstale":"2023-11-28T11:10:11.830710+0000","last_undegraded":"2023-11-28T11:10:11.830710+0000","last_fullsized":"2023-11-28T11:10:11.830710+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:14.491701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.396936+0000","last_change":"2023-11-28T11:10:10.415610+0000","last_active":"2023-11-28T11:10:12.396936+0000","last_peered":"2023-11-28T11:10:12.396936+0000","last_clean":"2023-11-28T11:10:12.396936+0000","last_became_active":"2023-11-28T11:10:10.415318+0000","last_became_peered":"2023-11-28T11:10:10.415318+0000","last_unstale":"2023-11-28T11:10:12.396936+0000","last_undegraded":"2023-11-28T11:10:12.396936+0000","last_fullsized":"2023-11-28T11:10:12.396936+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:52:02.253938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":28,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:10:10.836681+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:10:10.836681+0000","last_peered":"2023-11-28T11:10:10.836681+0000","last_clean":"2023-11-28T11:10:10.836681+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:10:10.836681+0000","last_undegraded":"2023-11-28T11:10:10.836681+0000","last_fullsized":"2023-11-28T11:10:10.836681+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:12.382864+0000","last_change":"2023-11-28T11:10:10.839727+0000","last_active":"2023-11-28T11:10:12.382864+0000","last_peered":"2023-11-28T11:10:12.382864+0000","last_clean":"2023-11-28T11:10:12.382864+0000","last_became_active":"2023-11-28T11:10:10.839472+0000","last_became_peered":"2023-11-28T11:10:10.839472+0000","last_unstale":"2023-11-28T11:10:12.382864+0000","last_undegraded":"2023-11-28T11:10:12.382864+0000","last_fullsized":"2023-11-28T11:10:12.382864+0000","mapping_epoch":38,"log_start":"0'0","ondisk_log_start":"0'0","created":38,"last_epoch_clean":39,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:10.399114+0000","last_clean_scrub_stamp":"2023-11-28T11:10:10.399114+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:29:40.483759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":11,"reported_epoch":40,"state":"active+clean","last_fresh":"2023-11-28T11:10:11.948433+0000","last_change":"2023-11-28T11:10:10.424191+0000","last_active":"2023-11-28T11:10:11.948433+0000","last_peered":"2023-11-28T11:10:11.948433+0000","last_clean":"2023-11-28T11:10:11.948433+0000","last_became_active":"2023-11-28T11:10:10.423032+0000","last_became_peered":"2023-11-28T11:10:10.423032+0000","last_unstale":"2023-11-28T11:10:11.948433+0000","last_undegraded":"2023-11-28T11:10:11.948433+0000","last_fullsized":"2023-11-28T11:10:11.948433+0000","mapping_epoch":37,"log_start":"0'0","ondisk_log_start":"0'0","created":37,"last_epoch_clean":38,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:09.392014+0000","last_clean_scrub_stamp":"2023-11-28T11:10:09.392014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:52:02.253938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":28,"reported_epoch":39,"state":"active+clean","last_fresh":"2023-11-28T11:10:10.843875+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:10:10.843875+0000","last_peered":"2023-11-28T11:10:10.843875+0000","last_clean":"2023-11-28T11:10:10.843875+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:10:10.843875+0000","last_undegraded":"2023-11-28T11:10:10.843875+0000","last_fullsized":"2023-11-28T11:10:10.843875+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9748,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362092,"statfs":{"total":96636764160,"available":96626782208,"internally_reserved":0,"allocated":1060864,"data_stored":798734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":3.2189999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":3.2000000000000002,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.66600000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":3.1360000000000001,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.53400000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":3.3820000000000001,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":3.3969999999999998,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.56299999999999994}]},{"osd":3,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":3.3069999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":3.294,"5min":3.294,"15min":3.294},"last":0.60099999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.624},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.752}]},{"osd":6,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89700000000000002,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.64600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8596,"kb_used_data":460,"kb_used_omap":0,"kb_used_meta":8128,"kb_avail":94363244,"statfs":{"total":96636764160,"available":96627961856,"internally_reserved":0,"allocated":471040,"data_stored":208932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8323072},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.32400000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54700000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.497,"15min":0.497},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":3.7429999999999999,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.51800000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":3.7450000000000001,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":3.7749999999999999,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.77600000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.48199999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.83699999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"last":0.74399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9748,"kb_used_data":1036,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362092,"statfs":{"total":96636764160,"available":96626782208,"internally_reserved":0,"allocated":1060864,"data_stored":798714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.50900000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.315,"5min":0.315,"15min":0.315},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.58999999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.57399999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.88200000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.66200000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.3220000000000001,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.74099999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.67500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8720,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363120,"statfs":{"total":96636764160,"available":96627834880,"internally_reserved":0,"allocated":466944,"data_stored":208310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.5,"5min":0.5,"15min":0.5},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"last":0.311}]},{"osd":2,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"last":0.37},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.35999999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.371,"15min":0.371},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.83599999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39100000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.59399999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.70299999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.3759999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.58099999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":1.105,"5min":1.105,"15min":1.105},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.60299999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8536,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":8064,"kb_avail":94363304,"statfs":{"total":96636764160,"available":96628023296,"internally_reserved":0,"allocated":475136,"data_stored":208885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8257536},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.378,"15min":0.378},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.32100000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.528,"5min":1.528,"15min":1.528},"last":0.47299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.313},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.38200000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.439}]},{"osd":5,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.95499999999999996,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.42399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.496,"5min":0.496,"15min":0.496},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.49399999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.50700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574870,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8728,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363112,"statfs":{"total":96636764160,"available":96627826688,"internally_reserved":0,"allocated":475136,"data_stored":208898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":11.965,"5min":11.965,"15min":11.965},"last":0.33500000000000002},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":12.888999999999999,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.36399999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"min":{"1min":0.215,"5min":0.215,"15min":0.215},"max":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.316},{"interface":"front","average":{"1min":1.276,"5min":1.276,"15min":1.276},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":16.988,"5min":16.988,"15min":16.988},"last":0.29699999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":2.0609999999999999,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":1.895,"5min":1.895,"15min":1.895},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":27.887,"5min":27.887,"15min":27.887},"last":0.42399999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":1.8899999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.0019999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.55600000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.98,"5min":1.98,"15min":1.98},"last":0.61599999999999999},{"interface":"front","average":{"1min":1.254,"5min":1.254,"15min":1.254},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":11.933999999999999,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.70199999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":11.888,"5min":11.888,"15min":11.888},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.76000000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:09:38 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.9490000000000001,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":2.0310000000000001,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.64100000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8728,"kb_used_data":464,"kb_used_omap":0,"kb_used_meta":8256,"kb_avail":94363112,"statfs":{"total":96636764160,"available":96627826688,"internally_reserved":0,"allocated":475136,"data_stored":209824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8454144},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.53500000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.996,"5min":0.996,"15min":0.996},"last":0.438},{"interface":"front","average":{"1min":0.996,"5min":0.996,"15min":0.996},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":10.045999999999999,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.70099999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.60399999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.67000000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.77400000000000002,"15min":0.77400000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.68500000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.51400000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":1.145,"5min":1.145,"15min":1.145},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":1.085,"5min":1.085,"15min":1.085},"last":0.749}]},{"osd":7,"last update":"Tue Nov 28 11:09:39 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":1.1799999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.76000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8656,"kb_used_data":456,"kb_used_omap":0,"kb_used_meta":8192,"kb_avail":94363184,"statfs":{"total":96636764160,"available":96627900416,"internally_reserved":0,"allocated":466944,"data_stored":208346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8388608},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.47299999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":2.8719999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.8849999999999998,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.51400000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.628},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79200000000000004,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.65900000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":2.8999999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.70499999999999996}]},{"osd":4,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.55000000000000004},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.68899999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.036,"5min":1.036,"15min":1.036},"last":0.71699999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:09:40 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.70399999999999996},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":1.3839999999999999,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.754}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":622,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:10:15.349 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:10:15.349 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:10:15.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.512+0000 7f47b32d7700 1 Processor -- start 2023-11-28T11:10:15.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.512+0000 7f47b32d7700 1 -- start start 2023-11-28T11:10:15.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.513+0000 7f47b32d7700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f47ac0b1630 0x7f47ac0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.513+0000 7f47b32d7700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f47ac0b1a00 0x7f47ac0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.513+0000 7f47b32d7700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47ac0ce550 con 0x7f47ac0cb3e0 2023-11-28T11:10:15.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.513+0000 7f47b32d7700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47ac0ce830 con 0x7f47ac0b1a00 2023-11-28T11:10:15.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.513+0000 7f47b32d7700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47ac0ceb10 con 0x7f47ac0b1630 2023-11-28T11:10:15.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.513+0000 7f47b0872700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f47ac0b1a00 0x7f47ac0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.514+0000 7f47b1874700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f47ac0cb3e0 0x7f47ac160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37562/0 (socket says 172.21.15.28:37562) 2023-11-28T11:10:15.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.514+0000 7f47b1874700 1 -- 172.21.15.28:0/707139664 learned_addr learned my addr 172.21.15.28:0/707139664 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:15.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.514+0000 7f47b1073700 1 --2- 172.21.15.28:0/707139664 >> v2:172.21.15.59:3300/0 conn(0x7f47ac0b1630 0x7f47ac0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.514+0000 7f47b0872700 1 -- 172.21.15.28:0/707139664 >> v2:172.21.15.59:3300/0 conn(0x7f47ac0b1630 msgr2=0x7f47ac0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:15.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.514+0000 7f47b0872700 1 --2- 172.21.15.28:0/707139664 >> v2:172.21.15.59:3300/0 conn(0x7f47ac0b1630 0x7f47ac0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.514+0000 7f47b0872700 1 -- 172.21.15.28:0/707139664 >> v1:172.21.15.28:6789/0 conn(0x7f47ac0cb3e0 legacy=0x7f47ac160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.514+0000 7f47b0872700 1 -- 172.21.15.28:0/707139664 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47ac0cedf0 con 0x7f47ac0b1a00 2023-11-28T11:10:15.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.514+0000 7f47a3fff700 1 -- 172.21.15.28:0/707139664 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408376552 0 0) 0x7f47ac0ce550 con 0x7f47ac0cb3e0 2023-11-28T11:10:15.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.514+0000 7f47b0872700 1 --2- 172.21.15.28:0/707139664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47ac0b1a00 0x7f47ac0caea0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009840 tx=0x7f47a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba0f8dfbbe3a347 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:15.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.514+0000 7f47a3fff700 1 -- 172.21.15.28:0/707139664 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8011830 con 0x7f47ac0b1a00 2023-11-28T11:10:15.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.514+0000 7f47a3fff700 1 -- 172.21.15.28:0/707139664 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a8011990 con 0x7f47ac0b1a00 2023-11-28T11:10:15.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.514+0000 7f47a3fff700 1 -- 172.21.15.28:0/707139664 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a8011d20 con 0x7f47ac0b1a00 2023-11-28T11:10:15.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.515+0000 7f47b32d7700 1 -- 172.21.15.28:0/707139664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47ac0b1a00 msgr2=0x7f47ac0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.515+0000 7f47b32d7700 1 --2- 172.21.15.28:0/707139664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47ac0b1a00 0x7f47ac0caea0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f47a8009840 tx=0x7f47a8009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:15.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.515+0000 7f47b32d7700 1 -- 172.21.15.28:0/707139664 shutdown_connections 2023-11-28T11:10:15.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.515+0000 7f47b32d7700 1 --2- 172.21.15.28:0/707139664 >> v2:172.21.15.59:3300/0 conn(0x7f47ac0b1630 0x7f47ac0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.515+0000 7f47b32d7700 1 --2- 172.21.15.28:0/707139664 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47ac0b1a00 0x7f47ac0caea0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.515+0000 7f47b32d7700 1 -- 172.21.15.28:0/707139664 >> 172.21.15.28:0/707139664 conn(0x7f47ac0cc630 msgr2=0x7f47ac0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:15.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.515+0000 7f47b32d7700 1 -- 172.21.15.28:0/707139664 shutdown_connections 2023-11-28T11:10:15.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.515+0000 7f47b32d7700 1 -- 172.21.15.28:0/707139664 wait complete. 2023-11-28T11:10:15.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.516+0000 7f47b32d7700 1 Processor -- start 2023-11-28T11:10:15.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.516+0000 7f47b32d7700 1 -- start start 2023-11-28T11:10:15.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.517+0000 7f47b32d7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac0b1630 0x7f47ac15c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.517+0000 7f47b32d7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47ac0b1a00 0x7f47ac0b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.517+0000 7f47b32d7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47ac0cb3e0 0x7f47ac1e7470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.517+0000 7f47b32d7700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac15c6c0 con 0x7f47ac0cb3e0 2023-11-28T11:10:15.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.517+0000 7f47b32d7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac1f2310 con 0x7f47ac0b1630 2023-11-28T11:10:15.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.517+0000 7f47b32d7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47ac0b3c40 con 0x7f47ac0b1a00 2023-11-28T11:10:15.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.517+0000 7f47b1874700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47ac0cb3e0 0x7f47ac1e7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.517+0000 7f47b1874700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47ac0cb3e0 0x7f47ac1e7470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49510/0 (socket says 172.21.15.28:49510) 2023-11-28T11:10:15.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.517+0000 7f47b1874700 1 -- 172.21.15.28:0/4075928730 learned_addr learned my addr 172.21.15.28:0/4075928730 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:15.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.518+0000 7f47b1073700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac0b1630 0x7f47ac15c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.518+0000 7f47b0872700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47ac0b1a00 0x7f47ac0b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.518+0000 7f47b1874700 1 -- 172.21.15.28:0/4075928730 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47ac0b1a00 msgr2=0x7f47ac0b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.518+0000 7f47b1874700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47ac0b1a00 0x7f47ac0b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.518+0000 7f47b1874700 1 -- 172.21.15.28:0/4075928730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac0b1630 msgr2=0x7f47ac15c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.518+0000 7f47b1874700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac0b1630 0x7f47ac15c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.518+0000 7f47b1874700 1 -- 172.21.15.28:0/4075928730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47a80094a0 con 0x7f47ac0cb3e0 2023-11-28T11:10:15.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.518+0000 7f47b1073700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac0b1630 0x7f47ac15c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:15.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.518+0000 7f47b0872700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47ac0b1a00 0x7f47ac0b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:15.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.518+0000 7f47b1874700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47ac0cb3e0 0x7f47ac1e7470 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f47a4002230 tx=0x7f47a400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:15.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.518+0000 7f47a1ffb700 1 -- 172.21.15.28:0/4075928730 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a400ac20 con 0x7f47ac0cb3e0 2023-11-28T11:10:15.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.518+0000 7f47a1ffb700 1 -- 172.21.15.28:0/4075928730 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47a400ada0 con 0x7f47ac0cb3e0 2023-11-28T11:10:15.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.518+0000 7f47a1ffb700 1 -- 172.21.15.28:0/4075928730 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a4007310 con 0x7f47ac0cb3e0 2023-11-28T11:10:15.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.518+0000 7f47b32d7700 1 -- 172.21.15.28:0/4075928730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47ac0b1ea0 con 0x7f47ac0cb3e0 2023-11-28T11:10:15.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.518+0000 7f47b32d7700 1 -- 172.21.15.28:0/4075928730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47ac0cf0d0 con 0x7f47ac0cb3e0 2023-11-28T11:10:15.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.519+0000 7f47b32d7700 1 -- 172.21.15.28:0/4075928730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47ac0ce140 con 0x7f47ac0cb3e0 2023-11-28T11:10:15.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.520+0000 7f47a1ffb700 1 -- 172.21.15.28:0/4075928730 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f47a4010070 con 0x7f47ac0cb3e0 2023-11-28T11:10:15.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.520+0000 7f47a1ffb700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f47900682f0 0x7f479006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.520+0000 7f47a1ffb700 1 -- 172.21.15.28:0/4075928730 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f47a4085800 con 0x7f47ac0cb3e0 2023-11-28T11:10:15.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.520+0000 7f47b1073700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f47900682f0 0x7f479006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.520+0000 7f47b1073700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f47900682f0 0x7f479006a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f479c009ac0 tx=0x7f479c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:15.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.524+0000 7f47a1ffb700 1 -- 172.21.15.28:0/4075928730 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f47a4051f20 con 0x7f47ac0cb3e0 2023-11-28T11:10:15.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.796+0000 7f47b32d7700 1 -- 172.21.15.28:0/4075928730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7f47ac0739f0 con 0x7f47ac0cb3e0 2023-11-28T11:10:15.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.799+0000 7f47a1ffb700 1 -- 172.21.15.28:0/4075928730 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 184+0+0 (secure 0 0 0) 0x7f47a4055540 con 0x7f47ac0cb3e0 2023-11-28T11:10:15.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.801+0000 7f47b32d7700 1 -- 172.21.15.28:0/4075928730 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f47900682f0 msgr2=0x7f479006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.801+0000 7f47b32d7700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f47900682f0 0x7f479006a7a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f479c009ac0 tx=0x7f479c008040 comp rx=0 tx=0).stop 2023-11-28T11:10:15.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.801+0000 7f47b32d7700 1 -- 172.21.15.28:0/4075928730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47ac0cb3e0 msgr2=0x7f47ac1e7470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.801+0000 7f47b32d7700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47ac0cb3e0 0x7f47ac1e7470 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f47a4002230 tx=0x7f47a400a060 comp rx=0 tx=0).stop 2023-11-28T11:10:15.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.801+0000 7f47b32d7700 1 -- 172.21.15.28:0/4075928730 shutdown_connections 2023-11-28T11:10:15.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.801+0000 7f47b32d7700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f47900682f0 0x7f479006a7a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.801+0000 7f47b32d7700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47ac0cb3e0 0x7f47ac1e7470 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.801+0000 7f47b32d7700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f47ac0b1630 0x7f47ac15c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.801+0000 7f47b32d7700 1 --2- 172.21.15.28:0/4075928730 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47ac0b1a00 0x7f47ac0b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.801+0000 7f47b32d7700 1 -- 172.21.15.28:0/4075928730 >> 172.21.15.28:0/4075928730 conn(0x7f47ac0cc630 msgr2=0x7f47ac06c9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:15.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.801+0000 7f47b32d7700 1 -- 172.21.15.28:0/4075928730 shutdown_connections 2023-11-28T11:10:15.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.801+0000 7f47b32d7700 1 -- 172.21.15.28:0/4075928730 wait complete. 2023-11-28T11:10:15.804 INFO:teuthology.orchestra.run.smithi028.stderr:updated caps for client.0 2023-11-28T11:10:15.814 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:15.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.938+0000 7f55a351a700 1 Processor -- start 2023-11-28T11:10:15.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.939+0000 7f55a351a700 1 -- start start 2023-11-28T11:10:15.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.939+0000 7f55a351a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f559c15c990 0x7f559c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.939+0000 7f55a351a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f559c161090 0x7f559c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.939+0000 7f55a351a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f559c0caeb0 0x7f559c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.939+0000 7f55a351a700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f559c0ce340 con 0x7f559c161090 2023-11-28T11:10:15.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.939+0000 7f55a351a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f559c0cebe0 con 0x7f559c15c990 2023-11-28T11:10:15.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.939+0000 7f55a351a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559c0cb3f0 con 0x7f559c0caeb0 2023-11-28T11:10:15.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.940+0000 7f55a0ab5700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f559c161090 0x7f559c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.940+0000 7f55a0ab5700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f559c161090 0x7f559c0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49514/0 (socket says 172.21.15.28:49514) 2023-11-28T11:10:15.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.940+0000 7f55a0ab5700 1 -- 172.21.15.28:0/3438550149 learned_addr learned my addr 172.21.15.28:0/3438550149 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:15.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.940+0000 7f55a1ab7700 1 --2- 172.21.15.28:0/3438550149 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f559c0caeb0 0x7f559c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.940+0000 7f55a12b6700 1 --2- 172.21.15.28:0/3438550149 >> v2:172.21.15.59:3300/0 conn(0x7f559c15c990 0x7f559c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.940+0000 7f55a0ab5700 1 -- 172.21.15.28:0/3438550149 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f559c0caeb0 msgr2=0x7f559c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:15.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.940+0000 7f55a0ab5700 1 --2- 172.21.15.28:0/3438550149 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f559c0caeb0 0x7f559c0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.940+0000 7f55a0ab5700 1 -- 172.21.15.28:0/3438550149 >> v2:172.21.15.59:3300/0 conn(0x7f559c15c990 msgr2=0x7f559c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:15.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.940+0000 7f55a0ab5700 1 --2- 172.21.15.28:0/3438550149 >> v2:172.21.15.59:3300/0 conn(0x7f559c15c990 0x7f559c0cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.940+0000 7f55a0ab5700 1 -- 172.21.15.28:0/3438550149 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c0ce590 con 0x7f559c161090 2023-11-28T11:10:15.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.940+0000 7f55a0ab5700 1 --2- 172.21.15.28:0/3438550149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f559c161090 0x7f559c0ca970 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f559c06ea90 tx=0x7f5598009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a07cf896d75790d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:15.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.940+0000 7f5593fff700 1 -- 172.21.15.28:0/3438550149 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559800b870 con 0x7f559c161090 2023-11-28T11:10:15.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.940+0000 7f5593fff700 1 -- 172.21.15.28:0/3438550149 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f559800b9d0 con 0x7f559c161090 2023-11-28T11:10:15.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.940+0000 7f5593fff700 1 -- 172.21.15.28:0/3438550149 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f559800bd60 con 0x7f559c161090 2023-11-28T11:10:15.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.941+0000 7f55a351a700 1 -- 172.21.15.28:0/3438550149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f559c161090 msgr2=0x7f559c0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.941+0000 7f55a351a700 1 --2- 172.21.15.28:0/3438550149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f559c161090 0x7f559c0ca970 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f559c06ea90 tx=0x7f5598009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.941+0000 7f55a351a700 1 -- 172.21.15.28:0/3438550149 shutdown_connections 2023-11-28T11:10:15.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.941+0000 7f55a351a700 1 --2- 172.21.15.28:0/3438550149 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f559c0caeb0 0x7f559c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.941+0000 7f55a351a700 1 --2- 172.21.15.28:0/3438550149 >> v2:172.21.15.59:3300/0 conn(0x7f559c15c990 0x7f559c0cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.941+0000 7f55a351a700 1 --2- 172.21.15.28:0/3438550149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f559c161090 0x7f559c0ca970 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.941+0000 7f55a351a700 1 -- 172.21.15.28:0/3438550149 >> 172.21.15.28:0/3438550149 conn(0x7f559c0b1650 msgr2=0x7f559c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:15.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.941+0000 7f55a351a700 1 -- 172.21.15.28:0/3438550149 shutdown_connections 2023-11-28T11:10:15.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.941+0000 7f55a351a700 1 -- 172.21.15.28:0/3438550149 wait complete. 2023-11-28T11:10:15.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.942+0000 7f55a351a700 1 Processor -- start 2023-11-28T11:10:15.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.942+0000 7f55a351a700 1 -- start start 2023-11-28T11:10:15.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a351a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f559c0caeb0 0x7f559c14ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a351a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f559c15c990 0x7f559c1f5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a351a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f559c161090 0x7f559c1f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a351a700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c150580 con 0x7f559c15c990 2023-11-28T11:10:15.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a351a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c14fa00 con 0x7f559c161090 2023-11-28T11:10:15.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a351a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559c14fce0 con 0x7f559c0caeb0 2023-11-28T11:10:15.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a0ab5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f559c15c990 0x7f559c1f5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a0ab5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f559c15c990 0x7f559c1f5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49522/0 (socket says 172.21.15.28:49522) 2023-11-28T11:10:15.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a0ab5700 1 -- 172.21.15.28:0/2702093805 learned_addr learned my addr 172.21.15.28:0/2702093805 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:15.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a12b6700 1 --2- 172.21.15.28:0/2702093805 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f559c0caeb0 0x7f559c14ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a1ab7700 1 --2- 172.21.15.28:0/2702093805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f559c161090 0x7f559c1f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a0ab5700 1 -- 172.21.15.28:0/2702093805 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f559c0caeb0 msgr2=0x7f559c14ae80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a0ab5700 1 --2- 172.21.15.28:0/2702093805 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f559c0caeb0 0x7f559c14ae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a0ab5700 1 -- 172.21.15.28:0/2702093805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f559c161090 msgr2=0x7f559c1f5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:15.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a0ab5700 1 --2- 172.21.15.28:0/2702093805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f559c161090 0x7f559c1f5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:15.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a0ab5700 1 -- 172.21.15.28:0/2702093805 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5598009550 con 0x7f559c15c990 2023-11-28T11:10:15.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a1ab7700 1 --2- 172.21.15.28:0/2702093805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f559c161090 0x7f559c1f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:15.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.943+0000 7f55a0ab5700 1 --2- 172.21.15.28:0/2702093805 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f559c15c990 0x7f559c1f5a30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5598009c10 tx=0x7f5598004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:15.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.944+0000 7f5591ffb700 1 -- 172.21.15.28:0/2702093805 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598004590 con 0x7f559c15c990 2023-11-28T11:10:15.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.944+0000 7f5591ffb700 1 -- 172.21.15.28:0/2702093805 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55980036a0 con 0x7f559c15c990 2023-11-28T11:10:15.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.944+0000 7f55a351a700 1 -- 172.21.15.28:0/2702093805 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559c0cf5b0 con 0x7f559c15c990 2023-11-28T11:10:15.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.944+0000 7f5591ffb700 1 -- 172.21.15.28:0/2702093805 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5598003a50 con 0x7f559c15c990 2023-11-28T11:10:15.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.944+0000 7f55a351a700 1 -- 172.21.15.28:0/2702093805 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559c0ce870 con 0x7f559c15c990 2023-11-28T11:10:15.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.945+0000 7f55a351a700 1 -- 172.21.15.28:0/2702093805 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f559c0cf740 con 0x7f559c15c990 2023-11-28T11:10:15.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.945+0000 7f5591ffb700 1 -- 172.21.15.28:0/2702093805 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f559800f050 con 0x7f559c15c990 2023-11-28T11:10:15.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.946+0000 7f5591ffb700 1 --2- 172.21.15.28:0/2702093805 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f55800682f0 0x7f558006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:15.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.946+0000 7f5591ffb700 1 -- 172.21.15.28:0/2702093805 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5598086fe0 con 0x7f559c15c990 2023-11-28T11:10:15.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.946+0000 7f55a12b6700 1 --2- 172.21.15.28:0/2702093805 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f55800682f0 0x7f558006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:15.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.946+0000 7f55a12b6700 1 --2- 172.21.15.28:0/2702093805 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f55800682f0 0x7f558006a7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f558c006b20 tx=0x7f558c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:15.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:15.950+0000 7f5591ffb700 1 -- 172.21.15.28:0/2702093805 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5598053700 con 0x7f559c15c990 2023-11-28T11:10:16.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.186+0000 7f55a351a700 1 -- 172.21.15.28:0/2702093805 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f559c0675a0 con 0x7f559c15c990 2023-11-28T11:10:16.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.187+0000 7f5591ffb700 1 -- 172.21.15.28:0/2702093805 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f5598056d20 con 0x7f559c15c990 2023-11-28T11:10:16.188 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:16.188 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2867100265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2867100265},{"type":"v1","addr":"172.21.15.59:6833","nonce":2867100265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3808190953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3808190953},{"type":"v1","addr":"172.21.15.28:6837","nonce":3808190953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3920280453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3920280453},{"type":"v1","addr":"172.21.15.28:6836","nonce":3920280453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:10.820923+0000","modified":"2023-11-28T11:10:13.833216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6835/707419743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":707419743},{"type":"v1","addr":"172.21.15.59:6835","nonce":707419743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:10:16.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.189+0000 7f55a351a700 1 -- 172.21.15.28:0/2702093805 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f55800682f0 msgr2=0x7f558006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.189+0000 7f55a351a700 1 --2- 172.21.15.28:0/2702093805 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f55800682f0 0x7f558006a7a0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f558c006b20 tx=0x7f558c008040 comp rx=0 tx=0).stop 2023-11-28T11:10:16.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.189+0000 7f55a351a700 1 -- 172.21.15.28:0/2702093805 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f559c15c990 msgr2=0x7f559c1f5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.189+0000 7f55a351a700 1 --2- 172.21.15.28:0/2702093805 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f559c15c990 0x7f559c1f5a30 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f5598009c10 tx=0x7f5598004750 comp rx=0 tx=0).stop 2023-11-28T11:10:16.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.189+0000 7f55a351a700 1 -- 172.21.15.28:0/2702093805 shutdown_connections 2023-11-28T11:10:16.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.189+0000 7f55a351a700 1 --2- 172.21.15.28:0/2702093805 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f55800682f0 0x7f558006a7a0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.189+0000 7f55a351a700 1 --2- 172.21.15.28:0/2702093805 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f559c161090 0x7f559c1f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.189+0000 7f55a351a700 1 --2- 172.21.15.28:0/2702093805 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f559c0caeb0 0x7f559c14ae80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.189+0000 7f55a351a700 1 --2- 172.21.15.28:0/2702093805 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f559c15c990 0x7f559c1f5a30 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.189+0000 7f55a351a700 1 -- 172.21.15.28:0/2702093805 >> 172.21.15.28:0/2702093805 conn(0x7f559c0b1650 msgr2=0x7f559c0cc310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:16.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.189+0000 7f55a351a700 1 -- 172.21.15.28:0/2702093805 shutdown_connections 2023-11-28T11:10:16.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.189+0000 7f55a351a700 1 -- 172.21.15.28:0/2702093805 wait complete. 2023-11-28T11:10:16.192 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2023-11-28T11:10:16.605 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:10:10.820923+0000', 'modified': '2023-11-28T11:10:13.833216+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5547}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5547': {'gid': 5547, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.59:6835/707419743', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6834', 'nonce': 707419743}, {'type': 'v1', 'addr': '172.21.15.59:6835', 'nonce': 707419743}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:10:16.605 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:10:16.605 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T11:10:16.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.739+0000 7fb45c772700 1 Processor -- start 2023-11-28T11:10:16.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.739+0000 7fb45c772700 1 -- start start 2023-11-28T11:10:16.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.740+0000 7fb45c772700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb454159590 0x7fb45406e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.740+0000 7fb45c772700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb4540ce890 0x7fb4540ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.740+0000 7fb45c772700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb45415d120 con 0x7fb45415db70 2023-11-28T11:10:16.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.740+0000 7fb45c772700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4540cf250 con 0x7fb4540ce890 2023-11-28T11:10:16.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.740+0000 7fb45c772700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb45415d6e0 con 0x7fb454159590 2023-11-28T11:10:16.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.740+0000 7fb45ad0f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb4540ce890 0x7fb4540ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.740+0000 7fb45ad0f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb4540ce890 0x7fb4540ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49532/0 (socket says 172.21.15.28:49532) 2023-11-28T11:10:16.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.740+0000 7fb45ad0f700 1 -- 172.21.15.28:0/2473860904 learned_addr learned my addr 172.21.15.28:0/2473860904 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:16.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.740+0000 7fb45a50e700 1 --2- 172.21.15.28:0/2473860904 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb454159590 0x7fb45406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.741+0000 7fb45ad0f700 1 -- 172.21.15.28:0/2473860904 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb454159590 msgr2=0x7fb45406e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:16.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.741+0000 7fb45ad0f700 1 --2- 172.21.15.28:0/2473860904 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb454159590 0x7fb45406e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.741+0000 7fb45ad0f700 1 -- 172.21.15.28:0/2473860904 >> v1:172.21.15.59:6789/0 conn(0x7fb45415db70 legacy=0x7fb4540ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.741+0000 7fb45ad0f700 1 -- 172.21.15.28:0/2473860904 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4540aef60 con 0x7fb4540ce890 2023-11-28T11:10:16.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.741+0000 7fb45ad0f700 1 --2- 172.21.15.28:0/2473860904 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4540ce890 0x7fb4540ced10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00d150 tx=0x7fb44c00d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d20dbfa1c3ae1c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:16.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.741+0000 7fb45950c700 1 -- 172.21.15.28:0/2473860904 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c010070 con 0x7fb4540ce890 2023-11-28T11:10:16.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.741+0000 7fb45950c700 1 -- 172.21.15.28:0/2473860904 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb44c00ddc0 con 0x7fb4540ce890 2023-11-28T11:10:16.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.741+0000 7fb45950c700 1 -- 172.21.15.28:0/2473860904 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44c00b310 con 0x7fb4540ce890 2023-11-28T11:10:16.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.741+0000 7fb45c772700 1 -- 172.21.15.28:0/2473860904 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4540ce890 msgr2=0x7fb4540ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.741+0000 7fb45c772700 1 --2- 172.21.15.28:0/2473860904 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4540ce890 0x7fb4540ced10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00d150 tx=0x7fb44c00d460 comp rx=0 tx=0).stop 2023-11-28T11:10:16.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.742+0000 7fb45c772700 1 -- 172.21.15.28:0/2473860904 shutdown_connections 2023-11-28T11:10:16.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.742+0000 7fb45c772700 1 --2- 172.21.15.28:0/2473860904 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4540ce890 0x7fb4540ced10 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.742+0000 7fb45c772700 1 --2- 172.21.15.28:0/2473860904 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb454159590 0x7fb45406e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.742+0000 7fb45c772700 1 -- 172.21.15.28:0/2473860904 >> 172.21.15.28:0/2473860904 conn(0x7fb4540b1650 msgr2=0x7fb4540b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:16.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.742+0000 7fb45c772700 1 -- 172.21.15.28:0/2473860904 shutdown_connections 2023-11-28T11:10:16.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.742+0000 7fb45c772700 1 -- 172.21.15.28:0/2473860904 wait complete. 2023-11-28T11:10:16.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.743+0000 7fb45c772700 1 Processor -- start 2023-11-28T11:10:16.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.743+0000 7fb45c772700 1 -- start start 2023-11-28T11:10:16.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45c772700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4540ce890 0x7fb454151a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45c772700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb454159590 0x7fb4540b5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45c772700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb45415db70 0x7fb4541ee1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45c772700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4540c87f0 con 0x7fb4540ce890 2023-11-28T11:10:16.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45c772700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb454151890 con 0x7fb45415db70 2023-11-28T11:10:16.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45c772700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4540b5a80 con 0x7fb454159590 2023-11-28T11:10:16.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45a50e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4540ce890 0x7fb454151a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45a50e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4540ce890 0x7fb454151a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49534/0 (socket says 172.21.15.28:49534) 2023-11-28T11:10:16.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45a50e700 1 -- 172.21.15.28:0/2503536874 learned_addr learned my addr 172.21.15.28:0/2503536874 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:16.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45ad0f700 1 --2- 172.21.15.28:0/2503536874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb45415db70 0x7fb4541ee1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb459d0d700 1 --2- 172.21.15.28:0/2503536874 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb454159590 0x7fb4540b5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45a50e700 1 -- 172.21.15.28:0/2503536874 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb454159590 msgr2=0x7fb4540b5400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45a50e700 1 --2- 172.21.15.28:0/2503536874 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb454159590 0x7fb4540b5400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45a50e700 1 -- 172.21.15.28:0/2503536874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb45415db70 msgr2=0x7fb4541ee1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45a50e700 1 --2- 172.21.15.28:0/2503536874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb45415db70 0x7fb4541ee1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45a50e700 1 -- 172.21.15.28:0/2503536874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44c00bd40 con 0x7fb4540ce890 2023-11-28T11:10:16.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb459d0d700 1 --2- 172.21.15.28:0/2503536874 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb454159590 0x7fb4540b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:16.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45a50e700 1 --2- 172.21.15.28:0/2503536874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4540ce890 0x7fb454151a70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb44400e810 tx=0x7fb44400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:16.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb44affd700 1 -- 172.21.15.28:0/2503536874 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444013610 con 0x7fb4540ce890 2023-11-28T11:10:16.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb44affd700 1 -- 172.21.15.28:0/2503536874 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb444013770 con 0x7fb4540ce890 2023-11-28T11:10:16.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb44affd700 1 -- 172.21.15.28:0/2503536874 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444013b20 con 0x7fb4540ce890 2023-11-28T11:10:16.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45c772700 1 -- 172.21.15.28:0/2503536874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb45415d370 con 0x7fb4540ce890 2023-11-28T11:10:16.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.744+0000 7fb45c772700 1 -- 172.21.15.28:0/2503536874 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4540af240 con 0x7fb4540ce890 2023-11-28T11:10:16.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.746+0000 7fb44affd700 1 -- 172.21.15.28:0/2503536874 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb444023070 con 0x7fb4540ce890 2023-11-28T11:10:16.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.746+0000 7fb44affd700 1 --2- 172.21.15.28:0/2503536874 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb4380682f0 0x7fb43806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.746+0000 7fb44affd700 1 -- 172.21.15.28:0/2503536874 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb444085b20 con 0x7fb4540ce890 2023-11-28T11:10:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.746+0000 7fb459d0d700 1 --2- 172.21.15.28:0/2503536874 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb4380682f0 0x7fb43806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.747+0000 7fb459d0d700 1 --2- 172.21.15.28:0/2503536874 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb4380682f0 0x7fb43806a7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb4540b2e70 tx=0x7fb450009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:16.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.748+0000 7fb4688fbb80 1 Processor -- start 2023-11-28T11:10:16.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.749+0000 7fb4688fbb80 1 -- start start 2023-11-28T11:10:16.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.749+0000 7fb4688fbb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d54810 0x561e80d2aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.749+0000 7fb4688fbb80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d54be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.749+0000 7fb4688fbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d70770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb4688fbb80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e80c14bc0 con 0x561e80d52130 2023-11-28T11:10:16.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb4688fbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e80c83350 con 0x561e80d55120 2023-11-28T11:10:16.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb4688fbb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e80c7c880 con 0x561e80d54810 2023-11-28T11:10:16.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb459d0d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d54be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb459d0d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d54be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49544/0 (socket says 172.21.15.28:49544) 2023-11-28T11:10:16.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb459d0d700 1 -- 172.21.15.28:0/1458467063 learned_addr learned my addr 172.21.15.28:0/1458467063 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:16.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb45a50e700 1 --2- 172.21.15.28:0/1458467063 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d54810 0x561e80d2aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb45ad0f700 1 --2- 172.21.15.28:0/1458467063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d70770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb459d0d700 1 -- 172.21.15.28:0/1458467063 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d54810 msgr2=0x561e80d2aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:16.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb459d0d700 1 --2- 172.21.15.28:0/1458467063 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d54810 0x561e80d2aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb459d0d700 1 -- 172.21.15.28:0/1458467063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 msgr2=0x561e80d70770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:16.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb459d0d700 1 --2- 172.21.15.28:0/1458467063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d70770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb459d0d700 1 -- 172.21.15.28:0/1458467063 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e80afe5e0 con 0x561e80d52130 2023-11-28T11:10:16.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb459d0d700 1 --2- 172.21.15.28:0/1458467063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d54be0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb450005150 tx=0x7fb450011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67286e40021c508f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:16.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb437fff700 1 -- 172.21.15.28:0/1458467063 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb450003b10 con 0x561e80d52130 2023-11-28T11:10:16.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb437fff700 1 -- 172.21.15.28:0/1458467063 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb450003c70 con 0x561e80d52130 2023-11-28T11:10:16.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.750+0000 7fb437fff700 1 -- 172.21.15.28:0/1458467063 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45000e620 con 0x561e80d52130 2023-11-28T11:10:16.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.751+0000 7fb4688fbb80 1 -- 172.21.15.28:0/1458467063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 msgr2=0x561e80d54be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.751+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/1458467063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d54be0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fb450005150 tx=0x7fb450011000 comp rx=0 tx=0).stop 2023-11-28T11:10:16.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.751+0000 7fb4688fbb80 1 -- 172.21.15.28:0/1458467063 shutdown_connections 2023-11-28T11:10:16.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.751+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/1458467063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d70770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.751+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/1458467063 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d54810 0x561e80d2aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.751+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/1458467063 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d54be0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.751+0000 7fb4688fbb80 1 -- 172.21.15.28:0/1458467063 >> 172.21.15.28:0/1458467063 conn(0x561e80d27d60 msgr2=0x561e80d286f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:16.763 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.751+0000 7fb4688fbb80 1 -- 172.21.15.28:0/1458467063 shutdown_connections 2023-11-28T11:10:16.763 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.751+0000 7fb4688fbb80 1 -- 172.21.15.28:0/1458467063 wait complete. 2023-11-28T11:10:16.763 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.752+0000 7fb4688fbb80 1 -- start start 2023-11-28T11:10:16.763 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.752+0000 7fb4688fbb80 1 Processor -- start 2023-11-28T11:10:16.763 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.753+0000 7fb4688fbb80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d68090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.763 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.753+0000 7fb4688fbb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d54810 0x561e80d72f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.763 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.753+0000 7fb4688fbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d68b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.753+0000 7fb4688fbb80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e80b008d0 con 0x561e80d52130 2023-11-28T11:10:16.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.753+0000 7fb4688fbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e80d4e850 con 0x561e80d55120 2023-11-28T11:10:16.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.753+0000 7fb4688fbb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e80d693b0 con 0x561e80d54810 2023-11-28T11:10:16.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.753+0000 7fb4688fbb80 10 client.0 _ll_register_callbacks cb 0x561e80b22240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:16.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.753+0000 7fb45a50e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d68090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.753+0000 7fb45a50e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d68090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49548/0 (socket says 172.21.15.28:49548) 2023-11-28T11:10:16.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.753+0000 7fb45a50e700 1 -- 172.21.15.28:0/1058506730 learned_addr learned my addr 172.21.15.28:0/1058506730 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:16.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb45ad0f700 1 --2- 172.21.15.28:0/1058506730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d68b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb459d0d700 1 --2- 172.21.15.28:0/1058506730 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d54810 0x561e80d72f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb45a50e700 1 -- 172.21.15.28:0/1058506730 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d54810 msgr2=0x561e80d72f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:16.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb45a50e700 1 --2- 172.21.15.28:0/1058506730 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d54810 0x561e80d72f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb45a50e700 1 -- 172.21.15.28:0/1058506730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 msgr2=0x561e80d68b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:16.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb45a50e700 1 --2- 172.21.15.28:0/1058506730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d68b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb45a50e700 1 -- 172.21.15.28:0/1058506730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb45000f040 con 0x561e80d52130 2023-11-28T11:10:16.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb45ad0f700 1 --2- 172.21.15.28:0/1058506730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d68b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:16.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb45a50e700 1 --2- 172.21.15.28:0/1058506730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d68090 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb444052290 tx=0x7fb444053ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2705f523d42ab216 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:16.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb4357fa700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:10:16.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb4357fa700 1 -- 172.21.15.28:0/1058506730 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44405dbf0 con 0x561e80d52130 2023-11-28T11:10:16.766 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb4357fa700 1 -- 172.21.15.28:0/1058506730 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb444059040 con 0x561e80d52130 2023-11-28T11:10:16.766 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb4688fbb80 1 -- 172.21.15.28:0/1058506730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e80c4a140 con 0x561e80d52130 2023-11-28T11:10:16.766 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb4357fa700 1 -- 172.21.15.28:0/1058506730 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb444054cf0 con 0x561e80d52130 2023-11-28T11:10:16.766 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb4688fbb80 10 client.5736 fetch_fsmap learned FSMap version 17 2023-11-28T11:10:16.766 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb4688fbb80 1 -- 172.21.15.28:0/1058506730 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x561e80aaad90 con 0x561e80d52130 2023-11-28T11:10:16.766 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb4357fa700 1 -- 172.21.15.28:0/1058506730 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e80c4a140 con 0x561e80d52130 2023-11-28T11:10:16.766 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb4357fa700 1 -- 172.21.15.28:0/1058506730 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 17) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb444054e50 con 0x561e80d52130 2023-11-28T11:10:16.766 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb4688fbb80 10 client.5736 fetch_fsmap finished waiting for FSMap version 17 2023-11-28T11:10:16.766 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb4688fbb80 10 client.5736 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:10:16.767 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb4688fbb80 20 client.5736 populate_metadata read hostname 'smithi028' 2023-11-28T11:10:16.767 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/1058506730 >> [v2:172.21.15.59:6834/707419743,v1:172.21.15.59:6835/707419743] conn(0x561e80d74000 0x561e80d743d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.767 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb4688fbb80 4 client.5736 mds_command: new command op to 5547 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:10:16.767 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.754+0000 7fb4688fbb80 1 -- 172.21.15.28:0/1058506730 --> [v2:172.21.15.59:6834/707419743,v1:172.21.15.59:6835/707419743] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561e80b00f10 con 0x561e80d74000 2023-11-28T11:10:16.767 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.755+0000 7fb459d0d700 1 --2- 172.21.15.28:0/1058506730 >> [v2:172.21.15.59:6834/707419743,v1:172.21.15.59:6835/707419743] conn(0x561e80d74000 0x561e80d743d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.767 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.755+0000 7fb459d0d700 1 --2- 172.21.15.28:0/1058506730 >> [v2:172.21.15.59:6834/707419743,v1:172.21.15.59:6835/707419743] conn(0x561e80d74000 0x561e80d743d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ead5fe2462ebe712 server_cookie=55e198644432d93e in_seq=0 out_seq=0 2023-11-28T11:10:16.767 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.755+0000 7fb4357fa700 10 client.5736 ms_handle_connect on v2:172.21.15.59:6834/707419743 2023-11-28T11:10:16.767 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.756+0000 7fb4357fa700 1 -- 172.21.15.28:0/1058506730 <== mds.0 v2:172.21.15.59:6834/707419743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x561e80b00f10 con 0x561e80d74000 2023-11-28T11:10:16.767 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.756+0000 7fb4357fa700 10 client.5736 handle_command_reply: tid=0 2023-11-28T11:10:16.767 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.756+0000 7fb4688fbb80 1 client.5736 shutdown 2023-11-28T11:10:16.768 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.756+0000 7fb4688fbb80 1 -- 172.21.15.28:0/1058506730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 msgr2=0x561e80d68090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.768 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.756+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/1058506730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d68090 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7fb444052290 tx=0x7fb444053ff0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.768 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.756+0000 7fb4688fbb80 1 -- 172.21.15.28:0/1058506730 shutdown_connections 2023-11-28T11:10:16.768 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.756+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/1058506730 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d68090 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.768 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.756+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/1058506730 >> [v2:172.21.15.59:6834/707419743,v1:172.21.15.59:6835/707419743] conn(0x561e80d74000 0x561e80d743d0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.768 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.756+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/1058506730 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d68b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.756+0000 7fb4357fa700 0 client.5736 ms_handle_reset on v2:172.21.15.59:6834/707419743 2023-11-28T11:10:16.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.756+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/1058506730 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d54810 0x561e80d72f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.756+0000 7fb4688fbb80 1 -- 172.21.15.28:0/1058506730 >> 172.21.15.28:0/1058506730 conn(0x561e80d27d60 msgr2=0x561e80d53e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:16.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.756+0000 7fb4688fbb80 1 -- 172.21.15.28:0/1058506730 shutdown_connections 2023-11-28T11:10:16.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.756+0000 7fb4688fbb80 1 -- 172.21.15.28:0/1058506730 wait complete. 2023-11-28T11:10:16.769 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.756+0000 7fb4688fbb80 20 client.5736 trim_cache size 0 max 16384 2023-11-28T11:10:16.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.774+0000 7fb4688fbb80 1 Processor -- start 2023-11-28T11:10:16.774 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.774+0000 7fb4688fbb80 1 -- start start 2023-11-28T11:10:16.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.774+0000 7fb4688fbb80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d689b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.774+0000 7fb4688fbb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d54810 0x561e80d288b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.774+0000 7fb4688fbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d2b960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.774+0000 7fb4688fbb80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e80d68ef0 con 0x561e80d52130 2023-11-28T11:10:16.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.774+0000 7fb4688fbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e80d722d0 con 0x561e80d55120 2023-11-28T11:10:16.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.774+0000 7fb4688fbb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e80d278a0 con 0x561e80d54810 2023-11-28T11:10:16.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.774+0000 7fb45a50e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.774+0000 7fb45a50e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d689b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49556/0 (socket says 172.21.15.28:49556) 2023-11-28T11:10:16.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.774+0000 7fb45a50e700 1 -- 172.21.15.28:0/4287701354 learned_addr learned my addr 172.21.15.28:0/4287701354 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:16.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.775+0000 7fb45ad0f700 1 --2- 172.21.15.28:0/4287701354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d2b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.775+0000 7fb459d0d700 1 --2- 172.21.15.28:0/4287701354 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d54810 0x561e80d288b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.775+0000 7fb45a50e700 1 -- 172.21.15.28:0/4287701354 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d54810 msgr2=0x561e80d288b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:16.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.775+0000 7fb45a50e700 1 --2- 172.21.15.28:0/4287701354 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d54810 0x561e80d288b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.775+0000 7fb45a50e700 1 -- 172.21.15.28:0/4287701354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 msgr2=0x561e80d2b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:16.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.775+0000 7fb45a50e700 1 --2- 172.21.15.28:0/4287701354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d2b960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.775+0000 7fb45a50e700 1 -- 172.21.15.28:0/4287701354 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561e80c43140 con 0x561e80d52130 2023-11-28T11:10:16.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.775+0000 7fb45a50e700 1 --2- 172.21.15.28:0/4287701354 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d689b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb444000be0 tx=0x7fb44401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14cefe1defdf25ab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:16.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.775+0000 7fb448ff9700 1 -- 172.21.15.28:0/4287701354 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb44400c070 con 0x561e80d52130 2023-11-28T11:10:16.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.775+0000 7fb448ff9700 1 -- 172.21.15.28:0/4287701354 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb444054cf0 con 0x561e80d52130 2023-11-28T11:10:16.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.775+0000 7fb448ff9700 1 -- 172.21.15.28:0/4287701354 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb444005a20 con 0x561e80d52130 2023-11-28T11:10:16.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.775+0000 7fb4688fbb80 1 -- 172.21.15.28:0/4287701354 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 msgr2=0x561e80d689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.775+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/4287701354 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d689b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7fb444000be0 tx=0x7fb44401d000 comp rx=0 tx=0).stop 2023-11-28T11:10:16.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.776+0000 7fb4688fbb80 1 -- 172.21.15.28:0/4287701354 shutdown_connections 2023-11-28T11:10:16.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.776+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/4287701354 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d52130 0x561e80d689b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.776+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/4287701354 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d2b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.776+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/4287701354 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d54810 0x561e80d288b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.776+0000 7fb4688fbb80 1 -- 172.21.15.28:0/4287701354 >> 172.21.15.28:0/4287701354 conn(0x561e80d27d60 msgr2=0x561e80d29230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:16.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.776+0000 7fb4688fbb80 1 -- 172.21.15.28:0/4287701354 shutdown_connections 2023-11-28T11:10:16.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.776+0000 7fb4688fbb80 1 -- 172.21.15.28:0/4287701354 wait complete. 2023-11-28T11:10:16.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.776+0000 7fb4688fbb80 1 -- start start 2023-11-28T11:10:16.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.776+0000 7fb4688fbb80 1 Processor -- start 2023-11-28T11:10:16.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.777+0000 7fb4688fbb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d52130 0x561e80dbb830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.777+0000 7fb4688fbb80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d54810 0x561e80d51500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.777+0000 7fb4688fbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d72d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.777+0000 7fb4688fbb80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e80d660f0 con 0x561e80d54810 2023-11-28T11:10:16.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.777+0000 7fb4688fbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561e80d662f0 con 0x561e80d55120 2023-11-28T11:10:16.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.777+0000 7fb4688fbb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x561e80d67150 con 0x561e80d52130 2023-11-28T11:10:16.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.777+0000 7fb4688fbb80 10 client.0 _ll_register_callbacks cb 0x561e80b22240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:10:16.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.777+0000 7fb459d0d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d54810 0x561e80d51500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.777+0000 7fb459d0d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d54810 0x561e80d51500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49564/0 (socket says 172.21.15.28:49564) 2023-11-28T11:10:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.777+0000 7fb459d0d700 1 -- 172.21.15.28:0/3813958806 learned_addr learned my addr 172.21.15.28:0/3813958806 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.777+0000 7fb45a50e700 1 --2- 172.21.15.28:0/3813958806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d52130 0x561e80dbb830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.777+0000 7fb45ad0f700 1 --2- 172.21.15.28:0/3813958806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d72d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.777+0000 7fb459d0d700 1 -- 172.21.15.28:0/3813958806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d52130 msgr2=0x561e80dbb830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.777+0000 7fb459d0d700 1 --2- 172.21.15.28:0/3813958806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d52130 0x561e80dbb830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb459d0d700 1 -- 172.21.15.28:0/3813958806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 msgr2=0x561e80d72d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:16.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb459d0d700 1 --2- 172.21.15.28:0/3813958806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d72d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb459d0d700 1 -- 172.21.15.28:0/3813958806 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561e80aaad90 con 0x561e80d54810 2023-11-28T11:10:16.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb45a50e700 1 --2- 172.21.15.28:0/3813958806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d52130 0x561e80dbb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:16.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb45ad0f700 1 --2- 172.21.15.28:0/3813958806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d72d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:16.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb459d0d700 1 --2- 172.21.15.28:0/3813958806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d54810 0x561e80d51500 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb450009010 tx=0x7fb450000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3205b7d362b9143f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:16.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb4357fa700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:10:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb4357fa700 1 -- 172.21.15.28:0/3813958806 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb450028040 con 0x561e80d54810 2023-11-28T11:10:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb4357fa700 1 -- 172.21.15.28:0/3813958806 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb45002f3f0 con 0x561e80d54810 2023-11-28T11:10:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb4688fbb80 1 -- 172.21.15.28:0/3813958806 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561e80c3a160 con 0x561e80d54810 2023-11-28T11:10:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb4357fa700 1 -- 172.21.15.28:0/3813958806 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb45002f8a0 con 0x561e80d54810 2023-11-28T11:10:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb4688fbb80 10 client.5742 fetch_fsmap learned FSMap version 17 2023-11-28T11:10:16.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb4688fbb80 1 -- 172.21.15.28:0/3813958806 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x561e80ad1200 con 0x561e80d54810 2023-11-28T11:10:16.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb4357fa700 1 -- 172.21.15.28:0/3813958806 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561e80c3a160 con 0x561e80d54810 2023-11-28T11:10:16.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb4357fa700 1 -- 172.21.15.28:0/3813958806 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 17) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fb45002fa00 con 0x561e80d54810 2023-11-28T11:10:16.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb4688fbb80 10 client.5742 fetch_fsmap finished waiting for FSMap version 17 2023-11-28T11:10:16.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb4688fbb80 10 client.5742 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:10:16.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb4688fbb80 20 client.5742 populate_metadata read hostname 'smithi028' 2023-11-28T11:10:16.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/3813958806 >> [v2:172.21.15.59:6834/707419743,v1:172.21.15.59:6835/707419743] conn(0x561e80dd04c0 0x561e80dd0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb4688fbb80 4 client.5742 mds_command: new command op to 5547 tid=0[{"prefix": "status"}] 2023-11-28T11:10:16.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.778+0000 7fb4688fbb80 1 -- 172.21.15.28:0/3813958806 --> [v2:172.21.15.59:6834/707419743,v1:172.21.15.59:6835/707419743] -- command(tid 0: {"prefix": "status"}) v1 -- 0x561e80c24170 con 0x561e80dd04c0 2023-11-28T11:10:16.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb45a50e700 1 --2- 172.21.15.28:0/3813958806 >> [v2:172.21.15.59:6834/707419743,v1:172.21.15.59:6835/707419743] conn(0x561e80dd04c0 0x561e80dd0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb4357fa700 10 client.5742 ms_handle_connect on v2:172.21.15.59:6834/707419743 2023-11-28T11:10:16.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb45a50e700 1 --2- 172.21.15.28:0/3813958806 >> [v2:172.21.15.59:6834/707419743,v1:172.21.15.59:6835/707419743] conn(0x561e80dd04c0 0x561e80dd0890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5db880778113afc6 server_cookie=40c8fbae12e41359 in_seq=0 out_seq=0 2023-11-28T11:10:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb4357fa700 1 -- 172.21.15.28:0/3813958806 <== mds.0 v2:172.21.15.59:6834/707419743 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x561e80c24170 con 0x561e80dd04c0 2023-11-28T11:10:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb4357fa700 10 client.5742 handle_command_reply: tid=0 2023-11-28T11:10:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb4688fbb80 1 client.5742 shutdown 2023-11-28T11:10:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb4688fbb80 1 -- 172.21.15.28:0/3813958806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d54810 msgr2=0x561e80d51500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/3813958806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d54810 0x561e80d51500 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7fb450009010 tx=0x7fb450000ed0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb4688fbb80 1 -- 172.21.15.28:0/3813958806 shutdown_connections 2023-11-28T11:10:16.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/3813958806 >> [v2:172.21.15.59:6834/707419743,v1:172.21.15.59:6835/707419743] conn(0x561e80dd04c0 0x561e80dd0890 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/3813958806 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e80d55120 0x561e80d72d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/3813958806 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561e80d52130 0x561e80dbb830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb4688fbb80 1 --2- 172.21.15.28:0/3813958806 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561e80d54810 0x561e80d51500 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb4357fa700 0 client.5742 ms_handle_reset on v2:172.21.15.59:6834/707419743 2023-11-28T11:10:16.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb4688fbb80 1 -- 172.21.15.28:0/3813958806 >> 172.21.15.28:0/3813958806 conn(0x561e80d27d60 msgr2=0x561e80d23b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:16.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb4688fbb80 1 -- 172.21.15.28:0/3813958806 shutdown_connections 2023-11-28T11:10:16.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.779+0000 7fb4688fbb80 1 -- 172.21.15.28:0/3813958806 wait complete. 2023-11-28T11:10:16.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.780+0000 7fb4688fbb80 20 client.5742 trim_cache size 0 max 16384 2023-11-28T11:10:16.796 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:10:16.796 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "6e299adc-7a2a-4cf1-afa2-7ce104b29a51", 2023-11-28T11:10:16.796 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2023-11-28T11:10:16.796 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 5547, 2023-11-28T11:10:16.796 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2023-11-28T11:10:16.796 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2023-11-28T11:10:16.796 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2023-11-28T11:10:16.796 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 5.956899183, 2023-11-28T11:10:16.797 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 17, 2023-11-28T11:10:16.797 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 40, 2023-11-28T11:10:16.797 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 40, 2023-11-28T11:10:16.797 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 14.220149203 2023-11-28T11:10:16.797 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:10:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.780+0000 7fb45c772700 1 -- 172.21.15.28:0/2503536874 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb4380682f0 msgr2=0x7fb43806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.780+0000 7fb45c772700 1 --2- 172.21.15.28:0/2503536874 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb4380682f0 0x7fb43806a7a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fb4540b2e70 tx=0x7fb450009040 comp rx=0 tx=0).stop 2023-11-28T11:10:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.780+0000 7fb45c772700 1 -- 172.21.15.28:0/2503536874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4540ce890 msgr2=0x7fb454151a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.780+0000 7fb45c772700 1 --2- 172.21.15.28:0/2503536874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4540ce890 0x7fb454151a70 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fb44400e810 tx=0x7fb44400eb20 comp rx=0 tx=0).stop 2023-11-28T11:10:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.780+0000 7fb45c772700 1 -- 172.21.15.28:0/2503536874 shutdown_connections 2023-11-28T11:10:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.780+0000 7fb45c772700 1 --2- 172.21.15.28:0/2503536874 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb4380682f0 0x7fb43806a7a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.780+0000 7fb45c772700 1 --2- 172.21.15.28:0/2503536874 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb4540ce890 0x7fb454151a70 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.780+0000 7fb45c772700 1 --2- 172.21.15.28:0/2503536874 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb45415db70 0x7fb4541ee1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.780+0000 7fb45c772700 1 --2- 172.21.15.28:0/2503536874 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb454159590 0x7fb4540b5400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.780+0000 7fb45c772700 1 -- 172.21.15.28:0/2503536874 >> 172.21.15.28:0/2503536874 conn(0x7fb4540b1650 msgr2=0x7fb454153a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:16.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.780+0000 7fb45c772700 1 -- 172.21.15.28:0/2503536874 shutdown_connections 2023-11-28T11:10:16.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.780+0000 7fb45c772700 1 -- 172.21.15.28:0/2503536874 wait complete. 2023-11-28T11:10:16.800 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:10:16.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.925+0000 7f4403b2a700 1 Processor -- start 2023-11-28T11:10:16.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.925+0000 7f4403b2a700 1 -- start start 2023-11-28T11:10:16.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.926+0000 7f4403b2a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f43fc15a5a0 0x7f43fc06b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.926+0000 7f4403b2a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f43fc0cc4b0 0x7f43fc0cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.926+0000 7f4403b2a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43fc0cb570 con 0x7f43fc0ca280 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.926+0000 7f4403b2a700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43fc0cce70 con 0x7f43fc0cc4b0 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.926+0000 7f4403b2a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc0ca6f0 con 0x7f43fc15a5a0 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.926+0000 7f44020c7700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f43fc0cc4b0 0x7f43fc0cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.926+0000 7f44020c7700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f43fc0cc4b0 0x7f43fc0cc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49574/0 (socket says 172.21.15.28:49574) 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.926+0000 7f44020c7700 1 -- 172.21.15.28:0/3484219344 learned_addr learned my addr 172.21.15.28:0/3484219344 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.926+0000 7f44018c6700 1 --2- 172.21.15.28:0/3484219344 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f43fc15a5a0 0x7f43fc06b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.926+0000 7f44020c7700 1 -- 172.21.15.28:0/3484219344 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f43fc15a5a0 msgr2=0x7f43fc06b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:16.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.926+0000 7f44020c7700 1 --2- 172.21.15.28:0/3484219344 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f43fc15a5a0 0x7f43fc06b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.926+0000 7f44020c7700 1 -- 172.21.15.28:0/3484219344 >> v1:172.21.15.59:6789/0 conn(0x7f43fc0ca280 legacy=0x7f43fc0cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:16.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.926+0000 7f44020c7700 1 -- 172.21.15.28:0/3484219344 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc0cc110 con 0x7f43fc0cc4b0 2023-11-28T11:10:16.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.926+0000 7f44020c7700 1 --2- 172.21.15.28:0/3484219344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43fc0cc4b0 0x7f43fc0cc930 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f43f4009e50 tx=0x7f43f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0cd4d3f89e4d3e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:16.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.927+0000 7f44008c4700 1 -- 172.21.15.28:0/3484219344 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f400e070 con 0x7f43fc0cc4b0 2023-11-28T11:10:16.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.927+0000 7f44008c4700 1 -- 172.21.15.28:0/3484219344 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f4003a20 con 0x7f43fc0cc4b0 2023-11-28T11:10:16.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.927+0000 7f44008c4700 1 -- 172.21.15.28:0/3484219344 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4003dd0 con 0x7f43fc0cc4b0 2023-11-28T11:10:16.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.927+0000 7f4403b2a700 1 -- 172.21.15.28:0/3484219344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43fc0cc4b0 msgr2=0x7f43fc0cc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.927+0000 7f4403b2a700 1 --2- 172.21.15.28:0/3484219344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43fc0cc4b0 0x7f43fc0cc930 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f43f4009e50 tx=0x7f43f400b330 comp rx=0 tx=0).stop 2023-11-28T11:10:16.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.927+0000 7f4403b2a700 1 -- 172.21.15.28:0/3484219344 shutdown_connections 2023-11-28T11:10:16.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.927+0000 7f4403b2a700 1 --2- 172.21.15.28:0/3484219344 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43fc0cc4b0 0x7f43fc0cc930 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.927+0000 7f4403b2a700 1 --2- 172.21.15.28:0/3484219344 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f43fc15a5a0 0x7f43fc06b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.927+0000 7f4403b2a700 1 -- 172.21.15.28:0/3484219344 >> 172.21.15.28:0/3484219344 conn(0x7f43fc0b1650 msgr2=0x7f43fc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:16.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.927+0000 7f4403b2a700 1 -- 172.21.15.28:0/3484219344 shutdown_connections 2023-11-28T11:10:16.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.927+0000 7f4403b2a700 1 -- 172.21.15.28:0/3484219344 wait complete. 2023-11-28T11:10:16.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.928+0000 7f4403b2a700 1 Processor -- start 2023-11-28T11:10:16.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.928+0000 7f4403b2a700 1 -- start start 2023-11-28T11:10:16.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.929+0000 7f4403b2a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f43fc0ca280 0x7f43fc0cd670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.929+0000 7f4403b2a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43fc0cc4b0 0x7f43fc0c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.929+0000 7f4403b2a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43fc15a5a0 0x7f43fc15d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.929+0000 7f4403b2a700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc15bc70 con 0x7f43fc15a5a0 2023-11-28T11:10:16.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.929+0000 7f4403b2a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc15d9f0 con 0x7f43fc0cc4b0 2023-11-28T11:10:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.929+0000 7f4403b2a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc0cdc70 con 0x7f43fc0ca280 2023-11-28T11:10:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.929+0000 7f44020c7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43fc15a5a0 0x7f43fc15d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.929+0000 7f44020c7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43fc15a5a0 0x7f43fc15d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49580/0 (socket says 172.21.15.28:49580) 2023-11-28T11:10:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.929+0000 7f44020c7700 1 -- 172.21.15.28:0/3098416902 learned_addr learned my addr 172.21.15.28:0/3098416902 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.929+0000 7f44018c6700 1 --2- 172.21.15.28:0/3098416902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f43fc0ca280 0x7f43fc0cd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.929+0000 7f44010c5700 1 --2- 172.21.15.28:0/3098416902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43fc0cc4b0 0x7f43fc0c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.930+0000 7f44020c7700 1 -- 172.21.15.28:0/3098416902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f43fc0ca280 msgr2=0x7f43fc0cd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.930+0000 7f44020c7700 1 --2- 172.21.15.28:0/3098416902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f43fc0ca280 0x7f43fc0cd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.930+0000 7f44020c7700 1 -- 172.21.15.28:0/3098416902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43fc0cc4b0 msgr2=0x7f43fc0c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.930+0000 7f44020c7700 1 --2- 172.21.15.28:0/3098416902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43fc0cc4b0 0x7f43fc0c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:16.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.930+0000 7f44020c7700 1 -- 172.21.15.28:0/3098416902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f4009b30 con 0x7f43fc15a5a0 2023-11-28T11:10:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.930+0000 7f44020c7700 1 --2- 172.21.15.28:0/3098416902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43fc15a5a0 0x7f43fc15d4b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f43fc06ebb0 tx=0x7f43f4004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.930+0000 7f43f27fc700 1 -- 172.21.15.28:0/3098416902 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4004440 con 0x7f43fc15a5a0 2023-11-28T11:10:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.930+0000 7f43f27fc700 1 -- 172.21.15.28:0/3098416902 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f40045a0 con 0x7f43fc15a5a0 2023-11-28T11:10:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.930+0000 7f4403b2a700 1 -- 172.21.15.28:0/3098416902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43fc0cac40 con 0x7f43fc15a5a0 2023-11-28T11:10:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.930+0000 7f43f27fc700 1 -- 172.21.15.28:0/3098416902 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4004950 con 0x7f43fc15a5a0 2023-11-28T11:10:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.930+0000 7f4403b2a700 1 -- 172.21.15.28:0/3098416902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43fc0ca960 con 0x7f43fc15a5a0 2023-11-28T11:10:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.931+0000 7f4403b2a700 1 -- 172.21.15.28:0/3098416902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43fc0caf20 con 0x7f43fc15a5a0 2023-11-28T11:10:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.931+0000 7f43f27fc700 1 -- 172.21.15.28:0/3098416902 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f43f4022030 con 0x7f43fc15a5a0 2023-11-28T11:10:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.932+0000 7f43f27fc700 1 --2- 172.21.15.28:0/3098416902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f43e00682f0 0x7f43e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.932+0000 7f43f27fc700 1 -- 172.21.15.28:0/3098416902 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f43f4087260 con 0x7f43fc15a5a0 2023-11-28T11:10:16.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.932+0000 7f44018c6700 1 --2- 172.21.15.28:0/3098416902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f43e00682f0 0x7f43e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:16.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.932+0000 7f44018c6700 1 --2- 172.21.15.28:0/3098416902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f43e00682f0 0x7f43e006a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00a950 tx=0x7f43ec008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:16.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:16.936+0000 7f43f27fc700 1 -- 172.21.15.28:0/3098416902 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f43f4053900 con 0x7f43fc15a5a0 2023-11-28T11:10:17.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:17.176+0000 7f4403b2a700 1 -- 172.21.15.28:0/3098416902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f43fc0675a0 con 0x7f43fc15a5a0 2023-11-28T11:10:17.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:17.176+0000 7f43f27fc700 1 -- 172.21.15.28:0/3098416902 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f43f402c300 con 0x7f43fc15a5a0 2023-11-28T11:10:17.177 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:10:17.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:17.178+0000 7f4403b2a700 1 -- 172.21.15.28:0/3098416902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f43e00682f0 msgr2=0x7f43e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:17.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:17.178+0000 7f4403b2a700 1 --2- 172.21.15.28:0/3098416902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f43e00682f0 0x7f43e006a7a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f43ec00a950 tx=0x7f43ec008040 comp rx=0 tx=0).stop 2023-11-28T11:10:17.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:17.178+0000 7f4403b2a700 1 -- 172.21.15.28:0/3098416902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43fc15a5a0 msgr2=0x7f43fc15d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:17.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:17.178+0000 7f4403b2a700 1 --2- 172.21.15.28:0/3098416902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43fc15a5a0 0x7f43fc15d4b0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f43fc06ebb0 tx=0x7f43f4004ad0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:17.178+0000 7f4403b2a700 1 -- 172.21.15.28:0/3098416902 shutdown_connections 2023-11-28T11:10:17.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:17.178+0000 7f4403b2a700 1 --2- 172.21.15.28:0/3098416902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f43e00682f0 0x7f43e006a7a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:17.178+0000 7f4403b2a700 1 --2- 172.21.15.28:0/3098416902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43fc15a5a0 0x7f43fc15d4b0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:17.178+0000 7f4403b2a700 1 --2- 172.21.15.28:0/3098416902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f43fc0ca280 0x7f43fc0cd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:17.178+0000 7f4403b2a700 1 --2- 172.21.15.28:0/3098416902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f43fc0cc4b0 0x7f43fc0c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:17.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:17.178+0000 7f4403b2a700 1 -- 172.21.15.28:0/3098416902 >> 172.21.15.28:0/3098416902 conn(0x7f43fc0b1650 msgr2=0x7f43fc0b0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:17.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:17.179+0000 7f4403b2a700 1 -- 172.21.15.28:0/3098416902 shutdown_connections 2023-11-28T11:10:17.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:17.179+0000 7f4403b2a700 1 -- 172.21.15.28:0/3098416902 wait complete. 2023-11-28T11:10:17.191 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:10:17.191 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:10:17.191 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2023-11-28T11:10:17.192 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:10:17.192 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:17.192 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:10:17.192 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:10:17.192 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:10:17.192 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:10:17.192 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:17.192 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2023-11-28T11:10:17.210 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:10:17.210 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:10:17.210 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:10:17.210 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:17.210 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2023-11-28T11:10:17.210 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:17.210 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:10:17.210 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:10:17.211 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:10:17.211 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:10:17.211 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:10:17.211 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2023-11-28T11:10:17.211 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T11:10:17.211 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42241sec preferred_lft 42241sec 2023-11-28T11:10:17.211 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2023-11-28T11:10:17.211 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:17.211 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:10:17.211 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2023-11-28T11:10:17.211 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:10:17.212 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 62:ec:54:d3:7a:f0 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:10:17.212 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:10:17.212 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:17.212 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::407f:32ff:fe2d:9b2f/64 scope link 2023-11-28T11:10:17.212 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:17.212 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:10:17.212 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 62:ec:54:d3:7a:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:10:17.212 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::60ec:54ff:fed3:7af0/64 scope link 2023-11-28T11:10:17.212 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:17.212 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:10:17.212 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether c6:1d:6e:49:de:b9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:10:17.213 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::c41d:6eff:fe49:deb9/64 scope link 2023-11-28T11:10:17.213 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:10:17.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:17.213 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:10:17.276 INFO:teuthology.orchestra.run.smithi028.stdout:1 2023-11-28T11:10:17.278 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:17.278 DEBUG:teuthology.orchestra.run.smithi028:> route 2023-11-28T11:10:17.338 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2023-11-28T11:10:17.338 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:10:17.339 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T11:10:17.339 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T11:10:17.339 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:10:17.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:17.340 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:10:17.340 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:10:17.340 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T11:10:17.340 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T11:10:17.341 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T11:10:17.341 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:10:17.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:17.445 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2023-11-28T11:10:17.462 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:10:17.462 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:10:17.463 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:10:17.463 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:17.463 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2023-11-28T11:10:17.530 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T11:10:17.531 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:10:17.531 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:17.587 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:10:17.588 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:10:17.588 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:17.644 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2023-11-28T11:10:17.712 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:10:17.712 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:10:17.779 INFO:teuthology.orchestra.run.smithi028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:10:17.780 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:10:17.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:17.781 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2023-11-28T11:10:17.838 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T11:10:17.839 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/) 2023-11-28T11:10:17.881 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2023-11-28T11:10:17.907 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:10:17.908 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:10:17.970 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:2023-11-28T11:10:17.969+0000 7fa7ec2f25c0 -1 init, newargv = 0x5638246c55f0 newargc=15 2023-11-28T11:10:17.970 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[121827]: starting ceph client 2023-11-28T11:10:17.975 INFO:teuthology.orchestra.run.smithi028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:10:17.976 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[121827]: starting fuse 2023-11-28T11:10:17.976 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:10:17.977 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:17.977 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2023-11-28T11:10:18.035 INFO:teuthology.orchestra.run.smithi028.stdout:47 2023-11-28T11:10:18.035 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T11:10:18.035 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:10:18.036 DEBUG:teuthology.orchestra.run.smithi028:> import glob 2023-11-28T11:10:18.036 DEBUG:teuthology.orchestra.run.smithi028:> import re 2023-11-28T11:10:18.036 DEBUG:teuthology.orchestra.run.smithi028:> import os 2023-11-28T11:10:18.036 DEBUG:teuthology.orchestra.run.smithi028:> import subprocess 2023-11-28T11:10:18.036 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:10:18.036 DEBUG:teuthology.orchestra.run.smithi028:> def _find_admin_socket(client_name): 2023-11-28T11:10:18.036 DEBUG:teuthology.orchestra.run.smithi028:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T11:10:18.036 DEBUG:teuthology.orchestra.run.smithi028:> files = glob.glob(asok_path) 2023-11-28T11:10:18.036 DEBUG:teuthology.orchestra.run.smithi028:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T11:10:18.036 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:10:18.036 DEBUG:teuthology.orchestra.run.smithi028:> # Given a non-glob path, it better be there 2023-11-28T11:10:18.036 DEBUG:teuthology.orchestra.run.smithi028:> if "*" not in asok_path: 2023-11-28T11:10:18.036 DEBUG:teuthology.orchestra.run.smithi028:> assert(len(files) == 1) 2023-11-28T11:10:18.037 DEBUG:teuthology.orchestra.run.smithi028:> return files[0] 2023-11-28T11:10:18.037 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:10:18.037 DEBUG:teuthology.orchestra.run.smithi028:> for f in files: 2023-11-28T11:10:18.037 DEBUG:teuthology.orchestra.run.smithi028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T11:10:18.037 DEBUG:teuthology.orchestra.run.smithi028:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T11:10:18.037 DEBUG:teuthology.orchestra.run.smithi028:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T11:10:18.037 DEBUG:teuthology.orchestra.run.smithi028:> contents = proc_f.read() 2023-11-28T11:10:18.037 DEBUG:teuthology.orchestra.run.smithi028:> if mountpoint in contents: 2023-11-28T11:10:18.037 DEBUG:teuthology.orchestra.run.smithi028:> return f 2023-11-28T11:10:18.037 DEBUG:teuthology.orchestra.run.smithi028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T11:10:18.037 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:10:18.037 DEBUG:teuthology.orchestra.run.smithi028:> print(_find_admin_socket("client.0")) 2023-11-28T11:10:18.037 DEBUG:teuthology.orchestra.run.smithi028:> ' 2023-11-28T11:10:18.161 INFO:teuthology.orchestra.run.smithi028.stdout:/var/run/ceph/ceph-client.0.121827.asok 2023-11-28T11:10:18.343 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.121827.asok 2023-11-28T11:10:18.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:18.344 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.121827.asok status 2023-11-28T11:10:18.506 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:10:18.506 INFO:teuthology.orchestra.run.smithi028.stdout: "metadata": { 2023-11-28T11:10:18.506 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T11:10:18.506 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T11:10:18.507 INFO:teuthology.orchestra.run.smithi028.stdout: "entity_id": "0", 2023-11-28T11:10:18.507 INFO:teuthology.orchestra.run.smithi028.stdout: "hostname": "smithi028", 2023-11-28T11:10:18.507 INFO:teuthology.orchestra.run.smithi028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T11:10:18.507 INFO:teuthology.orchestra.run.smithi028.stdout: "pid": "121827", 2023-11-28T11:10:18.507 INFO:teuthology.orchestra.run.smithi028.stdout: "root": "/" 2023-11-28T11:10:18.507 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:18.507 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_count": 0, 2023-11-28T11:10:18.508 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_pinned_count": 0, 2023-11-28T11:10:18.508 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 5757, 2023-11-28T11:10:18.508 INFO:teuthology.orchestra.run.smithi028.stdout: "inst": { 2023-11-28T11:10:18.508 INFO:teuthology.orchestra.run.smithi028.stdout: "name": { 2023-11-28T11:10:18.508 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "client", 2023-11-28T11:10:18.508 INFO:teuthology.orchestra.run.smithi028.stdout: "num": 5757 2023-11-28T11:10:18.508 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:18.508 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2023-11-28T11:10:18.508 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2023-11-28T11:10:18.508 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2023-11-28T11:10:18.509 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2518549734 2023-11-28T11:10:18.509 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:18.509 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:18.509 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2023-11-28T11:10:18.510 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2023-11-28T11:10:18.510 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2023-11-28T11:10:18.510 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 2518549734 2023-11-28T11:10:18.510 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:18.510 INFO:teuthology.orchestra.run.smithi028.stdout: "inst_str": "client.5757 192.168.0.1:0/2518549734", 2023-11-28T11:10:18.511 INFO:teuthology.orchestra.run.smithi028.stdout: "addr_str": "192.168.0.1:0/2518549734", 2023-11-28T11:10:18.511 INFO:teuthology.orchestra.run.smithi028.stdout: "inode_count": 1, 2023-11-28T11:10:18.511 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_epoch": 17, 2023-11-28T11:10:18.511 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch": 40, 2023-11-28T11:10:18.512 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch_barrier": 0, 2023-11-28T11:10:18.512 INFO:teuthology.orchestra.run.smithi028.stdout: "blocklisted": false, 2023-11-28T11:10:18.512 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs" 2023-11-28T11:10:18.513 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:10:18.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:18.517 DEBUG:teuthology.orchestra.run.smithi028:> stat --file-system '--printf=%T 2023-11-28T11:10:18.517 DEBUG:teuthology.orchestra.run.smithi028:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:18.534 INFO:teuthology.orchestra.run.smithi028.stdout:fuseblk 2023-11-28T11:10:18.535 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:18.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:18.536 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:18.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:18.604 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir) 2023-11-28T11:10:18.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:18.668 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parentdir/childdir) 2023-11-28T11:10:18.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:18.732 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch rfile) 2023-11-28T11:10:18.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:18.793 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/pfile) 2023-11-28T11:10:18.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:18.857 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parentdir/childdir/cfile) 2023-11-28T11:10:18.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:18.918 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec find ./) 2023-11-28T11:10:18.985 INFO:teuthology.orchestra.run.smithi028.stdout:./ 2023-11-28T11:10:18.985 INFO:teuthology.orchestra.run.smithi028.stdout:./rfile 2023-11-28T11:10:18.985 INFO:teuthology.orchestra.run.smithi028.stdout:./parentdir 2023-11-28T11:10:18.985 INFO:teuthology.orchestra.run.smithi028.stdout:./parentdir/pfile 2023-11-28T11:10:18.985 INFO:teuthology.orchestra.run.smithi028.stdout:./parentdir/childdir 2023-11-28T11:10:18.985 INFO:teuthology.orchestra.run.smithi028.stdout:./parentdir/childdir/cfile 2023-11-28T11:10:18.986 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:10:18.986 DEBUG:teuthology.orchestra.run.smithi028:> import os 2023-11-28T11:10:18.986 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2023-11-28T11:10:18.986 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:10:18.986 DEBUG:teuthology.orchestra.run.smithi028:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./").st_ino) 2023-11-28T11:10:18.986 DEBUG:teuthology.orchestra.run.smithi028:> ' 2023-11-28T11:10:19.093 INFO:teuthology.orchestra.run.smithi028.stdout:1 2023-11-28T11:10:19.281 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:10:19.281 DEBUG:teuthology.orchestra.run.smithi028:> import os 2023-11-28T11:10:19.281 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2023-11-28T11:10:19.282 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:10:19.282 DEBUG:teuthology.orchestra.run.smithi028:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./rfile").st_ino) 2023-11-28T11:10:19.282 DEBUG:teuthology.orchestra.run.smithi028:> ' 2023-11-28T11:10:19.390 INFO:teuthology.orchestra.run.smithi028.stdout:1099511627778 2023-11-28T11:10:19.579 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:10:19.579 DEBUG:teuthology.orchestra.run.smithi028:> import os 2023-11-28T11:10:19.579 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2023-11-28T11:10:19.580 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:10:19.580 DEBUG:teuthology.orchestra.run.smithi028:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir").st_ino) 2023-11-28T11:10:19.580 DEBUG:teuthology.orchestra.run.smithi028:> ' 2023-11-28T11:10:19.687 INFO:teuthology.orchestra.run.smithi028.stdout:1099511627776 2023-11-28T11:10:19.876 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:10:19.876 DEBUG:teuthology.orchestra.run.smithi028:> import os 2023-11-28T11:10:19.876 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2023-11-28T11:10:19.876 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:10:19.877 DEBUG:teuthology.orchestra.run.smithi028:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/pfile").st_ino) 2023-11-28T11:10:19.877 DEBUG:teuthology.orchestra.run.smithi028:> ' 2023-11-28T11:10:19.986 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628279 2023-11-28T11:10:20.175 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:10:20.175 DEBUG:teuthology.orchestra.run.smithi028:> import os 2023-11-28T11:10:20.175 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2023-11-28T11:10:20.175 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:10:20.175 DEBUG:teuthology.orchestra.run.smithi028:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir").st_ino) 2023-11-28T11:10:20.175 DEBUG:teuthology.orchestra.run.smithi028:> ' 2023-11-28T11:10:20.286 INFO:teuthology.orchestra.run.smithi028.stdout:1099511627777 2023-11-28T11:10:20.474 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:10:20.474 DEBUG:teuthology.orchestra.run.smithi028:> import os 2023-11-28T11:10:20.475 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2023-11-28T11:10:20.475 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:10:20.475 DEBUG:teuthology.orchestra.run.smithi028:> print(os.stat("/home/ubuntu/cephtest/mnt.0/./parentdir/childdir/cfile").st_ino) 2023-11-28T11:10:20.475 DEBUG:teuthology.orchestra.run.smithi028:> ' 2023-11-28T11:10:20.586 INFO:teuthology.orchestra.run.smithi028.stdout:1099511628280 2023-11-28T11:10:20.775 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:10:20.775 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:20.834 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:20.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.997+0000 7f3b73f93700 1 Processor -- start 2023-11-28T11:10:20.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.997+0000 7f3b73f93700 1 -- start start 2023-11-28T11:10:20.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.998+0000 7f3b73f93700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b6c159590 0x7f3b6c06e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:20.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.998+0000 7f3b73f93700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3b6c15db70 0x7f3b6c0ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.998+0000 7f3b73f93700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b6c15d120 con 0x7f3b6c0ce890 2023-11-28T11:10:21.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.998+0000 7f3b73f93700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b6c15d6e0 con 0x7f3b6c15db70 2023-11-28T11:10:21.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.998+0000 7f3b73f93700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c0aef60 con 0x7f3b6c159590 2023-11-28T11:10:21.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.998+0000 7f3b71d2f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b6c159590 0x7f3b6c06e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.998+0000 7f3b7152e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3b6c15db70 0x7f3b6c0ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.998+0000 7f3b71d2f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b6c159590 0x7f3b6c06e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:55356/0 (socket says 172.21.15.28:55356) 2023-11-28T11:10:21.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.998+0000 7f3b71d2f700 1 -- 172.21.15.28:0/1319917801 learned_addr learned my addr 172.21.15.28:0/1319917801 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:21.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.999+0000 7f3b71d2f700 1 -- 172.21.15.28:0/1319917801 >> v2:172.21.15.59:3300/0 conn(0x7f3b6c15db70 msgr2=0x7f3b6c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:21.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.999+0000 7f3b71d2f700 1 --2- 172.21.15.28:0/1319917801 >> v2:172.21.15.59:3300/0 conn(0x7f3b6c15db70 0x7f3b6c0ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.999+0000 7f3b71d2f700 1 -- 172.21.15.28:0/1319917801 >> v1:172.21.15.59:6789/0 conn(0x7f3b6c0ce890 legacy=0x7f3b6c0caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.999+0000 7f3b71d2f700 1 -- 172.21.15.28:0/1319917801 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b6c0af240 con 0x7f3b6c159590 2023-11-28T11:10:21.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.999+0000 7f3b7152e700 1 --2- 172.21.15.28:0/1319917801 >> v2:172.21.15.59:3300/0 conn(0x7f3b6c15db70 0x7f3b6c0ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:21.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.999+0000 7f3b71d2f700 1 --2- 172.21.15.28:0/1319917801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b6c159590 0x7f3b6c06e400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3b680098f0 tx=0x7f3b68009c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9f07325ca7086e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:21.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.999+0000 7f3b70d2d700 1 -- 172.21.15.28:0/1319917801 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68011830 con 0x7f3b6c159590 2023-11-28T11:10:21.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.999+0000 7f3b70d2d700 1 -- 172.21.15.28:0/1319917801 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b68011990 con 0x7f3b6c159590 2023-11-28T11:10:21.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:20.999+0000 7f3b70d2d700 1 -- 172.21.15.28:0/1319917801 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68011d20 con 0x7f3b6c159590 2023-11-28T11:10:21.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.000+0000 7f3b73f93700 1 -- 172.21.15.28:0/1319917801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b6c159590 msgr2=0x7f3b6c06e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.000+0000 7f3b73f93700 1 --2- 172.21.15.28:0/1319917801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b6c159590 0x7f3b6c06e400 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3b680098f0 tx=0x7f3b68009c00 comp rx=0 tx=0).stop 2023-11-28T11:10:21.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.000+0000 7f3b73f93700 1 -- 172.21.15.28:0/1319917801 shutdown_connections 2023-11-28T11:10:21.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.000+0000 7f3b73f93700 1 --2- 172.21.15.28:0/1319917801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b6c159590 0x7f3b6c06e400 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.000+0000 7f3b73f93700 1 --2- 172.21.15.28:0/1319917801 >> v2:172.21.15.59:3300/0 conn(0x7f3b6c15db70 0x7f3b6c0ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.000+0000 7f3b73f93700 1 -- 172.21.15.28:0/1319917801 >> 172.21.15.28:0/1319917801 conn(0x7f3b6c0b1650 msgr2=0x7f3b6c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:21.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.000+0000 7f3b73f93700 1 -- 172.21.15.28:0/1319917801 shutdown_connections 2023-11-28T11:10:21.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.000+0000 7f3b73f93700 1 -- 172.21.15.28:0/1319917801 wait complete. 2023-11-28T11:10:21.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.001+0000 7f3b73f93700 1 Processor -- start 2023-11-28T11:10:21.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.001+0000 7f3b73f93700 1 -- start start 2023-11-28T11:10:21.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b73f93700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b6c0ce890 0x7f3b6c0c8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b73f93700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b6c159590 0x7f3b6c15bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b73f93700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b6c15db70 0x7f3b6c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b73f93700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c0c7a50 con 0x7f3b6c0ce890 2023-11-28T11:10:21.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b73f93700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c0c7d30 con 0x7f3b6c15db70 2023-11-28T11:10:21.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b73f93700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b6c0c8010 con 0x7f3b6c159590 2023-11-28T11:10:21.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b71d2f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b6c0ce890 0x7f3b6c0c8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b71d2f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b6c0ce890 0x7f3b6c0c8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49604/0 (socket says 172.21.15.28:49604) 2023-11-28T11:10:21.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b71d2f700 1 -- 172.21.15.28:0/2691466822 learned_addr learned my addr 172.21.15.28:0/2691466822 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:21.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b72530700 1 --2- 172.21.15.28:0/2691466822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b6c15db70 0x7f3b6c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b7152e700 1 --2- 172.21.15.28:0/2691466822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b6c159590 0x7f3b6c15bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b71d2f700 1 -- 172.21.15.28:0/2691466822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b6c159590 msgr2=0x7f3b6c15bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b71d2f700 1 --2- 172.21.15.28:0/2691466822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b6c159590 0x7f3b6c15bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b71d2f700 1 -- 172.21.15.28:0/2691466822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b6c15db70 msgr2=0x7f3b6c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b71d2f700 1 --2- 172.21.15.28:0/2691466822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b6c15db70 0x7f3b6c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.002+0000 7f3b71d2f700 1 -- 172.21.15.28:0/2691466822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b5c0094a0 con 0x7f3b6c0ce890 2023-11-28T11:10:21.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.003+0000 7f3b71d2f700 1 --2- 172.21.15.28:0/2691466822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b6c0ce890 0x7f3b6c0c8af0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009840 tx=0x7f3b68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:21.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.003+0000 7f3b627fc700 1 -- 172.21.15.28:0/2691466822 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b680036a0 con 0x7f3b6c0ce890 2023-11-28T11:10:21.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.003+0000 7f3b627fc700 1 -- 172.21.15.28:0/2691466822 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b68003800 con 0x7f3b6c0ce890 2023-11-28T11:10:21.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.003+0000 7f3b73f93700 1 -- 172.21.15.28:0/2691466822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b68009550 con 0x7f3b6c0ce890 2023-11-28T11:10:21.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.003+0000 7f3b627fc700 1 -- 172.21.15.28:0/2691466822 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b68003b70 con 0x7f3b6c0ce890 2023-11-28T11:10:21.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.003+0000 7f3b73f93700 1 -- 172.21.15.28:0/2691466822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b6c0aec70 con 0x7f3b6c0ce890 2023-11-28T11:10:21.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.004+0000 7f3b73f93700 1 -- 172.21.15.28:0/2691466822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b6c15d370 con 0x7f3b6c0ce890 2023-11-28T11:10:21.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.004+0000 7f3b627fc700 1 -- 172.21.15.28:0/2691466822 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3b6802d070 con 0x7f3b6c0ce890 2023-11-28T11:10:21.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.004+0000 7f3b627fc700 1 --2- 172.21.15.28:0/2691466822 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3b500682f0 0x7f3b5006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.005+0000 7f3b627fc700 1 -- 172.21.15.28:0/2691466822 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3b68021030 con 0x7f3b6c0ce890 2023-11-28T11:10:21.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.005+0000 7f3b7152e700 1 --2- 172.21.15.28:0/2691466822 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3b500682f0 0x7f3b5006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.005+0000 7f3b7152e700 1 --2- 172.21.15.28:0/2691466822 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3b500682f0 0x7f3b5006a7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0b55b0 tx=0x7f3b5c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:21.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.009+0000 7f3b627fc700 1 -- 172.21.15.28:0/2691466822 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3b68053ca0 con 0x7f3b6c0ce890 2023-11-28T11:10:21.058 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:10:21.060 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:10:21.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.205+0000 7f3b73f93700 1 -- 172.21.15.28:0/2691466822 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3b6c15dfe0 con 0x7f3b6c0ce890 2023-11-28T11:10:21.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.206+0000 7f3b627fc700 1 -- 172.21.15.28:0/2691466822 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+18022 (secure 0 0 0) 0x7f3b6802b090 con 0x7f3b6c0ce890 2023-11-28T11:10:21.208 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:21.208 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":40,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:10:11.822678+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:08.429278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:09.797038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.208+0000 7f3b73f93700 1 -- 172.21.15.28:0/2691466822 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3b500682f0 msgr2=0x7f3b5006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.208+0000 7f3b73f93700 1 --2- 172.21.15.28:0/2691466822 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3b500682f0 0x7f3b5006a7a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3b6c0b55b0 tx=0x7f3b5c008040 comp rx=0 tx=0).stop 2023-11-28T11:10:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.208+0000 7f3b73f93700 1 -- 172.21.15.28:0/2691466822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b6c0ce890 msgr2=0x7f3b6c0c8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.208+0000 7f3b73f93700 1 --2- 172.21.15.28:0/2691466822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b6c0ce890 0x7f3b6c0c8af0 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f3b68009840 tx=0x7f3b68004750 comp rx=0 tx=0).stop 2023-11-28T11:10:21.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.209+0000 7f3b73f93700 1 -- 172.21.15.28:0/2691466822 shutdown_connections 2023-11-28T11:10:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.209+0000 7f3b73f93700 1 --2- 172.21.15.28:0/2691466822 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3b500682f0 0x7f3b5006a7a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.209+0000 7f3b73f93700 1 --2- 172.21.15.28:0/2691466822 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3b6c0ce890 0x7f3b6c0c8af0 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.209+0000 7f3b73f93700 1 --2- 172.21.15.28:0/2691466822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3b6c15db70 0x7f3b6c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.209+0000 7f3b73f93700 1 --2- 172.21.15.28:0/2691466822 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3b6c159590 0x7f3b6c15bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.209+0000 7f3b73f93700 1 -- 172.21.15.28:0/2691466822 >> 172.21.15.28:0/2691466822 conn(0x7f3b6c0b1650 msgr2=0x7f3b6c0af480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.209+0000 7f3b73f93700 1 -- 172.21.15.28:0/2691466822 shutdown_connections 2023-11-28T11:10:21.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.209+0000 7f3b73f93700 1 -- 172.21.15.28:0/2691466822 wait complete. 2023-11-28T11:10:21.221 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:10:08.429278+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:10:09.797038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:10:21.222 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:10:21.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.349+0000 7f0733dc1700 1 Processor -- start 2023-11-28T11:10:21.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.349+0000 7f0733dc1700 1 -- start start 2023-11-28T11:10:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.350+0000 7f0733dc1700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f072c0cf5f0 0x7f072c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.350+0000 7f0733dc1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f072c0cb3e0 0x7f072c06afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.350+0000 7f0733dc1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f072c0ce550 con 0x7f072c0b1630 2023-11-28T11:10:21.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.350+0000 7f0733dc1700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f072c0ce830 con 0x7f072c0cf5f0 2023-11-28T11:10:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.350+0000 7f0733dc1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0ceb10 con 0x7f072c0cb3e0 2023-11-28T11:10:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.350+0000 7f073135c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f072c0cf5f0 0x7f072c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.350+0000 7f073135c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f072c0cf5f0 0x7f072c0caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49614/0 (socket says 172.21.15.28:49614) 2023-11-28T11:10:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.350+0000 7f073135c700 1 -- 172.21.15.28:0/4286087376 learned_addr learned my addr 172.21.15.28:0/4286087376 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.350+0000 7f073135c700 1 -- 172.21.15.28:0/4286087376 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f072c0cb3e0 msgr2=0x7f072c06afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.350+0000 7f073135c700 1 --2- 172.21.15.28:0/4286087376 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f072c0cb3e0 0x7f072c06afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.350+0000 7f073135c700 1 -- 172.21.15.28:0/4286087376 >> v1:172.21.15.59:6789/0 conn(0x7f072c0b1630 legacy=0x7f072c0b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.350+0000 7f073135c700 1 -- 172.21.15.28:0/4286087376 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f072c0cedf0 con 0x7f072c0cf5f0 2023-11-28T11:10:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.351+0000 7f073135c700 1 --2- 172.21.15.28:0/4286087376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f072c0cf5f0 0x7f072c0caea0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0728009920 tx=0x7f0728009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80844b94d8ed25cb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.351+0000 7f0730b5b700 1 -- 172.21.15.28:0/4286087376 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728004230 con 0x7f072c0cf5f0 2023-11-28T11:10:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.351+0000 7f0730b5b700 1 -- 172.21.15.28:0/4286087376 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0728004390 con 0x7f072c0cf5f0 2023-11-28T11:10:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.351+0000 7f0730b5b700 1 -- 172.21.15.28:0/4286087376 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0728009e70 con 0x7f072c0cf5f0 2023-11-28T11:10:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.351+0000 7f0733dc1700 1 -- 172.21.15.28:0/4286087376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f072c0cf5f0 msgr2=0x7f072c0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.351+0000 7f0733dc1700 1 --2- 172.21.15.28:0/4286087376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f072c0cf5f0 0x7f072c0caea0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f0728009920 tx=0x7f0728009c30 comp rx=0 tx=0).stop 2023-11-28T11:10:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.351+0000 7f0733dc1700 1 -- 172.21.15.28:0/4286087376 shutdown_connections 2023-11-28T11:10:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.351+0000 7f0733dc1700 1 --2- 172.21.15.28:0/4286087376 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f072c0cb3e0 0x7f072c06afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.351+0000 7f0733dc1700 1 --2- 172.21.15.28:0/4286087376 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f072c0cf5f0 0x7f072c0caea0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.351+0000 7f0733dc1700 1 -- 172.21.15.28:0/4286087376 >> 172.21.15.28:0/4286087376 conn(0x7f072c0cc630 msgr2=0x7f072c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.351+0000 7f0733dc1700 1 -- 172.21.15.28:0/4286087376 shutdown_connections 2023-11-28T11:10:21.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.351+0000 7f0733dc1700 1 -- 172.21.15.28:0/4286087376 wait complete. 2023-11-28T11:10:21.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.353+0000 7f0733dc1700 1 Processor -- start 2023-11-28T11:10:21.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.353+0000 7f0733dc1700 1 -- start start 2023-11-28T11:10:21.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.354+0000 7f0733dc1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f072c0b1630 0x7f072c15b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.354+0000 7f0733dc1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f072c0cb3e0 0x7f072c15f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.354+0000 7f0733dc1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f072c0cf5f0 0x7f072c0b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.354+0000 7f0733dc1700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c15bc00 con 0x7f072c0b1630 2023-11-28T11:10:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.354+0000 7f0733dc1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0c71f0 con 0x7f072c0cf5f0 2023-11-28T11:10:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.354+0000 7f0733dc1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072c0c74d0 con 0x7f072c0cb3e0 2023-11-28T11:10:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.354+0000 7f0731b5d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f072c0b1630 0x7f072c15b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.354+0000 7f0731b5d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f072c0b1630 0x7f072c15b6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49618/0 (socket says 172.21.15.28:49618) 2023-11-28T11:10:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.354+0000 7f0731b5d700 1 -- 172.21.15.28:0/1355664897 learned_addr learned my addr 172.21.15.28:0/1355664897 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:21.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.354+0000 7f073135c700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f072c0cb3e0 0x7f072c15f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.354+0000 7f073235e700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f072c0cf5f0 0x7f072c0b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.354+0000 7f0731b5d700 1 -- 172.21.15.28:0/1355664897 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f072c0cb3e0 msgr2=0x7f072c15f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.354+0000 7f0731b5d700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f072c0cb3e0 0x7f072c15f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.354+0000 7f0731b5d700 1 -- 172.21.15.28:0/1355664897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f072c0cf5f0 msgr2=0x7f072c0b3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.354+0000 7f0731b5d700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f072c0cf5f0 0x7f072c0b3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.355+0000 7f0731b5d700 1 -- 172.21.15.28:0/1355664897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0728009550 con 0x7f072c0b1630 2023-11-28T11:10:21.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.355+0000 7f073135c700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f072c0cb3e0 0x7f072c15f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.355+0000 7f073235e700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f072c0cf5f0 0x7f072c0b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.355+0000 7f0731b5d700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f072c0b1630 0x7f072c15b6c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f071c00c4a0 tx=0x7f071c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.355+0000 7f07227fc700 1 -- 172.21.15.28:0/1355664897 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c0054e0 con 0x7f072c0b1630 2023-11-28T11:10:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.355+0000 7f07227fc700 1 -- 172.21.15.28:0/1355664897 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f071c003680 con 0x7f072c0b1630 2023-11-28T11:10:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.355+0000 7f0733dc1700 1 -- 172.21.15.28:0/1355664897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f072c0ce140 con 0x7f072c0b1630 2023-11-28T11:10:21.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.355+0000 7f07227fc700 1 -- 172.21.15.28:0/1355664897 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f071c003a30 con 0x7f072c0b1630 2023-11-28T11:10:21.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.355+0000 7f0733dc1700 1 -- 172.21.15.28:0/1355664897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f072c0cf0d0 con 0x7f072c0b1630 2023-11-28T11:10:21.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.356+0000 7f0733dc1700 1 -- 172.21.15.28:0/1355664897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f072c0ce2d0 con 0x7f072c0b1630 2023-11-28T11:10:21.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.356+0000 7f07227fc700 1 -- 172.21.15.28:0/1355664897 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f071c019030 con 0x7f072c0b1630 2023-11-28T11:10:21.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.357+0000 7f07227fc700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f07100682f0 0x7f071006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.357+0000 7f07227fc700 1 -- 172.21.15.28:0/1355664897 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f071c0851d0 con 0x7f072c0b1630 2023-11-28T11:10:21.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.357+0000 7f073135c700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f07100682f0 0x7f071006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.357+0000 7f073135c700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f07100682f0 0x7f071006a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f072c15d010 tx=0x7f0728008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:21.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.361+0000 7f07227fc700 1 -- 172.21.15.28:0/1355664897 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f071c0518f0 con 0x7f072c0b1630 2023-11-28T11:10:21.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.552+0000 7f0733dc1700 1 -- 172.21.15.28:0/1355664897 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f072c0739f0 con 0x7f072c0b1630 2023-11-28T11:10:21.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.552+0000 7f07227fc700 1 -- 172.21.15.28:0/1355664897 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7f071c017020 con 0x7f072c0b1630 2023-11-28T11:10:21.553 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:10:21.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.554+0000 7f0733dc1700 1 -- 172.21.15.28:0/1355664897 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f07100682f0 msgr2=0x7f071006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.554+0000 7f0733dc1700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f07100682f0 0x7f071006a7a0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f072c15d010 tx=0x7f0728008030 comp rx=0 tx=0).stop 2023-11-28T11:10:21.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.554+0000 7f0733dc1700 1 -- 172.21.15.28:0/1355664897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f072c0b1630 msgr2=0x7f072c15b6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.554+0000 7f0733dc1700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f072c0b1630 0x7f072c15b6c0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f071c00c4a0 tx=0x7f071c00c860 comp rx=0 tx=0).stop 2023-11-28T11:10:21.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.554+0000 7f0733dc1700 1 -- 172.21.15.28:0/1355664897 shutdown_connections 2023-11-28T11:10:21.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.554+0000 7f0733dc1700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f07100682f0 0x7f071006a7a0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.554+0000 7f0733dc1700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f072c0b1630 0x7f072c15b6c0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.554+0000 7f0733dc1700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f072c0cf5f0 0x7f072c0b3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.554+0000 7f0733dc1700 1 --2- 172.21.15.28:0/1355664897 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f072c0cb3e0 0x7f072c15f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.554+0000 7f0733dc1700 1 -- 172.21.15.28:0/1355664897 >> 172.21.15.28:0/1355664897 conn(0x7f072c0cc630 msgr2=0x7f072c06af80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:21.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.554+0000 7f0733dc1700 1 -- 172.21.15.28:0/1355664897 shutdown_connections 2023-11-28T11:10:21.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.554+0000 7f0733dc1700 1 -- 172.21.15.28:0/1355664897 wait complete. 2023-11-28T11:10:21.567 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:10:21.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.693+0000 7f4229ba6700 1 Processor -- start 2023-11-28T11:10:21.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.693+0000 7f4229ba6700 1 -- start start 2023-11-28T11:10:21.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.694+0000 7f4229ba6700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f42240cf5f0 0x7f42240caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.694+0000 7f4229ba6700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42240ce550 con 0x7f42240cb3e0 2023-11-28T11:10:21.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.694+0000 7f4229ba6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42240ce830 con 0x7f42240b1630 2023-11-28T11:10:21.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.694+0000 7f4229ba6700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42240ceb10 con 0x7f42240cf5f0 2023-11-28T11:10:21.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.694+0000 7f4222ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f42240cf5f0 0x7f42240caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.694+0000 7f4222ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f42240cf5f0 0x7f42240caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49628/0 (socket says 172.21.15.28:49628) 2023-11-28T11:10:21.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.694+0000 7f4222ffd700 1 -- 172.21.15.28:0/3498997460 learned_addr learned my addr 172.21.15.28:0/3498997460 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:21.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.695+0000 7f4222ffd700 1 -- 172.21.15.28:0/3498997460 >> v1:172.21.15.59:6789/0 conn(0x7f42240b1630 legacy=0x7f42240b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:21.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.695+0000 7f4222ffd700 1 -- 172.21.15.28:0/3498997460 >> v1:172.21.15.28:6789/0 conn(0x7f42240cb3e0 legacy=0x7f422406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.695+0000 7f42227fc700 1 -- 172.21.15.28:0/3498997460 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990515715 0 0) 0x7f42240ce550 con 0x7f42240cb3e0 2023-11-28T11:10:21.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.695+0000 7f4222ffd700 1 -- 172.21.15.28:0/3498997460 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42240cedf0 con 0x7f42240cf5f0 2023-11-28T11:10:21.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.695+0000 7f4222ffd700 1 --2- 172.21.15.28:0/3498997460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42240cf5f0 0x7f42240caea0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f420c009920 tx=0x7f420c009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5450b501ac1ebd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.695+0000 7f42227fc700 1 -- 172.21.15.28:0/3498997460 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c004230 con 0x7f42240cf5f0 2023-11-28T11:10:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.695+0000 7f42227fc700 1 -- 172.21.15.28:0/3498997460 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c004390 con 0x7f42240cf5f0 2023-11-28T11:10:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.695+0000 7f42227fc700 1 -- 172.21.15.28:0/3498997460 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c009e70 con 0x7f42240cf5f0 2023-11-28T11:10:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.695+0000 7f4229ba6700 1 -- 172.21.15.28:0/3498997460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42240cf5f0 msgr2=0x7f42240caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.695+0000 7f4229ba6700 1 --2- 172.21.15.28:0/3498997460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42240cf5f0 0x7f42240caea0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f420c009920 tx=0x7f420c009c30 comp rx=0 tx=0).stop 2023-11-28T11:10:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.696+0000 7f4229ba6700 1 -- 172.21.15.28:0/3498997460 shutdown_connections 2023-11-28T11:10:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.696+0000 7f4229ba6700 1 --2- 172.21.15.28:0/3498997460 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42240cf5f0 0x7f42240caea0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.696+0000 7f4229ba6700 1 -- 172.21.15.28:0/3498997460 >> 172.21.15.28:0/3498997460 conn(0x7f42240cc630 msgr2=0x7f42240cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.696+0000 7f4229ba6700 1 -- 172.21.15.28:0/3498997460 shutdown_connections 2023-11-28T11:10:21.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.696+0000 7f4229ba6700 1 -- 172.21.15.28:0/3498997460 wait complete. 2023-11-28T11:10:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.697+0000 7f4229ba6700 1 Processor -- start 2023-11-28T11:10:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.697+0000 7f4229ba6700 1 -- start start 2023-11-28T11:10:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4229ba6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42240b1630 0x7f422415bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4229ba6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42240cb3e0 0x7f42240c85d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4229ba6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42240cf5f0 0x7f42240c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4229ba6700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42240c9a00 con 0x7f42240cb3e0 2023-11-28T11:10:21.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4229ba6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42240c7830 con 0x7f42240b1630 2023-11-28T11:10:21.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4229ba6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42240c7b10 con 0x7f42240cf5f0 2023-11-28T11:10:21.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4222ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42240cb3e0 0x7f42240c85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4222ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42240cb3e0 0x7f42240c85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49634/0 (socket says 172.21.15.28:49634) 2023-11-28T11:10:21.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4222ffd700 1 -- 172.21.15.28:0/1073801790 learned_addr learned my addr 172.21.15.28:0/1073801790 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:21.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4222ffd700 1 -- 172.21.15.28:0/1073801790 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42240cf5f0 msgr2=0x7f42240c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f42237fe700 1 --2- 172.21.15.28:0/1073801790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42240b1630 0x7f422415bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4223fff700 1 --2- 172.21.15.28:0/1073801790 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42240cf5f0 0x7f42240c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4222ffd700 1 --2- 172.21.15.28:0/1073801790 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42240cf5f0 0x7f42240c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4222ffd700 1 -- 172.21.15.28:0/1073801790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42240b1630 msgr2=0x7f422415bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4222ffd700 1 --2- 172.21.15.28:0/1073801790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42240b1630 0x7f422415bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4222ffd700 1 -- 172.21.15.28:0/1073801790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f420c009550 con 0x7f42240cb3e0 2023-11-28T11:10:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f42237fe700 1 --2- 172.21.15.28:0/1073801790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42240b1630 0x7f422415bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f4222ffd700 1 --2- 172.21.15.28:0/1073801790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42240cb3e0 0x7f42240c85d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f420c009870 tx=0x7f420c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.698+0000 7f421bfff700 1 -- 172.21.15.28:0/1073801790 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c0038a0 con 0x7f42240cb3e0 2023-11-28T11:10:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.699+0000 7f421bfff700 1 -- 172.21.15.28:0/1073801790 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f420c003a00 con 0x7f42240cb3e0 2023-11-28T11:10:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.699+0000 7f4229ba6700 1 -- 172.21.15.28:0/1073801790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42240ce140 con 0x7f42240cb3e0 2023-11-28T11:10:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.699+0000 7f421bfff700 1 -- 172.21.15.28:0/1073801790 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f420c003db0 con 0x7f42240cb3e0 2023-11-28T11:10:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.699+0000 7f4229ba6700 1 -- 172.21.15.28:0/1073801790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42240cf0d0 con 0x7f42240cb3e0 2023-11-28T11:10:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.700+0000 7f4229ba6700 1 -- 172.21.15.28:0/1073801790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42240ce2d0 con 0x7f42240cb3e0 2023-11-28T11:10:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.700+0000 7f421bfff700 1 -- 172.21.15.28:0/1073801790 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f420c017020 con 0x7f42240cb3e0 2023-11-28T11:10:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.700+0000 7f421bfff700 1 --2- 172.21.15.28:0/1073801790 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42040682f0 0x7f420406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.701+0000 7f421bfff700 1 -- 172.21.15.28:0/1073801790 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f420c0876e0 con 0x7f42240cb3e0 2023-11-28T11:10:21.703 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.701+0000 7f42237fe700 1 --2- 172.21.15.28:0/1073801790 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42040682f0 0x7f420406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:21.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.701+0000 7f42237fe700 1 --2- 172.21.15.28:0/1073801790 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42040682f0 0x7f420406a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f42240c9450 tx=0x7f4214009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:21.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.705+0000 7f421bfff700 1 -- 172.21.15.28:0/1073801790 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f420c053e00 con 0x7f42240cb3e0 2023-11-28T11:10:21.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.897+0000 7f4229ba6700 1 -- 172.21.15.28:0/1073801790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f42240739f0 con 0x7f42240cb3e0 2023-11-28T11:10:21.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.898+0000 7f421bfff700 1 -- 172.21.15.28:0/1073801790 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f420c057420 con 0x7f42240cb3e0 2023-11-28T11:10:21.899 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:10:21.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.900+0000 7f4229ba6700 1 -- 172.21.15.28:0/1073801790 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42040682f0 msgr2=0x7f420406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.900+0000 7f4229ba6700 1 --2- 172.21.15.28:0/1073801790 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42040682f0 0x7f420406a7a0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f42240c9450 tx=0x7f4214009040 comp rx=0 tx=0).stop 2023-11-28T11:10:21.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.900+0000 7f4229ba6700 1 -- 172.21.15.28:0/1073801790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42240cb3e0 msgr2=0x7f42240c85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:21.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.900+0000 7f4229ba6700 1 --2- 172.21.15.28:0/1073801790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42240cb3e0 0x7f42240c85d0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f420c009870 tx=0x7f420c004750 comp rx=0 tx=0).stop 2023-11-28T11:10:21.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.900+0000 7f4229ba6700 1 -- 172.21.15.28:0/1073801790 shutdown_connections 2023-11-28T11:10:21.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.900+0000 7f4229ba6700 1 --2- 172.21.15.28:0/1073801790 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42040682f0 0x7f420406a7a0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.900+0000 7f4229ba6700 1 --2- 172.21.15.28:0/1073801790 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42240cf5f0 0x7f42240c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.900+0000 7f4229ba6700 1 --2- 172.21.15.28:0/1073801790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42240b1630 0x7f422415bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.900+0000 7f4229ba6700 1 --2- 172.21.15.28:0/1073801790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42240cb3e0 0x7f42240c85d0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:21.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.900+0000 7f4229ba6700 1 -- 172.21.15.28:0/1073801790 >> 172.21.15.28:0/1073801790 conn(0x7f42240cc630 msgr2=0x7f42240adae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:21.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.900+0000 7f4229ba6700 1 -- 172.21.15.28:0/1073801790 shutdown_connections 2023-11-28T11:10:21.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:21.900+0000 7f4229ba6700 1 -- 172.21.15.28:0/1073801790 wait complete. 2023-11-28T11:10:21.912 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:10:22.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.039+0000 7f68648e2700 1 Processor -- start 2023-11-28T11:10:22.040 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.039+0000 7f68648e2700 1 -- start start 2023-11-28T11:10:22.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.040+0000 7f68648e2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f68600b2d90 0x7f68600ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.040+0000 7f68648e2700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68600cd2a0 con 0x7f68600cb630 2023-11-28T11:10:22.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.040+0000 7f68648e2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68600d0400 con 0x7f68600b62d0 2023-11-28T11:10:22.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.040+0000 7f68648e2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68600d06e0 con 0x7f68600b2d90 2023-11-28T11:10:22.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.040+0000 7f685dd9b700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f68600cb630 0x7f68600b2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37700/0 (socket says 172.21.15.28:37700) 2023-11-28T11:10:22.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.040+0000 7f685dd9b700 1 -- 172.21.15.28:0/551734365 learned_addr learned my addr 172.21.15.28:0/551734365 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:22.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.040+0000 7f685ed9d700 1 --2- 172.21.15.28:0/551734365 >> v2:172.21.15.59:3300/0 conn(0x7f68600b2d90 0x7f68600ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.040+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351905576 0 0) 0x7f68600cd2a0 con 0x7f68600cb630 2023-11-28T11:10:22.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.040+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68480035b0 con 0x7f68600cb630 2023-11-28T11:10:22.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1693250081 0 0) 0x7f68600d0400 con 0x7f68600b62d0 2023-11-28T11:10:22.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68600cd2a0 con 0x7f68600b62d0 2023-11-28T11:10:22.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3507987671 0 0) 0x7f68480035b0 con 0x7f68600cb630 2023-11-28T11:10:22.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 >> v2:172.21.15.59:3300/0 conn(0x7f68600b2d90 msgr2=0x7f68600ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:22.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 --2- 172.21.15.28:0/551734365 >> v2:172.21.15.59:3300/0 conn(0x7f68600b2d90 0x7f68600ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.046 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 >> v1:172.21.15.59:6789/0 conn(0x7f68600b62d0 legacy=0x7f686006f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.046 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68600cffb0 con 0x7f68600cb630 2023-11-28T11:10:22.046 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f68500015f0 con 0x7f68600cb630 2023-11-28T11:10:22.046 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 >> v1:172.21.15.28:6789/0 conn(0x7f68600cb630 legacy=0x7f68600b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 --2- 172.21.15.28:0/551734365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6848003be0 0x7f6848003fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 --2- 172.21.15.28:0/551734365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6848004850 0x7f6848004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 --2- 172.21.15.28:0/551734365 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6848005200 0x7f684800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68600d06e0 con 0x7f6848003be0 2023-11-28T11:10:22.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f684800c0e0 con 0x7f6848004850 2023-11-28T11:10:22.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f684800c270 con 0x7f6848005200 2023-11-28T11:10:22.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685e59c700 1 --2- 172.21.15.28:0/551734365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6848003be0 0x7f6848003fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685dd9b700 1 --2- 172.21.15.28:0/551734365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6848004850 0x7f6848004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685ed9d700 1 --2- 172.21.15.28:0/551734365 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6848005200 0x7f684800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685e59c700 1 -- 172.21.15.28:0/551734365 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6848005200 msgr2=0x7f684800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685e59c700 1 --2- 172.21.15.28:0/551734365 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6848005200 0x7f684800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685e59c700 1 -- 172.21.15.28:0/551734365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6848004850 msgr2=0x7f6848004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685e59c700 1 --2- 172.21.15.28:0/551734365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6848004850 0x7f6848004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685e59c700 1 -- 172.21.15.28:0/551734365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68480035b0 con 0x7f6848003be0 2023-11-28T11:10:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685ed9d700 1 -- 172.21.15.28:0/551734365 reap_dead start 2023-11-28T11:10:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685e59c700 1 --2- 172.21.15.28:0/551734365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6848003be0 0x7f6848003fd0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6854007fd0 tx=0x7f6848005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13548e095b2040ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.041+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854001430 con 0x7f6848003be0 2023-11-28T11:10:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.042+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68540052f0 con 0x7f6848003be0 2023-11-28T11:10:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.042+0000 7f685d59a700 1 -- 172.21.15.28:0/551734365 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854001d40 con 0x7f6848003be0 2023-11-28T11:10:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.042+0000 7f68648e2700 1 -- 172.21.15.28:0/551734365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6848003be0 msgr2=0x7f6848003fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.042+0000 7f68648e2700 1 --2- 172.21.15.28:0/551734365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6848003be0 0x7f6848003fd0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f6854007fd0 tx=0x7f6848005880 comp rx=0 tx=0).stop 2023-11-28T11:10:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.042+0000 7f68648e2700 1 -- 172.21.15.28:0/551734365 shutdown_connections 2023-11-28T11:10:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.042+0000 7f68648e2700 1 --2- 172.21.15.28:0/551734365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6848003be0 0x7f6848003fd0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.042+0000 7f68648e2700 1 -- 172.21.15.28:0/551734365 >> 172.21.15.28:0/551734365 conn(0x7f68600cd680 msgr2=0x7f68600cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.042+0000 7f68648e2700 1 -- 172.21.15.28:0/551734365 shutdown_connections 2023-11-28T11:10:22.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.042+0000 7f68648e2700 1 -- 172.21.15.28:0/551734365 wait complete. 2023-11-28T11:10:22.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.044+0000 7f68648e2700 1 Processor -- start 2023-11-28T11:10:22.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.044+0000 7f68648e2700 1 -- start start 2023-11-28T11:10:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f68648e2700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6848003be0 0x7f6860156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f68648e2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68601540e0 0x7f68601f3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f68648e2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68601f3b90 0x7f68600c9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f68648e2700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68601f4030 con 0x7f6848003be0 2023-11-28T11:10:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f68648e2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6860157230 con 0x7f68601540e0 2023-11-28T11:10:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f68648e2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68600c8e50 con 0x7f68601f3b90 2023-11-28T11:10:22.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f685e59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6848003be0 0x7f6860156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f685e59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6848003be0 0x7f6860156690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49652/0 (socket says 172.21.15.28:49652) 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f685e59c700 1 -- 172.21.15.28:0/784005696 learned_addr learned my addr 172.21.15.28:0/784005696 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f685ed9d700 1 --2- 172.21.15.28:0/784005696 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68601f3b90 0x7f68600c9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f685dd9b700 1 --2- 172.21.15.28:0/784005696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68601540e0 0x7f68601f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f685e59c700 1 -- 172.21.15.28:0/784005696 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68601f3b90 msgr2=0x7f68600c9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f685e59c700 1 --2- 172.21.15.28:0/784005696 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68601f3b90 0x7f68600c9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f685e59c700 1 -- 172.21.15.28:0/784005696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68601540e0 msgr2=0x7f68601f3650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f685e59c700 1 --2- 172.21.15.28:0/784005696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68601540e0 0x7f68601f3650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f685e59c700 1 -- 172.21.15.28:0/784005696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68600cce50 con 0x7f6848003be0 2023-11-28T11:10:22.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.045+0000 7f685dd9b700 1 --2- 172.21.15.28:0/784005696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68601540e0 0x7f68601f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.046+0000 7f685e59c700 1 --2- 172.21.15.28:0/784005696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6848003be0 0x7f6860156690 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6854005040 tx=0x7f68540012d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.046+0000 7f684effd700 1 -- 172.21.15.28:0/784005696 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854006b30 con 0x7f6848003be0 2023-11-28T11:10:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.046+0000 7f684effd700 1 -- 172.21.15.28:0/784005696 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6854006c90 con 0x7f6848003be0 2023-11-28T11:10:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.046+0000 7f68648e2700 1 -- 172.21.15.28:0/784005696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68600b66a0 con 0x7f6848003be0 2023-11-28T11:10:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.046+0000 7f684effd700 1 -- 172.21.15.28:0/784005696 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854007040 con 0x7f6848003be0 2023-11-28T11:10:22.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.046+0000 7f68648e2700 1 -- 172.21.15.28:0/784005696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68600ccfe0 con 0x7f6848003be0 2023-11-28T11:10:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.047+0000 7f68648e2700 1 -- 172.21.15.28:0/784005696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68600c9460 con 0x7f6848003be0 2023-11-28T11:10:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.047+0000 7f684effd700 1 -- 172.21.15.28:0/784005696 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f685402d070 con 0x7f6848003be0 2023-11-28T11:10:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.048+0000 7f684effd700 1 --2- 172.21.15.28:0/784005696 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68400682f0 0x7f684006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.048+0000 7f685dd9b700 1 --2- 172.21.15.28:0/784005696 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68400682f0 0x7f684006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.048+0000 7f684effd700 1 -- 172.21.15.28:0/784005696 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f68540868b0 con 0x7f6848003be0 2023-11-28T11:10:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.048+0000 7f685dd9b700 1 --2- 172.21.15.28:0/784005696 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68400682f0 0x7f684006a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6850000c00 tx=0x7f6850009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.052+0000 7f684effd700 1 -- 172.21.15.28:0/784005696 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6854052f50 con 0x7f6848003be0 2023-11-28T11:10:22.253 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.252+0000 7f68648e2700 1 -- 172.21.15.28:0/784005696 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f68600739f0 con 0x7f6848003be0 2023-11-28T11:10:22.253 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.252+0000 7f684effd700 1 -- 172.21.15.28:0/784005696 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6854021020 con 0x7f6848003be0 2023-11-28T11:10:22.254 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:10:22.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.254+0000 7f68648e2700 1 -- 172.21.15.28:0/784005696 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68400682f0 msgr2=0x7f684006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.254+0000 7f68648e2700 1 --2- 172.21.15.28:0/784005696 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68400682f0 0x7f684006a7a0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f6850000c00 tx=0x7f6850009040 comp rx=0 tx=0).stop 2023-11-28T11:10:22.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.254+0000 7f68648e2700 1 -- 172.21.15.28:0/784005696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6848003be0 msgr2=0x7f6860156690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.254+0000 7f68648e2700 1 --2- 172.21.15.28:0/784005696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6848003be0 0x7f6860156690 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f6854005040 tx=0x7f68540012d0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.254+0000 7f68648e2700 1 -- 172.21.15.28:0/784005696 shutdown_connections 2023-11-28T11:10:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.254+0000 7f68648e2700 1 --2- 172.21.15.28:0/784005696 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f68400682f0 0x7f684006a7a0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.254+0000 7f68648e2700 1 --2- 172.21.15.28:0/784005696 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6848003be0 0x7f6860156690 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.254+0000 7f68648e2700 1 --2- 172.21.15.28:0/784005696 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f68601f3b90 0x7f68600c9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.254+0000 7f68648e2700 1 --2- 172.21.15.28:0/784005696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f68601540e0 0x7f68601f3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.254+0000 7f68648e2700 1 -- 172.21.15.28:0/784005696 >> 172.21.15.28:0/784005696 conn(0x7f68600cd680 msgr2=0x7f686015c630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:22.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.255+0000 7f68648e2700 1 -- 172.21.15.28:0/784005696 shutdown_connections 2023-11-28T11:10:22.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.255+0000 7f68648e2700 1 -- 172.21.15.28:0/784005696 wait complete. 2023-11-28T11:10:22.267 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:10:22.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.394+0000 7f75361cb700 1 Processor -- start 2023-11-28T11:10:22.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.394+0000 7f75361cb700 1 -- start start 2023-11-28T11:10:22.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.395+0000 7f75361cb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75300ca360 0x7f753006f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.395+0000 7f75361cb700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f75300cae50 0x7f75300cb2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.395+0000 7f75361cb700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75300caab0 con 0x7f753015c950 2023-11-28T11:10:22.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.395+0000 7f75361cb700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75300cb810 con 0x7f75300cae50 2023-11-28T11:10:22.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.395+0000 7f75361cb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753015cd20 con 0x7f75300ca360 2023-11-28T11:10:22.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.395+0000 7f752ffff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f75300cae50 0x7f75300cb2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.395+0000 7f752ffff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f75300cae50 0x7f75300cb2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49664/0 (socket says 172.21.15.28:49664) 2023-11-28T11:10:22.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.395+0000 7f752ffff700 1 -- 172.21.15.28:0/1412309617 learned_addr learned my addr 172.21.15.28:0/1412309617 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:22.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.395+0000 7f752effd700 1 --2- 172.21.15.28:0/1412309617 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75300ca360 0x7f753006f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.395+0000 7f752ffff700 1 -- 172.21.15.28:0/1412309617 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75300ca360 msgr2=0x7f753006f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:22.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.395+0000 7f752ffff700 1 --2- 172.21.15.28:0/1412309617 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75300ca360 0x7f753006f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.395+0000 7f752ffff700 1 -- 172.21.15.28:0/1412309617 >> v1:172.21.15.59:6789/0 conn(0x7f753015c950 legacy=0x7f753006a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:22.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.395+0000 7f752ffff700 1 -- 172.21.15.28:0/1412309617 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753006e770 con 0x7f75300cae50 2023-11-28T11:10:22.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.396+0000 7f752ffff700 1 --2- 172.21.15.28:0/1412309617 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75300cae50 0x7f75300cb2d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f752800d150 tx=0x7f752800d460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e7215a24e0d1b6a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.396+0000 7f752e7fc700 1 -- 172.21.15.28:0/1412309617 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7528010070 con 0x7f75300cae50 2023-11-28T11:10:22.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.396+0000 7f752e7fc700 1 -- 172.21.15.28:0/1412309617 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f752800dd80 con 0x7f75300cae50 2023-11-28T11:10:22.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.396+0000 7f752e7fc700 1 -- 172.21.15.28:0/1412309617 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f752800b310 con 0x7f75300cae50 2023-11-28T11:10:22.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.396+0000 7f75361cb700 1 -- 172.21.15.28:0/1412309617 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75300cae50 msgr2=0x7f75300cb2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.396+0000 7f75361cb700 1 --2- 172.21.15.28:0/1412309617 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75300cae50 0x7f75300cb2d0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f752800d150 tx=0x7f752800d460 comp rx=0 tx=0).stop 2023-11-28T11:10:22.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.396+0000 7f75361cb700 1 -- 172.21.15.28:0/1412309617 shutdown_connections 2023-11-28T11:10:22.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.396+0000 7f75361cb700 1 --2- 172.21.15.28:0/1412309617 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75300cae50 0x7f75300cb2d0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.396+0000 7f75361cb700 1 --2- 172.21.15.28:0/1412309617 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75300ca360 0x7f753006f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.396+0000 7f75361cb700 1 -- 172.21.15.28:0/1412309617 >> 172.21.15.28:0/1412309617 conn(0x7f75300b0df0 msgr2=0x7f75300b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:22.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.396+0000 7f75361cb700 1 -- 172.21.15.28:0/1412309617 shutdown_connections 2023-11-28T11:10:22.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.396+0000 7f75361cb700 1 -- 172.21.15.28:0/1412309617 wait complete. 2023-11-28T11:10:22.401 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.398+0000 7f75361cb700 1 Processor -- start 2023-11-28T11:10:22.401 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.398+0000 7f75361cb700 1 -- start start 2023-11-28T11:10:22.401 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.399+0000 7f75361cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75300ca360 0x7f753014cd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.401 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.399+0000 7f75361cb700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75300cae50 0x7f75301580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.399+0000 7f75361cb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f753015c950 0x7f7530159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.399+0000 7f75361cb700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7530153890 con 0x7f75300cae50 2023-11-28T11:10:22.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.399+0000 7f75361cb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7530158a90 con 0x7f75300ca360 2023-11-28T11:10:22.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.399+0000 7f75361cb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75301543c0 con 0x7f753015c950 2023-11-28T11:10:22.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.399+0000 7f752effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75300cae50 0x7f75301580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.399+0000 7f752effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75300cae50 0x7f75301580b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49668/0 (socket says 172.21.15.28:49668) 2023-11-28T11:10:22.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.399+0000 7f752effd700 1 -- 172.21.15.28:0/1318457180 learned_addr learned my addr 172.21.15.28:0/1318457180 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:22.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.400+0000 7f752effd700 1 -- 172.21.15.28:0/1318457180 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f753015c950 msgr2=0x7f7530159f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.400+0000 7f752effd700 1 --2- 172.21.15.28:0/1318457180 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f753015c950 0x7f7530159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.400+0000 7f752effd700 1 -- 172.21.15.28:0/1318457180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75300ca360 msgr2=0x7f753014cd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.400+0000 7f752effd700 1 --2- 172.21.15.28:0/1318457180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75300ca360 0x7f753014cd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.400+0000 7f752effd700 1 -- 172.21.15.28:0/1318457180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f752800bd40 con 0x7f75300cae50 2023-11-28T11:10:22.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.400+0000 7f752effd700 1 --2- 172.21.15.28:0/1318457180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75300cae50 0x7f75301580b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f751800e810 tx=0x7f751800ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.400+0000 7f7527fff700 1 -- 172.21.15.28:0/1318457180 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518013610 con 0x7f75300cae50 2023-11-28T11:10:22.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.400+0000 7f7527fff700 1 -- 172.21.15.28:0/1318457180 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7518013770 con 0x7f75300cae50 2023-11-28T11:10:22.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.400+0000 7f75361cb700 1 -- 172.21.15.28:0/1318457180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753006ea10 con 0x7f75300cae50 2023-11-28T11:10:22.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.400+0000 7f7527fff700 1 -- 172.21.15.28:0/1318457180 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7518013b20 con 0x7f75300cae50 2023-11-28T11:10:22.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.400+0000 7f75361cb700 1 -- 172.21.15.28:0/1318457180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75300ca7d0 con 0x7f75300cae50 2023-11-28T11:10:22.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.401+0000 7f75361cb700 1 -- 172.21.15.28:0/1318457180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753006ecf0 con 0x7f75300cae50 2023-11-28T11:10:22.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.401+0000 7f7527fff700 1 -- 172.21.15.28:0/1318457180 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7518023070 con 0x7f75300cae50 2023-11-28T11:10:22.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.402+0000 7f7527fff700 1 --2- 172.21.15.28:0/1318457180 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75100682f0 0x7f751006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.402+0000 7f7527fff700 1 -- 172.21.15.28:0/1318457180 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7518085ae0 con 0x7f75300cae50 2023-11-28T11:10:22.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.402+0000 7f752f7fe700 1 --2- 172.21.15.28:0/1318457180 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75100682f0 0x7f751006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.402+0000 7f752f7fe700 1 --2- 172.21.15.28:0/1318457180 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75100682f0 0x7f751006a7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f75200096f0 tx=0x7f7520006c50 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.406+0000 7f7527fff700 1 -- 172.21.15.28:0/1318457180 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7518052180 con 0x7f75300cae50 2023-11-28T11:10:22.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.598+0000 7f75361cb700 1 -- 172.21.15.28:0/1318457180 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f753006fcb0 con 0x7f75300cae50 2023-11-28T11:10:22.599 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.599+0000 7f7527fff700 1 -- 172.21.15.28:0/1318457180 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f75180557a0 con 0x7f75300cae50 2023-11-28T11:10:22.599 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:10:22.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.601+0000 7f75361cb700 1 -- 172.21.15.28:0/1318457180 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75100682f0 msgr2=0x7f751006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.601+0000 7f75361cb700 1 --2- 172.21.15.28:0/1318457180 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75100682f0 0x7f751006a7a0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f75200096f0 tx=0x7f7520006c50 comp rx=0 tx=0).stop 2023-11-28T11:10:22.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.601+0000 7f75361cb700 1 -- 172.21.15.28:0/1318457180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75300cae50 msgr2=0x7f75301580b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.601+0000 7f75361cb700 1 --2- 172.21.15.28:0/1318457180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75300cae50 0x7f75301580b0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f751800e810 tx=0x7f751800ebd0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.601+0000 7f75361cb700 1 -- 172.21.15.28:0/1318457180 shutdown_connections 2023-11-28T11:10:22.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.601+0000 7f75361cb700 1 --2- 172.21.15.28:0/1318457180 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75100682f0 0x7f751006a7a0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.601+0000 7f75361cb700 1 --2- 172.21.15.28:0/1318457180 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f753015c950 0x7f7530159f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.601+0000 7f75361cb700 1 --2- 172.21.15.28:0/1318457180 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75300ca360 0x7f753014cd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.601+0000 7f75361cb700 1 --2- 172.21.15.28:0/1318457180 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75300cae50 0x7f75301580b0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.601+0000 7f75361cb700 1 -- 172.21.15.28:0/1318457180 >> 172.21.15.28:0/1318457180 conn(0x7f75300b0df0 msgr2=0x7f7530150ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:22.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.601+0000 7f75361cb700 1 -- 172.21.15.28:0/1318457180 shutdown_connections 2023-11-28T11:10:22.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.601+0000 7f75361cb700 1 -- 172.21.15.28:0/1318457180 wait complete. 2023-11-28T11:10:22.613 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:10:22.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.738+0000 7f75a0f26700 1 Processor -- start 2023-11-28T11:10:22.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.738+0000 7f75a0f26700 1 -- start start 2023-11-28T11:10:22.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f75a0f26700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f759c15c560 0x7f759c06f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f75a0f26700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f759c0b2660 0x7f759c0ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f75a0f26700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f759c0b30b0 con 0x7f759c0cecf0 2023-11-28T11:10:22.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f75a0f26700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f759c06c020 con 0x7f759c15c560 2023-11-28T11:10:22.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f75a0f26700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0cf110 con 0x7f759c0b2660 2023-11-28T11:10:22.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f759a59c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f759c15c560 0x7f759c06f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f759ad9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f759c0cecf0 0x7f759c160200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37736/0 (socket says 172.21.15.28:37736) 2023-11-28T11:10:22.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f759a59c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f759c15c560 0x7f759c06f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49674/0 (socket says 172.21.15.28:49674) 2023-11-28T11:10:22.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f759ad9d700 1 -- 172.21.15.28:0/1046292977 learned_addr learned my addr 172.21.15.28:0/1046292977 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:22.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f7599d9b700 1 --2- 172.21.15.28:0/1046292977 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f759c0b2660 0x7f759c0ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f759a59c700 1 -- 172.21.15.28:0/1046292977 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f759c0b2660 msgr2=0x7f759c0ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:22.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f759a59c700 1 --2- 172.21.15.28:0/1046292977 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f759c0b2660 0x7f759c0ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f759a59c700 1 -- 172.21.15.28:0/1046292977 >> v1:172.21.15.28:6789/0 conn(0x7f759c0cecf0 legacy=0x7f759c160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f759a59c700 1 -- 172.21.15.28:0/1046292977 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f759c0d09c0 con 0x7f759c15c560 2023-11-28T11:10:22.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f759959a700 1 -- 172.21.15.28:0/1046292977 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584897187 0 0) 0x7f759c0b30b0 con 0x7f759c0cecf0 2023-11-28T11:10:22.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f759a59c700 1 --2- 172.21.15.28:0/1046292977 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f759c15c560 0x7f759c06f8b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f758c009840 tx=0x7f758c009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefbf5d783a51779 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f759959a700 1 -- 172.21.15.28:0/1046292977 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c011830 con 0x7f759c15c560 2023-11-28T11:10:22.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f759959a700 1 -- 172.21.15.28:0/1046292977 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758c011990 con 0x7f759c15c560 2023-11-28T11:10:22.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.739+0000 7f759959a700 1 -- 172.21.15.28:0/1046292977 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c011d20 con 0x7f759c15c560 2023-11-28T11:10:22.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.740+0000 7f75a0f26700 1 -- 172.21.15.28:0/1046292977 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f759c15c560 msgr2=0x7f759c06f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.740+0000 7f75a0f26700 1 --2- 172.21.15.28:0/1046292977 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f759c15c560 0x7f759c06f8b0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f758c009840 tx=0x7f758c009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:22.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.740+0000 7f75a0f26700 1 -- 172.21.15.28:0/1046292977 shutdown_connections 2023-11-28T11:10:22.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.740+0000 7f75a0f26700 1 --2- 172.21.15.28:0/1046292977 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f759c15c560 0x7f759c06f8b0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.740+0000 7f75a0f26700 1 --2- 172.21.15.28:0/1046292977 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f759c0b2660 0x7f759c0ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.740+0000 7f75a0f26700 1 -- 172.21.15.28:0/1046292977 >> 172.21.15.28:0/1046292977 conn(0x7f759c0cd660 msgr2=0x7f759c0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:22.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.740+0000 7f75a0f26700 1 -- 172.21.15.28:0/1046292977 shutdown_connections 2023-11-28T11:10:22.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.740+0000 7f75a0f26700 1 -- 172.21.15.28:0/1046292977 wait complete. 2023-11-28T11:10:22.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.741+0000 7f75a0f26700 1 Processor -- start 2023-11-28T11:10:22.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.741+0000 7f75a0f26700 1 -- start start 2023-11-28T11:10:22.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.742+0000 7f75a0f26700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f759c0b2660 0x7f759c0c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.742+0000 7f75a0f26700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f759c0cecf0 0x7f759c1f2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.742+0000 7f75a0f26700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f759c15c560 0x7f759c0ad710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.742+0000 7f75a0f26700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0c7a80 con 0x7f759c0b2660 2023-11-28T11:10:22.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.742+0000 7f75a0f26700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0c48b0 con 0x7f759c0cecf0 2023-11-28T11:10:22.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.742+0000 7f75a0f26700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f759c0c4b90 con 0x7f759c15c560 2023-11-28T11:10:22.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.742+0000 7f759a59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f759c0b2660 0x7f759c0c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.742+0000 7f759a59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f759c0b2660 0x7f759c0c7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49680/0 (socket says 172.21.15.28:49680) 2023-11-28T11:10:22.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.742+0000 7f759a59c700 1 -- 172.21.15.28:0/1422505322 learned_addr learned my addr 172.21.15.28:0/1422505322 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:22.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.743+0000 7f759a59c700 1 -- 172.21.15.28:0/1422505322 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f759c15c560 msgr2=0x7f759c0ad710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.743+0000 7f759a59c700 1 --2- 172.21.15.28:0/1422505322 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f759c15c560 0x7f759c0ad710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.743+0000 7f759a59c700 1 -- 172.21.15.28:0/1422505322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f759c0cecf0 msgr2=0x7f759c1f2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.743+0000 7f759a59c700 1 --2- 172.21.15.28:0/1422505322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f759c0cecf0 0x7f759c1f2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.743+0000 7f759a59c700 1 -- 172.21.15.28:0/1422505322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f758c0094a0 con 0x7f759c0b2660 2023-11-28T11:10:22.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.743+0000 7f759a59c700 1 --2- 172.21.15.28:0/1422505322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f759c0b2660 0x7f759c0c7180 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f758c014040 tx=0x7f758c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.743+0000 7f758affd700 1 -- 172.21.15.28:0/1422505322 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c0038a0 con 0x7f759c0b2660 2023-11-28T11:10:22.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.743+0000 7f758affd700 1 -- 172.21.15.28:0/1422505322 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f758c01e070 con 0x7f759c0b2660 2023-11-28T11:10:22.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.743+0000 7f75a0f26700 1 -- 172.21.15.28:0/1422505322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f759c0b2af0 con 0x7f759c0b2660 2023-11-28T11:10:22.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.743+0000 7f758affd700 1 -- 172.21.15.28:0/1422505322 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f758c003c20 con 0x7f759c0b2660 2023-11-28T11:10:22.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.743+0000 7f75a0f26700 1 -- 172.21.15.28:0/1422505322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f759c06fdf0 con 0x7f759c0b2660 2023-11-28T11:10:22.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.744+0000 7f75a0f26700 1 -- 172.21.15.28:0/1422505322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f759c0b2dd0 con 0x7f759c0b2660 2023-11-28T11:10:22.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.744+0000 7f758affd700 1 -- 172.21.15.28:0/1422505322 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f758c00f050 con 0x7f759c0b2660 2023-11-28T11:10:22.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.745+0000 7f758affd700 1 --2- 172.21.15.28:0/1422505322 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f757c0682f0 0x7f757c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:22.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.745+0000 7f758affd700 1 -- 172.21.15.28:0/1422505322 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f758c086e90 con 0x7f759c0b2660 2023-11-28T11:10:22.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.745+0000 7f7599d9b700 1 --2- 172.21.15.28:0/1422505322 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f757c0682f0 0x7f757c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:22.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.745+0000 7f7599d9b700 1 --2- 172.21.15.28:0/1422505322 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f757c0682f0 0x7f757c06a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f759c0b1d30 tx=0x7f7590008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:22.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.749+0000 7f758affd700 1 -- 172.21.15.28:0/1422505322 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f758c0535b0 con 0x7f759c0b2660 2023-11-28T11:10:22.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.956+0000 7f75a0f26700 1 -- 172.21.15.28:0/1422505322 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f759c0675a0 con 0x7f759c0b2660 2023-11-28T11:10:22.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.956+0000 7f758affd700 1 -- 172.21.15.28:0/1422505322 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7f758c015080 con 0x7f759c0b2660 2023-11-28T11:10:22.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.958+0000 7f75a0f26700 1 -- 172.21.15.28:0/1422505322 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f757c0682f0 msgr2=0x7f757c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.958+0000 7f75a0f26700 1 --2- 172.21.15.28:0/1422505322 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f757c0682f0 0x7f757c06a7a0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f759c0b1d30 tx=0x7f7590008040 comp rx=0 tx=0).stop 2023-11-28T11:10:22.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.958+0000 7f75a0f26700 1 -- 172.21.15.28:0/1422505322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f759c0b2660 msgr2=0x7f759c0c7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:22.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.958+0000 7f75a0f26700 1 --2- 172.21.15.28:0/1422505322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f759c0b2660 0x7f759c0c7180 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f758c014040 tx=0x7f758c004750 comp rx=0 tx=0).stop 2023-11-28T11:10:22.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.958+0000 7f75a0f26700 1 -- 172.21.15.28:0/1422505322 shutdown_connections 2023-11-28T11:10:22.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.958+0000 7f75a0f26700 1 --2- 172.21.15.28:0/1422505322 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f757c0682f0 0x7f757c06a7a0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.958+0000 7f75a0f26700 1 --2- 172.21.15.28:0/1422505322 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f759c0b2660 0x7f759c0c7180 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.958+0000 7f75a0f26700 1 --2- 172.21.15.28:0/1422505322 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f759c15c560 0x7f759c0ad710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.958+0000 7f75a0f26700 1 --2- 172.21.15.28:0/1422505322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f759c0cecf0 0x7f759c1f2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:22.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.958+0000 7f75a0f26700 1 -- 172.21.15.28:0/1422505322 >> 172.21.15.28:0/1422505322 conn(0x7f759c0cd660 msgr2=0x7f759c15b360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:22.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.958+0000 7f75a0f26700 1 -- 172.21.15.28:0/1422505322 shutdown_connections 2023-11-28T11:10:22.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:22.958+0000 7f75a0f26700 1 -- 172.21.15.28:0/1422505322 wait complete. 2023-11-28T11:10:22.960 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2023-11-28T11:10:22.970 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:10:22.971 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:22.988 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:23.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.152+0000 7f388fa02700 1 Processor -- start 2023-11-28T11:10:23.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.153+0000 7f388fa02700 1 -- start start 2023-11-28T11:10:23.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.153+0000 7f388fa02700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f38880b4fe0 0x7f388815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.153+0000 7f388fa02700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f38880cd4a0 0x7f38880d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.153+0000 7f388fa02700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f388806abb0 con 0x7f388815a230 2023-11-28T11:10:23.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.153+0000 7f388fa02700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f388815a600 con 0x7f38880cd4a0 2023-11-28T11:10:23.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.153+0000 7f388fa02700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f388806a280 con 0x7f38880b4fe0 2023-11-28T11:10:23.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.154+0000 7f388df9f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f38880cd4a0 0x7f38880d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.154+0000 7f388df9f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f38880cd4a0 0x7f38880d0a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49692/0 (socket says 172.21.15.28:49692) 2023-11-28T11:10:23.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.154+0000 7f388df9f700 1 -- 172.21.15.28:0/2684022769 learned_addr learned my addr 172.21.15.28:0/2684022769 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:23.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.154+0000 7f388cf9d700 1 --2- 172.21.15.28:0/2684022769 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f38880b4fe0 0x7f388815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.154+0000 7f388df9f700 1 -- 172.21.15.28:0/2684022769 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f38880b4fe0 msgr2=0x7f388815d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:23.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.154+0000 7f388df9f700 1 --2- 172.21.15.28:0/2684022769 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f38880b4fe0 0x7f388815d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.154+0000 7f388df9f700 1 -- 172.21.15.28:0/2684022769 >> v1:172.21.15.59:6789/0 conn(0x7f388815a230 legacy=0x7f38880b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.154+0000 7f388df9f700 1 -- 172.21.15.28:0/2684022769 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388806a560 con 0x7f38880cd4a0 2023-11-28T11:10:23.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.154+0000 7f388df9f700 1 --2- 172.21.15.28:0/2684022769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38880cd4a0 0x7f38880d0a20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3880009e50 tx=0x7f388000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92e66c5fe012d5fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.154+0000 7f387ffff700 1 -- 172.21.15.28:0/2684022769 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388000e070 con 0x7f38880cd4a0 2023-11-28T11:10:23.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.154+0000 7f387ffff700 1 -- 172.21.15.28:0/2684022769 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388000b910 con 0x7f38880cd4a0 2023-11-28T11:10:23.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.154+0000 7f387ffff700 1 -- 172.21.15.28:0/2684022769 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388000bcc0 con 0x7f38880cd4a0 2023-11-28T11:10:23.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.155+0000 7f388fa02700 1 -- 172.21.15.28:0/2684022769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38880cd4a0 msgr2=0x7f38880d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.155+0000 7f388fa02700 1 --2- 172.21.15.28:0/2684022769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38880cd4a0 0x7f38880d0a20 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f3880009e50 tx=0x7f388000b330 comp rx=0 tx=0).stop 2023-11-28T11:10:23.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.155+0000 7f388fa02700 1 -- 172.21.15.28:0/2684022769 shutdown_connections 2023-11-28T11:10:23.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.155+0000 7f388fa02700 1 --2- 172.21.15.28:0/2684022769 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38880cd4a0 0x7f38880d0a20 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.155+0000 7f388fa02700 1 --2- 172.21.15.28:0/2684022769 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f38880b4fe0 0x7f388815d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.155+0000 7f388fa02700 1 -- 172.21.15.28:0/2684022769 >> 172.21.15.28:0/2684022769 conn(0x7f38880cf8d0 msgr2=0x7f38880cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:23.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.155+0000 7f388fa02700 1 -- 172.21.15.28:0/2684022769 shutdown_connections 2023-11-28T11:10:23.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.155+0000 7f388fa02700 1 -- 172.21.15.28:0/2684022769 wait complete. 2023-11-28T11:10:23.160 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.156+0000 7f388fa02700 1 Processor -- start 2023-11-28T11:10:23.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.156+0000 7f388fa02700 1 -- start start 2023-11-28T11:10:23.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388fa02700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38880b4fe0 0x7f38880ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.162 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388fa02700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38880cd4a0 0x7f388814ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.162 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388fa02700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f388815a230 0x7f388814b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.162 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388fa02700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38881f24b0 con 0x7f38880cd4a0 2023-11-28T11:10:23.162 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388fa02700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38880caca0 con 0x7f38880b4fe0 2023-11-28T11:10:23.162 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388fa02700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38881e4f10 con 0x7f388815a230 2023-11-28T11:10:23.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388cf9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38880cd4a0 0x7f388814ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388cf9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38880cd4a0 0x7f388814ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49696/0 (socket says 172.21.15.28:49696) 2023-11-28T11:10:23.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388cf9d700 1 -- 172.21.15.28:0/223784258 learned_addr learned my addr 172.21.15.28:0/223784258 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:23.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388df9f700 1 --2- 172.21.15.28:0/223784258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f388815a230 0x7f388814b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388d79e700 1 --2- 172.21.15.28:0/223784258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38880b4fe0 0x7f38880ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388cf9d700 1 -- 172.21.15.28:0/223784258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f388815a230 msgr2=0x7f388814b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388cf9d700 1 --2- 172.21.15.28:0/223784258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f388815a230 0x7f388814b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388cf9d700 1 -- 172.21.15.28:0/223784258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38880b4fe0 msgr2=0x7f38880ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388cf9d700 1 --2- 172.21.15.28:0/223784258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38880b4fe0 0x7f38880ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388cf9d700 1 -- 172.21.15.28:0/223784258 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3880009b30 con 0x7f38880cd4a0 2023-11-28T11:10:23.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.157+0000 7f388cf9d700 1 --2- 172.21.15.28:0/223784258 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38880cd4a0 0x7f388814ad40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f388400e980 tx=0x7f388400ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.158+0000 7f387dffb700 1 -- 172.21.15.28:0/223784258 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3884009710 con 0x7f38880cd4a0 2023-11-28T11:10:23.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.158+0000 7f387dffb700 1 -- 172.21.15.28:0/223784258 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f388400c850 con 0x7f38880cd4a0 2023-11-28T11:10:23.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.158+0000 7f388fa02700 1 -- 172.21.15.28:0/223784258 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38880cd920 con 0x7f38880cd4a0 2023-11-28T11:10:23.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.158+0000 7f387dffb700 1 -- 172.21.15.28:0/223784258 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f388400cc00 con 0x7f38880cd4a0 2023-11-28T11:10:23.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.158+0000 7f388fa02700 1 -- 172.21.15.28:0/223784258 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388806a840 con 0x7f38880cd4a0 2023-11-28T11:10:23.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.159+0000 7f388fa02700 1 -- 172.21.15.28:0/223784258 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f388806b040 con 0x7f38880cd4a0 2023-11-28T11:10:23.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.159+0000 7f387dffb700 1 -- 172.21.15.28:0/223784258 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3884019030 con 0x7f38880cd4a0 2023-11-28T11:10:23.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.160+0000 7f387dffb700 1 --2- 172.21.15.28:0/223784258 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f386c0682f0 0x7f386c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.160+0000 7f387dffb700 1 -- 172.21.15.28:0/223784258 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3884085da0 con 0x7f38880cd4a0 2023-11-28T11:10:23.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.160+0000 7f388d79e700 1 --2- 172.21.15.28:0/223784258 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f386c0682f0 0x7f386c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.160+0000 7f388d79e700 1 --2- 172.21.15.28:0/223784258 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f386c0682f0 0x7f386c06a7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3878000f60 tx=0x7f3878009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.167 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.164+0000 7f387dffb700 1 -- 172.21.15.28:0/223784258 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3884052580 con 0x7f38880cd4a0 2023-11-28T11:10:23.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.363+0000 7f388fa02700 1 -- 172.21.15.28:0/223784258 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38880675a0 con 0x7f38880cd4a0 2023-11-28T11:10:23.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.364+0000 7f387dffb700 1 -- 172.21.15.28:0/223784258 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+18022 (secure 0 0 0) 0x7f3884017090 con 0x7f38880cd4a0 2023-11-28T11:10:23.365 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:23.365 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":40,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:10:11.822678+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:08.429278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:09.797038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:23.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.366+0000 7f388fa02700 1 -- 172.21.15.28:0/223784258 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f386c0682f0 msgr2=0x7f386c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.366+0000 7f388fa02700 1 --2- 172.21.15.28:0/223784258 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f386c0682f0 0x7f386c06a7a0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f3878000f60 tx=0x7f3878009040 comp rx=0 tx=0).stop 2023-11-28T11:10:23.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.366+0000 7f388fa02700 1 -- 172.21.15.28:0/223784258 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38880cd4a0 msgr2=0x7f388814ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.366+0000 7f388fa02700 1 --2- 172.21.15.28:0/223784258 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38880cd4a0 0x7f388814ad40 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f388400e980 tx=0x7f388400ed40 comp rx=0 tx=0).stop 2023-11-28T11:10:23.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.366+0000 7f388fa02700 1 -- 172.21.15.28:0/223784258 shutdown_connections 2023-11-28T11:10:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.366+0000 7f388fa02700 1 --2- 172.21.15.28:0/223784258 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f386c0682f0 0x7f386c06a7a0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.366+0000 7f388fa02700 1 --2- 172.21.15.28:0/223784258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f388815a230 0x7f388814b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.366+0000 7f388fa02700 1 --2- 172.21.15.28:0/223784258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f38880b4fe0 0x7f38880ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.366+0000 7f388fa02700 1 --2- 172.21.15.28:0/223784258 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f38880cd4a0 0x7f388814ad40 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.366+0000 7f388fa02700 1 -- 172.21.15.28:0/223784258 >> 172.21.15.28:0/223784258 conn(0x7f38880cf8d0 msgr2=0x7f38880d2200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.367+0000 7f388fa02700 1 -- 172.21.15.28:0/223784258 shutdown_connections 2023-11-28T11:10:23.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.367+0000 7f388fa02700 1 -- 172.21.15.28:0/223784258 wait complete. 2023-11-28T11:10:23.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:10:08.429278+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:10:09.797038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:10:23.380 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:10:23.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.506+0000 7ff069080700 1 Processor -- start 2023-11-28T11:10:23.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.506+0000 7ff069080700 1 -- start start 2023-11-28T11:10:23.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.507+0000 7ff069080700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff06415a640 0x7ff0640c3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.507+0000 7ff069080700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff06406b900 0x7ff0640cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.507+0000 7ff069080700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0640ae730 con 0x7ff0640cd140 2023-11-28T11:10:23.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.507+0000 7ff069080700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0640cb210 con 0x7ff06415a640 2023-11-28T11:10:23.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.507+0000 7ff069080700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff06406e9e0 con 0x7ff06406b900 2023-11-28T11:10:23.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.507+0000 7ff06259c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff0640cd140 0x7ff06406b1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37760/0 (socket says 172.21.15.28:37760) 2023-11-28T11:10:23.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.507+0000 7ff06259c700 1 -- 172.21.15.28:0/2427567149 learned_addr learned my addr 172.21.15.28:0/2427567149 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:23.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 -- 172.21.15.28:0/2427567149 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194566546 0 0) 0x7ff0640ae730 con 0x7ff0640cd140 2023-11-28T11:10:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 -- 172.21.15.28:0/2427567149 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff04c0035b0 con 0x7ff0640cd140 2023-11-28T11:10:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff06359e700 1 --2- 172.21.15.28:0/2427567149 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff06406b900 0x7ff0640cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff062d9d700 1 --2- 172.21.15.28:0/2427567149 >> v2:172.21.15.59:3300/0 conn(0x7ff06415a640 0x7ff0640c3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 -- 172.21.15.28:0/2427567149 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1946014469 0 0) 0x7ff04c0035b0 con 0x7ff0640cd140 2023-11-28T11:10:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 -- 172.21.15.28:0/2427567149 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff06406b900 msgr2=0x7ff0640cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 --2- 172.21.15.28:0/2427567149 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff06406b900 0x7ff0640cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 -- 172.21.15.28:0/2427567149 >> v2:172.21.15.59:3300/0 conn(0x7ff06415a640 msgr2=0x7ff0640c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 --2- 172.21.15.28:0/2427567149 >> v2:172.21.15.59:3300/0 conn(0x7ff06415a640 0x7ff0640c3700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 -- 172.21.15.28:0/2427567149 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0640ae980 con 0x7ff0640cd140 2023-11-28T11:10:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 -- 172.21.15.28:0/2427567149 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7ff054001670 con 0x7ff0640cd140 2023-11-28T11:10:23.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff06359e700 1 --2- 172.21.15.28:0/2427567149 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff06406b900 0x7ff0640cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff062d9d700 1 --2- 172.21.15.28:0/2427567149 >> v2:172.21.15.59:3300/0 conn(0x7ff06415a640 0x7ff0640c3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 -- 172.21.15.28:0/2427567149 >> v1:172.21.15.28:6789/0 conn(0x7ff0640cd140 legacy=0x7ff06406b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 --2- 172.21.15.28:0/2427567149 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff06406b900 0x7ff04c003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 --2- 172.21.15.28:0/2427567149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff04c003ff0 0x7ff04c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 --2- 172.21.15.28:0/2427567149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff04c004d90 0x7ff04c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 -- 172.21.15.28:0/2427567149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640cb210 con 0x7ff04c003ff0 2023-11-28T11:10:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 -- 172.21.15.28:0/2427567149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06406e9e0 con 0x7ff04c004d90 2023-11-28T11:10:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff061d9b700 1 -- 172.21.15.28:0/2427567149 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c00bac0 con 0x7ff06406b900 2023-11-28T11:10:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff06259c700 1 --2- 172.21.15.28:0/2427567149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff04c003ff0 0x7ff04c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff062d9d700 1 --2- 172.21.15.28:0/2427567149 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff06406b900 0x7ff04c003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff06359e700 1 --2- 172.21.15.28:0/2427567149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff04c004d90 0x7ff04c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff06259c700 1 -- 172.21.15.28:0/2427567149 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff06406b900 msgr2=0x7ff04c003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff06259c700 1 --2- 172.21.15.28:0/2427567149 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff06406b900 0x7ff04c003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff06259c700 1 -- 172.21.15.28:0/2427567149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff04c004d90 msgr2=0x7ff04c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff06259c700 1 --2- 172.21.15.28:0/2427567149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff04c004d90 0x7ff04c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.508+0000 7ff06259c700 1 -- 172.21.15.28:0/2427567149 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c0035b0 con 0x7ff04c003ff0 2023-11-28T11:10:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.509+0000 7ff06259c700 1 --2- 172.21.15.28:0/2427567149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff04c003ff0 0x7ff04c004850 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff05400c2f0 tx=0x7ff04c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eab634f895ca3c39 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.509+0000 7ff061d9b700 1 -- 172.21.15.28:0/2427567149 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05400ce30 con 0x7ff04c003ff0 2023-11-28T11:10:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.509+0000 7ff061d9b700 1 -- 172.21.15.28:0/2427567149 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff054005960 con 0x7ff04c003ff0 2023-11-28T11:10:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.509+0000 7ff061d9b700 1 -- 172.21.15.28:0/2427567149 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff054005d10 con 0x7ff04c003ff0 2023-11-28T11:10:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.509+0000 7ff069080700 1 -- 172.21.15.28:0/2427567149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff04c003ff0 msgr2=0x7ff04c004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.509+0000 7ff069080700 1 --2- 172.21.15.28:0/2427567149 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff04c003ff0 0x7ff04c004850 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ff05400c2f0 tx=0x7ff04c005880 comp rx=0 tx=0).stop 2023-11-28T11:10:23.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.509+0000 7ff06359e700 1 -- 172.21.15.28:0/2427567149 reap_dead start 2023-11-28T11:10:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.509+0000 7ff069080700 1 -- 172.21.15.28:0/2427567149 shutdown_connections 2023-11-28T11:10:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.509+0000 7ff069080700 1 -- 172.21.15.28:0/2427567149 >> 172.21.15.28:0/2427567149 conn(0x7ff0640cc630 msgr2=0x7ff0640cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.509+0000 7ff069080700 1 -- 172.21.15.28:0/2427567149 shutdown_connections 2023-11-28T11:10:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.509+0000 7ff069080700 1 -- 172.21.15.28:0/2427567149 wait complete. 2023-11-28T11:10:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.511+0000 7ff069080700 1 Processor -- start 2023-11-28T11:10:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.511+0000 7ff069080700 1 -- start start 2023-11-28T11:10:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.511+0000 7ff069080700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff04c003ff0 0x7ff0640adc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.511+0000 7ff069080700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0640c3910 0x7ff0640af6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff069080700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0640cd780 0x7ff0640cdbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff069080700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640c67b0 con 0x7ff0640cd780 2023-11-28T11:10:23.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff069080700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640c5bc0 con 0x7ff04c003ff0 2023-11-28T11:10:23.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff069080700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640c5e10 con 0x7ff0640c3910 2023-11-28T11:10:23.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff06359e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0640cd780 0x7ff0640cdbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff06359e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0640cd780 0x7ff0640cdbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49716/0 (socket says 172.21.15.28:49716) 2023-11-28T11:10:23.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff06359e700 1 -- 172.21.15.28:0/1081058887 learned_addr learned my addr 172.21.15.28:0/1081058887 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:23.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff06259c700 1 --2- 172.21.15.28:0/1081058887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0640c3910 0x7ff0640af6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff062d9d700 1 --2- 172.21.15.28:0/1081058887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff04c003ff0 0x7ff0640adc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff06359e700 1 -- 172.21.15.28:0/1081058887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0640c3910 msgr2=0x7ff0640af6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff06359e700 1 --2- 172.21.15.28:0/1081058887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0640c3910 0x7ff0640af6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff06359e700 1 -- 172.21.15.28:0/1081058887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff04c003ff0 msgr2=0x7ff0640adc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff06359e700 1 --2- 172.21.15.28:0/1081058887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff04c003ff0 0x7ff0640adc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff06359e700 1 -- 172.21.15.28:0/1081058887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0640aef40 con 0x7ff0640cd780 2023-11-28T11:10:23.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff06359e700 1 --2- 172.21.15.28:0/1081058887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0640cd780 0x7ff0640cdbe0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff0640718b0 tx=0x7ff05c010440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff04b7fe700 1 -- 172.21.15.28:0/1081058887 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c01b070 con 0x7ff0640cd780 2023-11-28T11:10:23.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff04b7fe700 1 -- 172.21.15.28:0/1081058887 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05c010a80 con 0x7ff0640cd780 2023-11-28T11:10:23.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff069080700 1 -- 172.21.15.28:0/1081058887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0640aec60 con 0x7ff0640cd780 2023-11-28T11:10:23.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff04b7fe700 1 -- 172.21.15.28:0/1081058887 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c010c10 con 0x7ff0640cd780 2023-11-28T11:10:23.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.512+0000 7ff069080700 1 -- 172.21.15.28:0/1081058887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0640af220 con 0x7ff0640cd780 2023-11-28T11:10:23.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.513+0000 7ff069080700 1 -- 172.21.15.28:0/1081058887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0640739f0 con 0x7ff0640cd780 2023-11-28T11:10:23.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.514+0000 7ff04b7fe700 1 -- 172.21.15.28:0/1081058887 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff05c010d70 con 0x7ff0640cd780 2023-11-28T11:10:23.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.514+0000 7ff04b7fe700 1 --2- 172.21.15.28:0/1081058887 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff0500682f0 0x7ff05006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.514+0000 7ff04b7fe700 1 -- 172.21.15.28:0/1081058887 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff05c086050 con 0x7ff0640cd780 2023-11-28T11:10:23.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.514+0000 7ff062d9d700 1 --2- 172.21.15.28:0/1081058887 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff0500682f0 0x7ff05006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.515+0000 7ff062d9d700 1 --2- 172.21.15.28:0/1081058887 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff0500682f0 0x7ff05006a7a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff058009a80 tx=0x7ff058008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.519+0000 7ff04b7fe700 1 -- 172.21.15.28:0/1081058887 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff05c052690 con 0x7ff0640cd780 2023-11-28T11:10:23.712 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.710+0000 7ff069080700 1 -- 172.21.15.28:0/1081058887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7ff064159810 con 0x7ff0640cd780 2023-11-28T11:10:23.712 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.711+0000 7ff04b7fe700 1 -- 172.21.15.28:0/1081058887 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v40) v1 ==== 92+0+10 (secure 0 0 0) 0x7ff05c055cb0 con 0x7ff0640cd780 2023-11-28T11:10:23.713 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:10:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.713+0000 7ff069080700 1 -- 172.21.15.28:0/1081058887 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff0500682f0 msgr2=0x7ff05006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.713+0000 7ff069080700 1 --2- 172.21.15.28:0/1081058887 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff0500682f0 0x7ff05006a7a0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7ff058009a80 tx=0x7ff058008040 comp rx=0 tx=0).stop 2023-11-28T11:10:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.713+0000 7ff069080700 1 -- 172.21.15.28:0/1081058887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0640cd780 msgr2=0x7ff0640cdbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.713+0000 7ff069080700 1 --2- 172.21.15.28:0/1081058887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0640cd780 0x7ff0640cdbe0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ff0640718b0 tx=0x7ff05c010440 comp rx=0 tx=0).stop 2023-11-28T11:10:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.713+0000 7ff069080700 1 -- 172.21.15.28:0/1081058887 shutdown_connections 2023-11-28T11:10:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.713+0000 7ff069080700 1 --2- 172.21.15.28:0/1081058887 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff0500682f0 0x7ff05006a7a0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.713+0000 7ff069080700 1 --2- 172.21.15.28:0/1081058887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff0640cd780 0x7ff0640cdbe0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.713+0000 7ff069080700 1 --2- 172.21.15.28:0/1081058887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff04c003ff0 0x7ff0640adc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.716 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.713+0000 7ff069080700 1 --2- 172.21.15.28:0/1081058887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff0640c3910 0x7ff0640af6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.716 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.713+0000 7ff069080700 1 -- 172.21.15.28:0/1081058887 >> 172.21.15.28:0/1081058887 conn(0x7ff0640cc630 msgr2=0x7ff0640b0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:23.716 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.713+0000 7ff069080700 1 -- 172.21.15.28:0/1081058887 shutdown_connections 2023-11-28T11:10:23.716 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.713+0000 7ff069080700 1 -- 172.21.15.28:0/1081058887 wait complete. 2023-11-28T11:10:23.727 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:10:23.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.853+0000 7f086e8c1700 1 Processor -- start 2023-11-28T11:10:23.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.854+0000 7f086e8c1700 1 -- start start 2023-11-28T11:10:23.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.854+0000 7f086e8c1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f086815c4b0 0x7f086815d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.854+0000 7f086e8c1700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08680cae80 con 0x7f086815c880 2023-11-28T11:10:23.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.854+0000 7f086e8c1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08680c9de0 con 0x7f08680c9950 2023-11-28T11:10:23.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.854+0000 7f086e8c1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f086815dba0 con 0x7f086815c4b0 2023-11-28T11:10:23.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.855+0000 7f08677fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f086815c880 0x7f086806a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37778/0 (socket says 172.21.15.28:37778) 2023-11-28T11:10:23.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.855+0000 7f08677fe700 1 -- 172.21.15.28:0/752197070 learned_addr learned my addr 172.21.15.28:0/752197070 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:23.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.855+0000 7f0867fff700 1 --2- 172.21.15.28:0/752197070 >> v2:172.21.15.59:3300/0 conn(0x7f086815c4b0 0x7f086815d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.855+0000 7f0866ffd700 1 -- 172.21.15.28:0/752197070 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2071487317 0 0) 0x7f08680cae80 con 0x7f086815c880 2023-11-28T11:10:23.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.855+0000 7f0866ffd700 1 -- 172.21.15.28:0/752197070 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08500035b0 con 0x7f086815c880 2023-11-28T11:10:23.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.855+0000 7f0867fff700 1 -- 172.21.15.28:0/752197070 >> v1:172.21.15.59:6789/0 conn(0x7f08680c9950 legacy=0x7f08680cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.855+0000 7f0867fff700 1 -- 172.21.15.28:0/752197070 >> v1:172.21.15.28:6789/0 conn(0x7f086815c880 legacy=0x7f086806a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.855+0000 7f0867fff700 1 -- 172.21.15.28:0/752197070 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08680ca0c0 con 0x7f086815c4b0 2023-11-28T11:10:23.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.855+0000 7f0867fff700 1 --2- 172.21.15.28:0/752197070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f086815c4b0 0x7f086815d660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f085c009840 tx=0x7f085c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b93b6adcf2e98b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.855+0000 7f0866ffd700 1 -- 172.21.15.28:0/752197070 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c011830 con 0x7f086815c4b0 2023-11-28T11:10:23.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.855+0000 7f0866ffd700 1 -- 172.21.15.28:0/752197070 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f085c011990 con 0x7f086815c4b0 2023-11-28T11:10:23.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.855+0000 7f0866ffd700 1 -- 172.21.15.28:0/752197070 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c011d00 con 0x7f086815c4b0 2023-11-28T11:10:23.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.856+0000 7f086e8c1700 1 -- 172.21.15.28:0/752197070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f086815c4b0 msgr2=0x7f086815d660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.856+0000 7f086e8c1700 1 --2- 172.21.15.28:0/752197070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f086815c4b0 0x7f086815d660 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f085c009840 tx=0x7f085c009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:23.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.856+0000 7f086e8c1700 1 -- 172.21.15.28:0/752197070 shutdown_connections 2023-11-28T11:10:23.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.856+0000 7f086e8c1700 1 --2- 172.21.15.28:0/752197070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f086815c4b0 0x7f086815d660 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.856+0000 7f086e8c1700 1 -- 172.21.15.28:0/752197070 >> 172.21.15.28:0/752197070 conn(0x7f08680b0df0 msgr2=0x7f08680b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:23.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.856+0000 7f086e8c1700 1 -- 172.21.15.28:0/752197070 shutdown_connections 2023-11-28T11:10:23.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.856+0000 7f086e8c1700 1 -- 172.21.15.28:0/752197070 wait complete. 2023-11-28T11:10:23.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.857+0000 7f086e8c1700 1 Processor -- start 2023-11-28T11:10:23.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.857+0000 7f086e8c1700 1 -- start start 2023-11-28T11:10:23.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.858+0000 7f086e8c1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08680c9950 0x7f08681536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.858+0000 7f086e8c1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f086815c4b0 0x7f08680ac460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.858+0000 7f086e8c1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f086815c880 0x7f086815ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.858+0000 7f086e8c1700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680ae000 con 0x7f086815c4b0 2023-11-28T11:10:23.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.858+0000 7f086e8c1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08680ad3f0 con 0x7f08680c9950 2023-11-28T11:10:23.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.858+0000 7f086e8c1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08680ad6d0 con 0x7f086815c880 2023-11-28T11:10:23.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.858+0000 7f08677fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f086815c4b0 0x7f08680ac460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.858+0000 7f08677fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f086815c4b0 0x7f08680ac460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49728/0 (socket says 172.21.15.28:49728) 2023-11-28T11:10:23.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.858+0000 7f08677fe700 1 -- 172.21.15.28:0/1880029039 learned_addr learned my addr 172.21.15.28:0/1880029039 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:23.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.858+0000 7f0867fff700 1 --2- 172.21.15.28:0/1880029039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08680c9950 0x7f08681536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.859+0000 7f086ce5e700 1 --2- 172.21.15.28:0/1880029039 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f086815c880 0x7f086815ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.859+0000 7f08677fe700 1 -- 172.21.15.28:0/1880029039 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f086815c880 msgr2=0x7f086815ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.859+0000 7f08677fe700 1 --2- 172.21.15.28:0/1880029039 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f086815c880 0x7f086815ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.859+0000 7f08677fe700 1 -- 172.21.15.28:0/1880029039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08680c9950 msgr2=0x7f08681536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:23.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.859+0000 7f08677fe700 1 --2- 172.21.15.28:0/1880029039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08680c9950 0x7f08681536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:23.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.859+0000 7f08677fe700 1 -- 172.21.15.28:0/1880029039 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c0094a0 con 0x7f086815c4b0 2023-11-28T11:10:23.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.859+0000 7f0867fff700 1 --2- 172.21.15.28:0/1880029039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08680c9950 0x7f08681536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:23.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.859+0000 7f08677fe700 1 --2- 172.21.15.28:0/1880029039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f086815c4b0 0x7f08680ac460 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0858001eb0 tx=0x7f085800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.859+0000 7f0864ff9700 1 -- 172.21.15.28:0/1880029039 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858005d70 con 0x7f086815c4b0 2023-11-28T11:10:23.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.859+0000 7f0864ff9700 1 -- 172.21.15.28:0/1880029039 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08580070d0 con 0x7f086815c4b0 2023-11-28T11:10:23.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.859+0000 7f086e8c1700 1 -- 172.21.15.28:0/1880029039 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08680ca680 con 0x7f086815c4b0 2023-11-28T11:10:23.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.859+0000 7f0864ff9700 1 -- 172.21.15.28:0/1880029039 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858007460 con 0x7f086815c4b0 2023-11-28T11:10:23.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.859+0000 7f086e8c1700 1 -- 172.21.15.28:0/1880029039 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08680ca3a0 con 0x7f086815c4b0 2023-11-28T11:10:23.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.860+0000 7f086e8c1700 1 -- 172.21.15.28:0/1880029039 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08680ca960 con 0x7f086815c4b0 2023-11-28T11:10:23.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.860+0000 7f0864ff9700 1 -- 172.21.15.28:0/1880029039 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0858019030 con 0x7f086815c4b0 2023-11-28T11:10:23.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.861+0000 7f0864ff9700 1 --2- 172.21.15.28:0/1880029039 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0854068320 0x7f085406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:23.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.861+0000 7f0864ff9700 1 -- 172.21.15.28:0/1880029039 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0858085160 con 0x7f086815c4b0 2023-11-28T11:10:23.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.861+0000 7f0867fff700 1 --2- 172.21.15.28:0/1880029039 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0854068320 0x7f085406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:23.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.861+0000 7f0867fff700 1 --2- 172.21.15.28:0/1880029039 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0854068320 0x7f085406a7d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f085c0090b0 tx=0x7f085c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:23.866 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:23.865+0000 7f0864ff9700 1 -- 172.21.15.28:0/1880029039 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0858051880 con 0x7f086815c4b0 2023-11-28T11:10:24.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.056+0000 7f086e8c1700 1 -- 172.21.15.28:0/1880029039 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f08680b0760 con 0x7f086815c4b0 2023-11-28T11:10:24.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.056+0000 7f0864ff9700 1 -- 172.21.15.28:0/1880029039 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0858054ea0 con 0x7f086815c4b0 2023-11-28T11:10:24.058 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:10:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.058+0000 7f086e8c1700 1 -- 172.21.15.28:0/1880029039 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0854068320 msgr2=0x7f085406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.058+0000 7f086e8c1700 1 --2- 172.21.15.28:0/1880029039 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0854068320 0x7f085406a7d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f085c0090b0 tx=0x7f085c008030 comp rx=0 tx=0).stop 2023-11-28T11:10:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.058+0000 7f086e8c1700 1 -- 172.21.15.28:0/1880029039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f086815c4b0 msgr2=0x7f08680ac460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.058+0000 7f086e8c1700 1 --2- 172.21.15.28:0/1880029039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f086815c4b0 0x7f08680ac460 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f0858001eb0 tx=0x7f085800c860 comp rx=0 tx=0).stop 2023-11-28T11:10:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.058+0000 7f086e8c1700 1 -- 172.21.15.28:0/1880029039 shutdown_connections 2023-11-28T11:10:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.058+0000 7f086e8c1700 1 --2- 172.21.15.28:0/1880029039 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0854068320 0x7f085406a7d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.058+0000 7f086e8c1700 1 --2- 172.21.15.28:0/1880029039 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f086815c880 0x7f086815ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.058+0000 7f086e8c1700 1 --2- 172.21.15.28:0/1880029039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f08680c9950 0x7f08681536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.058+0000 7f086e8c1700 1 --2- 172.21.15.28:0/1880029039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f086815c4b0 0x7f08680ac460 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.058+0000 7f086e8c1700 1 -- 172.21.15.28:0/1880029039 >> 172.21.15.28:0/1880029039 conn(0x7f08680b0df0 msgr2=0x7f08680b0050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:24.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.058+0000 7f086e8c1700 1 -- 172.21.15.28:0/1880029039 shutdown_connections 2023-11-28T11:10:24.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.058+0000 7f086e8c1700 1 -- 172.21.15.28:0/1880029039 wait complete. 2023-11-28T11:10:24.071 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:10:24.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.198+0000 7f0212d6f700 1 Processor -- start 2023-11-28T11:10:24.199 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.198+0000 7f0212d6f700 1 -- start start 2023-11-28T11:10:24.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.198+0000 7f0212d6f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f020c159500 0x7f020c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.198+0000 7f0212d6f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f020c15dbf0 0x7f020c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f0212d6f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f020c0b1630 0x7f020c0b1ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f0212d6f700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c0b2010 con 0x7f020c159500 2023-11-28T11:10:24.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f0212d6f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020c0cede0 con 0x7f020c15dbf0 2023-11-28T11:10:24.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f0212d6f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c15e040 con 0x7f020c0b1630 2023-11-28T11:10:24.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f0210b0b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f020c159500 0x7f020c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f0210b0b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f020c159500 0x7f020c0cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49734/0 (socket says 172.21.15.28:49734) 2023-11-28T11:10:24.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f0210b0b700 1 -- 172.21.15.28:0/2280057684 learned_addr learned my addr 172.21.15.28:0/2280057684 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:24.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f020bfff700 1 --2- 172.21.15.28:0/2280057684 >> v2:172.21.15.59:3300/0 conn(0x7f020c15dbf0 0x7f020c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f021130c700 1 --2- 172.21.15.28:0/2280057684 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f020c0b1630 0x7f020c0b1ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f0210b0b700 1 -- 172.21.15.28:0/2280057684 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f020c0b1630 msgr2=0x7f020c0b1ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:24.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f0210b0b700 1 --2- 172.21.15.28:0/2280057684 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f020c0b1630 0x7f020c0b1ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f0210b0b700 1 -- 172.21.15.28:0/2280057684 >> v2:172.21.15.59:3300/0 conn(0x7f020c15dbf0 msgr2=0x7f020c0ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f0210b0b700 1 --2- 172.21.15.28:0/2280057684 >> v2:172.21.15.59:3300/0 conn(0x7f020c15dbf0 0x7f020c0ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f0210b0b700 1 -- 172.21.15.28:0/2280057684 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0ceff0 con 0x7f020c159500 2023-11-28T11:10:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f020bfff700 1 --2- 172.21.15.28:0/2280057684 >> v2:172.21.15.59:3300/0 conn(0x7f020c15dbf0 0x7f020c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f021130c700 1 --2- 172.21.15.28:0/2280057684 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f020c0b1630 0x7f020c0b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f0210b0b700 1 --2- 172.21.15.28:0/2280057684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f020c159500 0x7f020c0cd140 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f020c0d0c50 tx=0x7f0200009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4138f016fd7d06d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f020b7fe700 1 -- 172.21.15.28:0/2280057684 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020000b870 con 0x7f020c159500 2023-11-28T11:10:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f020b7fe700 1 -- 172.21.15.28:0/2280057684 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f020000b9d0 con 0x7f020c159500 2023-11-28T11:10:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.199+0000 7f020b7fe700 1 -- 172.21.15.28:0/2280057684 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020000bd60 con 0x7f020c159500 2023-11-28T11:10:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.200+0000 7f0212d6f700 1 -- 172.21.15.28:0/2280057684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f020c159500 msgr2=0x7f020c0cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.200+0000 7f0212d6f700 1 --2- 172.21.15.28:0/2280057684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f020c159500 0x7f020c0cd140 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f020c0d0c50 tx=0x7f0200009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.200+0000 7f0212d6f700 1 -- 172.21.15.28:0/2280057684 shutdown_connections 2023-11-28T11:10:24.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.200+0000 7f0212d6f700 1 --2- 172.21.15.28:0/2280057684 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f020c0b1630 0x7f020c0b1ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.200+0000 7f0212d6f700 1 --2- 172.21.15.28:0/2280057684 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f020c159500 0x7f020c0cd140 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.200+0000 7f0212d6f700 1 --2- 172.21.15.28:0/2280057684 >> v2:172.21.15.59:3300/0 conn(0x7f020c15dbf0 0x7f020c0ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.200+0000 7f0212d6f700 1 -- 172.21.15.28:0/2280057684 >> 172.21.15.28:0/2280057684 conn(0x7f020c0cc630 msgr2=0x7f020c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:24.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.200+0000 7f0212d6f700 1 -- 172.21.15.28:0/2280057684 shutdown_connections 2023-11-28T11:10:24.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.200+0000 7f0212d6f700 1 -- 172.21.15.28:0/2280057684 wait complete. 2023-11-28T11:10:24.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.201+0000 7f0212d6f700 1 Processor -- start 2023-11-28T11:10:24.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.201+0000 7f0212d6f700 1 -- start start 2023-11-28T11:10:24.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f0212d6f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f020c0b1630 0x7f020c154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f0212d6f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f020c159500 0x7f020c15c870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f0212d6f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f020c15dbf0 0x7f020c0c8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f0212d6f700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c154740 con 0x7f020c0b1630 2023-11-28T11:10:24.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f0212d6f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c1f1260 con 0x7f020c159500 2023-11-28T11:10:24.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f0212d6f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c1f1b00 con 0x7f020c15dbf0 2023-11-28T11:10:24.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f0210b0b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f020c0b1630 0x7f020c154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f0210b0b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f020c0b1630 0x7f020c154200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49740/0 (socket says 172.21.15.28:49740) 2023-11-28T11:10:24.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f0210b0b700 1 -- 172.21.15.28:0/3501306127 learned_addr learned my addr 172.21.15.28:0/3501306127 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:24.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f021130c700 1 --2- 172.21.15.28:0/3501306127 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f020c15dbf0 0x7f020c0c8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f020bfff700 1 --2- 172.21.15.28:0/3501306127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f020c159500 0x7f020c15c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f0210b0b700 1 -- 172.21.15.28:0/3501306127 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f020c15dbf0 msgr2=0x7f020c0c8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f0210b0b700 1 --2- 172.21.15.28:0/3501306127 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f020c15dbf0 0x7f020c0c8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f0210b0b700 1 -- 172.21.15.28:0/3501306127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f020c159500 msgr2=0x7f020c15c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f0210b0b700 1 --2- 172.21.15.28:0/3501306127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f020c159500 0x7f020c15c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.202+0000 7f0210b0b700 1 -- 172.21.15.28:0/3501306127 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0200009550 con 0x7f020c0b1630 2023-11-28T11:10:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.203+0000 7f021130c700 1 --2- 172.21.15.28:0/3501306127 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f020c15dbf0 0x7f020c0c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.203+0000 7f0210b0b700 1 --2- 172.21.15.28:0/3501306127 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f020c0b1630 0x7f020c154200 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0200014040 tx=0x7f0200004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.203+0000 7f02097fa700 1 -- 172.21.15.28:0/3501306127 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200004500 con 0x7f020c0b1630 2023-11-28T11:10:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.203+0000 7f02097fa700 1 -- 172.21.15.28:0/3501306127 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02000036a0 con 0x7f020c0b1630 2023-11-28T11:10:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.203+0000 7f0212d6f700 1 -- 172.21.15.28:0/3501306127 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c15d110 con 0x7f020c0b1630 2023-11-28T11:10:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.203+0000 7f02097fa700 1 -- 172.21.15.28:0/3501306127 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0200003a30 con 0x7f020c0b1630 2023-11-28T11:10:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.203+0000 7f0212d6f700 1 -- 172.21.15.28:0/3501306127 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c0cf2d0 con 0x7f020c0b1630 2023-11-28T11:10:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.204+0000 7f0212d6f700 1 -- 172.21.15.28:0/3501306127 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c15d3f0 con 0x7f020c0b1630 2023-11-28T11:10:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.204+0000 7f02097fa700 1 -- 172.21.15.28:0/3501306127 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f020000f050 con 0x7f020c0b1630 2023-11-28T11:10:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.205+0000 7f02097fa700 1 --2- 172.21.15.28:0/3501306127 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01f00682f0 0x7f01f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.205+0000 7f02097fa700 1 -- 172.21.15.28:0/3501306127 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f02000873a0 con 0x7f020c0b1630 2023-11-28T11:10:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.205+0000 7f020bfff700 1 --2- 172.21.15.28:0/3501306127 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01f00682f0 0x7f01f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.205+0000 7f020bfff700 1 --2- 172.21.15.28:0/3501306127 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01f00682f0 0x7f01f006a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f01f8006fd0 tx=0x7f01f8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.209+0000 7f02097fa700 1 -- 172.21.15.28:0/3501306127 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0200053ac0 con 0x7f020c0b1630 2023-11-28T11:10:24.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.402+0000 7f0212d6f700 1 -- 172.21.15.28:0/3501306127 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f020c0739f0 con 0x7f020c0b1630 2023-11-28T11:10:24.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.402+0000 7f02097fa700 1 -- 172.21.15.28:0/3501306127 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v40) v1 ==== 104+0+11 (secure 0 0 0) 0x7f02000570e0 con 0x7f020c0b1630 2023-11-28T11:10:24.404 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:10:24.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.404+0000 7f0212d6f700 1 -- 172.21.15.28:0/3501306127 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01f00682f0 msgr2=0x7f01f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.404+0000 7f0212d6f700 1 --2- 172.21.15.28:0/3501306127 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01f00682f0 0x7f01f006a7a0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f01f8006fd0 tx=0x7f01f8008040 comp rx=0 tx=0).stop 2023-11-28T11:10:24.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.404+0000 7f0212d6f700 1 -- 172.21.15.28:0/3501306127 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f020c0b1630 msgr2=0x7f020c154200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.404+0000 7f0212d6f700 1 --2- 172.21.15.28:0/3501306127 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f020c0b1630 0x7f020c154200 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f0200014040 tx=0x7f0200004750 comp rx=0 tx=0).stop 2023-11-28T11:10:24.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.404+0000 7f0212d6f700 1 -- 172.21.15.28:0/3501306127 shutdown_connections 2023-11-28T11:10:24.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.404+0000 7f0212d6f700 1 --2- 172.21.15.28:0/3501306127 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f01f00682f0 0x7f01f006a7a0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.404+0000 7f0212d6f700 1 --2- 172.21.15.28:0/3501306127 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f020c0b1630 0x7f020c154200 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.404+0000 7f0212d6f700 1 --2- 172.21.15.28:0/3501306127 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f020c15dbf0 0x7f020c0c8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.404+0000 7f0212d6f700 1 --2- 172.21.15.28:0/3501306127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f020c159500 0x7f020c15c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.404+0000 7f0212d6f700 1 -- 172.21.15.28:0/3501306127 >> 172.21.15.28:0/3501306127 conn(0x7f020c0cc630 msgr2=0x7f020c0cd760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:24.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.404+0000 7f0212d6f700 1 -- 172.21.15.28:0/3501306127 shutdown_connections 2023-11-28T11:10:24.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.404+0000 7f0212d6f700 1 -- 172.21.15.28:0/3501306127 wait complete. 2023-11-28T11:10:24.418 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:10:24.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.545+0000 7f9ab215c700 1 Processor -- start 2023-11-28T11:10:24.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.545+0000 7f9ab215c700 1 -- start start 2023-11-28T11:10:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9ab215c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9aac0b1630 0x7f9aac0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9ab215c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9aac0b1a00 0x7f9aac0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9ab215c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9aac0cb3e0 0x7f9aac160350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9ab215c700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aac0ce550 con 0x7f9aac0b1630 2023-11-28T11:10:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9ab215c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9aac0ce830 con 0x7f9aac0cb3e0 2023-11-28T11:10:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9ab215c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac0ceb10 con 0x7f9aac0b1a00 2023-11-28T11:10:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9aab7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9aac0b1630 0x7f9aac0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9aab7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9aac0b1630 0x7f9aac0cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49748/0 (socket says 172.21.15.28:49748) 2023-11-28T11:10:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9aab7fe700 1 -- 172.21.15.28:0/3913263029 learned_addr learned my addr 172.21.15.28:0/3913263029 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9aaaffd700 1 --2- 172.21.15.28:0/3913263029 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9aac0b1a00 0x7f9aac0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9aabfff700 1 --2- 172.21.15.28:0/3913263029 >> v2:172.21.15.59:3300/0 conn(0x7f9aac0cb3e0 0x7f9aac160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9aab7fe700 1 -- 172.21.15.28:0/3913263029 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9aac0b1a00 msgr2=0x7f9aac0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9aab7fe700 1 --2- 172.21.15.28:0/3913263029 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9aac0b1a00 0x7f9aac0caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9aab7fe700 1 -- 172.21.15.28:0/3913263029 >> v2:172.21.15.59:3300/0 conn(0x7f9aac0cb3e0 msgr2=0x7f9aac160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9aab7fe700 1 --2- 172.21.15.28:0/3913263029 >> v2:172.21.15.59:3300/0 conn(0x7f9aac0cb3e0 0x7f9aac160350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9aab7fe700 1 -- 172.21.15.28:0/3913263029 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac0cedf0 con 0x7f9aac0b1630 2023-11-28T11:10:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9aab7fe700 1 --2- 172.21.15.28:0/3913263029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9aac0b1630 0x7f9aac0cf5f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9aac06eb10 tx=0x7f9a9c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a82777c4cdb4e5f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:24.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9aaa7fc700 1 -- 172.21.15.28:0/3913263029 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c00b870 con 0x7f9aac0b1630 2023-11-28T11:10:24.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9aaa7fc700 1 -- 172.21.15.28:0/3913263029 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9c00b9d0 con 0x7f9aac0b1630 2023-11-28T11:10:24.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.546+0000 7f9aaa7fc700 1 -- 172.21.15.28:0/3913263029 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9c00bd60 con 0x7f9aac0b1630 2023-11-28T11:10:24.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.547+0000 7f9ab215c700 1 -- 172.21.15.28:0/3913263029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9aac0b1630 msgr2=0x7f9aac0cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.547+0000 7f9ab215c700 1 --2- 172.21.15.28:0/3913263029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9aac0b1630 0x7f9aac0cf5f0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f9aac06eb10 tx=0x7f9a9c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.547+0000 7f9ab215c700 1 -- 172.21.15.28:0/3913263029 shutdown_connections 2023-11-28T11:10:24.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.547+0000 7f9ab215c700 1 --2- 172.21.15.28:0/3913263029 >> v2:172.21.15.59:3300/0 conn(0x7f9aac0cb3e0 0x7f9aac160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.547+0000 7f9ab215c700 1 --2- 172.21.15.28:0/3913263029 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9aac0b1630 0x7f9aac0cf5f0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.547+0000 7f9ab215c700 1 --2- 172.21.15.28:0/3913263029 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9aac0b1a00 0x7f9aac0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.547+0000 7f9ab215c700 1 -- 172.21.15.28:0/3913263029 >> 172.21.15.28:0/3913263029 conn(0x7f9aac0cc630 msgr2=0x7f9aac0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:24.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.547+0000 7f9ab215c700 1 -- 172.21.15.28:0/3913263029 shutdown_connections 2023-11-28T11:10:24.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.547+0000 7f9ab215c700 1 -- 172.21.15.28:0/3913263029 wait complete. 2023-11-28T11:10:24.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.548+0000 7f9ab215c700 1 Processor -- start 2023-11-28T11:10:24.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.548+0000 7f9ab215c700 1 -- start start 2023-11-28T11:10:24.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.549+0000 7f9ab215c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9aac0b1630 0x7f9aac0b2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.549+0000 7f9ab215c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9aac0b1a00 0x7f9aac14dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.549+0000 7f9ab215c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9aac0cb3e0 0x7f9aac0c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.549+0000 7f9ab215c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac15e3f0 con 0x7f9aac0b1a00 2023-11-28T11:10:24.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.549+0000 7f9ab215c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac0b3100 con 0x7f9aac0b1630 2023-11-28T11:10:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.549+0000 7f9ab215c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9aac15d060 con 0x7f9aac0cb3e0 2023-11-28T11:10:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.549+0000 7f9aaaffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9aac0b1a00 0x7f9aac14dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.549+0000 7f9aaaffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9aac0b1a00 0x7f9aac14dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49756/0 (socket says 172.21.15.28:49756) 2023-11-28T11:10:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.549+0000 7f9aaaffd700 1 -- 172.21.15.28:0/4229008542 learned_addr learned my addr 172.21.15.28:0/4229008542 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.549+0000 7f9aabfff700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9aac0cb3e0 0x7f9aac0c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.549+0000 7f9aab7fe700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9aac0b1630 0x7f9aac0b2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.549+0000 7f9aaaffd700 1 -- 172.21.15.28:0/4229008542 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9aac0cb3e0 msgr2=0x7f9aac0c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.549+0000 7f9aaaffd700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9aac0cb3e0 0x7f9aac0c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.550+0000 7f9aaaffd700 1 -- 172.21.15.28:0/4229008542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9aac0b1630 msgr2=0x7f9aac0b2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.550+0000 7f9aaaffd700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9aac0b1630 0x7f9aac0b2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.550+0000 7f9aaaffd700 1 -- 172.21.15.28:0/4229008542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a9c009550 con 0x7f9aac0b1a00 2023-11-28T11:10:24.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.550+0000 7f9aabfff700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9aac0cb3e0 0x7f9aac0c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:24.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.550+0000 7f9aab7fe700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9aac0b1630 0x7f9aac0b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:24.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.550+0000 7f9aaaffd700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9aac0b1a00 0x7f9aac14dfc0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000c7f0 tx=0x7f9aa000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:24.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.550+0000 7f9a93fff700 1 -- 172.21.15.28:0/4229008542 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0011800 con 0x7f9aac0b1a00 2023-11-28T11:10:24.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.550+0000 7f9a93fff700 1 -- 172.21.15.28:0/4229008542 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa0011960 con 0x7f9aac0b1a00 2023-11-28T11:10:24.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.550+0000 7f9ab215c700 1 -- 172.21.15.28:0/4229008542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9aac0b1e70 con 0x7f9aac0b1a00 2023-11-28T11:10:24.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.550+0000 7f9a93fff700 1 -- 172.21.15.28:0/4229008542 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa0011d10 con 0x7f9aac0b1a00 2023-11-28T11:10:24.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.550+0000 7f9ab215c700 1 -- 172.21.15.28:0/4229008542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9aac0cf0d0 con 0x7f9aac0b1a00 2023-11-28T11:10:24.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.551+0000 7f9ab215c700 1 -- 172.21.15.28:0/4229008542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9aac0ce140 con 0x7f9aac0b1a00 2023-11-28T11:10:24.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.551+0000 7f9a93fff700 1 -- 172.21.15.28:0/4229008542 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9aa0021070 con 0x7f9aac0b1a00 2023-11-28T11:10:24.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.552+0000 7f9a93fff700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9a980682f0 0x7f9a9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.552+0000 7f9a93fff700 1 -- 172.21.15.28:0/4229008542 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9aa0085c20 con 0x7f9aac0b1a00 2023-11-28T11:10:24.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.552+0000 7f9aab7fe700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9a980682f0 0x7f9a9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.552+0000 7f9aab7fe700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9a980682f0 0x7f9a9806a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c000c00 tx=0x7f9a9c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:24.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.556+0000 7f9a93fff700 1 -- 172.21.15.28:0/4229008542 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9aa00522b0 con 0x7f9aac0b1a00 2023-11-28T11:10:24.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.749+0000 7f9ab215c700 1 -- 172.21.15.28:0/4229008542 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9aac15e530 con 0x7f9aac0b1a00 2023-11-28T11:10:24.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.750+0000 7f9a93fff700 1 -- 172.21.15.28:0/4229008542 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v40) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9aac15e530 con 0x7f9aac0b1a00 2023-11-28T11:10:24.752 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:10:24.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.752+0000 7f9ab215c700 1 -- 172.21.15.28:0/4229008542 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9a980682f0 msgr2=0x7f9a9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.752+0000 7f9ab215c700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9a980682f0 0x7f9a9806a7a0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c000c00 tx=0x7f9a9c008030 comp rx=0 tx=0).stop 2023-11-28T11:10:24.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.752+0000 7f9ab215c700 1 -- 172.21.15.28:0/4229008542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9aac0b1a00 msgr2=0x7f9aac14dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.752+0000 7f9ab215c700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9aac0b1a00 0x7f9aac14dfc0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f9aa000c7f0 tx=0x7f9aa000cbb0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.752+0000 7f9ab215c700 1 -- 172.21.15.28:0/4229008542 shutdown_connections 2023-11-28T11:10:24.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.752+0000 7f9ab215c700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9a980682f0 0x7f9a9806a7a0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.752+0000 7f9ab215c700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9aac0cb3e0 0x7f9aac0c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.752+0000 7f9ab215c700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9aac0b1630 0x7f9aac0b2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.752+0000 7f9ab215c700 1 --2- 172.21.15.28:0/4229008542 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9aac0b1a00 0x7f9aac14dfc0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.752+0000 7f9ab215c700 1 -- 172.21.15.28:0/4229008542 >> 172.21.15.28:0/4229008542 conn(0x7f9aac0cc630 msgr2=0x7f9aac06c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:24.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.752+0000 7f9ab215c700 1 -- 172.21.15.28:0/4229008542 shutdown_connections 2023-11-28T11:10:24.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.752+0000 7f9ab215c700 1 -- 172.21.15.28:0/4229008542 wait complete. 2023-11-28T11:10:24.765 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:10:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.889+0000 7fcd4c732700 1 Processor -- start 2023-11-28T11:10:24.891 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.889+0000 7fcd4c732700 1 -- start start 2023-11-28T11:10:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.890+0000 7fcd4c732700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcd44159590 0x7fcd4406e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.890+0000 7fcd4c732700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd4415d120 con 0x7fcd440ce890 2023-11-28T11:10:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.890+0000 7fcd4c732700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd4415d6e0 con 0x7fcd4415db70 2023-11-28T11:10:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.890+0000 7fcd4c732700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd440aef60 con 0x7fcd44159590 2023-11-28T11:10:24.892 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.890+0000 7fcd4a4ce700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcd44159590 0x7fcd4406e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.890+0000 7fcd4accf700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fcd440ce890 0x7fcd440caf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37822/0 (socket says 172.21.15.28:37822) 2023-11-28T11:10:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.890+0000 7fcd4accf700 1 -- 172.21.15.28:0/3455694736 learned_addr learned my addr 172.21.15.28:0/3455694736 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.891+0000 7fcd4a4ce700 1 -- 172.21.15.28:0/3455694736 >> v1:172.21.15.59:6789/0 conn(0x7fcd4415db70 legacy=0x7fcd440ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.891+0000 7fcd4a4ce700 1 -- 172.21.15.28:0/3455694736 >> v1:172.21.15.28:6789/0 conn(0x7fcd440ce890 legacy=0x7fcd440caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.891+0000 7fcd494cc700 1 -- 172.21.15.28:0/3455694736 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465333313 0 0) 0x7fcd4415d120 con 0x7fcd440ce890 2023-11-28T11:10:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.891+0000 7fcd4a4ce700 1 -- 172.21.15.28:0/3455694736 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd440af240 con 0x7fcd44159590 2023-11-28T11:10:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.891+0000 7fcd4a4ce700 1 --2- 172.21.15.28:0/3455694736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd44159590 0x7fcd4406e400 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcd34009840 tx=0x7fcd34009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99562272e6893712 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.891+0000 7fcd494cc700 1 -- 172.21.15.28:0/3455694736 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd34004230 con 0x7fcd44159590 2023-11-28T11:10:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.891+0000 7fcd494cc700 1 -- 172.21.15.28:0/3455694736 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd34004390 con 0x7fcd44159590 2023-11-28T11:10:24.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.891+0000 7fcd494cc700 1 -- 172.21.15.28:0/3455694736 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd340036a0 con 0x7fcd44159590 2023-11-28T11:10:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.891+0000 7fcd4c732700 1 -- 172.21.15.28:0/3455694736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd44159590 msgr2=0x7fcd4406e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.891+0000 7fcd4c732700 1 --2- 172.21.15.28:0/3455694736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd44159590 0x7fcd4406e400 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fcd34009840 tx=0x7fcd34009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.891+0000 7fcd4c732700 1 -- 172.21.15.28:0/3455694736 shutdown_connections 2023-11-28T11:10:24.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.891+0000 7fcd4c732700 1 --2- 172.21.15.28:0/3455694736 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd44159590 0x7fcd4406e400 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.891+0000 7fcd4c732700 1 -- 172.21.15.28:0/3455694736 >> 172.21.15.28:0/3455694736 conn(0x7fcd440b1650 msgr2=0x7fcd440b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.891+0000 7fcd4c732700 1 -- 172.21.15.28:0/3455694736 shutdown_connections 2023-11-28T11:10:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.892+0000 7fcd4c732700 1 -- 172.21.15.28:0/3455694736 wait complete. 2023-11-28T11:10:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.893+0000 7fcd4c732700 1 Processor -- start 2023-11-28T11:10:24.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.893+0000 7fcd4c732700 1 -- start start 2023-11-28T11:10:24.897 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.893+0000 7fcd4c732700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcd440ce890 0x7fcd44156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.897 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.893+0000 7fcd4c732700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd44159590 0x7fcd440c96f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.893+0000 7fcd4c732700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd4415db70 0x7fcd44153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.893+0000 7fcd4c732700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd44155470 con 0x7fcd44159590 2023-11-28T11:10:24.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.893+0000 7fcd4c732700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd440c9c30 con 0x7fcd4415db70 2023-11-28T11:10:24.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.893+0000 7fcd4c732700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd441535c0 con 0x7fcd440ce890 2023-11-28T11:10:24.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.893+0000 7fcd49ccd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd44159590 0x7fcd440c96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd49ccd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd44159590 0x7fcd440c96f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49768/0 (socket says 172.21.15.28:49768) 2023-11-28T11:10:24.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd49ccd700 1 -- 172.21.15.28:0/2295403302 learned_addr learned my addr 172.21.15.28:0/2295403302 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:24.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd4a4ce700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcd440ce890 0x7fcd44156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd4accf700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd4415db70 0x7fcd44153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd49ccd700 1 -- 172.21.15.28:0/2295403302 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcd440ce890 msgr2=0x7fcd44156410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd49ccd700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcd440ce890 0x7fcd44156410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd49ccd700 1 -- 172.21.15.28:0/2295403302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd4415db70 msgr2=0x7fcd44153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:24.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd49ccd700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd4415db70 0x7fcd44153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:24.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd49ccd700 1 -- 172.21.15.28:0/2295403302 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd340094a0 con 0x7fcd44159590 2023-11-28T11:10:24.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd4a4ce700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcd440ce890 0x7fcd44156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:24.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd4accf700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd4415db70 0x7fcd44153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:24.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd49ccd700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd44159590 0x7fcd440c96f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000b4d0 tx=0x7fcd4000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:24.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd3affd700 1 -- 172.21.15.28:0/2295403302 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40011830 con 0x7fcd44159590 2023-11-28T11:10:24.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd3affd700 1 -- 172.21.15.28:0/2295403302 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd40011990 con 0x7fcd44159590 2023-11-28T11:10:24.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd3affd700 1 -- 172.21.15.28:0/2295403302 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd40011d40 con 0x7fcd44159590 2023-11-28T11:10:24.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd4c732700 1 -- 172.21.15.28:0/2295403302 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd4415e010 con 0x7fcd44159590 2023-11-28T11:10:24.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.894+0000 7fcd4c732700 1 -- 172.21.15.28:0/2295403302 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd4415d370 con 0x7fcd44159590 2023-11-28T11:10:24.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.895+0000 7fcd4c732700 1 -- 172.21.15.28:0/2295403302 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd44159960 con 0x7fcd44159590 2023-11-28T11:10:24.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.896+0000 7fcd3affd700 1 -- 172.21.15.28:0/2295403302 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcd4000f050 con 0x7fcd44159590 2023-11-28T11:10:24.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.896+0000 7fcd3affd700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcd280682f0 0x7fcd2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:24.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.896+0000 7fcd3affd700 1 -- 172.21.15.28:0/2295403302 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fcd40085980 con 0x7fcd44159590 2023-11-28T11:10:24.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.896+0000 7fcd4a4ce700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcd280682f0 0x7fcd2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:24.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.897+0000 7fcd4a4ce700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcd280682f0 0x7fcd2806a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcd34000c00 tx=0x7fcd3401c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:24.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:24.900+0000 7fcd3affd700 1 -- 172.21.15.28:0/2295403302 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcd400520a0 con 0x7fcd44159590 2023-11-28T11:10:25.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:25.107+0000 7fcd4c732700 1 -- 172.21.15.28:0/2295403302 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fcd4406c1b0 con 0x7fcd44159590 2023-11-28T11:10:25.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:25.108+0000 7fcd3affd700 1 -- 172.21.15.28:0/2295403302 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v40) v1 ==== 80+0+0 (secure 0 0 0) 0x7fcd400556c0 con 0x7fcd44159590 2023-11-28T11:10:25.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:25.110+0000 7fcd4c732700 1 -- 172.21.15.28:0/2295403302 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcd280682f0 msgr2=0x7fcd2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:25.110+0000 7fcd4c732700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcd280682f0 0x7fcd2806a7a0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fcd34000c00 tx=0x7fcd3401c040 comp rx=0 tx=0).stop 2023-11-28T11:10:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:25.110+0000 7fcd4c732700 1 -- 172.21.15.28:0/2295403302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd44159590 msgr2=0x7fcd440c96f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:25.110+0000 7fcd4c732700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd44159590 0x7fcd440c96f0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fcd4000b4d0 tx=0x7fcd4000b890 comp rx=0 tx=0).stop 2023-11-28T11:10:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:25.110+0000 7fcd4c732700 1 -- 172.21.15.28:0/2295403302 shutdown_connections 2023-11-28T11:10:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:25.110+0000 7fcd4c732700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcd280682f0 0x7fcd2806a7a0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:25.110+0000 7fcd4c732700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcd4415db70 0x7fcd44153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:25.110+0000 7fcd4c732700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcd440ce890 0x7fcd44156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:25.110+0000 7fcd4c732700 1 --2- 172.21.15.28:0/2295403302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcd44159590 0x7fcd440c96f0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:25.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:25.110+0000 7fcd4c732700 1 -- 172.21.15.28:0/2295403302 >> 172.21.15.28:0/2295403302 conn(0x7fcd440b1650 msgr2=0x7fcd440cc610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:25.110+0000 7fcd4c732700 1 -- 172.21.15.28:0/2295403302 shutdown_connections 2023-11-28T11:10:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:25.110+0000 7fcd4c732700 1 -- 172.21.15.28:0/2295403302 wait complete. 2023-11-28T11:10:25.113 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2023-11-28T11:10:25.123 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi028.front.sepia.ceph.com... 2023-11-28T11:10:25.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:25.124 DEBUG:teuthology.orchestra.run.smithi028:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T11:10:25.157 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[121827]: fuse finished with error 0 and tester_r 0 2023-11-28T11:10:25.159 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:31.261 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:10:31.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:31.262 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:10:31.284 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:31.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.462+0000 7f4c25b28700 1 Processor -- start 2023-11-28T11:10:31.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.462+0000 7f4c25b28700 1 -- start start 2023-11-28T11:10:31.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.462+0000 7f4c25b28700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c2015c990 0x7f4c200cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.462+0000 7f4c25b28700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4c20161090 0x7f4c200ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.462+0000 7f4c25b28700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c200caeb0 0x7f4c200cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.462+0000 7f4c25b28700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c200ce340 con 0x7f4c200caeb0 2023-11-28T11:10:31.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.462+0000 7f4c25b28700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c200cebe0 con 0x7f4c20161090 2023-11-28T11:10:31.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.462+0000 7f4c25b28700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c200cb3f0 con 0x7f4c2015c990 2023-11-28T11:10:31.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.463+0000 7f4c1ffff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c200caeb0 0x7f4c200cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.463+0000 7f4c1ffff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4c200caeb0 0x7f4c200cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49778/0 (socket says 172.21.15.28:49778) 2023-11-28T11:10:31.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.463+0000 7f4c1ffff700 1 -- 172.21.15.28:0/1623866494 learned_addr learned my addr 172.21.15.28:0/1623866494 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:31.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.463+0000 7f4c1f7fe700 1 --2- 172.21.15.28:0/1623866494 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c2015c990 0x7f4c200cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.463+0000 7f4c1effd700 1 --2- 172.21.15.28:0/1623866494 >> v2:172.21.15.59:3300/0 conn(0x7f4c20161090 0x7f4c200ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.463+0000 7f4c1ffff700 1 -- 172.21.15.28:0/1623866494 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c2015c990 msgr2=0x7f4c200cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:31.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.463+0000 7f4c1ffff700 1 --2- 172.21.15.28:0/1623866494 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c2015c990 0x7f4c200cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.463+0000 7f4c1ffff700 1 -- 172.21.15.28:0/1623866494 >> v2:172.21.15.59:3300/0 conn(0x7f4c20161090 msgr2=0x7f4c200ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:31.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.463+0000 7f4c1ffff700 1 --2- 172.21.15.28:0/1623866494 >> v2:172.21.15.59:3300/0 conn(0x7f4c20161090 0x7f4c200ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.463+0000 7f4c1ffff700 1 -- 172.21.15.28:0/1623866494 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c200ce590 con 0x7f4c200caeb0 2023-11-28T11:10:31.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.463+0000 7f4c1ffff700 1 --2- 172.21.15.28:0/1623866494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c200caeb0 0x7f4c200cc310 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4c2006f850 tx=0x7f4c1800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc713e370baf27f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:31.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.463+0000 7f4c1e7fc700 1 -- 172.21.15.28:0/1623866494 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1800e070 con 0x7f4c200caeb0 2023-11-28T11:10:31.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.463+0000 7f4c1e7fc700 1 -- 172.21.15.28:0/1623866494 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1800b910 con 0x7f4c200caeb0 2023-11-28T11:10:31.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.463+0000 7f4c1e7fc700 1 -- 172.21.15.28:0/1623866494 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1800bcc0 con 0x7f4c200caeb0 2023-11-28T11:10:31.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.464+0000 7f4c25b28700 1 -- 172.21.15.28:0/1623866494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c200caeb0 msgr2=0x7f4c200cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:31.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.464+0000 7f4c25b28700 1 --2- 172.21.15.28:0/1623866494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c200caeb0 0x7f4c200cc310 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f4c2006f850 tx=0x7f4c1800b330 comp rx=0 tx=0).stop 2023-11-28T11:10:31.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.464+0000 7f4c25b28700 1 -- 172.21.15.28:0/1623866494 shutdown_connections 2023-11-28T11:10:31.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.464+0000 7f4c25b28700 1 --2- 172.21.15.28:0/1623866494 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c200caeb0 0x7f4c200cc310 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.464+0000 7f4c25b28700 1 --2- 172.21.15.28:0/1623866494 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c2015c990 0x7f4c200cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.464+0000 7f4c25b28700 1 --2- 172.21.15.28:0/1623866494 >> v2:172.21.15.59:3300/0 conn(0x7f4c20161090 0x7f4c200ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.464+0000 7f4c25b28700 1 -- 172.21.15.28:0/1623866494 >> 172.21.15.28:0/1623866494 conn(0x7f4c200b1650 msgr2=0x7f4c200b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:31.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.464+0000 7f4c25b28700 1 -- 172.21.15.28:0/1623866494 shutdown_connections 2023-11-28T11:10:31.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.464+0000 7f4c25b28700 1 -- 172.21.15.28:0/1623866494 wait complete. 2023-11-28T11:10:31.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.465+0000 7f4c25b28700 1 Processor -- start 2023-11-28T11:10:31.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.465+0000 7f4c25b28700 1 -- start start 2023-11-28T11:10:31.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.466+0000 7f4c25b28700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c200caeb0 0x7f4c201e8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.466+0000 7f4c25b28700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c2015c990 0x7f4c2015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.466+0000 7f4c25b28700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c20161090 0x7f4c200c3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.466+0000 7f4c25b28700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c200b3f30 con 0x7f4c2015c990 2023-11-28T11:10:31.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.466+0000 7f4c25b28700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c200b3320 con 0x7f4c200caeb0 2023-11-28T11:10:31.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.466+0000 7f4c25b28700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c200b3600 con 0x7f4c20161090 2023-11-28T11:10:31.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.466+0000 7f4c1effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c2015c990 0x7f4c2015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.466+0000 7f4c1effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c2015c990 0x7f4c2015eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49784/0 (socket says 172.21.15.28:49784) 2023-11-28T11:10:31.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.466+0000 7f4c1effd700 1 -- 172.21.15.28:0/159182775 learned_addr learned my addr 172.21.15.28:0/159182775 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.466+0000 7f4c1ffff700 1 --2- 172.21.15.28:0/159182775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c20161090 0x7f4c200c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.466+0000 7f4c1f7fe700 1 --2- 172.21.15.28:0/159182775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c200caeb0 0x7f4c201e8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.467+0000 7f4c1effd700 1 -- 172.21.15.28:0/159182775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c20161090 msgr2=0x7f4c200c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.467+0000 7f4c1effd700 1 --2- 172.21.15.28:0/159182775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c20161090 0x7f4c200c3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.467+0000 7f4c1effd700 1 -- 172.21.15.28:0/159182775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c200caeb0 msgr2=0x7f4c201e8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.467+0000 7f4c1effd700 1 --2- 172.21.15.28:0/159182775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c200caeb0 0x7f4c201e8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.467+0000 7f4c1effd700 1 -- 172.21.15.28:0/159182775 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c18009b30 con 0x7f4c2015c990 2023-11-28T11:10:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.467+0000 7f4c1effd700 1 --2- 172.21.15.28:0/159182775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c2015c990 0x7f4c2015eef0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400e840 tx=0x7f4c1400eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.467+0000 7f4c07fff700 1 -- 172.21.15.28:0/159182775 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14009590 con 0x7f4c2015c990 2023-11-28T11:10:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.467+0000 7f4c07fff700 1 -- 172.21.15.28:0/159182775 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c14013680 con 0x7f4c2015c990 2023-11-28T11:10:31.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.467+0000 7f4c25b28700 1 -- 172.21.15.28:0/159182775 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c200cf5b0 con 0x7f4c2015c990 2023-11-28T11:10:31.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.467+0000 7f4c07fff700 1 -- 172.21.15.28:0/159182775 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14013a30 con 0x7f4c2015c990 2023-11-28T11:10:31.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.467+0000 7f4c25b28700 1 -- 172.21.15.28:0/159182775 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c200ce870 con 0x7f4c2015c990 2023-11-28T11:10:31.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.468+0000 7f4c25b28700 1 -- 172.21.15.28:0/159182775 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c200cf740 con 0x7f4c2015c990 2023-11-28T11:10:31.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.468+0000 7f4c07fff700 1 -- 172.21.15.28:0/159182775 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4c14019030 con 0x7f4c2015c990 2023-11-28T11:10:31.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.469+0000 7f4c07fff700 1 --2- 172.21.15.28:0/159182775 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4c0c0682f0 0x7f4c0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:31.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.469+0000 7f4c07fff700 1 -- 172.21.15.28:0/159182775 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4c14085a80 con 0x7f4c2015c990 2023-11-28T11:10:31.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.469+0000 7f4c1f7fe700 1 --2- 172.21.15.28:0/159182775 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4c0c0682f0 0x7f4c0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:31.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.469+0000 7f4c1f7fe700 1 --2- 172.21.15.28:0/159182775 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4c0c0682f0 0x7f4c0c06a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4c200ca920 tx=0x7f4c10009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:31.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.473+0000 7f4c07fff700 1 -- 172.21.15.28:0/159182775 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4c140521a0 con 0x7f4c2015c990 2023-11-28T11:10:31.717 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.716+0000 7f4c25b28700 1 -- 172.21.15.28:0/159182775 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c200675a0 con 0x7f4c2015c990 2023-11-28T11:10:31.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.717+0000 7f4c07fff700 1 -- 172.21.15.28:0/159182775 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f4c140557c0 con 0x7f4c2015c990 2023-11-28T11:10:31.718 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:31.718 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2867100265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2867100265},{"type":"v1","addr":"172.21.15.59:6833","nonce":2867100265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3808190953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3808190953},{"type":"v1","addr":"172.21.15.28:6837","nonce":3808190953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3920280453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3920280453},{"type":"v1","addr":"172.21.15.28:6836","nonce":3920280453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:10.820923+0000","modified":"2023-11-28T11:10:13.833216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6835/707419743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":707419743},{"type":"v1","addr":"172.21.15.59:6835","nonce":707419743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:10:31.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.719+0000 7f4c25b28700 1 -- 172.21.15.28:0/159182775 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4c0c0682f0 msgr2=0x7f4c0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:31.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.719+0000 7f4c25b28700 1 --2- 172.21.15.28:0/159182775 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4c0c0682f0 0x7f4c0c06a7a0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4c200ca920 tx=0x7f4c10009210 comp rx=0 tx=0).stop 2023-11-28T11:10:31.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.719+0000 7f4c25b28700 1 -- 172.21.15.28:0/159182775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c2015c990 msgr2=0x7f4c2015eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:31.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.719+0000 7f4c25b28700 1 --2- 172.21.15.28:0/159182775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c2015c990 0x7f4c2015eef0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f4c1400e840 tx=0x7f4c1400eb50 comp rx=0 tx=0).stop 2023-11-28T11:10:31.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.719+0000 7f4c25b28700 1 -- 172.21.15.28:0/159182775 shutdown_connections 2023-11-28T11:10:31.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.719+0000 7f4c25b28700 1 --2- 172.21.15.28:0/159182775 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4c0c0682f0 0x7f4c0c06a7a0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.719+0000 7f4c25b28700 1 --2- 172.21.15.28:0/159182775 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4c20161090 0x7f4c200c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.719+0000 7f4c25b28700 1 --2- 172.21.15.28:0/159182775 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4c200caeb0 0x7f4c201e8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.719+0000 7f4c25b28700 1 --2- 172.21.15.28:0/159182775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4c2015c990 0x7f4c2015eef0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:31.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.719+0000 7f4c25b28700 1 -- 172.21.15.28:0/159182775 >> 172.21.15.28:0/159182775 conn(0x7f4c200b1650 msgr2=0x7f4c2006aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:31.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.719+0000 7f4c25b28700 1 -- 172.21.15.28:0/159182775 shutdown_connections 2023-11-28T11:10:31.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:31.719+0000 7f4c25b28700 1 -- 172.21.15.28:0/159182775 wait complete. 2023-11-28T11:10:31.722 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2023-11-28T11:10:31.733 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-28T11:10:31.898 INFO:teuthology.orchestra.run.smithi059.stdout:{"message":"","return_code":0} 2023-11-28T11:10:31.898 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:10:31.898 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:32.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.023+0000 7f0880c6d700 1 Processor -- start 2023-11-28T11:10:32.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.023+0000 7f0880c6d700 1 -- start start 2023-11-28T11:10:32.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.024+0000 7f0880c6d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f087c06d9d0 0x7f087c15b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.024+0000 7f0880c6d700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087c06c3a0 con 0x7f087c06dda0 2023-11-28T11:10:32.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.024+0000 7f0880c6d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f087c06c0c0 con 0x7f087c06bc30 2023-11-28T11:10:32.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.024+0000 7f0880c6d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c15ab60 con 0x7f087c06d9d0 2023-11-28T11:10:32.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.024+0000 7f087ad9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f087c06dda0 0x7f087c0c9930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37846/0 (socket says 172.21.15.28:37846) 2023-11-28T11:10:32.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.024+0000 7f087ad9d700 1 -- 172.21.15.28:0/2986078820 learned_addr learned my addr 172.21.15.28:0/2986078820 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:32.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.024+0000 7f087a59c700 1 --2- 172.21.15.28:0/2986078820 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f087c06d9d0 0x7f087c15b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.025+0000 7f087959a700 1 -- 172.21.15.28:0/2986078820 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2238341787 0 0) 0x7f087c06c3a0 con 0x7f087c06dda0 2023-11-28T11:10:32.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.025+0000 7f087959a700 1 -- 172.21.15.28:0/2986078820 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08640035b0 con 0x7f087c06dda0 2023-11-28T11:10:32.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.025+0000 7f087959a700 1 -- 172.21.15.28:0/2986078820 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2611308793 0 0) 0x7f087c06c0c0 con 0x7f087c06bc30 2023-11-28T11:10:32.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.025+0000 7f087959a700 1 -- 172.21.15.28:0/2986078820 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f087c06c3a0 con 0x7f087c06bc30 2023-11-28T11:10:32.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.025+0000 7f087a59c700 1 -- 172.21.15.28:0/2986078820 >> v1:172.21.15.59:6789/0 conn(0x7f087c06bc30 legacy=0x7f087c150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.025+0000 7f087a59c700 1 -- 172.21.15.28:0/2986078820 >> v1:172.21.15.28:6789/0 conn(0x7f087c06dda0 legacy=0x7f087c0c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.025+0000 7f087a59c700 1 -- 172.21.15.28:0/2986078820 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f087c15ae40 con 0x7f087c06d9d0 2023-11-28T11:10:32.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.025+0000 7f087a59c700 1 --2- 172.21.15.28:0/2986078820 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f087c06d9d0 0x7f087c15b640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0870009870 tx=0x7f0870009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b15dfb765cee529 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.025+0000 7f087959a700 1 -- 172.21.15.28:0/2986078820 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870011830 con 0x7f087c06d9d0 2023-11-28T11:10:32.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.025+0000 7f087959a700 1 -- 172.21.15.28:0/2986078820 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0870011990 con 0x7f087c06d9d0 2023-11-28T11:10:32.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.025+0000 7f087959a700 1 -- 172.21.15.28:0/2986078820 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0870011d00 con 0x7f087c06d9d0 2023-11-28T11:10:32.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.026+0000 7f0880c6d700 1 -- 172.21.15.28:0/2986078820 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f087c06d9d0 msgr2=0x7f087c15b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.026+0000 7f0880c6d700 1 --2- 172.21.15.28:0/2986078820 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f087c06d9d0 0x7f087c15b640 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0870009870 tx=0x7f0870009b80 comp rx=0 tx=0).stop 2023-11-28T11:10:32.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.026+0000 7f0880c6d700 1 -- 172.21.15.28:0/2986078820 shutdown_connections 2023-11-28T11:10:32.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.026+0000 7f0880c6d700 1 --2- 172.21.15.28:0/2986078820 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f087c06d9d0 0x7f087c15b640 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.026+0000 7f0880c6d700 1 -- 172.21.15.28:0/2986078820 >> 172.21.15.28:0/2986078820 conn(0x7f087c0b0dd0 msgr2=0x7f087c0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:32.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.026+0000 7f0880c6d700 1 -- 172.21.15.28:0/2986078820 shutdown_connections 2023-11-28T11:10:32.029 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.026+0000 7f0880c6d700 1 -- 172.21.15.28:0/2986078820 wait complete. 2023-11-28T11:10:32.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.027+0000 7f0880c6d700 1 Processor -- start 2023-11-28T11:10:32.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.027+0000 7f0880c6d700 1 -- start start 2023-11-28T11:10:32.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f0880c6d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f087c06bc30 0x7f087c0cceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f0880c6d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f087c06d9d0 0x7f087c153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f0880c6d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f087c06dda0 0x7f087c14db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f0880c6d700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c1542a0 con 0x7f087c06bc30 2023-11-28T11:10:32.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f0880c6d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0cd5b0 con 0x7f087c06d9d0 2023-11-28T11:10:32.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f0880c6d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f087c0cd890 con 0x7f087c06dda0 2023-11-28T11:10:32.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f087a59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f087c06bc30 0x7f087c0cceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f087a59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f087c06bc30 0x7f087c0cceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49794/0 (socket says 172.21.15.28:49794) 2023-11-28T11:10:32.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f087a59c700 1 -- 172.21.15.28:0/1233098191 learned_addr learned my addr 172.21.15.28:0/1233098191 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:32.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f087ad9d700 1 --2- 172.21.15.28:0/1233098191 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f087c06dda0 0x7f087c14db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f0879d9b700 1 --2- 172.21.15.28:0/1233098191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f087c06d9d0 0x7f087c153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f087a59c700 1 -- 172.21.15.28:0/1233098191 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f087c06dda0 msgr2=0x7f087c14db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f087a59c700 1 --2- 172.21.15.28:0/1233098191 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f087c06dda0 0x7f087c14db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f087a59c700 1 -- 172.21.15.28:0/1233098191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f087c06d9d0 msgr2=0x7f087c153d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f087a59c700 1 --2- 172.21.15.28:0/1233098191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f087c06d9d0 0x7f087c153d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f087a59c700 1 -- 172.21.15.28:0/1233098191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08700094a0 con 0x7f087c06bc30 2023-11-28T11:10:32.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f0879d9b700 1 --2- 172.21.15.28:0/1233098191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f087c06d9d0 0x7f087c153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:32.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.028+0000 7f087a59c700 1 --2- 172.21.15.28:0/1233098191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f087c06bc30 0x7f087c0cceb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f08700097c0 tx=0x7f0870004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.029+0000 7f086affd700 1 -- 172.21.15.28:0/1233098191 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08700044d0 con 0x7f087c06bc30 2023-11-28T11:10:32.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.029+0000 7f086affd700 1 -- 172.21.15.28:0/1233098191 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f087001f070 con 0x7f087c06bc30 2023-11-28T11:10:32.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.029+0000 7f0880c6d700 1 -- 172.21.15.28:0/1233098191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f087c06fac0 con 0x7f087c06bc30 2023-11-28T11:10:32.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.029+0000 7f086affd700 1 -- 172.21.15.28:0/1233098191 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08700038e0 con 0x7f087c06bc30 2023-11-28T11:10:32.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.029+0000 7f0880c6d700 1 -- 172.21.15.28:0/1233098191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f087c15b120 con 0x7f087c06bc30 2023-11-28T11:10:32.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.030+0000 7f0880c6d700 1 -- 172.21.15.28:0/1233098191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f087c1515c0 con 0x7f087c06bc30 2023-11-28T11:10:32.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.030+0000 7f086affd700 1 -- 172.21.15.28:0/1233098191 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f087000f050 con 0x7f087c06bc30 2023-11-28T11:10:32.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.031+0000 7f086affd700 1 --2- 172.21.15.28:0/1233098191 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f085c0682f0 0x7f085c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:32.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.031+0000 7f086affd700 1 -- 172.21.15.28:0/1233098191 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0870021070 con 0x7f087c06bc30 2023-11-28T11:10:32.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.031+0000 7f0879d9b700 1 --2- 172.21.15.28:0/1233098191 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f085c0682f0 0x7f085c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:32.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.031+0000 7f0879d9b700 1 --2- 172.21.15.28:0/1233098191 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f085c0682f0 0x7f085c06a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f087c150d50 tx=0x7f086c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:32.036 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.035+0000 7f086affd700 1 -- 172.21.15.28:0/1233098191 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0870053c60 con 0x7f087c06bc30 2023-11-28T11:10:32.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.273+0000 7f0880c6d700 1 -- 172.21.15.28:0/1233098191 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f087c0675a0 con 0x7f087c06bc30 2023-11-28T11:10:32.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.274+0000 7f086affd700 1 -- 172.21.15.28:0/1233098191 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f0870057280 con 0x7f087c06bc30 2023-11-28T11:10:32.275 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:32.276 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2867100265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2867100265},{"type":"v1","addr":"172.21.15.59:6833","nonce":2867100265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3808190953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3808190953},{"type":"v1","addr":"172.21.15.28:6837","nonce":3808190953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3920280453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3920280453},{"type":"v1","addr":"172.21.15.28:6836","nonce":3920280453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:10.820923+0000","modified":"2023-11-28T11:10:13.833216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6835/707419743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":707419743},{"type":"v1","addr":"172.21.15.59:6835","nonce":707419743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:10:32.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.276+0000 7f0880c6d700 1 -- 172.21.15.28:0/1233098191 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f085c0682f0 msgr2=0x7f085c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.276+0000 7f0880c6d700 1 --2- 172.21.15.28:0/1233098191 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f085c0682f0 0x7f085c06a7a0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f087c150d50 tx=0x7f086c009040 comp rx=0 tx=0).stop 2023-11-28T11:10:32.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.276+0000 7f0880c6d700 1 -- 172.21.15.28:0/1233098191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f087c06bc30 msgr2=0x7f087c0cceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:32.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.276+0000 7f0880c6d700 1 --2- 172.21.15.28:0/1233098191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f087c06bc30 0x7f087c0cceb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f08700097c0 tx=0x7f0870004750 comp rx=0 tx=0).stop 2023-11-28T11:10:32.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.276+0000 7f0880c6d700 1 -- 172.21.15.28:0/1233098191 shutdown_connections 2023-11-28T11:10:32.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.276+0000 7f0880c6d700 1 --2- 172.21.15.28:0/1233098191 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f085c0682f0 0x7f085c06a7a0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.276+0000 7f0880c6d700 1 --2- 172.21.15.28:0/1233098191 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f087c06bc30 0x7f087c0cceb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.276+0000 7f0880c6d700 1 --2- 172.21.15.28:0/1233098191 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f087c06dda0 0x7f087c14db20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.276+0000 7f0880c6d700 1 --2- 172.21.15.28:0/1233098191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f087c06d9d0 0x7f087c153d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:32.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.276+0000 7f0880c6d700 1 -- 172.21.15.28:0/1233098191 >> 172.21.15.28:0/1233098191 conn(0x7f087c0b0dd0 msgr2=0x7f087c1e4ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:32.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.276+0000 7f0880c6d700 1 -- 172.21.15.28:0/1233098191 shutdown_connections 2023-11-28T11:10:32.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:32.276+0000 7f0880c6d700 1 -- 172.21.15.28:0/1233098191 wait complete. 2023-11-28T11:10:32.279 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2023-11-28T11:10:32.290 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json tag path /parentdir mytag 2023-11-28T11:10:32.440 INFO:teuthology.orchestra.run.smithi059.stdout:{"return_code":0,"scrub_tag":"mytag","mode":"asynchronous"} 2023-11-28T11:10:32.441 DEBUG:tasks.cephfs.filesystem:_json_asok output { "mode": "asynchronous", "return_code": 0, "scrub_tag": "mytag" } 2023-11-28T11:10:42.441 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000000.00000000 scrub_tag 2023-11-28T11:10:42.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.492+0000 7fe6657dc080 1 Processor -- start 2023-11-28T11:10:42.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.492+0000 7fe6657dc080 1 -- start start 2023-11-28T11:10:42.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.493+0000 7fe6657dc080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56276c32c6c0 0x56276c3fcb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.493+0000 7fe6657dc080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x56276c32ca90 0x56276c3f77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.493+0000 7fe6657dc080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56276c3ffda0 0x56276c3f7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.493+0000 7fe6657dc080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276c2d2000 con 0x56276c32c6c0 2023-11-28T11:10:42.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.493+0000 7fe6657dc080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276c264fe0 con 0x56276c3ffda0 2023-11-28T11:10:42.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.493+0000 7fe6657dc080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x56276c32da80 con 0x56276c32ca90 2023-11-28T11:10:42.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.493+0000 7fe65b8df700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56276c32c6c0 0x56276c3fcb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.493+0000 7fe65b8df700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56276c32c6c0 0x56276c3fcb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49806/0 (socket says 172.21.15.28:49806) 2023-11-28T11:10:42.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.493+0000 7fe65b8df700 1 -- 172.21.15.28:0/3662192202 learned_addr learned my addr 172.21.15.28:0/3662192202 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:42.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.493+0000 7fe65b0de700 1 --2- 172.21.15.28:0/3662192202 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x56276c32ca90 0x56276c3f77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.493+0000 7fe65c0e0700 1 --2- 172.21.15.28:0/3662192202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56276c3ffda0 0x56276c3f7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe65b8df700 1 -- 172.21.15.28:0/3662192202 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x56276c32ca90 msgr2=0x56276c3f77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:42.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe65b8df700 1 --2- 172.21.15.28:0/3662192202 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x56276c32ca90 0x56276c3f77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe65b8df700 1 -- 172.21.15.28:0/3662192202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56276c3ffda0 msgr2=0x56276c3f7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:42.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe65b8df700 1 --2- 172.21.15.28:0/3662192202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56276c3ffda0 0x56276c3f7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe65b8df700 1 -- 172.21.15.28:0/3662192202 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe650005de0 con 0x56276c32c6c0 2023-11-28T11:10:42.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe65b0de700 1 --2- 172.21.15.28:0/3662192202 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x56276c32ca90 0x56276c3f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:42.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe65c0e0700 1 --2- 172.21.15.28:0/3662192202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56276c3ffda0 0x56276c3f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:42.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe65b8df700 1 --2- 172.21.15.28:0/3662192202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56276c32c6c0 0x56276c3fcb90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe64c001db0 tx=0x7fe64c00b510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39468468a424bf67 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe65a0dc700 1 -- 172.21.15.28:0/3662192202 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c014070 con 0x56276c32c6c0 2023-11-28T11:10:42.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe65a0dc700 1 -- 172.21.15.28:0/3662192202 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe64c00be70 con 0x56276c32c6c0 2023-11-28T11:10:42.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe65a0dc700 1 -- 172.21.15.28:0/3662192202 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe64c00f650 con 0x56276c32c6c0 2023-11-28T11:10:42.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe6657dc080 1 -- 172.21.15.28:0/3662192202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56276c32c6c0 msgr2=0x56276c3fcb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe6657dc080 1 --2- 172.21.15.28:0/3662192202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56276c32c6c0 0x56276c3fcb90 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fe64c001db0 tx=0x7fe64c00b510 comp rx=0 tx=0).stop 2023-11-28T11:10:42.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe6657dc080 1 -- 172.21.15.28:0/3662192202 shutdown_connections 2023-11-28T11:10:42.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe6657dc080 1 --2- 172.21.15.28:0/3662192202 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56276c32c6c0 0x56276c3fcb90 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe6657dc080 1 --2- 172.21.15.28:0/3662192202 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56276c3ffda0 0x56276c3f7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe6657dc080 1 --2- 172.21.15.28:0/3662192202 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x56276c32ca90 0x56276c3f77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.494+0000 7fe6657dc080 1 -- 172.21.15.28:0/3662192202 >> 172.21.15.28:0/3662192202 conn(0x56276c372060 msgr2=0x56276c3297d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:42.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.495+0000 7fe6657dc080 1 -- 172.21.15.28:0/3662192202 shutdown_connections 2023-11-28T11:10:42.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.495+0000 7fe6657dc080 1 -- 172.21.15.28:0/3662192202 wait complete. 2023-11-28T11:10:42.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.495+0000 7fe6657dc080 1 Processor -- start 2023-11-28T11:10:42.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.496+0000 7fe6657dc080 1 -- start start 2023-11-28T11:10:42.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.496+0000 7fe6657dc080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56276c32c6c0 0x56276c3f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe6657dc080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56276c32ca90 0x56276c4015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe6657dc080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x56276c3ffda0 0x56276c401b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe6657dc080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276c3f9bb0 con 0x56276c32ca90 2023-11-28T11:10:42.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe6657dc080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x56276c3f9e90 con 0x56276c32c6c0 2023-11-28T11:10:42.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe6657dc080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x56276c3fa170 con 0x56276c3ffda0 2023-11-28T11:10:42.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe65b0de700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56276c32ca90 0x56276c4015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe65b0de700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56276c32ca90 0x56276c4015c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49814/0 (socket says 172.21.15.28:49814) 2023-11-28T11:10:42.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe65b0de700 1 -- 172.21.15.28:0/3002758479 learned_addr learned my addr 172.21.15.28:0/3002758479 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:42.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe65b8df700 1 --2- 172.21.15.28:0/3002758479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56276c32c6c0 0x56276c3f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe65c0e0700 1 --2- 172.21.15.28:0/3002758479 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x56276c3ffda0 0x56276c401b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe65b0de700 1 -- 172.21.15.28:0/3002758479 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x56276c3ffda0 msgr2=0x56276c401b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe65b0de700 1 --2- 172.21.15.28:0/3002758479 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x56276c3ffda0 0x56276c401b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe65b0de700 1 -- 172.21.15.28:0/3002758479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56276c32c6c0 msgr2=0x56276c3f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe65b0de700 1 --2- 172.21.15.28:0/3002758479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56276c32c6c0 0x56276c3f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe65b0de700 1 -- 172.21.15.28:0/3002758479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56276c2704a0 con 0x56276c32ca90 2023-11-28T11:10:42.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe65b0de700 1 --2- 172.21.15.28:0/3002758479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56276c32ca90 0x56276c4015c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe650000c00 tx=0x7fe650002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe6590da700 1 -- 172.21.15.28:0/3002758479 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe650005310 con 0x56276c32ca90 2023-11-28T11:10:42.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe6590da700 1 -- 172.21.15.28:0/3002758479 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe650008d40 con 0x56276c32ca90 2023-11-28T11:10:42.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe6657dc080 1 -- 172.21.15.28:0/3002758479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x56276c270830 con 0x56276c32ca90 2023-11-28T11:10:42.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.497+0000 7fe6590da700 1 -- 172.21.15.28:0/3002758479 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe65001d680 con 0x56276c32ca90 2023-11-28T11:10:42.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.498+0000 7fe6657dc080 1 -- 172.21.15.28:0/3002758479 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56276c27d360 con 0x56276c32ca90 2023-11-28T11:10:42.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.499+0000 7fe6590da700 1 -- 172.21.15.28:0/3002758479 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe65001d7e0 con 0x56276c32ca90 2023-11-28T11:10:42.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.499+0000 7fe6590da700 1 --2- 172.21.15.28:0/3002758479 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe648066c10 0x7fe6480690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.499+0000 7fe6590da700 1 -- 172.21.15.28:0/3002758479 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fe650085a10 con 0x56276c32ca90 2023-11-28T11:10:42.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.499+0000 7fe65b8df700 1 --2- 172.21.15.28:0/3002758479 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe648066c10 0x7fe6480690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.500+0000 7fe6657dc080 1 --2- 172.21.15.28:0/3002758479 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x56276c359860 0x56276c359c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.500+0000 7fe6657dc080 1 -- 172.21.15.28:0/3002758479 --> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] -- osd_op(unknown.0.0:1 6.10 6:0cf6ad0f:::10000000000.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x56276c35a1a0 con 0x56276c359860 2023-11-28T11:10:42.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.500+0000 7fe65c0e0700 1 --2- 172.21.15.28:0/3002758479 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x56276c359860 0x56276c359c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.500+0000 7fe65b8df700 1 --2- 172.21.15.28:0/3002758479 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe648066c10 0x7fe6480690c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe64c001db0 tx=0x7fe64c011040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.500+0000 7fe65c0e0700 1 --2- 172.21.15.28:0/3002758479 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x56276c359860 0x56276c359c60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.501+0000 7fe65c0e0700 1 -- 172.21.15.28:0/3002758479 <== osd.0 v2:172.21.15.28:6800/103508 1 ==== osd_op_reply(1 10000000000.00000000 [getxattr (9) out=9b] v0'0 uv11 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fe654006020 con 0x56276c359860 2023-11-28T11:10:42.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.501+0000 7fe6657dc080 1 -- 172.21.15.28:0/3002758479 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x56276c359860 msgr2=0x56276c359c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.501+0000 7fe6657dc080 1 --2- 172.21.15.28:0/3002758479 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x56276c359860 0x56276c359c60 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.501+0000 7fe6657dc080 1 -- 172.21.15.28:0/3002758479 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe648066c10 msgr2=0x7fe6480690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.501+0000 7fe6657dc080 1 --2- 172.21.15.28:0/3002758479 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe648066c10 0x7fe6480690c0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fe64c001db0 tx=0x7fe64c011040 comp rx=0 tx=0).stop 2023-11-28T11:10:42.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.501+0000 7fe6657dc080 1 -- 172.21.15.28:0/3002758479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56276c32ca90 msgr2=0x56276c4015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.501+0000 7fe6657dc080 1 --2- 172.21.15.28:0/3002758479 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x56276c32ca90 0x56276c4015c0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fe650000c00 tx=0x7fe650002fb0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.501+0000 7fe65c0e0700 1 -- 172.21.15.28:0/3002758479 reap_dead start 2023-11-28T11:10:42.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.501+0000 7fe6657dc080 1 -- 172.21.15.28:0/3002758479 shutdown_connections 2023-11-28T11:10:42.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.501+0000 7fe6657dc080 1 -- 172.21.15.28:0/3002758479 >> 172.21.15.28:0/3002758479 conn(0x56276c372060 msgr2=0x56276c328730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:42.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.501+0000 7fe6657dc080 1 -- 172.21.15.28:0/3002758479 shutdown_connections 2023-11-28T11:10:42.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.501+0000 7fe6657dc080 1 -- 172.21.15.28:0/3002758479 wait complete. 2023-11-28T11:10:42.509 INFO:teuthology.orchestra.run.smithi028.stdout:mytag 2023-11-28T11:10:42.509 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_metadata getxattr 10000000001.00000000 scrub_tag 2023-11-28T11:10:42.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.554+0000 7fa5094df080 1 Processor -- start 2023-11-28T11:10:42.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.554+0000 7fa5094df080 1 -- start start 2023-11-28T11:10:42.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa5094df080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c43ed13bc0 0x55c43ede7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa5094df080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c43ede8720 0x55c43ed3f890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa5094df080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c43edeb410 0x55c43ed3e560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa5094df080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43ec4dfe0 con 0x55c43edeb410 2023-11-28T11:10:42.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa5094df080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43eddf3d0 con 0x55c43ed13bc0 2023-11-28T11:10:42.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa5094df080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c43ede7b70 con 0x55c43ede8720 2023-11-28T11:10:42.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa4ffde3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c43edeb410 0x55c43ed3e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa4ffde3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c43edeb410 0x55c43ed3e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49832/0 (socket says 172.21.15.28:49832) 2023-11-28T11:10:42.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa4ffde3700 1 -- 172.21.15.28:0/60825803 learned_addr learned my addr 172.21.15.28:0/60825803 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:42.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa4fede1700 1 --2- 172.21.15.28:0/60825803 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c43ede8720 0x55c43ed3f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa4ff5e2700 1 --2- 172.21.15.28:0/60825803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c43ed13bc0 0x55c43ede7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa4ffde3700 1 -- 172.21.15.28:0/60825803 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c43ede8720 msgr2=0x55c43ed3f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:42.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa4ffde3700 1 --2- 172.21.15.28:0/60825803 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c43ede8720 0x55c43ed3f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa4ffde3700 1 -- 172.21.15.28:0/60825803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c43ed13bc0 msgr2=0x55c43ede7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:42.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa4ffde3700 1 --2- 172.21.15.28:0/60825803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c43ed13bc0 0x55c43ede7630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.555+0000 7fa4ffde3700 1 -- 172.21.15.28:0/60825803 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e8005de0 con 0x55c43edeb410 2023-11-28T11:10:42.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.556+0000 7fa4ffde3700 1 --2- 172.21.15.28:0/60825803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c43edeb410 0x55c43ed3e560 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800a4b0 tx=0x7fa4f800a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bd845a398282028 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.556+0000 7fa4fdddf700 1 -- 172.21.15.28:0/60825803 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f800e710 con 0x55c43edeb410 2023-11-28T11:10:42.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.556+0000 7fa4fdddf700 1 -- 172.21.15.28:0/60825803 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f800e890 con 0x55c43edeb410 2023-11-28T11:10:42.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.556+0000 7fa4fdddf700 1 -- 172.21.15.28:0/60825803 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f800ec40 con 0x55c43edeb410 2023-11-28T11:10:42.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.556+0000 7fa5094df080 1 -- 172.21.15.28:0/60825803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c43edeb410 msgr2=0x55c43ed3e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.556+0000 7fa5094df080 1 --2- 172.21.15.28:0/60825803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c43edeb410 0x55c43ed3e560 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fa4f800a4b0 tx=0x7fa4f800a7c0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.556+0000 7fa5094df080 1 -- 172.21.15.28:0/60825803 shutdown_connections 2023-11-28T11:10:42.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.556+0000 7fa5094df080 1 --2- 172.21.15.28:0/60825803 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c43edeb410 0x55c43ed3e560 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.556+0000 7fa5094df080 1 --2- 172.21.15.28:0/60825803 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c43ed13bc0 0x55c43ede7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.556+0000 7fa5094df080 1 --2- 172.21.15.28:0/60825803 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c43ede8720 0x55c43ed3f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.556+0000 7fa5094df080 1 -- 172.21.15.28:0/60825803 >> 172.21.15.28:0/60825803 conn(0x55c43ed40300 msgr2=0x55c43ed5e9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:42.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.556+0000 7fa5094df080 1 -- 172.21.15.28:0/60825803 shutdown_connections 2023-11-28T11:10:42.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.556+0000 7fa5094df080 1 -- 172.21.15.28:0/60825803 wait complete. 2023-11-28T11:10:42.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.557+0000 7fa5094df080 1 Processor -- start 2023-11-28T11:10:42.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.557+0000 7fa5094df080 1 -- start start 2023-11-28T11:10:42.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa5094df080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c43ed13bc0 0x55c43ed5e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa5094df080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c43ede8720 0x55c43ee79300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa5094df080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c43edeb410 0x55c43ed58b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa5094df080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43ed5d240 con 0x55c43ede8720 2023-11-28T11:10:42.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa5094df080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43ed3e030 con 0x55c43ed13bc0 2023-11-28T11:10:42.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa5094df080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c43edddda0 con 0x55c43edeb410 2023-11-28T11:10:42.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa4fede1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c43ede8720 0x55c43ee79300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa4fede1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c43ede8720 0x55c43ee79300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49836/0 (socket says 172.21.15.28:49836) 2023-11-28T11:10:42.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa4fede1700 1 -- 172.21.15.28:0/2066686672 learned_addr learned my addr 172.21.15.28:0/2066686672 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:42.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa4ff5e2700 1 --2- 172.21.15.28:0/2066686672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c43ed13bc0 0x55c43ed5e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa4ffde3700 1 --2- 172.21.15.28:0/2066686672 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c43edeb410 0x55c43ed58b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa4fede1700 1 -- 172.21.15.28:0/2066686672 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c43edeb410 msgr2=0x55c43ed58b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa4fede1700 1 --2- 172.21.15.28:0/2066686672 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c43edeb410 0x55c43ed58b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa4fede1700 1 -- 172.21.15.28:0/2066686672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c43ed13bc0 msgr2=0x55c43ed5e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa4fede1700 1 --2- 172.21.15.28:0/2066686672 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c43ed13bc0 0x55c43ed5e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa4fede1700 1 -- 172.21.15.28:0/2066686672 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f800a0e0 con 0x55c43ede8720 2023-11-28T11:10:42.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa4fede1700 1 --2- 172.21.15.28:0/2066686672 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c43ede8720 0x55c43ee79300 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009f20 tx=0x7fa4f00086e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.558+0000 7fa4fcddd700 1 -- 172.21.15.28:0/2066686672 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f00145f0 con 0x55c43ede8720 2023-11-28T11:10:42.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.559+0000 7fa4fcddd700 1 -- 172.21.15.28:0/2066686672 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4f0010040 con 0x55c43ede8720 2023-11-28T11:10:42.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.559+0000 7fa5094df080 1 -- 172.21.15.28:0/2066686672 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c43ec66120 con 0x55c43ede8720 2023-11-28T11:10:42.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.559+0000 7fa4fcddd700 1 -- 172.21.15.28:0/2066686672 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4f0014990 con 0x55c43ede8720 2023-11-28T11:10:42.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.559+0000 7fa5094df080 1 -- 172.21.15.28:0/2066686672 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c43ec59830 con 0x55c43ede8720 2023-11-28T11:10:42.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.560+0000 7fa4fcddd700 1 -- 172.21.15.28:0/2066686672 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa4f0014af0 con 0x55c43ede8720 2023-11-28T11:10:42.567 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.560+0000 7fa4fcddd700 1 --2- 172.21.15.28:0/2066686672 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa4e0066c10 0x7fa4e00690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.560+0000 7fa4fcddd700 1 -- 172.21.15.28:0/2066686672 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa4f0084b50 con 0x55c43ede8720 2023-11-28T11:10:42.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.560+0000 7fa4ff5e2700 1 --2- 172.21.15.28:0/2066686672 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa4e0066c10 0x7fa4e00690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.561+0000 7fa5094df080 1 --2- 172.21.15.28:0/2066686672 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x55c43ed5ee00 0x55c43ee84f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.561+0000 7fa5094df080 1 -- 172.21.15.28:0/2066686672 --> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] -- osd_op(unknown.0.0:1 6.7 6:e2e07930:::10000000001.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43edde680 con 0x55c43ed5ee00 2023-11-28T11:10:42.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.561+0000 7fa4ff5e2700 1 --2- 172.21.15.28:0/2066686672 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa4e0066c10 0x7fa4e00690c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8014040 tx=0x7fa4e800f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.561+0000 7fa4ffde3700 1 --2- 172.21.15.28:0/2066686672 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x55c43ed5ee00 0x55c43ee84f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.561+0000 7fa4ffde3700 1 --2- 172.21.15.28:0/2066686672 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x55c43ed5ee00 0x55c43ee84f70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.562+0000 7fa4ffde3700 1 -- 172.21.15.28:0/2066686672 <== osd.5 v2:172.21.15.59:6800/103696 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (9) out=9b] v0'0 uv5 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fa4f8017040 con 0x55c43ed5ee00 2023-11-28T11:10:42.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.562+0000 7fa5094df080 1 -- 172.21.15.28:0/2066686672 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x55c43ed5ee00 msgr2=0x55c43ee84f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.562+0000 7fa5094df080 1 --2- 172.21.15.28:0/2066686672 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x55c43ed5ee00 0x55c43ee84f70 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.568 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.562+0000 7fa5094df080 1 -- 172.21.15.28:0/2066686672 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa4e0066c10 msgr2=0x7fa4e00690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.562+0000 7fa5094df080 1 --2- 172.21.15.28:0/2066686672 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa4e0066c10 0x7fa4e00690c0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fa4e8014040 tx=0x7fa4e800f000 comp rx=0 tx=0).stop 2023-11-28T11:10:42.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.562+0000 7fa5094df080 1 -- 172.21.15.28:0/2066686672 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c43ede8720 msgr2=0x55c43ee79300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.562+0000 7fa5094df080 1 --2- 172.21.15.28:0/2066686672 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c43ede8720 0x55c43ee79300 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0009f20 tx=0x7fa4f00086e0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.562+0000 7fa4ffde3700 1 -- 172.21.15.28:0/2066686672 reap_dead start 2023-11-28T11:10:42.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.562+0000 7fa5094df080 1 -- 172.21.15.28:0/2066686672 shutdown_connections 2023-11-28T11:10:42.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.562+0000 7fa5094df080 1 -- 172.21.15.28:0/2066686672 >> 172.21.15.28:0/2066686672 conn(0x55c43ed40300 msgr2=0x55c43ed3e560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:42.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.562+0000 7fa5094df080 1 -- 172.21.15.28:0/2066686672 shutdown_connections 2023-11-28T11:10:42.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.562+0000 7fa5094df080 1 -- 172.21.15.28:0/2066686672 wait complete. 2023-11-28T11:10:42.570 INFO:teuthology.orchestra.run.smithi028.stdout:mytag 2023-11-28T11:10:42.570 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f7.00000000 scrub_tag 2023-11-28T11:10:42.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.613+0000 7f168695c080 1 Processor -- start 2023-11-28T11:10:42.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.614+0000 7f168695c080 1 -- start start 2023-11-28T11:10:42.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.614+0000 7f168695c080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557a93fd2bc0 0x557a9409bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.614+0000 7f168695c080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557a940a7720 0x557a93ffd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.614+0000 7f168695c080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557a940aa410 0x557a93ffd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.614+0000 7f168695c080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a93f0cfe0 con 0x557a93fd2bc0 2023-11-28T11:10:42.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.614+0000 7f168695c080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9401e3a0 con 0x557a940a7720 2023-11-28T11:10:42.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.614+0000 7f168695c080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a940ac160 con 0x557a940aa410 2023-11-28T11:10:42.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.614+0000 7f167ca5f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557a93fd2bc0 0x557a9409bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.614+0000 7f167ca5f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557a93fd2bc0 0x557a9409bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49850/0 (socket says 172.21.15.28:49850) 2023-11-28T11:10:42.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.614+0000 7f167ca5f700 1 -- 172.21.15.28:0/3939509965 learned_addr learned my addr 172.21.15.28:0/3939509965 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:42.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.614+0000 7f1677fff700 1 --2- 172.21.15.28:0/3939509965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557a940a7720 0x557a93ffd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.614+0000 7f167d260700 1 --2- 172.21.15.28:0/3939509965 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557a940aa410 0x557a93ffd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f167ca5f700 1 -- 172.21.15.28:0/3939509965 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557a940aa410 msgr2=0x557a93ffd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:42.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f167ca5f700 1 --2- 172.21.15.28:0/3939509965 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557a940aa410 0x557a93ffd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f167ca5f700 1 -- 172.21.15.28:0/3939509965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557a940a7720 msgr2=0x557a93ffd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:42.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f167ca5f700 1 --2- 172.21.15.28:0/3939509965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557a940a7720 0x557a93ffd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f167ca5f700 1 -- 172.21.15.28:0/3939509965 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1668005de0 con 0x557a93fd2bc0 2023-11-28T11:10:42.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f167ca5f700 1 --2- 172.21.15.28:0/3939509965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557a93fd2bc0 0x557a9409bcf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1670009790 tx=0x7f1670009aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e29ac048b3a26f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f1676ffd700 1 -- 172.21.15.28:0/3939509965 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f167000e070 con 0x557a93fd2bc0 2023-11-28T11:10:42.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f1676ffd700 1 -- 172.21.15.28:0/3939509965 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16700064a0 con 0x557a93fd2bc0 2023-11-28T11:10:42.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f1676ffd700 1 -- 172.21.15.28:0/3939509965 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1670006830 con 0x557a93fd2bc0 2023-11-28T11:10:42.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f168695c080 1 -- 172.21.15.28:0/3939509965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557a93fd2bc0 msgr2=0x557a9409bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f168695c080 1 --2- 172.21.15.28:0/3939509965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557a93fd2bc0 0x557a9409bcf0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f1670009790 tx=0x7f1670009aa0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f168695c080 1 -- 172.21.15.28:0/3939509965 shutdown_connections 2023-11-28T11:10:42.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f168695c080 1 --2- 172.21.15.28:0/3939509965 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557a93fd2bc0 0x557a9409bcf0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f168695c080 1 --2- 172.21.15.28:0/3939509965 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557a940aa410 0x557a93ffd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f168695c080 1 --2- 172.21.15.28:0/3939509965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557a940a7720 0x557a93ffd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f168695c080 1 -- 172.21.15.28:0/3939509965 >> 172.21.15.28:0/3939509965 conn(0x557a93fff300 msgr2=0x557a9401d9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:42.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f168695c080 1 -- 172.21.15.28:0/3939509965 shutdown_connections 2023-11-28T11:10:42.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.615+0000 7f168695c080 1 -- 172.21.15.28:0/3939509965 wait complete. 2023-11-28T11:10:42.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.616+0000 7f168695c080 1 Processor -- start 2023-11-28T11:10:42.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.616+0000 7f168695c080 1 -- start start 2023-11-28T11:10:42.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f168695c080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557a93fd2bc0 0x557a9401d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f168695c080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557a940a7720 0x557a94016b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f168695c080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557a940aa410 0x557a9401b3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f168695c080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a93ffe750 con 0x557a940a7720 2023-11-28T11:10:42.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f168695c080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a93ffe9a0 con 0x557a940aa410 2023-11-28T11:10:42.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f168695c080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x557a940ab700 con 0x557a93fd2bc0 2023-11-28T11:10:42.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f1677fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557a940a7720 0x557a94016b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f1677fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557a940a7720 0x557a94016b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49858/0 (socket says 172.21.15.28:49858) 2023-11-28T11:10:42.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f1677fff700 1 -- 172.21.15.28:0/750602432 learned_addr learned my addr 172.21.15.28:0/750602432 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:42.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f167ca5f700 1 --2- 172.21.15.28:0/750602432 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557a93fd2bc0 0x557a9401d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f167d260700 1 --2- 172.21.15.28:0/750602432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557a940aa410 0x557a9401b3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f1677fff700 1 -- 172.21.15.28:0/750602432 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557a93fd2bc0 msgr2=0x557a9401d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f1677fff700 1 --2- 172.21.15.28:0/750602432 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557a93fd2bc0 0x557a9401d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f1677fff700 1 -- 172.21.15.28:0/750602432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557a940aa410 msgr2=0x557a9401b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f1677fff700 1 --2- 172.21.15.28:0/750602432 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557a940aa410 0x557a9401b3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f1677fff700 1 -- 172.21.15.28:0/750602432 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1670009410 con 0x557a940a7720 2023-11-28T11:10:42.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.617+0000 7f1677fff700 1 --2- 172.21.15.28:0/750602432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557a940a7720 0x557a94016b80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f166800c010 tx=0x7f1668002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.618+0000 7f1675ffb700 1 -- 172.21.15.28:0/750602432 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166801c810 con 0x557a940a7720 2023-11-28T11:10:42.629 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.618+0000 7f1675ffb700 1 -- 172.21.15.28:0/750602432 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1668020070 con 0x557a940a7720 2023-11-28T11:10:42.629 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.618+0000 7f1675ffb700 1 -- 172.21.15.28:0/750602432 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f166801cbb0 con 0x557a940a7720 2023-11-28T11:10:42.629 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.618+0000 7f168695c080 1 -- 172.21.15.28:0/750602432 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x557a93f25120 con 0x557a940a7720 2023-11-28T11:10:42.629 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.618+0000 7f168695c080 1 -- 172.21.15.28:0/750602432 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557a93f18830 con 0x557a940a7720 2023-11-28T11:10:42.629 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.619+0000 7f1675ffb700 1 -- 172.21.15.28:0/750602432 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f166801cd10 con 0x557a940a7720 2023-11-28T11:10:42.629 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.619+0000 7f1675ffb700 1 --2- 172.21.15.28:0/750602432 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1660066c10 0x7f16600690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.629 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.619+0000 7f1675ffb700 1 -- 172.21.15.28:0/750602432 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f1668022070 con 0x557a940a7720 2023-11-28T11:10:42.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.619+0000 7f167ca5f700 1 --2- 172.21.15.28:0/750602432 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1660066c10 0x7f16600690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.620+0000 7f168695c080 1 --2- 172.21.15.28:0/750602432 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x557a940170c0 0x557a94018df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.620+0000 7f168695c080 1 -- 172.21.15.28:0/750602432 --> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] -- osd_op(unknown.0.0:1 7.36 7:6fe38dc2:::100000001f7.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x557a940a4980 con 0x557a940170c0 2023-11-28T11:10:42.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.620+0000 7f167d260700 1 --2- 172.21.15.28:0/750602432 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x557a940170c0 0x557a94018df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.620+0000 7f167ca5f700 1 --2- 172.21.15.28:0/750602432 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1660066c10 0x7f16600690c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1670015040 tx=0x7f167000f000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.620+0000 7f167d260700 1 --2- 172.21.15.28:0/750602432 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x557a940170c0 0x557a94018df0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.620+0000 7f167d260700 1 -- 172.21.15.28:0/750602432 <== osd.0 v2:172.21.15.28:6800/103508 1 ==== osd_op_reply(1 100000001f7.00000000 [getxattr (9) out=9b] v0'0 uv17 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7f167800d690 con 0x557a940170c0 2023-11-28T11:10:42.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.620+0000 7f168695c080 1 -- 172.21.15.28:0/750602432 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x557a940170c0 msgr2=0x557a94018df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.620+0000 7f168695c080 1 --2- 172.21.15.28:0/750602432 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x557a940170c0 0x557a94018df0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.620+0000 7f168695c080 1 -- 172.21.15.28:0/750602432 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1660066c10 msgr2=0x7f16600690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.620+0000 7f168695c080 1 --2- 172.21.15.28:0/750602432 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1660066c10 0x7f16600690c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f1670015040 tx=0x7f167000f000 comp rx=0 tx=0).stop 2023-11-28T11:10:42.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.620+0000 7f168695c080 1 -- 172.21.15.28:0/750602432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557a940a7720 msgr2=0x557a94016b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.620+0000 7f168695c080 1 --2- 172.21.15.28:0/750602432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557a940a7720 0x557a94016b80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f166800c010 tx=0x7f1668002fb0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.620+0000 7f167d260700 1 -- 172.21.15.28:0/750602432 reap_dead start 2023-11-28T11:10:42.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.620+0000 7f168695c080 1 -- 172.21.15.28:0/750602432 shutdown_connections 2023-11-28T11:10:42.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.621+0000 7f168695c080 1 -- 172.21.15.28:0/750602432 >> 172.21.15.28:0/750602432 conn(0x557a93fff300 msgr2=0x557a9401d120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:42.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.621+0000 7f168695c080 1 -- 172.21.15.28:0/750602432 shutdown_connections 2023-11-28T11:10:42.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.621+0000 7f168695c080 1 -- 172.21.15.28:0/750602432 wait complete. 2023-11-28T11:10:42.632 INFO:teuthology.orchestra.run.smithi028.stdout:mytag 2023-11-28T11:10:42.632 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 100000001f8.00000000 scrub_tag 2023-11-28T11:10:42.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.675+0000 7fed5bc40080 1 Processor -- start 2023-11-28T11:10:42.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.675+0000 7fed5bc40080 1 -- start start 2023-11-28T11:10:42.679 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed5bc40080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55f0f0f875b0 0x55f0f0ffc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.679 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed5bc40080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0f0f87dc0 0x55f0f1000fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.679 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed5bc40080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f0f10047f0 0x55f0f0f714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed5bc40080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f0eeb000 con 0x55f0f10047f0 2023-11-28T11:10:42.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed5bc40080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f0e7dfe0 con 0x55f0f0f87dc0 2023-11-28T11:10:42.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed5bc40080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0f1000190 con 0x55f0f0f875b0 2023-11-28T11:10:42.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed52544700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f0f10047f0 0x55f0f0f714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed52544700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f0f10047f0 0x55f0f0f714c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49876/0 (socket says 172.21.15.28:49876) 2023-11-28T11:10:42.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed52544700 1 -- 172.21.15.28:0/169730258 learned_addr learned my addr 172.21.15.28:0/169730258 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:42.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed51d43700 1 --2- 172.21.15.28:0/169730258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55f0f0f875b0 0x55f0f0ffc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed52544700 1 -- 172.21.15.28:0/169730258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55f0f0f875b0 msgr2=0x55f0f0ffc090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:42.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed52544700 1 --2- 172.21.15.28:0/169730258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55f0f0f875b0 0x55f0f0ffc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed52544700 1 -- 172.21.15.28:0/169730258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0f0f87dc0 msgr2=0x55f0f1000fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:42.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed51542700 1 --2- 172.21.15.28:0/169730258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0f0f87dc0 0x55f0f1000fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed52544700 1 --2- 172.21.15.28:0/169730258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0f0f87dc0 0x55f0f1000fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed52544700 1 -- 172.21.15.28:0/169730258 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed48005d20 con 0x55f0f10047f0 2023-11-28T11:10:42.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed52544700 1 --2- 172.21.15.28:0/169730258 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f0f10047f0 0x55f0f0f714c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fed4c006e70 tx=0x7fed4c001930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae97b3c018f19aa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed43fff700 1 -- 172.21.15.28:0/169730258 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c011440 con 0x55f0f10047f0 2023-11-28T11:10:42.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.676+0000 7fed43fff700 1 -- 172.21.15.28:0/169730258 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4c0115a0 con 0x55f0f10047f0 2023-11-28T11:10:42.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.677+0000 7fed43fff700 1 -- 172.21.15.28:0/169730258 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4c011930 con 0x55f0f10047f0 2023-11-28T11:10:42.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.677+0000 7fed5bc40080 1 -- 172.21.15.28:0/169730258 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f0f10047f0 msgr2=0x55f0f0f714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.677+0000 7fed5bc40080 1 --2- 172.21.15.28:0/169730258 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f0f10047f0 0x55f0f0f714c0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fed4c006e70 tx=0x7fed4c001930 comp rx=0 tx=0).stop 2023-11-28T11:10:42.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.677+0000 7fed5bc40080 1 -- 172.21.15.28:0/169730258 shutdown_connections 2023-11-28T11:10:42.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.677+0000 7fed5bc40080 1 --2- 172.21.15.28:0/169730258 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f0f10047f0 0x55f0f0f714c0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.677+0000 7fed5bc40080 1 --2- 172.21.15.28:0/169730258 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55f0f0f875b0 0x55f0f0ffc090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.677+0000 7fed5bc40080 1 --2- 172.21.15.28:0/169730258 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0f0f87dc0 0x55f0f1000fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.677+0000 7fed5bc40080 1 -- 172.21.15.28:0/169730258 >> 172.21.15.28:0/169730258 conn(0x55f0f0f8a7e0 msgr2=0x55f0f0f42770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:42.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.677+0000 7fed5bc40080 1 -- 172.21.15.28:0/169730258 shutdown_connections 2023-11-28T11:10:42.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.677+0000 7fed5bc40080 1 -- 172.21.15.28:0/169730258 wait complete. 2023-11-28T11:10:42.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.678+0000 7fed5bc40080 1 Processor -- start 2023-11-28T11:10:42.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.678+0000 7fed5bc40080 1 -- start start 2023-11-28T11:10:42.686 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed5bc40080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f0f0f875b0 0x55f0f10183d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.686 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed5bc40080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0f0f87dc0 0x55f0f1019940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.686 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed5bc40080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55f0f10047f0 0x55f0f0ffcd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.686 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed5bc40080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f1002610 con 0x55f0f0f875b0 2023-11-28T11:10:42.686 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed5bc40080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f0f10028f0 con 0x55f0f0f87dc0 2023-11-28T11:10:42.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed5bc40080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f0f1002bd0 con 0x55f0f10047f0 2023-11-28T11:10:42.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed51d43700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f0f0f875b0 0x55f0f10183d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed51d43700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f0f0f875b0 0x55f0f10183d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49878/0 (socket says 172.21.15.28:49878) 2023-11-28T11:10:42.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed51d43700 1 -- 172.21.15.28:0/1034847069 learned_addr learned my addr 172.21.15.28:0/1034847069 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:42.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed52544700 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55f0f10047f0 0x55f0f0ffcd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed51542700 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0f0f87dc0 0x55f0f1019940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed51d43700 1 -- 172.21.15.28:0/1034847069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55f0f10047f0 msgr2=0x55f0f0ffcd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed51d43700 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55f0f10047f0 0x55f0f0ffcd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed51d43700 1 -- 172.21.15.28:0/1034847069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0f0f87dc0 msgr2=0x55f0f1019940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed51d43700 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0f0f87dc0 0x55f0f1019940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.687 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed51d43700 1 -- 172.21.15.28:0/1034847069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f0f0e894a0 con 0x55f0f0f875b0 2023-11-28T11:10:42.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed51542700 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f0f0f87dc0 0x55f0f1019940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:42.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed52544700 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55f0f10047f0 0x55f0f0ffcd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:42.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed51d43700 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f0f0f875b0 0x55f0f10183d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fed44001eb0 tx=0x7fed4400a4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.679+0000 7fed42ffd700 1 -- 172.21.15.28:0/1034847069 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4400abf0 con 0x55f0f0f875b0 2023-11-28T11:10:42.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.680+0000 7fed42ffd700 1 -- 172.21.15.28:0/1034847069 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed4400ad70 con 0x55f0f0f875b0 2023-11-28T11:10:42.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.680+0000 7fed5bc40080 1 -- 172.21.15.28:0/1034847069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f0f0e89830 con 0x55f0f0f875b0 2023-11-28T11:10:42.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.680+0000 7fed42ffd700 1 -- 172.21.15.28:0/1034847069 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed4400f640 con 0x55f0f0f875b0 2023-11-28T11:10:42.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.680+0000 7fed5bc40080 1 -- 172.21.15.28:0/1034847069 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f0f0e96360 con 0x55f0f0f875b0 2023-11-28T11:10:42.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.681+0000 7fed42ffd700 1 -- 172.21.15.28:0/1034847069 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fed44020070 con 0x55f0f0f875b0 2023-11-28T11:10:42.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.681+0000 7fed42ffd700 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed34066c10 0x7fed340690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.681+0000 7fed42ffd700 1 -- 172.21.15.28:0/1034847069 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fed44084930 con 0x55f0f0f875b0 2023-11-28T11:10:42.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.681+0000 7fed51542700 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed34066c10 0x7fed340690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.682+0000 7fed5bc40080 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x55f0f1002180 0x55f0f10036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.682+0000 7fed5bc40080 1 -- 172.21.15.28:0/1034847069 --> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] -- osd_op(unknown.0.0:1 7.19 7:997b7129:::100000001f8.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f0f0ffd240 con 0x55f0f1002180 2023-11-28T11:10:42.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.682+0000 7fed51542700 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed34066c10 0x7fed340690c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55f0f0f87580 tx=0x7fed4800e000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.682+0000 7fed52544700 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x55f0f1002180 0x55f0f10036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.682+0000 7fed52544700 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x55f0f1002180 0x55f0f10036b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.682+0000 7fed52544700 1 -- 172.21.15.28:0/1034847069 <== osd.6 v2:172.21.15.59:6816/103697 1 ==== osd_op_reply(1 100000001f8.00000000 [getxattr (9) out=9b] v0'0 uv12 ondisk = 0) v8 ==== 164+0+9 (crc 0 0 0) 0x7fed4c011710 con 0x55f0f1002180 2023-11-28T11:10:42.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.682+0000 7fed5bc40080 1 -- 172.21.15.28:0/1034847069 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x55f0f1002180 msgr2=0x55f0f10036b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.682+0000 7fed5bc40080 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x55f0f1002180 0x55f0f10036b0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.683+0000 7fed5bc40080 1 -- 172.21.15.28:0/1034847069 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed34066c10 msgr2=0x7fed340690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.683+0000 7fed5bc40080 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed34066c10 0x7fed340690c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55f0f0f87580 tx=0x7fed4800e000 comp rx=0 tx=0).stop 2023-11-28T11:10:42.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.683+0000 7fed5bc40080 1 -- 172.21.15.28:0/1034847069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f0f0f875b0 msgr2=0x55f0f10183d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.683+0000 7fed5bc40080 1 --2- 172.21.15.28:0/1034847069 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55f0f0f875b0 0x55f0f10183d0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fed44001eb0 tx=0x7fed4400a4e0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.683+0000 7fed52544700 1 -- 172.21.15.28:0/1034847069 reap_dead start 2023-11-28T11:10:42.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.683+0000 7fed5bc40080 1 -- 172.21.15.28:0/1034847069 shutdown_connections 2023-11-28T11:10:42.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.683+0000 7fed5bc40080 1 -- 172.21.15.28:0/1034847069 >> 172.21.15.28:0/1034847069 conn(0x55f0f0f8a7e0 msgr2=0x55f0f0f72d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:42.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.683+0000 7fed5bc40080 1 -- 172.21.15.28:0/1034847069 shutdown_connections 2023-11-28T11:10:42.690 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.683+0000 7fed5bc40080 1 -- 172.21.15.28:0/1034847069 wait complete. 2023-11-28T11:10:42.691 INFO:teuthology.orchestra.run.smithi028.stdout:mytag 2023-11-28T11:10:42.691 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000002.00000000 scrub_tag 2023-11-28T11:10:42.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.734+0000 7f4cd7b39080 1 Processor -- start 2023-11-28T11:10:42.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.734+0000 7f4cd7b39080 1 -- start start 2023-11-28T11:10:42.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.734+0000 7f4cd7b39080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56270961aa40 0x56270961f520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.734+0000 7f4cd7b39080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5627096ab780 0x5627096aca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.734+0000 7f4cd7b39080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627096abb50 0x5627096acf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.734+0000 7f4cd7b39080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x562709510fe0 con 0x5627096abb50 2023-11-28T11:10:42.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.734+0000 7f4cd7b39080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627096adc80 con 0x56270961aa40 2023-11-28T11:10:42.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.734+0000 7f4cd7b39080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x562709620110 con 0x5627096ab780 2023-11-28T11:10:42.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.735+0000 7f4cce43d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627096abb50 0x5627096acf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.735+0000 7f4ccdc3c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56270961aa40 0x56270961f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.735+0000 7f4cce43d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627096abb50 0x5627096acf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49898/0 (socket says 172.21.15.28:49898) 2023-11-28T11:10:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.735+0000 7f4cce43d700 1 -- 172.21.15.28:0/256987455 learned_addr learned my addr 172.21.15.28:0/256987455 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.735+0000 7f4ccd43b700 1 --2- 172.21.15.28:0/256987455 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5627096ab780 0x5627096aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.735+0000 7f4cce43d700 1 -- 172.21.15.28:0/256987455 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5627096ab780 msgr2=0x5627096aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.735+0000 7f4cce43d700 1 --2- 172.21.15.28:0/256987455 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5627096ab780 0x5627096aca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.735+0000 7f4cce43d700 1 -- 172.21.15.28:0/256987455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56270961aa40 msgr2=0x56270961f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.735+0000 7f4cce43d700 1 --2- 172.21.15.28:0/256987455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56270961aa40 0x56270961f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.735+0000 7f4cce43d700 1 -- 172.21.15.28:0/256987455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cc4005de0 con 0x5627096abb50 2023-11-28T11:10:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.735+0000 7f4cce43d700 1 --2- 172.21.15.28:0/256987455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627096abb50 0x5627096acf80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8006f80 tx=0x7f4cc8001820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fc9565a17645c74 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.735+0000 7f4cbffff700 1 -- 172.21.15.28:0/256987455 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8012600 con 0x5627096abb50 2023-11-28T11:10:42.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.735+0000 7f4cbffff700 1 -- 172.21.15.28:0/256987455 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc8012760 con 0x5627096abb50 2023-11-28T11:10:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.735+0000 7f4cbffff700 1 -- 172.21.15.28:0/256987455 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc8012b10 con 0x5627096abb50 2023-11-28T11:10:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.736+0000 7f4cd7b39080 1 -- 172.21.15.28:0/256987455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627096abb50 msgr2=0x5627096acf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.736+0000 7f4cd7b39080 1 --2- 172.21.15.28:0/256987455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627096abb50 0x5627096acf80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8006f80 tx=0x7f4cc8001820 comp rx=0 tx=0).stop 2023-11-28T11:10:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.736+0000 7f4cd7b39080 1 -- 172.21.15.28:0/256987455 shutdown_connections 2023-11-28T11:10:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.736+0000 7f4cd7b39080 1 --2- 172.21.15.28:0/256987455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627096abb50 0x5627096acf80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.736+0000 7f4cd7b39080 1 --2- 172.21.15.28:0/256987455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56270961aa40 0x56270961f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.736+0000 7f4cd7b39080 1 --2- 172.21.15.28:0/256987455 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5627096ab780 0x5627096aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.736+0000 7f4cd7b39080 1 -- 172.21.15.28:0/256987455 >> 172.21.15.28:0/256987455 conn(0x56270961e060 msgr2=0x56270961e430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.736+0000 7f4cd7b39080 1 -- 172.21.15.28:0/256987455 shutdown_connections 2023-11-28T11:10:42.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.736+0000 7f4cd7b39080 1 -- 172.21.15.28:0/256987455 wait complete. 2023-11-28T11:10:42.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.736+0000 7f4cd7b39080 1 Processor -- start 2023-11-28T11:10:42.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.736+0000 7f4cd7b39080 1 -- start start 2023-11-28T11:10:42.745 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:10:42.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.737+0000 7f4cd7b39080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x56270961aa40 0x56270968f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.737+0000 7f4cd7b39080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627096ab780 0x56270968ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.737+0000 7f4cd7b39080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5627096abb50 0x562709690480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.737+0000 7f4cd7b39080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x562709693f00 con 0x5627096ab780 2023-11-28T11:10:42.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.737+0000 7f4cd7b39080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5627096aa3b0 con 0x5627096abb50 2023-11-28T11:10:42.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.737+0000 7f4cd7b39080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x5627096aa690 con 0x56270961aa40 2023-11-28T11:10:42.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.737+0000 7f4ccd43b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627096ab780 0x56270968ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.737+0000 7f4ccd43b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627096ab780 0x56270968ff40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49902/0 (socket says 172.21.15.28:49902) 2023-11-28T11:10:42.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.737+0000 7f4ccd43b700 1 -- 172.21.15.28:0/861735311 learned_addr learned my addr 172.21.15.28:0/861735311 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:42.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.737+0000 7f4ccdc3c700 1 --2- 172.21.15.28:0/861735311 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x56270961aa40 0x56270968f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.737+0000 7f4cce43d700 1 --2- 172.21.15.28:0/861735311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5627096abb50 0x562709690480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.738+0000 7f4ccd43b700 1 -- 172.21.15.28:0/861735311 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x56270961aa40 msgr2=0x56270968f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.738+0000 7f4ccd43b700 1 --2- 172.21.15.28:0/861735311 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x56270961aa40 0x56270968f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.738+0000 7f4ccd43b700 1 -- 172.21.15.28:0/861735311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5627096abb50 msgr2=0x562709690480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.738+0000 7f4ccd43b700 1 --2- 172.21.15.28:0/861735311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5627096abb50 0x562709690480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.738+0000 7f4ccd43b700 1 -- 172.21.15.28:0/861735311 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56270951c4a0 con 0x5627096ab780 2023-11-28T11:10:42.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.738+0000 7f4ccd43b700 1 --2- 172.21.15.28:0/861735311 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627096ab780 0x56270968ff40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4015040 tx=0x7f4cc4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.738+0000 7f4cbeffd700 1 -- 172.21.15.28:0/861735311 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4027610 con 0x5627096ab780 2023-11-28T11:10:42.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.738+0000 7f4cbeffd700 1 -- 172.21.15.28:0/861735311 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cc40277d0 con 0x5627096ab780 2023-11-28T11:10:42.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.738+0000 7f4cd7b39080 1 -- 172.21.15.28:0/861735311 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x562709529120 con 0x5627096ab780 2023-11-28T11:10:42.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.738+0000 7f4cbeffd700 1 -- 172.21.15.28:0/861735311 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cc4027b80 con 0x5627096ab780 2023-11-28T11:10:42.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.738+0000 7f4cd7b39080 1 -- 172.21.15.28:0/861735311 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56270951c830 con 0x5627096ab780 2023-11-28T11:10:42.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.739+0000 7f4cbeffd700 1 -- 172.21.15.28:0/861735311 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4cc4027ce0 con 0x5627096ab780 2023-11-28T11:10:42.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.740+0000 7f4cbeffd700 1 --2- 172.21.15.28:0/861735311 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4cb0066c10 0x7f4cb00690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.740+0000 7f4cbeffd700 1 -- 172.21.15.28:0/861735311 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f4cc4086700 con 0x5627096ab780 2023-11-28T11:10:42.748 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.740+0000 7f4ccdc3c700 1 --2- 172.21.15.28:0/861735311 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4cb0066c10 0x7f4cb00690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.740+0000 7f4cd7b39080 1 --2- 172.21.15.28:0/861735311 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x562709604900 0x562709691f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:42.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.740+0000 7f4cd7b39080 1 -- 172.21.15.28:0/861735311 --> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] -- osd_op(unknown.0.0:1 7.12 7:4bec3886:::10000000002.00000000:head [getxattr scrub_tag in=9b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x5627095d7d40 con 0x562709604900 2023-11-28T11:10:42.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.740+0000 7f4ccdc3c700 1 --2- 172.21.15.28:0/861735311 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4cb0066c10 0x7f4cb00690c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0005400 tx=0x7f4cc000c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.740+0000 7f4cce43d700 1 --2- 172.21.15.28:0/861735311 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x562709604900 0x562709691f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:42.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.740+0000 7f4cce43d700 1 --2- 172.21.15.28:0/861735311 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x562709604900 0x562709691f80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:42.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.741+0000 7f4cce43d700 1 -- 172.21.15.28:0/861735311 <== osd.6 v2:172.21.15.59:6816/103697 1 ==== osd_op_reply(1 10000000002.00000000 [getxattr] v0'0 uv0 ondisk = -61 ((61) No data available)) v8 ==== 164+0+0 (crc 0 0 0) 0x7f4cc8012400 con 0x562709604900 2023-11-28T11:10:42.749 INFO:teuthology.orchestra.run.smithi028.stderr:error getting xattr cephfs_data/10000000002.00000000/scrub_tag: (61) No data available 2023-11-28T11:10:42.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.741+0000 7f4cd7b39080 1 -- 172.21.15.28:0/861735311 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x562709604900 msgr2=0x562709691f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.749 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.741+0000 7f4cd7b39080 1 --2- 172.21.15.28:0/861735311 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x562709604900 0x562709691f80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.741+0000 7f4cd7b39080 1 -- 172.21.15.28:0/861735311 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4cb0066c10 msgr2=0x7f4cb00690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.741+0000 7f4cd7b39080 1 --2- 172.21.15.28:0/861735311 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4cb0066c10 0x7f4cb00690c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f4cc0005400 tx=0x7f4cc000c040 comp rx=0 tx=0).stop 2023-11-28T11:10:42.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.741+0000 7f4cd7b39080 1 -- 172.21.15.28:0/861735311 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627096ab780 msgr2=0x56270968ff40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:42.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.741+0000 7f4cd7b39080 1 --2- 172.21.15.28:0/861735311 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5627096ab780 0x56270968ff40 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f4cc4015040 tx=0x7f4cc4002fb0 comp rx=0 tx=0).stop 2023-11-28T11:10:42.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.741+0000 7f4cce43d700 1 -- 172.21.15.28:0/861735311 reap_dead start 2023-11-28T11:10:42.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.741+0000 7f4cd7b39080 1 -- 172.21.15.28:0/861735311 shutdown_connections 2023-11-28T11:10:42.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.741+0000 7f4cd7b39080 1 -- 172.21.15.28:0/861735311 >> 172.21.15.28:0/861735311 conn(0x56270961e060 msgr2=0x56270961f520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:42.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.741+0000 7f4cd7b39080 1 -- 172.21.15.28:0/861735311 shutdown_connections 2023-11-28T11:10:42.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:42.741+0000 7f4cd7b39080 1 -- 172.21.15.28:0/861735311 wait complete. 2023-11-28T11:10:42.751 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:10:42.751 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:10:42.788 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:10:42.838 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:10:42.838 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:10:42.838 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:42.856 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:10:42.857 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:10:42.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:42.857 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:10:42.914 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:10:42.914 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.0': No such file or directory 2023-11-28T11:10:42.915 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:10:42.915 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:42.936 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3961 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:10:42.937 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:10:42.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:42.937 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T11:10:42.996 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T11:10:42.997 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:10:42.997 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:10:42.998 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:43.016 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:10:43.016 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:10:43.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:43.016 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T11:10:43.073 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:10:43.073 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T11:10:43.074 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:10:43.074 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:43.095 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3961 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:10:43.095 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:10:43.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:10:43.095 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T11:10:43.155 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:10:43.155 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T11:10:43.156 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:43.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.280+0000 7fd7e0a43700 1 Processor -- start 2023-11-28T11:10:43.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.280+0000 7fd7e0a43700 1 -- start start 2023-11-28T11:10:43.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.281+0000 7fd7e0a43700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd7dc06e400 0x7fd7dc15db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.281+0000 7fd7e0a43700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd7dc0ce890 0x7fd7dc0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.281+0000 7fd7e0a43700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7dc15e0b0 con 0x7fd7dc159590 2023-11-28T11:10:43.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.281+0000 7fd7e0a43700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7dc0cf250 con 0x7fd7dc0ce890 2023-11-28T11:10:43.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.281+0000 7fd7e0a43700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7dc15d120 con 0x7fd7dc06e400 2023-11-28T11:10:43.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.281+0000 7fd7dad9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd7dc0ce890 0x7fd7dc0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.282+0000 7fd7dad9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd7dc0ce890 0x7fd7dc0ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49914/0 (socket says 172.21.15.28:49914) 2023-11-28T11:10:43.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.282+0000 7fd7dad9d700 1 -- 172.21.15.28:0/241298833 learned_addr learned my addr 172.21.15.28:0/241298833 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:43.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.282+0000 7fd7d9d9b700 1 --2- 172.21.15.28:0/241298833 >> v2:172.21.15.59:3300/0 conn(0x7fd7dc06e400 0x7fd7dc15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.282+0000 7fd7dad9d700 1 -- 172.21.15.28:0/241298833 >> v2:172.21.15.59:3300/0 conn(0x7fd7dc06e400 msgr2=0x7fd7dc15db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.282+0000 7fd7dad9d700 1 --2- 172.21.15.28:0/241298833 >> v2:172.21.15.59:3300/0 conn(0x7fd7dc06e400 0x7fd7dc15db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.282+0000 7fd7dad9d700 1 -- 172.21.15.28:0/241298833 >> v1:172.21.15.59:6789/0 conn(0x7fd7dc159590 legacy=0x7fd7dc0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.282+0000 7fd7dad9d700 1 -- 172.21.15.28:0/241298833 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc15d6e0 con 0x7fd7dc0ce890 2023-11-28T11:10:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.282+0000 7fd7d9d9b700 1 --2- 172.21.15.28:0/241298833 >> v2:172.21.15.59:3300/0 conn(0x7fd7dc06e400 0x7fd7dc15db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.282+0000 7fd7dad9d700 1 --2- 172.21.15.28:0/241298833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7dc0ce890 0x7fd7dc0ced10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc15d080 tx=0x7fd7cc00be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4c829ec6f538e9d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.282+0000 7fd7d959a700 1 -- 172.21.15.28:0/241298833 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc00dd30 con 0x7fd7dc0ce890 2023-11-28T11:10:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.282+0000 7fd7d959a700 1 -- 172.21.15.28:0/241298833 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7cc00de90 con 0x7fd7dc0ce890 2023-11-28T11:10:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.282+0000 7fd7d959a700 1 -- 172.21.15.28:0/241298833 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc010690 con 0x7fd7dc0ce890 2023-11-28T11:10:43.286 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.283+0000 7fd7e0a43700 1 -- 172.21.15.28:0/241298833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7dc0ce890 msgr2=0x7fd7dc0ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.283+0000 7fd7e0a43700 1 --2- 172.21.15.28:0/241298833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7dc0ce890 0x7fd7dc0ced10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc15d080 tx=0x7fd7cc00be70 comp rx=0 tx=0).stop 2023-11-28T11:10:43.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.283+0000 7fd7e0a43700 1 -- 172.21.15.28:0/241298833 shutdown_connections 2023-11-28T11:10:43.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.283+0000 7fd7e0a43700 1 --2- 172.21.15.28:0/241298833 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7dc0ce890 0x7fd7dc0ced10 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.283+0000 7fd7e0a43700 1 --2- 172.21.15.28:0/241298833 >> v2:172.21.15.59:3300/0 conn(0x7fd7dc06e400 0x7fd7dc15db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.283+0000 7fd7e0a43700 1 -- 172.21.15.28:0/241298833 >> 172.21.15.28:0/241298833 conn(0x7fd7dc0b1650 msgr2=0x7fd7dc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:43.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.283+0000 7fd7e0a43700 1 -- 172.21.15.28:0/241298833 shutdown_connections 2023-11-28T11:10:43.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.283+0000 7fd7e0a43700 1 -- 172.21.15.28:0/241298833 wait complete. 2023-11-28T11:10:43.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.284+0000 7fd7e0a43700 1 Processor -- start 2023-11-28T11:10:43.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.284+0000 7fd7e0a43700 1 -- start start 2023-11-28T11:10:43.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7e0a43700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc06e400 0x7fd7dc0b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7e0a43700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7dc0ce890 0x7fd7dc0cc2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7e0a43700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7dc159590 0x7fd7dc1f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7e0a43700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc0b5a10 con 0x7fd7dc159590 2023-11-28T11:10:43.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7e0a43700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc1f62f0 con 0x7fd7dc06e400 2023-11-28T11:10:43.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7e0a43700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7dc1f65d0 con 0x7fd7dc0ce890 2023-11-28T11:10:43.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7dad9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7dc159590 0x7fd7dc1f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7dad9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7dc159590 0x7fd7dc1f5300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49920/0 (socket says 172.21.15.28:49920) 2023-11-28T11:10:43.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7dad9d700 1 -- 172.21.15.28:0/4076347655 learned_addr learned my addr 172.21.15.28:0/4076347655 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:43.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7d9d9b700 1 --2- 172.21.15.28:0/4076347655 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7dc0ce890 0x7fd7dc0cc2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7da59c700 1 --2- 172.21.15.28:0/4076347655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc06e400 0x7fd7dc0b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7dad9d700 1 -- 172.21.15.28:0/4076347655 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7dc0ce890 msgr2=0x7fd7dc0cc2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7dad9d700 1 --2- 172.21.15.28:0/4076347655 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7dc0ce890 0x7fd7dc0cc2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7dad9d700 1 -- 172.21.15.28:0/4076347655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc06e400 msgr2=0x7fd7dc0b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7dad9d700 1 --2- 172.21.15.28:0/4076347655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc06e400 0x7fd7dc0b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7dad9d700 1 -- 172.21.15.28:0/4076347655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc00b7f0 con 0x7fd7dc159590 2023-11-28T11:10:43.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.285+0000 7fd7dad9d700 1 --2- 172.21.15.28:0/4076347655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7dc159590 0x7fd7dc1f5300 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00b6f0 tx=0x7fd7cc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:43.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.286+0000 7fd7d2ffd700 1 -- 172.21.15.28:0/4076347655 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc003d20 con 0x7fd7dc159590 2023-11-28T11:10:43.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.286+0000 7fd7d2ffd700 1 -- 172.21.15.28:0/4076347655 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7cc003e80 con 0x7fd7dc159590 2023-11-28T11:10:43.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.286+0000 7fd7e0a43700 1 -- 172.21.15.28:0/4076347655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7dc0af240 con 0x7fd7dc159590 2023-11-28T11:10:43.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.286+0000 7fd7d2ffd700 1 -- 172.21.15.28:0/4076347655 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc010670 con 0x7fd7dc159590 2023-11-28T11:10:43.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.286+0000 7fd7e0a43700 1 -- 172.21.15.28:0/4076347655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7dc0aef60 con 0x7fd7dc159590 2023-11-28T11:10:43.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.287+0000 7fd7e0a43700 1 -- 172.21.15.28:0/4076347655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7dc15d370 con 0x7fd7dc159590 2023-11-28T11:10:43.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.287+0000 7fd7d2ffd700 1 -- 172.21.15.28:0/4076347655 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd7cc019400 con 0x7fd7dc159590 2023-11-28T11:10:43.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.288+0000 7fd7d2ffd700 1 --2- 172.21.15.28:0/4076347655 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7bc0682f0 0x7fd7bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.288+0000 7fd7d2ffd700 1 -- 172.21.15.28:0/4076347655 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fd7cc0875a0 con 0x7fd7dc159590 2023-11-28T11:10:43.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.288+0000 7fd7da59c700 1 --2- 172.21.15.28:0/4076347655 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7bc0682f0 0x7fd7bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.288+0000 7fd7da59c700 1 --2- 172.21.15.28:0/4076347655 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7bc0682f0 0x7fd7bc06a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc15ae10 tx=0x7fd7d4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:43.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.292+0000 7fd7d2ffd700 1 -- 172.21.15.28:0/4076347655 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd7cc053c20 con 0x7fd7dc159590 2023-11-28T11:10:43.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.531+0000 7fd7e0a43700 1 -- 172.21.15.28:0/4076347655 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd7dc0675a0 con 0x7fd7dc159590 2023-11-28T11:10:43.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.532+0000 7fd7d2ffd700 1 -- 172.21.15.28:0/4076347655 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7fd7cc057240 con 0x7fd7dc159590 2023-11-28T11:10:43.533 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:43.533 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2867100265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2867100265},{"type":"v1","addr":"172.21.15.59:6833","nonce":2867100265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3808190953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3808190953},{"type":"v1","addr":"172.21.15.28:6837","nonce":3808190953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3920280453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3920280453},{"type":"v1","addr":"172.21.15.28:6836","nonce":3920280453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:10.820923+0000","modified":"2023-11-28T11:10:13.833216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6835/707419743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":707419743},{"type":"v1","addr":"172.21.15.59:6835","nonce":707419743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:10:43.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.534+0000 7fd7e0a43700 1 -- 172.21.15.28:0/4076347655 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7bc0682f0 msgr2=0x7fd7bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.534+0000 7fd7e0a43700 1 --2- 172.21.15.28:0/4076347655 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7bc0682f0 0x7fd7bc06a7a0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc15ae10 tx=0x7fd7d4009040 comp rx=0 tx=0).stop 2023-11-28T11:10:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.534+0000 7fd7e0a43700 1 -- 172.21.15.28:0/4076347655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7dc159590 msgr2=0x7fd7dc1f5300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.534+0000 7fd7e0a43700 1 --2- 172.21.15.28:0/4076347655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7dc159590 0x7fd7dc1f5300 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc00b6f0 tx=0x7fd7cc004750 comp rx=0 tx=0).stop 2023-11-28T11:10:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.534+0000 7fd7e0a43700 1 -- 172.21.15.28:0/4076347655 shutdown_connections 2023-11-28T11:10:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.534+0000 7fd7e0a43700 1 --2- 172.21.15.28:0/4076347655 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd7bc0682f0 0x7fd7bc06a7a0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.534+0000 7fd7e0a43700 1 --2- 172.21.15.28:0/4076347655 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd7dc159590 0x7fd7dc1f5300 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.534+0000 7fd7e0a43700 1 --2- 172.21.15.28:0/4076347655 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd7dc06e400 0x7fd7dc0b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.534+0000 7fd7e0a43700 1 --2- 172.21.15.28:0/4076347655 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd7dc0ce890 0x7fd7dc0cc2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.534+0000 7fd7e0a43700 1 -- 172.21.15.28:0/4076347655 >> 172.21.15.28:0/4076347655 conn(0x7fd7dc0b1650 msgr2=0x7fd7dc154450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:43.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.534+0000 7fd7e0a43700 1 -- 172.21.15.28:0/4076347655 shutdown_connections 2023-11-28T11:10:43.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.534+0000 7fd7e0a43700 1 -- 172.21.15.28:0/4076347655 wait complete. 2023-11-28T11:10:43.537 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2023-11-28T11:10:43.548 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:43.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.671+0000 7fa675251700 1 Processor -- start 2023-11-28T11:10:43.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.671+0000 7fa675251700 1 -- start start 2023-11-28T11:10:43.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa675251700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa67015c990 0x7fa6700cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa675251700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa6700cc310 0x7fa6700cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa675251700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6700ce340 con 0x7fa670161090 2023-11-28T11:10:43.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa675251700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6700cebe0 con 0x7fa6700cc310 2023-11-28T11:10:43.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa675251700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6700cccf0 con 0x7fa67015c990 2023-11-28T11:10:43.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa66f59e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa6700cc310 0x7fa6700cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa66f59e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa6700cc310 0x7fa6700cc7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49928/0 (socket says 172.21.15.28:49928) 2023-11-28T11:10:43.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa66f59e700 1 -- 172.21.15.28:0/2156323368 learned_addr learned my addr 172.21.15.28:0/2156323368 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:43.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa66ed9d700 1 --2- 172.21.15.28:0/2156323368 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa67015c990 0x7fa6700cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa66f59e700 1 -- 172.21.15.28:0/2156323368 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa67015c990 msgr2=0x7fa6700cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:43.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa66f59e700 1 --2- 172.21.15.28:0/2156323368 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa67015c990 0x7fa6700cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa66f59e700 1 -- 172.21.15.28:0/2156323368 >> v1:172.21.15.59:6789/0 conn(0x7fa670161090 legacy=0x7fa6700ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa66f59e700 1 -- 172.21.15.28:0/2156323368 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6700cb3f0 con 0x7fa6700cc310 2023-11-28T11:10:43.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa66f59e700 1 --2- 172.21.15.28:0/2156323368 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6700cc310 0x7fa6700cc7b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa668009e50 tx=0x7fa66800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a89ac553efbfe80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:43.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa66dd9b700 1 -- 172.21.15.28:0/2156323368 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800e070 con 0x7fa6700cc310 2023-11-28T11:10:43.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa66dd9b700 1 -- 172.21.15.28:0/2156323368 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa66800b9c0 con 0x7fa6700cc310 2023-11-28T11:10:43.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.672+0000 7fa66dd9b700 1 -- 172.21.15.28:0/2156323368 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa66800bd70 con 0x7fa6700cc310 2023-11-28T11:10:43.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.673+0000 7fa675251700 1 -- 172.21.15.28:0/2156323368 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6700cc310 msgr2=0x7fa6700cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.673+0000 7fa675251700 1 --2- 172.21.15.28:0/2156323368 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6700cc310 0x7fa6700cc7b0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fa668009e50 tx=0x7fa66800b330 comp rx=0 tx=0).stop 2023-11-28T11:10:43.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.673+0000 7fa675251700 1 -- 172.21.15.28:0/2156323368 shutdown_connections 2023-11-28T11:10:43.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.673+0000 7fa675251700 1 --2- 172.21.15.28:0/2156323368 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa6700cc310 0x7fa6700cc7b0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.673+0000 7fa675251700 1 --2- 172.21.15.28:0/2156323368 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa67015c990 0x7fa6700cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.673+0000 7fa675251700 1 -- 172.21.15.28:0/2156323368 >> 172.21.15.28:0/2156323368 conn(0x7fa6700b1650 msgr2=0x7fa6700b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:43.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.673+0000 7fa675251700 1 -- 172.21.15.28:0/2156323368 shutdown_connections 2023-11-28T11:10:43.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.673+0000 7fa675251700 1 -- 172.21.15.28:0/2156323368 wait complete. 2023-11-28T11:10:43.679 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.675+0000 7fa675251700 1 Processor -- start 2023-11-28T11:10:43.679 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.675+0000 7fa675251700 1 -- start start 2023-11-28T11:10:43.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa675251700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6700cc310 0x7fa6701f2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa675251700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa67015c990 0x7fa6701f6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa675251700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa670161090 0x7fa67015f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa675251700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6701f3330 con 0x7fa67015c990 2023-11-28T11:10:43.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa675251700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6701f2a90 con 0x7fa670161090 2023-11-28T11:10:43.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa675251700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6701f2d70 con 0x7fa6700cc310 2023-11-28T11:10:43.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa66e59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa67015c990 0x7fa6701f6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa66e59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa67015c990 0x7fa6701f6390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49932/0 (socket says 172.21.15.28:49932) 2023-11-28T11:10:43.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa66e59c700 1 -- 172.21.15.28:0/2077697386 learned_addr learned my addr 172.21.15.28:0/2077697386 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:43.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa66ed9d700 1 --2- 172.21.15.28:0/2077697386 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6700cc310 0x7fa6701f2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa66f59e700 1 --2- 172.21.15.28:0/2077697386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa670161090 0x7fa67015f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa66e59c700 1 -- 172.21.15.28:0/2077697386 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6700cc310 msgr2=0x7fa6701f2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa66e59c700 1 --2- 172.21.15.28:0/2077697386 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6700cc310 0x7fa6701f2390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa66e59c700 1 -- 172.21.15.28:0/2077697386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa670161090 msgr2=0x7fa67015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa66e59c700 1 --2- 172.21.15.28:0/2077697386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa670161090 0x7fa67015f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa66e59c700 1 -- 172.21.15.28:0/2077697386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa668009b30 con 0x7fa67015c990 2023-11-28T11:10:43.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.676+0000 7fa66f59e700 1 --2- 172.21.15.28:0/2077697386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa670161090 0x7fa67015f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:43.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.677+0000 7fa66e59c700 1 --2- 172.21.15.28:0/2077697386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa67015c990 0x7fa6701f6390 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa66400c4b0 tx=0x7fa66400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:43.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.677+0000 7fa6577fe700 1 -- 172.21.15.28:0/2077697386 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6640052d0 con 0x7fa67015c990 2023-11-28T11:10:43.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.677+0000 7fa6577fe700 1 -- 172.21.15.28:0/2077697386 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa664005430 con 0x7fa67015c990 2023-11-28T11:10:43.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.677+0000 7fa6577fe700 1 -- 172.21.15.28:0/2077697386 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6640038c0 con 0x7fa67015c990 2023-11-28T11:10:43.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.677+0000 7fa675251700 1 -- 172.21.15.28:0/2077697386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6700ce870 con 0x7fa67015c990 2023-11-28T11:10:43.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.677+0000 7fa675251700 1 -- 172.21.15.28:0/2077697386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6700ce590 con 0x7fa67015c990 2023-11-28T11:10:43.681 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.678+0000 7fa675251700 1 -- 172.21.15.28:0/2077697386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6700cb080 con 0x7fa67015c990 2023-11-28T11:10:43.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.678+0000 7fa6577fe700 1 -- 172.21.15.28:0/2077697386 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa664019030 con 0x7fa67015c990 2023-11-28T11:10:43.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.679+0000 7fa6577fe700 1 --2- 172.21.15.28:0/2077697386 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa65c0682f0 0x7fa65c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:43.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.679+0000 7fa6577fe700 1 -- 172.21.15.28:0/2077697386 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa664085260 con 0x7fa67015c990 2023-11-28T11:10:43.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.679+0000 7fa66ed9d700 1 --2- 172.21.15.28:0/2077697386 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa65c0682f0 0x7fa65c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:43.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.679+0000 7fa66ed9d700 1 --2- 172.21.15.28:0/2077697386 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa65c0682f0 0x7fa65c06a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa660009950 tx=0x7fa660008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:43.684 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.683+0000 7fa6577fe700 1 -- 172.21.15.28:0/2077697386 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa664051850 con 0x7fa67015c990 2023-11-28T11:10:43.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.877+0000 7fa675251700 1 -- 172.21.15.28:0/2077697386 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa6700675a0 con 0x7fa67015c990 2023-11-28T11:10:43.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.878+0000 7fa6577fe700 1 -- 172.21.15.28:0/2077697386 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18174 (secure 0 0 0) 0x7fa664054e70 con 0x7fa67015c990 2023-11-28T11:10:43.880 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:43.880 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":42,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:10:35.252810+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:08.429278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:09.797038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.18","mappings":[{"from":0,"to":4}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:43.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.880+0000 7fa675251700 1 -- 172.21.15.28:0/2077697386 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa65c0682f0 msgr2=0x7fa65c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.880+0000 7fa675251700 1 --2- 172.21.15.28:0/2077697386 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa65c0682f0 0x7fa65c06a7a0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fa660009950 tx=0x7fa660008040 comp rx=0 tx=0).stop 2023-11-28T11:10:43.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.880+0000 7fa675251700 1 -- 172.21.15.28:0/2077697386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa67015c990 msgr2=0x7fa6701f6390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:43.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.880+0000 7fa675251700 1 --2- 172.21.15.28:0/2077697386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa67015c990 0x7fa6701f6390 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa66400c4b0 tx=0x7fa66400c870 comp rx=0 tx=0).stop 2023-11-28T11:10:43.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.880+0000 7fa675251700 1 -- 172.21.15.28:0/2077697386 shutdown_connections 2023-11-28T11:10:43.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.880+0000 7fa675251700 1 --2- 172.21.15.28:0/2077697386 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa65c0682f0 0x7fa65c06a7a0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.880+0000 7fa675251700 1 --2- 172.21.15.28:0/2077697386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa670161090 0x7fa67015f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.880+0000 7fa675251700 1 --2- 172.21.15.28:0/2077697386 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa6700cc310 0x7fa6701f2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.880+0000 7fa675251700 1 --2- 172.21.15.28:0/2077697386 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa67015c990 0x7fa6701f6390 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:43.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.880+0000 7fa675251700 1 -- 172.21.15.28:0/2077697386 >> 172.21.15.28:0/2077697386 conn(0x7fa6700b1650 msgr2=0x7fa67006cbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:43.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.880+0000 7fa675251700 1 -- 172.21.15.28:0/2077697386 shutdown_connections 2023-11-28T11:10:43.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:43.880+0000 7fa675251700 1 -- 172.21.15.28:0/2077697386 wait complete. 2023-11-28T11:10:43.894 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:10:08.429278+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '39', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:10:09.797038+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '40', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:10:43.895 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:10:44.022 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.021+0000 7f34dff1f700 1 Processor -- start 2023-11-28T11:10:44.022 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.021+0000 7f34dff1f700 1 -- start start 2023-11-28T11:10:44.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34dff1f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34d80cb630 0x7f34d80b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34dff1f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f34d80b2bc0 0x7f34d80ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34dff1f700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34d80cd2a0 con 0x7f34d80b62d0 2023-11-28T11:10:44.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34dff1f700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d80cba80 con 0x7f34d80b2bc0 2023-11-28T11:10:44.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34dff1f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d80b3040 con 0x7f34d80cb630 2023-11-28T11:10:44.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34de4bc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f34d80b2bc0 0x7f34d80ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.025 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34ddcbb700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f34d80b62d0 0x7f34d806f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:37996/0 (socket says 172.21.15.28:37996) 2023-11-28T11:10:44.026 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34de4bc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f34d80b2bc0 0x7f34d80ce7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49942/0 (socket says 172.21.15.28:49942) 2023-11-28T11:10:44.026 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34ddcbb700 1 -- 172.21.15.28:0/2461807422 learned_addr learned my addr 172.21.15.28:0/2461807422 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:44.026 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34dd4ba700 1 --2- 172.21.15.28:0/2461807422 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34d80cb630 0x7f34d80b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.026 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34de4bc700 1 -- 172.21.15.28:0/2461807422 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34d80cb630 msgr2=0x7f34d80b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:44.026 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34de4bc700 1 --2- 172.21.15.28:0/2461807422 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34d80cb630 0x7f34d80b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.026 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34de4bc700 1 -- 172.21.15.28:0/2461807422 >> v1:172.21.15.28:6789/0 conn(0x7f34d80b62d0 legacy=0x7f34d806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.026 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34de4bc700 1 -- 172.21.15.28:0/2461807422 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d80d0400 con 0x7f34d80b2bc0 2023-11-28T11:10:44.026 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34dccb9700 1 -- 172.21.15.28:0/2461807422 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 526279606 0 0) 0x7f34d80cd2a0 con 0x7f34d80b62d0 2023-11-28T11:10:44.026 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34dd4ba700 1 --2- 172.21.15.28:0/2461807422 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34d80cb630 0x7f34d80b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:44.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.022+0000 7f34de4bc700 1 --2- 172.21.15.28:0/2461807422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34d80b2bc0 0x7f34d80ce7d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009d90 tx=0x7f34d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b79f007b75196 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.023+0000 7f34dccb9700 1 -- 172.21.15.28:0/2461807422 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d000e070 con 0x7f34d80b2bc0 2023-11-28T11:10:44.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.023+0000 7f34dccb9700 1 -- 172.21.15.28:0/2461807422 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d000bbd0 con 0x7f34d80b2bc0 2023-11-28T11:10:44.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.023+0000 7f34dccb9700 1 -- 172.21.15.28:0/2461807422 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d0012430 con 0x7f34d80b2bc0 2023-11-28T11:10:44.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.023+0000 7f34dff1f700 1 -- 172.21.15.28:0/2461807422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34d80b2bc0 msgr2=0x7f34d80ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.023+0000 7f34dff1f700 1 --2- 172.21.15.28:0/2461807422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34d80b2bc0 0x7f34d80ce7d0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f34d0009d90 tx=0x7f34d000b330 comp rx=0 tx=0).stop 2023-11-28T11:10:44.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.023+0000 7f34dff1f700 1 -- 172.21.15.28:0/2461807422 shutdown_connections 2023-11-28T11:10:44.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.023+0000 7f34dff1f700 1 --2- 172.21.15.28:0/2461807422 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34d80b2bc0 0x7f34d80ce7d0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.023+0000 7f34dff1f700 1 --2- 172.21.15.28:0/2461807422 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34d80cb630 0x7f34d80b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.023+0000 7f34dff1f700 1 -- 172.21.15.28:0/2461807422 >> 172.21.15.28:0/2461807422 conn(0x7f34d80cd680 msgr2=0x7f34d80cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:44.027 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.023+0000 7f34dff1f700 1 -- 172.21.15.28:0/2461807422 shutdown_connections 2023-11-28T11:10:44.028 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.023+0000 7f34dff1f700 1 -- 172.21.15.28:0/2461807422 wait complete. 2023-11-28T11:10:44.030 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.025+0000 7f34dff1f700 1 Processor -- start 2023-11-28T11:10:44.030 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.025+0000 7f34dff1f700 1 -- start start 2023-11-28T11:10:44.031 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34dff1f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d80b2bc0 0x7f34d81572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.031 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34dff1f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34d80b62d0 0x7f34d8156510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.031 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34dff1f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34d80cb630 0x7f34d81f02f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.031 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34dff1f700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d80c9c70 con 0x7f34d80b62d0 2023-11-28T11:10:44.031 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34dff1f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d80c9380 con 0x7f34d80b2bc0 2023-11-28T11:10:44.031 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34dff1f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d80c9620 con 0x7f34d80cb630 2023-11-28T11:10:44.031 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34dd4ba700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34d80b62d0 0x7f34d8156510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.031 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34dd4ba700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34d80b62d0 0x7f34d8156510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49946/0 (socket says 172.21.15.28:49946) 2023-11-28T11:10:44.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34dd4ba700 1 -- 172.21.15.28:0/2084724711 learned_addr learned my addr 172.21.15.28:0/2084724711 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:44.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34ddcbb700 1 --2- 172.21.15.28:0/2084724711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d80b2bc0 0x7f34d81572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34de4bc700 1 --2- 172.21.15.28:0/2084724711 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34d80cb630 0x7f34d81f02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34dd4ba700 1 -- 172.21.15.28:0/2084724711 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34d80cb630 msgr2=0x7f34d81f02f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34dd4ba700 1 --2- 172.21.15.28:0/2084724711 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34d80cb630 0x7f34d81f02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34dd4ba700 1 -- 172.21.15.28:0/2084724711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d80b2bc0 msgr2=0x7f34d81572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34dd4ba700 1 --2- 172.21.15.28:0/2084724711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d80b2bc0 0x7f34d81572e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34dd4ba700 1 -- 172.21.15.28:0/2084724711 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d0009a70 con 0x7f34d80b62d0 2023-11-28T11:10:44.032 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.026+0000 7f34dd4ba700 1 --2- 172.21.15.28:0/2084724711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34d80b62d0 0x7f34d8156510 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f34d400e810 tx=0x7f34d400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.027+0000 7f34ce7fc700 1 -- 172.21.15.28:0/2084724711 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d400c020 con 0x7f34d80b62d0 2023-11-28T11:10:44.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.027+0000 7f34ce7fc700 1 -- 172.21.15.28:0/2084724711 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34d4004250 con 0x7f34d80b62d0 2023-11-28T11:10:44.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.027+0000 7f34dff1f700 1 -- 172.21.15.28:0/2084724711 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d80cffb0 con 0x7f34d80b62d0 2023-11-28T11:10:44.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.027+0000 7f34ce7fc700 1 -- 172.21.15.28:0/2084724711 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34d40036a0 con 0x7f34d80b62d0 2023-11-28T11:10:44.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.027+0000 7f34dff1f700 1 -- 172.21.15.28:0/2084724711 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d80d06e0 con 0x7f34d80b62d0 2023-11-28T11:10:44.033 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.027+0000 7f34dff1f700 1 -- 172.21.15.28:0/2084724711 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d80b66a0 con 0x7f34d80b62d0 2023-11-28T11:10:44.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.028+0000 7f34ce7fc700 1 -- 172.21.15.28:0/2084724711 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f34d4003800 con 0x7f34d80b62d0 2023-11-28T11:10:44.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.028+0000 7f34ce7fc700 1 --2- 172.21.15.28:0/2084724711 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f34bc0682f0 0x7f34bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.028+0000 7f34ce7fc700 1 -- 172.21.15.28:0/2084724711 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f34d4085b90 con 0x7f34d80b62d0 2023-11-28T11:10:44.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.028+0000 7f34ddcbb700 1 --2- 172.21.15.28:0/2084724711 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f34bc0682f0 0x7f34bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.029+0000 7f34ddcbb700 1 --2- 172.21.15.28:0/2084724711 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f34bc0682f0 0x7f34bc06a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f34c8000f60 tx=0x7f34c8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.033+0000 7f34ce7fc700 1 -- 172.21.15.28:0/2084724711 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f34d4052270 con 0x7f34d80b62d0 2023-11-28T11:10:44.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.227+0000 7f34dff1f700 1 -- 172.21.15.28:0/2084724711 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f34d815dc50 con 0x7f34d80b62d0 2023-11-28T11:10:44.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.227+0000 7f34ce7fc700 1 -- 172.21.15.28:0/2084724711 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v42) v1 ==== 92+0+10 (secure 0 0 0) 0x7f34d400f040 con 0x7f34d80b62d0 2023-11-28T11:10:44.229 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:10:44.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.229+0000 7f34dff1f700 1 -- 172.21.15.28:0/2084724711 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f34bc0682f0 msgr2=0x7f34bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.229+0000 7f34dff1f700 1 --2- 172.21.15.28:0/2084724711 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f34bc0682f0 0x7f34bc06a7a0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f34c8000f60 tx=0x7f34c8009040 comp rx=0 tx=0).stop 2023-11-28T11:10:44.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.229+0000 7f34dff1f700 1 -- 172.21.15.28:0/2084724711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34d80b62d0 msgr2=0x7f34d8156510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.229+0000 7f34dff1f700 1 --2- 172.21.15.28:0/2084724711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34d80b62d0 0x7f34d8156510 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f34d400e810 tx=0x7f34d400ebd0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.229+0000 7f34dff1f700 1 -- 172.21.15.28:0/2084724711 shutdown_connections 2023-11-28T11:10:44.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.230+0000 7f34dff1f700 1 --2- 172.21.15.28:0/2084724711 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f34bc0682f0 0x7f34bc06a7a0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.230+0000 7f34dff1f700 1 --2- 172.21.15.28:0/2084724711 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f34d80cb630 0x7f34d81f02f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.230+0000 7f34dff1f700 1 --2- 172.21.15.28:0/2084724711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f34d80b2bc0 0x7f34d81572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.230+0000 7f34dff1f700 1 --2- 172.21.15.28:0/2084724711 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f34d80b62d0 0x7f34d8156510 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.230+0000 7f34dff1f700 1 -- 172.21.15.28:0/2084724711 >> 172.21.15.28:0/2084724711 conn(0x7f34d80cd680 msgr2=0x7f34d80b13f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:44.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.230+0000 7f34dff1f700 1 -- 172.21.15.28:0/2084724711 shutdown_connections 2023-11-28T11:10:44.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.230+0000 7f34dff1f700 1 -- 172.21.15.28:0/2084724711 wait complete. 2023-11-28T11:10:44.243 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:10:44.370 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.368+0000 7f8d329fa700 1 Processor -- start 2023-11-28T11:10:44.370 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.369+0000 7f8d329fa700 1 -- start start 2023-11-28T11:10:44.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.369+0000 7f8d329fa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8d2c0b1630 0x7f8d2c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.369+0000 7f8d329fa700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8d2c06afc0 0x7f8d2c06b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.369+0000 7f8d329fa700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d2c0ce550 con 0x7f8d2c0b1a00 2023-11-28T11:10:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.369+0000 7f8d329fa700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c0ce830 con 0x7f8d2c06afc0 2023-11-28T11:10:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.369+0000 7f8d329fa700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d2c0ceb10 con 0x7f8d2c0b1630 2023-11-28T11:10:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.369+0000 7f8d30f97700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8d2c06afc0 0x7f8d2c06b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.370+0000 7f8d30f97700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8d2c06afc0 0x7f8d2c06b420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49954/0 (socket says 172.21.15.28:49954) 2023-11-28T11:10:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.370+0000 7f8d30f97700 1 -- 172.21.15.28:0/2314596128 learned_addr learned my addr 172.21.15.28:0/2314596128 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.370+0000 7f8d2bfff700 1 --2- 172.21.15.28:0/2314596128 >> v2:172.21.15.59:3300/0 conn(0x7f8d2c0b1630 0x7f8d2c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.370+0000 7f8d30f97700 1 -- 172.21.15.28:0/2314596128 >> v2:172.21.15.59:3300/0 conn(0x7f8d2c0b1630 msgr2=0x7f8d2c0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.370+0000 7f8d30f97700 1 --2- 172.21.15.28:0/2314596128 >> v2:172.21.15.59:3300/0 conn(0x7f8d2c0b1630 0x7f8d2c0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.370+0000 7f8d30f97700 1 -- 172.21.15.28:0/2314596128 >> v1:172.21.15.59:6789/0 conn(0x7f8d2c0b1a00 legacy=0x7f8d2c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.370+0000 7f8d30f97700 1 -- 172.21.15.28:0/2314596128 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c0cedf0 con 0x7f8d2c06afc0 2023-11-28T11:10:44.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.370+0000 7f8d30f97700 1 --2- 172.21.15.28:0/2314596128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d2c06afc0 0x7f8d2c06b420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8d24009e50 tx=0x7f8d2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42042ccab821dc59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.370+0000 7f8d2affd700 1 -- 172.21.15.28:0/2314596128 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400e070 con 0x7f8d2c06afc0 2023-11-28T11:10:44.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.370+0000 7f8d2affd700 1 -- 172.21.15.28:0/2314596128 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d2400b980 con 0x7f8d2c06afc0 2023-11-28T11:10:44.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.370+0000 7f8d2affd700 1 -- 172.21.15.28:0/2314596128 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d2400bd30 con 0x7f8d2c06afc0 2023-11-28T11:10:44.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.371+0000 7f8d329fa700 1 -- 172.21.15.28:0/2314596128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d2c06afc0 msgr2=0x7f8d2c06b420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.371+0000 7f8d329fa700 1 --2- 172.21.15.28:0/2314596128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d2c06afc0 0x7f8d2c06b420 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f8d24009e50 tx=0x7f8d2400b330 comp rx=0 tx=0).stop 2023-11-28T11:10:44.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.371+0000 7f8d329fa700 1 -- 172.21.15.28:0/2314596128 shutdown_connections 2023-11-28T11:10:44.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.371+0000 7f8d329fa700 1 --2- 172.21.15.28:0/2314596128 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d2c06afc0 0x7f8d2c06b420 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.371+0000 7f8d329fa700 1 --2- 172.21.15.28:0/2314596128 >> v2:172.21.15.59:3300/0 conn(0x7f8d2c0b1630 0x7f8d2c0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.371+0000 7f8d329fa700 1 -- 172.21.15.28:0/2314596128 >> 172.21.15.28:0/2314596128 conn(0x7f8d2c0cc630 msgr2=0x7f8d2c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:44.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.371+0000 7f8d329fa700 1 -- 172.21.15.28:0/2314596128 shutdown_connections 2023-11-28T11:10:44.375 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.371+0000 7f8d329fa700 1 -- 172.21.15.28:0/2314596128 wait complete. 2023-11-28T11:10:44.377 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.372+0000 7f8d329fa700 1 Processor -- start 2023-11-28T11:10:44.377 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.372+0000 7f8d329fa700 1 -- start start 2023-11-28T11:10:44.377 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d329fa700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d2c06afc0 0x7f8d2c0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.377 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d329fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d2c0b1630 0x7f8d2c157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.378 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d329fa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d2c0b1a00 0x7f8d2c1580e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.378 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d329fa700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0b3ef0 con 0x7f8d2c06afc0 2023-11-28T11:10:44.378 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d329fa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0b41d0 con 0x7f8d2c0b1630 2023-11-28T11:10:44.378 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d329fa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d2c0b44b0 con 0x7f8d2c0b1a00 2023-11-28T11:10:44.378 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d2c06afc0 0x7f8d2c0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.378 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d2bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d2c06afc0 0x7f8d2c0b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49956/0 (socket says 172.21.15.28:49956) 2023-11-28T11:10:44.378 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d2bfff700 1 -- 172.21.15.28:0/1631183738 learned_addr learned my addr 172.21.15.28:0/1631183738 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:44.378 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d2b7fe700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d2c0b1630 0x7f8d2c157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.378 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d30f97700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d2c0b1a00 0x7f8d2c1580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.378 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d2bfff700 1 -- 172.21.15.28:0/1631183738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d2c0b1a00 msgr2=0x7f8d2c1580e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.378 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d2bfff700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d2c0b1a00 0x7f8d2c1580e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d2bfff700 1 -- 172.21.15.28:0/1631183738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d2c0b1630 msgr2=0x7f8d2c157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d2bfff700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d2c0b1630 0x7f8d2c157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d2bfff700 1 -- 172.21.15.28:0/1631183738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d24009b30 con 0x7f8d2c06afc0 2023-11-28T11:10:44.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d30f97700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d2c0b1a00 0x7f8d2c1580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:44.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d2b7fe700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d2c0b1630 0x7f8d2c157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:44.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d2bfff700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d2c06afc0 0x7f8d2c0b4f90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00e810 tx=0x7f8d1c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.373+0000 7f8d28ff9700 1 -- 172.21.15.28:0/1631183738 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c013610 con 0x7f8d2c06afc0 2023-11-28T11:10:44.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.374+0000 7f8d28ff9700 1 -- 172.21.15.28:0/1631183738 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d1c013770 con 0x7f8d2c06afc0 2023-11-28T11:10:44.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.374+0000 7f8d329fa700 1 -- 172.21.15.28:0/1631183738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d2c0b1ea0 con 0x7f8d2c06afc0 2023-11-28T11:10:44.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.374+0000 7f8d28ff9700 1 -- 172.21.15.28:0/1631183738 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d1c013b20 con 0x7f8d2c06afc0 2023-11-28T11:10:44.379 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.374+0000 7f8d329fa700 1 -- 172.21.15.28:0/1631183738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d2c0cf0d0 con 0x7f8d2c06afc0 2023-11-28T11:10:44.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.374+0000 7f8d329fa700 1 -- 172.21.15.28:0/1631183738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d2c0cb5b0 con 0x7f8d2c06afc0 2023-11-28T11:10:44.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.375+0000 7f8d28ff9700 1 -- 172.21.15.28:0/1631183738 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8d1c023070 con 0x7f8d2c06afc0 2023-11-28T11:10:44.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.375+0000 7f8d28ff9700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8d0c0682f0 0x7f8d0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.375+0000 7f8d28ff9700 1 -- 172.21.15.28:0/1631183738 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f8d1c085b20 con 0x7f8d2c06afc0 2023-11-28T11:10:44.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.375+0000 7f8d2b7fe700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8d0c0682f0 0x7f8d0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.380 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.376+0000 7f8d2b7fe700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8d0c0682f0 0x7f8d0c06a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c160220 tx=0x7f8d14009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.380+0000 7f8d28ff9700 1 -- 172.21.15.28:0/1631183738 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8d1c052180 con 0x7f8d2c06afc0 2023-11-28T11:10:44.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.572+0000 7f8d329fa700 1 -- 172.21.15.28:0/1631183738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d2c0b4de0 con 0x7f8d2c06afc0 2023-11-28T11:10:44.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.573+0000 7f8d28ff9700 1 -- 172.21.15.28:0/1631183738 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d1c0557a0 con 0x7f8d2c06afc0 2023-11-28T11:10:44.575 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:10:44.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.575+0000 7f8d329fa700 1 -- 172.21.15.28:0/1631183738 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8d0c0682f0 msgr2=0x7f8d0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.575+0000 7f8d329fa700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8d0c0682f0 0x7f8d0c06a7a0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c160220 tx=0x7f8d14009040 comp rx=0 tx=0).stop 2023-11-28T11:10:44.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.575+0000 7f8d329fa700 1 -- 172.21.15.28:0/1631183738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d2c06afc0 msgr2=0x7f8d2c0b4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.575+0000 7f8d329fa700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d2c06afc0 0x7f8d2c0b4f90 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f8d1c00e810 tx=0x7f8d1c00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.575+0000 7f8d329fa700 1 -- 172.21.15.28:0/1631183738 shutdown_connections 2023-11-28T11:10:44.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.575+0000 7f8d329fa700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8d0c0682f0 0x7f8d0c06a7a0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.575+0000 7f8d329fa700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8d2c06afc0 0x7f8d2c0b4f90 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.575+0000 7f8d329fa700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8d2c0b1a00 0x7f8d2c1580e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.575+0000 7f8d329fa700 1 --2- 172.21.15.28:0/1631183738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d2c0b1630 0x7f8d2c157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.575+0000 7f8d329fa700 1 -- 172.21.15.28:0/1631183738 >> 172.21.15.28:0/1631183738 conn(0x7f8d2c0cc630 msgr2=0x7f8d2c0cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:44.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.575+0000 7f8d329fa700 1 -- 172.21.15.28:0/1631183738 shutdown_connections 2023-11-28T11:10:44.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.575+0000 7f8d329fa700 1 -- 172.21.15.28:0/1631183738 wait complete. 2023-11-28T11:10:44.588 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:10:44.716 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.714+0000 7f5ba1fdf700 1 Processor -- start 2023-11-28T11:10:44.716 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.714+0000 7f5ba1fdf700 1 -- start start 2023-11-28T11:10:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5ba1fdf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5b9c0b62d0 0x7f5b9c0cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5ba1fdf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b9c06f8b0 0x7f5b9c0b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5ba1fdf700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b9c0b3020 con 0x7f5b9c0b2bc0 2023-11-28T11:10:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5ba1fdf700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b9c0cd2a0 con 0x7f5b9c0b62d0 2023-11-28T11:10:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5ba1fdf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c0d0400 con 0x7f5b9c06f8b0 2023-11-28T11:10:44.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5b9bfff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5b9c0b2bc0 0x7f5b9c0ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38026/0 (socket says 172.21.15.28:38026) 2023-11-28T11:10:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5b9bfff700 1 -- 172.21.15.28:0/2729851768 learned_addr learned my addr 172.21.15.28:0/2729851768 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5b9b7fe700 1 --2- 172.21.15.28:0/2729851768 >> v2:172.21.15.59:3300/0 conn(0x7f5b9c0b62d0 0x7f5b9c0cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5b9affd700 1 --2- 172.21.15.28:0/2729851768 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b9c06f8b0 0x7f5b9c0b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5b9a7fc700 1 -- 172.21.15.28:0/2729851768 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2766267548 0 0) 0x7f5b9c0b3020 con 0x7f5b9c0b2bc0 2023-11-28T11:10:44.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5b9a7fc700 1 -- 172.21.15.28:0/2729851768 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5b840035b0 con 0x7f5b9c0b2bc0 2023-11-28T11:10:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5b9a7fc700 1 -- 172.21.15.28:0/2729851768 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2598285358 0 0) 0x7f5b840035b0 con 0x7f5b9c0b2bc0 2023-11-28T11:10:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5b9a7fc700 1 -- 172.21.15.28:0/2729851768 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b9c06f8b0 msgr2=0x7f5b9c0b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5b9a7fc700 1 --2- 172.21.15.28:0/2729851768 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b9c06f8b0 0x7f5b9c0b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5b9a7fc700 1 -- 172.21.15.28:0/2729851768 >> v2:172.21.15.59:3300/0 conn(0x7f5b9c0b62d0 msgr2=0x7f5b9c0cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.715+0000 7f5b9a7fc700 1 --2- 172.21.15.28:0/2729851768 >> v2:172.21.15.59:3300/0 conn(0x7f5b9c0b62d0 0x7f5b9c0cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9a7fc700 1 -- 172.21.15.28:0/2729851768 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c0d06e0 con 0x7f5b9c0b2bc0 2023-11-28T11:10:44.723 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9a7fc700 1 -- 172.21.15.28:0/2729851768 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f5b940019f0 con 0x7f5b9c0b2bc0 2023-11-28T11:10:44.723 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9a7fc700 1 -- 172.21.15.28:0/2729851768 >> v1:172.21.15.28:6789/0 conn(0x7f5b9c0b2bc0 legacy=0x7f5b9c0ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.723 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9a7fc700 1 --2- 172.21.15.28:0/2729851768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b84003a00 0x7f5b84003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.723 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9a7fc700 1 --2- 172.21.15.28:0/2729851768 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b9c06f8b0 0x7f5b84004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.723 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9a7fc700 1 --2- 172.21.15.28:0/2729851768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b84004d90 0x7f5b84005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.723 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9a7fc700 1 -- 172.21.15.28:0/2729851768 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c0cd2a0 con 0x7f5b84003a00 2023-11-28T11:10:44.724 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9a7fc700 1 -- 172.21.15.28:0/2729851768 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c0d0400 con 0x7f5b84004d90 2023-11-28T11:10:44.724 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9a7fc700 1 -- 172.21.15.28:0/2729851768 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b8400ba10 con 0x7f5b9c06f8b0 2023-11-28T11:10:44.724 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9b7fe700 1 --2- 172.21.15.28:0/2729851768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b84003a00 0x7f5b84003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.724 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9affd700 1 --2- 172.21.15.28:0/2729851768 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b9c06f8b0 0x7f5b84004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.724 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9bfff700 1 --2- 172.21.15.28:0/2729851768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b84004d90 0x7f5b84005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.724 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9b7fe700 1 -- 172.21.15.28:0/2729851768 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b9c06f8b0 msgr2=0x7f5b84004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:44.724 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9b7fe700 1 --2- 172.21.15.28:0/2729851768 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b9c06f8b0 0x7f5b84004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.725 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9b7fe700 1 -- 172.21.15.28:0/2729851768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b84004d90 msgr2=0x7f5b84005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:44.725 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9b7fe700 1 --2- 172.21.15.28:0/2729851768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b84004d90 0x7f5b84005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.725 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9b7fe700 1 -- 172.21.15.28:0/2729851768 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b840035b0 con 0x7f5b84003a00 2023-11-28T11:10:44.725 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9b7fe700 1 --2- 172.21.15.28:0/2729851768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b84003a00 0x7f5b84003e10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009b60 tx=0x7f5b84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6265f26b1c86811b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.725 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9a7fc700 1 -- 172.21.15.28:0/2729851768 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00e660 con 0x7f5b84003a00 2023-11-28T11:10:44.725 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9a7fc700 1 -- 172.21.15.28:0/2729851768 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8c00e7c0 con 0x7f5b84003a00 2023-11-28T11:10:44.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.716+0000 7f5b9a7fc700 1 -- 172.21.15.28:0/2729851768 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8c00eb30 con 0x7f5b84003a00 2023-11-28T11:10:44.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.717+0000 7f5ba1fdf700 1 -- 172.21.15.28:0/2729851768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b84003a00 msgr2=0x7f5b84003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.717+0000 7f5ba1fdf700 1 --2- 172.21.15.28:0/2729851768 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b84003a00 0x7f5b84003e10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c009b60 tx=0x7f5b84005880 comp rx=0 tx=0).stop 2023-11-28T11:10:44.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.717+0000 7f5b9bfff700 1 -- 172.21.15.28:0/2729851768 reap_dead start 2023-11-28T11:10:44.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.717+0000 7f5ba1fdf700 1 -- 172.21.15.28:0/2729851768 shutdown_connections 2023-11-28T11:10:44.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.717+0000 7f5ba1fdf700 1 -- 172.21.15.28:0/2729851768 >> 172.21.15.28:0/2729851768 conn(0x7f5b9c0cd680 msgr2=0x7f5b9c0cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:44.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.717+0000 7f5ba1fdf700 1 -- 172.21.15.28:0/2729851768 shutdown_connections 2023-11-28T11:10:44.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.717+0000 7f5ba1fdf700 1 -- 172.21.15.28:0/2729851768 wait complete. 2023-11-28T11:10:44.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.719+0000 7f5ba1fdf700 1 Processor -- start 2023-11-28T11:10:44.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.719+0000 7f5ba1fdf700 1 -- start start 2023-11-28T11:10:44.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5ba1fdf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b84003a00 0x7f5b9c0c4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5ba1fdf700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b9c154160 0x7f5b9c1efd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5ba1fdf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b9c0c5f00 0x7f5b9c0c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5ba1fdf700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c0c4c50 con 0x7f5b9c154160 2023-11-28T11:10:44.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5ba1fdf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c15c6c0 con 0x7f5b9c0c5f00 2023-11-28T11:10:44.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5ba1fdf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b9c0c4e60 con 0x7f5b84003a00 2023-11-28T11:10:44.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5b9affd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b9c154160 0x7f5b9c1efd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5b9affd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b9c154160 0x7f5b9c1efd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49978/0 (socket says 172.21.15.28:49978) 2023-11-28T11:10:44.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5b9affd700 1 -- 172.21.15.28:0/2942570716 learned_addr learned my addr 172.21.15.28:0/2942570716 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:44.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5b9b7fe700 1 --2- 172.21.15.28:0/2942570716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b84003a00 0x7f5b9c0c4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5b9bfff700 1 --2- 172.21.15.28:0/2942570716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b9c0c5f00 0x7f5b9c0c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5b9affd700 1 -- 172.21.15.28:0/2942570716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b84003a00 msgr2=0x7f5b9c0c4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5b9affd700 1 --2- 172.21.15.28:0/2942570716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b84003a00 0x7f5b9c0c4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5b9affd700 1 -- 172.21.15.28:0/2942570716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b9c0c5f00 msgr2=0x7f5b9c0c6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5b9affd700 1 --2- 172.21.15.28:0/2942570716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b9c0c5f00 0x7f5b9c0c6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5b9affd700 1 -- 172.21.15.28:0/2942570716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b9c0b66a0 con 0x7f5b9c154160 2023-11-28T11:10:44.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5b9affd700 1 --2- 172.21.15.28:0/2942570716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b9c154160 0x7f5b9c1efd80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5b90011660 tx=0x7f5b90011970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5b83fff700 1 -- 172.21.15.28:0/2942570716 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90015870 con 0x7f5b9c154160 2023-11-28T11:10:44.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5b83fff700 1 -- 172.21.15.28:0/2942570716 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b900159d0 con 0x7f5b9c154160 2023-11-28T11:10:44.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.720+0000 7f5ba1fdf700 1 -- 172.21.15.28:0/2942570716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b9c0cffb0 con 0x7f5b9c154160 2023-11-28T11:10:44.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.721+0000 7f5b83fff700 1 -- 172.21.15.28:0/2942570716 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b90015d40 con 0x7f5b9c154160 2023-11-28T11:10:44.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.721+0000 7f5ba1fdf700 1 -- 172.21.15.28:0/2942570716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b9c06fd00 con 0x7f5b9c154160 2023-11-28T11:10:44.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.721+0000 7f5ba1fdf700 1 -- 172.21.15.28:0/2942570716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b9c0c5d50 con 0x7f5b9c154160 2023-11-28T11:10:44.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.722+0000 7f5b83fff700 1 -- 172.21.15.28:0/2942570716 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5b9001b030 con 0x7f5b9c154160 2023-11-28T11:10:44.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.722+0000 7f5b83fff700 1 --2- 172.21.15.28:0/2942570716 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5b880682f0 0x7f5b8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:44.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.722+0000 7f5b83fff700 1 -- 172.21.15.28:0/2942570716 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f5b90085ff0 con 0x7f5b9c154160 2023-11-28T11:10:44.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.722+0000 7f5b9b7fe700 1 --2- 172.21.15.28:0/2942570716 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5b880682f0 0x7f5b8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:44.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.723+0000 7f5b9b7fe700 1 --2- 172.21.15.28:0/2942570716 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5b880682f0 0x7f5b8806a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0090b0 tx=0x7f5b8c01f040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:44.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.726+0000 7f5b83fff700 1 -- 172.21.15.28:0/2942570716 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5b90052600 con 0x7f5b9c154160 2023-11-28T11:10:44.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.917+0000 7f5ba1fdf700 1 -- 172.21.15.28:0/2942570716 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5b9c0739f0 con 0x7f5b9c154160 2023-11-28T11:10:44.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.917+0000 7f5b83fff700 1 -- 172.21.15.28:0/2942570716 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5b90023020 con 0x7f5b9c154160 2023-11-28T11:10:44.919 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:10:44.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.919+0000 7f5ba1fdf700 1 -- 172.21.15.28:0/2942570716 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5b880682f0 msgr2=0x7f5b8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.919+0000 7f5ba1fdf700 1 --2- 172.21.15.28:0/2942570716 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5b880682f0 0x7f5b8806a7a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f5b8c0090b0 tx=0x7f5b8c01f040 comp rx=0 tx=0).stop 2023-11-28T11:10:44.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.919+0000 7f5ba1fdf700 1 -- 172.21.15.28:0/2942570716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b9c154160 msgr2=0x7f5b9c1efd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:44.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.919+0000 7f5ba1fdf700 1 --2- 172.21.15.28:0/2942570716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b9c154160 0x7f5b9c1efd80 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f5b90011660 tx=0x7f5b90011970 comp rx=0 tx=0).stop 2023-11-28T11:10:44.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.919+0000 7f5ba1fdf700 1 -- 172.21.15.28:0/2942570716 shutdown_connections 2023-11-28T11:10:44.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.919+0000 7f5ba1fdf700 1 --2- 172.21.15.28:0/2942570716 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5b880682f0 0x7f5b8806a7a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.919+0000 7f5ba1fdf700 1 --2- 172.21.15.28:0/2942570716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b9c0c5f00 0x7f5b9c0c6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.919+0000 7f5ba1fdf700 1 --2- 172.21.15.28:0/2942570716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5b84003a00 0x7f5b9c0c4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.919+0000 7f5ba1fdf700 1 --2- 172.21.15.28:0/2942570716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5b9c154160 0x7f5b9c1efd80 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:44.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.919+0000 7f5ba1fdf700 1 -- 172.21.15.28:0/2942570716 >> 172.21.15.28:0/2942570716 conn(0x7f5b9c0cd680 msgr2=0x7f5b9c0ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:44.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.919+0000 7f5ba1fdf700 1 -- 172.21.15.28:0/2942570716 shutdown_connections 2023-11-28T11:10:44.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:44.919+0000 7f5ba1fdf700 1 -- 172.21.15.28:0/2942570716 wait complete. 2023-11-28T11:10:44.933 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:10:45.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.059+0000 7ff615ad1700 1 Processor -- start 2023-11-28T11:10:45.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.059+0000 7ff615ad1700 1 -- start start 2023-11-28T11:10:45.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.060+0000 7ff615ad1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff610159500 0x7ff6100cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.060+0000 7ff615ad1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100b1d40 0x7ff6100ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.060+0000 7ff615ad1700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61015e040 con 0x7ff61015dbf0 2023-11-28T11:10:45.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.060+0000 7ff615ad1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6100cede0 con 0x7ff610159500 2023-11-28T11:10:45.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.060+0000 7ff615ad1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100ceff0 con 0x7ff6100b1d40 2023-11-28T11:10:45.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.060+0000 7ff60effd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff61015dbf0 0x7ff6100b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38042/0 (socket says 172.21.15.28:38042) 2023-11-28T11:10:45.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.060+0000 7ff60effd700 1 -- 172.21.15.28:0/2521832797 learned_addr learned my addr 172.21.15.28:0/2521832797 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:45.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.060+0000 7ff60ffff700 1 --2- 172.21.15.28:0/2521832797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100b1d40 0x7ff6100ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.060+0000 7ff60f7fe700 1 --2- 172.21.15.28:0/2521832797 >> v2:172.21.15.59:3300/0 conn(0x7ff610159500 0x7ff6100cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.061+0000 7ff60e7fc700 1 -- 172.21.15.28:0/2521832797 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 588382989 0 0) 0x7ff61015e040 con 0x7ff61015dbf0 2023-11-28T11:10:45.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.061+0000 7ff60e7fc700 1 -- 172.21.15.28:0/2521832797 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5fc0035b0 con 0x7ff61015dbf0 2023-11-28T11:10:45.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.061+0000 7ff60f7fe700 1 -- 172.21.15.28:0/2521832797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100b1d40 msgr2=0x7ff6100ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:45.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.061+0000 7ff60f7fe700 1 --2- 172.21.15.28:0/2521832797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100b1d40 0x7ff6100ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.061+0000 7ff60f7fe700 1 -- 172.21.15.28:0/2521832797 >> v1:172.21.15.28:6789/0 conn(0x7ff61015dbf0 legacy=0x7ff6100b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.061+0000 7ff60f7fe700 1 -- 172.21.15.28:0/2521832797 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6100cf2d0 con 0x7ff610159500 2023-11-28T11:10:45.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.061+0000 7ff60ffff700 1 --2- 172.21.15.28:0/2521832797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100b1d40 0x7ff6100ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:45.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.061+0000 7ff60f7fe700 1 --2- 172.21.15.28:0/2521832797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff610159500 0x7ff6100cd140 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff600009840 tx=0x7ff600009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cfdb0cf8acf84f07 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:45.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.061+0000 7ff60e7fc700 1 -- 172.21.15.28:0/2521832797 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600011830 con 0x7ff610159500 2023-11-28T11:10:45.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.061+0000 7ff60e7fc700 1 -- 172.21.15.28:0/2521832797 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff600011990 con 0x7ff610159500 2023-11-28T11:10:45.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.061+0000 7ff60e7fc700 1 -- 172.21.15.28:0/2521832797 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff600011d00 con 0x7ff610159500 2023-11-28T11:10:45.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.062+0000 7ff615ad1700 1 -- 172.21.15.28:0/2521832797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff610159500 msgr2=0x7ff6100cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.062+0000 7ff615ad1700 1 --2- 172.21.15.28:0/2521832797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff610159500 0x7ff6100cd140 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff600009840 tx=0x7ff600009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:45.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.062+0000 7ff615ad1700 1 -- 172.21.15.28:0/2521832797 shutdown_connections 2023-11-28T11:10:45.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.062+0000 7ff615ad1700 1 --2- 172.21.15.28:0/2521832797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100b1d40 0x7ff6100ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.062+0000 7ff615ad1700 1 --2- 172.21.15.28:0/2521832797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff610159500 0x7ff6100cd140 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.062+0000 7ff615ad1700 1 -- 172.21.15.28:0/2521832797 >> 172.21.15.28:0/2521832797 conn(0x7ff6100cc630 msgr2=0x7ff6100cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:45.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.062+0000 7ff615ad1700 1 -- 172.21.15.28:0/2521832797 shutdown_connections 2023-11-28T11:10:45.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.062+0000 7ff615ad1700 1 -- 172.21.15.28:0/2521832797 wait complete. 2023-11-28T11:10:45.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.063+0000 7ff615ad1700 1 Processor -- start 2023-11-28T11:10:45.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.063+0000 7ff615ad1700 1 -- start start 2023-11-28T11:10:45.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff615ad1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100b1d40 0x7ff6100b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff615ad1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610159500 0x7ff610152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff615ad1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff61015dbf0 0x7ff61015b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff615ad1700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100b2f20 con 0x7ff610159500 2023-11-28T11:10:45.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff615ad1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff610153530 con 0x7ff61015dbf0 2023-11-28T11:10:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff615ad1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6100b31d0 con 0x7ff6100b1d40 2023-11-28T11:10:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff60effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610159500 0x7ff610152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff60effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610159500 0x7ff610152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:49992/0 (socket says 172.21.15.28:49992) 2023-11-28T11:10:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff60effd700 1 -- 172.21.15.28:0/1643054751 learned_addr learned my addr 172.21.15.28:0/1643054751 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff60ffff700 1 --2- 172.21.15.28:0/1643054751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff61015dbf0 0x7ff61015b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff60f7fe700 1 --2- 172.21.15.28:0/1643054751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100b1d40 0x7ff6100b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff60effd700 1 -- 172.21.15.28:0/1643054751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100b1d40 msgr2=0x7ff6100b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff60effd700 1 --2- 172.21.15.28:0/1643054751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100b1d40 0x7ff6100b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff60effd700 1 -- 172.21.15.28:0/1643054751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff61015dbf0 msgr2=0x7ff61015b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff60effd700 1 --2- 172.21.15.28:0/1643054751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff61015dbf0 0x7ff61015b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff60effd700 1 -- 172.21.15.28:0/1643054751 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff608009a70 con 0x7ff610159500 2023-11-28T11:10:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff60f7fe700 1 --2- 172.21.15.28:0/1643054751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100b1d40 0x7ff6100b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.064+0000 7ff60effd700 1 --2- 172.21.15.28:0/1643054751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610159500 0x7ff610152ff0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8001eb0 tx=0x7ff5f800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.065+0000 7ff607fff700 1 -- 172.21.15.28:0/1643054751 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8005d70 con 0x7ff610159500 2023-11-28T11:10:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.065+0000 7ff607fff700 1 -- 172.21.15.28:0/1643054751 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5f80070d0 con 0x7ff610159500 2023-11-28T11:10:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.065+0000 7ff615ad1700 1 -- 172.21.15.28:0/1643054751 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6000094a0 con 0x7ff610159500 2023-11-28T11:10:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.065+0000 7ff607fff700 1 -- 172.21.15.28:0/1643054751 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5f8007480 con 0x7ff610159500 2023-11-28T11:10:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.065+0000 7ff615ad1700 1 -- 172.21.15.28:0/1643054751 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61015d3f0 con 0x7ff610159500 2023-11-28T11:10:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.066+0000 7ff615ad1700 1 -- 172.21.15.28:0/1643054751 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61015d110 con 0x7ff610159500 2023-11-28T11:10:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.066+0000 7ff607fff700 1 -- 172.21.15.28:0/1643054751 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff5f80075e0 con 0x7ff610159500 2023-11-28T11:10:45.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.067+0000 7ff607fff700 1 --2- 172.21.15.28:0/1643054751 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff5f00682f0 0x7ff5f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.067+0000 7ff607fff700 1 -- 172.21.15.28:0/1643054751 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff5f8087aa0 con 0x7ff610159500 2023-11-28T11:10:45.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.067+0000 7ff60f7fe700 1 --2- 172.21.15.28:0/1643054751 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff5f00682f0 0x7ff5f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.067+0000 7ff60f7fe700 1 --2- 172.21.15.28:0/1643054751 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff5f00682f0 0x7ff5f006a7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff600009fd0 tx=0x7ff600008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:45.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.071+0000 7ff607fff700 1 -- 172.21.15.28:0/1643054751 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff5f8054240 con 0x7ff610159500 2023-11-28T11:10:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.264+0000 7ff615ad1700 1 -- 172.21.15.28:0/1643054751 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff61015d6d0 con 0x7ff610159500 2023-11-28T11:10:45.266 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.264+0000 7ff607fff700 1 -- 172.21.15.28:0/1643054751 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff5f80150a0 con 0x7ff610159500 2023-11-28T11:10:45.266 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:10:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.266+0000 7ff615ad1700 1 -- 172.21.15.28:0/1643054751 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff5f00682f0 msgr2=0x7ff5f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.266+0000 7ff615ad1700 1 --2- 172.21.15.28:0/1643054751 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff5f00682f0 0x7ff5f006a7a0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff600009fd0 tx=0x7ff600008030 comp rx=0 tx=0).stop 2023-11-28T11:10:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.266+0000 7ff615ad1700 1 -- 172.21.15.28:0/1643054751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610159500 msgr2=0x7ff610152ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.266+0000 7ff615ad1700 1 --2- 172.21.15.28:0/1643054751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610159500 0x7ff610152ff0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7ff5f8001eb0 tx=0x7ff5f800c860 comp rx=0 tx=0).stop 2023-11-28T11:10:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.266+0000 7ff615ad1700 1 -- 172.21.15.28:0/1643054751 shutdown_connections 2023-11-28T11:10:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.266+0000 7ff615ad1700 1 --2- 172.21.15.28:0/1643054751 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff5f00682f0 0x7ff5f006a7a0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.266+0000 7ff615ad1700 1 --2- 172.21.15.28:0/1643054751 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff61015dbf0 0x7ff61015b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.266+0000 7ff615ad1700 1 --2- 172.21.15.28:0/1643054751 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff6100b1d40 0x7ff6100b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.266+0000 7ff615ad1700 1 --2- 172.21.15.28:0/1643054751 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff610159500 0x7ff610152ff0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.266+0000 7ff615ad1700 1 -- 172.21.15.28:0/1643054751 >> 172.21.15.28:0/1643054751 conn(0x7ff6100cc630 msgr2=0x7ff61006bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.267+0000 7ff615ad1700 1 -- 172.21.15.28:0/1643054751 shutdown_connections 2023-11-28T11:10:45.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.267+0000 7ff615ad1700 1 -- 172.21.15.28:0/1643054751 wait complete. 2023-11-28T11:10:45.280 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:45.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.403+0000 7f3443da1700 1 Processor -- start 2023-11-28T11:10:45.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.403+0000 7f3443da1700 1 -- start start 2023-11-28T11:10:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.404+0000 7f3443da1700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f343c161090 0x7f343c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.404+0000 7f3443da1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f343c0caeb0 0x7f343c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.404+0000 7f3443da1700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f343c0ce340 con 0x7f343c15c990 2023-11-28T11:10:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.404+0000 7f3443da1700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343c0cebe0 con 0x7f343c161090 2023-11-28T11:10:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.404+0000 7f3443da1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f343c0cf780 con 0x7f343c0caeb0 2023-11-28T11:10:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.404+0000 7f344133c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f343c161090 0x7f343c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.404+0000 7f344133c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f343c161090 0x7f343c0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50000/0 (socket says 172.21.15.28:50000) 2023-11-28T11:10:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.404+0000 7f3441b3d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f343c15c990 0x7f343c0cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38056/0 (socket says 172.21.15.28:38056) 2023-11-28T11:10:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.404+0000 7f344133c700 1 -- 172.21.15.28:0/3880822438 learned_addr learned my addr 172.21.15.28:0/3880822438 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.405+0000 7f344233e700 1 --2- 172.21.15.28:0/3880822438 >> v2:172.21.15.59:3300/0 conn(0x7f343c0caeb0 0x7f343c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.405+0000 7f344133c700 1 -- 172.21.15.28:0/3880822438 >> v2:172.21.15.59:3300/0 conn(0x7f343c0caeb0 msgr2=0x7f343c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.405+0000 7f344133c700 1 --2- 172.21.15.28:0/3880822438 >> v2:172.21.15.59:3300/0 conn(0x7f343c0caeb0 0x7f343c0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.405+0000 7f344133c700 1 -- 172.21.15.28:0/3880822438 >> v1:172.21.15.28:6789/0 conn(0x7f343c15c990 legacy=0x7f343c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.405+0000 7f344133c700 1 -- 172.21.15.28:0/3880822438 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f343c0cb3f0 con 0x7f343c161090 2023-11-28T11:10:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.405+0000 7f344133c700 1 --2- 172.21.15.28:0/3880822438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343c161090 0x7f343c0ca970 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f343c071a50 tx=0x7f3438009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b783e7898bb6ed0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.405+0000 7f3440b3b700 1 -- 172.21.15.28:0/3880822438 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3438011830 con 0x7f343c161090 2023-11-28T11:10:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.405+0000 7f3440b3b700 1 -- 172.21.15.28:0/3880822438 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3438011990 con 0x7f343c161090 2023-11-28T11:10:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.405+0000 7f3440b3b700 1 -- 172.21.15.28:0/3880822438 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3438011d20 con 0x7f343c161090 2023-11-28T11:10:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.406+0000 7f3443da1700 1 -- 172.21.15.28:0/3880822438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343c161090 msgr2=0x7f343c0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.406+0000 7f3443da1700 1 --2- 172.21.15.28:0/3880822438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343c161090 0x7f343c0ca970 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f343c071a50 tx=0x7f3438009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.409 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.406+0000 7f3443da1700 1 -- 172.21.15.28:0/3880822438 shutdown_connections 2023-11-28T11:10:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.406+0000 7f3443da1700 1 --2- 172.21.15.28:0/3880822438 >> v2:172.21.15.59:3300/0 conn(0x7f343c0caeb0 0x7f343c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.406+0000 7f3443da1700 1 --2- 172.21.15.28:0/3880822438 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343c161090 0x7f343c0ca970 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.406+0000 7f3443da1700 1 -- 172.21.15.28:0/3880822438 >> 172.21.15.28:0/3880822438 conn(0x7f343c0b1650 msgr2=0x7f343c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:45.410 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.406+0000 7f3443da1700 1 -- 172.21.15.28:0/3880822438 shutdown_connections 2023-11-28T11:10:45.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.406+0000 7f3443da1700 1 -- 172.21.15.28:0/3880822438 wait complete. 2023-11-28T11:10:45.413 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.407+0000 7f3443da1700 1 Processor -- start 2023-11-28T11:10:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.407+0000 7f3443da1700 1 -- start start 2023-11-28T11:10:45.414 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f3443da1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c0caeb0 0x7f343c1f1a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f3443da1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343c15c990 0x7f343c0b5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f3443da1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f343c161090 0x7f343c14a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f3443da1700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0c9750 con 0x7f343c15c990 2023-11-28T11:10:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f3443da1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0b45f0 con 0x7f343c0caeb0 2023-11-28T11:10:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f3443da1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f343c0b48d0 con 0x7f343c161090 2023-11-28T11:10:45.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f344133c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343c15c990 0x7f343c0b5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f344133c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343c15c990 0x7f343c0b5690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50006/0 (socket says 172.21.15.28:50006) 2023-11-28T11:10:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f344133c700 1 -- 172.21.15.28:0/3391822644 learned_addr learned my addr 172.21.15.28:0/3391822644 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f344233e700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f343c161090 0x7f343c14a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f3441b3d700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c0caeb0 0x7f343c1f1a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f344133c700 1 -- 172.21.15.28:0/3391822644 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f343c161090 msgr2=0x7f343c14a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f344133c700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f343c161090 0x7f343c14a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f344133c700 1 -- 172.21.15.28:0/3391822644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c0caeb0 msgr2=0x7f343c1f1a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f344133c700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c0caeb0 0x7f343c1f1a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f344133c700 1 -- 172.21.15.28:0/3391822644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3438009550 con 0x7f343c15c990 2023-11-28T11:10:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f344233e700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f343c161090 0x7f343c14a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f3441b3d700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c0caeb0 0x7f343c1f1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f344133c700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343c15c990 0x7f343c0b5690 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3438000c00 tx=0x7f3438004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:45.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f34327fc700 1 -- 172.21.15.28:0/3391822644 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34380045b0 con 0x7f343c15c990 2023-11-28T11:10:45.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.408+0000 7f34327fc700 1 -- 172.21.15.28:0/3391822644 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f343801f070 con 0x7f343c15c990 2023-11-28T11:10:45.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.409+0000 7f3443da1700 1 -- 172.21.15.28:0/3391822644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f343c0ce870 con 0x7f343c15c990 2023-11-28T11:10:45.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.409+0000 7f34327fc700 1 -- 172.21.15.28:0/3391822644 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3438003900 con 0x7f343c15c990 2023-11-28T11:10:45.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.409+0000 7f3443da1700 1 -- 172.21.15.28:0/3391822644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f343c0ce590 con 0x7f343c15c990 2023-11-28T11:10:45.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.409+0000 7f3443da1700 1 -- 172.21.15.28:0/3391822644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f343c0cc850 con 0x7f343c15c990 2023-11-28T11:10:45.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.410+0000 7f34327fc700 1 -- 172.21.15.28:0/3391822644 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f343802a020 con 0x7f343c15c990 2023-11-28T11:10:45.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.410+0000 7f34327fc700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f34200682f0 0x7f342006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.410+0000 7f34327fc700 1 -- 172.21.15.28:0/3391822644 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f3438087430 con 0x7f343c15c990 2023-11-28T11:10:45.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.410+0000 7f3441b3d700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f34200682f0 0x7f342006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.411+0000 7f3441b3d700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f34200682f0 0x7f342006a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f342c000f60 tx=0x7f342c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:45.420 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.415+0000 7f34327fc700 1 -- 172.21.15.28:0/3391822644 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3438053b10 con 0x7f343c15c990 2023-11-28T11:10:45.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.650+0000 7f3443da1700 1 -- 172.21.15.28:0/3391822644 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f343c0675a0 con 0x7f343c15c990 2023-11-28T11:10:45.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.650+0000 7f34327fc700 1 -- 172.21.15.28:0/3391822644 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f3438057130 con 0x7f343c15c990 2023-11-28T11:10:45.652 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:45.652 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2867100265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2867100265},{"type":"v1","addr":"172.21.15.59:6833","nonce":2867100265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3808190953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3808190953},{"type":"v1","addr":"172.21.15.28:6837","nonce":3808190953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3920280453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3920280453},{"type":"v1","addr":"172.21.15.28:6836","nonce":3920280453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:10.820923+0000","modified":"2023-11-28T11:10:13.833216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6835/707419743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":707419743},{"type":"v1","addr":"172.21.15.59:6835","nonce":707419743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:10:45.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.652+0000 7f3443da1700 1 -- 172.21.15.28:0/3391822644 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f34200682f0 msgr2=0x7f342006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.652+0000 7f3443da1700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f34200682f0 0x7f342006a7a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f342c000f60 tx=0x7f342c009040 comp rx=0 tx=0).stop 2023-11-28T11:10:45.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.652+0000 7f3443da1700 1 -- 172.21.15.28:0/3391822644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343c15c990 msgr2=0x7f343c0b5690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.652+0000 7f3443da1700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343c15c990 0x7f343c0b5690 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f3438000c00 tx=0x7f3438004750 comp rx=0 tx=0).stop 2023-11-28T11:10:45.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.652+0000 7f3443da1700 1 -- 172.21.15.28:0/3391822644 shutdown_connections 2023-11-28T11:10:45.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.652+0000 7f3443da1700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f34200682f0 0x7f342006a7a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.652+0000 7f3443da1700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f343c161090 0x7f343c14a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.652+0000 7f3443da1700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c0caeb0 0x7f343c1f1a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.652+0000 7f3443da1700 1 --2- 172.21.15.28:0/3391822644 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f343c15c990 0x7f343c0b5690 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.652+0000 7f3443da1700 1 -- 172.21.15.28:0/3391822644 >> 172.21.15.28:0/3391822644 conn(0x7f343c0b1650 msgr2=0x7f343c15bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:45.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.653+0000 7f3443da1700 1 -- 172.21.15.28:0/3391822644 shutdown_connections 2023-11-28T11:10:45.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.653+0000 7f3443da1700 1 -- 172.21.15.28:0/3391822644 wait complete. 2023-11-28T11:10:45.656 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2023-11-28T11:10:45.667 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:45.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.795+0000 7fc35ba51700 1 Processor -- start 2023-11-28T11:10:45.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.795+0000 7fc35ba51700 1 -- start start 2023-11-28T11:10:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.796+0000 7fc35ba51700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc35415a6c0 0x7fc3540cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.796+0000 7fc35ba51700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc3540b2660 0x7fc3540b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.796+0000 7fc35ba51700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3540cfb90 con 0x7fc3540d0020 2023-11-28T11:10:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.796+0000 7fc35ba51700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3540b3020 con 0x7fc35415a6c0 2023-11-28T11:10:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.796+0000 7fc35ba51700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3540cb930 con 0x7fc3540b2660 2023-11-28T11:10:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.796+0000 7fc3597ed700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc35415a6c0 0x7fc3540cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.796+0000 7fc358fec700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc3540d0020 0x7fc3540ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38072/0 (socket says 172.21.15.28:38072) 2023-11-28T11:10:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.796+0000 7fc358fec700 1 -- 172.21.15.28:0/53300174 learned_addr learned my addr 172.21.15.28:0/53300174 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.796+0000 7fc359fee700 1 --2- 172.21.15.28:0/53300174 >> v2:172.21.15.59:3300/0 conn(0x7fc3540b2660 0x7fc3540b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.797+0000 7fc3597ed700 1 -- 172.21.15.28:0/53300174 >> v2:172.21.15.59:3300/0 conn(0x7fc3540b2660 msgr2=0x7fc3540b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.797+0000 7fc3597ed700 1 --2- 172.21.15.28:0/53300174 >> v2:172.21.15.59:3300/0 conn(0x7fc3540b2660 0x7fc3540b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.797+0000 7fc3597ed700 1 -- 172.21.15.28:0/53300174 >> v1:172.21.15.28:6789/0 conn(0x7fc3540d0020 legacy=0x7fc3540ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.797+0000 7fc3597ed700 1 -- 172.21.15.28:0/53300174 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3540cf260 con 0x7fc35415a6c0 2023-11-28T11:10:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.797+0000 7fc359fee700 1 --2- 172.21.15.28:0/53300174 >> v2:172.21.15.59:3300/0 conn(0x7fc3540b2660 0x7fc3540b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.797+0000 7fc3597ed700 1 --2- 172.21.15.28:0/53300174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc35415a6c0 0x7fc3540cb230 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc3540cb1c0 tx=0x7fc348009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9370b8e2cd33fd66 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.797+0000 7fc347fff700 1 -- 172.21.15.28:0/53300174 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348011830 con 0x7fc35415a6c0 2023-11-28T11:10:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.797+0000 7fc347fff700 1 -- 172.21.15.28:0/53300174 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc348011990 con 0x7fc35415a6c0 2023-11-28T11:10:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.797+0000 7fc347fff700 1 -- 172.21.15.28:0/53300174 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348011d20 con 0x7fc35415a6c0 2023-11-28T11:10:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.797+0000 7fc35ba51700 1 -- 172.21.15.28:0/53300174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc35415a6c0 msgr2=0x7fc3540cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.797+0000 7fc35ba51700 1 --2- 172.21.15.28:0/53300174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc35415a6c0 0x7fc3540cb230 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc3540cb1c0 tx=0x7fc348009b20 comp rx=0 tx=0).stop 2023-11-28T11:10:45.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.797+0000 7fc35ba51700 1 -- 172.21.15.28:0/53300174 shutdown_connections 2023-11-28T11:10:45.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.797+0000 7fc35ba51700 1 --2- 172.21.15.28:0/53300174 >> v2:172.21.15.59:3300/0 conn(0x7fc3540b2660 0x7fc3540b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.797+0000 7fc35ba51700 1 --2- 172.21.15.28:0/53300174 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc35415a6c0 0x7fc3540cb230 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.797+0000 7fc35ba51700 1 -- 172.21.15.28:0/53300174 >> 172.21.15.28:0/53300174 conn(0x7fc3540cd660 msgr2=0x7fc3540cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:45.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.798+0000 7fc35ba51700 1 -- 172.21.15.28:0/53300174 shutdown_connections 2023-11-28T11:10:45.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.798+0000 7fc35ba51700 1 -- 172.21.15.28:0/53300174 wait complete. 2023-11-28T11:10:45.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.799+0000 7fc35ba51700 1 Processor -- start 2023-11-28T11:10:45.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.799+0000 7fc35ba51700 1 -- start start 2023-11-28T11:10:45.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.799+0000 7fc35ba51700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3540b2660 0x7fc3540c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.799+0000 7fc35ba51700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3540d0020 0x7fc354152430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.799+0000 7fc35ba51700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc35415a6c0 0x7fc35415b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc35ba51700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc354152970 con 0x7fc3540b2660 2023-11-28T11:10:45.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc35ba51700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc35415b9e0 con 0x7fc3540d0020 2023-11-28T11:10:45.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc35ba51700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3540c4cd0 con 0x7fc35415a6c0 2023-11-28T11:10:45.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc3597ed700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3540b2660 0x7fc3540c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc3597ed700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3540b2660 0x7fc3540c5d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50018/0 (socket says 172.21.15.28:50018) 2023-11-28T11:10:45.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc3597ed700 1 -- 172.21.15.28:0/2846386807 learned_addr learned my addr 172.21.15.28:0/2846386807 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:45.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc358fec700 1 --2- 172.21.15.28:0/2846386807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3540d0020 0x7fc354152430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc359fee700 1 --2- 172.21.15.28:0/2846386807 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc35415a6c0 0x7fc35415b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc3597ed700 1 -- 172.21.15.28:0/2846386807 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc35415a6c0 msgr2=0x7fc35415b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc3597ed700 1 --2- 172.21.15.28:0/2846386807 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc35415a6c0 0x7fc35415b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc3597ed700 1 -- 172.21.15.28:0/2846386807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3540d0020 msgr2=0x7fc354152430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:45.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc3597ed700 1 --2- 172.21.15.28:0/2846386807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3540d0020 0x7fc354152430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:45.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc3597ed700 1 -- 172.21.15.28:0/2846386807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3480094a0 con 0x7fc3540b2660 2023-11-28T11:10:45.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc359fee700 1 --2- 172.21.15.28:0/2846386807 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc35415a6c0 0x7fc35415b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:45.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc3597ed700 1 --2- 172.21.15.28:0/2846386807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3540b2660 0x7fc3540c5d70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc3480090b0 tx=0x7fc348004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:45.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc345ffb700 1 -- 172.21.15.28:0/2846386807 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3480038a0 con 0x7fc3540b2660 2023-11-28T11:10:45.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc345ffb700 1 -- 172.21.15.28:0/2846386807 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc34801e070 con 0x7fc3540b2660 2023-11-28T11:10:45.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc35ba51700 1 -- 172.21.15.28:0/2846386807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3540cf820 con 0x7fc3540b2660 2023-11-28T11:10:45.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc345ffb700 1 -- 172.21.15.28:0/2846386807 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc348003c60 con 0x7fc3540b2660 2023-11-28T11:10:45.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.800+0000 7fc35ba51700 1 -- 172.21.15.28:0/2846386807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3540cf540 con 0x7fc3540b2660 2023-11-28T11:10:45.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.802+0000 7fc35ba51700 1 -- 172.21.15.28:0/2846386807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3540d0470 con 0x7fc3540b2660 2023-11-28T11:10:45.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.802+0000 7fc345ffb700 1 -- 172.21.15.28:0/2846386807 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc34800f050 con 0x7fc3540b2660 2023-11-28T11:10:45.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.802+0000 7fc345ffb700 1 --2- 172.21.15.28:0/2846386807 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3380682f0 0x7fc33806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:45.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.802+0000 7fc345ffb700 1 -- 172.21.15.28:0/2846386807 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fc348086ea0 con 0x7fc3540b2660 2023-11-28T11:10:45.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.802+0000 7fc358fec700 1 --2- 172.21.15.28:0/2846386807 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3380682f0 0x7fc33806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:45.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.803+0000 7fc358fec700 1 --2- 172.21.15.28:0/2846386807 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3380682f0 0x7fc33806a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc3540cef40 tx=0x7fc350009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:45.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:45.807+0000 7fc345ffb700 1 -- 172.21.15.28:0/2846386807 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc348053580 con 0x7fc3540b2660 2023-11-28T11:10:46.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.004+0000 7fc35ba51700 1 -- 172.21.15.28:0/2846386807 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc3540675a0 con 0x7fc3540b2660 2023-11-28T11:10:46.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.005+0000 7fc345ffb700 1 -- 172.21.15.28:0/2846386807 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18174 (secure 0 0 0) 0x7fc348020090 con 0x7fc3540b2660 2023-11-28T11:10:46.007 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:46.007 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":42,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:10:35.252810+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:08.429278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:09.797038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.18","mappings":[{"from":0,"to":4}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:46.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.007+0000 7fc35ba51700 1 -- 172.21.15.28:0/2846386807 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3380682f0 msgr2=0x7fc33806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.007+0000 7fc35ba51700 1 --2- 172.21.15.28:0/2846386807 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3380682f0 0x7fc33806a7a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc3540cef40 tx=0x7fc350009040 comp rx=0 tx=0).stop 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.007+0000 7fc35ba51700 1 -- 172.21.15.28:0/2846386807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3540b2660 msgr2=0x7fc3540c5d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.007+0000 7fc35ba51700 1 --2- 172.21.15.28:0/2846386807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3540b2660 0x7fc3540c5d70 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fc3480090b0 tx=0x7fc348004750 comp rx=0 tx=0).stop 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.007+0000 7fc35ba51700 1 -- 172.21.15.28:0/2846386807 shutdown_connections 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.007+0000 7fc35ba51700 1 --2- 172.21.15.28:0/2846386807 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3380682f0 0x7fc33806a7a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.007+0000 7fc35ba51700 1 --2- 172.21.15.28:0/2846386807 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc3540b2660 0x7fc3540c5d70 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.007+0000 7fc35ba51700 1 --2- 172.21.15.28:0/2846386807 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc35415a6c0 0x7fc35415b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.007+0000 7fc35ba51700 1 --2- 172.21.15.28:0/2846386807 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc3540d0020 0x7fc354152430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.007+0000 7fc35ba51700 1 -- 172.21.15.28:0/2846386807 >> 172.21.15.28:0/2846386807 conn(0x7fc3540cd660 msgr2=0x7fc3540cda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.007+0000 7fc35ba51700 1 -- 172.21.15.28:0/2846386807 shutdown_connections 2023-11-28T11:10:46.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.007+0000 7fc35ba51700 1 -- 172.21.15.28:0/2846386807 wait complete. 2023-11-28T11:10:46.021 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:10:46.021 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:46.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.144+0000 7f8487546700 1 Processor -- start 2023-11-28T11:10:46.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.144+0000 7f8487546700 1 -- start start 2023-11-28T11:10:46.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f8487546700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8480161090 0x7f84800ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f8487546700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84800ce340 con 0x7f848015c990 2023-11-28T11:10:46.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f8487546700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84800cebe0 con 0x7f84800caeb0 2023-11-28T11:10:46.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f8487546700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84800cf780 con 0x7f8480161090 2023-11-28T11:10:46.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f84852e2700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f848015c990 0x7f84800cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38084/0 (socket says 172.21.15.28:38084) 2023-11-28T11:10:46.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f84852e2700 1 -- 172.21.15.28:0/469734851 learned_addr learned my addr 172.21.15.28:0/469734851 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:46.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f8477fff700 1 -- 172.21.15.28:0/469734851 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374764190 0 0) 0x7f84800ce340 con 0x7f848015c990 2023-11-28T11:10:46.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f8477fff700 1 -- 172.21.15.28:0/469734851 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f846c0035b0 con 0x7f848015c990 2023-11-28T11:10:46.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f8484ae1700 1 --2- 172.21.15.28:0/469734851 >> v2:172.21.15.59:3300/0 conn(0x7f8480161090 0x7f84800ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f8477fff700 1 -- 172.21.15.28:0/469734851 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1411255204 0 0) 0x7f846c0035b0 con 0x7f848015c990 2023-11-28T11:10:46.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f8477fff700 1 -- 172.21.15.28:0/469734851 >> v2:172.21.15.59:3300/0 conn(0x7f8480161090 msgr2=0x7f84800ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:46.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f8477fff700 1 --2- 172.21.15.28:0/469734851 >> v2:172.21.15.59:3300/0 conn(0x7f8480161090 0x7f84800ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f8477fff700 1 -- 172.21.15.28:0/469734851 >> v1:172.21.15.59:6789/0 conn(0x7f84800caeb0 legacy=0x7f84800cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f8477fff700 1 -- 172.21.15.28:0/469734851 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84800cb3f0 con 0x7f848015c990 2023-11-28T11:10:46.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f8477fff700 1 -- 172.21.15.28:0/469734851 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f847c001670 con 0x7f848015c990 2023-11-28T11:10:46.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.145+0000 7f8477fff700 1 -- 172.21.15.28:0/469734851 >> v1:172.21.15.28:6789/0 conn(0x7f848015c990 legacy=0x7f84800cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8477fff700 1 --2- 172.21.15.28:0/469734851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846c003a00 0x7f846c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8477fff700 1 --2- 172.21.15.28:0/469734851 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f846c004850 0x7f846c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8477fff700 1 --2- 172.21.15.28:0/469734851 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f846c005400 0x7f846c00ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8477fff700 1 -- 172.21.15.28:0/469734851 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84800cf780 con 0x7f846c004850 2023-11-28T11:10:46.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8477fff700 1 -- 172.21.15.28:0/469734851 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c00c280 con 0x7f846c003a00 2023-11-28T11:10:46.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8477fff700 1 -- 172.21.15.28:0/469734851 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846c00c440 con 0x7f846c005400 2023-11-28T11:10:46.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8484ae1700 1 --2- 172.21.15.28:0/469734851 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f846c004850 0x7f846c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f84852e2700 1 --2- 172.21.15.28:0/469734851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846c003a00 0x7f846c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8485ae3700 1 --2- 172.21.15.28:0/469734851 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f846c005400 0x7f846c00ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8484ae1700 1 -- 172.21.15.28:0/469734851 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f846c005400 msgr2=0x7f846c00ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:46.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8484ae1700 1 --2- 172.21.15.28:0/469734851 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f846c005400 0x7f846c00ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8484ae1700 1 -- 172.21.15.28:0/469734851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846c003a00 msgr2=0x7f846c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:46.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8484ae1700 1 --2- 172.21.15.28:0/469734851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846c003a00 0x7f846c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8484ae1700 1 -- 172.21.15.28:0/469734851 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846c0035b0 con 0x7f846c004850 2023-11-28T11:10:46.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8485ae3700 1 --2- 172.21.15.28:0/469734851 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f846c005400 0x7f846c00ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:46.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8485ae3700 1 -- 172.21.15.28:0/469734851 reap_dead start 2023-11-28T11:10:46.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8484ae1700 1 --2- 172.21.15.28:0/469734851 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f846c004850 0x7f846c004cc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8470009890 tx=0x7f846c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35dddde81ab42ec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8477fff700 1 -- 172.21.15.28:0/469734851 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847000c070 con 0x7f846c004850 2023-11-28T11:10:46.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8477fff700 1 -- 172.21.15.28:0/469734851 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8470013610 con 0x7f846c004850 2023-11-28T11:10:46.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.146+0000 7f8477fff700 1 -- 172.21.15.28:0/469734851 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700139a0 con 0x7f846c004850 2023-11-28T11:10:46.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.147+0000 7f8487546700 1 -- 172.21.15.28:0/469734851 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f846c004850 msgr2=0x7f846c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.147+0000 7f8487546700 1 --2- 172.21.15.28:0/469734851 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f846c004850 0x7f846c004cc0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f8470009890 tx=0x7f846c005880 comp rx=0 tx=0).stop 2023-11-28T11:10:46.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.147+0000 7f8487546700 1 -- 172.21.15.28:0/469734851 shutdown_connections 2023-11-28T11:10:46.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.147+0000 7f8487546700 1 --2- 172.21.15.28:0/469734851 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f846c004850 0x7f846c004cc0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.147+0000 7f8487546700 1 -- 172.21.15.28:0/469734851 >> 172.21.15.28:0/469734851 conn(0x7f84800b1650 msgr2=0x7f84800b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:46.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.147+0000 7f8487546700 1 -- 172.21.15.28:0/469734851 shutdown_connections 2023-11-28T11:10:46.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.147+0000 7f8487546700 1 -- 172.21.15.28:0/469734851 wait complete. 2023-11-28T11:10:46.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.148+0000 7f8487546700 1 Processor -- start 2023-11-28T11:10:46.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.148+0000 7f8487546700 1 -- start start 2023-11-28T11:10:46.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f8487546700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f846c004850 0x7f84800b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f8487546700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84800b41d0 0x7f84800c4fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f8487546700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84800c58c0 0x7f84800c5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f8487546700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84800c6280 con 0x7f846c004850 2023-11-28T11:10:46.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f8487546700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84800c79d0 con 0x7f84800b41d0 2023-11-28T11:10:46.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f8487546700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84800b3ab0 con 0x7f84800c58c0 2023-11-28T11:10:46.156 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f84852e2700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f846c004850 0x7f84800b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f84852e2700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f846c004850 0x7f84800b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50038/0 (socket says 172.21.15.28:50038) 2023-11-28T11:10:46.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f84852e2700 1 -- 172.21.15.28:0/1316883656 learned_addr learned my addr 172.21.15.28:0/1316883656 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:46.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f8485ae3700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84800c58c0 0x7f84800c5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f8484ae1700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84800b41d0 0x7f84800c4fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.157 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f84852e2700 1 -- 172.21.15.28:0/1316883656 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84800c58c0 msgr2=0x7f84800c5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f84852e2700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84800c58c0 0x7f84800c5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f84852e2700 1 -- 172.21.15.28:0/1316883656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84800b41d0 msgr2=0x7f84800c4fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f84852e2700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84800b41d0 0x7f84800c4fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f84852e2700 1 -- 172.21.15.28:0/1316883656 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84800ce870 con 0x7f846c004850 2023-11-28T11:10:46.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f8485ae3700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84800c58c0 0x7f84800c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:46.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f8484ae1700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84800b41d0 0x7f84800c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:46.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.149+0000 7f84852e2700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f846c004850 0x7f84800b2950 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f847c0072f0 tx=0x7f847c0076b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.150+0000 7f8475ffb700 1 -- 172.21.15.28:0/1316883656 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c00f8d0 con 0x7f846c004850 2023-11-28T11:10:46.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.150+0000 7f8475ffb700 1 -- 172.21.15.28:0/1316883656 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f847c00fa70 con 0x7f846c004850 2023-11-28T11:10:46.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.150+0000 7f8475ffb700 1 -- 172.21.15.28:0/1316883656 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847c00fe20 con 0x7f846c004850 2023-11-28T11:10:46.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.150+0000 7f8487546700 1 -- 172.21.15.28:0/1316883656 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84800ce590 con 0x7f846c004850 2023-11-28T11:10:46.160 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.150+0000 7f8487546700 1 -- 172.21.15.28:0/1316883656 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84800ca970 con 0x7f846c004850 2023-11-28T11:10:46.160 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.151+0000 7f8487546700 1 -- 172.21.15.28:0/1316883656 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84800c7b60 con 0x7f846c004850 2023-11-28T11:10:46.160 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.151+0000 7f8475ffb700 1 -- 172.21.15.28:0/1316883656 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f847c01d460 con 0x7f846c004850 2023-11-28T11:10:46.160 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.151+0000 7f8475ffb700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f84640682f0 0x7f846406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.160 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.152+0000 7f8475ffb700 1 -- 172.21.15.28:0/1316883656 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f847c086000 con 0x7f846c004850 2023-11-28T11:10:46.160 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.152+0000 7f8484ae1700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f84640682f0 0x7f846406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.152+0000 7f8484ae1700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f84640682f0 0x7f846406a7a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8470008010 tx=0x7f847000c000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.161 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.156+0000 7f8475ffb700 1 -- 172.21.15.28:0/1316883656 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f84800c7b60 con 0x7f846c004850 2023-11-28T11:10:46.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.394+0000 7f8487546700 1 -- 172.21.15.28:0/1316883656 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f84800c7b60 con 0x7f846c004850 2023-11-28T11:10:46.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.394+0000 7f8475ffb700 1 -- 172.21.15.28:0/1316883656 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f84800c7b60 con 0x7f846c004850 2023-11-28T11:10:46.396 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:46.396 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2867100265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2867100265},{"type":"v1","addr":"172.21.15.59:6833","nonce":2867100265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3808190953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3808190953},{"type":"v1","addr":"172.21.15.28:6837","nonce":3808190953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3920280453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3920280453},{"type":"v1","addr":"172.21.15.28:6836","nonce":3920280453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:10.820923+0000","modified":"2023-11-28T11:10:13.833216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6835/707419743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":707419743},{"type":"v1","addr":"172.21.15.59:6835","nonce":707419743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:10:46.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.396+0000 7f8487546700 1 -- 172.21.15.28:0/1316883656 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f84640682f0 msgr2=0x7f846406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.396+0000 7f8487546700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f84640682f0 0x7f846406a7a0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f8470008010 tx=0x7f847000c000 comp rx=0 tx=0).stop 2023-11-28T11:10:46.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.396+0000 7f8487546700 1 -- 172.21.15.28:0/1316883656 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f846c004850 msgr2=0x7f84800b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.396+0000 7f8487546700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f846c004850 0x7f84800b2950 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f847c0072f0 tx=0x7f847c0076b0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.397+0000 7f8487546700 1 -- 172.21.15.28:0/1316883656 shutdown_connections 2023-11-28T11:10:46.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.397+0000 7f8487546700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f84640682f0 0x7f846406a7a0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.397+0000 7f8487546700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f846c004850 0x7f84800b2950 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.397+0000 7f8487546700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f84800c58c0 0x7f84800c5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.397+0000 7f8487546700 1 --2- 172.21.15.28:0/1316883656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84800b41d0 0x7f84800c4fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.397+0000 7f8487546700 1 -- 172.21.15.28:0/1316883656 >> 172.21.15.28:0/1316883656 conn(0x7f84800b1650 msgr2=0x7f848006d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:46.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.397+0000 7f8487546700 1 -- 172.21.15.28:0/1316883656 shutdown_connections 2023-11-28T11:10:46.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.397+0000 7f8487546700 1 -- 172.21.15.28:0/1316883656 wait complete. 2023-11-28T11:10:46.399 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2023-11-28T11:10:46.410 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:46.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.533+0000 7fc352ce8700 1 Processor -- start 2023-11-28T11:10:46.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.533+0000 7fc352ce8700 1 -- start start 2023-11-28T11:10:46.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.534+0000 7fc352ce8700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc34c159270 0x7fc34c0cab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.534+0000 7fc352ce8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc34c0cb0a0 0x7fc34c0cd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.534+0000 7fc352ce8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc34c0cdfe0 con 0x7fc34c0cdaa0 2023-11-28T11:10:46.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.534+0000 7fc352ce8700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc34c06bf40 con 0x7fc34c159270 2023-11-28T11:10:46.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.534+0000 7fc352ce8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc34c06c150 con 0x7fc34c0cb0a0 2023-11-28T11:10:46.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.534+0000 7fc350a84700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc34c159270 0x7fc34c0cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.534+0000 7fc350a84700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc34c159270 0x7fc34c0cab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50046/0 (socket says 172.21.15.28:50046) 2023-11-28T11:10:46.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.534+0000 7fc350a84700 1 -- 172.21.15.28:0/3002521262 learned_addr learned my addr 172.21.15.28:0/3002521262 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:46.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.534+0000 7fc34bfff700 1 --2- 172.21.15.28:0/3002521262 >> v2:172.21.15.59:3300/0 conn(0x7fc34c0cb0a0 0x7fc34c0cd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.534+0000 7fc350a84700 1 -- 172.21.15.28:0/3002521262 >> v2:172.21.15.59:3300/0 conn(0x7fc34c0cb0a0 msgr2=0x7fc34c0cd560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:46.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.534+0000 7fc350a84700 1 --2- 172.21.15.28:0/3002521262 >> v2:172.21.15.59:3300/0 conn(0x7fc34c0cb0a0 0x7fc34c0cd560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.534+0000 7fc350a84700 1 -- 172.21.15.28:0/3002521262 >> v1:172.21.15.59:6789/0 conn(0x7fc34c0cdaa0 legacy=0x7fc34c0ce410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:46.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.534+0000 7fc350a84700 1 -- 172.21.15.28:0/3002521262 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc34c06c430 con 0x7fc34c159270 2023-11-28T11:10:46.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.535+0000 7fc34bfff700 1 --2- 172.21.15.28:0/3002521262 >> v2:172.21.15.59:3300/0 conn(0x7fc34c0cb0a0 0x7fc34c0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:46.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.535+0000 7fc350a84700 1 --2- 172.21.15.28:0/3002521262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc34c159270 0x7fc34c0cab60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0b2550 tx=0x7fc340009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1545ccb2dded1c37 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.535+0000 7fc34b7fe700 1 -- 172.21.15.28:0/3002521262 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc340004230 con 0x7fc34c159270 2023-11-28T11:10:46.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.535+0000 7fc34b7fe700 1 -- 172.21.15.28:0/3002521262 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc340004390 con 0x7fc34c159270 2023-11-28T11:10:46.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.535+0000 7fc34b7fe700 1 -- 172.21.15.28:0/3002521262 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3400036a0 con 0x7fc34c159270 2023-11-28T11:10:46.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.535+0000 7fc352ce8700 1 -- 172.21.15.28:0/3002521262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc34c159270 msgr2=0x7fc34c0cab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.535+0000 7fc352ce8700 1 --2- 172.21.15.28:0/3002521262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc34c159270 0x7fc34c0cab60 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0b2550 tx=0x7fc340009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.535+0000 7fc352ce8700 1 -- 172.21.15.28:0/3002521262 shutdown_connections 2023-11-28T11:10:46.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.535+0000 7fc352ce8700 1 --2- 172.21.15.28:0/3002521262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc34c159270 0x7fc34c0cab60 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.535+0000 7fc352ce8700 1 --2- 172.21.15.28:0/3002521262 >> v2:172.21.15.59:3300/0 conn(0x7fc34c0cb0a0 0x7fc34c0cd560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.535+0000 7fc352ce8700 1 -- 172.21.15.28:0/3002521262 >> 172.21.15.28:0/3002521262 conn(0x7fc34c06dad0 msgr2=0x7fc34c06dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:46.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.535+0000 7fc352ce8700 1 -- 172.21.15.28:0/3002521262 shutdown_connections 2023-11-28T11:10:46.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.535+0000 7fc352ce8700 1 -- 172.21.15.28:0/3002521262 wait complete. 2023-11-28T11:10:46.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.537+0000 7fc352ce8700 1 Processor -- start 2023-11-28T11:10:46.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.537+0000 7fc352ce8700 1 -- start start 2023-11-28T11:10:46.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.537+0000 7fc352ce8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc34c0cb0a0 0x7fc34c1500e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.537+0000 7fc352ce8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc34c0cdaa0 0x7fc34c155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.537+0000 7fc352ce8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c0ab8c0 0x7fc34c0c9220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc352ce8700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0abe30 con 0x7fc34c0cdaa0 2023-11-28T11:10:46.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc352ce8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0ac110 con 0x7fc34c0ab8c0 2023-11-28T11:10:46.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc352ce8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc34c0ac3f0 con 0x7fc34c0cb0a0 2023-11-28T11:10:46.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc34bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc34c0cdaa0 0x7fc34c155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc34bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc34c0cdaa0 0x7fc34c155150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50054/0 (socket says 172.21.15.28:50054) 2023-11-28T11:10:46.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc34bfff700 1 -- 172.21.15.28:0/1081652964 learned_addr learned my addr 172.21.15.28:0/1081652964 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:46.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc350a84700 1 --2- 172.21.15.28:0/1081652964 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc34c0cb0a0 0x7fc34c1500e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc351285700 1 --2- 172.21.15.28:0/1081652964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c0ab8c0 0x7fc34c0c9220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc34bfff700 1 -- 172.21.15.28:0/1081652964 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc34c0cb0a0 msgr2=0x7fc34c1500e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc34bfff700 1 --2- 172.21.15.28:0/1081652964 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc34c0cb0a0 0x7fc34c1500e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc34bfff700 1 -- 172.21.15.28:0/1081652964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c0ab8c0 msgr2=0x7fc34c0c9220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc34bfff700 1 --2- 172.21.15.28:0/1081652964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c0ab8c0 0x7fc34c0c9220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc34bfff700 1 -- 172.21.15.28:0/1081652964 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc340009550 con 0x7fc34c0cdaa0 2023-11-28T11:10:46.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc34bfff700 1 --2- 172.21.15.28:0/1081652964 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc34c0cdaa0 0x7fc34c155150 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc33800c7f0 tx=0x7fc33800cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc3497fa700 1 -- 172.21.15.28:0/1081652964 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338011800 con 0x7fc34c0cdaa0 2023-11-28T11:10:46.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc3497fa700 1 -- 172.21.15.28:0/1081652964 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc338011960 con 0x7fc34c0cdaa0 2023-11-28T11:10:46.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc352ce8700 1 -- 172.21.15.28:0/1081652964 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc34c159640 con 0x7fc34c0cdaa0 2023-11-28T11:10:46.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc3497fa700 1 -- 172.21.15.28:0/1081652964 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc338011cf0 con 0x7fc34c0cdaa0 2023-11-28T11:10:46.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.538+0000 7fc352ce8700 1 -- 172.21.15.28:0/1081652964 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc34c06c710 con 0x7fc34c0cdaa0 2023-11-28T11:10:46.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.539+0000 7fc352ce8700 1 -- 172.21.15.28:0/1081652964 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc34c15dc00 con 0x7fc34c0cdaa0 2023-11-28T11:10:46.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.540+0000 7fc3497fa700 1 -- 172.21.15.28:0/1081652964 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc338021070 con 0x7fc34c0cdaa0 2023-11-28T11:10:46.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.540+0000 7fc3497fa700 1 --2- 172.21.15.28:0/1081652964 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3300682f0 0x7fc33006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.540+0000 7fc3497fa700 1 -- 172.21.15.28:0/1081652964 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fc338085be0 con 0x7fc34c0cdaa0 2023-11-28T11:10:46.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.540+0000 7fc350a84700 1 --2- 172.21.15.28:0/1081652964 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3300682f0 0x7fc33006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.540+0000 7fc350a84700 1 --2- 172.21.15.28:0/1081652964 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3300682f0 0x7fc33006a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0b2490 tx=0x7fc340008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.544+0000 7fc3497fa700 1 -- 172.21.15.28:0/1081652964 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc338052260 con 0x7fc34c0cdaa0 2023-11-28T11:10:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.782+0000 7fc352ce8700 1 -- 172.21.15.28:0/1081652964 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc34c0739f0 con 0x7fc34c0cdaa0 2023-11-28T11:10:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.782+0000 7fc3497fa700 1 -- 172.21.15.28:0/1081652964 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4645 (secure 0 0 0) 0x7fc338055880 con 0x7fc34c0cdaa0 2023-11-28T11:10:46.784 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:46.784 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2867100265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2867100265},{"type":"v1","addr":"172.21.15.59:6833","nonce":2867100265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3808190953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3808190953},{"type":"v1","addr":"172.21.15.28:6837","nonce":3808190953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3920280453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3920280453},{"type":"v1","addr":"172.21.15.28:6836","nonce":3920280453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:10.820923+0000","modified":"2023-11-28T11:10:13.833216+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5547},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5547":{"gid":5547,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":4,"addr":"172.21.15.59:6835/707419743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":707419743},{"type":"v1","addr":"172.21.15.59:6835","nonce":707419743}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-11-28T11:10:46.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.784+0000 7fc352ce8700 1 -- 172.21.15.28:0/1081652964 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3300682f0 msgr2=0x7fc33006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.784+0000 7fc352ce8700 1 --2- 172.21.15.28:0/1081652964 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3300682f0 0x7fc33006a7a0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc34c0b2490 tx=0x7fc340008030 comp rx=0 tx=0).stop 2023-11-28T11:10:46.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.784+0000 7fc352ce8700 1 -- 172.21.15.28:0/1081652964 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc34c0cdaa0 msgr2=0x7fc34c155150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.784+0000 7fc352ce8700 1 --2- 172.21.15.28:0/1081652964 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc34c0cdaa0 0x7fc34c155150 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fc33800c7f0 tx=0x7fc33800cb00 comp rx=0 tx=0).stop 2023-11-28T11:10:46.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.784+0000 7fc352ce8700 1 -- 172.21.15.28:0/1081652964 shutdown_connections 2023-11-28T11:10:46.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.784+0000 7fc352ce8700 1 --2- 172.21.15.28:0/1081652964 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3300682f0 0x7fc33006a7a0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.784+0000 7fc352ce8700 1 --2- 172.21.15.28:0/1081652964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c0ab8c0 0x7fc34c0c9220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.784+0000 7fc352ce8700 1 --2- 172.21.15.28:0/1081652964 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc34c0cb0a0 0x7fc34c1500e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.784+0000 7fc352ce8700 1 --2- 172.21.15.28:0/1081652964 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc34c0cdaa0 0x7fc34c155150 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.784+0000 7fc352ce8700 1 -- 172.21.15.28:0/1081652964 >> 172.21.15.28:0/1081652964 conn(0x7fc34c06dad0 msgr2=0x7fc34c0af710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:46.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.784+0000 7fc352ce8700 1 -- 172.21.15.28:0/1081652964 shutdown_connections 2023-11-28T11:10:46.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.784+0000 7fc352ce8700 1 -- 172.21.15.28:0/1081652964 wait complete. 2023-11-28T11:10:46.787 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 17 2023-11-28T11:10:46.798 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:46.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.920+0000 7fb6055cf700 1 Processor -- start 2023-11-28T11:10:46.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.920+0000 7fb6055cf700 1 -- start start 2023-11-28T11:10:46.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.922+0000 7fb6055cf700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb60015a5a0 0x7fb60006b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.922+0000 7fb6055cf700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6000cb570 con 0x7fb6000cc4b0 2023-11-28T11:10:46.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.922+0000 7fb6055cf700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6000ca6f0 con 0x7fb6000ca280 2023-11-28T11:10:46.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.922+0000 7fb6055cf700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6000cc110 con 0x7fb60015a5a0 2023-11-28T11:10:46.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.922+0000 7fb5feffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb60015a5a0 0x7fb60006b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.922+0000 7fb5ff7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb6000cc4b0 0x7fb6000c35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38122/0 (socket says 172.21.15.28:38122) 2023-11-28T11:10:46.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.922+0000 7fb5ff7fe700 1 -- 172.21.15.28:0/2581070775 learned_addr learned my addr 172.21.15.28:0/2581070775 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:46.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.922+0000 7fb5feffd700 1 -- 172.21.15.28:0/2581070775 >> v1:172.21.15.59:6789/0 conn(0x7fb6000ca280 legacy=0x7fb6000cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.922+0000 7fb5feffd700 1 -- 172.21.15.28:0/2581070775 >> v1:172.21.15.28:6789/0 conn(0x7fb6000cc4b0 legacy=0x7fb6000c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.922+0000 7fb5feffd700 1 -- 172.21.15.28:0/2581070775 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6000ca960 con 0x7fb60015a5a0 2023-11-28T11:10:46.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.922+0000 7fb5feffd700 1 --2- 172.21.15.28:0/2581070775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb60015a5a0 0x7fb60006b930 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009840 tx=0x7fb5f0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58db3303da792ce6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.922+0000 7fb5fdffb700 1 -- 172.21.15.28:0/2581070775 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0011830 con 0x7fb60015a5a0 2023-11-28T11:10:46.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.922+0000 7fb5fdffb700 1 -- 172.21.15.28:0/2581070775 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f0011990 con 0x7fb60015a5a0 2023-11-28T11:10:46.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.922+0000 7fb5fdffb700 1 -- 172.21.15.28:0/2581070775 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f0011d20 con 0x7fb60015a5a0 2023-11-28T11:10:46.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.923+0000 7fb6055cf700 1 -- 172.21.15.28:0/2581070775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb60015a5a0 msgr2=0x7fb60006b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.923+0000 7fb6055cf700 1 --2- 172.21.15.28:0/2581070775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb60015a5a0 0x7fb60006b930 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb5f0009840 tx=0x7fb5f0009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:46.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.923+0000 7fb6055cf700 1 -- 172.21.15.28:0/2581070775 shutdown_connections 2023-11-28T11:10:46.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.923+0000 7fb6055cf700 1 --2- 172.21.15.28:0/2581070775 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb60015a5a0 0x7fb60006b930 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.923+0000 7fb6055cf700 1 -- 172.21.15.28:0/2581070775 >> 172.21.15.28:0/2581070775 conn(0x7fb6000b1650 msgr2=0x7fb6000b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:46.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.923+0000 7fb6055cf700 1 -- 172.21.15.28:0/2581070775 shutdown_connections 2023-11-28T11:10:46.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.923+0000 7fb6055cf700 1 -- 172.21.15.28:0/2581070775 wait complete. 2023-11-28T11:10:46.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.924+0000 7fb6055cf700 1 Processor -- start 2023-11-28T11:10:46.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.924+0000 7fb6055cf700 1 -- start start 2023-11-28T11:10:46.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.925+0000 7fb6055cf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb6000ca280 0x7fb60014d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.925+0000 7fb6055cf700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6000cc4b0 0x7fb600159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.925+0000 7fb6055cf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb60015a5a0 0x7fb6001f3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.925+0000 7fb6055cf700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb600144760 con 0x7fb6000cc4b0 2023-11-28T11:10:46.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.925+0000 7fb6055cf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb600141940 con 0x7fb60015a5a0 2023-11-28T11:10:46.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.925+0000 7fb6055cf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb600141c20 con 0x7fb6000ca280 2023-11-28T11:10:46.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.925+0000 7fb5fe7fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6000cc4b0 0x7fb600159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.925+0000 7fb5fe7fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6000cc4b0 0x7fb600159990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50068/0 (socket says 172.21.15.28:50068) 2023-11-28T11:10:46.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.925+0000 7fb5fe7fc700 1 -- 172.21.15.28:0/2760038654 learned_addr learned my addr 172.21.15.28:0/2760038654 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:46.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.925+0000 7fb5ff7fe700 1 --2- 172.21.15.28:0/2760038654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb60015a5a0 0x7fb6001f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.925+0000 7fb5feffd700 1 --2- 172.21.15.28:0/2760038654 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb6000ca280 0x7fb60014d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.926+0000 7fb5fe7fc700 1 -- 172.21.15.28:0/2760038654 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb6000ca280 msgr2=0x7fb60014d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.926+0000 7fb5fe7fc700 1 --2- 172.21.15.28:0/2760038654 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb6000ca280 0x7fb60014d400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.926+0000 7fb5fe7fc700 1 -- 172.21.15.28:0/2760038654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb60015a5a0 msgr2=0x7fb6001f3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:46.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.926+0000 7fb5fe7fc700 1 --2- 172.21.15.28:0/2760038654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb60015a5a0 0x7fb6001f3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:46.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.926+0000 7fb5fe7fc700 1 -- 172.21.15.28:0/2760038654 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5f00094a0 con 0x7fb6000cc4b0 2023-11-28T11:10:46.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.926+0000 7fb5fe7fc700 1 --2- 172.21.15.28:0/2760038654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6000cc4b0 0x7fb600159990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400c4e0 tx=0x7fb5f400c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.926+0000 7fb5e77fe700 1 -- 172.21.15.28:0/2760038654 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40052d0 con 0x7fb6000cc4b0 2023-11-28T11:10:46.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.926+0000 7fb5e77fe700 1 -- 172.21.15.28:0/2760038654 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5f4005430 con 0x7fb6000cc4b0 2023-11-28T11:10:46.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.926+0000 7fb6055cf700 1 -- 172.21.15.28:0/2760038654 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6000caf20 con 0x7fb6000cc4b0 2023-11-28T11:10:46.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.926+0000 7fb5e77fe700 1 -- 172.21.15.28:0/2760038654 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5f40038c0 con 0x7fb6000cc4b0 2023-11-28T11:10:46.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.926+0000 7fb6055cf700 1 -- 172.21.15.28:0/2760038654 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6000cac40 con 0x7fb6000cc4b0 2023-11-28T11:10:46.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.927+0000 7fb6055cf700 1 -- 172.21.15.28:0/2760038654 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6000cb200 con 0x7fb6000cc4b0 2023-11-28T11:10:46.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.927+0000 7fb5e77fe700 1 -- 172.21.15.28:0/2760038654 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb5f4019030 con 0x7fb6000cc4b0 2023-11-28T11:10:46.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.928+0000 7fb5e77fe700 1 --2- 172.21.15.28:0/2760038654 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb5ec0682f0 0x7fb5ec06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:46.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.928+0000 7fb5e77fe700 1 -- 172.21.15.28:0/2760038654 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fb5f4085030 con 0x7fb6000cc4b0 2023-11-28T11:10:46.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.928+0000 7fb5feffd700 1 --2- 172.21.15.28:0/2760038654 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb5ec0682f0 0x7fb5ec06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:46.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.928+0000 7fb5feffd700 1 --2- 172.21.15.28:0/2760038654 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb5ec0682f0 0x7fb5ec06a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00090b0 tx=0x7fb5f001c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:46.933 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:46.932+0000 7fb5e77fe700 1 -- 172.21.15.28:0/2760038654 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb5f4051710 con 0x7fb6000cc4b0 2023-11-28T11:10:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.126+0000 7fb6055cf700 1 -- 172.21.15.28:0/2760038654 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb6000675a0 con 0x7fb6000cc4b0 2023-11-28T11:10:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.127+0000 7fb5e77fe700 1 -- 172.21.15.28:0/2760038654 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18174 (secure 0 0 0) 0x7fb5f4017090 con 0x7fb6000cc4b0 2023-11-28T11:10:47.128 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:47.129 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":42,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:10:35.252810+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:08.429278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"39","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:09.797038+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"40","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.18","mappings":[{"from":0,"to":4}]},{"pgid":"7.3e","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.129+0000 7fb6055cf700 1 -- 172.21.15.28:0/2760038654 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb5ec0682f0 msgr2=0x7fb5ec06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.129+0000 7fb6055cf700 1 --2- 172.21.15.28:0/2760038654 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb5ec0682f0 0x7fb5ec06a7a0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb5f00090b0 tx=0x7fb5f001c040 comp rx=0 tx=0).stop 2023-11-28T11:10:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.129+0000 7fb6055cf700 1 -- 172.21.15.28:0/2760038654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6000cc4b0 msgr2=0x7fb600159990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.129+0000 7fb6055cf700 1 --2- 172.21.15.28:0/2760038654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6000cc4b0 0x7fb600159990 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb5f400c4e0 tx=0x7fb5f400c7f0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.129+0000 7fb6055cf700 1 -- 172.21.15.28:0/2760038654 shutdown_connections 2023-11-28T11:10:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.129+0000 7fb6055cf700 1 --2- 172.21.15.28:0/2760038654 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb5ec0682f0 0x7fb5ec06a7a0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.129+0000 7fb6055cf700 1 --2- 172.21.15.28:0/2760038654 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb60015a5a0 0x7fb6001f3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.129+0000 7fb6055cf700 1 --2- 172.21.15.28:0/2760038654 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb6000ca280 0x7fb60014d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.129+0000 7fb6055cf700 1 --2- 172.21.15.28:0/2760038654 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb6000cc4b0 0x7fb600159990 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.129+0000 7fb6055cf700 1 -- 172.21.15.28:0/2760038654 >> 172.21.15.28:0/2760038654 conn(0x7fb6000b1650 msgr2=0x7fb60015bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.129+0000 7fb6055cf700 1 -- 172.21.15.28:0/2760038654 shutdown_connections 2023-11-28T11:10:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.129+0000 7fb6055cf700 1 -- 172.21.15.28:0/2760038654 wait complete. 2023-11-28T11:10:47.143 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:10:47.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.267+0000 7f42bd870700 1 Processor -- start 2023-11-28T11:10:47.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.267+0000 7f42bd870700 1 -- start start 2023-11-28T11:10:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42bd870700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f42b815c640 0x7f42b80cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42bd870700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f42b80d2820 0x7f42b80b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42bd870700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b80d1470 con 0x7f42b806bf30 2023-11-28T11:10:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42bd870700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b815ca10 con 0x7f42b815c640 2023-11-28T11:10:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42bd870700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b80cd4e0 con 0x7f42b80d2820 2023-11-28T11:10:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42b6ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f42b815c640 0x7f42b80cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42b6ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f42b815c640 0x7f42b80cf040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50074/0 (socket says 172.21.15.28:50074) 2023-11-28T11:10:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42b77fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f42b806bf30 0x7f42b806c300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38136/0 (socket says 172.21.15.28:38136) 2023-11-28T11:10:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42b6ffd700 1 -- 172.21.15.28:0/158373860 learned_addr learned my addr 172.21.15.28:0/158373860 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42b67fc700 1 --2- 172.21.15.28:0/158373860 >> v2:172.21.15.59:3300/0 conn(0x7f42b80d2820 0x7f42b80b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:47.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42b6ffd700 1 -- 172.21.15.28:0/158373860 >> v2:172.21.15.59:3300/0 conn(0x7f42b80d2820 msgr2=0x7f42b80b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42b6ffd700 1 --2- 172.21.15.28:0/158373860 >> v2:172.21.15.59:3300/0 conn(0x7f42b80d2820 0x7f42b80b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42b6ffd700 1 -- 172.21.15.28:0/158373860 >> v1:172.21.15.28:6789/0 conn(0x7f42b806bf30 legacy=0x7f42b806c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42b6ffd700 1 -- 172.21.15.28:0/158373860 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b80cd7c0 con 0x7f42b815c640 2023-11-28T11:10:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42b5ffb700 1 -- 172.21.15.28:0/158373860 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1786982411 0 0) 0x7f42b80d1470 con 0x7f42b806bf30 2023-11-28T11:10:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42b67fc700 1 --2- 172.21.15.28:0/158373860 >> v2:172.21.15.59:3300/0 conn(0x7f42b80d2820 0x7f42b80b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42b6ffd700 1 --2- 172.21.15.28:0/158373860 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b815c640 0x7f42b80cf040 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f42b80c9e50 tx=0x7f42a8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25da3f691f89b97a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42b5ffb700 1 -- 172.21.15.28:0/158373860 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a800b870 con 0x7f42b815c640 2023-11-28T11:10:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42b5ffb700 1 -- 172.21.15.28:0/158373860 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a800b9d0 con 0x7f42b815c640 2023-11-28T11:10:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.268+0000 7f42b5ffb700 1 -- 172.21.15.28:0/158373860 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a800bd60 con 0x7f42b815c640 2023-11-28T11:10:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.269+0000 7f42bd870700 1 -- 172.21.15.28:0/158373860 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b815c640 msgr2=0x7f42b80cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.269+0000 7f42bd870700 1 --2- 172.21.15.28:0/158373860 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b815c640 0x7f42b80cf040 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f42b80c9e50 tx=0x7f42a8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.269+0000 7f42bd870700 1 -- 172.21.15.28:0/158373860 shutdown_connections 2023-11-28T11:10:47.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.269+0000 7f42bd870700 1 --2- 172.21.15.28:0/158373860 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b815c640 0x7f42b80cf040 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.269+0000 7f42bd870700 1 --2- 172.21.15.28:0/158373860 >> v2:172.21.15.59:3300/0 conn(0x7f42b80d2820 0x7f42b80b5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.269+0000 7f42bd870700 1 -- 172.21.15.28:0/158373860 >> 172.21.15.28:0/158373860 conn(0x7f42b80d2df0 msgr2=0x7f42b80d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:47.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.269+0000 7f42bd870700 1 -- 172.21.15.28:0/158373860 shutdown_connections 2023-11-28T11:10:47.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.269+0000 7f42bd870700 1 -- 172.21.15.28:0/158373860 wait complete. 2023-11-28T11:10:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.270+0000 7f42bd870700 1 Processor -- start 2023-11-28T11:10:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.270+0000 7f42bd870700 1 -- start start 2023-11-28T11:10:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.271+0000 7f42bd870700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42b806bf30 0x7f42b81f6660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.271+0000 7f42bd870700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b80d2820 0x7f42b8152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.271+0000 7f42bd870700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42b815c640 0x7f42b80b7c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:47.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.271+0000 7f42bd870700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80c9ec0 con 0x7f42b80d2820 2023-11-28T11:10:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.271+0000 7f42bd870700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b80ca760 con 0x7f42b806bf30 2023-11-28T11:10:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.271+0000 7f42bd870700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b81f6ba0 con 0x7f42b815c640 2023-11-28T11:10:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.271+0000 7f42b67fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b80d2820 0x7f42b8152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.271+0000 7f42b67fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b80d2820 0x7f42b8152e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50082/0 (socket says 172.21.15.28:50082) 2023-11-28T11:10:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.271+0000 7f42b67fc700 1 -- 172.21.15.28:0/3773055259 learned_addr learned my addr 172.21.15.28:0/3773055259 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.271+0000 7f42b67fc700 1 -- 172.21.15.28:0/3773055259 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42b815c640 msgr2=0x7f42b80b7c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.271+0000 7f42b67fc700 1 --2- 172.21.15.28:0/3773055259 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42b815c640 0x7f42b80b7c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.271+0000 7f42b67fc700 1 -- 172.21.15.28:0/3773055259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42b806bf30 msgr2=0x7f42b81f6660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.271+0000 7f42b67fc700 1 --2- 172.21.15.28:0/3773055259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42b806bf30 0x7f42b81f6660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.271+0000 7f42b67fc700 1 -- 172.21.15.28:0/3773055259 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a8009550 con 0x7f42b80d2820 2023-11-28T11:10:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.272+0000 7f42b67fc700 1 --2- 172.21.15.28:0/3773055259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b80d2820 0x7f42b8152e40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00e810 tx=0x7f42ac00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:47.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.272+0000 7f429ffff700 1 -- 172.21.15.28:0/3773055259 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac004230 con 0x7f42b80d2820 2023-11-28T11:10:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.272+0000 7f429ffff700 1 -- 172.21.15.28:0/3773055259 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42ac004390 con 0x7f42b80d2820 2023-11-28T11:10:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.272+0000 7f42bd870700 1 -- 172.21.15.28:0/3773055259 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b80cf580 con 0x7f42b80d2820 2023-11-28T11:10:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.272+0000 7f429ffff700 1 -- 172.21.15.28:0/3773055259 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42ac0036a0 con 0x7f42b80d2820 2023-11-28T11:10:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.272+0000 7f42bd870700 1 -- 172.21.15.28:0/3773055259 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b80d1710 con 0x7f42b80d2820 2023-11-28T11:10:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.273+0000 7f42bd870700 1 -- 172.21.15.28:0/3773055259 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b80cdaa0 con 0x7f42b80d2820 2023-11-28T11:10:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.273+0000 7f429ffff700 1 -- 172.21.15.28:0/3773055259 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f42ac022070 con 0x7f42b80d2820 2023-11-28T11:10:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.274+0000 7f429ffff700 1 --2- 172.21.15.28:0/3773055259 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42a40682f0 0x7f42a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.274+0000 7f42b6ffd700 1 --2- 172.21.15.28:0/3773055259 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42a40682f0 0x7f42a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:47.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.274+0000 7f429ffff700 1 -- 172.21.15.28:0/3773055259 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f42ac085940 con 0x7f42b80d2820 2023-11-28T11:10:47.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.274+0000 7f42b6ffd700 1 --2- 172.21.15.28:0/3773055259 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42a40682f0 0x7f42a406a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009c10 tx=0x7f42a8010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:47.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.278+0000 7f429ffff700 1 -- 172.21.15.28:0/3773055259 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f42ac051fc0 con 0x7f42b80d2820 2023-11-28T11:10:47.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:47.521+0000 7f42bd870700 1 -- 172.21.15.28:0/3773055259 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f42b80ca8d0 con 0x7f42b80d2820 2023-11-28T11:10:48.253 INFO:tasks.ceph.mon.a.smithi028.stderr:2023-11-28T11:10:48.252+0000 7fde5abc8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:10:48.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.256+0000 7f429ffff700 1 -- 172.21.15.28:0/3773055259 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7f42ac0555e0 con 0x7f42b80d2820 2023-11-28T11:10:48.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.260+0000 7f42bd870700 1 -- 172.21.15.28:0/3773055259 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42a40682f0 msgr2=0x7f42a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.260+0000 7f42bd870700 1 --2- 172.21.15.28:0/3773055259 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42a40682f0 0x7f42a406a7a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f42a8009c10 tx=0x7f42a8010000 comp rx=0 tx=0).stop 2023-11-28T11:10:48.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.260+0000 7f42bd870700 1 -- 172.21.15.28:0/3773055259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b80d2820 msgr2=0x7f42b8152e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.260+0000 7f42bd870700 1 --2- 172.21.15.28:0/3773055259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b80d2820 0x7f42b8152e40 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f42ac00e810 tx=0x7f42ac00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.260+0000 7f42bd870700 1 -- 172.21.15.28:0/3773055259 shutdown_connections 2023-11-28T11:10:48.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.260+0000 7f42bd870700 1 --2- 172.21.15.28:0/3773055259 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42a40682f0 0x7f42a406a7a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.260+0000 7f42bd870700 1 --2- 172.21.15.28:0/3773055259 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42b815c640 0x7f42b80b7c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.260+0000 7f42bd870700 1 --2- 172.21.15.28:0/3773055259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f42b806bf30 0x7f42b81f6660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.260+0000 7f42bd870700 1 --2- 172.21.15.28:0/3773055259 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42b80d2820 0x7f42b8152e40 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.260+0000 7f42bd870700 1 -- 172.21.15.28:0/3773055259 >> 172.21.15.28:0/3773055259 conn(0x7f42b80d2df0 msgr2=0x7f42b80b2b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:48.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.260+0000 7f42bd870700 1 -- 172.21.15.28:0/3773055259 shutdown_connections 2023-11-28T11:10:48.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.260+0000 7f42bd870700 1 -- 172.21.15.28:0/3773055259 wait complete. 2023-11-28T11:10:48.263 INFO:tasks.ceph.mds.d.smithi059.stderr: -5742> 2023-11-28T11:10:13.392+0000 7f08d132c700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:10:48.263 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:10:48.277 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:10:48.294 INFO:tasks.ceph.mds.d.smithi059.stdout:starting mds.d at 2023-11-28T11:10:48.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.414+0000 7fe7a05cc700 1 Processor -- start 2023-11-28T11:10:48.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.414+0000 7fe7a05cc700 1 -- start start 2023-11-28T11:10:48.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.415+0000 7fe7a05cc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe79806d500 0x7fe7980ca840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.415+0000 7fe7a05cc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe7980c2970 0x7fe7980cad80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.415+0000 7fe7a05cc700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7980ca0a0 con 0x7fe7981593f0 2023-11-28T11:10:48.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.415+0000 7fe7a05cc700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7980ccf40 con 0x7fe7980c2970 2023-11-28T11:10:48.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.415+0000 7fe7a05cc700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980cd220 con 0x7fe79806d500 2023-11-28T11:10:48.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.416+0000 7fe79db67700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe79806d500 0x7fe7980ca840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.416+0000 7fe79eb69700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe7980c2970 0x7fe7980cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.416+0000 7fe79eb69700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe7980c2970 0x7fe7980cad80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:37382/0 (socket says 172.21.15.28:37382) 2023-11-28T11:10:48.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.416+0000 7fe79db67700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe79806d500 0x7fe7980ca840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:55848/0 (socket says 172.21.15.28:55848) 2023-11-28T11:10:48.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.416+0000 7fe79eb69700 1 -- 172.21.15.28:0/1612560745 learned_addr learned my addr 172.21.15.28:0/1612560745 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:48.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.416+0000 7fe79db67700 1 -- 172.21.15.28:0/1612560745 >> v2:172.21.15.59:3300/0 conn(0x7fe7980c2970 msgr2=0x7fe7980cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:48.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.416+0000 7fe79db67700 1 --2- 172.21.15.28:0/1612560745 >> v2:172.21.15.59:3300/0 conn(0x7fe7980c2970 0x7fe7980cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.416+0000 7fe79db67700 1 -- 172.21.15.28:0/1612560745 >> v1:172.21.15.59:6789/0 conn(0x7fe7981593f0 legacy=0x7fe79806cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.416+0000 7fe79db67700 1 -- 172.21.15.28:0/1612560745 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7980cd500 con 0x7fe79806d500 2023-11-28T11:10:48.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.416+0000 7fe79eb69700 1 --2- 172.21.15.28:0/1612560745 >> v2:172.21.15.59:3300/0 conn(0x7fe7980c2970 0x7fe7980cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:48.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.416+0000 7fe79db67700 1 --2- 172.21.15.28:0/1612560745 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe79806d500 0x7fe7980ca840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe794009840 tx=0x7fe794009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdca1d87498cb621 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:48.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.417+0000 7fe79d366700 1 -- 172.21.15.28:0/1612560745 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794011830 con 0x7fe79806d500 2023-11-28T11:10:48.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.417+0000 7fe79d366700 1 -- 172.21.15.28:0/1612560745 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe794011990 con 0x7fe79806d500 2023-11-28T11:10:48.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.417+0000 7fe79d366700 1 -- 172.21.15.28:0/1612560745 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794011d20 con 0x7fe79806d500 2023-11-28T11:10:48.421 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.417+0000 7fe7a05cc700 1 -- 172.21.15.28:0/1612560745 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe79806d500 msgr2=0x7fe7980ca840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.417+0000 7fe7a05cc700 1 --2- 172.21.15.28:0/1612560745 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe79806d500 0x7fe7980ca840 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe794009840 tx=0x7fe794009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:48.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.417+0000 7fe7a05cc700 1 -- 172.21.15.28:0/1612560745 shutdown_connections 2023-11-28T11:10:48.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.417+0000 7fe7a05cc700 1 --2- 172.21.15.28:0/1612560745 >> v2:172.21.15.59:3300/0 conn(0x7fe7980c2970 0x7fe7980cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.417+0000 7fe7a05cc700 1 --2- 172.21.15.28:0/1612560745 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe79806d500 0x7fe7980ca840 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.417+0000 7fe7a05cc700 1 -- 172.21.15.28:0/1612560745 >> 172.21.15.28:0/1612560745 conn(0x7fe7980b1640 msgr2=0x7fe7980cee20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:48.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.417+0000 7fe7a05cc700 1 -- 172.21.15.28:0/1612560745 shutdown_connections 2023-11-28T11:10:48.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.417+0000 7fe7a05cc700 1 -- 172.21.15.28:0/1612560745 wait complete. 2023-11-28T11:10:48.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.418+0000 7fe7a05cc700 1 Processor -- start 2023-11-28T11:10:48.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.419+0000 7fe7a05cc700 1 -- start start 2023-11-28T11:10:48.422 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.419+0000 7fe7a05cc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe79806d500 0x7fe798157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.419+0000 7fe7a05cc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe7980c2970 0x7fe7981e1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.419+0000 7fe7a05cc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7981593f0 0x7fe7980c8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.419+0000 7fe7a05cc700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7980b0380 con 0x7fe79806d500 2023-11-28T11:10:48.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.419+0000 7fe7a05cc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7981e24e0 con 0x7fe7981593f0 2023-11-28T11:10:48.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.419+0000 7fe7a05cc700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7981e27c0 con 0x7fe7980c2970 2023-11-28T11:10:48.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.419+0000 7fe79e368700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe79806d500 0x7fe798157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.419+0000 7fe79e368700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe79806d500 0x7fe798157f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50094/0 (socket says 172.21.15.28:50094) 2023-11-28T11:10:48.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.419+0000 7fe79e368700 1 -- 172.21.15.28:0/2311457010 learned_addr learned my addr 172.21.15.28:0/2311457010 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:48.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.420+0000 7fe79eb69700 1 --2- 172.21.15.28:0/2311457010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7981593f0 0x7fe7980c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.420+0000 7fe79db67700 1 --2- 172.21.15.28:0/2311457010 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe7980c2970 0x7fe7981e1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.420+0000 7fe79e368700 1 -- 172.21.15.28:0/2311457010 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe7980c2970 msgr2=0x7fe7981e1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.420+0000 7fe79e368700 1 --2- 172.21.15.28:0/2311457010 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe7980c2970 0x7fe7981e1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.420+0000 7fe79e368700 1 -- 172.21.15.28:0/2311457010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7981593f0 msgr2=0x7fe7980c8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:48.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.420+0000 7fe79e368700 1 --2- 172.21.15.28:0/2311457010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7981593f0 0x7fe7980c8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:48.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.420+0000 7fe79e368700 1 -- 172.21.15.28:0/2311457010 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe790009b30 con 0x7fe79806d500 2023-11-28T11:10:48.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.420+0000 7fe79e368700 1 --2- 172.21.15.28:0/2311457010 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe79806d500 0x7fe798157f00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe78800c4e0 tx=0x7fe78800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:48.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.420+0000 7fe78effd700 1 -- 172.21.15.28:0/2311457010 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7880052d0 con 0x7fe79806d500 2023-11-28T11:10:48.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.420+0000 7fe78effd700 1 -- 172.21.15.28:0/2311457010 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe788005430 con 0x7fe79806d500 2023-11-28T11:10:48.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.420+0000 7fe7a05cc700 1 -- 172.21.15.28:0/2311457010 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7940094a0 con 0x7fe79806d500 2023-11-28T11:10:48.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.420+0000 7fe78effd700 1 -- 172.21.15.28:0/2311457010 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7880038c0 con 0x7fe79806d500 2023-11-28T11:10:48.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.420+0000 7fe7a05cc700 1 -- 172.21.15.28:0/2311457010 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe79806b9c0 con 0x7fe79806d500 2023-11-28T11:10:48.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.421+0000 7fe7a05cc700 1 -- 172.21.15.28:0/2311457010 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7980ca320 con 0x7fe79806d500 2023-11-28T11:10:48.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.422+0000 7fe78effd700 1 -- 172.21.15.28:0/2311457010 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe788019030 con 0x7fe79806d500 2023-11-28T11:10:48.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.422+0000 7fe78effd700 1 --2- 172.21.15.28:0/2311457010 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe77c0682f0 0x7fe77c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:48.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.422+0000 7fe79db67700 1 --2- 172.21.15.28:0/2311457010 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe77c0682f0 0x7fe77c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:48.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.422+0000 7fe78effd700 1 -- 172.21.15.28:0/2311457010 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6526+0+0 (secure 0 0 0) 0x7fe7880850c0 con 0x7fe79806d500 2023-11-28T11:10:48.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.423+0000 7fe79db67700 1 --2- 172.21.15.28:0/2311457010 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe77c0682f0 0x7fe77c06a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe79806d8f0 tx=0x7fe79401c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:48.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.427+0000 7fe78effd700 1 -- 172.21.15.28:0/2311457010 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe788051750 con 0x7fe79806d500 2023-11-28T11:10:48.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:48.664+0000 7fe7a05cc700 1 -- 172.21.15.28:0/2311457010 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe79806bb50 con 0x7fe79806d500 2023-11-28T11:10:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.264+0000 7fe78effd700 1 -- 172.21.15.28:0/2311457010 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v19) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe788054d70 con 0x7fe79806d500 2023-11-28T11:10:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.267+0000 7fe7a05cc700 1 -- 172.21.15.28:0/2311457010 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe77c0682f0 msgr2=0x7fe77c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.267+0000 7fe7a05cc700 1 --2- 172.21.15.28:0/2311457010 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe77c0682f0 0x7fe77c06a7a0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe79806d8f0 tx=0x7fe79401c040 comp rx=0 tx=0).stop 2023-11-28T11:10:49.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.267+0000 7fe7a05cc700 1 -- 172.21.15.28:0/2311457010 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe79806d500 msgr2=0x7fe798157f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.267+0000 7fe7a05cc700 1 --2- 172.21.15.28:0/2311457010 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe79806d500 0x7fe798157f00 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fe78800c4e0 tx=0x7fe78800c7f0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.267+0000 7fe7a05cc700 1 -- 172.21.15.28:0/2311457010 shutdown_connections 2023-11-28T11:10:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.267+0000 7fe7a05cc700 1 --2- 172.21.15.28:0/2311457010 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe77c0682f0 0x7fe77c06a7a0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.267+0000 7fe7a05cc700 1 --2- 172.21.15.28:0/2311457010 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe79806d500 0x7fe798157f00 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.267+0000 7fe7a05cc700 1 --2- 172.21.15.28:0/2311457010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe7981593f0 0x7fe7980c8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.267+0000 7fe7a05cc700 1 --2- 172.21.15.28:0/2311457010 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe7980c2970 0x7fe7981e1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.267+0000 7fe7a05cc700 1 -- 172.21.15.28:0/2311457010 >> 172.21.15.28:0/2311457010 conn(0x7fe7980b1640 msgr2=0x7fe7980c4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:49.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.267+0000 7fe7a05cc700 1 -- 172.21.15.28:0/2311457010 shutdown_connections 2023-11-28T11:10:49.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.267+0000 7fe7a05cc700 1 -- 172.21.15.28:0/2311457010 wait complete. 2023-11-28T11:10:49.282 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:10:49.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.419+0000 7faf54cea700 1 Processor -- start 2023-11-28T11:10:49.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.419+0000 7faf54cea700 1 -- start start 2023-11-28T11:10:49.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.420+0000 7faf54cea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf5006b930 0x7faf500ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.420+0000 7faf54cea700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf500cb570 con 0x7faf500cc4b0 2023-11-28T11:10:49.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.420+0000 7faf54cea700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf500cc110 con 0x7faf5015a5a0 2023-11-28T11:10:49.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.420+0000 7faf54cea700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5006bda0 con 0x7faf5006b930 2023-11-28T11:10:49.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.420+0000 7faf4ed9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7faf500cc4b0 0x7faf500c35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38164/0 (socket says 172.21.15.28:38164) 2023-11-28T11:10:49.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.420+0000 7faf4ed9d700 1 -- 172.21.15.28:0/1275777137 learned_addr learned my addr 172.21.15.28:0/1275777137 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:49.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.420+0000 7faf4dd9b700 1 --2- 172.21.15.28:0/1275777137 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf5006b930 0x7faf500ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1309929526 0 0) 0x7faf500cb570 con 0x7faf500cc4b0 2023-11-28T11:10:49.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf380035b0 con 0x7faf500cc4b0 2023-11-28T11:10:49.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1836455499 0 0) 0x7faf500cc110 con 0x7faf5015a5a0 2023-11-28T11:10:49.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf500cb570 con 0x7faf5015a5a0 2023-11-28T11:10:49.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2377157824 0 0) 0x7faf380035b0 con 0x7faf500cc4b0 2023-11-28T11:10:49.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf5006b930 msgr2=0x7faf500ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:49.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 --2- 172.21.15.28:0/1275777137 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf5006b930 0x7faf500ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 >> v1:172.21.15.59:6789/0 conn(0x7faf5015a5a0 legacy=0x7faf500cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500ca960 con 0x7faf500cc4b0 2023-11-28T11:10:49.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7faf480019f0 con 0x7faf500cc4b0 2023-11-28T11:10:49.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 >> v1:172.21.15.28:6789/0 conn(0x7faf500cc4b0 legacy=0x7faf500c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 --2- 172.21.15.28:0/1275777137 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf5006b930 0x7faf38003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 --2- 172.21.15.28:0/1275777137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf38004120 0x7faf38004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 --2- 172.21.15.28:0/1275777137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf38004d90 0x7faf38005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5006bda0 con 0x7faf38004120 2023-11-28T11:10:49.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf3800bac0 con 0x7faf38004d90 2023-11-28T11:10:49.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3800bc80 con 0x7faf5006b930 2023-11-28T11:10:49.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4dd9b700 1 --2- 172.21.15.28:0/1275777137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf38004120 0x7faf38004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4ed9d700 1 --2- 172.21.15.28:0/1275777137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf38004d90 0x7faf38005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4e59c700 1 --2- 172.21.15.28:0/1275777137 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf5006b930 0x7faf38003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4dd9b700 1 -- 172.21.15.28:0/1275777137 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf5006b930 msgr2=0x7faf38003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:49.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4dd9b700 1 --2- 172.21.15.28:0/1275777137 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf5006b930 0x7faf38003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4dd9b700 1 -- 172.21.15.28:0/1275777137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf38004d90 msgr2=0x7faf38005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:49.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4dd9b700 1 --2- 172.21.15.28:0/1275777137 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf38004d90 0x7faf38005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.421+0000 7faf4dd9b700 1 -- 172.21.15.28:0/1275777137 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380035b0 con 0x7faf38004120 2023-11-28T11:10:49.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.422+0000 7faf4dd9b700 1 --2- 172.21.15.28:0/1275777137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf38004120 0x7faf38004850 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7faf44009b30 tx=0x7faf38005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acaa68899cc3528c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:49.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.422+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4400e6c0 con 0x7faf38004120 2023-11-28T11:10:49.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.422+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4400e820 con 0x7faf38004120 2023-11-28T11:10:49.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.422+0000 7faf4d59a700 1 -- 172.21.15.28:0/1275777137 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4400eb90 con 0x7faf38004120 2023-11-28T11:10:49.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.422+0000 7faf54cea700 1 -- 172.21.15.28:0/1275777137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf38004120 msgr2=0x7faf38004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.422+0000 7faf54cea700 1 --2- 172.21.15.28:0/1275777137 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf38004120 0x7faf38004850 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7faf44009b30 tx=0x7faf38005880 comp rx=0 tx=0).stop 2023-11-28T11:10:49.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.422+0000 7faf4ed9d700 1 -- 172.21.15.28:0/1275777137 reap_dead start 2023-11-28T11:10:49.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.422+0000 7faf54cea700 1 -- 172.21.15.28:0/1275777137 shutdown_connections 2023-11-28T11:10:49.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.422+0000 7faf54cea700 1 -- 172.21.15.28:0/1275777137 >> 172.21.15.28:0/1275777137 conn(0x7faf500b1650 msgr2=0x7faf500b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:49.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.422+0000 7faf54cea700 1 -- 172.21.15.28:0/1275777137 shutdown_connections 2023-11-28T11:10:49.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.422+0000 7faf54cea700 1 -- 172.21.15.28:0/1275777137 wait complete. 2023-11-28T11:10:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.424+0000 7faf54cea700 1 Processor -- start 2023-11-28T11:10:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.424+0000 7faf54cea700 1 -- start start 2023-11-28T11:10:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.424+0000 7faf54cea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf38004120 0x7faf501f10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.424+0000 7faf54cea700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf500c5850 0x7faf5015b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.424+0000 7faf54cea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf5015bf00 0x7faf50147ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.424+0000 7faf54cea700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5015c9b0 con 0x7faf500c5850 2023-11-28T11:10:49.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.424+0000 7faf54cea700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf5015c3f0 con 0x7faf38004120 2023-11-28T11:10:49.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.424+0000 7faf54cea700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf5015c6d0 con 0x7faf5015bf00 2023-11-28T11:10:49.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf4dd9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf500c5850 0x7faf5015b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf4dd9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf500c5850 0x7faf5015b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50118/0 (socket says 172.21.15.28:50118) 2023-11-28T11:10:49.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf4dd9b700 1 -- 172.21.15.28:0/2830087566 learned_addr learned my addr 172.21.15.28:0/2830087566 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:49.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf4ed9d700 1 --2- 172.21.15.28:0/2830087566 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf5015bf00 0x7faf50147ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf4e59c700 1 --2- 172.21.15.28:0/2830087566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf38004120 0x7faf501f10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf4dd9b700 1 -- 172.21.15.28:0/2830087566 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf5015bf00 msgr2=0x7faf50147ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf4dd9b700 1 --2- 172.21.15.28:0/2830087566 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf5015bf00 0x7faf50147ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf4dd9b700 1 -- 172.21.15.28:0/2830087566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf38004120 msgr2=0x7faf501f10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:49.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf4dd9b700 1 --2- 172.21.15.28:0/2830087566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf38004120 0x7faf501f10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:49.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf4dd9b700 1 -- 172.21.15.28:0/2830087566 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf500caf20 con 0x7faf500c5850 2023-11-28T11:10:49.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf4e59c700 1 --2- 172.21.15.28:0/2830087566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf38004120 0x7faf501f10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:49.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf4dd9b700 1 --2- 172.21.15.28:0/2830087566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf500c5850 0x7faf5015b9c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7faf380047d0 tx=0x7faf440047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:49.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf3effd700 1 -- 172.21.15.28:0/2830087566 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4400e6c0 con 0x7faf500c5850 2023-11-28T11:10:49.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf3effd700 1 -- 172.21.15.28:0/2830087566 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf44003ad0 con 0x7faf500c5850 2023-11-28T11:10:49.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf54cea700 1 -- 172.21.15.28:0/2830087566 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf500cac40 con 0x7faf500c5850 2023-11-28T11:10:49.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf3effd700 1 -- 172.21.15.28:0/2830087566 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44003e80 con 0x7faf500c5850 2023-11-28T11:10:49.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.425+0000 7faf54cea700 1 -- 172.21.15.28:0/2830087566 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf500cb200 con 0x7faf500c5850 2023-11-28T11:10:49.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.426+0000 7faf54cea700 1 -- 172.21.15.28:0/2830087566 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf5015c910 con 0x7faf500c5850 2023-11-28T11:10:49.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.427+0000 7faf3effd700 1 -- 172.21.15.28:0/2830087566 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7faf44025070 con 0x7faf500c5850 2023-11-28T11:10:49.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.427+0000 7faf3effd700 1 --2- 172.21.15.28:0/2830087566 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faf300682f0 0x7faf3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:49.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.427+0000 7faf3effd700 1 -- 172.21.15.28:0/2830087566 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6526+0+0 (secure 0 0 0) 0x7faf44087690 con 0x7faf500c5850 2023-11-28T11:10:49.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.427+0000 7faf4e59c700 1 --2- 172.21.15.28:0/2830087566 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faf300682f0 0x7faf3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:49.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.428+0000 7faf4e59c700 1 --2- 172.21.15.28:0/2830087566 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faf300682f0 0x7faf3006a7a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faf40007fd0 tx=0x7faf40009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:49.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.431+0000 7faf3effd700 1 -- 172.21.15.28:0/2830087566 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7faf44053d20 con 0x7faf500c5850 2023-11-28T11:10:49.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:49.625+0000 7faf54cea700 1 -- 172.21.15.28:0/2830087566 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faf500739f0 con 0x7faf500c5850 2023-11-28T11:10:50.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.316+0000 7faf3effd700 1 -- 172.21.15.28:0/2830087566 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v44) v1 ==== 181+0+0 (secure 0 0 0) 0x7faf44057340 con 0x7faf500c5850 2023-11-28T11:10:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.321+0000 7faf54cea700 1 -- 172.21.15.28:0/2830087566 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faf300682f0 msgr2=0x7faf3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.321+0000 7faf54cea700 1 --2- 172.21.15.28:0/2830087566 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faf300682f0 0x7faf3006a7a0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7faf40007fd0 tx=0x7faf40009040 comp rx=0 tx=0).stop 2023-11-28T11:10:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.321+0000 7faf54cea700 1 -- 172.21.15.28:0/2830087566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf500c5850 msgr2=0x7faf5015b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.321+0000 7faf54cea700 1 --2- 172.21.15.28:0/2830087566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf500c5850 0x7faf5015b9c0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7faf380047d0 tx=0x7faf440047c0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.321+0000 7faf54cea700 1 -- 172.21.15.28:0/2830087566 shutdown_connections 2023-11-28T11:10:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.321+0000 7faf54cea700 1 --2- 172.21.15.28:0/2830087566 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7faf300682f0 0x7faf3006a7a0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.321+0000 7faf54cea700 1 --2- 172.21.15.28:0/2830087566 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7faf5015bf00 0x7faf50147ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.321+0000 7faf54cea700 1 --2- 172.21.15.28:0/2830087566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf38004120 0x7faf501f10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.321+0000 7faf54cea700 1 --2- 172.21.15.28:0/2830087566 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7faf500c5850 0x7faf5015b9c0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.321+0000 7faf54cea700 1 -- 172.21.15.28:0/2830087566 >> 172.21.15.28:0/2830087566 conn(0x7faf500b1650 msgr2=0x7faf50159040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:50.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.321+0000 7faf54cea700 1 -- 172.21.15.28:0/2830087566 shutdown_connections 2023-11-28T11:10:50.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.321+0000 7faf54cea700 1 -- 172.21.15.28:0/2830087566 wait complete. 2023-11-28T11:10:50.323 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:10:50.338 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:10:50.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.477+0000 7fcb39df5700 1 Processor -- start 2023-11-28T11:10:50.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.477+0000 7fcb39df5700 1 -- start start 2023-11-28T11:10:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.477+0000 7fcb39df5700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcb34159390 0x7fcb3415d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.477+0000 7fcb39df5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcb340cdf80 0x7fcb340ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.477+0000 7fcb39df5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340cea10 0x7fcb340b1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.477+0000 7fcb39df5700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb34159760 con 0x7fcb34159390 2023-11-28T11:10:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.477+0000 7fcb39df5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb340cee50 con 0x7fcb340cdf80 2023-11-28T11:10:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.477+0000 7fcb39df5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3406c3a0 con 0x7fcb340cea10 2023-11-28T11:10:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb337fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcb34159390 0x7fcb3415d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb337fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcb34159390 0x7fcb3415d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50124/0 (socket says 172.21.15.28:50124) 2023-11-28T11:10:50.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb337fe700 1 -- 172.21.15.28:0/3889554774 learned_addr learned my addr 172.21.15.28:0/3889554774 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb32ffd700 1 --2- 172.21.15.28:0/3889554774 >> v2:172.21.15.59:3300/0 conn(0x7fcb340cdf80 0x7fcb340ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb33fff700 1 --2- 172.21.15.28:0/3889554774 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340cea10 0x7fcb340b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb337fe700 1 -- 172.21.15.28:0/3889554774 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340cea10 msgr2=0x7fcb340b1530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb337fe700 1 --2- 172.21.15.28:0/3889554774 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340cea10 0x7fcb340b1530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb337fe700 1 -- 172.21.15.28:0/3889554774 >> v2:172.21.15.59:3300/0 conn(0x7fcb340cdf80 msgr2=0x7fcb340ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb337fe700 1 --2- 172.21.15.28:0/3889554774 >> v2:172.21.15.59:3300/0 conn(0x7fcb340cdf80 0x7fcb340ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb337fe700 1 -- 172.21.15.28:0/3889554774 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3415dd50 con 0x7fcb34159390 2023-11-28T11:10:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb32ffd700 1 --2- 172.21.15.28:0/3889554774 >> v2:172.21.15.59:3300/0 conn(0x7fcb340cdf80 0x7fcb340ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb33fff700 1 --2- 172.21.15.28:0/3889554774 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340cea10 0x7fcb340b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb337fe700 1 --2- 172.21.15.28:0/3889554774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb34159390 0x7fcb3415d810 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcb340cf9f0 tx=0x7fcb2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1802bd9c57d23f05 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb327fc700 1 -- 172.21.15.28:0/3889554774 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2400e070 con 0x7fcb34159390 2023-11-28T11:10:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb327fc700 1 -- 172.21.15.28:0/3889554774 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb2400b9c0 con 0x7fcb34159390 2023-11-28T11:10:50.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.478+0000 7fcb327fc700 1 -- 172.21.15.28:0/3889554774 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb2400bd70 con 0x7fcb34159390 2023-11-28T11:10:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.479+0000 7fcb39df5700 1 -- 172.21.15.28:0/3889554774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb34159390 msgr2=0x7fcb3415d810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.479+0000 7fcb39df5700 1 --2- 172.21.15.28:0/3889554774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb34159390 0x7fcb3415d810 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fcb340cf9f0 tx=0x7fcb2400b330 comp rx=0 tx=0).stop 2023-11-28T11:10:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.479+0000 7fcb39df5700 1 -- 172.21.15.28:0/3889554774 shutdown_connections 2023-11-28T11:10:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.479+0000 7fcb39df5700 1 --2- 172.21.15.28:0/3889554774 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340cea10 0x7fcb340b1530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.479+0000 7fcb39df5700 1 --2- 172.21.15.28:0/3889554774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb34159390 0x7fcb3415d810 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.479+0000 7fcb39df5700 1 --2- 172.21.15.28:0/3889554774 >> v2:172.21.15.59:3300/0 conn(0x7fcb340cdf80 0x7fcb340ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.479+0000 7fcb39df5700 1 -- 172.21.15.28:0/3889554774 >> 172.21.15.28:0/3889554774 conn(0x7fcb3406dae0 msgr2=0x7fcb3406dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.479+0000 7fcb39df5700 1 -- 172.21.15.28:0/3889554774 shutdown_connections 2023-11-28T11:10:50.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.479+0000 7fcb39df5700 1 -- 172.21.15.28:0/3889554774 wait complete. 2023-11-28T11:10:50.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.480+0000 7fcb39df5700 1 Processor -- start 2023-11-28T11:10:50.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.480+0000 7fcb39df5700 1 -- start start 2023-11-28T11:10:50.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.481+0000 7fcb39df5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340cdf80 0x7fcb341f0480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:50.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.481+0000 7fcb39df5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb340cea10 0x7fcb340c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:50.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.481+0000 7fcb39df5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb34159390 0x7fcb341e4780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:50.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.481+0000 7fcb39df5700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340af410 con 0x7fcb340cea10 2023-11-28T11:10:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.481+0000 7fcb39df5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340af620 con 0x7fcb34159390 2023-11-28T11:10:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.481+0000 7fcb39df5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb340af900 con 0x7fcb340cdf80 2023-11-28T11:10:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.481+0000 7fcb32ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb340cea10 0x7fcb340c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.481+0000 7fcb32ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb340cea10 0x7fcb340c9670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50130/0 (socket says 172.21.15.28:50130) 2023-11-28T11:10:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.481+0000 7fcb32ffd700 1 -- 172.21.15.28:0/1867064846 learned_addr learned my addr 172.21.15.28:0/1867064846 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.481+0000 7fcb33fff700 1 --2- 172.21.15.28:0/1867064846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb34159390 0x7fcb341e4780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.481+0000 7fcb337fe700 1 --2- 172.21.15.28:0/1867064846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340cdf80 0x7fcb341f0480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.482+0000 7fcb32ffd700 1 -- 172.21.15.28:0/1867064846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340cdf80 msgr2=0x7fcb341f0480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.482+0000 7fcb32ffd700 1 --2- 172.21.15.28:0/1867064846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340cdf80 0x7fcb341f0480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.482+0000 7fcb32ffd700 1 -- 172.21.15.28:0/1867064846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb34159390 msgr2=0x7fcb341e4780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.482+0000 7fcb32ffd700 1 --2- 172.21.15.28:0/1867064846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb34159390 0x7fcb341e4780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:50.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.482+0000 7fcb32ffd700 1 -- 172.21.15.28:0/1867064846 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb24009b30 con 0x7fcb340cea10 2023-11-28T11:10:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.482+0000 7fcb337fe700 1 --2- 172.21.15.28:0/1867064846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340cdf80 0x7fcb341f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.482+0000 7fcb32ffd700 1 --2- 172.21.15.28:0/1867064846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb340cea10 0x7fcb340c9670 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800d800 tx=0x7fcb2800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.482+0000 7fcb1bfff700 1 -- 172.21.15.28:0/1867064846 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28004030 con 0x7fcb340cea10 2023-11-28T11:10:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.482+0000 7fcb1bfff700 1 -- 172.21.15.28:0/1867064846 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb28004190 con 0x7fcb340cea10 2023-11-28T11:10:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.482+0000 7fcb39df5700 1 -- 172.21.15.28:0/1867064846 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3406c920 con 0x7fcb340cea10 2023-11-28T11:10:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.482+0000 7fcb1bfff700 1 -- 172.21.15.28:0/1867064846 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb28004520 con 0x7fcb340cea10 2023-11-28T11:10:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.482+0000 7fcb39df5700 1 -- 172.21.15.28:0/1867064846 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3406c640 con 0x7fcb340cea10 2023-11-28T11:10:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.483+0000 7fcb39df5700 1 -- 172.21.15.28:0/1867064846 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3406cc00 con 0x7fcb340cea10 2023-11-28T11:10:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.483+0000 7fcb1bfff700 1 -- 172.21.15.28:0/1867064846 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcb2801b030 con 0x7fcb340cea10 2023-11-28T11:10:50.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.484+0000 7fcb1bfff700 1 --2- 172.21.15.28:0/1867064846 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcb200682f0 0x7fcb2006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:50.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.484+0000 7fcb1bfff700 1 -- 172.21.15.28:0/1867064846 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6088+0+0 (secure 0 0 0) 0x7fcb280853d0 con 0x7fcb340cea10 2023-11-28T11:10:50.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.484+0000 7fcb337fe700 1 --2- 172.21.15.28:0/1867064846 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcb200682f0 0x7fcb2006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:50.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.484+0000 7fcb337fe700 1 --2- 172.21.15.28:0/1867064846 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcb200682f0 0x7fcb2006a7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcb24000f80 tx=0x7fcb24009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:50.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.488+0000 7fcb1bfff700 1 -- 172.21.15.28:0/1867064846 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcb2804d5e0 con 0x7fcb340cea10 2023-11-28T11:10:50.686 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:50.686+0000 7fcb39df5700 1 -- 172.21.15.28:0/1867064846 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fcb340af280 con 0x7fcb340cea10 2023-11-28T11:10:51.132 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:10:51.135 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:10:51.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.323+0000 7fcb1bfff700 1 -- 172.21.15.28:0/1867064846 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v45) v1 ==== 169+0+0 (secure 0 0 0) 0x7fcb28051cc0 con 0x7fcb340cea10 2023-11-28T11:10:51.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.327+0000 7fcb39df5700 1 -- 172.21.15.28:0/1867064846 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcb200682f0 msgr2=0x7fcb2006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.327+0000 7fcb39df5700 1 --2- 172.21.15.28:0/1867064846 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcb200682f0 0x7fcb2006a7a0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fcb24000f80 tx=0x7fcb24009000 comp rx=0 tx=0).stop 2023-11-28T11:10:51.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.327+0000 7fcb39df5700 1 -- 172.21.15.28:0/1867064846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb340cea10 msgr2=0x7fcb340c9670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.327+0000 7fcb39df5700 1 --2- 172.21.15.28:0/1867064846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb340cea10 0x7fcb340c9670 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fcb2800d800 tx=0x7fcb2800dbc0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.329 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.327+0000 7fcb39df5700 1 -- 172.21.15.28:0/1867064846 shutdown_connections 2023-11-28T11:10:51.329 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.327+0000 7fcb39df5700 1 --2- 172.21.15.28:0/1867064846 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcb200682f0 0x7fcb2006a7a0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.329 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.327+0000 7fcb39df5700 1 --2- 172.21.15.28:0/1867064846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcb34159390 0x7fcb341e4780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.329 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.327+0000 7fcb39df5700 1 --2- 172.21.15.28:0/1867064846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcb340cdf80 0x7fcb341f0480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.329 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.327+0000 7fcb39df5700 1 --2- 172.21.15.28:0/1867064846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcb340cea10 0x7fcb340c9670 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.329 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.327+0000 7fcb39df5700 1 -- 172.21.15.28:0/1867064846 >> 172.21.15.28:0/1867064846 conn(0x7fcb3406dae0 msgr2=0x7fcb340cc4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:51.329 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.328+0000 7fcb39df5700 1 -- 172.21.15.28:0/1867064846 shutdown_connections 2023-11-28T11:10:51.329 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.328+0000 7fcb39df5700 1 -- 172.21.15.28:0/1867064846 wait complete. 2023-11-28T11:10:51.330 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' removed 2023-11-28T11:10:51.342 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:10:51.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.473+0000 7fe5cb39e700 1 Processor -- start 2023-11-28T11:10:51.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.473+0000 7fe5cb39e700 1 -- start start 2023-11-28T11:10:51.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5cb39e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe5c40cf5f0 0x7fe5c40caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5cb39e700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c40ce550 con 0x7fe5c40b1630 2023-11-28T11:10:51.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5cb39e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5c40ce830 con 0x7fe5c40cb3e0 2023-11-28T11:10:51.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5cb39e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5c40ceb10 con 0x7fe5c40cf5f0 2023-11-28T11:10:51.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5c913a700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe5c40b1630 0x7fe5c40b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38194/0 (socket says 172.21.15.28:38194) 2023-11-28T11:10:51.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5c913a700 1 -- 172.21.15.28:0/2510870621 learned_addr learned my addr 172.21.15.28:0/2510870621 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:51.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5bbfff700 1 -- 172.21.15.28:0/2510870621 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863705815 0 0) 0x7fe5c40ce550 con 0x7fe5c40b1630 2023-11-28T11:10:51.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5bbfff700 1 -- 172.21.15.28:0/2510870621 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5b00035b0 con 0x7fe5c40b1630 2023-11-28T11:10:51.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5c8939700 1 --2- 172.21.15.28:0/2510870621 >> v2:172.21.15.59:3300/0 conn(0x7fe5c40cf5f0 0x7fe5c40caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5bbfff700 1 -- 172.21.15.28:0/2510870621 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2787289766 0 0) 0x7fe5b00035b0 con 0x7fe5c40b1630 2023-11-28T11:10:51.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5bbfff700 1 -- 172.21.15.28:0/2510870621 >> v2:172.21.15.59:3300/0 conn(0x7fe5c40cf5f0 msgr2=0x7fe5c40caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:51.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5bbfff700 1 --2- 172.21.15.28:0/2510870621 >> v2:172.21.15.59:3300/0 conn(0x7fe5c40cf5f0 0x7fe5c40caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5bbfff700 1 -- 172.21.15.28:0/2510870621 >> v1:172.21.15.59:6789/0 conn(0x7fe5c40cb3e0 legacy=0x7fe5c406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5bbfff700 1 -- 172.21.15.28:0/2510870621 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c40cedf0 con 0x7fe5c40b1630 2023-11-28T11:10:51.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5bbfff700 1 -- 172.21.15.28:0/2510870621 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fe5b4001670 con 0x7fe5c40b1630 2023-11-28T11:10:51.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5c8939700 1 --2- 172.21.15.28:0/2510870621 >> v2:172.21.15.59:3300/0 conn(0x7fe5c40cf5f0 0x7fe5c40caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:51.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5bbfff700 1 -- 172.21.15.28:0/2510870621 >> v1:172.21.15.28:6789/0 conn(0x7fe5c40b1630 legacy=0x7fe5c40b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5bbfff700 1 --2- 172.21.15.28:0/2510870621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5b0003a30 0x7fe5b0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5bbfff700 1 --2- 172.21.15.28:0/2510870621 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5b0004850 0x7fe5b0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5bbfff700 1 --2- 172.21.15.28:0/2510870621 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b0005250 0x7fe5b000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5bbfff700 1 -- 172.21.15.28:0/2510870621 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40ceb10 con 0x7fe5b0005250 2023-11-28T11:10:51.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5bbfff700 1 -- 172.21.15.28:0/2510870621 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b000c160 con 0x7fe5b0003a30 2023-11-28T11:10:51.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.474+0000 7fe5bbfff700 1 -- 172.21.15.28:0/2510870621 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5b000c320 con 0x7fe5b0004850 2023-11-28T11:10:51.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.475+0000 7fe5c993b700 1 --2- 172.21.15.28:0/2510870621 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b0005250 0x7fe5b000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.475+0000 7fe5c913a700 1 --2- 172.21.15.28:0/2510870621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5b0003a30 0x7fe5b0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.475+0000 7fe5c8939700 1 --2- 172.21.15.28:0/2510870621 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5b0004850 0x7fe5b0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.475+0000 7fe5c993b700 1 -- 172.21.15.28:0/2510870621 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5b0004850 msgr2=0x7fe5b0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:51.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.475+0000 7fe5c993b700 1 --2- 172.21.15.28:0/2510870621 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5b0004850 0x7fe5b0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.475+0000 7fe5c993b700 1 -- 172.21.15.28:0/2510870621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5b0003a30 msgr2=0x7fe5b0003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:51.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.475+0000 7fe5c993b700 1 --2- 172.21.15.28:0/2510870621 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5b0003a30 0x7fe5b0003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.475+0000 7fe5c993b700 1 -- 172.21.15.28:0/2510870621 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5b00035b0 con 0x7fe5b0005250 2023-11-28T11:10:51.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.475+0000 7fe5c993b700 1 -- 172.21.15.28:0/2510870621 reap_dead start 2023-11-28T11:10:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.475+0000 7fe5c993b700 1 --2- 172.21.15.28:0/2510870621 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b0005250 0x7fe5b000ba10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00c190 tx=0x7fe5b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c8970709446bfc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.475+0000 7fe5bbfff700 1 -- 172.21.15.28:0/2510870621 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc001db0 con 0x7fe5b0005250 2023-11-28T11:10:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.475+0000 7fe5bbfff700 1 -- 172.21.15.28:0/2510870621 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc00ccf0 con 0x7fe5b0005250 2023-11-28T11:10:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.475+0000 7fe5bbfff700 1 -- 172.21.15.28:0/2510870621 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc003c40 con 0x7fe5b0005250 2023-11-28T11:10:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.476+0000 7fe5cb39e700 1 -- 172.21.15.28:0/2510870621 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b0005250 msgr2=0x7fe5b000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.476+0000 7fe5cb39e700 1 --2- 172.21.15.28:0/2510870621 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b0005250 0x7fe5b000ba10 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc00c190 tx=0x7fe5b0005880 comp rx=0 tx=0).stop 2023-11-28T11:10:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.476+0000 7fe5cb39e700 1 -- 172.21.15.28:0/2510870621 shutdown_connections 2023-11-28T11:10:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.476+0000 7fe5cb39e700 1 --2- 172.21.15.28:0/2510870621 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5b0005250 0x7fe5b000ba10 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.476+0000 7fe5cb39e700 1 -- 172.21.15.28:0/2510870621 >> 172.21.15.28:0/2510870621 conn(0x7fe5c40cc630 msgr2=0x7fe5c40cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.476+0000 7fe5cb39e700 1 -- 172.21.15.28:0/2510870621 shutdown_connections 2023-11-28T11:10:51.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.476+0000 7fe5cb39e700 1 -- 172.21.15.28:0/2510870621 wait complete. 2023-11-28T11:10:51.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.477+0000 7fe5cb39e700 1 Processor -- start 2023-11-28T11:10:51.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.477+0000 7fe5cb39e700 1 -- start start 2023-11-28T11:10:51.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5cb39e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5b0005250 0x7fe5c4156d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5cb39e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5c40b4c00 0x7fe5c40c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5cb39e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5c40c60c0 0x7fe5c40c7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5cb39e700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40b5a30 con 0x7fe5c40b4c00 2023-11-28T11:10:51.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5cb39e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c4157630 con 0x7fe5b0005250 2023-11-28T11:10:51.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5cb39e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5c40c6530 con 0x7fe5c40c60c0 2023-11-28T11:10:51.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5c8939700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5c40b4c00 0x7fe5c40c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5c8939700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5c40b4c00 0x7fe5c40c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50150/0 (socket says 172.21.15.28:50150) 2023-11-28T11:10:51.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5c8939700 1 -- 172.21.15.28:0/4278375090 learned_addr learned my addr 172.21.15.28:0/4278375090 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:51.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5c993b700 1 --2- 172.21.15.28:0/4278375090 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5c40c60c0 0x7fe5c40c7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5c913a700 1 --2- 172.21.15.28:0/4278375090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5b0005250 0x7fe5c4156d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5c8939700 1 -- 172.21.15.28:0/4278375090 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5c40c60c0 msgr2=0x7fe5c40c7f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5c8939700 1 --2- 172.21.15.28:0/4278375090 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5c40c60c0 0x7fe5c40c7f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5c8939700 1 -- 172.21.15.28:0/4278375090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5b0005250 msgr2=0x7fe5c4156d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5c8939700 1 --2- 172.21.15.28:0/4278375090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5b0005250 0x7fe5c4156d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5c8939700 1 -- 172.21.15.28:0/4278375090 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c40ce140 con 0x7fe5c40b4c00 2023-11-28T11:10:51.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5c993b700 1 --2- 172.21.15.28:0/4278375090 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5c40c60c0 0x7fe5c40c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:51.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5c8939700 1 --2- 172.21.15.28:0/4278375090 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5c40b4c00 0x7fe5c40c5b80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00096b0 tx=0x7fe5c0009a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:51.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5b9ffb700 1 -- 172.21.15.28:0/4278375090 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0015880 con 0x7fe5c40b4c00 2023-11-28T11:10:51.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5b9ffb700 1 -- 172.21.15.28:0/4278375090 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c00159e0 con 0x7fe5c40b4c00 2023-11-28T11:10:51.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5b9ffb700 1 -- 172.21.15.28:0/4278375090 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0015d70 con 0x7fe5c40b4c00 2023-11-28T11:10:51.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.478+0000 7fe5cb39e700 1 -- 172.21.15.28:0/4278375090 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5c40cf0d0 con 0x7fe5c40b4c00 2023-11-28T11:10:51.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.479+0000 7fe5cb39e700 1 -- 172.21.15.28:0/4278375090 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5c40ce2d0 con 0x7fe5c40b4c00 2023-11-28T11:10:51.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.479+0000 7fe5cb39e700 1 -- 172.21.15.28:0/4278375090 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5c40c7d90 con 0x7fe5c40b4c00 2023-11-28T11:10:51.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.480+0000 7fe5b9ffb700 1 -- 172.21.15.28:0/4278375090 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe5c0025070 con 0x7fe5c40b4c00 2023-11-28T11:10:51.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.480+0000 7fe5b9ffb700 1 --2- 172.21.15.28:0/4278375090 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe5a80682f0 0x7fe5a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.480+0000 7fe5b9ffb700 1 -- 172.21.15.28:0/4278375090 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fe5c0086090 con 0x7fe5c40b4c00 2023-11-28T11:10:51.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.480+0000 7fe5c913a700 1 --2- 172.21.15.28:0/4278375090 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe5a80682f0 0x7fe5a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.481+0000 7fe5c913a700 1 --2- 172.21.15.28:0/4278375090 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe5a80682f0 0x7fe5a806a7a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4001c90 tx=0x7fe5b4001b20 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:51.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.485+0000 7fe5b9ffb700 1 -- 172.21.15.28:0/4278375090 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe5c0052ab0 con 0x7fe5c40b4c00 2023-11-28T11:10:51.679 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.678+0000 7fe5cb39e700 1 -- 172.21.15.28:0/4278375090 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fe5c40739f0 con 0x7fe5c40b4c00 2023-11-28T11:10:51.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.682+0000 7fe5b9ffb700 1 -- 172.21.15.28:0/4278375090 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v11) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe5c00560d0 con 0x7fe5c40b4c00 2023-11-28T11:10:51.685 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.684+0000 7fe5cb39e700 1 -- 172.21.15.28:0/4278375090 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe5a80682f0 msgr2=0x7fe5a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.685 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.684+0000 7fe5cb39e700 1 --2- 172.21.15.28:0/4278375090 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe5a80682f0 0x7fe5a806a7a0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fe5b4001c90 tx=0x7fe5b4001b20 comp rx=0 tx=0).stop 2023-11-28T11:10:51.685 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.684+0000 7fe5cb39e700 1 -- 172.21.15.28:0/4278375090 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5c40b4c00 msgr2=0x7fe5c40c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.685 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.684+0000 7fe5cb39e700 1 --2- 172.21.15.28:0/4278375090 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5c40b4c00 0x7fe5c40c5b80 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fe5c00096b0 tx=0x7fe5c0009a70 comp rx=0 tx=0).stop 2023-11-28T11:10:51.685 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.684+0000 7fe5cb39e700 1 -- 172.21.15.28:0/4278375090 shutdown_connections 2023-11-28T11:10:51.685 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.684+0000 7fe5cb39e700 1 --2- 172.21.15.28:0/4278375090 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe5a80682f0 0x7fe5a806a7a0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.686 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.684+0000 7fe5cb39e700 1 --2- 172.21.15.28:0/4278375090 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe5c40c60c0 0x7fe5c40c7f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.686 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.684+0000 7fe5cb39e700 1 --2- 172.21.15.28:0/4278375090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe5b0005250 0x7fe5c4156d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.686 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.684+0000 7fe5cb39e700 1 --2- 172.21.15.28:0/4278375090 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe5c40b4c00 0x7fe5c40c5b80 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.686 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.684+0000 7fe5cb39e700 1 -- 172.21.15.28:0/4278375090 >> 172.21.15.28:0/4278375090 conn(0x7fe5c40cc630 msgr2=0x7fe5c40ae040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:51.686 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.684+0000 7fe5cb39e700 1 -- 172.21.15.28:0/4278375090 shutdown_connections 2023-11-28T11:10:51.686 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.684+0000 7fe5cb39e700 1 -- 172.21.15.28:0/4278375090 wait complete. 2023-11-28T11:10:51.697 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag' 2023-11-28T11:10:51.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.821+0000 7f3fddb40700 1 Processor -- start 2023-11-28T11:10:51.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.821+0000 7f3fddb40700 1 -- start start 2023-11-28T11:10:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.822+0000 7f3fddb40700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3fd806aa70 0x7f3fd80ce860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.822+0000 7f3fddb40700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd806ed80 con 0x7f3fd8158a30 2023-11-28T11:10:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.822+0000 7f3fddb40700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd806f060 con 0x7f3fd815c680 2023-11-28T11:10:51.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.822+0000 7f3fddb40700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd80cd950 con 0x7f3fd806aa70 2023-11-28T11:10:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.822+0000 7f3fd6ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3fd806aa70 0x7f3fd80ce860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.822+0000 7f3fd77fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3fd8158a30 0x7f3fd80ce150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38214/0 (socket says 172.21.15.28:38214) 2023-11-28T11:10:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.822+0000 7f3fd77fe700 1 -- 172.21.15.28:0/1711576004 learned_addr learned my addr 172.21.15.28:0/1711576004 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.822+0000 7f3fd67fc700 1 -- 172.21.15.28:0/1711576004 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2531904729 0 0) 0x7f3fd806ed80 con 0x7f3fd8158a30 2023-11-28T11:10:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.823+0000 7f3fd67fc700 1 -- 172.21.15.28:0/1711576004 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3fc40035b0 con 0x7f3fd8158a30 2023-11-28T11:10:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.823+0000 7f3fd6ffd700 1 -- 172.21.15.28:0/1711576004 >> v1:172.21.15.59:6789/0 conn(0x7f3fd815c680 legacy=0x7f3fd815cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.823+0000 7f3fd6ffd700 1 -- 172.21.15.28:0/1711576004 >> v1:172.21.15.28:6789/0 conn(0x7f3fd8158a30 legacy=0x7f3fd80ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.823+0000 7f3fd6ffd700 1 -- 172.21.15.28:0/1711576004 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd80cdc30 con 0x7f3fd806aa70 2023-11-28T11:10:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.823+0000 7f3fd6ffd700 1 --2- 172.21.15.28:0/1711576004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fd806aa70 0x7f3fd80ce860 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000a9e0 tx=0x7f3fc000acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1339cd0922a134f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:51.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.823+0000 7f3fd67fc700 1 -- 172.21.15.28:0/1711576004 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc000cb70 con 0x7f3fd806aa70 2023-11-28T11:10:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.823+0000 7f3fd67fc700 1 -- 172.21.15.28:0/1711576004 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc000ccd0 con 0x7f3fd806aa70 2023-11-28T11:10:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.823+0000 7f3fd67fc700 1 -- 172.21.15.28:0/1711576004 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc0018600 con 0x7f3fd806aa70 2023-11-28T11:10:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.823+0000 7f3fddb40700 1 -- 172.21.15.28:0/1711576004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fd806aa70 msgr2=0x7f3fd80ce860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.823+0000 7f3fddb40700 1 --2- 172.21.15.28:0/1711576004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fd806aa70 0x7f3fd80ce860 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f3fc000a9e0 tx=0x7f3fc000acf0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.823+0000 7f3fddb40700 1 -- 172.21.15.28:0/1711576004 shutdown_connections 2023-11-28T11:10:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.823+0000 7f3fddb40700 1 --2- 172.21.15.28:0/1711576004 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fd806aa70 0x7f3fd80ce860 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.823+0000 7f3fddb40700 1 -- 172.21.15.28:0/1711576004 >> 172.21.15.28:0/1711576004 conn(0x7f3fd80b0dd0 msgr2=0x7f3fd80b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.824+0000 7f3fddb40700 1 -- 172.21.15.28:0/1711576004 shutdown_connections 2023-11-28T11:10:51.826 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.824+0000 7f3fddb40700 1 -- 172.21.15.28:0/1711576004 wait complete. 2023-11-28T11:10:51.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.825+0000 7f3fddb40700 1 Processor -- start 2023-11-28T11:10:51.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.825+0000 7f3fddb40700 1 -- start start 2023-11-28T11:10:51.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fddb40700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fd806aa70 0x7f3fd8156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fddb40700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fd8158a30 0x7f3fd814e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fddb40700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3fd815c680 0x7f3fd80c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fddb40700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd814eed0 con 0x7f3fd806aa70 2023-11-28T11:10:51.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fddb40700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd815a620 con 0x7f3fd8158a30 2023-11-28T11:10:51.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fddb40700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd815a900 con 0x7f3fd815c680 2023-11-28T11:10:51.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fd77fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fd806aa70 0x7f3fd8156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fd77fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fd806aa70 0x7f3fd8156a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50162/0 (socket says 172.21.15.28:50162) 2023-11-28T11:10:51.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fd77fe700 1 -- 172.21.15.28:0/3644046779 learned_addr learned my addr 172.21.15.28:0/3644046779 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:51.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fd77fe700 1 -- 172.21.15.28:0/3644046779 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3fd815c680 msgr2=0x7f3fd80c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fd77fe700 1 --2- 172.21.15.28:0/3644046779 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3fd815c680 0x7f3fd80c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fd77fe700 1 -- 172.21.15.28:0/3644046779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fd8158a30 msgr2=0x7f3fd814e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:51.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fd6ffd700 1 --2- 172.21.15.28:0/3644046779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fd8158a30 0x7f3fd814e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fd77fe700 1 --2- 172.21.15.28:0/3644046779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fd8158a30 0x7f3fd814e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:51.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fd77fe700 1 -- 172.21.15.28:0/3644046779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fc000a640 con 0x7f3fd806aa70 2023-11-28T11:10:51.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fd77fe700 1 --2- 172.21.15.28:0/3644046779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fd806aa70 0x7f3fd8156a30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000c700 tx=0x7f3fd000cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:51.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fcffff700 1 -- 172.21.15.28:0/3644046779 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0005d70 con 0x7f3fd806aa70 2023-11-28T11:10:51.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fcffff700 1 -- 172.21.15.28:0/3644046779 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd00070d0 con 0x7f3fd806aa70 2023-11-28T11:10:51.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.826+0000 7f3fddb40700 1 -- 172.21.15.28:0/3644046779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd806f340 con 0x7f3fd806aa70 2023-11-28T11:10:51.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.827+0000 7f3fcffff700 1 -- 172.21.15.28:0/3644046779 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd0007460 con 0x7f3fd806aa70 2023-11-28T11:10:51.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.827+0000 7f3fddb40700 1 -- 172.21.15.28:0/3644046779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd80ceda0 con 0x7f3fd806aa70 2023-11-28T11:10:51.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.827+0000 7f3fddb40700 1 -- 172.21.15.28:0/3644046779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd806e9c0 con 0x7f3fd806aa70 2023-11-28T11:10:51.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.828+0000 7f3fcffff700 1 -- 172.21.15.28:0/3644046779 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3fd0019030 con 0x7f3fd806aa70 2023-11-28T11:10:51.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.828+0000 7f3fcffff700 1 --2- 172.21.15.28:0/3644046779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3fb80682f0 0x7f3fb806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:51.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.828+0000 7f3fcffff700 1 -- 172.21.15.28:0/3644046779 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f3fd0085010 con 0x7f3fd806aa70 2023-11-28T11:10:51.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.828+0000 7f3fd6ffd700 1 --2- 172.21.15.28:0/3644046779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3fb80682f0 0x7f3fb806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:51.832 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.829+0000 7f3fd6ffd700 1 --2- 172.21.15.28:0/3644046779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3fb80682f0 0x7f3fb806a7a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80cae50 tx=0x7f3fc0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:51.833 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:51.833+0000 7f3fcffff700 1 -- 172.21.15.28:0/3644046779 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3fd00519b0 con 0x7f3fd806aa70 2023-11-28T11:10:52.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.105+0000 7f3fddb40700 1 -- 172.21.15.28:0/3644046779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]} v 0) v1 -- 0x7f3fd80675a0 con 0x7f3fd806aa70 2023-11-28T11:10:52.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.323+0000 7f3fcffff700 1 -- 172.21.15.28:0/3644046779 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_apply_tag"]}]=0 v132) v1 ==== 142+0+0 (secure 0 0 0) 0x7f3fd0054fd0 con 0x7f3fd806aa70 2023-11-28T11:10:52.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.325+0000 7f3fddb40700 1 -- 172.21.15.28:0/3644046779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3fb80682f0 msgr2=0x7f3fb806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.325+0000 7f3fddb40700 1 --2- 172.21.15.28:0/3644046779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3fb80682f0 0x7f3fb806a7a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3fd80cae50 tx=0x7f3fc0008040 comp rx=0 tx=0).stop 2023-11-28T11:10:52.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.325+0000 7f3fddb40700 1 -- 172.21.15.28:0/3644046779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fd806aa70 msgr2=0x7f3fd8156a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.325+0000 7f3fddb40700 1 --2- 172.21.15.28:0/3644046779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fd806aa70 0x7f3fd8156a30 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f3fd000c700 tx=0x7f3fd000cac0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.325+0000 7f3fddb40700 1 -- 172.21.15.28:0/3644046779 shutdown_connections 2023-11-28T11:10:52.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.325+0000 7f3fddb40700 1 --2- 172.21.15.28:0/3644046779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3fb80682f0 0x7f3fb806a7a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.325+0000 7f3fddb40700 1 --2- 172.21.15.28:0/3644046779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3fd806aa70 0x7f3fd8156a30 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.325+0000 7f3fddb40700 1 --2- 172.21.15.28:0/3644046779 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3fd815c680 0x7f3fd80c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.325+0000 7f3fddb40700 1 --2- 172.21.15.28:0/3644046779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3fd8158a30 0x7f3fd814e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.325+0000 7f3fddb40700 1 -- 172.21.15.28:0/3644046779 >> 172.21.15.28:0/3644046779 conn(0x7f3fd80b0dd0 msgr2=0x7f3fd80aef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:52.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.325+0000 7f3fddb40700 1 -- 172.21.15.28:0/3644046779 shutdown_connections 2023-11-28T11:10:52.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.325+0000 7f3fddb40700 1 -- 172.21.15.28:0/3644046779 wait complete. 2023-11-28T11:10:52.338 INFO:tasks.cephfs_test_runner:test_apply_tag (tasks.cephfs.test_forward_scrub.TestForwardScrub) ... ok 2023-11-28T11:10:52.339 INFO:tasks.cephfs_test_runner:Starting test: test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) That the MDS can repair an inodes backtrace in the data pool 2023-11-28T11:10:52.339 INFO:tasks.cephfs_test_runner:test_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:10:52.339 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T11:10:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.464+0000 7f6f7026e700 1 Processor -- start 2023-11-28T11:10:52.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.464+0000 7f6f7026e700 1 -- start start 2023-11-28T11:10:52.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.465+0000 7f6f7026e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f68161090 0x7f6f680ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.465+0000 7f6f7026e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6f680caeb0 0x7f6f680cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.465+0000 7f6f7026e700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f680ce340 con 0x7f6f6815c990 2023-11-28T11:10:52.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.465+0000 7f6f7026e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f680cebe0 con 0x7f6f680caeb0 2023-11-28T11:10:52.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.465+0000 7f6f7026e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680cf780 con 0x7f6f68161090 2023-11-28T11:10:52.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.465+0000 7f6f6e00a700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6f6815c990 0x7f6f680cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38228/0 (socket says 172.21.15.28:38228) 2023-11-28T11:10:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.465+0000 7f6f6e00a700 1 -- 172.21.15.28:0/48968334 learned_addr learned my addr 172.21.15.28:0/48968334 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d809700 1 --2- 172.21.15.28:0/48968334 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f68161090 0x7f6f680ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 -- 172.21.15.28:0/48968334 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4107838960 0 0) 0x7f6f680ce340 con 0x7f6f6815c990 2023-11-28T11:10:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 -- 172.21.15.28:0/48968334 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f540035b0 con 0x7f6f6815c990 2023-11-28T11:10:52.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6e80b700 1 --2- 172.21.15.28:0/48968334 >> v2:172.21.15.59:3300/0 conn(0x7f6f680caeb0 0x7f6f680cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 -- 172.21.15.28:0/48968334 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4151813424 0 0) 0x7f6f540035b0 con 0x7f6f6815c990 2023-11-28T11:10:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 -- 172.21.15.28:0/48968334 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f68161090 msgr2=0x7f6f680ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 --2- 172.21.15.28:0/48968334 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f68161090 0x7f6f680ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 -- 172.21.15.28:0/48968334 >> v2:172.21.15.59:3300/0 conn(0x7f6f680caeb0 msgr2=0x7f6f680cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 --2- 172.21.15.28:0/48968334 >> v2:172.21.15.59:3300/0 conn(0x7f6f680caeb0 0x7f6f680cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 -- 172.21.15.28:0/48968334 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f680cb3f0 con 0x7f6f6815c990 2023-11-28T11:10:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 -- 172.21.15.28:0/48968334 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f6f64001670 con 0x7f6f6815c990 2023-11-28T11:10:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 -- 172.21.15.28:0/48968334 >> v1:172.21.15.28:6789/0 conn(0x7f6f6815c990 legacy=0x7f6f680cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 --2- 172.21.15.28:0/48968334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f54003a00 0x7f6f54003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 --2- 172.21.15.28:0/48968334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f54004850 0x7f6f54004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 --2- 172.21.15.28:0/48968334 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f68161090 0x7f6f54005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 -- 172.21.15.28:0/48968334 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680cebe0 con 0x7f6f54004850 2023-11-28T11:10:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 -- 172.21.15.28:0/48968334 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680cf780 con 0x7f6f54003a00 2023-11-28T11:10:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d008700 1 -- 172.21.15.28:0/48968334 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f5400ba90 con 0x7f6f68161090 2023-11-28T11:10:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6d809700 1 --2- 172.21.15.28:0/48968334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f54004850 0x7f6f54004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6e00a700 1 --2- 172.21.15.28:0/48968334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f54003a00 0x7f6f54003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.466+0000 7f6f6e80b700 1 --2- 172.21.15.28:0/48968334 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f68161090 0x7f6f54005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.467+0000 7f6f6d809700 1 -- 172.21.15.28:0/48968334 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f68161090 msgr2=0x7f6f54005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.467+0000 7f6f6d809700 1 --2- 172.21.15.28:0/48968334 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f68161090 0x7f6f54005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.467+0000 7f6f6d809700 1 -- 172.21.15.28:0/48968334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f54003a00 msgr2=0x7f6f54003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:52.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.467+0000 7f6f6d809700 1 --2- 172.21.15.28:0/48968334 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f54003a00 0x7f6f54003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.467+0000 7f6f6d809700 1 -- 172.21.15.28:0/48968334 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f540035b0 con 0x7f6f54004850 2023-11-28T11:10:52.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.467+0000 7f6f6d809700 1 --2- 172.21.15.28:0/48968334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f54004850 0x7f6f54004cc0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009b30 tx=0x7f6f54005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ba1202e500c2b78 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:52.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.467+0000 7f6f6d008700 1 -- 172.21.15.28:0/48968334 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c00e6c0 con 0x7f6f54004850 2023-11-28T11:10:52.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.467+0000 7f6f6d008700 1 -- 172.21.15.28:0/48968334 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c00e820 con 0x7f6f54004850 2023-11-28T11:10:52.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.467+0000 7f6f6d008700 1 -- 172.21.15.28:0/48968334 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c00eb90 con 0x7f6f54004850 2023-11-28T11:10:52.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.467+0000 7f6f7026e700 1 -- 172.21.15.28:0/48968334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f54004850 msgr2=0x7f6f54004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.467+0000 7f6f7026e700 1 --2- 172.21.15.28:0/48968334 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f54004850 0x7f6f54004cc0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009b30 tx=0x7f6f54005880 comp rx=0 tx=0).stop 2023-11-28T11:10:52.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.467+0000 7f6f6e80b700 1 -- 172.21.15.28:0/48968334 reap_dead start 2023-11-28T11:10:52.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.467+0000 7f6f7026e700 1 -- 172.21.15.28:0/48968334 shutdown_connections 2023-11-28T11:10:52.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.467+0000 7f6f7026e700 1 -- 172.21.15.28:0/48968334 >> 172.21.15.28:0/48968334 conn(0x7f6f680b1650 msgr2=0x7f6f680b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:52.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.468+0000 7f6f7026e700 1 -- 172.21.15.28:0/48968334 shutdown_connections 2023-11-28T11:10:52.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.468+0000 7f6f7026e700 1 -- 172.21.15.28:0/48968334 wait complete. 2023-11-28T11:10:52.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.469+0000 7f6f7026e700 1 Processor -- start 2023-11-28T11:10:52.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.469+0000 7f6f7026e700 1 -- start start 2023-11-28T11:10:52.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.469+0000 7f6f7026e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f54004850 0x7f6f6815eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f7026e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f68150ec0 0x7f6f6815cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f7026e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f680c5600 0x7f6f680c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f7026e700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6815e6a0 con 0x7f6f54004850 2023-11-28T11:10:52.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f7026e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680c5fb0 con 0x7f6f68150ec0 2023-11-28T11:10:52.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f7026e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6815e330 con 0x7f6f680c5600 2023-11-28T11:10:52.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f6e00a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f54004850 0x7f6f6815eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f6e00a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f54004850 0x7f6f6815eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50182/0 (socket says 172.21.15.28:50182) 2023-11-28T11:10:52.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f6e00a700 1 -- 172.21.15.28:0/214495633 learned_addr learned my addr 172.21.15.28:0/214495633 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:52.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f6d809700 1 --2- 172.21.15.28:0/214495633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f68150ec0 0x7f6f6815cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f6e80b700 1 --2- 172.21.15.28:0/214495633 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f680c5600 0x7f6f680c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f6e00a700 1 -- 172.21.15.28:0/214495633 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f680c5600 msgr2=0x7f6f680c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f6e00a700 1 --2- 172.21.15.28:0/214495633 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f680c5600 0x7f6f680c5a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f6e00a700 1 -- 172.21.15.28:0/214495633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f68150ec0 msgr2=0x7f6f6815cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:52.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f6e00a700 1 --2- 172.21.15.28:0/214495633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f68150ec0 0x7f6f6815cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:52.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f6e00a700 1 -- 172.21.15.28:0/214495633 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f680ce870 con 0x7f6f54004850 2023-11-28T11:10:52.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f6e00a700 1 --2- 172.21.15.28:0/214495633 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f54004850 0x7f6f6815eac0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6f64000fb0 tx=0x7f6f640075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:52.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f5affd700 1 -- 172.21.15.28:0/214495633 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400f6c0 con 0x7f6f54004850 2023-11-28T11:10:52.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f5affd700 1 -- 172.21.15.28:0/214495633 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6400f860 con 0x7f6f54004850 2023-11-28T11:10:52.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f7026e700 1 -- 172.21.15.28:0/214495633 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f680ce590 con 0x7f6f54004850 2023-11-28T11:10:52.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f5affd700 1 -- 172.21.15.28:0/214495633 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400fc10 con 0x7f6f54004850 2023-11-28T11:10:52.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.470+0000 7f6f7026e700 1 -- 172.21.15.28:0/214495633 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f680ca970 con 0x7f6f54004850 2023-11-28T11:10:52.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.471+0000 7f6f7026e700 1 -- 172.21.15.28:0/214495633 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6815e7e0 con 0x7f6f54004850 2023-11-28T11:10:52.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.472+0000 7f6f5affd700 1 -- 172.21.15.28:0/214495633 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6f6400fd70 con 0x7f6f54004850 2023-11-28T11:10:52.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.472+0000 7f6f5affd700 1 --2- 172.21.15.28:0/214495633 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6f4c0682f0 0x7f6f4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:52.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.472+0000 7f6f5affd700 1 -- 172.21.15.28:0/214495633 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f6f64085ba0 con 0x7f6f54004850 2023-11-28T11:10:52.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.472+0000 7f6f6d809700 1 --2- 172.21.15.28:0/214495633 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6f4c0682f0 0x7f6f4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:52.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.472+0000 7f6f6d809700 1 --2- 172.21.15.28:0/214495633 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6f4c0682f0 0x7f6f4c06a7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0090b0 tx=0x7f6f5c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:52.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.476+0000 7f6f5affd700 1 -- 172.21.15.28:0/214495633 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6f6815e7e0 con 0x7f6f54004850 2023-11-28T11:10:52.756 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:52.756+0000 7f6f7026e700 1 -- 172.21.15.28:0/214495633 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f6f6815e7e0 con 0x7f6f54004850 2023-11-28T11:10:53.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.328+0000 7f6f5affd700 1 -- 172.21.15.28:0/214495633 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v133) v1 ==== 152+0+0 (secure 0 0 0) 0x7f6f6815e7e0 con 0x7f6f54004850 2023-11-28T11:10:53.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.330+0000 7f6f7026e700 1 -- 172.21.15.28:0/214495633 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6f4c0682f0 msgr2=0x7f6f4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:53.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.330+0000 7f6f7026e700 1 --2- 172.21.15.28:0/214495633 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6f4c0682f0 0x7f6f4c06a7a0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0090b0 tx=0x7f6f5c008030 comp rx=0 tx=0).stop 2023-11-28T11:10:53.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.330+0000 7f6f7026e700 1 -- 172.21.15.28:0/214495633 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f54004850 msgr2=0x7f6f6815eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:53.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.330+0000 7f6f7026e700 1 --2- 172.21.15.28:0/214495633 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f54004850 0x7f6f6815eac0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f6f64000fb0 tx=0x7f6f640075f0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.331+0000 7f6f7026e700 1 -- 172.21.15.28:0/214495633 shutdown_connections 2023-11-28T11:10:53.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.331+0000 7f6f7026e700 1 --2- 172.21.15.28:0/214495633 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6f4c0682f0 0x7f6f4c06a7a0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.331+0000 7f6f7026e700 1 --2- 172.21.15.28:0/214495633 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6f54004850 0x7f6f6815eac0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.331+0000 7f6f7026e700 1 --2- 172.21.15.28:0/214495633 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6f680c5600 0x7f6f680c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.331+0000 7f6f7026e700 1 --2- 172.21.15.28:0/214495633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f68150ec0 0x7f6f6815cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.331+0000 7f6f7026e700 1 -- 172.21.15.28:0/214495633 >> 172.21.15.28:0/214495633 conn(0x7f6f680b1650 msgr2=0x7f6f680cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:53.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.331+0000 7f6f7026e700 1 -- 172.21.15.28:0/214495633 shutdown_connections 2023-11-28T11:10:53.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.331+0000 7f6f7026e700 1 -- 172.21.15.28:0/214495633 wait complete. 2023-11-28T11:10:53.346 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:10:53.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.480+0000 7fdbb2ec8700 1 Processor -- start 2023-11-28T11:10:53.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.480+0000 7fdbb2ec8700 1 -- start start 2023-11-28T11:10:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.480+0000 7fdbb2ec8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdbac0b1630 0x7fdbac0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.480+0000 7fdbb2ec8700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdbac0b1a00 0x7fdbac0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.480+0000 7fdbb2ec8700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbac0ce550 con 0x7fdbac0cb3e0 2023-11-28T11:10:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.480+0000 7fdbb2ec8700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbac0ce830 con 0x7fdbac0b1a00 2023-11-28T11:10:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.480+0000 7fdbb2ec8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbac0ceb10 con 0x7fdbac0b1630 2023-11-28T11:10:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.481+0000 7fdbb1465700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdbac0cb3e0 0x7fdbac160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38248/0 (socket says 172.21.15.28:38248) 2023-11-28T11:10:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.481+0000 7fdbabfff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdbac0b1a00 0x7fdbac0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:53.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.481+0000 7fdbb1465700 1 -- 172.21.15.28:0/1059150757 learned_addr learned my addr 172.21.15.28:0/1059150757 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.481+0000 7fdbb0c64700 1 --2- 172.21.15.28:0/1059150757 >> v2:172.21.15.59:3300/0 conn(0x7fdbac0b1630 0x7fdbac0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.481+0000 7fdbab7fe700 1 -- 172.21.15.28:0/1059150757 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531581627 0 0) 0x7fdbac0ce550 con 0x7fdbac0cb3e0 2023-11-28T11:10:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.481+0000 7fdbab7fe700 1 -- 172.21.15.28:0/1059150757 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb9c0035b0 con 0x7fdbac0cb3e0 2023-11-28T11:10:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.481+0000 7fdbabfff700 1 -- 172.21.15.28:0/1059150757 >> v2:172.21.15.59:3300/0 conn(0x7fdbac0b1630 msgr2=0x7fdbac0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.481+0000 7fdbabfff700 1 --2- 172.21.15.28:0/1059150757 >> v2:172.21.15.59:3300/0 conn(0x7fdbac0b1630 0x7fdbac0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.481+0000 7fdbabfff700 1 -- 172.21.15.28:0/1059150757 >> v1:172.21.15.28:6789/0 conn(0x7fdbac0cb3e0 legacy=0x7fdbac160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.481+0000 7fdbabfff700 1 -- 172.21.15.28:0/1059150757 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac0cedf0 con 0x7fdbac0b1a00 2023-11-28T11:10:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.481+0000 7fdbb0c64700 1 --2- 172.21.15.28:0/1059150757 >> v2:172.21.15.59:3300/0 conn(0x7fdbac0b1630 0x7fdbac0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.481+0000 7fdbabfff700 1 --2- 172.21.15.28:0/1059150757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbac0b1a00 0x7fdbac0caea0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009840 tx=0x7fdb98009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=750fcc68ec2c91ac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:53.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.481+0000 7fdbab7fe700 1 -- 172.21.15.28:0/1059150757 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98011830 con 0x7fdbac0b1a00 2023-11-28T11:10:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.481+0000 7fdbab7fe700 1 -- 172.21.15.28:0/1059150757 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb98011990 con 0x7fdbac0b1a00 2023-11-28T11:10:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.481+0000 7fdbab7fe700 1 -- 172.21.15.28:0/1059150757 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98011d20 con 0x7fdbac0b1a00 2023-11-28T11:10:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.482+0000 7fdbb2ec8700 1 -- 172.21.15.28:0/1059150757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbac0b1a00 msgr2=0x7fdbac0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.482+0000 7fdbb2ec8700 1 --2- 172.21.15.28:0/1059150757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbac0b1a00 0x7fdbac0caea0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009840 tx=0x7fdb98009b50 comp rx=0 tx=0).stop 2023-11-28T11:10:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.482+0000 7fdbb2ec8700 1 -- 172.21.15.28:0/1059150757 shutdown_connections 2023-11-28T11:10:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.482+0000 7fdbb2ec8700 1 --2- 172.21.15.28:0/1059150757 >> v2:172.21.15.59:3300/0 conn(0x7fdbac0b1630 0x7fdbac0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.482+0000 7fdbb2ec8700 1 --2- 172.21.15.28:0/1059150757 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbac0b1a00 0x7fdbac0caea0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.482+0000 7fdbb2ec8700 1 -- 172.21.15.28:0/1059150757 >> 172.21.15.28:0/1059150757 conn(0x7fdbac0cc630 msgr2=0x7fdbac0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.482+0000 7fdbb2ec8700 1 -- 172.21.15.28:0/1059150757 shutdown_connections 2023-11-28T11:10:53.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.482+0000 7fdbb2ec8700 1 -- 172.21.15.28:0/1059150757 wait complete. 2023-11-28T11:10:53.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.483+0000 7fdbb2ec8700 1 Processor -- start 2023-11-28T11:10:53.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.483+0000 7fdbb2ec8700 1 -- start start 2023-11-28T11:10:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.484+0000 7fdbb2ec8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbac0b1630 0x7fdbac1e7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.484+0000 7fdbb2ec8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdbac0b1a00 0x7fdbac1e21c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.484+0000 7fdbb2ec8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbac0cb3e0 0x7fdbac1f23c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.484+0000 7fdbb2ec8700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac1e3620 con 0x7fdbac0b1630 2023-11-28T11:10:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.484+0000 7fdbb2ec8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac1e2700 con 0x7fdbac0cb3e0 2023-11-28T11:10:53.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.484+0000 7fdbb2ec8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac1e29a0 con 0x7fdbac0b1a00 2023-11-28T11:10:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.484+0000 7fdbb0c64700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbac0b1630 0x7fdbac1e7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.484+0000 7fdbb0c64700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbac0b1630 0x7fdbac1e7220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50194/0 (socket says 172.21.15.28:50194) 2023-11-28T11:10:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.484+0000 7fdbb0c64700 1 -- 172.21.15.28:0/3617306936 learned_addr learned my addr 172.21.15.28:0/3617306936 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.484+0000 7fdbb1465700 1 --2- 172.21.15.28:0/3617306936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbac0cb3e0 0x7fdbac1f23c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.484+0000 7fdbb0c64700 1 -- 172.21.15.28:0/3617306936 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdbac0b1a00 msgr2=0x7fdbac1e21c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.484+0000 7fdbb0c64700 1 --2- 172.21.15.28:0/3617306936 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdbac0b1a00 0x7fdbac1e21c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.484+0000 7fdbb0c64700 1 -- 172.21.15.28:0/3617306936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbac0cb3e0 msgr2=0x7fdbac1f23c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.484+0000 7fdbb0c64700 1 --2- 172.21.15.28:0/3617306936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbac0cb3e0 0x7fdbac1f23c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.484+0000 7fdbb0c64700 1 -- 172.21.15.28:0/3617306936 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb980094a0 con 0x7fdbac0b1630 2023-11-28T11:10:53.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.485+0000 7fdbb0c64700 1 --2- 172.21.15.28:0/3617306936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbac0b1630 0x7fdbac1e7220 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdba000e810 tx=0x7fdba000ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.485+0000 7fdba97fa700 1 -- 172.21.15.28:0/3617306936 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba000c020 con 0x7fdbac0b1630 2023-11-28T11:10:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.485+0000 7fdba97fa700 1 -- 172.21.15.28:0/3617306936 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba0004250 con 0x7fdbac0b1630 2023-11-28T11:10:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.485+0000 7fdbb2ec8700 1 -- 172.21.15.28:0/3617306936 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbac0b1ea0 con 0x7fdbac0b1630 2023-11-28T11:10:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.485+0000 7fdba97fa700 1 -- 172.21.15.28:0/3617306936 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba00036a0 con 0x7fdbac0b1630 2023-11-28T11:10:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.485+0000 7fdbb2ec8700 1 -- 172.21.15.28:0/3617306936 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbac0cf0d0 con 0x7fdbac0b1630 2023-11-28T11:10:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.486+0000 7fdbb2ec8700 1 -- 172.21.15.28:0/3617306936 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbac0ce140 con 0x7fdbac0b1630 2023-11-28T11:10:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.486+0000 7fdba97fa700 1 -- 172.21.15.28:0/3617306936 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdba0003800 con 0x7fdbac0b1630 2023-11-28T11:10:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.487+0000 7fdba97fa700 1 --2- 172.21.15.28:0/3617306936 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdb900682f0 0x7fdb9006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.487+0000 7fdbabfff700 1 --2- 172.21.15.28:0/3617306936 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdb900682f0 0x7fdb9006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.487+0000 7fdba97fa700 1 -- 172.21.15.28:0/3617306936 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fdba0085850 con 0x7fdbac0b1630 2023-11-28T11:10:53.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.487+0000 7fdbabfff700 1 --2- 172.21.15.28:0/3617306936 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdb900682f0 0x7fdb9006a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdbac06cc30 tx=0x7fdb98009f90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:53.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.491+0000 7fdba97fa700 1 -- 172.21.15.28:0/3617306936 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdba0052270 con 0x7fdbac0b1630 2023-11-28T11:10:53.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.687+0000 7fdbb2ec8700 1 -- 172.21.15.28:0/3617306936 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fdbac0739f0 con 0x7fdbac0b1630 2023-11-28T11:10:53.691 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.691+0000 7fdba97fa700 1 -- 172.21.15.28:0/3617306936 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v12)=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdba000f040 con 0x7fdbac0b1630 2023-11-28T11:10:53.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.693+0000 7fdbb2ec8700 1 -- 172.21.15.28:0/3617306936 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdb900682f0 msgr2=0x7fdb9006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:53.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.693+0000 7fdbb2ec8700 1 --2- 172.21.15.28:0/3617306936 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdb900682f0 0x7fdb9006a7a0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fdbac06cc30 tx=0x7fdb98009f90 comp rx=0 tx=0).stop 2023-11-28T11:10:53.695 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.693+0000 7fdbb2ec8700 1 -- 172.21.15.28:0/3617306936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbac0b1630 msgr2=0x7fdbac1e7220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:53.695 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.693+0000 7fdbb2ec8700 1 --2- 172.21.15.28:0/3617306936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbac0b1630 0x7fdbac1e7220 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fdba000e810 tx=0x7fdba000ebd0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.695 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.693+0000 7fdbb2ec8700 1 -- 172.21.15.28:0/3617306936 shutdown_connections 2023-11-28T11:10:53.695 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.693+0000 7fdbb2ec8700 1 --2- 172.21.15.28:0/3617306936 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdb900682f0 0x7fdb9006a7a0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.695 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.693+0000 7fdbb2ec8700 1 --2- 172.21.15.28:0/3617306936 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbac0b1630 0x7fdbac1e7220 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.693+0000 7fdbb2ec8700 1 --2- 172.21.15.28:0/3617306936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbac0cb3e0 0x7fdbac1f23c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.693+0000 7fdbb2ec8700 1 --2- 172.21.15.28:0/3617306936 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdbac0b1a00 0x7fdbac1e21c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:53.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.693+0000 7fdbb2ec8700 1 -- 172.21.15.28:0/3617306936 >> 172.21.15.28:0/3617306936 conn(0x7fdbac0cc630 msgr2=0x7fdbac15c090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:53.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.693+0000 7fdbb2ec8700 1 -- 172.21.15.28:0/3617306936 shutdown_connections 2023-11-28T11:10:53.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:53.693+0000 7fdbb2ec8700 1 -- 172.21.15.28:0/3617306936 wait complete. 2023-11-28T11:10:53.706 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:10:53.706 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:10:53.745 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:10:53.788 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:10:53.788 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:10:53.788 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:53.806 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:10:53.806 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:10:53.806 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:53.848 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3961 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:10:53.848 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:10:53.848 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:53.866 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:10:53.866 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:10:53.866 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:10:53.909 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3961 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:10:53.909 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:10:53.910 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:10:54.034 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.034+0000 7f896cebd700 1 Processor -- start 2023-11-28T11:10:54.035 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.034+0000 7f896cebd700 1 -- start start 2023-11-28T11:10:54.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.035+0000 7f896cebd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f896815a650 0x7f89680b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.035+0000 7f896cebd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f89680b5500 0x7f89680cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.035+0000 7f896cebd700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89680ca990 con 0x7f896815aa20 2023-11-28T11:10:54.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.035+0000 7f896cebd700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89680caf50 con 0x7f896815a650 2023-11-28T11:10:54.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.035+0000 7f896cebd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89680aef10 con 0x7f89680b5500 2023-11-28T11:10:54.037 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.035+0000 7f896659c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f896815a650 0x7f89680b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:54.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.035+0000 7f8965d9b700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f896815aa20 0x7f89680ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38260/0 (socket says 172.21.15.28:38260) 2023-11-28T11:10:54.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.035+0000 7f8965d9b700 1 -- 172.21.15.28:0/3684269365 learned_addr learned my addr 172.21.15.28:0/3684269365 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:54.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.035+0000 7f8966d9d700 1 --2- 172.21.15.28:0/3684269365 >> v2:172.21.15.59:3300/0 conn(0x7f89680b5500 0x7f89680cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:54.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.035+0000 7f896659c700 1 -- 172.21.15.28:0/3684269365 >> v2:172.21.15.59:3300/0 conn(0x7f89680b5500 msgr2=0x7f89680cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:54.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.035+0000 7f896659c700 1 --2- 172.21.15.28:0/3684269365 >> v2:172.21.15.59:3300/0 conn(0x7f89680b5500 0x7f89680cc4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.035+0000 7f896659c700 1 -- 172.21.15.28:0/3684269365 >> v1:172.21.15.28:6789/0 conn(0x7f896815aa20 legacy=0x7f89680ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.035+0000 7f896659c700 1 -- 172.21.15.28:0/3684269365 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89680af1f0 con 0x7f896815a650 2023-11-28T11:10:54.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.035+0000 7f896559a700 1 -- 172.21.15.28:0/3684269365 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1484467301 0 0) 0x7f89680ca990 con 0x7f896815aa20 2023-11-28T11:10:54.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.035+0000 7f896659c700 1 --2- 172.21.15.28:0/3684269365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f896815a650 0x7f89680b2950 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f89680caec0 tx=0x7f8958009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa4bb65f783851fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:54.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.036+0000 7f896559a700 1 -- 172.21.15.28:0/3684269365 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958011830 con 0x7f896815a650 2023-11-28T11:10:54.038 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.036+0000 7f896559a700 1 -- 172.21.15.28:0/3684269365 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8958011990 con 0x7f896815a650 2023-11-28T11:10:54.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.036+0000 7f896559a700 1 -- 172.21.15.28:0/3684269365 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8958011d20 con 0x7f896815a650 2023-11-28T11:10:54.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.036+0000 7f896cebd700 1 -- 172.21.15.28:0/3684269365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f896815a650 msgr2=0x7f89680b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.036+0000 7f896cebd700 1 --2- 172.21.15.28:0/3684269365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f896815a650 0x7f89680b2950 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f89680caec0 tx=0x7f8958009b20 comp rx=0 tx=0).stop 2023-11-28T11:10:54.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.036+0000 7f896cebd700 1 -- 172.21.15.28:0/3684269365 shutdown_connections 2023-11-28T11:10:54.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.036+0000 7f896cebd700 1 --2- 172.21.15.28:0/3684269365 >> v2:172.21.15.59:3300/0 conn(0x7f89680b5500 0x7f89680cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.036+0000 7f896cebd700 1 --2- 172.21.15.28:0/3684269365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f896815a650 0x7f89680b2950 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.036+0000 7f896cebd700 1 -- 172.21.15.28:0/3684269365 >> 172.21.15.28:0/3684269365 conn(0x7f89680b1650 msgr2=0x7f89680b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:54.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.036+0000 7f896cebd700 1 -- 172.21.15.28:0/3684269365 shutdown_connections 2023-11-28T11:10:54.039 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.036+0000 7f896cebd700 1 -- 172.21.15.28:0/3684269365 wait complete. 2023-11-28T11:10:54.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.038+0000 7f896cebd700 1 Processor -- start 2023-11-28T11:10:54.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.038+0000 7f896cebd700 1 -- start start 2023-11-28T11:10:54.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.038+0000 7f896cebd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89680b5500 0x7f8968154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.038+0000 7f896cebd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f896815a650 0x7f8968152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.038+0000 7f896cebd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896815aa20 0x7f89680af430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.038+0000 7f896cebd700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89680b4830 con 0x7f896815a650 2023-11-28T11:10:54.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.038+0000 7f896cebd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89680cd630 con 0x7f896815aa20 2023-11-28T11:10:54.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.038+0000 7f896cebd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89680cd880 con 0x7f89680b5500 2023-11-28T11:10:54.042 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.038+0000 7f8965d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f896815a650 0x7f8968152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:54.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.038+0000 7f8965d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f896815a650 0x7f8968152dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50210/0 (socket says 172.21.15.28:50210) 2023-11-28T11:10:54.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.039+0000 7f8965d9b700 1 -- 172.21.15.28:0/1316186995 learned_addr learned my addr 172.21.15.28:0/1316186995 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:54.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.039+0000 7f8966d9d700 1 --2- 172.21.15.28:0/1316186995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896815aa20 0x7f89680af430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:54.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.039+0000 7f8965d9b700 1 -- 172.21.15.28:0/1316186995 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89680b5500 msgr2=0x7f8968154330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.039+0000 7f8965d9b700 1 --2- 172.21.15.28:0/1316186995 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89680b5500 0x7f8968154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.039+0000 7f8965d9b700 1 -- 172.21.15.28:0/1316186995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896815aa20 msgr2=0x7f89680af430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.039+0000 7f8965d9b700 1 --2- 172.21.15.28:0/1316186995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896815aa20 0x7f89680af430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.039+0000 7f8965d9b700 1 -- 172.21.15.28:0/1316186995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89580094a0 con 0x7f896815a650 2023-11-28T11:10:54.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.039+0000 7f8965d9b700 1 --2- 172.21.15.28:0/1316186995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f896815a650 0x7f8968152dd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f895c00c520 tx=0x7f895c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:54.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.039+0000 7f8956ffd700 1 -- 172.21.15.28:0/1316186995 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c004c80 con 0x7f896815a650 2023-11-28T11:10:54.043 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.039+0000 7f8956ffd700 1 -- 172.21.15.28:0/1316186995 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f895c004de0 con 0x7f896815a650 2023-11-28T11:10:54.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.039+0000 7f896cebd700 1 -- 172.21.15.28:0/1316186995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89680cc9f0 con 0x7f896815a650 2023-11-28T11:10:54.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.039+0000 7f8956ffd700 1 -- 172.21.15.28:0/1316186995 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895c0072d0 con 0x7f896815a650 2023-11-28T11:10:54.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.039+0000 7f896cebd700 1 -- 172.21.15.28:0/1316186995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89680cabe0 con 0x7f896815a650 2023-11-28T11:10:54.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.040+0000 7f896cebd700 1 -- 172.21.15.28:0/1316186995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89680ccb80 con 0x7f896815a650 2023-11-28T11:10:54.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.040+0000 7f8956ffd700 1 -- 172.21.15.28:0/1316186995 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f895c01a020 con 0x7f896815a650 2023-11-28T11:10:54.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.041+0000 7f8956ffd700 1 --2- 172.21.15.28:0/1316186995 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f89480682f0 0x7f894806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.041+0000 7f8956ffd700 1 -- 172.21.15.28:0/1316186995 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f895c0856a0 con 0x7f896815a650 2023-11-28T11:10:54.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.041+0000 7f896659c700 1 --2- 172.21.15.28:0/1316186995 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f89480682f0 0x7f894806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:54.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.041+0000 7f896659c700 1 --2- 172.21.15.28:0/1316186995 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f89480682f0 0x7f894806a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8958000c00 tx=0x7f8958010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:54.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.045+0000 7f8956ffd700 1 -- 172.21.15.28:0/1316186995 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f895c052170 con 0x7f896815a650 2023-11-28T11:10:54.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.284+0000 7f896cebd700 1 -- 172.21.15.28:0/1316186995 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f89680b0170 con 0x7f896815a650 2023-11-28T11:10:54.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.284+0000 7f8956ffd700 1 -- 172.21.15.28:0/1316186995 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 19 v19) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f895c055790 con 0x7f896815a650 2023-11-28T11:10:54.285 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:54.285 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":19,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4934,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6833/2867100265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2867100265},{"type":"v1","addr":"172.21.15.59:6833","nonce":2867100265}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5137,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3808190953","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3808190953},{"type":"v1","addr":"172.21.15.28:6837","nonce":3808190953}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":5431,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/4264898946","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":4264898946},{"type":"v1","addr":"172.21.15.59:6835","nonce":4264898946}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5526,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/3920280453","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":3920280453},{"type":"v1","addr":"172.21.15.28:6836","nonce":3920280453}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[]} 2023-11-28T11:10:54.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.286+0000 7f896cebd700 1 -- 172.21.15.28:0/1316186995 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f89480682f0 msgr2=0x7f894806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.286+0000 7f896cebd700 1 --2- 172.21.15.28:0/1316186995 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f89480682f0 0x7f894806a7a0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f8958000c00 tx=0x7f8958010000 comp rx=0 tx=0).stop 2023-11-28T11:10:54.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.286+0000 7f896cebd700 1 -- 172.21.15.28:0/1316186995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f896815a650 msgr2=0x7f8968152dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.286+0000 7f896cebd700 1 --2- 172.21.15.28:0/1316186995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f896815a650 0x7f8968152dd0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f895c00c520 tx=0x7f895c00c8e0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.287 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.286+0000 7f896cebd700 1 -- 172.21.15.28:0/1316186995 shutdown_connections 2023-11-28T11:10:54.288 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.286+0000 7f896cebd700 1 --2- 172.21.15.28:0/1316186995 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f89480682f0 0x7f894806a7a0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.288 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.286+0000 7f896cebd700 1 --2- 172.21.15.28:0/1316186995 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896815aa20 0x7f89680af430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.288 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.286+0000 7f896cebd700 1 --2- 172.21.15.28:0/1316186995 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f89680b5500 0x7f8968154330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.288 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.286+0000 7f896cebd700 1 --2- 172.21.15.28:0/1316186995 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f896815a650 0x7f8968152dd0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.288 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.286+0000 7f896cebd700 1 -- 172.21.15.28:0/1316186995 >> 172.21.15.28:0/1316186995 conn(0x7f89680b1650 msgr2=0x7f896806b970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:54.288 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.286+0000 7f896cebd700 1 -- 172.21.15.28:0/1316186995 shutdown_connections 2023-11-28T11:10:54.288 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.286+0000 7f896cebd700 1 -- 172.21.15.28:0/1316186995 wait complete. 2023-11-28T11:10:54.288 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 19 2023-11-28T11:10:54.299 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:10:54.299 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:10:54.300 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:10:54.300 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:54.300 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:10:54.300 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:10:54.301 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:10:54.301 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:54.301 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:10:54.301 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:10:54.302 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:10:54.302 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:54.302 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:10:54.302 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:10:54.302 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:10:54.302 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:10:54.309 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:10:54.310 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:10:54.310 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:10:54.311 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:10:54.313 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:10:54.313 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:10:54.314 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:10:54.314 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:10:54.352 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:10:54.352 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:54.353 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:10:54.353 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:54.355 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:10:54.356 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:54.356 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:10:54.356 DEBUG:teuthology.parallel:result is None 2023-11-28T11:10:54.357 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:10:54.434 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2023-11-28T11:10:54.434 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2023-11-28T11:10:54.448 INFO:tasks.ceph.mds.d.smithi059.stdout:starting mds.d at 2023-11-28T11:10:54.448 INFO:tasks.ceph.mds.b.smithi059.stdout:starting mds.b at 2023-11-28T11:10:54.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.492+0000 7f22deecc700 1 Processor -- start 2023-11-28T11:10:54.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.492+0000 7f22deecc700 1 -- start start 2023-11-28T11:10:54.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.492+0000 7f22deecc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f22d80b2660 0x7f22d80ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.492+0000 7f22deecc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f22d80cecf0 0x7f22d806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.492+0000 7f22deecc700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22d80b30b0 con 0x7f22d815c560 2023-11-28T11:10:54.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.492+0000 7f22deecc700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d806c020 con 0x7f22d80cecf0 2023-11-28T11:10:54.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.492+0000 7f22deecc700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22d80cf110 con 0x7f22d80b2660 2023-11-28T11:10:54.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.492+0000 7f22dd469700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f22d80cecf0 0x7f22d806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:54.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.492+0000 7f22dcc68700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f22d815c560 0x7f22d806f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38298/0 (socket says 172.21.15.28:38298) 2023-11-28T11:10:54.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.493+0000 7f22dcc68700 1 -- 172.21.15.28:0/2919859850 learned_addr learned my addr 172.21.15.28:0/2919859850 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:54.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.493+0000 7f22d7fff700 1 --2- 172.21.15.28:0/2919859850 >> v2:172.21.15.59:3300/0 conn(0x7f22d80b2660 0x7f22d80ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:54.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.493+0000 7f22dd469700 1 -- 172.21.15.28:0/2919859850 >> v2:172.21.15.59:3300/0 conn(0x7f22d80b2660 msgr2=0x7f22d80ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:54.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.493+0000 7f22dd469700 1 --2- 172.21.15.28:0/2919859850 >> v2:172.21.15.59:3300/0 conn(0x7f22d80b2660 0x7f22d80ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.493+0000 7f22dd469700 1 -- 172.21.15.28:0/2919859850 >> v1:172.21.15.28:6789/0 conn(0x7f22d815c560 legacy=0x7f22d806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.493+0000 7f22d77fe700 1 -- 172.21.15.28:0/2919859850 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441636863 0 0) 0x7f22d80b30b0 con 0x7f22d815c560 2023-11-28T11:10:54.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.493+0000 7f22dd469700 1 -- 172.21.15.28:0/2919859850 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d80d09c0 con 0x7f22d80cecf0 2023-11-28T11:10:54.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.493+0000 7f22dd469700 1 --2- 172.21.15.28:0/2919859850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22d80cecf0 0x7f22d806c4b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f22d8071e10 tx=0x7f22d000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72ac69cfdffdc3ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:54.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.493+0000 7f22d77fe700 1 -- 172.21.15.28:0/2919859850 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d000e070 con 0x7f22d80cecf0 2023-11-28T11:10:54.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.493+0000 7f22d77fe700 1 -- 172.21.15.28:0/2919859850 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22d000bbd0 con 0x7f22d80cecf0 2023-11-28T11:10:54.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.493+0000 7f22d77fe700 1 -- 172.21.15.28:0/2919859850 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22d0012430 con 0x7f22d80cecf0 2023-11-28T11:10:54.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.494+0000 7f22deecc700 1 -- 172.21.15.28:0/2919859850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22d80cecf0 msgr2=0x7f22d806c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.494+0000 7f22deecc700 1 --2- 172.21.15.28:0/2919859850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22d80cecf0 0x7f22d806c4b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f22d8071e10 tx=0x7f22d000b330 comp rx=0 tx=0).stop 2023-11-28T11:10:54.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.494+0000 7f22deecc700 1 -- 172.21.15.28:0/2919859850 shutdown_connections 2023-11-28T11:10:54.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.494+0000 7f22deecc700 1 --2- 172.21.15.28:0/2919859850 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22d80cecf0 0x7f22d806c4b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.494+0000 7f22deecc700 1 --2- 172.21.15.28:0/2919859850 >> v2:172.21.15.59:3300/0 conn(0x7f22d80b2660 0x7f22d80ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.494+0000 7f22deecc700 1 -- 172.21.15.28:0/2919859850 >> 172.21.15.28:0/2919859850 conn(0x7f22d80cd660 msgr2=0x7f22d80cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:54.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.494+0000 7f22deecc700 1 -- 172.21.15.28:0/2919859850 shutdown_connections 2023-11-28T11:10:54.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.494+0000 7f22deecc700 1 -- 172.21.15.28:0/2919859850 wait complete. 2023-11-28T11:10:54.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.495+0000 7f22deecc700 1 Processor -- start 2023-11-28T11:10:54.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.495+0000 7f22deecc700 1 -- start start 2023-11-28T11:10:54.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22deecc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22d80b2660 0x7f22d81f2a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22deecc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22d80cecf0 0x7f22d81605a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22deecc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22d815c560 0x7f22d80b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22deecc700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d8160110 con 0x7f22d80cecf0 2023-11-28T11:10:54.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22deecc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22d81f2f80 con 0x7f22d80b2660 2023-11-28T11:10:54.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22deecc700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22d815d350 con 0x7f22d815c560 2023-11-28T11:10:54.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22d7fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22d80cecf0 0x7f22d81605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:54.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22d7fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22d80cecf0 0x7f22d81605a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50246/0 (socket says 172.21.15.28:50246) 2023-11-28T11:10:54.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22d7fff700 1 -- 172.21.15.28:0/3690681267 learned_addr learned my addr 172.21.15.28:0/3690681267 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:54.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22dd469700 1 --2- 172.21.15.28:0/3690681267 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22d815c560 0x7f22d80b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:54.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22dcc68700 1 --2- 172.21.15.28:0/3690681267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22d80b2660 0x7f22d81f2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:54.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22d7fff700 1 -- 172.21.15.28:0/3690681267 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22d815c560 msgr2=0x7f22d80b1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22d7fff700 1 --2- 172.21.15.28:0/3690681267 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22d815c560 0x7f22d80b1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22d7fff700 1 -- 172.21.15.28:0/3690681267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22d80b2660 msgr2=0x7f22d81f2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:54.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22d7fff700 1 --2- 172.21.15.28:0/3690681267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22d80b2660 0x7f22d81f2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:54.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22d7fff700 1 -- 172.21.15.28:0/3690681267 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22d0009a70 con 0x7f22d80cecf0 2023-11-28T11:10:54.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.496+0000 7f22d7fff700 1 --2- 172.21.15.28:0/3690681267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22d80cecf0 0x7f22d81605a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f22c400e810 tx=0x7f22c400eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:54.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.497+0000 7f22d57fa700 1 -- 172.21.15.28:0/3690681267 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c400c070 con 0x7f22d80cecf0 2023-11-28T11:10:54.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.497+0000 7f22d57fa700 1 -- 172.21.15.28:0/3690681267 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22c4013610 con 0x7f22d80cecf0 2023-11-28T11:10:54.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.497+0000 7f22deecc700 1 -- 172.21.15.28:0/3690681267 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22d80b2dd0 con 0x7f22d80cecf0 2023-11-28T11:10:54.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.497+0000 7f22d57fa700 1 -- 172.21.15.28:0/3690681267 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22c40139c0 con 0x7f22d80cecf0 2023-11-28T11:10:54.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.497+0000 7f22deecc700 1 -- 172.21.15.28:0/3690681267 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22d80b2af0 con 0x7f22d80cecf0 2023-11-28T11:10:54.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.498+0000 7f22deecc700 1 -- 172.21.15.28:0/3690681267 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22d806bcb0 con 0x7f22d80cecf0 2023-11-28T11:10:54.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.498+0000 7f22d57fa700 1 -- 172.21.15.28:0/3690681267 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f22c4023070 con 0x7f22d80cecf0 2023-11-28T11:10:54.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.498+0000 7f22d57fa700 1 --2- 172.21.15.28:0/3690681267 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f22bc0682f0 0x7f22bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:54.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.499+0000 7f22d57fa700 1 -- 172.21.15.28:0/3690681267 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f22c40858a0 con 0x7f22d80cecf0 2023-11-28T11:10:54.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.499+0000 7f22dcc68700 1 --2- 172.21.15.28:0/3690681267 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f22bc0682f0 0x7f22bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:54.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.499+0000 7f22dcc68700 1 --2- 172.21.15.28:0/3690681267 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f22bc0682f0 0x7f22bc06a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f22cc000f60 tx=0x7f22cc007e70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:54.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.503+0000 7f22d57fa700 1 -- 172.21.15.28:0/3690681267 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f22c4052240 con 0x7f22d80cecf0 2023-11-28T11:10:54.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:54.710+0000 7f22deecc700 1 -- 172.21.15.28:0/3690681267 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f22d80675a0 con 0x7f22d80cecf0 2023-11-28T11:10:55.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.477+0000 7f22d57fa700 1 -- 172.21.15.28:0/3690681267 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v50) v1 ==== 97+0+0 (secure 0 0 0) 0x7f22c4059e60 con 0x7f22d80cecf0 2023-11-28T11:10:55.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.480+0000 7f22deecc700 1 -- 172.21.15.28:0/3690681267 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f22bc0682f0 msgr2=0x7f22bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.480+0000 7f22deecc700 1 --2- 172.21.15.28:0/3690681267 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f22bc0682f0 0x7f22bc06a7a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f22cc000f60 tx=0x7f22cc007e70 comp rx=0 tx=0).stop 2023-11-28T11:10:55.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.480+0000 7f22deecc700 1 -- 172.21.15.28:0/3690681267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22d80cecf0 msgr2=0x7f22d81605a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.480+0000 7f22deecc700 1 --2- 172.21.15.28:0/3690681267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22d80cecf0 0x7f22d81605a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f22c400e810 tx=0x7f22c400eb20 comp rx=0 tx=0).stop 2023-11-28T11:10:55.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.480+0000 7f22deecc700 1 -- 172.21.15.28:0/3690681267 shutdown_connections 2023-11-28T11:10:55.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.480+0000 7f22deecc700 1 --2- 172.21.15.28:0/3690681267 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f22bc0682f0 0x7f22bc06a7a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.480+0000 7f22deecc700 1 --2- 172.21.15.28:0/3690681267 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f22d815c560 0x7f22d80b1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.480+0000 7f22deecc700 1 --2- 172.21.15.28:0/3690681267 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f22d80b2660 0x7f22d81f2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.480+0000 7f22deecc700 1 --2- 172.21.15.28:0/3690681267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f22d80cecf0 0x7f22d81605a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.480+0000 7f22deecc700 1 -- 172.21.15.28:0/3690681267 >> 172.21.15.28:0/3690681267 conn(0x7f22d80cd660 msgr2=0x7f22d80afc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:55.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.480+0000 7f22deecc700 1 -- 172.21.15.28:0/3690681267 shutdown_connections 2023-11-28T11:10:55.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.480+0000 7f22deecc700 1 -- 172.21.15.28:0/3690681267 wait complete. 2023-11-28T11:10:55.482 INFO:teuthology.orchestra.run.smithi028.stderr: removed all blocklist entries 2023-11-28T11:10:55.496 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:10:55.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.627+0000 7fcf1aaa6700 1 Processor -- start 2023-11-28T11:10:55.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.627+0000 7fcf1aaa6700 1 -- start start 2023-11-28T11:10:55.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.628+0000 7fcf1aaa6700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcf1406e400 0x7fcf1415db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.628+0000 7fcf1aaa6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcf140ce890 0x7fcf140ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.628+0000 7fcf1aaa6700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf1415e0b0 con 0x7fcf14159590 2023-11-28T11:10:55.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.628+0000 7fcf1aaa6700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf140cf250 con 0x7fcf1406e400 2023-11-28T11:10:55.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.628+0000 7fcf1aaa6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf1415d120 con 0x7fcf140ce890 2023-11-28T11:10:55.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.628+0000 7fcf13fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcf1406e400 0x7fcf1415db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.628+0000 7fcf13fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcf1406e400 0x7fcf1415db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50260/0 (socket says 172.21.15.28:50260) 2023-11-28T11:10:55.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.628+0000 7fcf13fff700 1 -- 172.21.15.28:0/1434707314 learned_addr learned my addr 172.21.15.28:0/1434707314 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:55.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.628+0000 7fcf18842700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fcf14159590 0x7fcf140ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38316/0 (socket says 172.21.15.28:38316) 2023-11-28T11:10:55.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.628+0000 7fcf13fff700 1 -- 172.21.15.28:0/1434707314 >> v2:172.21.15.59:3300/0 conn(0x7fcf140ce890 msgr2=0x7fcf140ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:55.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.628+0000 7fcf13fff700 1 --2- 172.21.15.28:0/1434707314 >> v2:172.21.15.59:3300/0 conn(0x7fcf140ce890 0x7fcf140ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.628+0000 7fcf13fff700 1 -- 172.21.15.28:0/1434707314 >> v1:172.21.15.28:6789/0 conn(0x7fcf14159590 legacy=0x7fcf140ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.629+0000 7fcf13fff700 1 -- 172.21.15.28:0/1434707314 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf1415d6e0 con 0x7fcf1406e400 2023-11-28T11:10:55.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.628+0000 7fcf137fe700 1 -- 172.21.15.28:0/1434707314 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 819333969 0 0) 0x7fcf08001110 con 0x7fcf14159590 2023-11-28T11:10:55.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.629+0000 7fcf13fff700 1 --2- 172.21.15.28:0/1434707314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf1406e400 0x7fcf1415db70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcf140cd500 tx=0x7fcf0000be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e84d55a4b9569ce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:55.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.629+0000 7fcf137fe700 1 -- 172.21.15.28:0/1434707314 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0000dd30 con 0x7fcf1406e400 2023-11-28T11:10:55.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.629+0000 7fcf137fe700 1 -- 172.21.15.28:0/1434707314 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0000de90 con 0x7fcf1406e400 2023-11-28T11:10:55.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.629+0000 7fcf137fe700 1 -- 172.21.15.28:0/1434707314 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf00010690 con 0x7fcf1406e400 2023-11-28T11:10:55.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.629+0000 7fcf1aaa6700 1 -- 172.21.15.28:0/1434707314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf1406e400 msgr2=0x7fcf1415db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.629+0000 7fcf1aaa6700 1 --2- 172.21.15.28:0/1434707314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf1406e400 0x7fcf1415db70 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fcf140cd500 tx=0x7fcf0000be70 comp rx=0 tx=0).stop 2023-11-28T11:10:55.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.629+0000 7fcf1aaa6700 1 -- 172.21.15.28:0/1434707314 shutdown_connections 2023-11-28T11:10:55.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.629+0000 7fcf1aaa6700 1 --2- 172.21.15.28:0/1434707314 >> v2:172.21.15.59:3300/0 conn(0x7fcf140ce890 0x7fcf140ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.629+0000 7fcf1aaa6700 1 --2- 172.21.15.28:0/1434707314 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf1406e400 0x7fcf1415db70 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.629+0000 7fcf1aaa6700 1 -- 172.21.15.28:0/1434707314 >> 172.21.15.28:0/1434707314 conn(0x7fcf140b1650 msgr2=0x7fcf140b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:55.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.629+0000 7fcf1aaa6700 1 -- 172.21.15.28:0/1434707314 shutdown_connections 2023-11-28T11:10:55.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.630+0000 7fcf1aaa6700 1 -- 172.21.15.28:0/1434707314 wait complete. 2023-11-28T11:10:55.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.631+0000 7fcf1aaa6700 1 Processor -- start 2023-11-28T11:10:55.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.631+0000 7fcf1aaa6700 1 -- start start 2023-11-28T11:10:55.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf1aaa6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf1406e400 0x7fcf141588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf1aaa6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf140ce890 0x7fcf140b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf1aaa6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf14159590 0x7fcf140c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf1aaa6700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf140b4bd0 con 0x7fcf14159590 2023-11-28T11:10:55.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf1aaa6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf140b4eb0 con 0x7fcf1406e400 2023-11-28T11:10:55.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf1aaa6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf140b5190 con 0x7fcf140ce890 2023-11-28T11:10:55.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf19043700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf14159590 0x7fcf140c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf19043700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf14159590 0x7fcf140c94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50268/0 (socket says 172.21.15.28:50268) 2023-11-28T11:10:55.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf19043700 1 -- 172.21.15.28:0/1491400520 learned_addr learned my addr 172.21.15.28:0/1491400520 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:55.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf18842700 1 --2- 172.21.15.28:0/1491400520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf1406e400 0x7fcf141588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf13fff700 1 --2- 172.21.15.28:0/1491400520 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf140ce890 0x7fcf140b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf19043700 1 -- 172.21.15.28:0/1491400520 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf140ce890 msgr2=0x7fcf140b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf19043700 1 --2- 172.21.15.28:0/1491400520 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf140ce890 0x7fcf140b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf19043700 1 -- 172.21.15.28:0/1491400520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf1406e400 msgr2=0x7fcf141588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf19043700 1 --2- 172.21.15.28:0/1491400520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf1406e400 0x7fcf141588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf19043700 1 -- 172.21.15.28:0/1491400520 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf0000b7f0 con 0x7fcf14159590 2023-11-28T11:10:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf19043700 1 --2- 172.21.15.28:0/1491400520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf14159590 0x7fcf140c94c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00b970 tx=0x7fcf0c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf117fa700 1 -- 172.21.15.28:0/1491400520 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c00d860 con 0x7fcf14159590 2023-11-28T11:10:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf117fa700 1 -- 172.21.15.28:0/1491400520 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf0c00d9e0 con 0x7fcf14159590 2023-11-28T11:10:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf1aaa6700 1 -- 172.21.15.28:0/1491400520 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf140af240 con 0x7fcf14159590 2023-11-28T11:10:55.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf117fa700 1 -- 172.21.15.28:0/1491400520 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf0c00dd90 con 0x7fcf14159590 2023-11-28T11:10:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.632+0000 7fcf1aaa6700 1 -- 172.21.15.28:0/1491400520 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf140aef60 con 0x7fcf14159590 2023-11-28T11:10:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.633+0000 7fcf1aaa6700 1 -- 172.21.15.28:0/1491400520 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf140cafe0 con 0x7fcf14159590 2023-11-28T11:10:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.634+0000 7fcf117fa700 1 -- 172.21.15.28:0/1491400520 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcf0c00e430 con 0x7fcf14159590 2023-11-28T11:10:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.634+0000 7fcf117fa700 1 --2- 172.21.15.28:0/1491400520 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcef80682f0 0x7fcef806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.634+0000 7fcf117fa700 1 -- 172.21.15.28:0/1491400520 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fcf0c006e40 con 0x7fcf14159590 2023-11-28T11:10:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.634+0000 7fcf18842700 1 --2- 172.21.15.28:0/1491400520 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcef80682f0 0x7fcef806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:55.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.635+0000 7fcf18842700 1 --2- 172.21.15.28:0/1491400520 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcef80682f0 0x7fcef806a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcf08000f60 tx=0x7fcf08007e70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:55.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.639+0000 7fcf117fa700 1 -- 172.21.15.28:0/1491400520 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcf140cafe0 con 0x7fcf14159590 2023-11-28T11:10:55.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.905+0000 7fcf1aaa6700 1 -- 172.21.15.28:0/1491400520 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fcf140cafe0 con 0x7fcf14159590 2023-11-28T11:10:55.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.906+0000 7fcf117fa700 1 -- 172.21.15.28:0/1491400520 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5298 (secure 0 0 0) 0x7fcf140cafe0 con 0x7fcf14159590 2023-11-28T11:10:55.906 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:55.907 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:10:55.907 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_dump": [ 2023-11-28T11:10:55.907 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.907 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.a", 2023-11-28T11:10:55.907 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlx1F8HRAAqlv0qwBqCCSPbuXOK00uLQ==", 2023-11-28T11:10:55.907 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.907 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:55.907 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2023-11-28T11:10:55.907 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2023-11-28T11:10:55.907 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:55.907 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.908 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.908 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.908 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.b", 2023-11-28T11:10:55.908 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlqQd7KhAASvfFg9HJbZLZ48lOZ8rfdg==", 2023-11-28T11:10:55.908 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.908 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:55.908 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2023-11-28T11:10:55.908 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2023-11-28T11:10:55.908 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:55.908 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.908 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.908 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.909 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.c", 2023-11-28T11:10:55.909 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlz/TyJBAAug4qbJp/xfFm3aHFNMSlBA==", 2023-11-28T11:10:55.909 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.909 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:55.909 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2023-11-28T11:10:55.909 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2023-11-28T11:10:55.909 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:55.909 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.909 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.909 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.909 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.d", 2023-11-28T11:10:55.910 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlJA0pMhAAgmUFSzlTLDsYT7SPRaS4hg==", 2023-11-28T11:10:55.910 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.910 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:55.910 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2023-11-28T11:10:55.910 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2023-11-28T11:10:55.910 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:55.910 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.910 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.910 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.910 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.0", 2023-11-28T11:10:55.910 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQALymVlGI9xJBAAFY51VVoRxuhhNHGi8tlG5g==", 2023-11-28T11:10:55.910 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.911 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:55.911 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:55.911 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:55.911 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.911 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.911 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.911 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.1", 2023-11-28T11:10:55.911 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAOymVl6bn5IxAAE02o425v2koATSJg/CKx2Q==", 2023-11-28T11:10:55.911 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.911 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:55.911 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:55.912 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:55.912 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.912 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.912 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.912 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.2", 2023-11-28T11:10:55.912 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQARymVl0JCmIxAASw1qp+1l8/G+7uP2t2Gfcg==", 2023-11-28T11:10:55.912 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.912 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:55.912 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:55.912 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:55.912 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.912 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.914 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.915 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.3", 2023-11-28T11:10:55.916 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAUymVlHpKYIxAAeEFLqUE4cWvE29wkIdb4xQ==", 2023-11-28T11:10:55.916 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.916 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:55.916 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:55.916 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:55.916 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.916 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.916 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.917 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.4", 2023-11-28T11:10:55.923 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAcymVlRw7IDBAACbu1rTbZMEVtwM33xdVPSA==", 2023-11-28T11:10:55.924 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.924 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:55.924 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:55.924 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:55.924 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.925 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.925 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.925 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.5", 2023-11-28T11:10:55.925 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAfymVlDGXHDBAA5lGng1DglyCM+DnWCQ0iUg==", 2023-11-28T11:10:55.925 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.925 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:55.925 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:55.925 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:55.926 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.928 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.928 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.928 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.6", 2023-11-28T11:10:55.928 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAiymVlfw+IDBAAX5cXoXUhrY8BX3tZFVyqCA==", 2023-11-28T11:10:55.929 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.929 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:55.929 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:55.929 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:55.929 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.929 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.929 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.929 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.7", 2023-11-28T11:10:55.929 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAlymVliBSsDBAACyShS6zNLoUAlVFbB5XEag==", 2023-11-28T11:10:55.929 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.929 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:10:55.929 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:10:55.929 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:55.930 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.930 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.930 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.930 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.0", 2023-11-28T11:10:55.930 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVluSHRNhAAmBrJ4shI0iFZ+s0TazdTrw==", 2023-11-28T11:10:55.930 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.930 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:55.930 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow r", 2023-11-28T11:10:55.930 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T11:10:55.930 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.930 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.930 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.934 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.1", 2023-11-28T11:10:55.934 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAKymVl2+otAhAATGAAuzHz3lKdV0enZTr6qA==", 2023-11-28T11:10:55.934 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.934 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:55.934 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2023-11-28T11:10:55.935 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2023-11-28T11:10:55.935 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2023-11-28T11:10:55.935 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.935 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.935 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.935 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.2", 2023-11-28T11:10:55.935 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlxqlTOhAAU7U9b//feat8r6WDQNx/Gg==", 2023-11-28T11:10:55.935 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.935 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:55.935 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2023-11-28T11:10:55.935 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2023-11-28T11:10:55.936 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2023-11-28T11:10:55.936 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.936 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.936 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.936 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.3", 2023-11-28T11:10:55.936 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAKymVl592XBRAASQZMNs5hbHPm2CYq2ktYdg==", 2023-11-28T11:10:55.936 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.936 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:10:55.936 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2023-11-28T11:10:55.936 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2023-11-28T11:10:55.936 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2023-11-28T11:10:55.936 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.937 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.937 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.937 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.admin", 2023-11-28T11:10:55.937 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAIymVl5ZzXOhAAt2N6VJgsHFGvT1U6Uee9VA==", 2023-11-28T11:10:55.937 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.937 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2023-11-28T11:10:55.937 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2023-11-28T11:10:55.937 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2023-11-28T11:10:55.937 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:55.937 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.938 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.938 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.938 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.x", 2023-11-28T11:10:55.938 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlZrj9GRAAXu1z37jFMZo3YXUJLRB2+w==", 2023-11-28T11:10:55.938 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.938 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2023-11-28T11:10:55.938 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2023-11-28T11:10:55.938 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:55.938 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.938 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:55.939 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:55.939 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.y", 2023-11-28T11:10:55.939 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlyBOJFRAA6cjReBkYUD/3YOUHw7DHpg==", 2023-11-28T11:10:55.939 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:10:55.939 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2023-11-28T11:10:55.939 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2023-11-28T11:10:55.939 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:10:55.939 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.939 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:55.939 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2023-11-28T11:10:55.939 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:10:55.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.908+0000 7fcf1aaa6700 1 -- 172.21.15.28:0/1491400520 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcef80682f0 msgr2=0x7fcef806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.908+0000 7fcf1aaa6700 1 --2- 172.21.15.28:0/1491400520 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcef80682f0 0x7fcef806a7a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fcf08000f60 tx=0x7fcf08007e70 comp rx=0 tx=0).stop 2023-11-28T11:10:55.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.908+0000 7fcf1aaa6700 1 -- 172.21.15.28:0/1491400520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf14159590 msgr2=0x7fcf140c94c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:55.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.908+0000 7fcf1aaa6700 1 --2- 172.21.15.28:0/1491400520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf14159590 0x7fcf140c94c0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fcf0c00b970 tx=0x7fcf0c00bc80 comp rx=0 tx=0).stop 2023-11-28T11:10:55.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.908+0000 7fcf1aaa6700 1 -- 172.21.15.28:0/1491400520 shutdown_connections 2023-11-28T11:10:55.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.908+0000 7fcf1aaa6700 1 --2- 172.21.15.28:0/1491400520 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcef80682f0 0x7fcef806a7a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.908+0000 7fcf1aaa6700 1 --2- 172.21.15.28:0/1491400520 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf14159590 0x7fcf140c94c0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.908+0000 7fcf1aaa6700 1 --2- 172.21.15.28:0/1491400520 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf1406e400 0x7fcf141588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.908+0000 7fcf1aaa6700 1 --2- 172.21.15.28:0/1491400520 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf140ce890 0x7fcf140b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:55.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.908+0000 7fcf1aaa6700 1 -- 172.21.15.28:0/1491400520 >> 172.21.15.28:0/1491400520 conn(0x7fcf140b1650 msgr2=0x7fcf14152f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:55.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.908+0000 7fcf1aaa6700 1 -- 172.21.15.28:0/1491400520 shutdown_connections 2023-11-28T11:10:55.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:55.908+0000 7fcf1aaa6700 1 -- 172.21.15.28:0/1491400520 wait complete. 2023-11-28T11:10:55.943 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:56.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.069+0000 7f1ee965d700 1 Processor -- start 2023-11-28T11:10:56.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.070+0000 7f1ee965d700 1 -- start start 2023-11-28T11:10:56.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.070+0000 7f1ee965d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ee415a230 0x7f1ee415d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.070+0000 7f1ee965d700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee406abb0 con 0x7f1ee40b4fe0 2023-11-28T11:10:56.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.070+0000 7f1ee965d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ee406a280 con 0x7f1ee40cd4a0 2023-11-28T11:10:56.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.070+0000 7f1ee965d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee415a600 con 0x7f1ee415a230 2023-11-28T11:10:56.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.070+0000 7f1ee37fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1ee40b4fe0 0x7f1ee40d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38332/0 (socket says 172.21.15.28:38332) 2023-11-28T11:10:56.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.070+0000 7f1ee37fe700 1 -- 172.21.15.28:0/1497931763 learned_addr learned my addr 172.21.15.28:0/1497931763 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:56.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee2ffd700 1 --2- 172.21.15.28:0/1497931763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ee415a230 0x7f1ee415d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee1ffb700 1 -- 172.21.15.28:0/1497931763 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050129122 0 0) 0x7f1ee406abb0 con 0x7f1ee40b4fe0 2023-11-28T11:10:56.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee1ffb700 1 -- 172.21.15.28:0/1497931763 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ecc0035b0 con 0x7f1ee40b4fe0 2023-11-28T11:10:56.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee1ffb700 1 -- 172.21.15.28:0/1497931763 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3858629961 0 0) 0x7f1ecc0035b0 con 0x7f1ee40b4fe0 2023-11-28T11:10:56.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee1ffb700 1 -- 172.21.15.28:0/1497931763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ee415a230 msgr2=0x7f1ee415d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:56.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee1ffb700 1 --2- 172.21.15.28:0/1497931763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ee415a230 0x7f1ee415d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee1ffb700 1 -- 172.21.15.28:0/1497931763 >> v1:172.21.15.59:6789/0 conn(0x7f1ee40cd4a0 legacy=0x7f1ee40b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee1ffb700 1 -- 172.21.15.28:0/1497931763 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee406a560 con 0x7f1ee40b4fe0 2023-11-28T11:10:56.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee1ffb700 1 -- 172.21.15.28:0/1497931763 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f1edc0019f0 con 0x7f1ee40b4fe0 2023-11-28T11:10:56.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee1ffb700 1 -- 172.21.15.28:0/1497931763 >> v1:172.21.15.28:6789/0 conn(0x7f1ee40b4fe0 legacy=0x7f1ee40d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee1ffb700 1 --2- 172.21.15.28:0/1497931763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ecc003a30 0x7f1ecc003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee1ffb700 1 --2- 172.21.15.28:0/1497931763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ee415a230 0x7f1ecc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee1ffb700 1 --2- 172.21.15.28:0/1497931763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ecc004d90 0x7f1ecc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee1ffb700 1 -- 172.21.15.28:0/1497931763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee415a600 con 0x7f1ecc004d90 2023-11-28T11:10:56.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee1ffb700 1 -- 172.21.15.28:0/1497931763 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc00baf0 con 0x7f1ecc003a30 2023-11-28T11:10:56.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee1ffb700 1 -- 172.21.15.28:0/1497931763 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ecc00bcb0 con 0x7f1ee415a230 2023-11-28T11:10:56.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee37fe700 1 --2- 172.21.15.28:0/1497931763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ecc004d90 0x7f1ecc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee2ffd700 1 --2- 172.21.15.28:0/1497931763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ecc003a30 0x7f1ecc003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.071+0000 7f1ee27fc700 1 --2- 172.21.15.28:0/1497931763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ee415a230 0x7f1ecc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.072+0000 7f1ee37fe700 1 -- 172.21.15.28:0/1497931763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ee415a230 msgr2=0x7f1ecc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:56.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.072+0000 7f1ee37fe700 1 --2- 172.21.15.28:0/1497931763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ee415a230 0x7f1ecc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.072+0000 7f1ee37fe700 1 -- 172.21.15.28:0/1497931763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ecc003a30 msgr2=0x7f1ecc003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:56.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.072+0000 7f1ee37fe700 1 --2- 172.21.15.28:0/1497931763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ecc003a30 0x7f1ecc003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.072+0000 7f1ee37fe700 1 -- 172.21.15.28:0/1497931763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ecc0035b0 con 0x7f1ecc004d90 2023-11-28T11:10:56.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.072+0000 7f1ee27fc700 1 --2- 172.21.15.28:0/1497931763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ee415a230 0x7f1ecc004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:56.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.072+0000 7f1ee37fe700 1 --2- 172.21.15.28:0/1497931763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ecc004d90 0x7f1ecc005200 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1edc005a80 tx=0x7f1ecc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ddb45b31625193 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.072+0000 7f1ee1ffb700 1 -- 172.21.15.28:0/1497931763 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc005520 con 0x7f1ecc004d90 2023-11-28T11:10:56.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.072+0000 7f1ee1ffb700 1 -- 172.21.15.28:0/1497931763 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc00dab0 con 0x7f1ecc004d90 2023-11-28T11:10:56.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.072+0000 7f1ee1ffb700 1 -- 172.21.15.28:0/1497931763 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00de60 con 0x7f1ecc004d90 2023-11-28T11:10:56.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.072+0000 7f1ee965d700 1 -- 172.21.15.28:0/1497931763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ecc004d90 msgr2=0x7f1ecc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.072+0000 7f1ee965d700 1 --2- 172.21.15.28:0/1497931763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ecc004d90 0x7f1ecc005200 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1edc005a80 tx=0x7f1ecc005880 comp rx=0 tx=0).stop 2023-11-28T11:10:56.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.072+0000 7f1ee37fe700 1 -- 172.21.15.28:0/1497931763 reap_dead start 2023-11-28T11:10:56.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.072+0000 7f1ee965d700 1 -- 172.21.15.28:0/1497931763 shutdown_connections 2023-11-28T11:10:56.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.072+0000 7f1ee965d700 1 -- 172.21.15.28:0/1497931763 >> 172.21.15.28:0/1497931763 conn(0x7f1ee40cf8d0 msgr2=0x7f1ee40cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:56.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.073+0000 7f1ee965d700 1 -- 172.21.15.28:0/1497931763 shutdown_connections 2023-11-28T11:10:56.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.073+0000 7f1ee965d700 1 -- 172.21.15.28:0/1497931763 wait complete. 2023-11-28T11:10:56.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.074+0000 7f1ee965d700 1 Processor -- start 2023-11-28T11:10:56.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.074+0000 7f1ee965d700 1 -- start start 2023-11-28T11:10:56.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.075+0000 7f1ee965d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ee41491c0 0x7f1ee4156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.075+0000 7f1ee965d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ee4149590 0x7f1ee40ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.075+0000 7f1ee965d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ee40cc340 0x7f1ee40cc7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.075+0000 7f1ee965d700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee40c78d0 con 0x7f1ee4149590 2023-11-28T11:10:56.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.075+0000 7f1ee965d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee40ca610 con 0x7f1ee40cc340 2023-11-28T11:10:56.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.075+0000 7f1ee965d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee40cae10 con 0x7f1ee41491c0 2023-11-28T11:10:56.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.075+0000 7f1ee27fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ee4149590 0x7f1ee40ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.075+0000 7f1ee27fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ee4149590 0x7f1ee40ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50284/0 (socket says 172.21.15.28:50284) 2023-11-28T11:10:56.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.075+0000 7f1ee27fc700 1 -- 172.21.15.28:0/1925387686 learned_addr learned my addr 172.21.15.28:0/1925387686 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:56.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.076+0000 7f1ee2ffd700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ee41491c0 0x7f1ee4156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.076+0000 7f1ee27fc700 1 -- 172.21.15.28:0/1925387686 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ee41491c0 msgr2=0x7f1ee4156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.076+0000 7f1ee37fe700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ee40cc340 0x7f1ee40cc7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.076+0000 7f1ee27fc700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ee41491c0 0x7f1ee4156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.076+0000 7f1ee27fc700 1 -- 172.21.15.28:0/1925387686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ee40cc340 msgr2=0x7f1ee40cc7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.076+0000 7f1ee27fc700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ee40cc340 0x7f1ee40cc7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.076+0000 7f1ee27fc700 1 -- 172.21.15.28:0/1925387686 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee40cd910 con 0x7f1ee4149590 2023-11-28T11:10:56.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.076+0000 7f1ee2ffd700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ee41491c0 0x7f1ee4156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:56.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.076+0000 7f1ee37fe700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ee40cc340 0x7f1ee40cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:10:56.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.076+0000 7f1ee27fc700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ee4149590 0x7f1ee40ca8d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400e780 tx=0x7f1ed400eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.076+0000 7f1ecb7fe700 1 -- 172.21.15.28:0/1925387686 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40052e0 con 0x7f1ee4149590 2023-11-28T11:10:56.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.076+0000 7f1ecb7fe700 1 -- 172.21.15.28:0/1925387686 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ed4005440 con 0x7f1ee4149590 2023-11-28T11:10:56.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.076+0000 7f1ee965d700 1 -- 172.21.15.28:0/1925387686 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ee406a840 con 0x7f1ee4149590 2023-11-28T11:10:56.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.076+0000 7f1ecb7fe700 1 -- 172.21.15.28:0/1925387686 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ed40038c0 con 0x7f1ee4149590 2023-11-28T11:10:56.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.076+0000 7f1ee965d700 1 -- 172.21.15.28:0/1925387686 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee406b040 con 0x7f1ee4149590 2023-11-28T11:10:56.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.077+0000 7f1ee965d700 1 -- 172.21.15.28:0/1925387686 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ee40675a0 con 0x7f1ee4149590 2023-11-28T11:10:56.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.077+0000 7f1ecb7fe700 1 -- 172.21.15.28:0/1925387686 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1ed401b030 con 0x7f1ee4149590 2023-11-28T11:10:56.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.078+0000 7f1ecb7fe700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1ed00682f0 0x7f1ed006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.078+0000 7f1ecb7fe700 1 -- 172.21.15.28:0/1925387686 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f1ed40850d0 con 0x7f1ee4149590 2023-11-28T11:10:56.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.078+0000 7f1ee2ffd700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1ed00682f0 0x7f1ed006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.078+0000 7f1ee2ffd700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1ed00682f0 0x7f1ed006a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40c9980 tx=0x7f1ed8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.091 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.082+0000 7f1ecb7fe700 1 -- 172.21.15.28:0/1925387686 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1ed404d5e0 con 0x7f1ee4149590 2023-11-28T11:10:56.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.278+0000 7f1ee965d700 1 -- 172.21.15.28:0/1925387686 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1ee40c7a10 con 0x7f1ee4149590 2023-11-28T11:10:56.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.278+0000 7f1ecb7fe700 1 -- 172.21.15.28:0/1925387686 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+14366 (secure 0 0 0) 0x7f1ed4051b70 con 0x7f1ee4149590 2023-11-28T11:10:56.279 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:56.279 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":50,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:10:55.471665+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":38,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:10:56.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.281+0000 7f1ee965d700 1 -- 172.21.15.28:0/1925387686 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1ed00682f0 msgr2=0x7f1ed006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.281+0000 7f1ee965d700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1ed00682f0 0x7f1ed006a7a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f1ee40c9980 tx=0x7f1ed8008040 comp rx=0 tx=0).stop 2023-11-28T11:10:56.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.281+0000 7f1ee965d700 1 -- 172.21.15.28:0/1925387686 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ee4149590 msgr2=0x7f1ee40ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.281+0000 7f1ee965d700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ee4149590 0x7f1ee40ca8d0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1ed400e780 tx=0x7f1ed400eb40 comp rx=0 tx=0).stop 2023-11-28T11:10:56.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.281+0000 7f1ee965d700 1 -- 172.21.15.28:0/1925387686 shutdown_connections 2023-11-28T11:10:56.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.281+0000 7f1ee965d700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1ed00682f0 0x7f1ed006a7a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.281+0000 7f1ee965d700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ee40cc340 0x7f1ee40cc7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.281+0000 7f1ee965d700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ee41491c0 0x7f1ee4156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.281+0000 7f1ee965d700 1 --2- 172.21.15.28:0/1925387686 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ee4149590 0x7f1ee40ca8d0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.281+0000 7f1ee965d700 1 -- 172.21.15.28:0/1925387686 >> 172.21.15.28:0/1925387686 conn(0x7f1ee40cf8d0 msgr2=0x7f1ee415e180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:56.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.281+0000 7f1ee965d700 1 -- 172.21.15.28:0/1925387686 shutdown_connections 2023-11-28T11:10:56.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.281+0000 7f1ee965d700 1 -- 172.21.15.28:0/1925387686 wait complete. 2023-11-28T11:10:56.294 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:10:56.294 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:10:56.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.424+0000 7f6fd959c700 1 Processor -- start 2023-11-28T11:10:56.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.424+0000 7f6fd959c700 1 -- start start 2023-11-28T11:10:56.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.425+0000 7f6fd959c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40d2820 0x7f6fd40b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.425+0000 7f6fd959c700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd40d1470 con 0x7f6fd406bf30 2023-11-28T11:10:56.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.425+0000 7f6fd959c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fd40cd4e0 con 0x7f6fd415c640 2023-11-28T11:10:56.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.425+0000 7f6fd959c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd415ca10 con 0x7f6fd40d2820 2023-11-28T11:10:56.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.425+0000 7f6fd37fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6fd406bf30 0x7f6fd406c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38352/0 (socket says 172.21.15.28:38352) 2023-11-28T11:10:56.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.425+0000 7f6fd37fe700 1 -- 172.21.15.28:0/1724937842 learned_addr learned my addr 172.21.15.28:0/1724937842 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:56.429 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.425+0000 7f6fd27fc700 1 --2- 172.21.15.28:0/1724937842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40d2820 0x7f6fd40b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.425+0000 7f6fd1ffb700 1 -- 172.21.15.28:0/1724937842 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 697670566 0 0) 0x7f6fd40d1470 con 0x7f6fd406bf30 2023-11-28T11:10:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.425+0000 7f6fd1ffb700 1 -- 172.21.15.28:0/1724937842 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fbc0035b0 con 0x7f6fd406bf30 2023-11-28T11:10:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.425+0000 7f6fd1ffb700 1 -- 172.21.15.28:0/1724937842 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3298037802 0 0) 0x7f6fbc0035b0 con 0x7f6fd406bf30 2023-11-28T11:10:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.425+0000 7f6fd1ffb700 1 -- 172.21.15.28:0/1724937842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40d2820 msgr2=0x7f6fd40b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.425+0000 7f6fd1ffb700 1 --2- 172.21.15.28:0/1724937842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40d2820 0x7f6fd40b5c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.425+0000 7f6fd1ffb700 1 -- 172.21.15.28:0/1724937842 >> v1:172.21.15.59:6789/0 conn(0x7f6fd415c640 legacy=0x7f6fd40cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd1ffb700 1 -- 172.21.15.28:0/1724937842 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd40cd7c0 con 0x7f6fd406bf30 2023-11-28T11:10:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd1ffb700 1 -- 172.21.15.28:0/1724937842 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f6fcc0019f0 con 0x7f6fd406bf30 2023-11-28T11:10:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd1ffb700 1 -- 172.21.15.28:0/1724937842 >> v1:172.21.15.28:6789/0 conn(0x7f6fd406bf30 legacy=0x7f6fd406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd1ffb700 1 --2- 172.21.15.28:0/1724937842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fbc003a30 0x7f6fbc003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd1ffb700 1 --2- 172.21.15.28:0/1724937842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40d2820 0x7f6fbc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd1ffb700 1 --2- 172.21.15.28:0/1724937842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fbc004d90 0x7f6fbc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd1ffb700 1 -- 172.21.15.28:0/1724937842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd415ca10 con 0x7f6fbc004d90 2023-11-28T11:10:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd1ffb700 1 -- 172.21.15.28:0/1724937842 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc00bb80 con 0x7f6fbc003a30 2023-11-28T11:10:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd1ffb700 1 -- 172.21.15.28:0/1724937842 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fbc00bd40 con 0x7f6fd40d2820 2023-11-28T11:10:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd37fe700 1 --2- 172.21.15.28:0/1724937842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fbc004d90 0x7f6fbc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd2ffd700 1 --2- 172.21.15.28:0/1724937842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fbc003a30 0x7f6fbc003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd27fc700 1 --2- 172.21.15.28:0/1724937842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40d2820 0x7f6fbc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd37fe700 1 -- 172.21.15.28:0/1724937842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40d2820 msgr2=0x7f6fbc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd37fe700 1 --2- 172.21.15.28:0/1724937842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40d2820 0x7f6fbc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd37fe700 1 -- 172.21.15.28:0/1724937842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fbc003a30 msgr2=0x7f6fbc003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd37fe700 1 --2- 172.21.15.28:0/1724937842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fbc003a30 0x7f6fbc003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd37fe700 1 -- 172.21.15.28:0/1724937842 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fbc0035b0 con 0x7f6fbc004d90 2023-11-28T11:10:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd27fc700 1 --2- 172.21.15.28:0/1724937842 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fd40d2820 0x7f6fbc004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd37fe700 1 --2- 172.21.15.28:0/1724937842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fbc004d90 0x7f6fbc005200 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00a820 tx=0x7f6fbc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f75e7104d539460 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd1ffb700 1 -- 172.21.15.28:0/1724937842 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc00a2f0 con 0x7f6fbc004d90 2023-11-28T11:10:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd1ffb700 1 -- 172.21.15.28:0/1724937842 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fcc012600 con 0x7f6fbc004d90 2023-11-28T11:10:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.426+0000 7f6fd1ffb700 1 -- 172.21.15.28:0/1724937842 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc0129b0 con 0x7f6fbc004d90 2023-11-28T11:10:56.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.427+0000 7f6fd959c700 1 -- 172.21.15.28:0/1724937842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fbc004d90 msgr2=0x7f6fbc005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.427+0000 7f6fd959c700 1 --2- 172.21.15.28:0/1724937842 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fbc004d90 0x7f6fbc005200 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00a820 tx=0x7f6fbc005880 comp rx=0 tx=0).stop 2023-11-28T11:10:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.427+0000 7f6fd37fe700 1 -- 172.21.15.28:0/1724937842 reap_dead start 2023-11-28T11:10:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.427+0000 7f6fd959c700 1 -- 172.21.15.28:0/1724937842 shutdown_connections 2023-11-28T11:10:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.427+0000 7f6fd959c700 1 -- 172.21.15.28:0/1724937842 >> 172.21.15.28:0/1724937842 conn(0x7f6fd40d2df0 msgr2=0x7f6fd40d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:56.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.427+0000 7f6fd959c700 1 -- 172.21.15.28:0/1724937842 shutdown_connections 2023-11-28T11:10:56.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.427+0000 7f6fd959c700 1 -- 172.21.15.28:0/1724937842 wait complete. 2023-11-28T11:10:56.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.429+0000 7f6fd959c700 1 Processor -- start 2023-11-28T11:10:56.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.429+0000 7f6fd959c700 1 -- start start 2023-11-28T11:10:56.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd959c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fbc004d90 0x7f6fd40c9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd959c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd41589e0 0x7f6fd41581b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd959c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fd40c68f0 0x7f6fd40c6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd959c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd40c72a0 con 0x7f6fd40c68f0 2023-11-28T11:10:56.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd959c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd415f5a0 con 0x7f6fd41589e0 2023-11-28T11:10:56.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd959c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd415ed80 con 0x7f6fbc004d90 2023-11-28T11:10:56.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd37fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fd40c68f0 0x7f6fd40c6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd37fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fd40c68f0 0x7f6fd40c6d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50306/0 (socket says 172.21.15.28:50306) 2023-11-28T11:10:56.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd37fe700 1 -- 172.21.15.28:0/1248929381 learned_addr learned my addr 172.21.15.28:0/1248929381 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:56.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd27fc700 1 --2- 172.21.15.28:0/1248929381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd41589e0 0x7f6fd41581b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd2ffd700 1 --2- 172.21.15.28:0/1248929381 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fbc004d90 0x7f6fd40c9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd37fe700 1 -- 172.21.15.28:0/1248929381 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fbc004d90 msgr2=0x7f6fd40c9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd37fe700 1 --2- 172.21.15.28:0/1248929381 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fbc004d90 0x7f6fd40c9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd37fe700 1 -- 172.21.15.28:0/1248929381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd41589e0 msgr2=0x7f6fd41581b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd37fe700 1 --2- 172.21.15.28:0/1248929381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd41589e0 0x7f6fd41581b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd37fe700 1 -- 172.21.15.28:0/1248929381 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd40cdaa0 con 0x7f6fd40c68f0 2023-11-28T11:10:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd37fe700 1 --2- 172.21.15.28:0/1248929381 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fd40c68f0 0x7f6fd40c6d60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00ae10 tx=0x7f6fd40cef90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fbb7fe700 1 -- 172.21.15.28:0/1248929381 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc0017b0 con 0x7f6fd40c68f0 2023-11-28T11:10:56.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fbb7fe700 1 -- 172.21.15.28:0/1248929381 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fcc001950 con 0x7f6fd40c68f0 2023-11-28T11:10:56.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fd959c700 1 -- 172.21.15.28:0/1248929381 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fd40d1710 con 0x7f6fd40c68f0 2023-11-28T11:10:56.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.430+0000 7f6fbb7fe700 1 -- 172.21.15.28:0/1248929381 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fcc01e670 con 0x7f6fd40c68f0 2023-11-28T11:10:56.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.431+0000 7f6fd959c700 1 -- 172.21.15.28:0/1248929381 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fd40cdc30 con 0x7f6fd40c68f0 2023-11-28T11:10:56.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.431+0000 7f6fd959c700 1 -- 172.21.15.28:0/1248929381 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fd40739f0 con 0x7f6fd40c68f0 2023-11-28T11:10:56.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.432+0000 7f6fbb7fe700 1 -- 172.21.15.28:0/1248929381 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6fcc02d070 con 0x7f6fd40c68f0 2023-11-28T11:10:56.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.433+0000 7f6fbb7fe700 1 --2- 172.21.15.28:0/1248929381 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6fc00682f0 0x7f6fc006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.433+0000 7f6fbb7fe700 1 -- 172.21.15.28:0/1248929381 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f6fcc087cc0 con 0x7f6fd40c68f0 2023-11-28T11:10:56.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.433+0000 7f6fd2ffd700 1 --2- 172.21.15.28:0/1248929381 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6fc00682f0 0x7f6fc006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.433+0000 7f6fd2ffd700 1 --2- 172.21.15.28:0/1248929381 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6fc00682f0 0x7f6fc006a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8007fd0 tx=0x7f6fc8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.437+0000 7f6fbb7fe700 1 -- 172.21.15.28:0/1248929381 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6fcc0547d0 con 0x7f6fd40c68f0 2023-11-28T11:10:56.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.630+0000 7f6fd959c700 1 -- 172.21.15.28:0/1248929381 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6fd415f7f0 con 0x7f6fd40c68f0 2023-11-28T11:10:56.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.630+0000 7f6fbb7fe700 1 -- 172.21.15.28:0/1248929381 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v50) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6fcc02b0a0 con 0x7f6fd40c68f0 2023-11-28T11:10:56.631 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:10:56.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.632+0000 7f6fd959c700 1 -- 172.21.15.28:0/1248929381 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6fc00682f0 msgr2=0x7f6fc006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.632+0000 7f6fd959c700 1 --2- 172.21.15.28:0/1248929381 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6fc00682f0 0x7f6fc006a7a0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f6fc8007fd0 tx=0x7f6fc8009040 comp rx=0 tx=0).stop 2023-11-28T11:10:56.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.632+0000 7f6fd959c700 1 -- 172.21.15.28:0/1248929381 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fd40c68f0 msgr2=0x7f6fd40c6d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.632+0000 7f6fd959c700 1 --2- 172.21.15.28:0/1248929381 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fd40c68f0 0x7f6fd40c6d60 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f6fcc00ae10 tx=0x7f6fd40cef90 comp rx=0 tx=0).stop 2023-11-28T11:10:56.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.632+0000 7f6fd959c700 1 -- 172.21.15.28:0/1248929381 shutdown_connections 2023-11-28T11:10:56.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.632+0000 7f6fd959c700 1 --2- 172.21.15.28:0/1248929381 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6fc00682f0 0x7f6fc006a7a0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.632+0000 7f6fd959c700 1 --2- 172.21.15.28:0/1248929381 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6fd40c68f0 0x7f6fd40c6d60 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.632+0000 7f6fd959c700 1 --2- 172.21.15.28:0/1248929381 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6fbc004d90 0x7f6fd40c9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.632+0000 7f6fd959c700 1 --2- 172.21.15.28:0/1248929381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6fd41589e0 0x7f6fd41581b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.632+0000 7f6fd959c700 1 -- 172.21.15.28:0/1248929381 >> 172.21.15.28:0/1248929381 conn(0x7f6fd40d2df0 msgr2=0x7f6fd40d1b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:56.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.632+0000 7f6fd959c700 1 -- 172.21.15.28:0/1248929381 shutdown_connections 2023-11-28T11:10:56.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.632+0000 7f6fd959c700 1 -- 172.21.15.28:0/1248929381 wait complete. 2023-11-28T11:10:56.645 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:10:56.772 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.772+0000 7f3f4c2a3700 1 Processor -- start 2023-11-28T11:10:56.772 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.772+0000 7f3f4c2a3700 1 -- start start 2023-11-28T11:10:56.775 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.773+0000 7f3f4c2a3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f4415c950 0x7f3f4406a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.775 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.773+0000 7f3f4c2a3700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3f440ca360 0x7f3f4406f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.775 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.773+0000 7f3f4c2a3700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f440caab0 con 0x7f3f440cae50 2023-11-28T11:10:56.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.773+0000 7f3f4c2a3700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f4415cd20 con 0x7f3f440ca360 2023-11-28T11:10:56.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.773+0000 7f3f4c2a3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4406e770 con 0x7f3f4415c950 2023-11-28T11:10:56.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.773+0000 7f3f4983e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3f440ca360 0x7f3f4406f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.773+0000 7f3f4983e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3f440ca360 0x7f3f4406f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50310/0 (socket says 172.21.15.28:50310) 2023-11-28T11:10:56.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.773+0000 7f3f4a840700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3f440cae50 0x7f3f440cced0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38372/0 (socket says 172.21.15.28:38372) 2023-11-28T11:10:56.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.773+0000 7f3f4983e700 1 -- 172.21.15.28:0/2019460140 learned_addr learned my addr 172.21.15.28:0/2019460140 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:56.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.773+0000 7f3f4a03f700 1 --2- 172.21.15.28:0/2019460140 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f4415c950 0x7f3f4406a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.773+0000 7f3f4983e700 1 -- 172.21.15.28:0/2019460140 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f4415c950 msgr2=0x7f3f4406a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:56.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.773+0000 7f3f4983e700 1 --2- 172.21.15.28:0/2019460140 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f4415c950 0x7f3f4406a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.773+0000 7f3f4983e700 1 -- 172.21.15.28:0/2019460140 >> v1:172.21.15.28:6789/0 conn(0x7f3f440cae50 legacy=0x7f3f440cced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.773+0000 7f3f4983e700 1 -- 172.21.15.28:0/2019460140 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f440ca7d0 con 0x7f3f440ca360 2023-11-28T11:10:56.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.773+0000 7f3f4903d700 1 -- 172.21.15.28:0/2019460140 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626283454 0 0) 0x7f3f440caab0 con 0x7f3f440cae50 2023-11-28T11:10:56.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.774+0000 7f3f4983e700 1 --2- 172.21.15.28:0/2019460140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f440ca360 0x7f3f4406f4f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3f400098f0 tx=0x7f3f40009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad16ab31222a4de6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.774+0000 7f3f4903d700 1 -- 172.21.15.28:0/2019460140 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40011830 con 0x7f3f440ca360 2023-11-28T11:10:56.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.774+0000 7f3f4903d700 1 -- 172.21.15.28:0/2019460140 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f40011990 con 0x7f3f440ca360 2023-11-28T11:10:56.778 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.774+0000 7f3f4903d700 1 -- 172.21.15.28:0/2019460140 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40011d20 con 0x7f3f440ca360 2023-11-28T11:10:56.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.774+0000 7f3f4c2a3700 1 -- 172.21.15.28:0/2019460140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f440ca360 msgr2=0x7f3f4406f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.774+0000 7f3f4c2a3700 1 --2- 172.21.15.28:0/2019460140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f440ca360 0x7f3f4406f4f0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f3f400098f0 tx=0x7f3f40009c00 comp rx=0 tx=0).stop 2023-11-28T11:10:56.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.774+0000 7f3f4c2a3700 1 -- 172.21.15.28:0/2019460140 shutdown_connections 2023-11-28T11:10:56.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.774+0000 7f3f4c2a3700 1 --2- 172.21.15.28:0/2019460140 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f4415c950 0x7f3f4406a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.774+0000 7f3f4c2a3700 1 --2- 172.21.15.28:0/2019460140 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f440ca360 0x7f3f4406f4f0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.774+0000 7f3f4c2a3700 1 -- 172.21.15.28:0/2019460140 >> 172.21.15.28:0/2019460140 conn(0x7f3f440b0df0 msgr2=0x7f3f440b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:56.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.774+0000 7f3f4c2a3700 1 -- 172.21.15.28:0/2019460140 shutdown_connections 2023-11-28T11:10:56.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.774+0000 7f3f4c2a3700 1 -- 172.21.15.28:0/2019460140 wait complete. 2023-11-28T11:10:56.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.776+0000 7f3f4c2a3700 1 Processor -- start 2023-11-28T11:10:56.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.776+0000 7f3f4c2a3700 1 -- start start 2023-11-28T11:10:56.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.776+0000 7f3f4c2a3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f440ca360 0x7f3f4414c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.776+0000 7f3f4c2a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f440cae50 0x7f3f4414d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.776+0000 7f3f4c2a3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f4415c950 0x7f3f44071110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.776+0000 7f3f4c2a3700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4414c580 con 0x7f3f440ca360 2023-11-28T11:10:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.776+0000 7f3f4c2a3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4414c870 con 0x7f3f440cae50 2023-11-28T11:10:56.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.776+0000 7f3f4c2a3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f4414cb50 con 0x7f3f4415c950 2023-11-28T11:10:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.776+0000 7f3f4a03f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f440ca360 0x7f3f4414c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.776+0000 7f3f4a03f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f440ca360 0x7f3f4414c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50316/0 (socket says 172.21.15.28:50316) 2023-11-28T11:10:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.776+0000 7f3f4a03f700 1 -- 172.21.15.28:0/453323021 learned_addr learned my addr 172.21.15.28:0/453323021 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:56.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.777+0000 7f3f4a03f700 1 -- 172.21.15.28:0/453323021 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f4415c950 msgr2=0x7f3f44071110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.777+0000 7f3f4a840700 1 --2- 172.21.15.28:0/453323021 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f4415c950 0x7f3f44071110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.777+0000 7f3f4983e700 1 --2- 172.21.15.28:0/453323021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f440cae50 0x7f3f4414d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.777+0000 7f3f4a03f700 1 --2- 172.21.15.28:0/453323021 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f4415c950 0x7f3f44071110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.777+0000 7f3f4a03f700 1 -- 172.21.15.28:0/453323021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f440cae50 msgr2=0x7f3f4414d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.777+0000 7f3f4a03f700 1 --2- 172.21.15.28:0/453323021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f440cae50 0x7f3f4414d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.777+0000 7f3f4a03f700 1 -- 172.21.15.28:0/453323021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f40009550 con 0x7f3f440ca360 2023-11-28T11:10:56.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.777+0000 7f3f4a03f700 1 --2- 172.21.15.28:0/453323021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f440ca360 0x7f3f4414c040 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400e870 tx=0x7f3f3400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.777+0000 7f3f3affd700 1 -- 172.21.15.28:0/453323021 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f34009590 con 0x7f3f440ca360 2023-11-28T11:10:56.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.777+0000 7f3f3affd700 1 -- 172.21.15.28:0/453323021 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3400ee70 con 0x7f3f440ca360 2023-11-28T11:10:56.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.777+0000 7f3f4c2a3700 1 -- 172.21.15.28:0/453323021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f4406ecf0 con 0x7f3f440ca360 2023-11-28T11:10:56.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.777+0000 7f3f3affd700 1 -- 172.21.15.28:0/453323021 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3400ca50 con 0x7f3f440ca360 2023-11-28T11:10:56.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.777+0000 7f3f4c2a3700 1 -- 172.21.15.28:0/453323021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f4406ea10 con 0x7f3f440ca360 2023-11-28T11:10:56.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.778+0000 7f3f4c2a3700 1 -- 172.21.15.28:0/453323021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4406efd0 con 0x7f3f440ca360 2023-11-28T11:10:56.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.778+0000 7f3f3affd700 1 -- 172.21.15.28:0/453323021 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f34019030 con 0x7f3f440ca360 2023-11-28T11:10:56.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.779+0000 7f3f3affd700 1 --2- 172.21.15.28:0/453323021 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3f280682f0 0x7f3f2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:56.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.779+0000 7f3f3affd700 1 -- 172.21.15.28:0/453323021 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3f34085910 con 0x7f3f440ca360 2023-11-28T11:10:56.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.779+0000 7f3f4983e700 1 --2- 172.21.15.28:0/453323021 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3f280682f0 0x7f3f2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:56.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.779+0000 7f3f4983e700 1 --2- 172.21.15.28:0/453323021 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3f280682f0 0x7f3f2806a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3f440b5200 tx=0x7f3f40008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:56.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.783+0000 7f3f3affd700 1 -- 172.21.15.28:0/453323021 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3f34052380 con 0x7f3f440ca360 2023-11-28T11:10:56.974 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.974+0000 7f3f4c2a3700 1 -- 172.21.15.28:0/453323021 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3f4414d760 con 0x7f3f440ca360 2023-11-28T11:10:56.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.974+0000 7f3f3affd700 1 -- 172.21.15.28:0/453323021 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3f340559a0 con 0x7f3f440ca360 2023-11-28T11:10:56.975 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:10:56.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.976+0000 7f3f4c2a3700 1 -- 172.21.15.28:0/453323021 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3f280682f0 msgr2=0x7f3f2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.976+0000 7f3f4c2a3700 1 --2- 172.21.15.28:0/453323021 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3f280682f0 0x7f3f2806a7a0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f3f440b5200 tx=0x7f3f40008030 comp rx=0 tx=0).stop 2023-11-28T11:10:56.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.976+0000 7f3f4c2a3700 1 -- 172.21.15.28:0/453323021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f440ca360 msgr2=0x7f3f4414c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:56.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.976+0000 7f3f4c2a3700 1 --2- 172.21.15.28:0/453323021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f440ca360 0x7f3f4414c040 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400e870 tx=0x7f3f3400ec30 comp rx=0 tx=0).stop 2023-11-28T11:10:56.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.976+0000 7f3f4c2a3700 1 -- 172.21.15.28:0/453323021 shutdown_connections 2023-11-28T11:10:56.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.976+0000 7f3f4c2a3700 1 --2- 172.21.15.28:0/453323021 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3f280682f0 0x7f3f2806a7a0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.976+0000 7f3f4c2a3700 1 --2- 172.21.15.28:0/453323021 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f440ca360 0x7f3f4414c040 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.976+0000 7f3f4c2a3700 1 --2- 172.21.15.28:0/453323021 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f4415c950 0x7f3f44071110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.976+0000 7f3f4c2a3700 1 --2- 172.21.15.28:0/453323021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f440cae50 0x7f3f4414d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:56.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.976+0000 7f3f4c2a3700 1 -- 172.21.15.28:0/453323021 >> 172.21.15.28:0/453323021 conn(0x7f3f440b0df0 msgr2=0x7f3f440ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:56.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.976+0000 7f3f4c2a3700 1 -- 172.21.15.28:0/453323021 shutdown_connections 2023-11-28T11:10:56.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:56.976+0000 7f3f4c2a3700 1 -- 172.21.15.28:0/453323021 wait complete. 2023-11-28T11:10:56.989 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:10:57.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.113+0000 7f4df4cde700 1 Processor -- start 2023-11-28T11:10:57.114 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.113+0000 7f4df4cde700 1 -- start start 2023-11-28T11:10:57.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.114+0000 7f4df4cde700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4df0161090 0x7f4df00ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.114+0000 7f4df4cde700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4df00caeb0 0x7f4df00cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.114+0000 7f4df4cde700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4df00ce340 con 0x7f4df015c990 2023-11-28T11:10:57.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.114+0000 7f4df4cde700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4df00cebe0 con 0x7f4df00caeb0 2023-11-28T11:10:57.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.114+0000 7f4df4cde700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00cf780 con 0x7f4df0161090 2023-11-28T11:10:57.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.114+0000 7f4deed9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4df00caeb0 0x7f4df00cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.114+0000 7f4deed9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f4df00caeb0 0x7f4df00cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50326/0 (socket says 172.21.15.28:50326) 2023-11-28T11:10:57.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.114+0000 7f4deed9d700 1 -- 172.21.15.28:0/1594208398 learned_addr learned my addr 172.21.15.28:0/1594208398 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:57.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.114+0000 7f4dedd9b700 1 --2- 172.21.15.28:0/1594208398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4df0161090 0x7f4df00ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.114+0000 7f4deed9d700 1 -- 172.21.15.28:0/1594208398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4df0161090 msgr2=0x7f4df00ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:57.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.114+0000 7f4deed9d700 1 --2- 172.21.15.28:0/1594208398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4df0161090 0x7f4df00ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.114+0000 7f4deed9d700 1 -- 172.21.15.28:0/1594208398 >> v1:172.21.15.59:6789/0 conn(0x7f4df015c990 legacy=0x7f4df00cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.114+0000 7f4deed9d700 1 -- 172.21.15.28:0/1594208398 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4df00cb3f0 con 0x7f4df00caeb0 2023-11-28T11:10:57.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.115+0000 7f4deed9d700 1 --2- 172.21.15.28:0/1594208398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df00caeb0 0x7f4df00cc310 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009e50 tx=0x7f4de800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5eec377e50aeec7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:57.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.115+0000 7f4ded59a700 1 -- 172.21.15.28:0/1594208398 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de800e070 con 0x7f4df00caeb0 2023-11-28T11:10:57.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.115+0000 7f4ded59a700 1 -- 172.21.15.28:0/1594208398 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de800b9c0 con 0x7f4df00caeb0 2023-11-28T11:10:57.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.115+0000 7f4ded59a700 1 -- 172.21.15.28:0/1594208398 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de800bd70 con 0x7f4df00caeb0 2023-11-28T11:10:57.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.115+0000 7f4df4cde700 1 -- 172.21.15.28:0/1594208398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df00caeb0 msgr2=0x7f4df00cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.115+0000 7f4df4cde700 1 --2- 172.21.15.28:0/1594208398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df00caeb0 0x7f4df00cc310 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f4de8009e50 tx=0x7f4de800b330 comp rx=0 tx=0).stop 2023-11-28T11:10:57.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.115+0000 7f4df4cde700 1 -- 172.21.15.28:0/1594208398 shutdown_connections 2023-11-28T11:10:57.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.115+0000 7f4df4cde700 1 --2- 172.21.15.28:0/1594208398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df00caeb0 0x7f4df00cc310 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.115+0000 7f4df4cde700 1 --2- 172.21.15.28:0/1594208398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4df0161090 0x7f4df00ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.115+0000 7f4df4cde700 1 -- 172.21.15.28:0/1594208398 >> 172.21.15.28:0/1594208398 conn(0x7f4df00b1650 msgr2=0x7f4df00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:57.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.115+0000 7f4df4cde700 1 -- 172.21.15.28:0/1594208398 shutdown_connections 2023-11-28T11:10:57.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.115+0000 7f4df4cde700 1 -- 172.21.15.28:0/1594208398 wait complete. 2023-11-28T11:10:57.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.117+0000 7f4df4cde700 1 Processor -- start 2023-11-28T11:10:57.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.117+0000 7f4df4cde700 1 -- start start 2023-11-28T11:10:57.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4df4cde700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df00caeb0 0x7f4df00b5140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4df4cde700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4df015c990 0x7f4df00c9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4df4cde700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df0161090 0x7f4df00b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.121 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4df4cde700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00b5a50 con 0x7f4df0161090 2023-11-28T11:10:57.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4df4cde700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4df00c6fd0 con 0x7f4df00caeb0 2023-11-28T11:10:57.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4df4cde700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4df01483c0 con 0x7f4df015c990 2023-11-28T11:10:57.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4deed9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df0161090 0x7f4df00b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4deed9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df0161090 0x7f4df00b2b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50334/0 (socket says 172.21.15.28:50334) 2023-11-28T11:10:57.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4deed9d700 1 -- 172.21.15.28:0/3566464707 learned_addr learned my addr 172.21.15.28:0/3566464707 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:57.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4dedd9b700 1 --2- 172.21.15.28:0/3566464707 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4df015c990 0x7f4df00c9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4dee59c700 1 --2- 172.21.15.28:0/3566464707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df00caeb0 0x7f4df00b5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4deed9d700 1 -- 172.21.15.28:0/3566464707 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4df015c990 msgr2=0x7f4df00c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4deed9d700 1 --2- 172.21.15.28:0/3566464707 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4df015c990 0x7f4df00c9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4deed9d700 1 -- 172.21.15.28:0/3566464707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df00caeb0 msgr2=0x7f4df00b5140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4deed9d700 1 --2- 172.21.15.28:0/3566464707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df00caeb0 0x7f4df00b5140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4deed9d700 1 -- 172.21.15.28:0/3566464707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de8009b30 con 0x7f4df0161090 2023-11-28T11:10:57.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4dedd9b700 1 --2- 172.21.15.28:0/3566464707 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4df015c990 0x7f4df00c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:57.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4deed9d700 1 --2- 172.21.15.28:0/3566464707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df0161090 0x7f4df00b2b90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4df006eac0 tx=0x7f4de8004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:57.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4ddeffd700 1 -- 172.21.15.28:0/3566464707 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de8004870 con 0x7f4df0161090 2023-11-28T11:10:57.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4ddeffd700 1 -- 172.21.15.28:0/3566464707 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de80099b0 con 0x7f4df0161090 2023-11-28T11:10:57.124 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4df4cde700 1 -- 172.21.15.28:0/3566464707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4df00ce870 con 0x7f4df0161090 2023-11-28T11:10:57.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4ddeffd700 1 -- 172.21.15.28:0/3566464707 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de801c680 con 0x7f4df0161090 2023-11-28T11:10:57.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.118+0000 7f4df4cde700 1 -- 172.21.15.28:0/3566464707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4df00ce590 con 0x7f4df0161090 2023-11-28T11:10:57.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.119+0000 7f4df4cde700 1 -- 172.21.15.28:0/3566464707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4df00cc850 con 0x7f4df0161090 2023-11-28T11:10:57.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.120+0000 7f4ddeffd700 1 -- 172.21.15.28:0/3566464707 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4de802c070 con 0x7f4df0161090 2023-11-28T11:10:57.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.120+0000 7f4ddeffd700 1 --2- 172.21.15.28:0/3566464707 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4dd00682f0 0x7f4dd006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.120+0000 7f4ddeffd700 1 -- 172.21.15.28:0/3566464707 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f4de8087590 con 0x7f4df0161090 2023-11-28T11:10:57.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.120+0000 7f4dee59c700 1 --2- 172.21.15.28:0/3566464707 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4dd00682f0 0x7f4dd006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.121+0000 7f4dee59c700 1 --2- 172.21.15.28:0/3566464707 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4dd00682f0 0x7f4dd006a7a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4de0007fd0 tx=0x7f4de0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:57.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.125+0000 7f4ddeffd700 1 -- 172.21.15.28:0/3566464707 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4de8054000 con 0x7f4df0161090 2023-11-28T11:10:57.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.324+0000 7f4df4cde700 1 -- 172.21.15.28:0/3566464707 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4df0148530 con 0x7f4df0161090 2023-11-28T11:10:57.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.324+0000 7f4ddeffd700 1 -- 172.21.15.28:0/3566464707 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v50) v1 ==== 84+0+126 (secure 0 0 0) 0x7f4de8057620 con 0x7f4df0161090 2023-11-28T11:10:57.325 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:10:57.325 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2023-11-28T11:10:57.325 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:57.325 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 1, 2023-11-28T11:10:57.325 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": "rbd" 2023-11-28T11:10:57.325 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:10:57.325 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:10:57.326 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 2, 2023-11-28T11:10:57.326 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": ".mgr" 2023-11-28T11:10:57.326 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:10:57.326 INFO:teuthology.orchestra.run.smithi028.stdout:] 2023-11-28T11:10:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.326+0000 7f4df4cde700 1 -- 172.21.15.28:0/3566464707 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4dd00682f0 msgr2=0x7f4dd006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.326+0000 7f4df4cde700 1 --2- 172.21.15.28:0/3566464707 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4dd00682f0 0x7f4dd006a7a0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f4de0007fd0 tx=0x7f4de0009040 comp rx=0 tx=0).stop 2023-11-28T11:10:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.326+0000 7f4df4cde700 1 -- 172.21.15.28:0/3566464707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df0161090 msgr2=0x7f4df00b2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.326+0000 7f4df4cde700 1 --2- 172.21.15.28:0/3566464707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df0161090 0x7f4df00b2b90 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f4df006eac0 tx=0x7f4de8004ad0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.326+0000 7f4df4cde700 1 -- 172.21.15.28:0/3566464707 shutdown_connections 2023-11-28T11:10:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.326+0000 7f4df4cde700 1 --2- 172.21.15.28:0/3566464707 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4dd00682f0 0x7f4dd006a7a0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.326+0000 7f4df4cde700 1 --2- 172.21.15.28:0/3566464707 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4df0161090 0x7f4df00b2b90 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.327 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.326+0000 7f4df4cde700 1 --2- 172.21.15.28:0/3566464707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4df00caeb0 0x7f4df00b5140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.326+0000 7f4df4cde700 1 --2- 172.21.15.28:0/3566464707 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4df015c990 0x7f4df00c9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.326+0000 7f4df4cde700 1 -- 172.21.15.28:0/3566464707 >> 172.21.15.28:0/3566464707 conn(0x7f4df00b1650 msgr2=0x7f4df006c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:57.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.326+0000 7f4df4cde700 1 -- 172.21.15.28:0/3566464707 shutdown_connections 2023-11-28T11:10:57.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.326+0000 7f4df4cde700 1 -- 172.21.15.28:0/3566464707 wait complete. 2023-11-28T11:10:57.339 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:10:57.339 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:10:57.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.468+0000 7f3f3f3a1700 1 Processor -- start 2023-11-28T11:10:57.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.468+0000 7f3f3f3a1700 1 -- start start 2023-11-28T11:10:57.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.469+0000 7f3f3f3a1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f380cefa0 0x7f3f3815db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.469+0000 7f3f3f3a1700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f3815e0b0 con 0x7f3f3806e400 2023-11-28T11:10:57.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.469+0000 7f3f3f3a1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f3815d120 con 0x7f3f38159590 2023-11-28T11:10:57.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.469+0000 7f3f3f3a1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3815d6e0 con 0x7f3f380cefa0 2023-11-28T11:10:57.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.469+0000 7f3f3c93c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3f3806e400 0x7f3f380ce890 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38398/0 (socket says 172.21.15.28:38398) 2023-11-28T11:10:57.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.469+0000 7f3f3c93c700 1 -- 172.21.15.28:0/3950978780 learned_addr learned my addr 172.21.15.28:0/3950978780 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:57.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.469+0000 7f3f3d93e700 1 --2- 172.21.15.28:0/3950978780 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f380cefa0 0x7f3f3815db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.469+0000 7f3f37fff700 1 -- 172.21.15.28:0/3950978780 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2592222634 0 0) 0x7f3f2c001110 con 0x7f3f3806e400 2023-11-28T11:10:57.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.469+0000 7f3f37fff700 1 -- 172.21.15.28:0/3950978780 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f200035b0 con 0x7f3f3806e400 2023-11-28T11:10:57.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.470+0000 7f3f37fff700 1 -- 172.21.15.28:0/3950978780 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214862183 0 0) 0x7f3f3815d120 con 0x7f3f38159590 2023-11-28T11:10:57.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.470+0000 7f3f37fff700 1 -- 172.21.15.28:0/3950978780 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f2c001110 con 0x7f3f38159590 2023-11-28T11:10:57.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.470+0000 7f3f37fff700 1 -- 172.21.15.28:0/3950978780 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 247620713 0 0) 0x7f3f200035b0 con 0x7f3f3806e400 2023-11-28T11:10:57.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.470+0000 7f3f3d93e700 1 -- 172.21.15.28:0/3950978780 >> v1:172.21.15.59:6789/0 conn(0x7f3f38159590 legacy=0x7f3f380ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.470+0000 7f3f3d93e700 1 -- 172.21.15.28:0/3950978780 >> v1:172.21.15.28:6789/0 conn(0x7f3f3806e400 legacy=0x7f3f380ce890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.470+0000 7f3f3d93e700 1 -- 172.21.15.28:0/3950978780 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f380aef60 con 0x7f3f380cefa0 2023-11-28T11:10:57.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.470+0000 7f3f3d93e700 1 --2- 172.21.15.28:0/3950978780 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f380cefa0 0x7f3f3815db70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009dc0 tx=0x7f3f2800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4068acc52eac089c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:57.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.470+0000 7f3f37fff700 1 -- 172.21.15.28:0/3950978780 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2800e070 con 0x7f3f380cefa0 2023-11-28T11:10:57.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.470+0000 7f3f37fff700 1 -- 172.21.15.28:0/3950978780 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f28003a20 con 0x7f3f380cefa0 2023-11-28T11:10:57.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.470+0000 7f3f37fff700 1 -- 172.21.15.28:0/3950978780 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f2c001800 con 0x7f3f380cefa0 2023-11-28T11:10:57.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.471+0000 7f3f3f3a1700 1 -- 172.21.15.28:0/3950978780 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f380cefa0 msgr2=0x7f3f3815db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.471+0000 7f3f3f3a1700 1 --2- 172.21.15.28:0/3950978780 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f380cefa0 0x7f3f3815db70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3f28009dc0 tx=0x7f3f2800b330 comp rx=0 tx=0).stop 2023-11-28T11:10:57.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.471+0000 7f3f3f3a1700 1 -- 172.21.15.28:0/3950978780 shutdown_connections 2023-11-28T11:10:57.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.471+0000 7f3f3f3a1700 1 --2- 172.21.15.28:0/3950978780 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f380cefa0 0x7f3f3815db70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.471+0000 7f3f3f3a1700 1 -- 172.21.15.28:0/3950978780 >> 172.21.15.28:0/3950978780 conn(0x7f3f380b1650 msgr2=0x7f3f380b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:57.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.471+0000 7f3f3f3a1700 1 -- 172.21.15.28:0/3950978780 shutdown_connections 2023-11-28T11:10:57.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.471+0000 7f3f3f3a1700 1 -- 172.21.15.28:0/3950978780 wait complete. 2023-11-28T11:10:57.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.472+0000 7f3f3f3a1700 1 Processor -- start 2023-11-28T11:10:57.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.472+0000 7f3f3f3a1700 1 -- start start 2023-11-28T11:10:57.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3f3a1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f3806e400 0x7f3f380adde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3f3a1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f380cefa0 0x7f3f38150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3f3a1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f38159590 0x7f3f380ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3f3a1700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f380abf80 con 0x7f3f3806e400 2023-11-28T11:10:57.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3f3a1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f380ae320 con 0x7f3f380cefa0 2023-11-28T11:10:57.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3f3a1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f3815b7a0 con 0x7f3f38159590 2023-11-28T11:10:57.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3d13d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f3806e400 0x7f3f380adde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3d13d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f3806e400 0x7f3f380adde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50344/0 (socket says 172.21.15.28:50344) 2023-11-28T11:10:57.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3d13d700 1 -- 172.21.15.28:0/3679840949 learned_addr learned my addr 172.21.15.28:0/3679840949 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:57.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3d93e700 1 --2- 172.21.15.28:0/3679840949 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f38159590 0x7f3f380ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3c93c700 1 --2- 172.21.15.28:0/3679840949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f380cefa0 0x7f3f38150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3d13d700 1 -- 172.21.15.28:0/3679840949 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f38159590 msgr2=0x7f3f380ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3d13d700 1 --2- 172.21.15.28:0/3679840949 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f38159590 0x7f3f380ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3d13d700 1 -- 172.21.15.28:0/3679840949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f380cefa0 msgr2=0x7f3f38150d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3d13d700 1 --2- 172.21.15.28:0/3679840949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f380cefa0 0x7f3f38150d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3d13d700 1 -- 172.21.15.28:0/3679840949 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f28009a70 con 0x7f3f3806e400 2023-11-28T11:10:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.473+0000 7f3f3d13d700 1 --2- 172.21.15.28:0/3679840949 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f3806e400 0x7f3f380adde0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000c500 tx=0x7f3f3000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.474+0000 7f3f35ffb700 1 -- 172.21.15.28:0/3679840949 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f300052d0 con 0x7f3f3806e400 2023-11-28T11:10:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.474+0000 7f3f35ffb700 1 -- 172.21.15.28:0/3679840949 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f30005450 con 0x7f3f3806e400 2023-11-28T11:10:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.474+0000 7f3f3f3a1700 1 -- 172.21.15.28:0/3679840949 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f3815d370 con 0x7f3f3806e400 2023-11-28T11:10:57.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.474+0000 7f3f35ffb700 1 -- 172.21.15.28:0/3679840949 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f300038c0 con 0x7f3f3806e400 2023-11-28T11:10:57.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.474+0000 7f3f3f3a1700 1 -- 172.21.15.28:0/3679840949 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f380af240 con 0x7f3f3806e400 2023-11-28T11:10:57.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.475+0000 7f3f3f3a1700 1 -- 172.21.15.28:0/3679840949 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f38159960 con 0x7f3f3806e400 2023-11-28T11:10:57.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.475+0000 7f3f35ffb700 1 -- 172.21.15.28:0/3679840949 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3f30019030 con 0x7f3f3806e400 2023-11-28T11:10:57.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.476+0000 7f3f35ffb700 1 --2- 172.21.15.28:0/3679840949 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3f240682f0 0x7f3f2406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:57.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.476+0000 7f3f35ffb700 1 -- 172.21.15.28:0/3679840949 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f3f30084ee0 con 0x7f3f3806e400 2023-11-28T11:10:57.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.476+0000 7f3f3c93c700 1 --2- 172.21.15.28:0/3679840949 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3f240682f0 0x7f3f2406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:57.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.476+0000 7f3f3c93c700 1 --2- 172.21.15.28:0/3679840949 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3f240682f0 0x7f3f2406a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c001370 tx=0x7f3f2c001c00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:57.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.480+0000 7f3f35ffb700 1 -- 172.21.15.28:0/3679840949 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3f30051860 con 0x7f3f3806e400 2023-11-28T11:10:57.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:57.671+0000 7f3f3f3a1700 1 -- 172.21.15.28:0/3679840949 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3f380739f0 con 0x7f3f3806e400 2023-11-28T11:10:58.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.342+0000 7f3f35ffb700 1 -- 172.21.15.28:0/3679840949 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v51) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3f30054e80 con 0x7f3f3806e400 2023-11-28T11:10:58.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.345+0000 7f3f3f3a1700 1 -- 172.21.15.28:0/3679840949 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3f240682f0 msgr2=0x7f3f2406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:58.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.345+0000 7f3f3f3a1700 1 --2- 172.21.15.28:0/3679840949 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3f240682f0 0x7f3f2406a7a0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f3f2c001370 tx=0x7f3f2c001c00 comp rx=0 tx=0).stop 2023-11-28T11:10:58.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.345+0000 7f3f3f3a1700 1 -- 172.21.15.28:0/3679840949 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f3806e400 msgr2=0x7f3f380adde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:58.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.345+0000 7f3f3f3a1700 1 --2- 172.21.15.28:0/3679840949 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f3806e400 0x7f3f380adde0 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f3f3000c500 tx=0x7f3f3000c8c0 comp rx=0 tx=0).stop 2023-11-28T11:10:58.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.345+0000 7f3f3f3a1700 1 -- 172.21.15.28:0/3679840949 shutdown_connections 2023-11-28T11:10:58.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.345+0000 7f3f3f3a1700 1 --2- 172.21.15.28:0/3679840949 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3f240682f0 0x7f3f2406a7a0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:58.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.345+0000 7f3f3f3a1700 1 --2- 172.21.15.28:0/3679840949 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3f3806e400 0x7f3f380adde0 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:58.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.345+0000 7f3f3f3a1700 1 --2- 172.21.15.28:0/3679840949 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3f38159590 0x7f3f380ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:58.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.345+0000 7f3f3f3a1700 1 --2- 172.21.15.28:0/3679840949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3f380cefa0 0x7f3f38150d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:58.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.345+0000 7f3f3f3a1700 1 -- 172.21.15.28:0/3679840949 >> 172.21.15.28:0/3679840949 conn(0x7f3f380b1650 msgr2=0x7f3f380ae730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:58.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.345+0000 7f3f3f3a1700 1 -- 172.21.15.28:0/3679840949 shutdown_connections 2023-11-28T11:10:58.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.346+0000 7f3f3f3a1700 1 -- 172.21.15.28:0/3679840949 wait complete. 2023-11-28T11:10:58.349 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' created 2023-11-28T11:10:58.364 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:10:58.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.501+0000 7f600f71a700 1 Processor -- start 2023-11-28T11:10:58.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.501+0000 7f600f71a700 1 -- start start 2023-11-28T11:10:58.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.502+0000 7f600f71a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6008165670 0x7f6008167a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:58.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.502+0000 7f600f71a700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60080ce550 con 0x7f60080b1630 2023-11-28T11:10:58.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.502+0000 7f600f71a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60080ce830 con 0x7f60080cf5f0 2023-11-28T11:10:58.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.502+0000 7f600f71a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60080ceb10 con 0x7f6008165670 2023-11-28T11:10:58.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.502+0000 7f600d4b6700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f60080b1630 0x7f60080b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38410/0 (socket says 172.21.15.28:38410) 2023-11-28T11:10:58.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.502+0000 7f600d4b6700 1 -- 172.21.15.28:0/1632867489 learned_addr learned my addr 172.21.15.28:0/1632867489 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:58.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.502+0000 7f5ffffff700 1 -- 172.21.15.28:0/1632867489 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 287327071 0 0) 0x7f60080ce550 con 0x7f60080b1630 2023-11-28T11:10:58.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.502+0000 7f600dcb7700 1 --2- 172.21.15.28:0/1632867489 >> v2:172.21.15.59:3300/0 conn(0x7f6008165670 0x7f6008167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:58.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f5ffffff700 1 -- 172.21.15.28:0/1632867489 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ff40035b0 con 0x7f60080b1630 2023-11-28T11:10:58.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f5ffffff700 1 -- 172.21.15.28:0/1632867489 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3504366925 0 0) 0x7f5ff40035b0 con 0x7f60080b1630 2023-11-28T11:10:58.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f5ffffff700 1 -- 172.21.15.28:0/1632867489 >> v2:172.21.15.59:3300/0 conn(0x7f6008165670 msgr2=0x7f6008167a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:58.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f5ffffff700 1 --2- 172.21.15.28:0/1632867489 >> v2:172.21.15.59:3300/0 conn(0x7f6008165670 0x7f6008167a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:58.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f5ffffff700 1 -- 172.21.15.28:0/1632867489 >> v1:172.21.15.59:6789/0 conn(0x7f60080cf5f0 legacy=0x7f60080caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:58.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f5ffffff700 1 -- 172.21.15.28:0/1632867489 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60080cedf0 con 0x7f60080b1630 2023-11-28T11:10:58.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f5ffffff700 1 -- 172.21.15.28:0/1632867489 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f5ff8001670 con 0x7f60080b1630 2023-11-28T11:10:58.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f5ffffff700 1 -- 172.21.15.28:0/1632867489 >> v1:172.21.15.28:6789/0 conn(0x7f60080b1630 legacy=0x7f60080b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:58.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f5ffffff700 1 --2- 172.21.15.28:0/1632867489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4003a00 0x7f5ff4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:58.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f5ffffff700 1 --2- 172.21.15.28:0/1632867489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff4004850 0x7f5ff4004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f5ffffff700 1 --2- 172.21.15.28:0/1632867489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4005200 0x7f5ff400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f5ffffff700 1 -- 172.21.15.28:0/1632867489 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60080ceb10 con 0x7f5ff4004850 2023-11-28T11:10:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f5ffffff700 1 -- 172.21.15.28:0/1632867489 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff400c080 con 0x7f5ff4003a00 2023-11-28T11:10:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f5ffffff700 1 -- 172.21.15.28:0/1632867489 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ff400c240 con 0x7f5ff4005200 2023-11-28T11:10:58.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f600ccb5700 1 --2- 172.21.15.28:0/1632867489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff4004850 0x7f5ff4004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:58.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f600d4b6700 1 --2- 172.21.15.28:0/1632867489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4003a00 0x7f5ff4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:58.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f600dcb7700 1 --2- 172.21.15.28:0/1632867489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4005200 0x7f5ff400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:58.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f600ccb5700 1 -- 172.21.15.28:0/1632867489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4005200 msgr2=0x7f5ff400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:58.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f600ccb5700 1 --2- 172.21.15.28:0/1632867489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4005200 0x7f5ff400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:58.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f600ccb5700 1 -- 172.21.15.28:0/1632867489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4003a00 msgr2=0x7f5ff4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:58.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f600ccb5700 1 --2- 172.21.15.28:0/1632867489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4003a00 0x7f5ff4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:58.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f600ccb5700 1 -- 172.21.15.28:0/1632867489 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff40035b0 con 0x7f5ff4004850 2023-11-28T11:10:58.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f600d4b6700 1 --2- 172.21.15.28:0/1632867489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ff4003a00 0x7f5ff4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:58.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f600dcb7700 1 --2- 172.21.15.28:0/1632867489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4005200 0x7f5ff400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:58.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.503+0000 7f600dcb7700 1 -- 172.21.15.28:0/1632867489 reap_dead start 2023-11-28T11:10:58.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.504+0000 7f600ccb5700 1 --2- 172.21.15.28:0/1632867489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff4004850 0x7f5ff4004cc0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f60040017c0 tx=0x7f5ff4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=953333d1d453d54a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:58.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.504+0000 7f5ffffff700 1 -- 172.21.15.28:0/1632867489 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60040052d0 con 0x7f5ff4004850 2023-11-28T11:10:58.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.504+0000 7f5ffffff700 1 -- 172.21.15.28:0/1632867489 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6004005430 con 0x7f5ff4004850 2023-11-28T11:10:58.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.504+0000 7f5ffffff700 1 -- 172.21.15.28:0/1632867489 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60040038c0 con 0x7f5ff4004850 2023-11-28T11:10:58.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.504+0000 7f600f71a700 1 -- 172.21.15.28:0/1632867489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff4004850 msgr2=0x7f5ff4004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.504+0000 7f600f71a700 1 --2- 172.21.15.28:0/1632867489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff4004850 0x7f5ff4004cc0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f60040017c0 tx=0x7f5ff4005880 comp rx=0 tx=0).stop 2023-11-28T11:10:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.504+0000 7f600f71a700 1 -- 172.21.15.28:0/1632867489 shutdown_connections 2023-11-28T11:10:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.504+0000 7f600f71a700 1 --2- 172.21.15.28:0/1632867489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ff4004850 0x7f5ff4004cc0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.504+0000 7f600f71a700 1 -- 172.21.15.28:0/1632867489 >> 172.21.15.28:0/1632867489 conn(0x7f60080cc630 msgr2=0x7f60080cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.504+0000 7f600f71a700 1 -- 172.21.15.28:0/1632867489 shutdown_connections 2023-11-28T11:10:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.504+0000 7f600f71a700 1 -- 172.21.15.28:0/1632867489 wait complete. 2023-11-28T11:10:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.506+0000 7f600f71a700 1 Processor -- start 2023-11-28T11:10:58.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.506+0000 7f600f71a700 1 -- start start 2023-11-28T11:10:58.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.506+0000 7f600f71a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4004850 0x7f600815f670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:58.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.506+0000 7f600f71a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600814fa60 0x7f60080ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:58.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.506+0000 7f600f71a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60080abe00 0x7f600815e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:58.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.506+0000 7f600f71a700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600815dbe0 con 0x7f60080abe00 2023-11-28T11:10:58.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.506+0000 7f600f71a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60081502b0 con 0x7f600814fa60 2023-11-28T11:10:58.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.506+0000 7f600f71a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6008155c50 con 0x7f5ff4004850 2023-11-28T11:10:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.506+0000 7f600dcb7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60080abe00 0x7f600815e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.506+0000 7f600dcb7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60080abe00 0x7f600815e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50368/0 (socket says 172.21.15.28:50368) 2023-11-28T11:10:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.506+0000 7f600dcb7700 1 -- 172.21.15.28:0/393348251 learned_addr learned my addr 172.21.15.28:0/393348251 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.507+0000 7f600ccb5700 1 --2- 172.21.15.28:0/393348251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600814fa60 0x7f60080ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.507+0000 7f600d4b6700 1 --2- 172.21.15.28:0/393348251 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4004850 0x7f600815f670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:58.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.507+0000 7f600dcb7700 1 -- 172.21.15.28:0/393348251 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4004850 msgr2=0x7f600815f670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:58.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.507+0000 7f600dcb7700 1 --2- 172.21.15.28:0/393348251 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4004850 0x7f600815f670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:58.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.507+0000 7f600dcb7700 1 -- 172.21.15.28:0/393348251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600814fa60 msgr2=0x7f60080ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:58.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.507+0000 7f600dcb7700 1 --2- 172.21.15.28:0/393348251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600814fa60 0x7f60080ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:58.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.507+0000 7f600dcb7700 1 -- 172.21.15.28:0/393348251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60080ce140 con 0x7f60080abe00 2023-11-28T11:10:58.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.507+0000 7f600dcb7700 1 --2- 172.21.15.28:0/393348251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60080abe00 0x7f600815e660 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f60080d0c50 tx=0x7f60000104a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:58.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.507+0000 7f5ffdffb700 1 -- 172.21.15.28:0/393348251 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600001b070 con 0x7f60080abe00 2023-11-28T11:10:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.507+0000 7f5ffdffb700 1 -- 172.21.15.28:0/393348251 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f600000f040 con 0x7f60080abe00 2023-11-28T11:10:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.507+0000 7f600f71a700 1 -- 172.21.15.28:0/393348251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60080cf0d0 con 0x7f60080abe00 2023-11-28T11:10:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.507+0000 7f5ffdffb700 1 -- 172.21.15.28:0/393348251 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600000e050 con 0x7f60080abe00 2023-11-28T11:10:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.507+0000 7f600f71a700 1 -- 172.21.15.28:0/393348251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60080ce2d0 con 0x7f60080abe00 2023-11-28T11:10:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.508+0000 7f600f71a700 1 -- 172.21.15.28:0/393348251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60080739f0 con 0x7f60080abe00 2023-11-28T11:10:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.509+0000 7f5ffdffb700 1 -- 172.21.15.28:0/393348251 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6000003a20 con 0x7f60080abe00 2023-11-28T11:10:58.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.509+0000 7f5ffdffb700 1 --2- 172.21.15.28:0/393348251 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fec068320 0x7f5fec06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.509+0000 7f5ffdffb700 1 -- 172.21.15.28:0/393348251 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f6000085f00 con 0x7f60080abe00 2023-11-28T11:10:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.509+0000 7f600d4b6700 1 --2- 172.21.15.28:0/393348251 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fec068320 0x7f5fec06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:58.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.510+0000 7f600d4b6700 1 --2- 172.21.15.28:0/393348251 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fec068320 0x7f5fec06a7d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8001200 tx=0x7f5ff8001110 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:58.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.514+0000 7f5ffdffb700 1 -- 172.21.15.28:0/393348251 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6000052800 con 0x7f60080abe00 2023-11-28T11:10:58.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:58.708+0000 7f600f71a700 1 -- 172.21.15.28:0/393348251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f60080ad910 con 0x7f60080abe00 2023-11-28T11:10:59.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.348+0000 7f5ffdffb700 1 -- 172.21.15.28:0/393348251 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v52) v1 ==== 170+0+0 (secure 0 0 0) 0x7f6000003cb0 con 0x7f60080abe00 2023-11-28T11:10:59.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.356+0000 7f600f71a700 1 -- 172.21.15.28:0/393348251 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fec068320 msgr2=0x7f5fec06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.356+0000 7f600f71a700 1 --2- 172.21.15.28:0/393348251 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fec068320 0x7f5fec06a7d0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5ff8001200 tx=0x7f5ff8001110 comp rx=0 tx=0).stop 2023-11-28T11:10:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.356+0000 7f600f71a700 1 -- 172.21.15.28:0/393348251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60080abe00 msgr2=0x7f600815e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:59.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.356+0000 7f600f71a700 1 --2- 172.21.15.28:0/393348251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60080abe00 0x7f600815e660 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f60080d0c50 tx=0x7f60000104a0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.356+0000 7f600f71a700 1 -- 172.21.15.28:0/393348251 shutdown_connections 2023-11-28T11:10:59.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.356+0000 7f600f71a700 1 --2- 172.21.15.28:0/393348251 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fec068320 0x7f5fec06a7d0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.356+0000 7f600f71a700 1 --2- 172.21.15.28:0/393348251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f60080abe00 0x7f600815e660 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.356+0000 7f600f71a700 1 --2- 172.21.15.28:0/393348251 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ff4004850 0x7f600815f670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.356+0000 7f600f71a700 1 --2- 172.21.15.28:0/393348251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600814fa60 0x7f60080ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.356+0000 7f600f71a700 1 -- 172.21.15.28:0/393348251 >> 172.21.15.28:0/393348251 conn(0x7f60080cc630 msgr2=0x7f6008156360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:59.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.356+0000 7f600f71a700 1 -- 172.21.15.28:0/393348251 shutdown_connections 2023-11-28T11:10:59.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.359+0000 7f600f71a700 1 -- 172.21.15.28:0/393348251 wait complete. 2023-11-28T11:10:59.370 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' created 2023-11-28T11:10:59.385 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:10:59.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.517+0000 7fb0cc04e700 1 Processor -- start 2023-11-28T11:10:59.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.518+0000 7fb0cc04e700 1 -- start start 2023-11-28T11:10:59.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0cc04e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0c40b1d40 0x7fb0c40cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0cc04e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb0c415dbf0 0x7fb0c40ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0cc04e700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0c40ceff0 con 0x7fb0c4159500 2023-11-28T11:10:59.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0cc04e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0c415e090 con 0x7fb0c415dbf0 2023-11-28T11:10:59.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0cc04e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40cede0 con 0x7fb0c40b1d40 2023-11-28T11:10:59.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c9dea700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb0c4159500 0x7fb0c40b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38430/0 (socket says 172.21.15.28:38430) 2023-11-28T11:10:59.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c9dea700 1 -- 172.21.15.28:0/2058505024 learned_addr learned my addr 172.21.15.28:0/2058505024 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:59.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0ca5eb700 1 --2- 172.21.15.28:0/2058505024 >> v2:172.21.15.59:3300/0 conn(0x7fb0c415dbf0 0x7fb0c40ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c95e9700 1 --2- 172.21.15.28:0/2058505024 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0c40b1d40 0x7fb0c40cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 -- 172.21.15.28:0/2058505024 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2609546767 0 0) 0x7fb0c40ceff0 con 0x7fb0c4159500 2023-11-28T11:10:59.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 -- 172.21.15.28:0/2058505024 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0b00035b0 con 0x7fb0c4159500 2023-11-28T11:10:59.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 -- 172.21.15.28:0/2058505024 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3771444870 0 0) 0x7fb0b00035b0 con 0x7fb0c4159500 2023-11-28T11:10:59.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 -- 172.21.15.28:0/2058505024 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0c40b1d40 msgr2=0x7fb0c40cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:59.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 --2- 172.21.15.28:0/2058505024 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0c40b1d40 0x7fb0c40cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 -- 172.21.15.28:0/2058505024 >> v2:172.21.15.59:3300/0 conn(0x7fb0c415dbf0 msgr2=0x7fb0c40ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:59.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 --2- 172.21.15.28:0/2058505024 >> v2:172.21.15.59:3300/0 conn(0x7fb0c415dbf0 0x7fb0c40ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 -- 172.21.15.28:0/2058505024 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c40cf2d0 con 0x7fb0c4159500 2023-11-28T11:10:59.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 -- 172.21.15.28:0/2058505024 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fb0c0001670 con 0x7fb0c4159500 2023-11-28T11:10:59.528 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 -- 172.21.15.28:0/2058505024 >> v1:172.21.15.28:6789/0 conn(0x7fb0c4159500 legacy=0x7fb0c40b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:59.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 --2- 172.21.15.28:0/2058505024 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0b0003a00 0x7fb0b0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 --2- 172.21.15.28:0/2058505024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0b0004430 0x7fb0b00048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 --2- 172.21.15.28:0/2058505024 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0c40b1d40 0x7fb0b0004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 -- 172.21.15.28:0/2058505024 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c415e090 con 0x7fb0b0003a00 2023-11-28T11:10:59.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 -- 172.21.15.28:0/2058505024 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40cede0 con 0x7fb0b0004430 2023-11-28T11:10:59.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c8de8700 1 -- 172.21.15.28:0/2058505024 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b000ba10 con 0x7fb0c40b1d40 2023-11-28T11:10:59.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0ca5eb700 1 --2- 172.21.15.28:0/2058505024 >> v2:172.21.15.59:3300/0 conn(0x7fb0c415dbf0 0x7fb0c40ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:10:59.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.519+0000 7fb0c9dea700 1 --2- 172.21.15.28:0/2058505024 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0b0003a00 0x7fb0b0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.520+0000 7fb0c95e9700 1 --2- 172.21.15.28:0/2058505024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0b0004430 0x7fb0b00048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.520+0000 7fb0ca5eb700 1 --2- 172.21.15.28:0/2058505024 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0c40b1d40 0x7fb0b0004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.520+0000 7fb0c9dea700 1 -- 172.21.15.28:0/2058505024 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0c40b1d40 msgr2=0x7fb0b0004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:59.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.520+0000 7fb0c9dea700 1 --2- 172.21.15.28:0/2058505024 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0c40b1d40 0x7fb0b0004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.520+0000 7fb0c9dea700 1 -- 172.21.15.28:0/2058505024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0b0004430 msgr2=0x7fb0b00048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:59.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.520+0000 7fb0c9dea700 1 --2- 172.21.15.28:0/2058505024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0b0004430 0x7fb0b00048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.520+0000 7fb0c9dea700 1 -- 172.21.15.28:0/2058505024 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0b00035b0 con 0x7fb0b0003a00 2023-11-28T11:10:59.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.520+0000 7fb0c95e9700 1 --2- 172.21.15.28:0/2058505024 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0b0004430 0x7fb0b00048f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:59.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.520+0000 7fb0c9dea700 1 --2- 172.21.15.28:0/2058505024 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0b0003a00 0x7fb0b0003e10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000c2f0 tx=0x7fb0b0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88bc82adfd9ae83 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:59.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.520+0000 7fb0c8de8700 1 -- 172.21.15.28:0/2058505024 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c000ce30 con 0x7fb0b0003a00 2023-11-28T11:10:59.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.520+0000 7fb0c8de8700 1 -- 172.21.15.28:0/2058505024 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c0005960 con 0x7fb0b0003a00 2023-11-28T11:10:59.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.520+0000 7fb0c8de8700 1 -- 172.21.15.28:0/2058505024 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0005d10 con 0x7fb0b0003a00 2023-11-28T11:10:59.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.521+0000 7fb0cc04e700 1 -- 172.21.15.28:0/2058505024 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0b0003a00 msgr2=0x7fb0b0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:59.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.521+0000 7fb0cc04e700 1 --2- 172.21.15.28:0/2058505024 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0b0003a00 0x7fb0b0003e10 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000c2f0 tx=0x7fb0b0005880 comp rx=0 tx=0).stop 2023-11-28T11:10:59.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.521+0000 7fb0ca5eb700 1 -- 172.21.15.28:0/2058505024 reap_dead start 2023-11-28T11:10:59.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.521+0000 7fb0cc04e700 1 -- 172.21.15.28:0/2058505024 shutdown_connections 2023-11-28T11:10:59.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.521+0000 7fb0cc04e700 1 -- 172.21.15.28:0/2058505024 >> 172.21.15.28:0/2058505024 conn(0x7fb0c40cc630 msgr2=0x7fb0c40cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:59.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.521+0000 7fb0cc04e700 1 -- 172.21.15.28:0/2058505024 shutdown_connections 2023-11-28T11:10:59.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.521+0000 7fb0cc04e700 1 -- 172.21.15.28:0/2058505024 wait complete. 2023-11-28T11:10:59.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.522+0000 7fb0cc04e700 1 Processor -- start 2023-11-28T11:10:59.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.522+0000 7fb0cc04e700 1 -- start start 2023-11-28T11:10:59.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0cc04e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0b0003a00 0x7fb0c40af3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0cc04e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c4154220 0x7fb0c40b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0cc04e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0c406aa70 0x7fb0c406aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0cc04e700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c406b430 con 0x7fb0c4154220 2023-11-28T11:10:59.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0cc04e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c4154690 con 0x7fb0b0003a00 2023-11-28T11:10:59.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0cc04e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0c40af9a0 con 0x7fb0c406aa70 2023-11-28T11:10:59.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0c95e9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c4154220 0x7fb0c40b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0c95e9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c4154220 0x7fb0c40b0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50386/0 (socket says 172.21.15.28:50386) 2023-11-28T11:10:59.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0c95e9700 1 -- 172.21.15.28:0/3458259028 learned_addr learned my addr 172.21.15.28:0/3458259028 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:59.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0ca5eb700 1 --2- 172.21.15.28:0/3458259028 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0c406aa70 0x7fb0c406aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0c9dea700 1 --2- 172.21.15.28:0/3458259028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0b0003a00 0x7fb0c40af3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0c95e9700 1 -- 172.21.15.28:0/3458259028 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0c406aa70 msgr2=0x7fb0c406aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:59.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0c95e9700 1 --2- 172.21.15.28:0/3458259028 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0c406aa70 0x7fb0c406aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0c95e9700 1 -- 172.21.15.28:0/3458259028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0b0003a00 msgr2=0x7fb0c40af3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:59.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0c95e9700 1 --2- 172.21.15.28:0/3458259028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0b0003a00 0x7fb0c40af3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0c95e9700 1 -- 172.21.15.28:0/3458259028 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c415d3f0 con 0x7fb0c4154220 2023-11-28T11:10:59.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.523+0000 7fb0c95e9700 1 --2- 172.21.15.28:0/3458259028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c4154220 0x7fb0c40b0dd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80108c0 tx=0x7fb0b8010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:59.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.524+0000 7fb0b67fc700 1 -- 172.21.15.28:0/3458259028 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800e8d0 con 0x7fb0c4154220 2023-11-28T11:10:59.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.524+0000 7fb0b67fc700 1 -- 172.21.15.28:0/3458259028 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0b800ea30 con 0x7fb0c4154220 2023-11-28T11:10:59.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.524+0000 7fb0cc04e700 1 -- 172.21.15.28:0/3458259028 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0c415d110 con 0x7fb0c4154220 2023-11-28T11:10:59.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.524+0000 7fb0b67fc700 1 -- 172.21.15.28:0/3458259028 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0b800edc0 con 0x7fb0c4154220 2023-11-28T11:10:59.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.524+0000 7fb0cc04e700 1 -- 172.21.15.28:0/3458259028 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0c415d6d0 con 0x7fb0c4154220 2023-11-28T11:10:59.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.525+0000 7fb0cc04e700 1 -- 172.21.15.28:0/3458259028 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0c40c65a0 con 0x7fb0c4154220 2023-11-28T11:10:59.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.525+0000 7fb0b67fc700 1 -- 172.21.15.28:0/3458259028 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb0b801c030 con 0x7fb0c4154220 2023-11-28T11:10:59.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.526+0000 7fb0b67fc700 1 --2- 172.21.15.28:0/3458259028 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb0a80682f0 0x7fb0a806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.526+0000 7fb0c9dea700 1 --2- 172.21.15.28:0/3458259028 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb0a80682f0 0x7fb0a806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.526+0000 7fb0b67fc700 1 -- 172.21.15.28:0/3458259028 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6282+0+0 (secure 0 0 0) 0x7fb0b8085f70 con 0x7fb0c4154220 2023-11-28T11:10:59.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.526+0000 7fb0c9dea700 1 --2- 172.21.15.28:0/3458259028 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb0a80682f0 0x7fb0a806a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0008fd0 tx=0x7fb0c0007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:59.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.530+0000 7fb0b67fc700 1 -- 172.21.15.28:0/3458259028 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb0b80b75e0 con 0x7fb0c4154220 2023-11-28T11:10:59.771 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.769+0000 7fb0cc04e700 1 -- 172.21.15.28:0/3458259028 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb0c40ab8c0 con 0x7fb0c4154220 2023-11-28T11:10:59.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.775+0000 7fb0b67fc700 1 -- 172.21.15.28:0/3458259028 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v21) v1 ==== 172+0+0 (secure 0 0 0) 0x7fb0b80526f0 con 0x7fb0c4154220 2023-11-28T11:10:59.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.779+0000 7fb0cc04e700 1 -- 172.21.15.28:0/3458259028 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb0a80682f0 msgr2=0x7fb0a806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:59.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.779+0000 7fb0cc04e700 1 --2- 172.21.15.28:0/3458259028 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb0a80682f0 0x7fb0a806a7a0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb0c0008fd0 tx=0x7fb0c0007000 comp rx=0 tx=0).stop 2023-11-28T11:10:59.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.779+0000 7fb0cc04e700 1 -- 172.21.15.28:0/3458259028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c4154220 msgr2=0x7fb0c40b0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.779+0000 7fb0cc04e700 1 --2- 172.21.15.28:0/3458259028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c4154220 0x7fb0c40b0dd0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7fb0b80108c0 tx=0x7fb0b8010c80 comp rx=0 tx=0).stop 2023-11-28T11:10:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.779+0000 7fb0cc04e700 1 -- 172.21.15.28:0/3458259028 shutdown_connections 2023-11-28T11:10:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.779+0000 7fb0cc04e700 1 --2- 172.21.15.28:0/3458259028 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb0a80682f0 0x7fb0a806a7a0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.779+0000 7fb0cc04e700 1 --2- 172.21.15.28:0/3458259028 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0c406aa70 0x7fb0c406aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.779+0000 7fb0cc04e700 1 --2- 172.21.15.28:0/3458259028 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0b0003a00 0x7fb0c40af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.779+0000 7fb0cc04e700 1 --2- 172.21.15.28:0/3458259028 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0c4154220 0x7fb0c40b0dd0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.779+0000 7fb0cc04e700 1 -- 172.21.15.28:0/3458259028 >> 172.21.15.28:0/3458259028 conn(0x7fb0c40cc630 msgr2=0x7fb0c406be80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.779+0000 7fb0cc04e700 1 -- 172.21.15.28:0/3458259028 shutdown_connections 2023-11-28T11:10:59.782 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.779+0000 7fb0cc04e700 1 -- 172.21.15.28:0/3458259028 wait complete. 2023-11-28T11:10:59.786 INFO:teuthology.orchestra.run.smithi028.stderr:new fs with metadata pool 8 and data pool 9 2023-11-28T11:10:59.807 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:10:59.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.930+0000 7f91d989b700 1 Processor -- start 2023-11-28T11:10:59.932 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.931+0000 7f91d989b700 1 -- start start 2023-11-28T11:10:59.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.931+0000 7f91d989b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91d40ceec0 0x7f91d40b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.931+0000 7f91d989b700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d415d2d0 con 0x7f91d415c890 2023-11-28T11:10:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.931+0000 7f91d989b700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91d41611d0 con 0x7f91d4160d30 2023-11-28T11:10:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.931+0000 7f91d989b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40b2ba0 con 0x7f91d40ceec0 2023-11-28T11:10:59.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.931+0000 7f91d2ffd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f91d415c890 0x7f91d40ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38450/0 (socket says 172.21.15.28:38450) 2023-11-28T11:10:59.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.931+0000 7f91d2ffd700 1 -- 172.21.15.28:0/3337618456 learned_addr learned my addr 172.21.15.28:0/3337618456 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:10:59.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d27fc700 1 --2- 172.21.15.28:0/3337618456 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91d40ceec0 0x7f91d40b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968797549 0 0) 0x7f91d415d2d0 con 0x7f91d415c890 2023-11-28T11:10:59.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91bc0035b0 con 0x7f91d415c890 2023-11-28T11:10:59.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1706791883 0 0) 0x7f91d41611d0 con 0x7f91d4160d30 2023-11-28T11:10:59.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91d415d2d0 con 0x7f91d4160d30 2023-11-28T11:10:59.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1493862695 0 0) 0x7f91bc0035b0 con 0x7f91d415c890 2023-11-28T11:10:59.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91d40ceec0 msgr2=0x7f91d40b2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:59.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 --2- 172.21.15.28:0/3337618456 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91d40ceec0 0x7f91d40b2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 >> v1:172.21.15.59:6789/0 conn(0x7f91d4160d30 legacy=0x7f91d40cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:59.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d40b2d90 con 0x7f91d415c890 2023-11-28T11:10:59.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f91c4001670 con 0x7f91d415c890 2023-11-28T11:10:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 >> v1:172.21.15.28:6789/0 conn(0x7f91d415c890 legacy=0x7f91d40ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 --2- 172.21.15.28:0/3337618456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91bc003bc0 0x7f91bc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 --2- 172.21.15.28:0/3337618456 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91d40ceec0 0x7f91bc004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 --2- 172.21.15.28:0/3337618456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91bc004d90 0x7f91bc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc00bac0 con 0x7f91bc003bc0 2023-11-28T11:10:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc00bc00 con 0x7f91bc004d90 2023-11-28T11:10:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc00bdc0 con 0x7f91d40ceec0 2023-11-28T11:10:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d2ffd700 1 --2- 172.21.15.28:0/3337618456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91bc003bc0 0x7f91bc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d27fc700 1 --2- 172.21.15.28:0/3337618456 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91d40ceec0 0x7f91bc004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.932+0000 7f91d37fe700 1 --2- 172.21.15.28:0/3337618456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91bc004d90 0x7f91bc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d2ffd700 1 -- 172.21.15.28:0/3337618456 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91d40ceec0 msgr2=0x7f91bc004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d2ffd700 1 --2- 172.21.15.28:0/3337618456 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91d40ceec0 0x7f91bc004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d2ffd700 1 -- 172.21.15.28:0/3337618456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91bc004d90 msgr2=0x7f91bc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:10:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d2ffd700 1 --2- 172.21.15.28:0/3337618456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91bc004d90 0x7f91bc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d2ffd700 1 -- 172.21.15.28:0/3337618456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc0035b0 con 0x7f91bc003bc0 2023-11-28T11:10:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d27fc700 1 --2- 172.21.15.28:0/3337618456 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91d40ceec0 0x7f91bc004850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:10:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d2ffd700 1 --2- 172.21.15.28:0/3337618456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91bc003bc0 0x7f91bc003fb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f91c4001eb0 tx=0x7f91bc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b0cc5767ac204f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c400cc70 con 0x7f91bc003bc0 2023-11-28T11:10:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c400cdf0 con 0x7f91bc003bc0 2023-11-28T11:10:59.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d1ffb700 1 -- 172.21.15.28:0/3337618456 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c4005ba0 con 0x7f91bc003bc0 2023-11-28T11:10:59.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d989b700 1 -- 172.21.15.28:0/3337618456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91bc003bc0 msgr2=0x7f91bc003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:59.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d989b700 1 --2- 172.21.15.28:0/3337618456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91bc003bc0 0x7f91bc003fb0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f91c4001eb0 tx=0x7f91bc005880 comp rx=0 tx=0).stop 2023-11-28T11:10:59.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d37fe700 1 -- 172.21.15.28:0/3337618456 reap_dead start 2023-11-28T11:10:59.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d989b700 1 -- 172.21.15.28:0/3337618456 shutdown_connections 2023-11-28T11:10:59.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d989b700 1 -- 172.21.15.28:0/3337618456 >> 172.21.15.28:0/3337618456 conn(0x7f91d406e9b0 msgr2=0x7f91d40cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:10:59.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.933+0000 7f91d989b700 1 -- 172.21.15.28:0/3337618456 shutdown_connections 2023-11-28T11:10:59.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.934+0000 7f91d989b700 1 -- 172.21.15.28:0/3337618456 wait complete. 2023-11-28T11:10:59.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.935+0000 7f91d989b700 1 Processor -- start 2023-11-28T11:10:59.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.935+0000 7f91d989b700 1 -- start start 2023-11-28T11:10:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d989b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91bc003bc0 0x7f91d41586c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d989b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d4155a50 0x7f91d415aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d989b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91d415b3f0 0x7f91d41e73f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d989b700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40b60a0 con 0x7f91d4155a50 2023-11-28T11:10:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d989b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91d415b860 con 0x7f91d415b3f0 2023-11-28T11:10:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d989b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91d40b5520 con 0x7f91bc003bc0 2023-11-28T11:10:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d27fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d4155a50 0x7f91d415aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d27fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d4155a50 0x7f91d415aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50406/0 (socket says 172.21.15.28:50406) 2023-11-28T11:10:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d27fc700 1 -- 172.21.15.28:0/1118263619 learned_addr learned my addr 172.21.15.28:0/1118263619 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:10:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d37fe700 1 --2- 172.21.15.28:0/1118263619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91d415b3f0 0x7f91d41e73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d2ffd700 1 --2- 172.21.15.28:0/1118263619 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91bc003bc0 0x7f91d41586c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d27fc700 1 -- 172.21.15.28:0/1118263619 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91bc003bc0 msgr2=0x7f91d41586c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d27fc700 1 --2- 172.21.15.28:0/1118263619 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91bc003bc0 0x7f91d41586c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d27fc700 1 -- 172.21.15.28:0/1118263619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91d415b3f0 msgr2=0x7f91d41e73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:10:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d27fc700 1 --2- 172.21.15.28:0/1118263619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91d415b3f0 0x7f91d41e73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:10:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d27fc700 1 -- 172.21.15.28:0/1118263619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d415cf60 con 0x7f91d4155a50 2023-11-28T11:10:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91d27fc700 1 --2- 172.21.15.28:0/1118263619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d4155a50 0x7f91d415aeb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f91c80116d0 tx=0x7f91c80119e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91bbfff700 1 -- 172.21.15.28:0/1118263619 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c8004220 con 0x7f91d4155a50 2023-11-28T11:10:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.936+0000 7f91bbfff700 1 -- 172.21.15.28:0/1118263619 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c8004380 con 0x7f91d4155a50 2023-11-28T11:10:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.937+0000 7f91d989b700 1 -- 172.21.15.28:0/1118263619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91d40b3070 con 0x7f91d4155a50 2023-11-28T11:10:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.937+0000 7f91bbfff700 1 -- 172.21.15.28:0/1118263619 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c80036a0 con 0x7f91d4155a50 2023-11-28T11:10:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.937+0000 7f91d989b700 1 -- 172.21.15.28:0/1118263619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91d415cc60 con 0x7f91d4155a50 2023-11-28T11:10:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.938+0000 7f91d989b700 1 -- 172.21.15.28:0/1118263619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91d4068570 con 0x7f91d4155a50 2023-11-28T11:10:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.938+0000 7f91bbfff700 1 -- 172.21.15.28:0/1118263619 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f91c8024070 con 0x7f91d4155a50 2023-11-28T11:10:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.939+0000 7f91bbfff700 1 --2- 172.21.15.28:0/1118263619 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f91c00682f0 0x7f91c006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:10:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.939+0000 7f91d2ffd700 1 --2- 172.21.15.28:0/1118263619 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f91c00682f0 0x7f91c006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:10:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.939+0000 7f91bbfff700 1 -- 172.21.15.28:0/1118263619 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f91c80860d0 con 0x7f91d4155a50 2023-11-28T11:10:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.939+0000 7f91d2ffd700 1 --2- 172.21.15.28:0/1118263619 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f91c00682f0 0x7f91c006a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f91c4015040 tx=0x7f91c4007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:10:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:10:59.943+0000 7f91bbfff700 1 -- 172.21.15.28:0/1118263619 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f91c80526a0 con 0x7f91d4155a50 2023-11-28T11:11:00.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.135+0000 7f91d989b700 1 -- 172.21.15.28:0/1118263619 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91d40ce010 con 0x7f91d4155a50 2023-11-28T11:11:00.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.136+0000 7f91bbfff700 1 -- 172.21.15.28:0/1118263619 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18087 (secure 0 0 0) 0x7f91c8022090 con 0x7f91d4155a50 2023-11-28T11:11:00.138 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:00.138 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":53,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:10:59.771640+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:57.702079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:58.738753+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:00.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.138+0000 7f91d989b700 1 -- 172.21.15.28:0/1118263619 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f91c00682f0 msgr2=0x7f91c006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:00.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.138+0000 7f91d989b700 1 --2- 172.21.15.28:0/1118263619 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f91c00682f0 0x7f91c006a7a0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f91c4015040 tx=0x7f91c4007000 comp rx=0 tx=0).stop 2023-11-28T11:11:00.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.138+0000 7f91d989b700 1 -- 172.21.15.28:0/1118263619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d4155a50 msgr2=0x7f91d415aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:00.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.138+0000 7f91d989b700 1 --2- 172.21.15.28:0/1118263619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d4155a50 0x7f91d415aeb0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f91c80116d0 tx=0x7f91c80119e0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.138+0000 7f91d989b700 1 -- 172.21.15.28:0/1118263619 shutdown_connections 2023-11-28T11:11:00.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.138+0000 7f91d989b700 1 --2- 172.21.15.28:0/1118263619 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f91c00682f0 0x7f91c006a7a0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.138+0000 7f91d989b700 1 --2- 172.21.15.28:0/1118263619 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f91d415b3f0 0x7f91d41e73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.138+0000 7f91d989b700 1 --2- 172.21.15.28:0/1118263619 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f91bc003bc0 0x7f91d41586c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.138+0000 7f91d989b700 1 --2- 172.21.15.28:0/1118263619 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91d4155a50 0x7f91d415aeb0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.138+0000 7f91d989b700 1 -- 172.21.15.28:0/1118263619 >> 172.21.15.28:0/1118263619 conn(0x7f91d406e9b0 msgr2=0x7f91d40cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:00.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.138+0000 7f91d989b700 1 -- 172.21.15.28:0/1118263619 shutdown_connections 2023-11-28T11:11:00.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.138+0000 7f91d989b700 1 -- 172.21.15.28:0/1118263619 wait complete. 2023-11-28T11:11:00.153 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:11:00.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.276+0000 7f1af2256700 1 Processor -- start 2023-11-28T11:11:00.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.276+0000 7f1af2256700 1 -- start start 2023-11-28T11:11:00.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.277+0000 7f1af2256700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1aec0b4fe0 0x7f1aec15d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:00.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.277+0000 7f1af2256700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aec06abb0 con 0x7f1aec0cd4a0 2023-11-28T11:11:00.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.277+0000 7f1af2256700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1aec06a280 con 0x7f1aec15a230 2023-11-28T11:11:00.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.277+0000 7f1af2256700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1aec15a600 con 0x7f1aec0b4fe0 2023-11-28T11:11:00.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.277+0000 7f1aebfff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1aec0cd4a0 0x7f1aec0d0a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38474/0 (socket says 172.21.15.28:38474) 2023-11-28T11:11:00.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.277+0000 7f1aebfff700 1 -- 172.21.15.28:0/2762431420 learned_addr learned my addr 172.21.15.28:0/2762431420 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:00.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aea7fc700 1 -- 172.21.15.28:0/2762431420 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1661944227 0 0) 0x7f1aec06abb0 con 0x7f1aec0cd4a0 2023-11-28T11:11:00.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aeaffd700 1 --2- 172.21.15.28:0/2762431420 >> v2:172.21.15.59:3300/0 conn(0x7f1aec0b4fe0 0x7f1aec15d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:00.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aea7fc700 1 -- 172.21.15.28:0/2762431420 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ad80035b0 con 0x7f1aec0cd4a0 2023-11-28T11:11:00.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aea7fc700 1 -- 172.21.15.28:0/2762431420 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1833995648 0 0) 0x7f1ad80035b0 con 0x7f1aec0cd4a0 2023-11-28T11:11:00.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aea7fc700 1 -- 172.21.15.28:0/2762431420 >> v2:172.21.15.59:3300/0 conn(0x7f1aec0b4fe0 msgr2=0x7f1aec15d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:00.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aea7fc700 1 --2- 172.21.15.28:0/2762431420 >> v2:172.21.15.59:3300/0 conn(0x7f1aec0b4fe0 0x7f1aec15d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aea7fc700 1 -- 172.21.15.28:0/2762431420 >> v1:172.21.15.59:6789/0 conn(0x7f1aec15a230 legacy=0x7f1aec0b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:00.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aea7fc700 1 -- 172.21.15.28:0/2762431420 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec06a560 con 0x7f1aec0cd4a0 2023-11-28T11:11:00.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aea7fc700 1 -- 172.21.15.28:0/2762431420 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f1ae40019f0 con 0x7f1aec0cd4a0 2023-11-28T11:11:00.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aea7fc700 1 -- 172.21.15.28:0/2762431420 >> v1:172.21.15.28:6789/0 conn(0x7f1aec0cd4a0 legacy=0x7f1aec0d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:00.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aea7fc700 1 --2- 172.21.15.28:0/2762431420 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ad8003a00 0x7f1ad8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:00.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aea7fc700 1 --2- 172.21.15.28:0/2762431420 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ad8004850 0x7f1ad8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:00.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aea7fc700 1 --2- 172.21.15.28:0/2762431420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad8005200 0x7f1ad800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:00.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aea7fc700 1 -- 172.21.15.28:0/2762431420 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec15a600 con 0x7f1ad8004850 2023-11-28T11:11:00.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aea7fc700 1 -- 172.21.15.28:0/2762431420 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad800c080 con 0x7f1ad8005200 2023-11-28T11:11:00.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aea7fc700 1 -- 172.21.15.28:0/2762431420 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad800c240 con 0x7f1ad8003a00 2023-11-28T11:11:00.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aeaffd700 1 --2- 172.21.15.28:0/2762431420 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ad8004850 0x7f1ad8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:00.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aebfff700 1 --2- 172.21.15.28:0/2762431420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad8005200 0x7f1ad800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:00.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aeb7fe700 1 --2- 172.21.15.28:0/2762431420 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ad8003a00 0x7f1ad8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:00.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aeaffd700 1 -- 172.21.15.28:0/2762431420 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ad8003a00 msgr2=0x7f1ad8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:00.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aeaffd700 1 --2- 172.21.15.28:0/2762431420 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ad8003a00 0x7f1ad8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aeaffd700 1 -- 172.21.15.28:0/2762431420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad8005200 msgr2=0x7f1ad800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:00.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aeaffd700 1 --2- 172.21.15.28:0/2762431420 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1ad8005200 0x7f1ad800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.278+0000 7f1aebfff700 1 -- 172.21.15.28:0/2762431420 reap_dead start 2023-11-28T11:11:00.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.279+0000 7f1aeaffd700 1 -- 172.21.15.28:0/2762431420 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ad80035b0 con 0x7f1ad8004850 2023-11-28T11:11:00.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.279+0000 7f1aeaffd700 1 --2- 172.21.15.28:0/2762431420 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ad8004850 0x7f1ad8004cc0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009b60 tx=0x7f1ad8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17712785ae2daa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:00.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.279+0000 7f1aea7fc700 1 -- 172.21.15.28:0/2762431420 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400e6c0 con 0x7f1ad8004850 2023-11-28T11:11:00.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.279+0000 7f1aea7fc700 1 -- 172.21.15.28:0/2762431420 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ad400e820 con 0x7f1ad8004850 2023-11-28T11:11:00.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.279+0000 7f1aea7fc700 1 -- 172.21.15.28:0/2762431420 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ad400eb90 con 0x7f1ad8004850 2023-11-28T11:11:00.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.279+0000 7f1af2256700 1 -- 172.21.15.28:0/2762431420 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ad8004850 msgr2=0x7f1ad8004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:00.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.279+0000 7f1af2256700 1 --2- 172.21.15.28:0/2762431420 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ad8004850 0x7f1ad8004cc0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4009b60 tx=0x7f1ad8005880 comp rx=0 tx=0).stop 2023-11-28T11:11:00.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.279+0000 7f1af2256700 1 -- 172.21.15.28:0/2762431420 shutdown_connections 2023-11-28T11:11:00.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.279+0000 7f1af2256700 1 --2- 172.21.15.28:0/2762431420 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1ad8004850 0x7f1ad8004cc0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.279+0000 7f1af2256700 1 -- 172.21.15.28:0/2762431420 >> 172.21.15.28:0/2762431420 conn(0x7f1aec0cf8d0 msgr2=0x7f1aec0cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:00.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.279+0000 7f1af2256700 1 -- 172.21.15.28:0/2762431420 shutdown_connections 2023-11-28T11:11:00.285 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.280+0000 7f1af2256700 1 -- 172.21.15.28:0/2762431420 wait complete. 2023-11-28T11:11:00.288 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.281+0000 7f1af2256700 1 Processor -- start 2023-11-28T11:11:00.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.281+0000 7f1af2256700 1 -- start start 2023-11-28T11:11:00.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1af2256700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ad8004850 0x7f1aec1e89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:00.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1af2256700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec0cb980 0x7f1aec0cbdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:00.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1af2256700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aec0c9210 0x7f1aec0c96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:00.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1af2256700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec1e8ee0 con 0x7f1aec0c9210 2023-11-28T11:11:00.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1af2256700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec0c9bf0 con 0x7f1aec0cb980 2023-11-28T11:11:00.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1af2256700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1aec1587f0 con 0x7f1ad8004850 2023-11-28T11:11:00.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1aebfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aec0c9210 0x7f1aec0c96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:00.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1aebfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aec0c9210 0x7f1aec0c96b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50428/0 (socket says 172.21.15.28:50428) 2023-11-28T11:11:00.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1aebfff700 1 -- 172.21.15.28:0/391845299 learned_addr learned my addr 172.21.15.28:0/391845299 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:00.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1aeb7fe700 1 --2- 172.21.15.28:0/391845299 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ad8004850 0x7f1aec1e89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:00.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1aeaffd700 1 --2- 172.21.15.28:0/391845299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec0cb980 0x7f1aec0cbdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:00.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1aebfff700 1 -- 172.21.15.28:0/391845299 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ad8004850 msgr2=0x7f1aec1e89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:00.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1aebfff700 1 --2- 172.21.15.28:0/391845299 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ad8004850 0x7f1aec1e89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1aebfff700 1 -- 172.21.15.28:0/391845299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec0cb980 msgr2=0x7f1aec0cbdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:00.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1aebfff700 1 --2- 172.21.15.28:0/391845299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec0cb980 0x7f1aec0cbdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1aebfff700 1 -- 172.21.15.28:0/391845299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aec0cd8c0 con 0x7f1aec0c9210 2023-11-28T11:11:00.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1aeaffd700 1 --2- 172.21.15.28:0/391845299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec0cb980 0x7f1aec0cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:00.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1aebfff700 1 --2- 172.21.15.28:0/391845299 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aec0c9210 0x7f1aec0c96b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1aec08b050 tx=0x7f1ae400a5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:00.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1ae3fff700 1 -- 172.21.15.28:0/391845299 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae400ada0 con 0x7f1aec0c9210 2023-11-28T11:11:00.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1ae3fff700 1 -- 172.21.15.28:0/391845299 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ae40133f0 con 0x7f1aec0c9210 2023-11-28T11:11:00.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1af2256700 1 -- 172.21.15.28:0/391845299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1aec06a840 con 0x7f1aec0c9210 2023-11-28T11:11:00.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1ae3fff700 1 -- 172.21.15.28:0/391845299 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ae40137a0 con 0x7f1aec0c9210 2023-11-28T11:11:00.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.282+0000 7f1af2256700 1 -- 172.21.15.28:0/391845299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1aec06b040 con 0x7f1aec0c9210 2023-11-28T11:11:00.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.283+0000 7f1af2256700 1 -- 172.21.15.28:0/391845299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1aec0675a0 con 0x7f1aec0c9210 2023-11-28T11:11:00.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.284+0000 7f1ae3fff700 1 -- 172.21.15.28:0/391845299 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1ae4013900 con 0x7f1aec0c9210 2023-11-28T11:11:00.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.284+0000 7f1ae3fff700 1 --2- 172.21.15.28:0/391845299 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1acc0682f0 0x7f1acc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:00.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.284+0000 7f1ae3fff700 1 -- 172.21.15.28:0/391845299 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f1ae40867c0 con 0x7f1aec0c9210 2023-11-28T11:11:00.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.284+0000 7f1aeb7fe700 1 --2- 172.21.15.28:0/391845299 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1acc0682f0 0x7f1acc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:00.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.285+0000 7f1aeb7fe700 1 --2- 172.21.15.28:0/391845299 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1acc0682f0 0x7f1acc06a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1adc007fd0 tx=0x7f1adc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:00.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.289+0000 7f1ae3fff700 1 -- 172.21.15.28:0/391845299 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1ae4052ee0 con 0x7f1aec0c9210 2023-11-28T11:11:00.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.486+0000 7f1af2256700 1 -- 172.21.15.28:0/391845299 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1aec06b5e0 con 0x7f1aec0c9210 2023-11-28T11:11:00.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.487+0000 7f1ae3fff700 1 -- 172.21.15.28:0/391845299 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v53) v1 ==== 74+0+18087 (secure 0 0 0) 0x7f1ae4056500 con 0x7f1aec0c9210 2023-11-28T11:11:00.489 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:00.489 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":53,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:10:59.771640+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:57.702079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:58.738753+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:00.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.489+0000 7f1af2256700 1 -- 172.21.15.28:0/391845299 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1acc0682f0 msgr2=0x7f1acc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:00.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.489+0000 7f1af2256700 1 --2- 172.21.15.28:0/391845299 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1acc0682f0 0x7f1acc06a7a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f1adc007fd0 tx=0x7f1adc009040 comp rx=0 tx=0).stop 2023-11-28T11:11:00.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.489+0000 7f1af2256700 1 -- 172.21.15.28:0/391845299 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aec0c9210 msgr2=0x7f1aec0c96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:00.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.489+0000 7f1af2256700 1 --2- 172.21.15.28:0/391845299 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aec0c9210 0x7f1aec0c96b0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f1aec08b050 tx=0x7f1ae400a5f0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.489+0000 7f1af2256700 1 -- 172.21.15.28:0/391845299 shutdown_connections 2023-11-28T11:11:00.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.489+0000 7f1af2256700 1 --2- 172.21.15.28:0/391845299 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1acc0682f0 0x7f1acc06a7a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.489+0000 7f1af2256700 1 --2- 172.21.15.28:0/391845299 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1aec0c9210 0x7f1aec0c96b0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.489+0000 7f1af2256700 1 --2- 172.21.15.28:0/391845299 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1ad8004850 0x7f1aec1e89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.489+0000 7f1af2256700 1 --2- 172.21.15.28:0/391845299 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1aec0cb980 0x7f1aec0cbdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.489+0000 7f1af2256700 1 -- 172.21.15.28:0/391845299 >> 172.21.15.28:0/391845299 conn(0x7f1aec0cf8d0 msgr2=0x7f1aec0b7c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:00.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.489+0000 7f1af2256700 1 -- 172.21.15.28:0/391845299 shutdown_connections 2023-11-28T11:11:00.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.489+0000 7f1af2256700 1 -- 172.21.15.28:0/391845299 wait complete. 2023-11-28T11:11:00.503 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:11:00.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.629+0000 7fea80877700 1 Processor -- start 2023-11-28T11:11:00.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.629+0000 7fea80877700 1 -- start start 2023-11-28T11:11:00.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.630+0000 7fea80877700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea7c0b1630 0x7fea7c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:00.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.630+0000 7fea80877700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fea7c0b1a00 0x7fea7c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:00.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.630+0000 7fea80877700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea7c0ce550 con 0x7fea7c0cb3e0 2023-11-28T11:11:00.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.630+0000 7fea80877700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea7c0ce830 con 0x7fea7c0b1a00 2023-11-28T11:11:00.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.630+0000 7fea80877700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c0ceb10 con 0x7fea7c0b1630 2023-11-28T11:11:00.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.630+0000 7fea79d9b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fea7c0b1a00 0x7fea7c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:00.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.630+0000 7fea79d9b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fea7c0b1a00 0x7fea7c0caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50432/0 (socket says 172.21.15.28:50432) 2023-11-28T11:11:00.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.630+0000 7fea7ad9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fea7c0cb3e0 0x7fea7c160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38494/0 (socket says 172.21.15.28:38494) 2023-11-28T11:11:00.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.630+0000 7fea79d9b700 1 -- 172.21.15.28:0/3089717613 learned_addr learned my addr 172.21.15.28:0/3089717613 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:00.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.630+0000 7fea7a59c700 1 --2- 172.21.15.28:0/3089717613 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea7c0b1630 0x7fea7c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:00.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.631+0000 7fea79d9b700 1 -- 172.21.15.28:0/3089717613 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea7c0b1630 msgr2=0x7fea7c0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:00.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.631+0000 7fea79d9b700 1 --2- 172.21.15.28:0/3089717613 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea7c0b1630 0x7fea7c0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.631+0000 7fea79d9b700 1 -- 172.21.15.28:0/3089717613 >> v1:172.21.15.28:6789/0 conn(0x7fea7c0cb3e0 legacy=0x7fea7c160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:00.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.631+0000 7fea7959a700 1 -- 172.21.15.28:0/3089717613 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4073673062 0 0) 0x7fea7c0ce550 con 0x7fea7c0cb3e0 2023-11-28T11:11:00.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.631+0000 7fea79d9b700 1 -- 172.21.15.28:0/3089717613 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea7c0cedf0 con 0x7fea7c0b1a00 2023-11-28T11:11:00.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.631+0000 7fea79d9b700 1 --2- 172.21.15.28:0/3089717613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea7c0b1a00 0x7fea7c0caea0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0098f0 tx=0x7fea6c009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53002584913f0189 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:00.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.631+0000 7fea7959a700 1 -- 172.21.15.28:0/3089717613 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c011830 con 0x7fea7c0b1a00 2023-11-28T11:11:00.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.631+0000 7fea7959a700 1 -- 172.21.15.28:0/3089717613 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6c011990 con 0x7fea7c0b1a00 2023-11-28T11:11:00.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.631+0000 7fea7959a700 1 -- 172.21.15.28:0/3089717613 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c011d20 con 0x7fea7c0b1a00 2023-11-28T11:11:00.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.631+0000 7fea80877700 1 -- 172.21.15.28:0/3089717613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea7c0b1a00 msgr2=0x7fea7c0caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:00.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.631+0000 7fea80877700 1 --2- 172.21.15.28:0/3089717613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea7c0b1a00 0x7fea7c0caea0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fea6c0098f0 tx=0x7fea6c009c00 comp rx=0 tx=0).stop 2023-11-28T11:11:00.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.632+0000 7fea80877700 1 -- 172.21.15.28:0/3089717613 shutdown_connections 2023-11-28T11:11:00.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.632+0000 7fea80877700 1 --2- 172.21.15.28:0/3089717613 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea7c0b1630 0x7fea7c0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.632+0000 7fea80877700 1 --2- 172.21.15.28:0/3089717613 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea7c0b1a00 0x7fea7c0caea0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.632+0000 7fea80877700 1 -- 172.21.15.28:0/3089717613 >> 172.21.15.28:0/3089717613 conn(0x7fea7c0cc630 msgr2=0x7fea7c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:00.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.632+0000 7fea80877700 1 -- 172.21.15.28:0/3089717613 shutdown_connections 2023-11-28T11:11:00.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.632+0000 7fea80877700 1 -- 172.21.15.28:0/3089717613 wait complete. 2023-11-28T11:11:00.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.633+0000 7fea80877700 1 Processor -- start 2023-11-28T11:11:00.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.633+0000 7fea80877700 1 -- start start 2023-11-28T11:11:00.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea80877700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea7c0b1630 0x7fea7c0b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:00.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea80877700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea7c0b1a00 0x7fea7c0c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:00.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea80877700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea7c0cb3e0 0x7fea7c15bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:00.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea80877700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c15d720 con 0x7fea7c0b1a00 2023-11-28T11:11:00.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea80877700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c0b5240 con 0x7fea7c0cb3e0 2023-11-28T11:11:00.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea80877700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea7c15d520 con 0x7fea7c0b1630 2023-11-28T11:11:00.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea79d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea7c0b1a00 0x7fea7c0c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:00.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea79d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea7c0b1a00 0x7fea7c0c40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50440/0 (socket says 172.21.15.28:50440) 2023-11-28T11:11:00.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea79d9b700 1 -- 172.21.15.28:0/4142285616 learned_addr learned my addr 172.21.15.28:0/4142285616 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:00.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea79d9b700 1 -- 172.21.15.28:0/4142285616 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea7c0b1630 msgr2=0x7fea7c0b4d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:00.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea7a59c700 1 --2- 172.21.15.28:0/4142285616 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea7c0b1630 0x7fea7c0b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:00.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea7ad9d700 1 --2- 172.21.15.28:0/4142285616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea7c0cb3e0 0x7fea7c15bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:00.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea79d9b700 1 --2- 172.21.15.28:0/4142285616 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea7c0b1630 0x7fea7c0b4d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea79d9b700 1 -- 172.21.15.28:0/4142285616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea7c0cb3e0 msgr2=0x7fea7c15bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:00.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea79d9b700 1 --2- 172.21.15.28:0/4142285616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea7c0cb3e0 0x7fea7c15bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:00.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea79d9b700 1 -- 172.21.15.28:0/4142285616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6c009550 con 0x7fea7c0b1a00 2023-11-28T11:11:00.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea7ad9d700 1 --2- 172.21.15.28:0/4142285616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea7c0cb3e0 0x7fea7c15bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:00.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea79d9b700 1 --2- 172.21.15.28:0/4142285616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea7c0b1a00 0x7fea7c0c40f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fea7c066a20 tx=0x7fea6c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:00.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea6affd700 1 -- 172.21.15.28:0/4142285616 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0045b0 con 0x7fea7c0b1a00 2023-11-28T11:11:00.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.634+0000 7fea6affd700 1 -- 172.21.15.28:0/4142285616 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea6c01f070 con 0x7fea7c0b1a00 2023-11-28T11:11:00.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.635+0000 7fea80877700 1 -- 172.21.15.28:0/4142285616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea7c0b1e70 con 0x7fea7c0b1a00 2023-11-28T11:11:00.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.635+0000 7fea6affd700 1 -- 172.21.15.28:0/4142285616 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea6c0038e0 con 0x7fea7c0b1a00 2023-11-28T11:11:00.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.635+0000 7fea80877700 1 -- 172.21.15.28:0/4142285616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea7c0cf0d0 con 0x7fea7c0b1a00 2023-11-28T11:11:00.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.635+0000 7fea80877700 1 -- 172.21.15.28:0/4142285616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea7c0ce140 con 0x7fea7c0b1a00 2023-11-28T11:11:00.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.636+0000 7fea6affd700 1 -- 172.21.15.28:0/4142285616 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fea6c02a020 con 0x7fea7c0b1a00 2023-11-28T11:11:00.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.636+0000 7fea6affd700 1 --2- 172.21.15.28:0/4142285616 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fea5c0682f0 0x7fea5c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:00.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.636+0000 7fea6affd700 1 -- 172.21.15.28:0/4142285616 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(53..53 src has 1..53) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fea6c087480 con 0x7fea7c0b1a00 2023-11-28T11:11:00.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.636+0000 7fea7a59c700 1 --2- 172.21.15.28:0/4142285616 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fea5c0682f0 0x7fea5c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:00.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.637+0000 7fea7a59c700 1 --2- 172.21.15.28:0/4142285616 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fea5c0682f0 0x7fea5c06a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fea70009ab0 tx=0x7fea70008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:00.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.641+0000 7fea6affd700 1 -- 172.21.15.28:0/4142285616 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fea6c053ba0 con 0x7fea7c0b1a00 2023-11-28T11:11:00.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:00.882+0000 7fea80877700 1 -- 172.21.15.28:0/4142285616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fea7c15d860 con 0x7fea7c0b1a00 2023-11-28T11:11:01.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.784+0000 7fea6affd700 1 -- 172.21.15.28:0/4142285616 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v24) v1 ==== 118+0+0 (secure 0 0 0) 0x7fea7c15d860 con 0x7fea7c0b1a00 2023-11-28T11:11:01.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.787+0000 7fea80877700 1 -- 172.21.15.28:0/4142285616 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fea5c0682f0 msgr2=0x7fea5c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.787+0000 7fea80877700 1 --2- 172.21.15.28:0/4142285616 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fea5c0682f0 0x7fea5c06a7a0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fea70009ab0 tx=0x7fea70008040 comp rx=0 tx=0).stop 2023-11-28T11:11:01.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.787+0000 7fea80877700 1 -- 172.21.15.28:0/4142285616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea7c0b1a00 msgr2=0x7fea7c0c40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.787+0000 7fea80877700 1 --2- 172.21.15.28:0/4142285616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea7c0b1a00 0x7fea7c0c40f0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fea7c066a20 tx=0x7fea6c004750 comp rx=0 tx=0).stop 2023-11-28T11:11:01.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.787+0000 7fea80877700 1 -- 172.21.15.28:0/4142285616 shutdown_connections 2023-11-28T11:11:01.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.787+0000 7fea80877700 1 --2- 172.21.15.28:0/4142285616 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fea5c0682f0 0x7fea5c06a7a0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.787+0000 7fea80877700 1 --2- 172.21.15.28:0/4142285616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea7c0cb3e0 0x7fea7c15bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.787+0000 7fea80877700 1 --2- 172.21.15.28:0/4142285616 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fea7c0b1630 0x7fea7c0b4d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.787+0000 7fea80877700 1 --2- 172.21.15.28:0/4142285616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fea7c0b1a00 0x7fea7c0c40f0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.787+0000 7fea80877700 1 -- 172.21.15.28:0/4142285616 >> 172.21.15.28:0/4142285616 conn(0x7fea7c0cc630 msgr2=0x7fea7c0ae410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:01.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.787+0000 7fea80877700 1 -- 172.21.15.28:0/4142285616 shutdown_connections 2023-11-28T11:11:01.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.788+0000 7fea80877700 1 -- 172.21.15.28:0/4142285616 wait complete. 2023-11-28T11:11:01.804 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:11:01.804 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:11:01.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.938+0000 7f436cdb0700 1 Processor -- start 2023-11-28T11:11:01.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.938+0000 7f436cdb0700 1 -- start start 2023-11-28T11:11:01.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f436cdb0700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f43680b1d40 0x7f43680cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f436cdb0700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f436815dbf0 0x7f43680ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f436cdb0700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43680ceff0 con 0x7f4368159500 2023-11-28T11:11:01.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f436cdb0700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f436815e090 con 0x7f43680b1d40 2023-11-28T11:11:01.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f436cdb0700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43680cede0 con 0x7f436815dbf0 2023-11-28T11:11:01.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f4365d9b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f43680b1d40 0x7f43680cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f436659c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4368159500 0x7f43680b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38504/0 (socket says 172.21.15.28:38504) 2023-11-28T11:11:01.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f436659c700 1 -- 172.21.15.28:0/3315020750 learned_addr learned my addr 172.21.15.28:0/3315020750 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:01.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f4366d9d700 1 --2- 172.21.15.28:0/3315020750 >> v2:172.21.15.59:3300/0 conn(0x7f436815dbf0 0x7f43680ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f4365d9b700 1 -- 172.21.15.28:0/3315020750 >> v2:172.21.15.59:3300/0 conn(0x7f436815dbf0 msgr2=0x7f43680ce8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:01.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f4365d9b700 1 --2- 172.21.15.28:0/3315020750 >> v2:172.21.15.59:3300/0 conn(0x7f436815dbf0 0x7f43680ce8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f4365d9b700 1 -- 172.21.15.28:0/3315020750 >> v1:172.21.15.28:6789/0 conn(0x7f4368159500 legacy=0x7f43680b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f4365d9b700 1 -- 172.21.15.28:0/3315020750 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43680cf2d0 con 0x7f43680b1d40 2023-11-28T11:11:01.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f436559a700 1 -- 172.21.15.28:0/3315020750 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1965629354 0 0) 0x7f43680ceff0 con 0x7f4368159500 2023-11-28T11:11:01.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f4366d9d700 1 --2- 172.21.15.28:0/3315020750 >> v2:172.21.15.59:3300/0 conn(0x7f436815dbf0 0x7f43680ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:01.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f4365d9b700 1 --2- 172.21.15.28:0/3315020750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43680b1d40 0x7f43680cd140 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f43680718b0 tx=0x7f435c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce7f92acf92dcfc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:01.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f436559a700 1 -- 172.21.15.28:0/3315020750 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c011830 con 0x7f43680b1d40 2023-11-28T11:11:01.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f436559a700 1 -- 172.21.15.28:0/3315020750 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435c011990 con 0x7f43680b1d40 2023-11-28T11:11:01.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.939+0000 7f436559a700 1 -- 172.21.15.28:0/3315020750 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c011d20 con 0x7f43680b1d40 2023-11-28T11:11:01.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.940+0000 7f436cdb0700 1 -- 172.21.15.28:0/3315020750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43680b1d40 msgr2=0x7f43680cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.940+0000 7f436cdb0700 1 --2- 172.21.15.28:0/3315020750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43680b1d40 0x7f43680cd140 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f43680718b0 tx=0x7f435c009b20 comp rx=0 tx=0).stop 2023-11-28T11:11:01.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.940+0000 7f436cdb0700 1 -- 172.21.15.28:0/3315020750 shutdown_connections 2023-11-28T11:11:01.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.940+0000 7f436cdb0700 1 --2- 172.21.15.28:0/3315020750 >> v2:172.21.15.59:3300/0 conn(0x7f436815dbf0 0x7f43680ce8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.940+0000 7f436cdb0700 1 --2- 172.21.15.28:0/3315020750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43680b1d40 0x7f43680cd140 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.940+0000 7f436cdb0700 1 -- 172.21.15.28:0/3315020750 >> 172.21.15.28:0/3315020750 conn(0x7f43680cc630 msgr2=0x7f43680cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:01.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.940+0000 7f436cdb0700 1 -- 172.21.15.28:0/3315020750 shutdown_connections 2023-11-28T11:11:01.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.940+0000 7f436cdb0700 1 -- 172.21.15.28:0/3315020750 wait complete. 2023-11-28T11:11:01.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.942+0000 7f436cdb0700 1 Processor -- start 2023-11-28T11:11:01.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.942+0000 7f436cdb0700 1 -- start start 2023-11-28T11:11:01.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f436cdb0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43680b1d40 0x7f43680b5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f436cdb0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4368159500 0x7f43680c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f436cdb0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f436815dbf0 0x7f436815b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f436cdb0700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43680b3590 con 0x7f43680b1d40 2023-11-28T11:11:01.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f436cdb0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43680b3870 con 0x7f4368159500 2023-11-28T11:11:01.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f436cdb0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43680b3b50 con 0x7f436815dbf0 2023-11-28T11:11:01.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f436659c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43680b1d40 0x7f43680b5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f436659c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43680b1d40 0x7f43680b5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50452/0 (socket says 172.21.15.28:50452) 2023-11-28T11:11:01.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f436659c700 1 -- 172.21.15.28:0/2809974168 learned_addr learned my addr 172.21.15.28:0/2809974168 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:01.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f4365d9b700 1 --2- 172.21.15.28:0/2809974168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4368159500 0x7f43680c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f4366d9d700 1 --2- 172.21.15.28:0/2809974168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f436815dbf0 0x7f436815b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f436659c700 1 -- 172.21.15.28:0/2809974168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f436815dbf0 msgr2=0x7f436815b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f436659c700 1 --2- 172.21.15.28:0/2809974168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f436815dbf0 0x7f436815b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f436659c700 1 -- 172.21.15.28:0/2809974168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4368159500 msgr2=0x7f43680c5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:01.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f436659c700 1 --2- 172.21.15.28:0/2809974168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4368159500 0x7f43680c5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:01.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f436659c700 1 -- 172.21.15.28:0/2809974168 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c0094a0 con 0x7f43680b1d40 2023-11-28T11:11:01.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.943+0000 7f4365d9b700 1 --2- 172.21.15.28:0/2809974168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4368159500 0x7f43680c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:01.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.944+0000 7f436659c700 1 --2- 172.21.15.28:0/2809974168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43680b1d40 0x7f43680b5460 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f43580011e0 tx=0x7f435800cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:01.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.944+0000 7f4356ffd700 1 -- 172.21.15.28:0/2809974168 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358004c80 con 0x7f43680b1d40 2023-11-28T11:11:01.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.944+0000 7f4356ffd700 1 -- 172.21.15.28:0/2809974168 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4358004e00 con 0x7f43680b1d40 2023-11-28T11:11:01.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.944+0000 7f4356ffd700 1 -- 172.21.15.28:0/2809974168 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358007310 con 0x7f43680b1d40 2023-11-28T11:11:01.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.944+0000 7f436cdb0700 1 -- 172.21.15.28:0/2809974168 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f436815d3f0 con 0x7f43680b1d40 2023-11-28T11:11:01.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.944+0000 7f436cdb0700 1 -- 172.21.15.28:0/2809974168 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f436815d110 con 0x7f43680b1d40 2023-11-28T11:11:01.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.945+0000 7f436cdb0700 1 -- 172.21.15.28:0/2809974168 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f436815d6d0 con 0x7f43680b1d40 2023-11-28T11:11:01.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.945+0000 7f4356ffd700 1 -- 172.21.15.28:0/2809974168 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f435801a020 con 0x7f43680b1d40 2023-11-28T11:11:01.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.946+0000 7f4356ffd700 1 --2- 172.21.15.28:0/2809974168 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f43480682f0 0x7f434806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:01.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.946+0000 7f4356ffd700 1 -- 172.21.15.28:0/2809974168 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4358085a10 con 0x7f43680b1d40 2023-11-28T11:11:01.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.946+0000 7f4365d9b700 1 --2- 172.21.15.28:0/2809974168 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f43480682f0 0x7f434806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:01.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.946+0000 7f4365d9b700 1 --2- 172.21.15.28:0/2809974168 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f43480682f0 0x7f434806a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f43680718b0 tx=0x7f435c01c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:01.951 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:01.950+0000 7f4356ffd700 1 -- 172.21.15.28:0/2809974168 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f43580521e0 con 0x7f43680b1d40 2023-11-28T11:11:02.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:02.191+0000 7f436cdb0700 1 -- 172.21.15.28:0/2809974168 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f43680c69f0 con 0x7f43680b1d40 2023-11-28T11:11:03.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.112+0000 7f4356ffd700 1 -- 172.21.15.28:0/2809974168 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v25) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4358055800 con 0x7f43680b1d40 2023-11-28T11:11:03.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.114+0000 7f436cdb0700 1 -- 172.21.15.28:0/2809974168 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f43480682f0 msgr2=0x7f434806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.114+0000 7f436cdb0700 1 --2- 172.21.15.28:0/2809974168 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f43480682f0 0x7f434806a7a0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f43680718b0 tx=0x7f435c01c040 comp rx=0 tx=0).stop 2023-11-28T11:11:03.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.114+0000 7f436cdb0700 1 -- 172.21.15.28:0/2809974168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43680b1d40 msgr2=0x7f43680b5460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.114+0000 7f436cdb0700 1 --2- 172.21.15.28:0/2809974168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43680b1d40 0x7f43680b5460 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f43580011e0 tx=0x7f435800cae0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.115+0000 7f436cdb0700 1 -- 172.21.15.28:0/2809974168 shutdown_connections 2023-11-28T11:11:03.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.115+0000 7f436cdb0700 1 --2- 172.21.15.28:0/2809974168 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f43480682f0 0x7f434806a7a0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.115+0000 7f436cdb0700 1 --2- 172.21.15.28:0/2809974168 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f43680b1d40 0x7f43680b5460 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.115+0000 7f436cdb0700 1 --2- 172.21.15.28:0/2809974168 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f436815dbf0 0x7f436815b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.115+0000 7f436cdb0700 1 --2- 172.21.15.28:0/2809974168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4368159500 0x7f43680c5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.115+0000 7f436cdb0700 1 -- 172.21.15.28:0/2809974168 >> 172.21.15.28:0/2809974168 conn(0x7f43680cc630 msgr2=0x7f43680cbd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:03.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.115+0000 7f436cdb0700 1 -- 172.21.15.28:0/2809974168 shutdown_connections 2023-11-28T11:11:03.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.115+0000 7f436cdb0700 1 -- 172.21.15.28:0/2809974168 wait complete. 2023-11-28T11:11:03.131 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:11:03.131 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:03.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.262+0000 7feda0bb6700 1 Processor -- start 2023-11-28T11:11:03.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.262+0000 7feda0bb6700 1 -- start start 2023-11-28T11:11:03.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.263+0000 7feda0bb6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed9c161090 0x7fed9c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.263+0000 7feda0bb6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fed9c0caeb0 0x7fed9c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.263+0000 7feda0bb6700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed9c0ce340 con 0x7fed9c15c990 2023-11-28T11:11:03.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.263+0000 7feda0bb6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed9c0cebe0 con 0x7fed9c0caeb0 2023-11-28T11:11:03.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.263+0000 7feda0bb6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c0cf780 con 0x7fed9c161090 2023-11-28T11:11:03.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.263+0000 7fed9a59c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fed9c15c990 0x7fed9c0cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38518/0 (socket says 172.21.15.28:38518) 2023-11-28T11:11:03.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.263+0000 7fed9a59c700 1 -- 172.21.15.28:0/1799531824 learned_addr learned my addr 172.21.15.28:0/1799531824 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:03.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.263+0000 7fed9ad9d700 1 --2- 172.21.15.28:0/1799531824 >> v2:172.21.15.59:3300/0 conn(0x7fed9c0caeb0 0x7fed9c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.263+0000 7fed99d9b700 1 --2- 172.21.15.28:0/1799531824 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed9c161090 0x7fed9c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 -- 172.21.15.28:0/1799531824 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1542800994 0 0) 0x7fed9c0ce340 con 0x7fed9c15c990 2023-11-28T11:11:03.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 -- 172.21.15.28:0/1799531824 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed840035b0 con 0x7fed9c15c990 2023-11-28T11:11:03.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 -- 172.21.15.28:0/1799531824 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4274549035 0 0) 0x7fed840035b0 con 0x7fed9c15c990 2023-11-28T11:11:03.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 -- 172.21.15.28:0/1799531824 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed9c161090 msgr2=0x7fed9c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:03.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 --2- 172.21.15.28:0/1799531824 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed9c161090 0x7fed9c0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 -- 172.21.15.28:0/1799531824 >> v2:172.21.15.59:3300/0 conn(0x7fed9c0caeb0 msgr2=0x7fed9c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:03.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 --2- 172.21.15.28:0/1799531824 >> v2:172.21.15.59:3300/0 conn(0x7fed9c0caeb0 0x7fed9c0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 -- 172.21.15.28:0/1799531824 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9c0cb3f0 con 0x7fed9c15c990 2023-11-28T11:11:03.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 -- 172.21.15.28:0/1799531824 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fed90001670 con 0x7fed9c15c990 2023-11-28T11:11:03.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 -- 172.21.15.28:0/1799531824 >> v1:172.21.15.28:6789/0 conn(0x7fed9c15c990 legacy=0x7fed9c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 --2- 172.21.15.28:0/1799531824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed84003a00 0x7fed84003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 --2- 172.21.15.28:0/1799531824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed84004850 0x7fed84004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 --2- 172.21.15.28:0/1799531824 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed9c161090 0x7fed84005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 -- 172.21.15.28:0/1799531824 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c0cebe0 con 0x7fed84004850 2023-11-28T11:11:03.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 -- 172.21.15.28:0/1799531824 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c0cf780 con 0x7fed84003a00 2023-11-28T11:11:03.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9959a700 1 -- 172.21.15.28:0/1799531824 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed8400ba90 con 0x7fed9c161090 2023-11-28T11:11:03.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed99d9b700 1 --2- 172.21.15.28:0/1799531824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed84004850 0x7fed84004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9a59c700 1 --2- 172.21.15.28:0/1799531824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed84003a00 0x7fed84003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9ad9d700 1 --2- 172.21.15.28:0/1799531824 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed9c161090 0x7fed84005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed99d9b700 1 -- 172.21.15.28:0/1799531824 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed9c161090 msgr2=0x7fed84005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:03.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed99d9b700 1 --2- 172.21.15.28:0/1799531824 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed9c161090 0x7fed84005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed99d9b700 1 -- 172.21.15.28:0/1799531824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed84003a00 msgr2=0x7fed84003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:03.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed99d9b700 1 --2- 172.21.15.28:0/1799531824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed84003a00 0x7fed84003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed99d9b700 1 -- 172.21.15.28:0/1799531824 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed840035b0 con 0x7fed84004850 2023-11-28T11:11:03.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.264+0000 7fed9ad9d700 1 --2- 172.21.15.28:0/1799531824 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed9c161090 0x7fed84005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:03.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.265+0000 7fed99d9b700 1 --2- 172.21.15.28:0/1799531824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed84004850 0x7fed84004cc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009b30 tx=0x7fed84005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a380f756c1db7768 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:03.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.265+0000 7fed9959a700 1 -- 172.21.15.28:0/1799531824 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c00e6c0 con 0x7fed84004850 2023-11-28T11:11:03.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.265+0000 7fed9959a700 1 -- 172.21.15.28:0/1799531824 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed8c00e820 con 0x7fed84004850 2023-11-28T11:11:03.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.265+0000 7fed9959a700 1 -- 172.21.15.28:0/1799531824 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed8c00eb90 con 0x7fed84004850 2023-11-28T11:11:03.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.265+0000 7feda0bb6700 1 -- 172.21.15.28:0/1799531824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed84004850 msgr2=0x7fed84004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.265+0000 7feda0bb6700 1 --2- 172.21.15.28:0/1799531824 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed84004850 0x7fed84004cc0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fed8c009b30 tx=0x7fed84005880 comp rx=0 tx=0).stop 2023-11-28T11:11:03.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.265+0000 7fed9ad9d700 1 -- 172.21.15.28:0/1799531824 reap_dead start 2023-11-28T11:11:03.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.265+0000 7feda0bb6700 1 -- 172.21.15.28:0/1799531824 shutdown_connections 2023-11-28T11:11:03.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.265+0000 7feda0bb6700 1 -- 172.21.15.28:0/1799531824 >> 172.21.15.28:0/1799531824 conn(0x7fed9c0b1650 msgr2=0x7fed9c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:03.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.265+0000 7feda0bb6700 1 -- 172.21.15.28:0/1799531824 shutdown_connections 2023-11-28T11:11:03.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.265+0000 7feda0bb6700 1 -- 172.21.15.28:0/1799531824 wait complete. 2023-11-28T11:11:03.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.267+0000 7feda0bb6700 1 Processor -- start 2023-11-28T11:11:03.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.267+0000 7feda0bb6700 1 -- start start 2023-11-28T11:11:03.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7feda0bb6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed84004850 0x7fed9c15eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7feda0bb6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed9c150ec0 0x7fed9c15cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7feda0bb6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed9c0c5600 0x7fed9c0c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7feda0bb6700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c15e6a0 con 0x7fed84004850 2023-11-28T11:11:03.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7feda0bb6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c0c5fb0 con 0x7fed9c0c5600 2023-11-28T11:11:03.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7feda0bb6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed9c15e330 con 0x7fed9c150ec0 2023-11-28T11:11:03.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7fed9a59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed84004850 0x7fed9c15eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7fed9a59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed84004850 0x7fed9c15eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50472/0 (socket says 172.21.15.28:50472) 2023-11-28T11:11:03.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7fed9a59c700 1 -- 172.21.15.28:0/3340170378 learned_addr learned my addr 172.21.15.28:0/3340170378 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:03.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7fed99d9b700 1 --2- 172.21.15.28:0/3340170378 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed9c150ec0 0x7fed9c15cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7fed9ad9d700 1 --2- 172.21.15.28:0/3340170378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed9c0c5600 0x7fed9c0c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7fed9a59c700 1 -- 172.21.15.28:0/3340170378 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed9c150ec0 msgr2=0x7fed9c15cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7fed9a59c700 1 --2- 172.21.15.28:0/3340170378 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed9c150ec0 0x7fed9c15cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7fed9a59c700 1 -- 172.21.15.28:0/3340170378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed9c0c5600 msgr2=0x7fed9c0c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7fed9a59c700 1 --2- 172.21.15.28:0/3340170378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed9c0c5600 0x7fed9c0c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7fed9a59c700 1 -- 172.21.15.28:0/3340170378 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed9c0ce870 con 0x7fed84004850 2023-11-28T11:11:03.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7fed9a59c700 1 --2- 172.21.15.28:0/3340170378 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed84004850 0x7fed9c15eac0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fed90000fb0 tx=0x7fed900075f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:03.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7fed8affd700 1 -- 172.21.15.28:0/3340170378 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9000f6c0 con 0x7fed84004850 2023-11-28T11:11:03.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7fed8affd700 1 -- 172.21.15.28:0/3340170378 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed9000f860 con 0x7fed84004850 2023-11-28T11:11:03.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7feda0bb6700 1 -- 172.21.15.28:0/3340170378 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9c0ce590 con 0x7fed84004850 2023-11-28T11:11:03.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7fed8affd700 1 -- 172.21.15.28:0/3340170378 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed9000fc10 con 0x7fed84004850 2023-11-28T11:11:03.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.268+0000 7feda0bb6700 1 -- 172.21.15.28:0/3340170378 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9c0ca970 con 0x7fed84004850 2023-11-28T11:11:03.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.269+0000 7feda0bb6700 1 -- 172.21.15.28:0/3340170378 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed9c15e7e0 con 0x7fed84004850 2023-11-28T11:11:03.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.270+0000 7fed8affd700 1 -- 172.21.15.28:0/3340170378 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fed9000fd70 con 0x7fed84004850 2023-11-28T11:11:03.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.270+0000 7fed8affd700 1 --2- 172.21.15.28:0/3340170378 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed7c0682f0 0x7fed7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.271+0000 7fed8affd700 1 -- 172.21.15.28:0/3340170378 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fed90085ea0 con 0x7fed84004850 2023-11-28T11:11:03.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.271+0000 7fed99d9b700 1 --2- 172.21.15.28:0/3340170378 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed7c0682f0 0x7fed7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.271+0000 7fed99d9b700 1 --2- 172.21.15.28:0/3340170378 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed7c0682f0 0x7fed7c06a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fed8c0090b0 tx=0x7fed8c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:03.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.275+0000 7fed8affd700 1 -- 172.21.15.28:0/3340170378 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fed9c15e7e0 con 0x7fed84004850 2023-11-28T11:11:03.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.514+0000 7feda0bb6700 1 -- 172.21.15.28:0/3340170378 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fed9c15e7e0 con 0x7fed84004850 2023-11-28T11:11:03.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.514+0000 7fed8affd700 1 -- 172.21.15.28:0/3340170378 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4645 (secure 0 0 0) 0x7fed9c15e7e0 con 0x7fed84004850 2023-11-28T11:11:03.516 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:03.516 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5198,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/284515388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":284515388},{"type":"v1","addr":"172.21.15.59:6835","nonce":284515388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2160385010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2160385010},{"type":"v1","addr":"172.21.15.28:6836","nonce":2160385010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3775145347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3775145347},{"type":"v1","addr":"172.21.15.28:6837","nonce":3775145347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:59.771956+0000","modified":"2023-11-28T11:11:03.109382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6834/3261856585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3261856585},{"type":"v1","addr":"172.21.15.59:6834","nonce":3261856585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:11:03.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.516+0000 7feda0bb6700 1 -- 172.21.15.28:0/3340170378 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed7c0682f0 msgr2=0x7fed7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.516+0000 7feda0bb6700 1 --2- 172.21.15.28:0/3340170378 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed7c0682f0 0x7fed7c06a7a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fed8c0090b0 tx=0x7fed8c008030 comp rx=0 tx=0).stop 2023-11-28T11:11:03.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.516+0000 7feda0bb6700 1 -- 172.21.15.28:0/3340170378 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed84004850 msgr2=0x7fed9c15eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.516+0000 7feda0bb6700 1 --2- 172.21.15.28:0/3340170378 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed84004850 0x7fed9c15eac0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fed90000fb0 tx=0x7fed900075f0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.516+0000 7feda0bb6700 1 -- 172.21.15.28:0/3340170378 shutdown_connections 2023-11-28T11:11:03.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.516+0000 7feda0bb6700 1 --2- 172.21.15.28:0/3340170378 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fed7c0682f0 0x7fed7c06a7a0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.516+0000 7feda0bb6700 1 --2- 172.21.15.28:0/3340170378 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fed84004850 0x7fed9c15eac0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.516+0000 7feda0bb6700 1 --2- 172.21.15.28:0/3340170378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed9c0c5600 0x7fed9c0c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.516+0000 7feda0bb6700 1 --2- 172.21.15.28:0/3340170378 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fed9c150ec0 0x7fed9c15cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.516+0000 7feda0bb6700 1 -- 172.21.15.28:0/3340170378 >> 172.21.15.28:0/3340170378 conn(0x7fed9c0b1650 msgr2=0x7fed9c0cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:03.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.516+0000 7feda0bb6700 1 -- 172.21.15.28:0/3340170378 shutdown_connections 2023-11-28T11:11:03.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.516+0000 7feda0bb6700 1 -- 172.21.15.28:0/3340170378 wait complete. 2023-11-28T11:11:03.519 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 25 2023-11-28T11:11:03.530 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:11:03.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.653+0000 7f6257732700 1 Processor -- start 2023-11-28T11:11:03.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.653+0000 7f6257732700 1 -- start start 2023-11-28T11:11:03.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.654+0000 7f6257732700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f625015c6b0 0x7f62500ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.654+0000 7f6257732700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62500cb9b0 con 0x7f62500c9930 2023-11-28T11:11:03.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.654+0000 7f6257732700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62500cd980 con 0x7f625006a560 2023-11-28T11:11:03.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.654+0000 7f6257732700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62500caf50 con 0x7f625015c6b0 2023-11-28T11:11:03.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.654+0000 7f6254ccd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f62500c9930 0x7f625006d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38538/0 (socket says 172.21.15.28:38538) 2023-11-28T11:11:03.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.654+0000 7f6254ccd700 1 -- 172.21.15.28:0/1508607557 learned_addr learned my addr 172.21.15.28:0/1508607557 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:03.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f62554ce700 1 --2- 172.21.15.28:0/1508607557 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f625015c6b0 0x7f62500ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6247fff700 1 -- 172.21.15.28:0/1508607557 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 105278885 0 0) 0x7f62500cb9b0 con 0x7f62500c9930 2023-11-28T11:11:03.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6247fff700 1 -- 172.21.15.28:0/1508607557 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f623c0035b0 con 0x7f62500c9930 2023-11-28T11:11:03.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6247fff700 1 -- 172.21.15.28:0/1508607557 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3878115876 0 0) 0x7f623c0035b0 con 0x7f62500c9930 2023-11-28T11:11:03.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6247fff700 1 -- 172.21.15.28:0/1508607557 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f625015c6b0 msgr2=0x7f62500ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:03.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6247fff700 1 --2- 172.21.15.28:0/1508607557 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f625015c6b0 0x7f62500ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6247fff700 1 -- 172.21.15.28:0/1508607557 >> v1:172.21.15.59:6789/0 conn(0x7f625006a560 legacy=0x7f62500cde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6247fff700 1 -- 172.21.15.28:0/1508607557 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625015ca80 con 0x7f62500c9930 2023-11-28T11:11:03.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6247fff700 1 -- 172.21.15.28:0/1508607557 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f624c001670 con 0x7f62500c9930 2023-11-28T11:11:03.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6247fff700 1 -- 172.21.15.28:0/1508607557 >> v1:172.21.15.28:6789/0 conn(0x7f62500c9930 legacy=0x7f625006d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6247fff700 1 --2- 172.21.15.28:0/1508607557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f623c003a30 0x7f623c003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6247fff700 1 --2- 172.21.15.28:0/1508607557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f623c004850 0x7f623c004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6247fff700 1 --2- 172.21.15.28:0/1508607557 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f625015c6b0 0x7f623c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6247fff700 1 -- 172.21.15.28:0/1508607557 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62500caf50 con 0x7f623c004850 2023-11-28T11:11:03.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6247fff700 1 -- 172.21.15.28:0/1508607557 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f623c00bac0 con 0x7f623c003a30 2023-11-28T11:11:03.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6247fff700 1 -- 172.21.15.28:0/1508607557 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f623c00bc50 con 0x7f625015c6b0 2023-11-28T11:11:03.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6254ccd700 1 --2- 172.21.15.28:0/1508607557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f623c004850 0x7f623c004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f62554ce700 1 --2- 172.21.15.28:0/1508607557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f623c003a30 0x7f623c003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6255ccf700 1 --2- 172.21.15.28:0/1508607557 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f625015c6b0 0x7f623c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6254ccd700 1 -- 172.21.15.28:0/1508607557 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f625015c6b0 msgr2=0x7f623c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:03.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6254ccd700 1 --2- 172.21.15.28:0/1508607557 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f625015c6b0 0x7f623c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6254ccd700 1 -- 172.21.15.28:0/1508607557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f623c003a30 msgr2=0x7f623c003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:03.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6254ccd700 1 --2- 172.21.15.28:0/1508607557 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f623c003a30 0x7f623c003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6254ccd700 1 -- 172.21.15.28:0/1508607557 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c0035b0 con 0x7f623c004850 2023-11-28T11:11:03.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.655+0000 7f6255ccf700 1 --2- 172.21.15.28:0/1508607557 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f625015c6b0 0x7f623c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:03.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.656+0000 7f6254ccd700 1 --2- 172.21.15.28:0/1508607557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f623c004850 0x7f623c004cc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f624c00c240 tx=0x7f623c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=292d13d97830b6e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:03.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.656+0000 7f6247fff700 1 -- 172.21.15.28:0/1508607557 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c007c10 con 0x7f623c004850 2023-11-28T11:11:03.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.656+0000 7f6247fff700 1 -- 172.21.15.28:0/1508607557 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f624c00cde0 con 0x7f623c004850 2023-11-28T11:11:03.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.656+0000 7f6247fff700 1 -- 172.21.15.28:0/1508607557 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f624c005ba0 con 0x7f623c004850 2023-11-28T11:11:03.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.656+0000 7f6257732700 1 -- 172.21.15.28:0/1508607557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f623c004850 msgr2=0x7f623c004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.656+0000 7f6257732700 1 --2- 172.21.15.28:0/1508607557 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f623c004850 0x7f623c004cc0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f624c00c240 tx=0x7f623c005880 comp rx=0 tx=0).stop 2023-11-28T11:11:03.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.656+0000 7f6255ccf700 1 -- 172.21.15.28:0/1508607557 reap_dead start 2023-11-28T11:11:03.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.656+0000 7f6257732700 1 -- 172.21.15.28:0/1508607557 shutdown_connections 2023-11-28T11:11:03.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.656+0000 7f6257732700 1 -- 172.21.15.28:0/1508607557 >> 172.21.15.28:0/1508607557 conn(0x7f62500b0dd0 msgr2=0x7f62500b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:03.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.656+0000 7f6257732700 1 -- 172.21.15.28:0/1508607557 shutdown_connections 2023-11-28T11:11:03.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.656+0000 7f6257732700 1 -- 172.21.15.28:0/1508607557 wait complete. 2023-11-28T11:11:03.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.658+0000 7f6257732700 1 Processor -- start 2023-11-28T11:11:03.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.658+0000 7f6257732700 1 -- start start 2023-11-28T11:11:03.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.658+0000 7f6257732700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f623c004850 0x7f62500ae1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.658+0000 7f6257732700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6250150380 0x7f62500b4980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.658+0000 7f6257732700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62500c4e80 0x7f62500c52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.658+0000 7f6257732700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6250150ca0 con 0x7f62500c4e80 2023-11-28T11:11:03.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.658+0000 7f6257732700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62500ae6e0 con 0x7f623c004850 2023-11-28T11:11:03.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.658+0000 7f6257732700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62500c5830 con 0x7f6250150380 2023-11-28T11:11:03.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f6255ccf700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62500c4e80 0x7f62500c52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f6255ccf700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62500c4e80 0x7f62500c52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50496/0 (socket says 172.21.15.28:50496) 2023-11-28T11:11:03.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f6255ccf700 1 -- 172.21.15.28:0/1324554455 learned_addr learned my addr 172.21.15.28:0/1324554455 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:03.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f62554ce700 1 --2- 172.21.15.28:0/1324554455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f623c004850 0x7f62500ae1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f6254ccd700 1 --2- 172.21.15.28:0/1324554455 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6250150380 0x7f62500b4980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f6255ccf700 1 -- 172.21.15.28:0/1324554455 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6250150380 msgr2=0x7f62500b4980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f6255ccf700 1 --2- 172.21.15.28:0/1324554455 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6250150380 0x7f62500b4980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f6255ccf700 1 -- 172.21.15.28:0/1324554455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f623c004850 msgr2=0x7f62500ae1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f6255ccf700 1 --2- 172.21.15.28:0/1324554455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f623c004850 0x7f62500ae1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f6255ccf700 1 -- 172.21.15.28:0/1324554455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62500cd050 con 0x7f62500c4e80 2023-11-28T11:11:03.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f62554ce700 1 --2- 172.21.15.28:0/1324554455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f623c004850 0x7f62500ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:03.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f6255ccf700 1 --2- 172.21.15.28:0/1324554455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62500c4e80 0x7f62500c52f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6250070df0 tx=0x7f624800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:03.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f6245ffb700 1 -- 172.21.15.28:0/1324554455 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248003c10 con 0x7f62500c4e80 2023-11-28T11:11:03.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f6245ffb700 1 -- 172.21.15.28:0/1324554455 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6248003d90 con 0x7f62500c4e80 2023-11-28T11:11:03.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f6257732700 1 -- 172.21.15.28:0/1324554455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625006dd40 con 0x7f62500c4e80 2023-11-28T11:11:03.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.659+0000 7f6245ffb700 1 -- 172.21.15.28:0/1324554455 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6248005310 con 0x7f62500c4e80 2023-11-28T11:11:03.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.660+0000 7f6257732700 1 -- 172.21.15.28:0/1324554455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62500cd330 con 0x7f62500c4e80 2023-11-28T11:11:03.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.660+0000 7f6257732700 1 -- 172.21.15.28:0/1324554455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6250068570 con 0x7f62500c4e80 2023-11-28T11:11:03.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.661+0000 7f6245ffb700 1 -- 172.21.15.28:0/1324554455 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6248005470 con 0x7f62500c4e80 2023-11-28T11:11:03.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.661+0000 7f6245ffb700 1 --2- 172.21.15.28:0/1324554455 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f62340682f0 0x7f623406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:03.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.661+0000 7f62554ce700 1 --2- 172.21.15.28:0/1324554455 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f62340682f0 0x7f623406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:03.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.661+0000 7f6245ffb700 1 -- 172.21.15.28:0/1324554455 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6248085f50 con 0x7f62500c4e80 2023-11-28T11:11:03.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.662+0000 7f62554ce700 1 --2- 172.21.15.28:0/1324554455 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f62340682f0 0x7f623406a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6240009c60 tx=0x7f6240008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:03.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.666+0000 7f6245ffb700 1 -- 172.21.15.28:0/1324554455 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6248052670 con 0x7f62500c4e80 2023-11-28T11:11:03.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.863+0000 7f6257732700 1 -- 172.21.15.28:0/1324554455 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f62500b5260 con 0x7f62500c4e80 2023-11-28T11:11:03.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.864+0000 7f6245ffb700 1 -- 172.21.15.28:0/1324554455 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18074 (secure 0 0 0) 0x7f6248055c90 con 0x7f62500c4e80 2023-11-28T11:11:03.866 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:03.866 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":54,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:11:00.773581+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:57.702079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:58.738753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:03.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.866+0000 7f6257732700 1 -- 172.21.15.28:0/1324554455 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f62340682f0 msgr2=0x7f623406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.866+0000 7f6257732700 1 --2- 172.21.15.28:0/1324554455 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f62340682f0 0x7f623406a7a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f6240009c60 tx=0x7f6240008040 comp rx=0 tx=0).stop 2023-11-28T11:11:03.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.866+0000 7f6257732700 1 -- 172.21.15.28:0/1324554455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62500c4e80 msgr2=0x7f62500c52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:03.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.866+0000 7f6257732700 1 --2- 172.21.15.28:0/1324554455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62500c4e80 0x7f62500c52f0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f6250070df0 tx=0x7f624800ee70 comp rx=0 tx=0).stop 2023-11-28T11:11:03.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.866+0000 7f6257732700 1 -- 172.21.15.28:0/1324554455 shutdown_connections 2023-11-28T11:11:03.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.866+0000 7f6257732700 1 --2- 172.21.15.28:0/1324554455 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f62340682f0 0x7f623406a7a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.866+0000 7f6257732700 1 --2- 172.21.15.28:0/1324554455 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f62500c4e80 0x7f62500c52f0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.866+0000 7f6257732700 1 --2- 172.21.15.28:0/1324554455 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f623c004850 0x7f62500ae1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.866+0000 7f6257732700 1 --2- 172.21.15.28:0/1324554455 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6250150380 0x7f62500b4980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:03.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.866+0000 7f6257732700 1 -- 172.21.15.28:0/1324554455 >> 172.21.15.28:0/1324554455 conn(0x7f62500b0dd0 msgr2=0x7f62500aefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:03.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.866+0000 7f6257732700 1 -- 172.21.15.28:0/1324554455 shutdown_connections 2023-11-28T11:11:03.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:03.866+0000 7f6257732700 1 -- 172.21.15.28:0/1324554455 wait complete. 2023-11-28T11:11:03.880 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:11:03.881 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:11:04.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.004+0000 7f5f79082700 1 Processor -- start 2023-11-28T11:11:04.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.004+0000 7f5f79082700 1 -- start start 2023-11-28T11:11:04.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.004+0000 7f5f79082700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f740b4fe0 0x7f5f7415d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.004+0000 7f5f79082700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5f740cd4a0 0x7f5f740d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.004+0000 7f5f79082700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f7406abb0 con 0x7f5f7415a230 2023-11-28T11:11:04.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.004+0000 7f5f79082700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f7415a600 con 0x7f5f740cd4a0 2023-11-28T11:11:04.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f79082700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7406a280 con 0x7f5f740b4fe0 2023-11-28T11:11:04.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f72d9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5f7415a230 0x7f5f740b48d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38558/0 (socket says 172.21.15.28:38558) 2023-11-28T11:11:04.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f72d9d700 1 -- 172.21.15.28:0/3117914243 learned_addr learned my addr 172.21.15.28:0/3117914243 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:04.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f7259c700 1 --2- 172.21.15.28:0/3117914243 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f740b4fe0 0x7f5f7415d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f7359e700 1 --2- 172.21.15.28:0/3117914243 >> v2:172.21.15.59:3300/0 conn(0x7f5f740cd4a0 0x7f5f740d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 -- 172.21.15.28:0/3117914243 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651349601 0 0) 0x7f5f7406abb0 con 0x7f5f7415a230 2023-11-28T11:11:04.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 -- 172.21.15.28:0/3117914243 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5f5c0035b0 con 0x7f5f7415a230 2023-11-28T11:11:04.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 -- 172.21.15.28:0/3117914243 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 847037885 0 0) 0x7f5f5c0035b0 con 0x7f5f7415a230 2023-11-28T11:11:04.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 -- 172.21.15.28:0/3117914243 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f740b4fe0 msgr2=0x7f5f7415d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:04.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 --2- 172.21.15.28:0/3117914243 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f740b4fe0 0x7f5f7415d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 -- 172.21.15.28:0/3117914243 >> v2:172.21.15.59:3300/0 conn(0x7f5f740cd4a0 msgr2=0x7f5f740d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:04.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 --2- 172.21.15.28:0/3117914243 >> v2:172.21.15.59:3300/0 conn(0x7f5f740cd4a0 0x7f5f740d0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 -- 172.21.15.28:0/3117914243 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7406a560 con 0x7f5f7415a230 2023-11-28T11:11:04.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 -- 172.21.15.28:0/3117914243 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f5f64001670 con 0x7f5f7415a230 2023-11-28T11:11:04.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 -- 172.21.15.28:0/3117914243 >> v1:172.21.15.28:6789/0 conn(0x7f5f7415a230 legacy=0x7f5f740b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 --2- 172.21.15.28:0/3117914243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f5c003a00 0x7f5f5c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 --2- 172.21.15.28:0/3117914243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f5c004430 0x7f5f5c0048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 --2- 172.21.15.28:0/3117914243 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f740b4fe0 0x7f5f5c004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 -- 172.21.15.28:0/3117914243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7415a600 con 0x7f5f5c004430 2023-11-28T11:11:04.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 -- 172.21.15.28:0/3117914243 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f7406a280 con 0x7f5f5c003a00 2023-11-28T11:11:04.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f71d9b700 1 -- 172.21.15.28:0/3117914243 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f5c00ba10 con 0x7f5f740b4fe0 2023-11-28T11:11:04.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f7259c700 1 --2- 172.21.15.28:0/3117914243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f5c004430 0x7f5f5c0048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.005+0000 7f5f72d9d700 1 --2- 172.21.15.28:0/3117914243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f5c003a00 0x7f5f5c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.006+0000 7f5f7359e700 1 --2- 172.21.15.28:0/3117914243 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f740b4fe0 0x7f5f5c004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.006+0000 7f5f7259c700 1 -- 172.21.15.28:0/3117914243 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f740b4fe0 msgr2=0x7f5f5c004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:04.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.006+0000 7f5f7259c700 1 --2- 172.21.15.28:0/3117914243 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f740b4fe0 0x7f5f5c004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.006+0000 7f5f7259c700 1 -- 172.21.15.28:0/3117914243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f5c003a00 msgr2=0x7f5f5c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:04.016 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.006+0000 7f5f7259c700 1 --2- 172.21.15.28:0/3117914243 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f5c003a00 0x7f5f5c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.016 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.006+0000 7f5f7259c700 1 -- 172.21.15.28:0/3117914243 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f5c0035b0 con 0x7f5f5c004430 2023-11-28T11:11:04.016 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.006+0000 7f5f7259c700 1 --2- 172.21.15.28:0/3117914243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f5c004430 0x7f5f5c0048f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009b30 tx=0x7f5f5c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8e23fa2781b102 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.016 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.006+0000 7f5f71d9b700 1 -- 172.21.15.28:0/3117914243 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800e6c0 con 0x7f5f5c004430 2023-11-28T11:11:04.016 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.006+0000 7f5f71d9b700 1 -- 172.21.15.28:0/3117914243 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6800e820 con 0x7f5f5c004430 2023-11-28T11:11:04.016 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.006+0000 7f5f71d9b700 1 -- 172.21.15.28:0/3117914243 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6800eb90 con 0x7f5f5c004430 2023-11-28T11:11:04.016 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.006+0000 7f5f79082700 1 -- 172.21.15.28:0/3117914243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f5c004430 msgr2=0x7f5f5c0048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.006+0000 7f5f79082700 1 --2- 172.21.15.28:0/3117914243 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f5c004430 0x7f5f5c0048f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f5f68009b30 tx=0x7f5f5c005880 comp rx=0 tx=0).stop 2023-11-28T11:11:04.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.006+0000 7f5f7359e700 1 -- 172.21.15.28:0/3117914243 reap_dead start 2023-11-28T11:11:04.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.006+0000 7f5f79082700 1 -- 172.21.15.28:0/3117914243 shutdown_connections 2023-11-28T11:11:04.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.006+0000 7f5f79082700 1 -- 172.21.15.28:0/3117914243 >> 172.21.15.28:0/3117914243 conn(0x7f5f740cf8d0 msgr2=0x7f5f740cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:04.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.007+0000 7f5f79082700 1 -- 172.21.15.28:0/3117914243 shutdown_connections 2023-11-28T11:11:04.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.007+0000 7f5f79082700 1 -- 172.21.15.28:0/3117914243 wait complete. 2023-11-28T11:11:04.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.008+0000 7f5f79082700 1 Processor -- start 2023-11-28T11:11:04.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.008+0000 7f5f79082700 1 -- start start 2023-11-28T11:11:04.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f79082700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f5c004430 0x7f5f740c8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f79082700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f740c92e0 0x7f5f740c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f79082700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f741f6350 0x7f5f741f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f79082700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f74155df0 con 0x7f5f741f6350 2023-11-28T11:11:04.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f79082700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740cc8a0 con 0x7f5f740c92e0 2023-11-28T11:11:04.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f79082700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f740cc000 con 0x7f5f5c004430 2023-11-28T11:11:04.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f7359e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f741f6350 0x7f5f741f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f7359e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f741f6350 0x7f5f741f67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50516/0 (socket says 172.21.15.28:50516) 2023-11-28T11:11:04.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f7359e700 1 -- 172.21.15.28:0/3441224227 learned_addr learned my addr 172.21.15.28:0/3441224227 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:04.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f7259c700 1 --2- 172.21.15.28:0/3441224227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f740c92e0 0x7f5f740c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f72d9d700 1 --2- 172.21.15.28:0/3441224227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f5c004430 0x7f5f740c8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f7359e700 1 -- 172.21.15.28:0/3441224227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f5c004430 msgr2=0x7f5f740c8830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f7359e700 1 --2- 172.21.15.28:0/3441224227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f5c004430 0x7f5f740c8830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f7359e700 1 -- 172.21.15.28:0/3441224227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f740c92e0 msgr2=0x7f5f740c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f7359e700 1 --2- 172.21.15.28:0/3441224227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f740c92e0 0x7f5f740c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f7359e700 1 -- 172.21.15.28:0/3441224227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f740cd920 con 0x7f5f741f6350 2023-11-28T11:11:04.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.009+0000 7f5f7359e700 1 --2- 172.21.15.28:0/3441224227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f741f6350 0x7f5f741f67c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5f74160050 tx=0x7f5f6c0111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.010+0000 7f5f5b7fe700 1 -- 172.21.15.28:0/3441224227 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c00e070 con 0x7f5f741f6350 2023-11-28T11:11:04.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.010+0000 7f5f5b7fe700 1 -- 172.21.15.28:0/3441224227 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f6c003cf0 con 0x7f5f741f6350 2023-11-28T11:11:04.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.010+0000 7f5f79082700 1 -- 172.21.15.28:0/3441224227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f7406a840 con 0x7f5f741f6350 2023-11-28T11:11:04.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.010+0000 7f5f5b7fe700 1 -- 172.21.15.28:0/3441224227 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f6c003ea0 con 0x7f5f741f6350 2023-11-28T11:11:04.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.010+0000 7f5f79082700 1 -- 172.21.15.28:0/3441224227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f7406b040 con 0x7f5f741f6350 2023-11-28T11:11:04.022 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.011+0000 7f5f79082700 1 -- 172.21.15.28:0/3441224227 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f740675a0 con 0x7f5f741f6350 2023-11-28T11:11:04.022 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.011+0000 7f5f5b7fe700 1 -- 172.21.15.28:0/3441224227 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5f6c011770 con 0x7f5f741f6350 2023-11-28T11:11:04.022 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.011+0000 7f5f5b7fe700 1 --2- 172.21.15.28:0/3441224227 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5f600682f0 0x7f5f6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.022 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.011+0000 7f5f5b7fe700 1 -- 172.21.15.28:0/3441224227 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5f6c086420 con 0x7f5f741f6350 2023-11-28T11:11:04.022 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.011+0000 7f5f72d9d700 1 --2- 172.21.15.28:0/3441224227 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5f600682f0 0x7f5f6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.022 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.012+0000 7f5f72d9d700 1 --2- 172.21.15.28:0/3441224227 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5f600682f0 0x7f5f6006a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5f640015d0 tx=0x7f5f64007980 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.022 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.016+0000 7f5f5b7fe700 1 -- 172.21.15.28:0/3441224227 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5f6c052ae0 con 0x7f5f741f6350 2023-11-28T11:11:04.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.205+0000 7f5f79082700 1 -- 172.21.15.28:0/3441224227 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f740d3110 con 0x7f5f600682f0 2023-11-28T11:11:04.214 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.213+0000 7f5f5b7fe700 1 -- 172.21.15.28:0/3441224227 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296768 (secure 0 0 0) 0x7f5f740d3110 con 0x7f5f600682f0 2023-11-28T11:11:04.221 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:04.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.216+0000 7f5f79082700 1 -- 172.21.15.28:0/3441224227 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5f600682f0 msgr2=0x7f5f6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.216+0000 7f5f79082700 1 --2- 172.21.15.28:0/3441224227 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5f600682f0 0x7f5f6006a7a0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5f640015d0 tx=0x7f5f64007980 comp rx=0 tx=0).stop 2023-11-28T11:11:04.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.216+0000 7f5f79082700 1 -- 172.21.15.28:0/3441224227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f741f6350 msgr2=0x7f5f741f67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.216+0000 7f5f79082700 1 --2- 172.21.15.28:0/3441224227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f741f6350 0x7f5f741f67c0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f5f74160050 tx=0x7f5f6c0111d0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.216+0000 7f5f79082700 1 -- 172.21.15.28:0/3441224227 shutdown_connections 2023-11-28T11:11:04.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.216+0000 7f5f79082700 1 --2- 172.21.15.28:0/3441224227 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5f600682f0 0x7f5f6006a7a0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.216+0000 7f5f79082700 1 --2- 172.21.15.28:0/3441224227 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5f741f6350 0x7f5f741f67c0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.216+0000 7f5f79082700 1 --2- 172.21.15.28:0/3441224227 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5f5c004430 0x7f5f740c8830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.216+0000 7f5f79082700 1 --2- 172.21.15.28:0/3441224227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5f740c92e0 0x7f5f740c9750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.216+0000 7f5f79082700 1 -- 172.21.15.28:0/3441224227 >> 172.21.15.28:0/3441224227 conn(0x7f5f740cf8d0 msgr2=0x7f5f740b5c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:04.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.216+0000 7f5f79082700 1 -- 172.21.15.28:0/3441224227 shutdown_connections 2023-11-28T11:11:04.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.216+0000 7f5f79082700 1 -- 172.21.15.28:0/3441224227 wait complete. 2023-11-28T11:11:04.227 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:11:04.230 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2023-11-28T11:11:03.192682+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":74276,"kb_used_data":5796,"kb_used_omap":0,"kb_used_meta":68416,"kb_avail":754900444,"statfs":{"total":773094113280,"available":773018054656,"internally_reserved":0,"allocated":5935104,"data_stored":3345263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70057984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.851208"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:56.391440+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:10:56.391440+0000","last_peered":"2023-11-28T11:10:56.391440+0000","last_clean":"2023-11-28T11:10:56.391440+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:10:56.391440+0000","last_undegraded":"2023-11-28T11:10:56.391440+0000","last_fullsized":"2023-11-28T11:10:56.391440+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:10:54.469632+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:10:54.469632+0000","last_peered":"2023-11-28T11:10:54.469632+0000","last_clean":"2023-11-28T11:10:54.469632+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:10:54.469632+0000","last_undegraded":"2023-11-28T11:10:54.469632+0000","last_fullsized":"2023-11-28T11:10:54.469632+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:54.475122+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:10:54.475122+0000","last_peered":"2023-11-28T11:10:54.475122+0000","last_clean":"2023-11-28T11:10:54.475122+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:10:54.475122+0000","last_undegraded":"2023-11-28T11:10:54.475122+0000","last_fullsized":"2023-11-28T11:10:54.475122+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":189,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:56.391566+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:10:56.391566+0000","last_peered":"2023-11-28T11:10:56.391566+0000","last_clean":"2023-11-28T11:10:56.391566+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:10:56.391566+0000","last_undegraded":"2023-11-28T11:10:56.391566+0000","last_fullsized":"2023-11-28T11:10:56.391566+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:56.391607+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:10:56.391607+0000","last_peered":"2023-11-28T11:10:56.391607+0000","last_clean":"2023-11-28T11:10:56.391607+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:10:56.391607+0000","last_undegraded":"2023-11-28T11:10:56.391607+0000","last_fullsized":"2023-11-28T11:10:56.391607+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:56.391482+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:10:56.391482+0000","last_peered":"2023-11-28T11:10:56.391482+0000","last_clean":"2023-11-28T11:10:56.391482+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:10:56.391482+0000","last_undegraded":"2023-11-28T11:10:56.391482+0000","last_fullsized":"2023-11-28T11:10:56.391482+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:54.474339+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:10:54.474339+0000","last_peered":"2023-11-28T11:10:54.474339+0000","last_clean":"2023-11-28T11:10:54.474339+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:10:54.474339+0000","last_undegraded":"2023-11-28T11:10:54.474339+0000","last_fullsized":"2023-11-28T11:10:54.474339+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:54.691507+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:10:54.691507+0000","last_peered":"2023-11-28T11:10:54.691507+0000","last_clean":"2023-11-28T11:10:54.691507+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:10:54.691507+0000","last_undegraded":"2023-11-28T11:10:54.691507+0000","last_fullsized":"2023-11-28T11:10:54.691507+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:54.691486+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:10:54.691486+0000","last_peered":"2023-11-28T11:10:54.691486+0000","last_clean":"2023-11-28T11:10:54.691486+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:10:54.691486+0000","last_undegraded":"2023-11-28T11:10:54.691486+0000","last_fullsized":"2023-11-28T11:10:54.691486+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10064,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94361776,"statfs":{"total":96636764160,"available":96626458624,"internally_reserved":0,"allocated":1187840,"data_stored":861054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.621,"15min":0.64700000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60999999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.373},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.626,"15min":0.64400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69499999999999995,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.40000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.67100000000000004,"15min":0.68999999999999995},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.879,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.38,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71999999999999997,"5min":1.224,"15min":1.224},"last":0.52900000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.67100000000000004,"15min":0.69299999999999995},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72399999999999998,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.76000000000000001,"15min":0.78400000000000003},"min":{"1min":0.46400000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.80700000000000005,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.58699999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.68100000000000005,"15min":0.70199999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.75600000000000001,"15min":0.77600000000000002},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85699999999999998,"5min":3.294,"15min":3.294},"last":0.65600000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77600000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.36199999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.76100000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.629}]},{"osd":5,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.96299999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.41099999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67600000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86899999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.41199999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86499999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69699999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8972,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362868,"statfs":{"total":96636764160,"available":96627576832,"internally_reserved":0,"allocated":593920,"data_stored":270686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.46000000000000002,"15min":0.45400000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.72099999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.624,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.56599999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51200000000000001,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77100000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.503}]},{"osd":2,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34100000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.753,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.67100000000000004,"15min":0.68500000000000005},"min":{"1min":0.42999999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84099999999999997,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.45700000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77900000000000003,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90100000000000002,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.75700000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54500000000000004,"15min":0.54600000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10064,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94361776,"statfs":{"total":96636764160,"available":96626458624,"internally_reserved":0,"allocated":1187840,"data_stored":861054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.32800000000000001,"15min":0.32700000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.35599999999999998,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54400000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.32000000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38100000000000001,"15min":0.38},"min":{"1min":0.29099999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.58999999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.28999999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.39400000000000002,"15min":0.38800000000000001},"min":{"1min":0.28100000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40799999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69999999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.27900000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53000000000000003,"15min":0.52700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88200000000000001,"5min":1.014,"15min":1.014},"last":0.51900000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.39900000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.79000000000000004,"5min":0.998,"15min":0.998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.76000000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.442,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79300000000000004,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81000000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.627}]},{"osd":7,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85199999999999998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.59199999999999997},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.82399999999999995,"5min":1.071,"15min":1.071},"last":0.55900000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9100,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362740,"statfs":{"total":96636764160,"available":96627445760,"internally_reserved":0,"allocated":593920,"data_stored":270686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.316},{"interface":"front","average":{"1min":0.373,"5min":0.37,"15min":0.37},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38,"15min":0.38300000000000001},"min":{"1min":0.26800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.372},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39700000000000002,"15min":0.39900000000000002},"min":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57999999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.26700000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36799999999999999,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.48999999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.58999999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39200000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81599999999999995,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86299999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.65100000000000002,"15min":0.62},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":5.4269999999999996,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.62,"15min":0.622},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.79900000000000004,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.51100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89500000000000002,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.44800000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.88200000000000001,"5min":1.016,"15min":1.016},"last":0.65400000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85799999999999998,"5min":1.105,"15min":1.105},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.505,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66700000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8904,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94362936,"statfs":{"total":96636764160,"available":96627646464,"internally_reserved":0,"allocated":589824,"data_stored":269725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.36699999999999999,"15min":0.374},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.41699999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38100000000000001,"15min":0.38700000000000001},"min":{"1min":0.248,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51900000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.34100000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.39000000000000001,"15min":0.39400000000000002},"min":{"1min":0.26100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51000000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60899999999999999,"5min":1.528,"15min":1.528},"last":0.38300000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.39000000000000001,"15min":0.39600000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69599999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.46899999999999997,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.59699999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.36699999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79000000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.54600000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75600000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.622},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.85699999999999998,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53100000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.66400000000000003,"15min":0.67100000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.60799999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64300000000000002,"15min":0.64800000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.623,"15min":0.628},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.56699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9036,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362804,"statfs":{"total":96636764160,"available":96627511296,"internally_reserved":0,"allocated":593920,"data_stored":270686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.83399999999999996,"15min":0.92000000000000004},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.46300000000000002,"5min":11.965,"15min":11.965},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.87,"15min":0.95799999999999996},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55000000000000004,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.38100000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.33900000000000002,"15min":0.33700000000000002},"min":{"1min":0.23999999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":1.0940000000000001,"15min":1.2150000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56799999999999995,"5min":16.988,"15min":16.988},"last":0.40300000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.49399999999999999,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75600000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":1.6020000000000001,"15min":1.7969999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64100000000000001,"5min":27.887,"15min":27.887},"last":0.45700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.68799999999999994,"15min":0.70899999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.75900000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.65800000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.748,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.51100000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.67900000000000005,"15min":0.69199999999999995},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.745,"5min":1.98,"15min":1.98},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":1.1220000000000001,"15min":1.21},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78100000000000003,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.61399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.1279999999999999,"15min":1.2170000000000001},"min":{"1min":0.44,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73799999999999999,"5min":11.888,"15min":11.888},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.71799999999999997,"15min":0.73699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.76000000000000001,"5min":1.925,"15min":1.925},"last":0.626}]},{"osd":7,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77300000000000002,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.498,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.79700000000000004,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.70099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9164,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362676,"statfs":{"total":96636764160,"available":96627380224,"internally_reserved":0,"allocated":593920,"data_stored":270686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.748,"5min":0.996,"15min":0.996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.90300000000000002,"15min":0.96499999999999997},"min":{"1min":0.41099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.70099999999999996,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.34999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72499999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70499999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.66500000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67900000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.375,"5min":0.374,"15min":0.374},"max":{"1min":0.749,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61299999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.72899999999999998,"15min":0.75900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70399999999999996,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.47699999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71599999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.71099999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33100000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71999999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.35799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79300000000000004,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.73099999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59999999999999998,"15min":0.60799999999999998},"min":{"1min":0.35899999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76400000000000001,"5min":1.145,"15min":1.145},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.374,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":1.085,"15min":1.085},"last":0.747}]},{"osd":7,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.40600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.80300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8972,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362868,"statfs":{"total":96636764160,"available":96627576832,"internally_reserved":0,"allocated":593920,"data_stored":270686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.36499999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71199999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.443},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73099999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68000000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79300000000000004,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.66100000000000003,"15min":0.67600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.71199999999999997,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.56000000000000005}]},{"osd":2,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78300000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73699999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.496}]},{"osd":3,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.67200000000000004,"15min":0.69099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80400000000000005,"5min":1.042,"15min":1.042},"last":0.64900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.629,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.75900000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.71799999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73399999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72699999999999998,"5min":1.036,"15min":1.036},"last":0.60399999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.68200000000000005,"15min":0.69599999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78300000000000003,"5min":1.329,"15min":1.329},"last":0.749},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75800000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.69299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:11:04.237 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:11:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.394+0000 7f258aaac700 1 Processor -- start 2023-11-28T11:11:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.394+0000 7f258aaac700 1 -- start start 2023-11-28T11:11:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.395+0000 7f258aaac700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f25840d2820 0x7f25840b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.395+0000 7f258aaac700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f258406bf30 0x7f258406c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.395+0000 7f258aaac700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258406c870 con 0x7f258415c640 2023-11-28T11:11:04.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.395+0000 7f258aaac700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f258415ca10 con 0x7f25840d2820 2023-11-28T11:11:04.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.395+0000 7f258aaac700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25840d1470 con 0x7f258406bf30 2023-11-28T11:11:04.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.395+0000 7f2583fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f25840d2820 0x7f25840b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.395+0000 7f2588848700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f258415c640 0x7f25840cf040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38578/0 (socket says 172.21.15.28:38578) 2023-11-28T11:11:04.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.395+0000 7f2588848700 1 -- 172.21.15.28:0/2850295340 learned_addr learned my addr 172.21.15.28:0/2850295340 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:04.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.395+0000 7f2583fff700 1 -- 172.21.15.28:0/2850295340 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f258406bf30 msgr2=0x7f258406c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:04.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.395+0000 7f2583fff700 1 --2- 172.21.15.28:0/2850295340 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f258406bf30 0x7f258406c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.395+0000 7f2583fff700 1 -- 172.21.15.28:0/2850295340 >> v1:172.21.15.28:6789/0 conn(0x7f258415c640 legacy=0x7f25840cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.395+0000 7f25837fe700 1 -- 172.21.15.28:0/2850295340 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200636840 0 0) 0x7f258406c870 con 0x7f258415c640 2023-11-28T11:11:04.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.395+0000 7f2583fff700 1 -- 172.21.15.28:0/2850295340 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25840cd4e0 con 0x7f25840d2820 2023-11-28T11:11:04.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.396+0000 7f2583fff700 1 --2- 172.21.15.28:0/2850295340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25840d2820 0x7f25840b5c40 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2570009840 tx=0x7f2570009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=366b8584c0686291 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.396+0000 7f25837fe700 1 -- 172.21.15.28:0/2850295340 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570011830 con 0x7f25840d2820 2023-11-28T11:11:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.396+0000 7f25837fe700 1 -- 172.21.15.28:0/2850295340 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2570011990 con 0x7f25840d2820 2023-11-28T11:11:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.396+0000 7f25837fe700 1 -- 172.21.15.28:0/2850295340 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570011d20 con 0x7f25840d2820 2023-11-28T11:11:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.396+0000 7f258aaac700 1 -- 172.21.15.28:0/2850295340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25840d2820 msgr2=0x7f25840b5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.396+0000 7f258aaac700 1 --2- 172.21.15.28:0/2850295340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25840d2820 0x7f25840b5c40 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f2570009840 tx=0x7f2570009b50 comp rx=0 tx=0).stop 2023-11-28T11:11:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.396+0000 7f258aaac700 1 -- 172.21.15.28:0/2850295340 shutdown_connections 2023-11-28T11:11:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.396+0000 7f258aaac700 1 --2- 172.21.15.28:0/2850295340 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f258406bf30 0x7f258406c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.396+0000 7f258aaac700 1 --2- 172.21.15.28:0/2850295340 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25840d2820 0x7f25840b5c40 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.396+0000 7f258aaac700 1 -- 172.21.15.28:0/2850295340 >> 172.21.15.28:0/2850295340 conn(0x7f25840d2df0 msgr2=0x7f25840d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.396+0000 7f258aaac700 1 -- 172.21.15.28:0/2850295340 shutdown_connections 2023-11-28T11:11:04.404 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.396+0000 7f258aaac700 1 -- 172.21.15.28:0/2850295340 wait complete. 2023-11-28T11:11:04.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.397+0000 7f258aaac700 1 Processor -- start 2023-11-28T11:11:04.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.397+0000 7f258aaac700 1 -- start start 2023-11-28T11:11:04.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.398+0000 7f258aaac700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f258406bf30 0x7f258414b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.398+0000 7f258aaac700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25840d2820 0x7f25840c8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.398+0000 7f258aaac700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f258415c640 0x7f25840c8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.398+0000 7f258aaac700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25840cc270 con 0x7f25840d2820 2023-11-28T11:11:04.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.398+0000 7f258aaac700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258415cf00 con 0x7f258415c640 2023-11-28T11:11:04.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.398+0000 7f258aaac700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25840c9530 con 0x7f258406bf30 2023-11-28T11:11:04.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.398+0000 7f2583fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25840d2820 0x7f25840c8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.398+0000 7f2583fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25840d2820 0x7f25840c8950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50528/0 (socket says 172.21.15.28:50528) 2023-11-28T11:11:04.405 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.398+0000 7f2583fff700 1 -- 172.21.15.28:0/2524686060 learned_addr learned my addr 172.21.15.28:0/2524686060 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:04.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.399+0000 7f2588848700 1 --2- 172.21.15.28:0/2524686060 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f258406bf30 0x7f258414b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.399+0000 7f2589049700 1 --2- 172.21.15.28:0/2524686060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f258415c640 0x7f25840c8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.399+0000 7f2583fff700 1 -- 172.21.15.28:0/2524686060 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f258406bf30 msgr2=0x7f258414b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.399+0000 7f2583fff700 1 --2- 172.21.15.28:0/2524686060 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f258406bf30 0x7f258414b050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.399+0000 7f2583fff700 1 -- 172.21.15.28:0/2524686060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f258415c640 msgr2=0x7f25840c8e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.399+0000 7f2583fff700 1 --2- 172.21.15.28:0/2524686060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f258415c640 0x7f25840c8e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.399+0000 7f2583fff700 1 -- 172.21.15.28:0/2524686060 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25700094a0 con 0x7f25840d2820 2023-11-28T11:11:04.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.399+0000 7f2588848700 1 --2- 172.21.15.28:0/2524686060 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f258406bf30 0x7f258414b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:04.406 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.399+0000 7f2583fff700 1 --2- 172.21.15.28:0/2524686060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25840d2820 0x7f25840c8950 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f25700090b0 tx=0x7f2570004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.399+0000 7f25817fa700 1 -- 172.21.15.28:0/2524686060 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25700045f0 con 0x7f25840d2820 2023-11-28T11:11:04.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.399+0000 7f25817fa700 1 -- 172.21.15.28:0/2524686060 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257001f070 con 0x7f25840d2820 2023-11-28T11:11:04.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.399+0000 7f258aaac700 1 -- 172.21.15.28:0/2524686060 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25840d1710 con 0x7f25840d2820 2023-11-28T11:11:04.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.399+0000 7f25817fa700 1 -- 172.21.15.28:0/2524686060 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2570003b00 con 0x7f25840d2820 2023-11-28T11:11:04.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.399+0000 7f258aaac700 1 -- 172.21.15.28:0/2524686060 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25840cd7c0 con 0x7f25840d2820 2023-11-28T11:11:04.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.400+0000 7f258aaac700 1 -- 172.21.15.28:0/2524686060 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25840cdaa0 con 0x7f25840d2820 2023-11-28T11:11:04.407 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.401+0000 7f25817fa700 1 -- 172.21.15.28:0/2524686060 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f257000f050 con 0x7f25840d2820 2023-11-28T11:11:04.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.401+0000 7f25817fa700 1 --2- 172.21.15.28:0/2524686060 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2568068320 0x7f256806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.401+0000 7f25817fa700 1 -- 172.21.15.28:0/2524686060 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2570021070 con 0x7f25840d2820 2023-11-28T11:11:04.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.401+0000 7f2588848700 1 --2- 172.21.15.28:0/2524686060 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2568068320 0x7f256806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.401+0000 7f2588848700 1 --2- 172.21.15.28:0/2524686060 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2568068320 0x7f256806a7d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2578000f60 tx=0x7f2578007d30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.408 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.405+0000 7f25817fa700 1 -- 172.21.15.28:0/2524686060 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2570053940 con 0x7f25840d2820 2023-11-28T11:11:04.597 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.596+0000 7f258aaac700 1 -- 172.21.15.28:0/2524686060 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f25840ce3e0 con 0x7f2568068320 2023-11-28T11:11:04.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.604+0000 7f25817fa700 1 -- 172.21.15.28:0/2524686060 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296768 (secure 0 0 0) 0x7f25840ce3e0 con 0x7f2568068320 2023-11-28T11:11:04.609 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:04.611 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.607+0000 7f258aaac700 1 -- 172.21.15.28:0/2524686060 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2568068320 msgr2=0x7f256806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.607+0000 7f258aaac700 1 --2- 172.21.15.28:0/2524686060 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2568068320 0x7f256806a7d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2578000f60 tx=0x7f2578007d30 comp rx=0 tx=0).stop 2023-11-28T11:11:04.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.607+0000 7f258aaac700 1 -- 172.21.15.28:0/2524686060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25840d2820 msgr2=0x7f25840c8950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.607+0000 7f258aaac700 1 --2- 172.21.15.28:0/2524686060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25840d2820 0x7f25840c8950 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f25700090b0 tx=0x7f2570004750 comp rx=0 tx=0).stop 2023-11-28T11:11:04.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.607+0000 7f258aaac700 1 -- 172.21.15.28:0/2524686060 shutdown_connections 2023-11-28T11:11:04.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.607+0000 7f258aaac700 1 --2- 172.21.15.28:0/2524686060 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2568068320 0x7f256806a7d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.607+0000 7f258aaac700 1 --2- 172.21.15.28:0/2524686060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f258415c640 0x7f25840c8e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.607+0000 7f258aaac700 1 --2- 172.21.15.28:0/2524686060 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f258406bf30 0x7f258414b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.607+0000 7f258aaac700 1 --2- 172.21.15.28:0/2524686060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25840d2820 0x7f25840c8950 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.612 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.607+0000 7f258aaac700 1 -- 172.21.15.28:0/2524686060 >> 172.21.15.28:0/2524686060 conn(0x7f25840d2df0 msgr2=0x7f25840d3720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:04.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.607+0000 7f258aaac700 1 -- 172.21.15.28:0/2524686060 shutdown_connections 2023-11-28T11:11:04.613 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.607+0000 7f258aaac700 1 -- 172.21.15.28:0/2524686060 wait complete. 2023-11-28T11:11:04.613 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:11:04.621 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2023-11-28T11:11:03.192682+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":74276,"kb_used_data":5796,"kb_used_omap":0,"kb_used_meta":68416,"kb_avail":754900444,"statfs":{"total":773094113280,"available":773018054656,"internally_reserved":0,"allocated":5935104,"data_stored":3345263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70057984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.851208"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:56.391440+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:10:56.391440+0000","last_peered":"2023-11-28T11:10:56.391440+0000","last_clean":"2023-11-28T11:10:56.391440+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:10:56.391440+0000","last_undegraded":"2023-11-28T11:10:56.391440+0000","last_fullsized":"2023-11-28T11:10:56.391440+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:10:54.469632+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:10:54.469632+0000","last_peered":"2023-11-28T11:10:54.469632+0000","last_clean":"2023-11-28T11:10:54.469632+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:10:54.469632+0000","last_undegraded":"2023-11-28T11:10:54.469632+0000","last_fullsized":"2023-11-28T11:10:54.469632+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:54.475122+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:10:54.475122+0000","last_peered":"2023-11-28T11:10:54.475122+0000","last_clean":"2023-11-28T11:10:54.475122+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:10:54.475122+0000","last_undegraded":"2023-11-28T11:10:54.475122+0000","last_fullsized":"2023-11-28T11:10:54.475122+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":189,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:56.391566+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:10:56.391566+0000","last_peered":"2023-11-28T11:10:56.391566+0000","last_clean":"2023-11-28T11:10:56.391566+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:10:56.391566+0000","last_undegraded":"2023-11-28T11:10:56.391566+0000","last_fullsized":"2023-11-28T11:10:56.391566+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:56.391607+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:10:56.391607+0000","last_peered":"2023-11-28T11:10:56.391607+0000","last_clean":"2023-11-28T11:10:56.391607+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:10:56.391607+0000","last_undegraded":"2023-11-28T11:10:56.391607+0000","last_fullsized":"2023-11-28T11:10:56.391607+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:56.391482+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:10:56.391482+0000","last_peered":"2023-11-28T11:10:56.391482+0000","last_clean":"2023-11-28T11:10:56.391482+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:10:56.391482+0000","last_undegraded":"2023-11-28T11:10:56.391482+0000","last_fullsized":"2023-11-28T11:10:56.391482+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:54.474339+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:10:54.474339+0000","last_peered":"2023-11-28T11:10:54.474339+0000","last_clean":"2023-11-28T11:10:54.474339+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:10:54.474339+0000","last_undegraded":"2023-11-28T11:10:54.474339+0000","last_fullsized":"2023-11-28T11:10:54.474339+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:54.691507+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:10:54.691507+0000","last_peered":"2023-11-28T11:10:54.691507+0000","last_clean":"2023-11-28T11:10:54.691507+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:10:54.691507+0000","last_undegraded":"2023-11-28T11:10:54.691507+0000","last_fullsized":"2023-11-28T11:10:54.691507+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:54.691486+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:10:54.691486+0000","last_peered":"2023-11-28T11:10:54.691486+0000","last_clean":"2023-11-28T11:10:54.691486+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:10:54.691486+0000","last_undegraded":"2023-11-28T11:10:54.691486+0000","last_fullsized":"2023-11-28T11:10:54.691486+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10064,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94361776,"statfs":{"total":96636764160,"available":96626458624,"internally_reserved":0,"allocated":1187840,"data_stored":861054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.621,"15min":0.64700000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60999999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.373},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.626,"15min":0.64400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69499999999999995,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.40000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.67100000000000004,"15min":0.68999999999999995},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.879,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.38,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71999999999999997,"5min":1.224,"15min":1.224},"last":0.52900000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.67100000000000004,"15min":0.69299999999999995},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72399999999999998,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.76000000000000001,"15min":0.78400000000000003},"min":{"1min":0.46400000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.80700000000000005,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.58699999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.68100000000000005,"15min":0.70199999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.75600000000000001,"15min":0.77600000000000002},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85699999999999998,"5min":3.294,"15min":3.294},"last":0.65600000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77600000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.36199999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.76100000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.629}]},{"osd":5,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.96299999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.41099999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67600000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86899999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.41199999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86499999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69699999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8972,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362868,"statfs":{"total":96636764160,"available":96627576832,"internally_reserved":0,"allocated":593920,"data_stored":270686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.46000000000000002,"15min":0.45400000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.72099999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.624,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.56599999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51200000000000001,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77100000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.503}]},{"osd":2,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34100000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.753,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.67100000000000004,"15min":0.68500000000000005},"min":{"1min":0.42999999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84099999999999997,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.45700000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77900000000000003,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90100000000000002,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.75700000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54500000000000004,"15min":0.54600000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10064,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94361776,"statfs":{"total":96636764160,"available":96626458624,"internally_reserved":0,"allocated":1187840,"data_stored":861054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.32800000000000001,"15min":0.32700000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.35599999999999998,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54400000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.32000000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38100000000000001,"15min":0.38},"min":{"1min":0.29099999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.58999999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.28999999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.39400000000000002,"15min":0.38800000000000001},"min":{"1min":0.28100000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40799999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69999999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.27900000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53000000000000003,"15min":0.52700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88200000000000001,"5min":1.014,"15min":1.014},"last":0.51900000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.39900000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.79000000000000004,"5min":0.998,"15min":0.998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.76000000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.442,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79300000000000004,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81000000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.627}]},{"osd":7,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85199999999999998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.59199999999999997},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.82399999999999995,"5min":1.071,"15min":1.071},"last":0.55900000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9100,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362740,"statfs":{"total":96636764160,"available":96627445760,"internally_reserved":0,"allocated":593920,"data_stored":270686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.316},{"interface":"front","average":{"1min":0.373,"5min":0.37,"15min":0.37},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38,"15min":0.38300000000000001},"min":{"1min":0.26800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.372},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39700000000000002,"15min":0.39900000000000002},"min":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57999999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.26700000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36799999999999999,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.48999999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.58999999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39200000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81599999999999995,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86299999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.65100000000000002,"15min":0.62},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":5.4269999999999996,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.62,"15min":0.622},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.79900000000000004,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.51100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89500000000000002,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.44800000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.88200000000000001,"5min":1.016,"15min":1.016},"last":0.65400000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85799999999999998,"5min":1.105,"15min":1.105},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.505,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66700000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8904,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94362936,"statfs":{"total":96636764160,"available":96627646464,"internally_reserved":0,"allocated":589824,"data_stored":269725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.36699999999999999,"15min":0.374},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.41699999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38100000000000001,"15min":0.38700000000000001},"min":{"1min":0.248,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51900000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.34100000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.39000000000000001,"15min":0.39400000000000002},"min":{"1min":0.26100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51000000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60899999999999999,"5min":1.528,"15min":1.528},"last":0.38300000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.39000000000000001,"15min":0.39600000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69599999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.46899999999999997,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.59699999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.36699999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79000000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.54600000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75600000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.622},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.85699999999999998,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53100000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.66400000000000003,"15min":0.67100000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.60799999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64300000000000002,"15min":0.64800000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.623,"15min":0.628},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.56699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9036,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362804,"statfs":{"total":96636764160,"available":96627511296,"internally_reserved":0,"allocated":593920,"data_stored":270686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.83399999999999996,"15min":0.92000000000000004},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.46300000000000002,"5min":11.965,"15min":11.965},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.87,"15min":0.95799999999999996},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55000000000000004,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.38100000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.33900000000000002,"15min":0.33700000000000002},"min":{"1min":0.23999999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":1.0940000000000001,"15min":1.2150000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56799999999999995,"5min":16.988,"15min":16.988},"last":0.40300000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.49399999999999999,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75600000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":1.6020000000000001,"15min":1.7969999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64100000000000001,"5min":27.887,"15min":27.887},"last":0.45700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.68799999999999994,"15min":0.70899999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.75900000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.65800000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.748,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.51100000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.67900000000000005,"15min":0.69199999999999995},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.745,"5min":1.98,"15min":1.98},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":1.1220000000000001,"15min":1.21},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78100000000000003,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.61399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.1279999999999999,"15min":1.2170000000000001},"min":{"1min":0.44,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73799999999999999,"5min":11.888,"15min":11.888},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.71799999999999997,"15min":0.73699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.76000000000000001,"5min":1.925,"15min":1.925},"last":0.626}]},{"osd":7,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77300000000000002,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.498,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.79700000000000004,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.70099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9164,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362676,"statfs":{"total":96636764160,"available":96627380224,"internally_reserved":0,"allocated":593920,"data_stored":270686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.748,"5min":0.996,"15min":0.996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.90300000000000002,"15min":0.96499999999999997},"min":{"1min":0.41099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.70099999999999996,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.34999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72499999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70499999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.66500000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67900000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.375,"5min":0.374,"15min":0.374},"max":{"1min":0.749,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61299999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.72899999999999998,"15min":0.75900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70399999999999996,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.47699999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71599999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.71099999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33100000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71999999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.35799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79300000000000004,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.73099999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59999999999999998,"15min":0.60799999999999998},"min":{"1min":0.35899999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76400000000000001,"5min":1.145,"15min":1.145},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.374,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":1.085,"15min":1.085},"last":0.747}]},{"osd":7,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.40600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.80300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8972,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362868,"statfs":{"total":96636764160,"available":96627576832,"internally_reserved":0,"allocated":593920,"data_stored":270686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.36499999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71199999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.443},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73099999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68000000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79300000000000004,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.66100000000000003,"15min":0.67600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.71199999999999997,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.56000000000000005}]},{"osd":2,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78300000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73699999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.496}]},{"osd":3,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.67200000000000004,"15min":0.69099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80400000000000005,"5min":1.042,"15min":1.042},"last":0.64900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.629,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.75900000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.71799999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73399999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72699999999999998,"5min":1.036,"15min":1.036},"last":0.60399999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.68200000000000005,"15min":0.69599999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78300000000000003,"5min":1.329,"15min":1.329},"last":0.749},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75800000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.69299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:11:04.626 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:11:04.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.785+0000 7f78a1c15700 1 Processor -- start 2023-11-28T11:11:04.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.785+0000 7f78a1c15700 1 -- start start 2023-11-28T11:11:04.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.785+0000 7f78a1c15700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f789c161090 0x7f789c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.785+0000 7f78a1c15700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f789c0caeb0 0x7f789c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.785+0000 7f78a1c15700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f789c0ce340 con 0x7f789c15c990 2023-11-28T11:11:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.785+0000 7f78a1c15700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f789c0cebe0 con 0x7f789c161090 2023-11-28T11:11:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.785+0000 7f78a1c15700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789c0cf780 con 0x7f789c0caeb0 2023-11-28T11:11:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.786+0000 7f789affd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f789c161090 0x7f789c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.786+0000 7f789affd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f789c161090 0x7f789c0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50536/0 (socket says 172.21.15.28:50536) 2023-11-28T11:11:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.786+0000 7f789affd700 1 -- 172.21.15.28:0/847057347 learned_addr learned my addr 172.21.15.28:0/847057347 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.786+0000 7f789affd700 1 -- 172.21.15.28:0/847057347 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f789c0caeb0 msgr2=0x7f789c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.786+0000 7f789affd700 1 --2- 172.21.15.28:0/847057347 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f789c0caeb0 0x7f789c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.786+0000 7f789affd700 1 -- 172.21.15.28:0/847057347 >> v1:172.21.15.59:6789/0 conn(0x7f789c15c990 legacy=0x7f789c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.786+0000 7f789affd700 1 -- 172.21.15.28:0/847057347 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f789c0cb3f0 con 0x7f789c161090 2023-11-28T11:11:04.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.786+0000 7f789affd700 1 --2- 172.21.15.28:0/847057347 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f789c161090 0x7f789c0ca970 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f78900098f0 tx=0x7f7890009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4b2aafd0942c90 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.786+0000 7f789a7fc700 1 -- 172.21.15.28:0/847057347 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890004230 con 0x7f789c161090 2023-11-28T11:11:04.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.786+0000 7f789a7fc700 1 -- 172.21.15.28:0/847057347 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7890004390 con 0x7f789c161090 2023-11-28T11:11:04.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.786+0000 7f789a7fc700 1 -- 172.21.15.28:0/847057347 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78900036a0 con 0x7f789c161090 2023-11-28T11:11:04.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.787+0000 7f78a1c15700 1 -- 172.21.15.28:0/847057347 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f789c161090 msgr2=0x7f789c0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.787+0000 7f78a1c15700 1 --2- 172.21.15.28:0/847057347 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f789c161090 0x7f789c0ca970 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f78900098f0 tx=0x7f7890009c00 comp rx=0 tx=0).stop 2023-11-28T11:11:04.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.787+0000 7f78a1c15700 1 -- 172.21.15.28:0/847057347 shutdown_connections 2023-11-28T11:11:04.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.787+0000 7f78a1c15700 1 --2- 172.21.15.28:0/847057347 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f789c0caeb0 0x7f789c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.787+0000 7f78a1c15700 1 --2- 172.21.15.28:0/847057347 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f789c161090 0x7f789c0ca970 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.787+0000 7f78a1c15700 1 -- 172.21.15.28:0/847057347 >> 172.21.15.28:0/847057347 conn(0x7f789c0b1650 msgr2=0x7f789c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:04.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.787+0000 7f78a1c15700 1 -- 172.21.15.28:0/847057347 shutdown_connections 2023-11-28T11:11:04.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.787+0000 7f78a1c15700 1 -- 172.21.15.28:0/847057347 wait complete. 2023-11-28T11:11:04.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.788+0000 7f78a1c15700 1 Processor -- start 2023-11-28T11:11:04.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.788+0000 7f78a1c15700 1 -- start start 2023-11-28T11:11:04.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.789+0000 7f78a1c15700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f789c0caeb0 0x7f789c1f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.789+0000 7f78a1c15700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f789c15c990 0x7f789c0adb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.789+0000 7f78a1c15700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f789c161090 0x7f789c1f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.789+0000 7f78a1c15700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c0b32a0 con 0x7f789c161090 2023-11-28T11:11:04.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.789+0000 7f78a1c15700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f789c0ae8d0 con 0x7f789c15c990 2023-11-28T11:11:04.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.789+0000 7f78a1c15700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f789c0ae5f0 con 0x7f789c0caeb0 2023-11-28T11:11:04.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.789+0000 7f789bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f789c161090 0x7f789c1f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.789+0000 7f789bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f789c161090 0x7f789c1f5ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50544/0 (socket says 172.21.15.28:50544) 2023-11-28T11:11:04.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.789+0000 7f789bfff700 1 -- 172.21.15.28:0/3811216274 learned_addr learned my addr 172.21.15.28:0/3811216274 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:04.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.789+0000 7f789affd700 1 --2- 172.21.15.28:0/3811216274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f789c15c990 0x7f789c0adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.789+0000 7f789b7fe700 1 --2- 172.21.15.28:0/3811216274 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f789c0caeb0 0x7f789c1f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.790+0000 7f789bfff700 1 -- 172.21.15.28:0/3811216274 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f789c0caeb0 msgr2=0x7f789c1f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.790+0000 7f789bfff700 1 --2- 172.21.15.28:0/3811216274 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f789c0caeb0 0x7f789c1f5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.790+0000 7f789bfff700 1 -- 172.21.15.28:0/3811216274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f789c15c990 msgr2=0x7f789c0adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:04.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.790+0000 7f789bfff700 1 --2- 172.21.15.28:0/3811216274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f789c15c990 0x7f789c0adb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:04.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.790+0000 7f789bfff700 1 -- 172.21.15.28:0/3811216274 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7890009550 con 0x7f789c161090 2023-11-28T11:11:04.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.790+0000 7f789bfff700 1 --2- 172.21.15.28:0/3811216274 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f789c161090 0x7f789c1f5ac0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f789400b970 tx=0x7f789400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.790+0000 7f7883fff700 1 -- 172.21.15.28:0/3811216274 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789400d860 con 0x7f789c161090 2023-11-28T11:11:04.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.790+0000 7f7883fff700 1 -- 172.21.15.28:0/3811216274 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789400d9e0 con 0x7f789c161090 2023-11-28T11:11:04.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.790+0000 7f78a1c15700 1 -- 172.21.15.28:0/3811216274 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f789c0ce870 con 0x7f789c161090 2023-11-28T11:11:04.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.790+0000 7f7883fff700 1 -- 172.21.15.28:0/3811216274 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789400dd90 con 0x7f789c161090 2023-11-28T11:11:04.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.790+0000 7f78a1c15700 1 -- 172.21.15.28:0/3811216274 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f789c0ce590 con 0x7f789c161090 2023-11-28T11:11:04.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.791+0000 7f78a1c15700 1 -- 172.21.15.28:0/3811216274 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f789c0cc850 con 0x7f789c161090 2023-11-28T11:11:04.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.791+0000 7f7883fff700 1 -- 172.21.15.28:0/3811216274 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f789400e430 con 0x7f789c161090 2023-11-28T11:11:04.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.792+0000 7f7883fff700 1 --2- 172.21.15.28:0/3811216274 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f78880682f0 0x7f788806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:04.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.792+0000 7f7883fff700 1 -- 172.21.15.28:0/3811216274 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7894006e40 con 0x7f789c161090 2023-11-28T11:11:04.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.792+0000 7f789b7fe700 1 --2- 172.21.15.28:0/3811216274 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f78880682f0 0x7f788806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:04.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.792+0000 7f789b7fe700 1 --2- 172.21.15.28:0/3811216274 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f78880682f0 0x7f788806a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f789c0b48a0 tx=0x7f788c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:04.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.796+0000 7f7883fff700 1 -- 172.21.15.28:0/3811216274 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f78940527b0 con 0x7f789c161090 2023-11-28T11:11:04.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.986+0000 7f78a1c15700 1 -- 172.21.15.28:0/3811216274 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f789c066700 con 0x7f78880682f0 2023-11-28T11:11:04.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.994+0000 7f7883fff700 1 -- 172.21.15.28:0/3811216274 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+296768 (secure 0 0 0) 0x7f789c066700 con 0x7f78880682f0 2023-11-28T11:11:05.000 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:05.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.997+0000 7f78a1c15700 1 -- 172.21.15.28:0/3811216274 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f78880682f0 msgr2=0x7f788806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.997+0000 7f78a1c15700 1 --2- 172.21.15.28:0/3811216274 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f78880682f0 0x7f788806a7a0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f789c0b48a0 tx=0x7f788c009040 comp rx=0 tx=0).stop 2023-11-28T11:11:05.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.997+0000 7f78a1c15700 1 -- 172.21.15.28:0/3811216274 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f789c161090 msgr2=0x7f789c1f5ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.997+0000 7f78a1c15700 1 --2- 172.21.15.28:0/3811216274 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f789c161090 0x7f789c1f5ac0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f789400b970 tx=0x7f789400bc80 comp rx=0 tx=0).stop 2023-11-28T11:11:05.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.997+0000 7f78a1c15700 1 -- 172.21.15.28:0/3811216274 shutdown_connections 2023-11-28T11:11:05.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.997+0000 7f78a1c15700 1 --2- 172.21.15.28:0/3811216274 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f78880682f0 0x7f788806a7a0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.997+0000 7f78a1c15700 1 --2- 172.21.15.28:0/3811216274 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f789c161090 0x7f789c1f5ac0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.997+0000 7f78a1c15700 1 --2- 172.21.15.28:0/3811216274 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f789c0caeb0 0x7f789c1f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.997+0000 7f78a1c15700 1 --2- 172.21.15.28:0/3811216274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f789c15c990 0x7f789c0adb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.998+0000 7f78a1c15700 1 -- 172.21.15.28:0/3811216274 >> 172.21.15.28:0/3811216274 conn(0x7f789c0b1650 msgr2=0x7f789c0af060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.998+0000 7f78a1c15700 1 -- 172.21.15.28:0/3811216274 shutdown_connections 2023-11-28T11:11:05.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:04.998+0000 7f78a1c15700 1 -- 172.21.15.28:0/3811216274 wait complete. 2023-11-28T11:11:05.004 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:11:05.011 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":120,"stamp":"2023-11-28T11:11:03.192682+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":74276,"kb_used_data":5796,"kb_used_omap":0,"kb_used_meta":68416,"kb_avail":754900444,"statfs":{"total":773094113280,"available":773018054656,"internally_reserved":0,"allocated":5935104,"data_stored":3345263,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":70057984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.851208"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":38,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:56.391440+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:10:56.391440+0000","last_peered":"2023-11-28T11:10:56.391440+0000","last_clean":"2023-11-28T11:10:56.391440+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:10:56.391440+0000","last_undegraded":"2023-11-28T11:10:56.391440+0000","last_fullsized":"2023-11-28T11:10:56.391440+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":38,"reported_epoch":48,"state":"active+clean","last_fresh":"2023-11-28T11:10:54.469632+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:10:54.469632+0000","last_peered":"2023-11-28T11:10:54.469632+0000","last_clean":"2023-11-28T11:10:54.469632+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:10:54.469632+0000","last_undegraded":"2023-11-28T11:10:54.469632+0000","last_fullsized":"2023-11-28T11:10:54.469632+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:54.475122+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:10:54.475122+0000","last_peered":"2023-11-28T11:10:54.475122+0000","last_clean":"2023-11-28T11:10:54.475122+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:10:54.475122+0000","last_undegraded":"2023-11-28T11:10:54.475122+0000","last_fullsized":"2023-11-28T11:10:54.475122+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":189,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:56.391566+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:10:56.391566+0000","last_peered":"2023-11-28T11:10:56.391566+0000","last_clean":"2023-11-28T11:10:56.391566+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:10:56.391566+0000","last_undegraded":"2023-11-28T11:10:56.391566+0000","last_fullsized":"2023-11-28T11:10:56.391566+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:56.391607+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:10:56.391607+0000","last_peered":"2023-11-28T11:10:56.391607+0000","last_clean":"2023-11-28T11:10:56.391607+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:10:56.391607+0000","last_undegraded":"2023-11-28T11:10:56.391607+0000","last_fullsized":"2023-11-28T11:10:56.391607+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":39,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:56.391482+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:10:56.391482+0000","last_peered":"2023-11-28T11:10:56.391482+0000","last_clean":"2023-11-28T11:10:56.391482+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:10:56.391482+0000","last_undegraded":"2023-11-28T11:10:56.391482+0000","last_fullsized":"2023-11-28T11:10:56.391482+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":42,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:54.474339+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:10:54.474339+0000","last_peered":"2023-11-28T11:10:54.474339+0000","last_clean":"2023-11-28T11:10:54.474339+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:10:54.474339+0000","last_undegraded":"2023-11-28T11:10:54.474339+0000","last_fullsized":"2023-11-28T11:10:54.474339+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:54.691507+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:10:54.691507+0000","last_peered":"2023-11-28T11:10:54.691507+0000","last_clean":"2023-11-28T11:10:54.691507+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:10:54.691507+0000","last_undegraded":"2023-11-28T11:10:54.691507+0000","last_fullsized":"2023-11-28T11:10:54.691507+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:59.341509+0000","last_change":"2023-11-28T11:10:59.341509+0000","last_active":"2023-11-28T11:10:59.341509+0000","last_peered":"2023-11-28T11:10:59.341509+0000","last_clean":"2023-11-28T11:10:59.341509+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:59.341509+0000","last_undegraded":"2023-11-28T11:10:59.341509+0000","last_fullsized":"2023-11-28T11:10:59.341509+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:10:58.336036+0000","last_change":"2023-11-28T11:10:58.336036+0000","last_active":"2023-11-28T11:10:58.336036+0000","last_peered":"2023-11-28T11:10:58.336036+0000","last_clean":"2023-11-28T11:10:58.336036+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:10:58.336036+0000","last_undegraded":"2023-11-28T11:10:58.336036+0000","last_fullsized":"2023-11-28T11:10:58.336036+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":37,"reported_epoch":49,"state":"active+clean","last_fresh":"2023-11-28T11:10:54.691486+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:10:54.691486+0000","last_peered":"2023-11-28T11:10:54.691486+0000","last_clean":"2023-11-28T11:10:54.691486+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:10:54.691486+0000","last_undegraded":"2023-11-28T11:10:54.691486+0000","last_fullsized":"2023-11-28T11:10:54.691486+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574880,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10064,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94361776,"statfs":{"total":96636764160,"available":96626458624,"internally_reserved":0,"allocated":1187840,"data_stored":861054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.621,"15min":0.64700000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60999999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.373},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.626,"15min":0.64400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69499999999999995,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.40000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.67100000000000004,"15min":0.68999999999999995},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.879,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.38,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71999999999999997,"5min":1.224,"15min":1.224},"last":0.52900000000000003}]},{"osd":2,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.67100000000000004,"15min":0.69299999999999995},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72399999999999998,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.76000000000000001,"15min":0.78400000000000003},"min":{"1min":0.46400000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.80700000000000005,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.58699999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.68100000000000005,"15min":0.70199999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.75600000000000001,"15min":0.77600000000000002},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85699999999999998,"5min":3.294,"15min":3.294},"last":0.65600000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77600000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.36199999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.76100000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.629}]},{"osd":5,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.96299999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.41099999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67600000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86899999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.41199999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86499999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.69699999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8972,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362868,"statfs":{"total":96636764160,"available":96627576832,"internally_reserved":0,"allocated":593920,"data_stored":270686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.46000000000000002,"15min":0.45400000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.72099999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.624,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.56599999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51200000000000001,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77100000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.503}]},{"osd":2,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34100000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.753,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.67100000000000004,"15min":0.68500000000000005},"min":{"1min":0.42999999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84099999999999997,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.45700000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77900000000000003,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90100000000000002,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.75700000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54500000000000004,"15min":0.54600000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.52900000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.59999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10064,"kb_used_data":1160,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94361776,"statfs":{"total":96636764160,"available":96626458624,"internally_reserved":0,"allocated":1187840,"data_stored":861054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.32800000000000001,"15min":0.32700000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.20599999999999999},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.35599999999999998,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54400000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.32000000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38100000000000001,"15min":0.38},"min":{"1min":0.29099999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.58999999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.28999999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.39400000000000002,"15min":0.38800000000000001},"min":{"1min":0.28100000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40799999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69999999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.27900000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53000000000000003,"15min":0.52700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88200000000000001,"5min":1.014,"15min":1.014},"last":0.51900000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.39900000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.79000000000000004,"5min":0.998,"15min":0.998},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.76000000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.442,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79300000000000004,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81000000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.627}]},{"osd":7,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85199999999999998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.59199999999999997},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.82399999999999995,"5min":1.071,"15min":1.071},"last":0.55900000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9100,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362740,"statfs":{"total":96636764160,"available":96627445760,"internally_reserved":0,"allocated":593920,"data_stored":270686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.316},{"interface":"front","average":{"1min":0.373,"5min":0.37,"15min":0.37},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38,"15min":0.38300000000000001},"min":{"1min":0.26800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.372},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39700000000000002,"15min":0.39900000000000002},"min":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57999999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.26700000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36799999999999999,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.48999999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.58999999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39200000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81599999999999995,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86299999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.65100000000000002,"15min":0.62},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":5.4269999999999996,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.62,"15min":0.622},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.79900000000000004,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.51100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89500000000000002,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.44800000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.88200000000000001,"5min":1.016,"15min":1.016},"last":0.65400000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85799999999999998,"5min":1.105,"15min":1.105},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.505,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66700000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8904,"kb_used_data":576,"kb_used_omap":0,"kb_used_meta":8320,"kb_avail":94362936,"statfs":{"total":96636764160,"available":96627646464,"internally_reserved":0,"allocated":589824,"data_stored":269725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8519680},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.36699999999999999,"15min":0.374},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.41699999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38100000000000001,"15min":0.38700000000000001},"min":{"1min":0.248,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51900000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.34100000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.39000000000000001,"15min":0.39400000000000002},"min":{"1min":0.26100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51000000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60899999999999999,"5min":1.528,"15min":1.528},"last":0.38300000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.39000000000000001,"15min":0.39600000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69599999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.46899999999999997,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.59699999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.36699999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79000000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.54600000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75600000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.622},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.85699999999999998,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53100000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.62,"5min":0.66400000000000003,"15min":0.67100000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.60799999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64300000000000002,"15min":0.64800000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.623,"15min":0.628},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.56699999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9036,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362804,"statfs":{"total":96636764160,"available":96627511296,"internally_reserved":0,"allocated":593920,"data_stored":270686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.83399999999999996,"15min":0.92000000000000004},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.46300000000000002,"5min":11.965,"15min":11.965},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.87,"15min":0.95799999999999996},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55000000000000004,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.38100000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.33900000000000002,"15min":0.33700000000000002},"min":{"1min":0.23999999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":1.0940000000000001,"15min":1.2150000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56799999999999995,"5min":16.988,"15min":16.988},"last":0.40300000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.49399999999999999,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75600000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":1.6020000000000001,"15min":1.7969999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64100000000000001,"5min":27.887,"15min":27.887},"last":0.45700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.68799999999999994,"15min":0.70899999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.75900000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.65800000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.748,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.51100000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.67900000000000005,"15min":0.69199999999999995},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.745,"5min":1.98,"15min":1.98},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":1.1220000000000001,"15min":1.21},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78100000000000003,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.61399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.1279999999999999,"15min":1.2170000000000001},"min":{"1min":0.44,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73799999999999999,"5min":11.888,"15min":11.888},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.71799999999999997,"15min":0.73699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.76000000000000001,"5min":1.925,"15min":1.925},"last":0.626}]},{"osd":7,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77300000000000002,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.498,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.79700000000000004,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.70099999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9164,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362676,"statfs":{"total":96636764160,"available":96627380224,"internally_reserved":0,"allocated":593920,"data_stored":270686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.748,"5min":0.996,"15min":0.996},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.90300000000000002,"15min":0.96499999999999997},"min":{"1min":0.41099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.70099999999999996,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.34999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72499999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70499999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.66500000000000004}]},{"osd":2,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67900000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.375,"5min":0.374,"15min":0.374},"max":{"1min":0.749,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.61299999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.72899999999999998,"15min":0.75900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70399999999999996,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.47699999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71599999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.71099999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33100000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71999999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.35799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79300000000000004,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.73099999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59999999999999998,"15min":0.60799999999999998},"min":{"1min":0.35899999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76400000000000001,"5min":1.145,"15min":1.145},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.374,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":1.085,"15min":1.085},"last":0.747}]},{"osd":7,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.40600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.80300000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":8972,"kb_used_data":580,"kb_used_omap":0,"kb_used_meta":8384,"kb_avail":94362868,"statfs":{"total":96636764160,"available":96627576832,"internally_reserved":0,"allocated":593920,"data_stored":270686,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8585216},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.36499999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71199999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.443},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73099999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68000000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79300000000000004,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.66100000000000003,"15min":0.67600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.71199999999999997,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.56000000000000005}]},{"osd":2,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78300000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73699999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.496}]},{"osd":3,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.67200000000000004,"15min":0.69099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80400000000000005,"5min":1.042,"15min":1.042},"last":0.64900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.629,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.75900000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.71799999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73399999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72699999999999998,"5min":1.036,"15min":1.036},"last":0.60399999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.68200000000000005,"15min":0.69599999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78300000000000003,"5min":1.329,"15min":1.329},"last":0.749},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75800000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.69299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:11:05.016 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3f is not active+clean 2023-11-28T11:11:05.016 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3e is not active+clean 2023-11-28T11:11:05.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3d is not active+clean 2023-11-28T11:11:05.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3c is not active+clean 2023-11-28T11:11:05.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3b is not active+clean 2023-11-28T11:11:05.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.017 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3a is not active+clean 2023-11-28T11:11:05.017 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.39 is not active+clean 2023-11-28T11:11:05.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.38 is not active+clean 2023-11-28T11:11:05.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.37 is not active+clean 2023-11-28T11:11:05.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.36 is not active+clean 2023-11-28T11:11:05.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.35 is not active+clean 2023-11-28T11:11:05.018 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.018 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.34 is not active+clean 2023-11-28T11:11:05.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.33 is not active+clean 2023-11-28T11:11:05.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.32 is not active+clean 2023-11-28T11:11:05.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.31 is not active+clean 2023-11-28T11:11:05.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.30 is not active+clean 2023-11-28T11:11:05.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.019 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2f is not active+clean 2023-11-28T11:11:05.019 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2e is not active+clean 2023-11-28T11:11:05.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2d is not active+clean 2023-11-28T11:11:05.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2c is not active+clean 2023-11-28T11:11:05.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2b is not active+clean 2023-11-28T11:11:05.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2a is not active+clean 2023-11-28T11:11:05.020 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.020 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.29 is not active+clean 2023-11-28T11:11:05.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.28 is not active+clean 2023-11-28T11:11:05.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.27 is not active+clean 2023-11-28T11:11:05.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.26 is not active+clean 2023-11-28T11:11:05.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.25 is not active+clean 2023-11-28T11:11:05.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.021 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.24 is not active+clean 2023-11-28T11:11:05.021 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.23 is not active+clean 2023-11-28T11:11:05.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.22 is not active+clean 2023-11-28T11:11:05.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.21 is not active+clean 2023-11-28T11:11:05.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.022 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.20 is not active+clean 2023-11-28T11:11:05.022 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1e is not active+clean 2023-11-28T11:11:05.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1f is not active+clean 2023-11-28T11:11:05.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1f is not active+clean 2023-11-28T11:11:05.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1e is not active+clean 2023-11-28T11:11:05.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1c is not active+clean 2023-11-28T11:11:05.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.023 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1d is not active+clean 2023-11-28T11:11:05.023 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1d is not active+clean 2023-11-28T11:11:05.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1c is not active+clean 2023-11-28T11:11:05.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1a is not active+clean 2023-11-28T11:11:05.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1b is not active+clean 2023-11-28T11:11:05.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1b is not active+clean 2023-11-28T11:11:05.024 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.024 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1a is not active+clean 2023-11-28T11:11:05.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.18 is not active+clean 2023-11-28T11:11:05.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.19 is not active+clean 2023-11-28T11:11:05.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.19 is not active+clean 2023-11-28T11:11:05.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.18 is not active+clean 2023-11-28T11:11:05.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.025 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.16 is not active+clean 2023-11-28T11:11:05.025 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.17 is not active+clean 2023-11-28T11:11:05.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.17 is not active+clean 2023-11-28T11:11:05.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.16 is not active+clean 2023-11-28T11:11:05.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.14 is not active+clean 2023-11-28T11:11:05.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.026 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.15 is not active+clean 2023-11-28T11:11:05.026 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.15 is not active+clean 2023-11-28T11:11:05.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.14 is not active+clean 2023-11-28T11:11:05.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.12 is not active+clean 2023-11-28T11:11:05.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.13 is not active+clean 2023-11-28T11:11:05.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.13 is not active+clean 2023-11-28T11:11:05.027 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.027 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.12 is not active+clean 2023-11-28T11:11:05.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.10 is not active+clean 2023-11-28T11:11:05.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.11 is not active+clean 2023-11-28T11:11:05.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.11 is not active+clean 2023-11-28T11:11:05.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.10 is not active+clean 2023-11-28T11:11:05.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.028 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.6 is not active+clean 2023-11-28T11:11:05.028 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.7 is not active+clean 2023-11-28T11:11:05.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.7 is not active+clean 2023-11-28T11:11:05.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.6 is not active+clean 2023-11-28T11:11:05.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.4 is not active+clean 2023-11-28T11:11:05.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.5 is not active+clean 2023-11-28T11:11:05.029 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.029 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.5 is not active+clean 2023-11-28T11:11:05.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.4 is not active+clean 2023-11-28T11:11:05.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.2 is not active+clean 2023-11-28T11:11:05.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.3 is not active+clean 2023-11-28T11:11:05.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.3 is not active+clean 2023-11-28T11:11:05.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.030 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.2 is not active+clean 2023-11-28T11:11:05.030 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.0 is not active+clean 2023-11-28T11:11:05.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.1 is not active+clean 2023-11-28T11:11:05.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.1 is not active+clean 2023-11-28T11:11:05.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.0 is not active+clean 2023-11-28T11:11:05.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.d is not active+clean 2023-11-28T11:11:05.031 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.031 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.c is not active+clean 2023-11-28T11:11:05.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.c is not active+clean 2023-11-28T11:11:05.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.d is not active+clean 2023-11-28T11:11:05.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.b is not active+clean 2023-11-28T11:11:05.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.a is not active+clean 2023-11-28T11:11:05.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.032 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.a is not active+clean 2023-11-28T11:11:05.032 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.b is not active+clean 2023-11-28T11:11:05.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.8 is not active+clean 2023-11-28T11:11:05.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.9 is not active+clean 2023-11-28T11:11:05.033 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.033 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.9 is not active+clean 2023-11-28T11:11:05.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.8 is not active+clean 2023-11-28T11:11:05.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.e is not active+clean 2023-11-28T11:11:05.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.f is not active+clean 2023-11-28T11:11:05.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 9.f is not active+clean 2023-11-28T11:11:05.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '9.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:59.341509+0000', 'last_change': '2023-11-28T11:10:59.341509+0000', 'last_active': '2023-11-28T11:10:59.341509+0000', 'last_peered': '2023-11-28T11:10:59.341509+0000', 'last_clean': '2023-11-28T11:10:59.341509+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:59.341509+0000', 'last_undegraded': '2023-11-28T11:10:59.341509+0000', 'last_fullsized': '2023-11-28T11:10:59.341509+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:59.341509+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.034 INFO:tasks.cephfs.filesystem.ceph_manager:PG 8.e is not active+clean 2023-11-28T11:11:05.034 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '8.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:10:58.336036+0000', 'last_change': '2023-11-28T11:10:58.336036+0000', 'last_active': '2023-11-28T11:10:58.336036+0000', 'last_peered': '2023-11-28T11:10:58.336036+0000', 'last_clean': '2023-11-28T11:10:58.336036+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:10:58.336036+0000', 'last_undegraded': '2023-11-28T11:10:58.336036+0000', 'last_fullsized': '2023-11-28T11:10:58.336036+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'last_clean_scrub_stamp': '2023-11-28T11:10:58.336036+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:11:05.035 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T11:11:05.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.175+0000 7f5ef5094700 1 Processor -- start 2023-11-28T11:11:05.177 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.175+0000 7f5ef5094700 1 -- start start 2023-11-28T11:11:05.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.176+0000 7f5ef5094700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5ef015c8a0 0x7f5ef00b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.179 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.176+0000 7f5ef5094700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5ef00b2290 0x7f5ef00ac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.176+0000 7f5ef5094700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ef00d0280 con 0x7f5ef006f120 2023-11-28T11:11:05.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.176+0000 7f5ef5094700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef00d0560 con 0x7f5ef015c8a0 2023-11-28T11:11:05.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.176+0000 7f5ef5094700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ef00d0840 con 0x7f5ef00b2290 2023-11-28T11:11:05.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.176+0000 7f5eeed9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5ef015c8a0 0x7f5ef00b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.176+0000 7f5eef59e700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5ef006f120 0x7f5ef00cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38610/0 (socket says 172.21.15.28:38610) 2023-11-28T11:11:05.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.176+0000 7f5eef59e700 1 -- 172.21.15.28:0/3608455836 learned_addr learned my addr 172.21.15.28:0/3608455836 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:05.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.176+0000 7f5eee59c700 1 --2- 172.21.15.28:0/3608455836 >> v2:172.21.15.59:3300/0 conn(0x7f5ef00b2290 0x7f5ef00ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.177+0000 7f5eeed9d700 1 -- 172.21.15.28:0/3608455836 >> v2:172.21.15.59:3300/0 conn(0x7f5ef00b2290 msgr2=0x7f5ef00ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.177+0000 7f5eeed9d700 1 --2- 172.21.15.28:0/3608455836 >> v2:172.21.15.59:3300/0 conn(0x7f5ef00b2290 0x7f5ef00ac660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.177+0000 7f5eeed9d700 1 -- 172.21.15.28:0/3608455836 >> v1:172.21.15.28:6789/0 conn(0x7f5ef006f120 legacy=0x7f5ef00cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.177+0000 7f5eeed9d700 1 -- 172.21.15.28:0/3608455836 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ef00b1550 con 0x7f5ef015c8a0 2023-11-28T11:11:05.180 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.177+0000 7f5eedd9b700 1 -- 172.21.15.28:0/3608455836 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3411743600 0 0) 0x7f5ef00d0280 con 0x7f5ef006f120 2023-11-28T11:11:05.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.177+0000 7f5eeed9d700 1 --2- 172.21.15.28:0/3608455836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ef015c8a0 0x7f5ef00b1d50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00726a0 tx=0x7f5ee0009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb98be3883814729 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.177+0000 7f5eedd9b700 1 -- 172.21.15.28:0/3608455836 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0011830 con 0x7f5ef015c8a0 2023-11-28T11:11:05.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.177+0000 7f5eedd9b700 1 -- 172.21.15.28:0/3608455836 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee0011990 con 0x7f5ef015c8a0 2023-11-28T11:11:05.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.177+0000 7f5eedd9b700 1 -- 172.21.15.28:0/3608455836 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0011d20 con 0x7f5ef015c8a0 2023-11-28T11:11:05.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.177+0000 7f5ef5094700 1 -- 172.21.15.28:0/3608455836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ef015c8a0 msgr2=0x7f5ef00b1d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.177+0000 7f5ef5094700 1 --2- 172.21.15.28:0/3608455836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ef015c8a0 0x7f5ef00b1d50 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f5ef00726a0 tx=0x7f5ee0009b20 comp rx=0 tx=0).stop 2023-11-28T11:11:05.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.177+0000 7f5ef5094700 1 -- 172.21.15.28:0/3608455836 shutdown_connections 2023-11-28T11:11:05.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.177+0000 7f5ef5094700 1 --2- 172.21.15.28:0/3608455836 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ef015c8a0 0x7f5ef00b1d50 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.177+0000 7f5ef5094700 1 --2- 172.21.15.28:0/3608455836 >> v2:172.21.15.59:3300/0 conn(0x7f5ef00b2290 0x7f5ef00ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.177+0000 7f5ef5094700 1 -- 172.21.15.28:0/3608455836 >> 172.21.15.28:0/3608455836 conn(0x7f5ef00b3610 msgr2=0x7f5ef00cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.178+0000 7f5ef5094700 1 -- 172.21.15.28:0/3608455836 shutdown_connections 2023-11-28T11:11:05.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.178+0000 7f5ef5094700 1 -- 172.21.15.28:0/3608455836 wait complete. 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.179+0000 7f5ef5094700 1 Processor -- start 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.179+0000 7f5ef5094700 1 -- start start 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5ef5094700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ef006f120 0x7f5ef00c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5ef5094700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef00b2290 0x7f5ef00c6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5ef5094700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ef015c8a0 0x7f5ef00cdb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5ef5094700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef0152750 con 0x7f5ef015c8a0 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5ef5094700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef00c63b0 con 0x7f5ef00b2290 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5ef5094700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ef01e3d10 con 0x7f5ef006f120 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5eef59e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ef015c8a0 0x7f5ef00cdb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5eef59e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ef015c8a0 0x7f5ef00cdb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50558/0 (socket says 172.21.15.28:50558) 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5eef59e700 1 -- 172.21.15.28:0/2877990811 learned_addr learned my addr 172.21.15.28:0/2877990811 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5eeed9d700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ef006f120 0x7f5ef00c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5eee59c700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef00b2290 0x7f5ef00c6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5eef59e700 1 -- 172.21.15.28:0/2877990811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ef006f120 msgr2=0x7f5ef00c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5eef59e700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ef006f120 0x7f5ef00c5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5eef59e700 1 -- 172.21.15.28:0/2877990811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef00b2290 msgr2=0x7f5ef00c6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5eef59e700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef00b2290 0x7f5ef00c6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5eef59e700 1 -- 172.21.15.28:0/2877990811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ee00094a0 con 0x7f5ef015c8a0 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5eee59c700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef00b2290 0x7f5ef00c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.180+0000 7f5eeed9d700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ef006f120 0x7f5ef00c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.181+0000 7f5eef59e700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ef015c8a0 0x7f5ef00cdb20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8002230 tx=0x7f5ee800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.181+0000 7f5ee77fe700 1 -- 172.21.15.28:0/2877990811 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee800ac20 con 0x7f5ef015c8a0 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.181+0000 7f5ee77fe700 1 -- 172.21.15.28:0/2877990811 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ee800ada0 con 0x7f5ef015c8a0 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.181+0000 7f5ef5094700 1 -- 172.21.15.28:0/2877990811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ef015cc70 con 0x7f5ef015c8a0 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.181+0000 7f5ee77fe700 1 -- 172.21.15.28:0/2877990811 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ee8007310 con 0x7f5ef015c8a0 2023-11-28T11:11:05.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.181+0000 7f5ef5094700 1 -- 172.21.15.28:0/2877990811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ef00b1830 con 0x7f5ef015c8a0 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.182+0000 7f5ef5094700 1 -- 172.21.15.28:0/2877990811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ef006a6c0 con 0x7f5ef015c8a0 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.182+0000 7f5ee77fe700 1 -- 172.21.15.28:0/2877990811 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5ee801a030 con 0x7f5ef015c8a0 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.183+0000 7f5ee77fe700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5ed00682f0 0x7f5ed006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.183+0000 7f5ee77fe700 1 -- 172.21.15.28:0/2877990811 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5ee8085bb0 con 0x7f5ef015c8a0 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.183+0000 7f5eeed9d700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5ed00682f0 0x7f5ed006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.183+0000 7f5eeed9d700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5ed00682f0 0x7f5ed006a7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00090b0 tx=0x7f5ee001c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.188 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.187+0000 7f5ee77fe700 1 -- 172.21.15.28:0/2877990811 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5ee808c080 con 0x7f5ef015c8a0 2023-11-28T11:11:05.341 INFO:tasks.ceph.mds.d.smithi059.stderr:2023-11-28T11:11:05.341+0000 7fae4d557700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:11:05.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.477+0000 7f5ef5094700 1 -- 172.21.15.28:0/2877990811 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5ef00675a0 con 0x7f5ef015c8a0 2023-11-28T11:11:05.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.478+0000 7f5ee77fe700 1 -- 172.21.15.28:0/2877990811 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1441 (secure 0 0 0) 0x7f5ee80185c0 con 0x7f5ef015c8a0 2023-11-28T11:11:05.479 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:05.479 INFO:teuthology.orchestra.run.smithi028.stdout:{"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":158,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":54,"num_osds":8,"num_up_osds":8,"osd_up_since":1701169717,"num_in_osds":8,"osd_in_since":1701169709,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":96},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":76058624,"bytes_avail":773018054656,"bytes_total":773094113280,"unknown_pgs_ratio":0.91428571939468384},"fsmap":{"epoch":25,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"d","status":"up:active","gid":6054}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-11-28T11:10:49.189942+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-28T11:11:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.480+0000 7f5ef5094700 1 -- 172.21.15.28:0/2877990811 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5ed00682f0 msgr2=0x7f5ed006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.480+0000 7f5ef5094700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5ed00682f0 0x7f5ed006a7a0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00090b0 tx=0x7f5ee001c040 comp rx=0 tx=0).stop 2023-11-28T11:11:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.480+0000 7f5ef5094700 1 -- 172.21.15.28:0/2877990811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ef015c8a0 msgr2=0x7f5ef00cdb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.480+0000 7f5ef5094700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ef015c8a0 0x7f5ef00cdb20 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f5ee8002230 tx=0x7f5ee800a060 comp rx=0 tx=0).stop 2023-11-28T11:11:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.480+0000 7f5ef5094700 1 -- 172.21.15.28:0/2877990811 shutdown_connections 2023-11-28T11:11:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.480+0000 7f5ef5094700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5ed00682f0 0x7f5ed006a7a0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.480+0000 7f5ef5094700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5ef015c8a0 0x7f5ef00cdb20 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.480+0000 7f5ef5094700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5ef006f120 0x7f5ef00c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.480+0000 7f5ef5094700 1 --2- 172.21.15.28:0/2877990811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef00b2290 0x7f5ef00c6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.480+0000 7f5ef5094700 1 -- 172.21.15.28:0/2877990811 >> 172.21.15.28:0/2877990811 conn(0x7f5ef00b3610 msgr2=0x7f5ef01e1b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.480+0000 7f5ef5094700 1 -- 172.21.15.28:0/2877990811 shutdown_connections 2023-11-28T11:11:05.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.480+0000 7f5ef5094700 1 -- 172.21.15.28:0/2877990811 wait complete. 2023-11-28T11:11:05.493 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T11:11:05.494 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:11:05.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.616+0000 7fd1090aa700 1 Processor -- start 2023-11-28T11:11:05.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.616+0000 7fd1090aa700 1 -- start start 2023-11-28T11:11:05.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd1090aa700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd10415c990 0x7fd1040cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd1090aa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd104161090 0x7fd1040ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd1090aa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd1040caeb0 0x7fd1040cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd1090aa700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1040ce340 con 0x7fd10415c990 2023-11-28T11:11:05.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd1090aa700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1040cebe0 con 0x7fd1040caeb0 2023-11-28T11:11:05.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd1090aa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1040cb3f0 con 0x7fd104161090 2023-11-28T11:11:05.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd102d9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd10415c990 0x7fd1040cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd102d9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd10415c990 0x7fd1040cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50562/0 (socket says 172.21.15.28:50562) 2023-11-28T11:11:05.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd102d9d700 1 -- 172.21.15.28:0/1921205054 learned_addr learned my addr 172.21.15.28:0/1921205054 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:05.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd10359e700 1 --2- 172.21.15.28:0/1921205054 >> v2:172.21.15.59:3300/0 conn(0x7fd1040caeb0 0x7fd1040cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd10259c700 1 --2- 172.21.15.28:0/1921205054 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd104161090 0x7fd1040ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd102d9d700 1 -- 172.21.15.28:0/1921205054 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd104161090 msgr2=0x7fd1040ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd102d9d700 1 --2- 172.21.15.28:0/1921205054 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd104161090 0x7fd1040ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd102d9d700 1 -- 172.21.15.28:0/1921205054 >> v2:172.21.15.59:3300/0 conn(0x7fd1040caeb0 msgr2=0x7fd1040cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:05.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd102d9d700 1 --2- 172.21.15.28:0/1921205054 >> v2:172.21.15.59:3300/0 conn(0x7fd1040caeb0 0x7fd1040cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.617+0000 7fd102d9d700 1 -- 172.21.15.28:0/1921205054 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1040ce590 con 0x7fd10415c990 2023-11-28T11:11:05.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.618+0000 7fd102d9d700 1 --2- 172.21.15.28:0/1921205054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd10415c990 0x7fd1040cf070 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd1040ca080 tx=0x7fd0f8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bb35e92ab6240b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.618+0000 7fd101d9b700 1 -- 172.21.15.28:0/1921205054 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f800b870 con 0x7fd10415c990 2023-11-28T11:11:05.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.618+0000 7fd101d9b700 1 -- 172.21.15.28:0/1921205054 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f800b9d0 con 0x7fd10415c990 2023-11-28T11:11:05.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.618+0000 7fd101d9b700 1 -- 172.21.15.28:0/1921205054 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f800bd60 con 0x7fd10415c990 2023-11-28T11:11:05.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.618+0000 7fd1090aa700 1 -- 172.21.15.28:0/1921205054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd10415c990 msgr2=0x7fd1040cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.618+0000 7fd1090aa700 1 --2- 172.21.15.28:0/1921205054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd10415c990 0x7fd1040cf070 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fd1040ca080 tx=0x7fd0f8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.618+0000 7fd1090aa700 1 -- 172.21.15.28:0/1921205054 shutdown_connections 2023-11-28T11:11:05.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.618+0000 7fd1090aa700 1 --2- 172.21.15.28:0/1921205054 >> v2:172.21.15.59:3300/0 conn(0x7fd1040caeb0 0x7fd1040cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.618+0000 7fd1090aa700 1 --2- 172.21.15.28:0/1921205054 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd10415c990 0x7fd1040cf070 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.618+0000 7fd1090aa700 1 --2- 172.21.15.28:0/1921205054 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd104161090 0x7fd1040ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.628 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.618+0000 7fd1090aa700 1 -- 172.21.15.28:0/1921205054 >> 172.21.15.28:0/1921205054 conn(0x7fd1040b1650 msgr2=0x7fd1040b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.629 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.618+0000 7fd1090aa700 1 -- 172.21.15.28:0/1921205054 shutdown_connections 2023-11-28T11:11:05.629 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.618+0000 7fd1090aa700 1 -- 172.21.15.28:0/1921205054 wait complete. 2023-11-28T11:11:05.629 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.620+0000 7fd1090aa700 1 Processor -- start 2023-11-28T11:11:05.629 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.620+0000 7fd1090aa700 1 -- start start 2023-11-28T11:11:05.629 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd1090aa700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1040caeb0 0x7fd10415ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.629 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd1090aa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd10415c990 0x7fd1040b5360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd1090aa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd104161090 0x7fd10406b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd1090aa700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041f54b0 con 0x7fd1040caeb0 2023-11-28T11:11:05.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd1090aa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041f62a0 con 0x7fd104161090 2023-11-28T11:11:05.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd1090aa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1041f5690 con 0x7fd10415c990 2023-11-28T11:11:05.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd102d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1040caeb0 0x7fd10415ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd102d9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1040caeb0 0x7fd10415ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50568/0 (socket says 172.21.15.28:50568) 2023-11-28T11:11:05.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd102d9d700 1 -- 172.21.15.28:0/731587251 learned_addr learned my addr 172.21.15.28:0/731587251 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:05.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd10359e700 1 --2- 172.21.15.28:0/731587251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd104161090 0x7fd10406b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd10259c700 1 --2- 172.21.15.28:0/731587251 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd10415c990 0x7fd1040b5360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd102d9d700 1 -- 172.21.15.28:0/731587251 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd10415c990 msgr2=0x7fd1040b5360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd102d9d700 1 --2- 172.21.15.28:0/731587251 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd10415c990 0x7fd1040b5360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd102d9d700 1 -- 172.21.15.28:0/731587251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd104161090 msgr2=0x7fd10406b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd102d9d700 1 --2- 172.21.15.28:0/731587251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd104161090 0x7fd10406b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd102d9d700 1 -- 172.21.15.28:0/731587251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f8009550 con 0x7fd1040caeb0 2023-11-28T11:11:05.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd102d9d700 1 --2- 172.21.15.28:0/731587251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1040caeb0 0x7fd10415ad30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8000c00 tx=0x7fd0f8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.621+0000 7fd0eb7fe700 1 -- 172.21.15.28:0/731587251 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8004540 con 0x7fd1040caeb0 2023-11-28T11:11:05.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.622+0000 7fd0eb7fe700 1 -- 172.21.15.28:0/731587251 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f80036a0 con 0x7fd1040caeb0 2023-11-28T11:11:05.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.622+0000 7fd1090aa700 1 -- 172.21.15.28:0/731587251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1040cf5b0 con 0x7fd1040caeb0 2023-11-28T11:11:05.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.622+0000 7fd0eb7fe700 1 -- 172.21.15.28:0/731587251 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f8003a50 con 0x7fd1040caeb0 2023-11-28T11:11:05.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.622+0000 7fd1090aa700 1 -- 172.21.15.28:0/731587251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1040ce870 con 0x7fd1040caeb0 2023-11-28T11:11:05.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.622+0000 7fd1090aa700 1 -- 172.21.15.28:0/731587251 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1040cf740 con 0x7fd1040caeb0 2023-11-28T11:11:05.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.623+0000 7fd0eb7fe700 1 -- 172.21.15.28:0/731587251 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd0f800f050 con 0x7fd1040caeb0 2023-11-28T11:11:05.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.623+0000 7fd0eb7fe700 1 --2- 172.21.15.28:0/731587251 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd0f00682f0 0x7fd0f006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:05.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.623+0000 7fd0eb7fe700 1 -- 172.21.15.28:0/731587251 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd0f8087710 con 0x7fd1040caeb0 2023-11-28T11:11:05.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.623+0000 7fd10259c700 1 --2- 172.21.15.28:0/731587251 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd0f00682f0 0x7fd0f006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:05.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.624+0000 7fd10259c700 1 --2- 172.21.15.28:0/731587251 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd0f00682f0 0x7fd0f006a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd10415b680 tx=0x7fd0f4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:05.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.628+0000 7fd0eb7fe700 1 -- 172.21.15.28:0/731587251 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd0f8053db0 con 0x7fd1040caeb0 2023-11-28T11:11:05.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.818+0000 7fd1090aa700 1 -- 172.21.15.28:0/731587251 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd1041f63e0 con 0x7fd0f00682f0 2023-11-28T11:11:05.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.827+0000 7fd0eb7fe700 1 -- 172.21.15.28:0/731587251 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311579 (secure 0 0 0) 0x7fd0f40094e0 con 0x7fd0f00682f0 2023-11-28T11:11:05.833 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:05.835 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.830+0000 7fd1090aa700 1 -- 172.21.15.28:0/731587251 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd0f00682f0 msgr2=0x7fd0f006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.835 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.830+0000 7fd1090aa700 1 --2- 172.21.15.28:0/731587251 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd0f00682f0 0x7fd0f006a7a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd10415b680 tx=0x7fd0f4008040 comp rx=0 tx=0).stop 2023-11-28T11:11:05.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.830+0000 7fd1090aa700 1 -- 172.21.15.28:0/731587251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1040caeb0 msgr2=0x7fd10415ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:05.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.830+0000 7fd1090aa700 1 --2- 172.21.15.28:0/731587251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1040caeb0 0x7fd10415ad30 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fd0f8000c00 tx=0x7fd0f8004750 comp rx=0 tx=0).stop 2023-11-28T11:11:05.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.830+0000 7fd1090aa700 1 -- 172.21.15.28:0/731587251 shutdown_connections 2023-11-28T11:11:05.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.830+0000 7fd1090aa700 1 --2- 172.21.15.28:0/731587251 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd0f00682f0 0x7fd0f006a7a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.830+0000 7fd1090aa700 1 --2- 172.21.15.28:0/731587251 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd1040caeb0 0x7fd10415ad30 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.830+0000 7fd1090aa700 1 --2- 172.21.15.28:0/731587251 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd104161090 0x7fd10406b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.830+0000 7fd1090aa700 1 --2- 172.21.15.28:0/731587251 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd10415c990 0x7fd1040b5360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:05.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.830+0000 7fd1090aa700 1 -- 172.21.15.28:0/731587251 >> 172.21.15.28:0/731587251 conn(0x7fd1040b1650 msgr2=0x7fd1040ca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:05.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.830+0000 7fd1090aa700 1 -- 172.21.15.28:0/731587251 shutdown_connections 2023-11-28T11:11:05.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:05.830+0000 7fd1090aa700 1 -- 172.21.15.28:0/731587251 wait complete. 2023-11-28T11:11:05.837 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:11:05.844 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2023-11-28T11:11:05.193001+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":143,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":75968,"kb_used_data":6208,"kb_used_omap":0,"kb_used_meta":69696,"kb_avail":754898752,"statfs":{"total":773094113280,"available":773016322048,"internally_reserved":0,"allocated":6356992,"data_stored":3551812,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":71368704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001165"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905490+0000","last_change":"2023-11-28T11:10:59.788613+0000","last_active":"2023-11-28T11:11:02.905490+0000","last_peered":"2023-11-28T11:11:02.905490+0000","last_clean":"2023-11-28T11:11:02.905490+0000","last_became_active":"2023-11-28T11:10:59.788354+0000","last_became_peered":"2023-11-28T11:10:59.788354+0000","last_unstale":"2023-11-28T11:11:02.905490+0000","last_undegraded":"2023-11-28T11:11:02.905490+0000","last_fullsized":"2023-11-28T11:11:02.905490+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:53.730776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905951+0000","last_change":"2023-11-28T11:10:59.802140+0000","last_active":"2023-11-28T11:11:02.905951+0000","last_peered":"2023-11-28T11:11:02.905951+0000","last_clean":"2023-11-28T11:11:02.905951+0000","last_became_active":"2023-11-28T11:10:59.801891+0000","last_became_peered":"2023-11-28T11:10:59.801891+0000","last_unstale":"2023-11-28T11:11:02.905951+0000","last_undegraded":"2023-11-28T11:11:02.905951+0000","last_fullsized":"2023-11-28T11:11:02.905951+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:51:59.306560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906388+0000","last_change":"2023-11-28T11:10:59.799426+0000","last_active":"2023-11-28T11:11:02.906388+0000","last_peered":"2023-11-28T11:11:02.906388+0000","last_clean":"2023-11-28T11:11:02.906388+0000","last_became_active":"2023-11-28T11:10:59.799085+0000","last_became_peered":"2023-11-28T11:10:59.799085+0000","last_unstale":"2023-11-28T11:11:02.906388+0000","last_undegraded":"2023-11-28T11:11:02.906388+0000","last_fullsized":"2023-11-28T11:11:02.906388+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:41:04.478315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784336+0000","last_change":"2023-11-28T11:10:59.791480+0000","last_active":"2023-11-28T11:11:00.784336+0000","last_peered":"2023-11-28T11:11:00.784336+0000","last_clean":"2023-11-28T11:11:00.784336+0000","last_became_active":"2023-11-28T11:10:59.791142+0000","last_became_peered":"2023-11-28T11:10:59.791142+0000","last_unstale":"2023-11-28T11:11:00.784336+0000","last_undegraded":"2023-11-28T11:11:00.784336+0000","last_fullsized":"2023-11-28T11:11:00.784336+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:41:25.021451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905306+0000","last_change":"2023-11-28T11:10:59.789548+0000","last_active":"2023-11-28T11:11:02.905306+0000","last_peered":"2023-11-28T11:11:02.905306+0000","last_clean":"2023-11-28T11:11:02.905306+0000","last_became_active":"2023-11-28T11:10:59.789280+0000","last_became_peered":"2023-11-28T11:10:59.789280+0000","last_unstale":"2023-11-28T11:11:02.905306+0000","last_undegraded":"2023-11-28T11:11:02.905306+0000","last_fullsized":"2023-11-28T11:11:02.905306+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:50:54.142216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906481+0000","last_change":"2023-11-28T11:10:59.798422+0000","last_active":"2023-11-28T11:11:02.906481+0000","last_peered":"2023-11-28T11:11:02.906481+0000","last_clean":"2023-11-28T11:11:02.906481+0000","last_became_active":"2023-11-28T11:10:59.798131+0000","last_became_peered":"2023-11-28T11:10:59.798131+0000","last_unstale":"2023-11-28T11:11:02.906481+0000","last_undegraded":"2023-11-28T11:11:02.906481+0000","last_fullsized":"2023-11-28T11:11:02.906481+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:26.055549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905293+0000","last_change":"2023-11-28T11:10:59.788420+0000","last_active":"2023-11-28T11:11:02.905293+0000","last_peered":"2023-11-28T11:11:02.905293+0000","last_clean":"2023-11-28T11:11:02.905293+0000","last_became_active":"2023-11-28T11:10:59.787673+0000","last_became_peered":"2023-11-28T11:10:59.787673+0000","last_unstale":"2023-11-28T11:11:02.905293+0000","last_undegraded":"2023-11-28T11:11:02.905293+0000","last_fullsized":"2023-11-28T11:11:02.905293+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:35:39.058059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905438+0000","last_change":"2023-11-28T11:10:59.782137+0000","last_active":"2023-11-28T11:11:02.905438+0000","last_peered":"2023-11-28T11:11:02.905438+0000","last_clean":"2023-11-28T11:11:02.905438+0000","last_became_active":"2023-11-28T11:10:59.781878+0000","last_became_peered":"2023-11-28T11:10:59.781878+0000","last_unstale":"2023-11-28T11:11:02.905438+0000","last_undegraded":"2023-11-28T11:11:02.905438+0000","last_fullsized":"2023-11-28T11:11:02.905438+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:39.866743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906413+0000","last_change":"2023-11-28T11:10:59.793042+0000","last_active":"2023-11-28T11:11:02.906413+0000","last_peered":"2023-11-28T11:11:02.906413+0000","last_clean":"2023-11-28T11:11:02.906413+0000","last_became_active":"2023-11-28T11:10:59.792726+0000","last_became_peered":"2023-11-28T11:10:59.792726+0000","last_unstale":"2023-11-28T11:11:02.906413+0000","last_undegraded":"2023-11-28T11:11:02.906413+0000","last_fullsized":"2023-11-28T11:11:02.906413+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:05:35.660459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780564+0000","last_change":"2023-11-28T11:10:59.800651+0000","last_active":"2023-11-28T11:11:00.780564+0000","last_peered":"2023-11-28T11:11:00.780564+0000","last_clean":"2023-11-28T11:11:00.780564+0000","last_became_active":"2023-11-28T11:10:59.800389+0000","last_became_peered":"2023-11-28T11:10:59.800389+0000","last_unstale":"2023-11-28T11:11:00.780564+0000","last_undegraded":"2023-11-28T11:11:00.780564+0000","last_fullsized":"2023-11-28T11:11:00.780564+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:46.598260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780379+0000","last_change":"2023-11-28T11:10:59.796275+0000","last_active":"2023-11-28T11:11:00.780379+0000","last_peered":"2023-11-28T11:11:00.780379+0000","last_clean":"2023-11-28T11:11:00.780379+0000","last_became_active":"2023-11-28T11:10:59.795017+0000","last_became_peered":"2023-11-28T11:10:59.795017+0000","last_unstale":"2023-11-28T11:11:00.780379+0000","last_undegraded":"2023-11-28T11:11:00.780379+0000","last_fullsized":"2023-11-28T11:11:00.780379+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:09:13.076060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784596+0000","last_change":"2023-11-28T11:10:59.803912+0000","last_active":"2023-11-28T11:11:00.784596+0000","last_peered":"2023-11-28T11:11:00.784596+0000","last_clean":"2023-11-28T11:11:00.784596+0000","last_became_active":"2023-11-28T11:10:59.803619+0000","last_became_peered":"2023-11-28T11:10:59.803619+0000","last_unstale":"2023-11-28T11:11:00.784596+0000","last_undegraded":"2023-11-28T11:11:00.784596+0000","last_fullsized":"2023-11-28T11:11:00.784596+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:03.916291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.292124+0000","last_change":"2023-11-28T11:10:59.800886+0000","last_active":"2023-11-28T11:11:02.292124+0000","last_peered":"2023-11-28T11:11:02.292124+0000","last_clean":"2023-11-28T11:11:02.292124+0000","last_became_active":"2023-11-28T11:10:59.800573+0000","last_became_peered":"2023-11-28T11:10:59.800573+0000","last_unstale":"2023-11-28T11:11:02.292124+0000","last_undegraded":"2023-11-28T11:11:02.292124+0000","last_fullsized":"2023-11-28T11:11:02.292124+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:44.618142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905494+0000","last_change":"2023-11-28T11:10:59.788195+0000","last_active":"2023-11-28T11:11:02.905494+0000","last_peered":"2023-11-28T11:11:02.905494+0000","last_clean":"2023-11-28T11:11:02.905494+0000","last_became_active":"2023-11-28T11:10:59.787925+0000","last_became_peered":"2023-11-28T11:10:59.787925+0000","last_unstale":"2023-11-28T11:11:02.905494+0000","last_undegraded":"2023-11-28T11:11:02.905494+0000","last_fullsized":"2023-11-28T11:11:02.905494+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:28.106265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905761+0000","last_change":"2023-11-28T11:10:59.801017+0000","last_active":"2023-11-28T11:11:02.905761+0000","last_peered":"2023-11-28T11:11:02.905761+0000","last_clean":"2023-11-28T11:11:02.905761+0000","last_became_active":"2023-11-28T11:10:59.800727+0000","last_became_peered":"2023-11-28T11:10:59.800727+0000","last_unstale":"2023-11-28T11:11:02.905761+0000","last_undegraded":"2023-11-28T11:11:02.905761+0000","last_fullsized":"2023-11-28T11:11:02.905761+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:38:05.209860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906104+0000","last_change":"2023-11-28T11:10:59.796635+0000","last_active":"2023-11-28T11:11:02.906104+0000","last_peered":"2023-11-28T11:11:02.906104+0000","last_clean":"2023-11-28T11:11:02.906104+0000","last_became_active":"2023-11-28T11:10:59.792803+0000","last_became_peered":"2023-11-28T11:10:59.792803+0000","last_unstale":"2023-11-28T11:11:02.906104+0000","last_undegraded":"2023-11-28T11:11:02.906104+0000","last_fullsized":"2023-11-28T11:11:02.906104+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:51:59.306560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905931+0000","last_change":"2023-11-28T11:10:59.791572+0000","last_active":"2023-11-28T11:11:02.905931+0000","last_peered":"2023-11-28T11:11:02.905931+0000","last_clean":"2023-11-28T11:11:02.905931+0000","last_became_active":"2023-11-28T11:10:59.789432+0000","last_became_peered":"2023-11-28T11:10:59.789432+0000","last_unstale":"2023-11-28T11:11:02.905931+0000","last_undegraded":"2023-11-28T11:11:02.905931+0000","last_fullsized":"2023-11-28T11:11:02.905931+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:14.482363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906317+0000","last_change":"2023-11-28T11:10:59.799877+0000","last_active":"2023-11-28T11:11:02.906317+0000","last_peered":"2023-11-28T11:11:02.906317+0000","last_clean":"2023-11-28T11:11:02.906317+0000","last_became_active":"2023-11-28T11:10:59.799607+0000","last_became_peered":"2023-11-28T11:10:59.799607+0000","last_unstale":"2023-11-28T11:11:02.906317+0000","last_undegraded":"2023-11-28T11:11:02.906317+0000","last_fullsized":"2023-11-28T11:11:02.906317+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:09:50.383846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905696+0000","last_change":"2023-11-28T11:10:59.788241+0000","last_active":"2023-11-28T11:11:02.905696+0000","last_peered":"2023-11-28T11:11:02.905696+0000","last_clean":"2023-11-28T11:11:02.905696+0000","last_became_active":"2023-11-28T11:10:59.787995+0000","last_became_peered":"2023-11-28T11:10:59.787995+0000","last_unstale":"2023-11-28T11:11:02.905696+0000","last_undegraded":"2023-11-28T11:11:02.905696+0000","last_fullsized":"2023-11-28T11:11:02.905696+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:45.705252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906093+0000","last_change":"2023-11-28T11:10:59.794292+0000","last_active":"2023-11-28T11:11:02.906093+0000","last_peered":"2023-11-28T11:11:02.906093+0000","last_clean":"2023-11-28T11:11:02.906093+0000","last_became_active":"2023-11-28T11:10:59.793249+0000","last_became_peered":"2023-11-28T11:10:59.793249+0000","last_unstale":"2023-11-28T11:11:02.906093+0000","last_undegraded":"2023-11-28T11:11:02.906093+0000","last_fullsized":"2023-11-28T11:11:02.906093+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:05:14.963396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905666+0000","last_change":"2023-11-28T11:10:59.800290+0000","last_active":"2023-11-28T11:11:02.905666+0000","last_peered":"2023-11-28T11:11:02.905666+0000","last_clean":"2023-11-28T11:11:02.905666+0000","last_became_active":"2023-11-28T11:10:59.798617+0000","last_became_peered":"2023-11-28T11:10:59.798617+0000","last_unstale":"2023-11-28T11:11:02.905666+0000","last_undegraded":"2023-11-28T11:11:02.905666+0000","last_fullsized":"2023-11-28T11:11:02.905666+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:47:42.462499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905559+0000","last_change":"2023-11-28T11:10:59.787921+0000","last_active":"2023-11-28T11:11:02.905559+0000","last_peered":"2023-11-28T11:11:02.905559+0000","last_clean":"2023-11-28T11:11:02.905559+0000","last_became_active":"2023-11-28T11:10:59.787453+0000","last_became_peered":"2023-11-28T11:10:59.787453+0000","last_unstale":"2023-11-28T11:11:02.905559+0000","last_undegraded":"2023-11-28T11:11:02.905559+0000","last_fullsized":"2023-11-28T11:11:02.905559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:56.378053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906247+0000","last_change":"2023-11-28T11:10:59.803707+0000","last_active":"2023-11-28T11:11:02.906247+0000","last_peered":"2023-11-28T11:11:02.906247+0000","last_clean":"2023-11-28T11:11:02.906247+0000","last_became_active":"2023-11-28T11:10:59.803411+0000","last_became_peered":"2023-11-28T11:10:59.803411+0000","last_unstale":"2023-11-28T11:11:02.906247+0000","last_undegraded":"2023-11-28T11:11:02.906247+0000","last_fullsized":"2023-11-28T11:11:02.906247+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:46.079984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.292020+0000","last_change":"2023-11-28T11:10:59.801163+0000","last_active":"2023-11-28T11:11:02.292020+0000","last_peered":"2023-11-28T11:11:02.292020+0000","last_clean":"2023-11-28T11:11:02.292020+0000","last_became_active":"2023-11-28T11:10:59.800902+0000","last_became_peered":"2023-11-28T11:10:59.800902+0000","last_unstale":"2023-11-28T11:11:02.292020+0000","last_undegraded":"2023-11-28T11:11:02.292020+0000","last_fullsized":"2023-11-28T11:11:02.292020+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:52:57.405614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905689+0000","last_change":"2023-11-28T11:10:59.802820+0000","last_active":"2023-11-28T11:11:02.905689+0000","last_peered":"2023-11-28T11:11:02.905689+0000","last_clean":"2023-11-28T11:11:02.905689+0000","last_became_active":"2023-11-28T11:10:59.802559+0000","last_became_peered":"2023-11-28T11:10:59.802559+0000","last_unstale":"2023-11-28T11:11:02.905689+0000","last_undegraded":"2023-11-28T11:11:02.905689+0000","last_fullsized":"2023-11-28T11:11:02.905689+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:19:08.724798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784461+0000","last_change":"2023-11-28T11:10:59.791720+0000","last_active":"2023-11-28T11:11:00.784461+0000","last_peered":"2023-11-28T11:11:00.784461+0000","last_clean":"2023-11-28T11:11:00.784461+0000","last_became_active":"2023-11-28T11:10:59.791413+0000","last_became_peered":"2023-11-28T11:10:59.791413+0000","last_unstale":"2023-11-28T11:11:00.784461+0000","last_undegraded":"2023-11-28T11:11:00.784461+0000","last_fullsized":"2023-11-28T11:11:00.784461+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:41:04.478315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906745+0000","last_change":"2023-11-28T11:10:59.798723+0000","last_active":"2023-11-28T11:11:02.906745+0000","last_peered":"2023-11-28T11:11:02.906745+0000","last_clean":"2023-11-28T11:11:02.906745+0000","last_became_active":"2023-11-28T11:10:59.796726+0000","last_became_peered":"2023-11-28T11:10:59.796726+0000","last_unstale":"2023-11-28T11:11:02.906745+0000","last_undegraded":"2023-11-28T11:11:02.906745+0000","last_fullsized":"2023-11-28T11:11:02.906745+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:58.186048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906436+0000","last_change":"2023-11-28T11:10:59.792848+0000","last_active":"2023-11-28T11:11:02.906436+0000","last_peered":"2023-11-28T11:11:02.906436+0000","last_clean":"2023-11-28T11:11:02.906436+0000","last_became_active":"2023-11-28T11:10:59.792526+0000","last_became_peered":"2023-11-28T11:10:59.792526+0000","last_unstale":"2023-11-28T11:11:02.906436+0000","last_undegraded":"2023-11-28T11:11:02.906436+0000","last_fullsized":"2023-11-28T11:11:02.906436+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:56:00.455574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905887+0000","last_change":"2023-11-28T11:10:59.788209+0000","last_active":"2023-11-28T11:11:02.905887+0000","last_peered":"2023-11-28T11:11:02.905887+0000","last_clean":"2023-11-28T11:11:02.905887+0000","last_became_active":"2023-11-28T11:10:59.787980+0000","last_became_peered":"2023-11-28T11:10:59.787980+0000","last_unstale":"2023-11-28T11:11:02.905887+0000","last_undegraded":"2023-11-28T11:11:02.905887+0000","last_fullsized":"2023-11-28T11:11:02.905887+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:35:10.703217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905973+0000","last_change":"2023-11-28T11:10:59.795807+0000","last_active":"2023-11-28T11:11:02.905973+0000","last_peered":"2023-11-28T11:11:02.905973+0000","last_clean":"2023-11-28T11:11:02.905973+0000","last_became_active":"2023-11-28T11:10:59.795521+0000","last_became_peered":"2023-11-28T11:10:59.795521+0000","last_unstale":"2023-11-28T11:11:02.905973+0000","last_undegraded":"2023-11-28T11:11:02.905973+0000","last_fullsized":"2023-11-28T11:11:02.905973+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:18.802241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905259+0000","last_change":"2023-11-28T11:10:59.802476+0000","last_active":"2023-11-28T11:11:02.905259+0000","last_peered":"2023-11-28T11:11:02.905259+0000","last_clean":"2023-11-28T11:11:02.905259+0000","last_became_active":"2023-11-28T11:10:59.802192+0000","last_became_peered":"2023-11-28T11:10:59.802192+0000","last_unstale":"2023-11-28T11:11:02.905259+0000","last_undegraded":"2023-11-28T11:11:02.905259+0000","last_fullsized":"2023-11-28T11:11:02.905259+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:59:10.861518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906014+0000","last_change":"2023-11-28T11:10:59.796452+0000","last_active":"2023-11-28T11:11:02.906014+0000","last_peered":"2023-11-28T11:11:02.906014+0000","last_clean":"2023-11-28T11:11:02.906014+0000","last_became_active":"2023-11-28T11:10:59.792787+0000","last_became_peered":"2023-11-28T11:10:59.792787+0000","last_unstale":"2023-11-28T11:11:02.906014+0000","last_undegraded":"2023-11-28T11:11:02.906014+0000","last_fullsized":"2023-11-28T11:11:02.906014+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:28.106265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906916+0000","last_change":"2023-11-28T11:10:59.796748+0000","last_active":"2023-11-28T11:11:02.906916+0000","last_peered":"2023-11-28T11:11:02.906916+0000","last_clean":"2023-11-28T11:11:02.906916+0000","last_became_active":"2023-11-28T11:10:59.796555+0000","last_became_peered":"2023-11-28T11:10:59.796555+0000","last_unstale":"2023-11-28T11:11:02.906916+0000","last_undegraded":"2023-11-28T11:11:02.906916+0000","last_fullsized":"2023-11-28T11:11:02.906916+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:36:15.362997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.907025+0000","last_change":"2023-11-28T11:10:59.365054+0000","last_active":"2023-11-28T11:11:02.907025+0000","last_peered":"2023-11-28T11:11:02.907025+0000","last_clean":"2023-11-28T11:11:02.907025+0000","last_became_active":"2023-11-28T11:10:59.364212+0000","last_became_peered":"2023-11-28T11:10:59.364212+0000","last_unstale":"2023-11-28T11:11:02.907025+0000","last_undegraded":"2023-11-28T11:11:02.907025+0000","last_fullsized":"2023-11-28T11:11:02.907025+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:01:08.672135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780209+0000","last_change":"2023-11-28T11:10:59.800697+0000","last_active":"2023-11-28T11:11:00.780209+0000","last_peered":"2023-11-28T11:11:00.780209+0000","last_clean":"2023-11-28T11:11:00.780209+0000","last_became_active":"2023-11-28T11:10:59.800376+0000","last_became_peered":"2023-11-28T11:10:59.800376+0000","last_unstale":"2023-11-28T11:11:00.780209+0000","last_undegraded":"2023-11-28T11:11:00.780209+0000","last_fullsized":"2023-11-28T11:11:00.780209+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:46.415640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906244+0000","last_change":"2023-11-28T11:10:59.371087+0000","last_active":"2023-11-28T11:11:02.906244+0000","last_peered":"2023-11-28T11:11:02.906244+0000","last_clean":"2023-11-28T11:11:02.906244+0000","last_became_active":"2023-11-28T11:10:59.370756+0000","last_became_peered":"2023-11-28T11:10:59.370756+0000","last_unstale":"2023-11-28T11:11:02.906244+0000","last_undegraded":"2023-11-28T11:11:02.906244+0000","last_fullsized":"2023-11-28T11:11:02.906244+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:41:03.472842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906659+0000","last_change":"2023-11-28T11:10:59.803851+0000","last_active":"2023-11-28T11:11:02.906659+0000","last_peered":"2023-11-28T11:11:02.906659+0000","last_clean":"2023-11-28T11:11:02.906659+0000","last_became_active":"2023-11-28T11:10:59.803504+0000","last_became_peered":"2023-11-28T11:10:59.803504+0000","last_unstale":"2023-11-28T11:11:02.906659+0000","last_undegraded":"2023-11-28T11:11:02.906659+0000","last_fullsized":"2023-11-28T11:11:02.906659+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:00.484044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906918+0000","last_change":"2023-11-28T11:10:59.365025+0000","last_active":"2023-11-28T11:11:02.906918+0000","last_peered":"2023-11-28T11:11:02.906918+0000","last_clean":"2023-11-28T11:11:02.906918+0000","last_became_active":"2023-11-28T11:10:59.364746+0000","last_became_peered":"2023-11-28T11:10:59.364746+0000","last_unstale":"2023-11-28T11:11:02.906918+0000","last_undegraded":"2023-11-28T11:11:02.906918+0000","last_fullsized":"2023-11-28T11:11:02.906918+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:17.796768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906772+0000","last_change":"2023-11-28T11:10:59.798706+0000","last_active":"2023-11-28T11:11:02.906772+0000","last_peered":"2023-11-28T11:11:02.906772+0000","last_clean":"2023-11-28T11:11:02.906772+0000","last_became_active":"2023-11-28T11:10:59.798287+0000","last_became_peered":"2023-11-28T11:10:59.798287+0000","last_unstale":"2023-11-28T11:11:02.906772+0000","last_undegraded":"2023-11-28T11:11:02.906772+0000","last_fullsized":"2023-11-28T11:11:02.906772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:56.378053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784575+0000","last_change":"2023-11-28T11:10:59.366716+0000","last_active":"2023-11-28T11:11:00.784575+0000","last_peered":"2023-11-28T11:11:00.784575+0000","last_clean":"2023-11-28T11:11:00.784575+0000","last_became_active":"2023-11-28T11:10:59.366426+0000","last_became_peered":"2023-11-28T11:10:59.366426+0000","last_unstale":"2023-11-28T11:11:00.784575+0000","last_undegraded":"2023-11-28T11:11:00.784575+0000","last_fullsized":"2023-11-28T11:11:00.784575+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:45.410168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905821+0000","last_change":"2023-11-28T11:10:59.788013+0000","last_active":"2023-11-28T11:11:02.905821+0000","last_peered":"2023-11-28T11:11:02.905821+0000","last_clean":"2023-11-28T11:11:02.905821+0000","last_became_active":"2023-11-28T11:10:59.787380+0000","last_became_peered":"2023-11-28T11:10:59.787380+0000","last_unstale":"2023-11-28T11:11:02.905821+0000","last_undegraded":"2023-11-28T11:11:02.905821+0000","last_fullsized":"2023-11-28T11:11:02.905821+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:58.186048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905452+0000","last_change":"2023-11-28T11:10:59.366526+0000","last_active":"2023-11-28T11:11:02.905452+0000","last_peered":"2023-11-28T11:11:02.905452+0000","last_clean":"2023-11-28T11:11:02.905452+0000","last_became_active":"2023-11-28T11:10:59.366249+0000","last_became_peered":"2023-11-28T11:10:59.366249+0000","last_unstale":"2023-11-28T11:11:02.905452+0000","last_undegraded":"2023-11-28T11:11:02.905452+0000","last_fullsized":"2023-11-28T11:11:02.905452+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:01:08.672135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906005+0000","last_change":"2023-11-28T11:10:59.794676+0000","last_active":"2023-11-28T11:11:02.906005+0000","last_peered":"2023-11-28T11:11:02.906005+0000","last_clean":"2023-11-28T11:11:02.906005+0000","last_became_active":"2023-11-28T11:10:59.793185+0000","last_became_peered":"2023-11-28T11:10:59.793185+0000","last_unstale":"2023-11-28T11:11:02.906005+0000","last_undegraded":"2023-11-28T11:11:02.906005+0000","last_fullsized":"2023-11-28T11:11:02.906005+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:17.744141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906517+0000","last_change":"2023-11-28T11:10:59.368381+0000","last_active":"2023-11-28T11:11:02.906517+0000","last_peered":"2023-11-28T11:11:02.906517+0000","last_clean":"2023-11-28T11:11:02.906517+0000","last_became_active":"2023-11-28T11:10:59.367921+0000","last_became_peered":"2023-11-28T11:10:59.367921+0000","last_unstale":"2023-11-28T11:11:02.906517+0000","last_undegraded":"2023-11-28T11:11:02.906517+0000","last_fullsized":"2023-11-28T11:11:02.906517+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:22.959630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.292011+0000","last_change":"2023-11-28T11:10:59.795964+0000","last_active":"2023-11-28T11:11:02.292011+0000","last_peered":"2023-11-28T11:11:02.292011+0000","last_clean":"2023-11-28T11:11:02.292011+0000","last_became_active":"2023-11-28T11:10:59.795666+0000","last_became_peered":"2023-11-28T11:10:59.795666+0000","last_unstale":"2023-11-28T11:11:02.292011+0000","last_undegraded":"2023-11-28T11:11:02.292011+0000","last_fullsized":"2023-11-28T11:11:02.292011+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:00:27.962233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784097+0000","last_change":"2023-11-28T11:10:59.358836+0000","last_active":"2023-11-28T11:11:00.784097+0000","last_peered":"2023-11-28T11:11:00.784097+0000","last_clean":"2023-11-28T11:11:00.784097+0000","last_became_active":"2023-11-28T11:10:59.358559+0000","last_became_peered":"2023-11-28T11:10:59.358559+0000","last_unstale":"2023-11-28T11:11:00.784097+0000","last_undegraded":"2023-11-28T11:11:00.784097+0000","last_fullsized":"2023-11-28T11:11:00.784097+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:28:43.457063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905683+0000","last_change":"2023-11-28T11:10:59.792069+0000","last_active":"2023-11-28T11:11:02.905683+0000","last_peered":"2023-11-28T11:11:02.905683+0000","last_clean":"2023-11-28T11:11:02.905683+0000","last_became_active":"2023-11-28T11:10:59.791286+0000","last_became_peered":"2023-11-28T11:10:59.791286+0000","last_unstale":"2023-11-28T11:11:02.905683+0000","last_undegraded":"2023-11-28T11:11:02.905683+0000","last_fullsized":"2023-11-28T11:11:02.905683+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:16:07.408042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905306+0000","last_change":"2023-11-28T11:10:59.362945+0000","last_active":"2023-11-28T11:11:02.905306+0000","last_peered":"2023-11-28T11:11:02.905306+0000","last_clean":"2023-11-28T11:11:02.905306+0000","last_became_active":"2023-11-28T11:10:59.362418+0000","last_became_peered":"2023-11-28T11:10:59.362418+0000","last_unstale":"2023-11-28T11:11:02.905306+0000","last_undegraded":"2023-11-28T11:11:02.905306+0000","last_fullsized":"2023-11-28T11:11:02.905306+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:12.320362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906363+0000","last_change":"2023-11-28T11:10:59.801036+0000","last_active":"2023-11-28T11:11:02.906363+0000","last_peered":"2023-11-28T11:11:02.906363+0000","last_clean":"2023-11-28T11:11:02.906363+0000","last_became_active":"2023-11-28T11:10:59.800738+0000","last_became_peered":"2023-11-28T11:10:59.800738+0000","last_unstale":"2023-11-28T11:11:02.906363+0000","last_undegraded":"2023-11-28T11:11:02.906363+0000","last_fullsized":"2023-11-28T11:11:02.906363+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:53:40.993953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906283+0000","last_change":"2023-11-28T11:10:59.370351+0000","last_active":"2023-11-28T11:11:02.906283+0000","last_peered":"2023-11-28T11:11:02.906283+0000","last_clean":"2023-11-28T11:11:02.906283+0000","last_became_active":"2023-11-28T11:10:59.370020+0000","last_became_peered":"2023-11-28T11:10:59.370020+0000","last_unstale":"2023-11-28T11:11:02.906283+0000","last_undegraded":"2023-11-28T11:11:02.906283+0000","last_fullsized":"2023-11-28T11:11:02.906283+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:21.232441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906148+0000","last_change":"2023-11-28T11:10:59.801568+0000","last_active":"2023-11-28T11:11:02.906148+0000","last_peered":"2023-11-28T11:11:02.906148+0000","last_clean":"2023-11-28T11:11:02.906148+0000","last_became_active":"2023-11-28T11:10:59.801229+0000","last_became_peered":"2023-11-28T11:10:59.801229+0000","last_unstale":"2023-11-28T11:11:02.906148+0000","last_undegraded":"2023-11-28T11:11:02.906148+0000","last_fullsized":"2023-11-28T11:11:02.906148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:03.916291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906301+0000","last_change":"2023-11-28T11:10:59.365589+0000","last_active":"2023-11-28T11:11:02.906301+0000","last_peered":"2023-11-28T11:11:02.906301+0000","last_clean":"2023-11-28T11:11:02.906301+0000","last_became_active":"2023-11-28T11:10:59.365321+0000","last_became_peered":"2023-11-28T11:10:59.365321+0000","last_unstale":"2023-11-28T11:11:02.906301+0000","last_undegraded":"2023-11-28T11:11:02.906301+0000","last_fullsized":"2023-11-28T11:11:02.906301+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:17.796768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780042+0000","last_change":"2023-11-28T11:10:59.801864+0000","last_active":"2023-11-28T11:11:00.780042+0000","last_peered":"2023-11-28T11:11:00.780042+0000","last_clean":"2023-11-28T11:11:00.780042+0000","last_became_active":"2023-11-28T11:10:59.801546+0000","last_became_peered":"2023-11-28T11:10:59.801546+0000","last_unstale":"2023-11-28T11:11:00.780042+0000","last_undegraded":"2023-11-28T11:11:00.780042+0000","last_fullsized":"2023-11-28T11:11:00.780042+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:49.736618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906036+0000","last_change":"2023-11-28T11:10:59.370539+0000","last_active":"2023-11-28T11:11:02.906036+0000","last_peered":"2023-11-28T11:11:02.906036+0000","last_clean":"2023-11-28T11:11:02.906036+0000","last_became_active":"2023-11-28T11:10:59.370111+0000","last_became_peered":"2023-11-28T11:10:59.370111+0000","last_unstale":"2023-11-28T11:11:02.906036+0000","last_undegraded":"2023-11-28T11:11:02.906036+0000","last_fullsized":"2023-11-28T11:11:02.906036+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:48.731146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905772+0000","last_change":"2023-11-28T11:10:59.805481+0000","last_active":"2023-11-28T11:11:02.905772+0000","last_peered":"2023-11-28T11:11:02.905772+0000","last_clean":"2023-11-28T11:11:02.905772+0000","last_became_active":"2023-11-28T11:10:59.805201+0000","last_became_peered":"2023-11-28T11:10:59.805201+0000","last_unstale":"2023-11-28T11:11:02.905772+0000","last_undegraded":"2023-11-28T11:11:02.905772+0000","last_fullsized":"2023-11-28T11:11:02.905772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:53:40.993953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905762+0000","last_change":"2023-11-28T11:10:59.365182+0000","last_active":"2023-11-28T11:11:02.905762+0000","last_peered":"2023-11-28T11:11:02.905762+0000","last_clean":"2023-11-28T11:11:02.905762+0000","last_became_active":"2023-11-28T11:10:59.364858+0000","last_became_peered":"2023-11-28T11:10:59.364858+0000","last_unstale":"2023-11-28T11:11:02.905762+0000","last_undegraded":"2023-11-28T11:11:02.905762+0000","last_fullsized":"2023-11-28T11:11:02.905762+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:17.796768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784163+0000","last_change":"2023-11-28T11:10:59.794457+0000","last_active":"2023-11-28T11:11:00.784163+0000","last_peered":"2023-11-28T11:11:00.784163+0000","last_clean":"2023-11-28T11:11:00.784163+0000","last_became_active":"2023-11-28T11:10:59.794187+0000","last_became_peered":"2023-11-28T11:10:59.794187+0000","last_unstale":"2023-11-28T11:11:00.784163+0000","last_undegraded":"2023-11-28T11:11:00.784163+0000","last_fullsized":"2023-11-28T11:11:00.784163+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:46.079984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906158+0000","last_change":"2023-11-28T11:10:59.366540+0000","last_active":"2023-11-28T11:11:02.906158+0000","last_peered":"2023-11-28T11:11:02.906158+0000","last_clean":"2023-11-28T11:11:02.906158+0000","last_became_active":"2023-11-28T11:10:59.366086+0000","last_became_peered":"2023-11-28T11:10:59.366086+0000","last_unstale":"2023-11-28T11:11:02.906158+0000","last_undegraded":"2023-11-28T11:11:02.906158+0000","last_fullsized":"2023-11-28T11:11:02.906158+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:35.736465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906065+0000","last_change":"2023-11-28T11:10:59.796322+0000","last_active":"2023-11-28T11:11:02.906065+0000","last_peered":"2023-11-28T11:11:02.906065+0000","last_clean":"2023-11-28T11:11:02.906065+0000","last_became_active":"2023-11-28T11:10:59.796041+0000","last_became_peered":"2023-11-28T11:10:59.796041+0000","last_unstale":"2023-11-28T11:11:02.906065+0000","last_undegraded":"2023-11-28T11:11:02.906065+0000","last_fullsized":"2023-11-28T11:11:02.906065+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:41:25.021451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.292124+0000","last_change":"2023-11-28T11:10:59.370497+0000","last_active":"2023-11-28T11:11:02.292124+0000","last_peered":"2023-11-28T11:11:02.292124+0000","last_clean":"2023-11-28T11:11:02.292124+0000","last_became_active":"2023-11-28T11:10:59.370219+0000","last_became_peered":"2023-11-28T11:10:59.370219+0000","last_unstale":"2023-11-28T11:11:02.292124+0000","last_undegraded":"2023-11-28T11:11:02.292124+0000","last_fullsized":"2023-11-28T11:11:02.292124+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:57:16.833452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905218+0000","last_change":"2023-11-28T11:10:59.796459+0000","last_active":"2023-11-28T11:11:02.905218+0000","last_peered":"2023-11-28T11:11:02.905218+0000","last_clean":"2023-11-28T11:11:02.905218+0000","last_became_active":"2023-11-28T11:10:59.796175+0000","last_became_peered":"2023-11-28T11:10:59.796175+0000","last_unstale":"2023-11-28T11:11:02.905218+0000","last_undegraded":"2023-11-28T11:11:02.905218+0000","last_fullsized":"2023-11-28T11:11:02.905218+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:26.055549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784167+0000","last_change":"2023-11-28T11:10:59.364716+0000","last_active":"2023-11-28T11:11:00.784167+0000","last_peered":"2023-11-28T11:11:00.784167+0000","last_clean":"2023-11-28T11:11:00.784167+0000","last_became_active":"2023-11-28T11:10:59.364388+0000","last_became_peered":"2023-11-28T11:10:59.364388+0000","last_unstale":"2023-11-28T11:11:00.784167+0000","last_undegraded":"2023-11-28T11:11:00.784167+0000","last_fullsized":"2023-11-28T11:11:00.784167+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:45.592788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906407+0000","last_change":"2023-11-28T11:10:59.793552+0000","last_active":"2023-11-28T11:11:02.906407+0000","last_peered":"2023-11-28T11:11:02.906407+0000","last_clean":"2023-11-28T11:11:02.906407+0000","last_became_active":"2023-11-28T11:10:59.793301+0000","last_became_peered":"2023-11-28T11:10:59.793301+0000","last_unstale":"2023-11-28T11:11:02.906407+0000","last_undegraded":"2023-11-28T11:11:02.906407+0000","last_fullsized":"2023-11-28T11:11:02.906407+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:23.965103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":19,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905654+0000","last_change":"2023-11-28T11:10:59.371819+0000","last_active":"2023-11-28T11:11:02.905654+0000","last_peered":"2023-11-28T11:11:02.905654+0000","last_clean":"2023-11-28T11:11:02.905654+0000","last_became_active":"2023-11-28T11:10:59.371521+0000","last_became_peered":"2023-11-28T11:10:59.371521+0000","last_unstale":"2023-11-28T11:11:02.905654+0000","last_undegraded":"2023-11-28T11:11:02.905654+0000","last_fullsized":"2023-11-28T11:11:02.905654+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:52:56.400142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784446+0000","last_change":"2023-11-28T11:10:59.791682+0000","last_active":"2023-11-28T11:11:00.784446+0000","last_peered":"2023-11-28T11:11:00.784446+0000","last_clean":"2023-11-28T11:11:00.784446+0000","last_became_active":"2023-11-28T11:10:59.791097+0000","last_became_peered":"2023-11-28T11:10:59.791097+0000","last_unstale":"2023-11-28T11:11:00.784446+0000","last_undegraded":"2023-11-28T11:11:00.784446+0000","last_fullsized":"2023-11-28T11:11:00.784446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:36.589155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780257+0000","last_change":"2023-11-28T11:10:59.365816+0000","last_active":"2023-11-28T11:11:00.780257+0000","last_peered":"2023-11-28T11:11:00.780257+0000","last_clean":"2023-11-28T11:11:00.780257+0000","last_became_active":"2023-11-28T11:10:59.365556+0000","last_became_peered":"2023-11-28T11:10:59.365556+0000","last_unstale":"2023-11-28T11:11:00.780257+0000","last_undegraded":"2023-11-28T11:11:00.780257+0000","last_fullsized":"2023-11-28T11:11:00.780257+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:38:46.615550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784573+0000","last_change":"2023-11-28T11:10:59.794871+0000","last_active":"2023-11-28T11:11:00.784573+0000","last_peered":"2023-11-28T11:11:00.784573+0000","last_clean":"2023-11-28T11:11:00.784573+0000","last_became_active":"2023-11-28T11:10:59.794570+0000","last_became_peered":"2023-11-28T11:10:59.794570+0000","last_unstale":"2023-11-28T11:11:00.784573+0000","last_undegraded":"2023-11-28T11:11:00.784573+0000","last_fullsized":"2023-11-28T11:11:00.784573+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:47:42.462499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906996+0000","last_change":"2023-11-28T11:10:59.366239+0000","last_active":"2023-11-28T11:11:02.906996+0000","last_peered":"2023-11-28T11:11:02.906996+0000","last_clean":"2023-11-28T11:11:02.906996+0000","last_became_active":"2023-11-28T11:10:59.365580+0000","last_became_peered":"2023-11-28T11:10:59.365580+0000","last_unstale":"2023-11-28T11:11:02.906996+0000","last_undegraded":"2023-11-28T11:11:02.906996+0000","last_fullsized":"2023-11-28T11:11:02.906996+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:41:03.472842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780106+0000","last_change":"2023-11-28T11:10:59.802145+0000","last_active":"2023-11-28T11:11:00.780106+0000","last_peered":"2023-11-28T11:11:00.780106+0000","last_clean":"2023-11-28T11:11:00.780106+0000","last_became_active":"2023-11-28T11:10:59.801683+0000","last_became_peered":"2023-11-28T11:10:59.801683+0000","last_unstale":"2023-11-28T11:11:00.780106+0000","last_undegraded":"2023-11-28T11:11:00.780106+0000","last_fullsized":"2023-11-28T11:11:00.780106+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:22.237914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.292182+0000","last_change":"2023-11-28T11:10:59.364244+0000","last_active":"2023-11-28T11:11:02.292182+0000","last_peered":"2023-11-28T11:11:02.292182+0000","last_clean":"2023-11-28T11:11:02.292182+0000","last_became_active":"2023-11-28T11:10:59.363887+0000","last_became_peered":"2023-11-28T11:10:59.363887+0000","last_unstale":"2023-11-28T11:11:02.292182+0000","last_undegraded":"2023-11-28T11:11:02.292182+0000","last_fullsized":"2023-11-28T11:11:02.292182+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:38:46.615550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906891+0000","last_change":"2023-11-28T11:10:59.798580+0000","last_active":"2023-11-28T11:11:02.906891+0000","last_peered":"2023-11-28T11:11:02.906891+0000","last_clean":"2023-11-28T11:11:02.906891+0000","last_became_active":"2023-11-28T11:10:59.798096+0000","last_became_peered":"2023-11-28T11:10:59.798096+0000","last_unstale":"2023-11-28T11:11:02.906891+0000","last_undegraded":"2023-11-28T11:11:02.906891+0000","last_fullsized":"2023-11-28T11:11:02.906891+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:39.866743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784462+0000","last_change":"2023-11-28T11:10:59.359526+0000","last_active":"2023-11-28T11:11:00.784462+0000","last_peered":"2023-11-28T11:11:00.784462+0000","last_clean":"2023-11-28T11:11:00.784462+0000","last_became_active":"2023-11-28T11:10:59.359227+0000","last_became_peered":"2023-11-28T11:10:59.359227+0000","last_unstale":"2023-11-28T11:11:00.784462+0000","last_undegraded":"2023-11-28T11:11:00.784462+0000","last_fullsized":"2023-11-28T11:11:00.784462+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:54.036483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905687+0000","last_change":"2023-11-28T11:10:59.800743+0000","last_active":"2023-11-28T11:11:02.905687+0000","last_peered":"2023-11-28T11:11:02.905687+0000","last_clean":"2023-11-28T11:11:02.905687+0000","last_became_active":"2023-11-28T11:10:59.800492+0000","last_became_peered":"2023-11-28T11:10:59.800492+0000","last_unstale":"2023-11-28T11:11:02.905687+0000","last_undegraded":"2023-11-28T11:11:02.905687+0000","last_fullsized":"2023-11-28T11:11:02.905687+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:00.484044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.779970+0000","last_change":"2023-11-28T11:10:59.360867+0000","last_active":"2023-11-28T11:11:00.779970+0000","last_peered":"2023-11-28T11:11:00.779970+0000","last_clean":"2023-11-28T11:11:00.779970+0000","last_became_active":"2023-11-28T11:10:59.360582+0000","last_became_peered":"2023-11-28T11:10:59.360582+0000","last_unstale":"2023-11-28T11:11:00.779970+0000","last_undegraded":"2023-11-28T11:11:00.779970+0000","last_fullsized":"2023-11-28T11:11:00.779970+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:25:03.498918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905927+0000","last_change":"2023-11-28T11:10:59.800578+0000","last_active":"2023-11-28T11:11:02.905927+0000","last_peered":"2023-11-28T11:11:02.905927+0000","last_clean":"2023-11-28T11:11:02.905927+0000","last_became_active":"2023-11-28T11:10:59.799028+0000","last_became_peered":"2023-11-28T11:10:59.799028+0000","last_unstale":"2023-11-28T11:11:02.905927+0000","last_undegraded":"2023-11-28T11:11:02.905927+0000","last_fullsized":"2023-11-28T11:11:02.905927+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:36.741937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905606+0000","last_change":"2023-11-28T11:10:59.365099+0000","last_active":"2023-11-28T11:11:02.905606+0000","last_peered":"2023-11-28T11:11:02.905606+0000","last_clean":"2023-11-28T11:11:02.905606+0000","last_became_active":"2023-11-28T11:10:59.364795+0000","last_became_peered":"2023-11-28T11:10:59.364795+0000","last_unstale":"2023-11-28T11:11:02.905606+0000","last_undegraded":"2023-11-28T11:11:02.905606+0000","last_fullsized":"2023-11-28T11:11:02.905606+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:41:03.472842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780687+0000","last_change":"2023-11-28T11:10:59.802156+0000","last_active":"2023-11-28T11:11:00.780687+0000","last_peered":"2023-11-28T11:11:00.780687+0000","last_clean":"2023-11-28T11:11:00.780687+0000","last_became_active":"2023-11-28T11:10:59.801852+0000","last_became_peered":"2023-11-28T11:10:59.801852+0000","last_unstale":"2023-11-28T11:11:00.780687+0000","last_undegraded":"2023-11-28T11:11:00.780687+0000","last_fullsized":"2023-11-28T11:11:00.780687+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:36.892073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.779819+0000","last_change":"2023-11-28T11:10:59.365299+0000","last_active":"2023-11-28T11:11:00.779819+0000","last_peered":"2023-11-28T11:11:00.779819+0000","last_clean":"2023-11-28T11:11:00.779819+0000","last_became_active":"2023-11-28T11:10:59.365006+0000","last_became_peered":"2023-11-28T11:10:59.365006+0000","last_unstale":"2023-11-28T11:11:00.779819+0000","last_undegraded":"2023-11-28T11:11:00.779819+0000","last_fullsized":"2023-11-28T11:11:00.779819+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:01:39.937685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905803+0000","last_change":"2023-11-28T11:10:59.791752+0000","last_active":"2023-11-28T11:11:02.905803+0000","last_peered":"2023-11-28T11:11:02.905803+0000","last_clean":"2023-11-28T11:11:02.905803+0000","last_became_active":"2023-11-28T11:10:59.790973+0000","last_became_peered":"2023-11-28T11:10:59.790973+0000","last_unstale":"2023-11-28T11:11:02.905803+0000","last_undegraded":"2023-11-28T11:11:02.905803+0000","last_fullsized":"2023-11-28T11:11:02.905803+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:49.410919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905730+0000","last_change":"2023-11-28T11:10:59.364424+0000","last_active":"2023-11-28T11:11:02.905730+0000","last_peered":"2023-11-28T11:11:02.905730+0000","last_clean":"2023-11-28T11:11:02.905730+0000","last_became_active":"2023-11-28T11:10:59.361681+0000","last_became_peered":"2023-11-28T11:10:59.361681+0000","last_unstale":"2023-11-28T11:11:02.905730+0000","last_undegraded":"2023-11-28T11:11:02.905730+0000","last_fullsized":"2023-11-28T11:11:02.905730+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:12.320362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905917+0000","last_change":"2023-11-28T11:10:59.805162+0000","last_active":"2023-11-28T11:11:02.905917+0000","last_peered":"2023-11-28T11:11:02.905917+0000","last_clean":"2023-11-28T11:11:02.905917+0000","last_became_active":"2023-11-28T11:10:59.804866+0000","last_became_peered":"2023-11-28T11:10:59.804866+0000","last_unstale":"2023-11-28T11:11:02.905917+0000","last_undegraded":"2023-11-28T11:11:02.905917+0000","last_fullsized":"2023-11-28T11:11:02.905917+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:36.589155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906802+0000","last_change":"2023-11-28T11:10:59.369331+0000","last_active":"2023-11-28T11:11:02.906802+0000","last_peered":"2023-11-28T11:11:02.906802+0000","last_clean":"2023-11-28T11:11:02.906802+0000","last_became_active":"2023-11-28T11:10:59.369122+0000","last_became_peered":"2023-11-28T11:10:59.369122+0000","last_unstale":"2023-11-28T11:11:02.906802+0000","last_undegraded":"2023-11-28T11:11:02.906802+0000","last_fullsized":"2023-11-28T11:11:02.906802+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:12.320362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":42,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.778121+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:10:59.778121+0000","last_peered":"2023-11-28T11:10:59.778121+0000","last_clean":"2023-11-28T11:10:59.778121+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:10:59.778121+0000","last_undegraded":"2023-11-28T11:10:59.778121+0000","last_fullsized":"2023-11-28T11:10:59.778121+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906171+0000","last_change":"2023-11-28T11:10:59.786892+0000","last_active":"2023-11-28T11:11:02.906171+0000","last_peered":"2023-11-28T11:11:02.906171+0000","last_clean":"2023-11-28T11:11:02.906171+0000","last_became_active":"2023-11-28T11:10:59.786632+0000","last_became_peered":"2023-11-28T11:10:59.786632+0000","last_unstale":"2023-11-28T11:11:02.906171+0000","last_undegraded":"2023-11-28T11:11:02.906171+0000","last_fullsized":"2023-11-28T11:11:02.906171+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:03.606113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784692+0000","last_change":"2023-11-28T11:10:59.364836+0000","last_active":"2023-11-28T11:11:00.784692+0000","last_peered":"2023-11-28T11:11:00.784692+0000","last_clean":"2023-11-28T11:11:00.784692+0000","last_became_active":"2023-11-28T11:10:59.364463+0000","last_became_peered":"2023-11-28T11:10:59.364463+0000","last_unstale":"2023-11-28T11:11:00.784692+0000","last_undegraded":"2023-11-28T11:11:00.784692+0000","last_fullsized":"2023-11-28T11:11:00.784692+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:35.886600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.776479+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:10:59.776479+0000","last_peered":"2023-11-28T11:10:59.776479+0000","last_clean":"2023-11-28T11:10:59.776479+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:10:59.776479+0000","last_undegraded":"2023-11-28T11:10:59.776479+0000","last_fullsized":"2023-11-28T11:10:59.776479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906840+0000","last_change":"2023-11-28T11:10:59.792194+0000","last_active":"2023-11-28T11:11:02.906840+0000","last_peered":"2023-11-28T11:11:02.906840+0000","last_clean":"2023-11-28T11:11:02.906840+0000","last_became_active":"2023-11-28T11:10:59.791942+0000","last_became_peered":"2023-11-28T11:10:59.791942+0000","last_unstale":"2023-11-28T11:11:02.906840+0000","last_undegraded":"2023-11-28T11:11:02.906840+0000","last_fullsized":"2023-11-28T11:11:02.906840+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:35:10.703217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780075+0000","last_change":"2023-11-28T11:10:59.365263+0000","last_active":"2023-11-28T11:11:00.780075+0000","last_peered":"2023-11-28T11:11:00.780075+0000","last_clean":"2023-11-28T11:11:00.780075+0000","last_became_active":"2023-11-28T11:10:59.364962+0000","last_became_peered":"2023-11-28T11:10:59.364962+0000","last_unstale":"2023-11-28T11:11:00.780075+0000","last_undegraded":"2023-11-28T11:11:00.780075+0000","last_fullsized":"2023-11-28T11:11:00.780075+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:57:16.833452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":43,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.781531+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:10:59.781531+0000","last_peered":"2023-11-28T11:10:59.781531+0000","last_clean":"2023-11-28T11:10:59.781531+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:10:59.781531+0000","last_undegraded":"2023-11-28T11:10:59.781531+0000","last_fullsized":"2023-11-28T11:10:59.781531+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":193,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.789300+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:10:59.789300+0000","last_peered":"2023-11-28T11:10:59.789300+0000","last_clean":"2023-11-28T11:10:59.789300+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:10:59.789300+0000","last_undegraded":"2023-11-28T11:10:59.789300+0000","last_fullsized":"2023-11-28T11:10:59.789300+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784636+0000","last_change":"2023-11-28T11:10:59.802124+0000","last_active":"2023-11-28T11:11:00.784636+0000","last_peered":"2023-11-28T11:11:00.784636+0000","last_clean":"2023-11-28T11:11:00.784636+0000","last_became_active":"2023-11-28T11:10:59.801897+0000","last_became_peered":"2023-11-28T11:10:59.801897+0000","last_unstale":"2023-11-28T11:11:00.784636+0000","last_undegraded":"2023-11-28T11:11:00.784636+0000","last_fullsized":"2023-11-28T11:11:00.784636+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:36.741937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906204+0000","last_change":"2023-11-28T11:10:59.364007+0000","last_active":"2023-11-28T11:11:02.906204+0000","last_peered":"2023-11-28T11:11:02.906204+0000","last_clean":"2023-11-28T11:11:02.906204+0000","last_became_active":"2023-11-28T11:10:59.363733+0000","last_became_peered":"2023-11-28T11:10:59.363733+0000","last_unstale":"2023-11-28T11:11:02.906204+0000","last_undegraded":"2023-11-28T11:11:02.906204+0000","last_fullsized":"2023-11-28T11:11:02.906204+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:23.407481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":43,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.787257+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:10:59.787257+0000","last_peered":"2023-11-28T11:10:59.787257+0000","last_clean":"2023-11-28T11:10:59.787257+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:10:59.787257+0000","last_undegraded":"2023-11-28T11:10:59.787257+0000","last_fullsized":"2023-11-28T11:10:59.787257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905628+0000","last_change":"2023-11-28T11:10:59.794401+0000","last_active":"2023-11-28T11:11:02.905628+0000","last_peered":"2023-11-28T11:11:02.905628+0000","last_clean":"2023-11-28T11:11:02.905628+0000","last_became_active":"2023-11-28T11:10:59.794032+0000","last_became_peered":"2023-11-28T11:10:59.794032+0000","last_unstale":"2023-11-28T11:11:02.905628+0000","last_undegraded":"2023-11-28T11:11:02.905628+0000","last_fullsized":"2023-11-28T11:11:02.905628+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:00.979070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.292057+0000","last_change":"2023-11-28T11:10:59.364552+0000","last_active":"2023-11-28T11:11:02.292057+0000","last_peered":"2023-11-28T11:11:02.292057+0000","last_clean":"2023-11-28T11:11:02.292057+0000","last_became_active":"2023-11-28T11:10:59.364156+0000","last_became_peered":"2023-11-28T11:10:59.364156+0000","last_unstale":"2023-11-28T11:11:02.292057+0000","last_undegraded":"2023-11-28T11:11:02.292057+0000","last_fullsized":"2023-11-28T11:11:02.292057+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:01:39.937685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":43,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.789181+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:10:59.789181+0000","last_peered":"2023-11-28T11:10:59.789181+0000","last_clean":"2023-11-28T11:10:59.789181+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:10:59.789181+0000","last_undegraded":"2023-11-28T11:10:59.789181+0000","last_fullsized":"2023-11-28T11:10:59.789181+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906014+0000","last_change":"2023-11-28T11:10:59.798700+0000","last_active":"2023-11-28T11:11:02.906014+0000","last_peered":"2023-11-28T11:11:02.906014+0000","last_clean":"2023-11-28T11:11:02.906014+0000","last_became_active":"2023-11-28T11:10:59.798436+0000","last_became_peered":"2023-11-28T11:10:59.798436+0000","last_unstale":"2023-11-28T11:11:02.906014+0000","last_undegraded":"2023-11-28T11:11:02.906014+0000","last_fullsized":"2023-11-28T11:11:02.906014+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:52.792611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784096+0000","last_change":"2023-11-28T11:10:59.363284+0000","last_active":"2023-11-28T11:11:00.784096+0000","last_peered":"2023-11-28T11:11:00.784096+0000","last_clean":"2023-11-28T11:11:00.784096+0000","last_became_active":"2023-11-28T11:10:59.362970+0000","last_became_peered":"2023-11-28T11:10:59.362970+0000","last_unstale":"2023-11-28T11:11:00.784096+0000","last_undegraded":"2023-11-28T11:11:00.784096+0000","last_fullsized":"2023-11-28T11:11:00.784096+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:14:29.172511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":46,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.781552+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:10:59.781552+0000","last_peered":"2023-11-28T11:10:59.781552+0000","last_clean":"2023-11-28T11:10:59.781552+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:10:59.781552+0000","last_undegraded":"2023-11-28T11:10:59.781552+0000","last_fullsized":"2023-11-28T11:10:59.781552+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906429+0000","last_change":"2023-11-28T11:10:59.798836+0000","last_active":"2023-11-28T11:11:02.906429+0000","last_peered":"2023-11-28T11:11:02.906429+0000","last_clean":"2023-11-28T11:11:02.906429+0000","last_became_active":"2023-11-28T11:10:59.798528+0000","last_became_peered":"2023-11-28T11:10:59.798528+0000","last_unstale":"2023-11-28T11:11:02.906429+0000","last_undegraded":"2023-11-28T11:11:02.906429+0000","last_fullsized":"2023-11-28T11:11:02.906429+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:13.325835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906041+0000","last_change":"2023-11-28T11:10:59.370468+0000","last_active":"2023-11-28T11:11:02.906041+0000","last_peered":"2023-11-28T11:11:02.906041+0000","last_clean":"2023-11-28T11:11:02.906041+0000","last_became_active":"2023-11-28T11:10:59.370130+0000","last_became_peered":"2023-11-28T11:10:59.370130+0000","last_unstale":"2023-11-28T11:11:02.906041+0000","last_undegraded":"2023-11-28T11:11:02.906041+0000","last_fullsized":"2023-11-28T11:11:02.906041+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:22.959630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.782724+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:10:59.782724+0000","last_peered":"2023-11-28T11:10:59.782724+0000","last_clean":"2023-11-28T11:10:59.782724+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:10:59.782724+0000","last_undegraded":"2023-11-28T11:10:59.782724+0000","last_fullsized":"2023-11-28T11:10:59.782724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784524+0000","last_change":"2023-11-28T11:10:59.801532+0000","last_active":"2023-11-28T11:11:00.784524+0000","last_peered":"2023-11-28T11:11:00.784524+0000","last_clean":"2023-11-28T11:11:00.784524+0000","last_became_active":"2023-11-28T11:10:59.801211+0000","last_became_peered":"2023-11-28T11:10:59.801211+0000","last_unstale":"2023-11-28T11:11:00.784524+0000","last_undegraded":"2023-11-28T11:11:00.784524+0000","last_fullsized":"2023-11-28T11:11:00.784524+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:23.965103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.907001+0000","last_change":"2023-11-28T11:10:59.366434+0000","last_active":"2023-11-28T11:11:02.907001+0000","last_peered":"2023-11-28T11:11:02.907001+0000","last_clean":"2023-11-28T11:11:02.907001+0000","last_became_active":"2023-11-28T11:10:59.365971+0000","last_became_peered":"2023-11-28T11:10:59.365971+0000","last_unstale":"2023-11-28T11:11:02.907001+0000","last_undegraded":"2023-11-28T11:11:02.907001+0000","last_fullsized":"2023-11-28T11:11:02.907001+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:35.736465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":41,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.784222+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:10:59.784222+0000","last_peered":"2023-11-28T11:10:59.784222+0000","last_clean":"2023-11-28T11:10:59.784222+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:10:59.784222+0000","last_undegraded":"2023-11-28T11:10:59.784222+0000","last_fullsized":"2023-11-28T11:10:59.784222+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574881,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10300,"kb_used_data":1204,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361540,"statfs":{"total":96636764160,"available":96626216960,"internally_reserved":0,"allocated":1232896,"data_stored":882092,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.621,"15min":0.64700000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60999999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.626,"15min":0.64400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69499999999999995,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.69599999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.67100000000000004,"15min":0.68999999999999995},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.879,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.38,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71999999999999997,"5min":1.224,"15min":1.224},"last":0.87}]},{"osd":2,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.67100000000000004,"15min":0.69299999999999995},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72399999999999998,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.76000000000000001,"15min":0.78400000000000003},"min":{"1min":0.46400000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.80700000000000005,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.751}]},{"osd":3,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.68100000000000005,"15min":0.70199999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.75600000000000001,"15min":0.77600000000000002},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85699999999999998,"5min":3.294,"15min":3.294},"last":0.67400000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77600000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.36199999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.76100000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.76400000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.96299999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.41099999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.78000000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86899999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.41199999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86499999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.73299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574880,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9148,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362692,"statfs":{"total":96636764160,"available":96627396608,"internally_reserved":0,"allocated":643072,"data_stored":298537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.46000000000000002,"15min":0.45400000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.72099999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.624,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60399999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51200000000000001,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77100000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.73899999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34100000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.753,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.67100000000000004,"15min":0.68500000000000005},"min":{"1min":0.42999999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84099999999999997,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.63800000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77900000000000003,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90100000000000002,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.78900000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54500000000000004,"15min":0.54600000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.80800000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.86799999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.85099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10244,"kb_used_data":1212,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361596,"statfs":{"total":96636764160,"available":96626274304,"internally_reserved":0,"allocated":1241088,"data_stored":888995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.32800000000000001,"15min":0.32700000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.35599999999999998,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54400000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.34599999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38100000000000001,"15min":0.38},"min":{"1min":0.29099999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.58999999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.52100000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.39400000000000002,"15min":0.38800000000000001},"min":{"1min":0.28100000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40799999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69999999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.41899999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53000000000000003,"15min":0.52700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88200000000000001,"5min":1.014,"15min":1.014},"last":0.79100000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.39900000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.79000000000000004,"5min":0.998,"15min":0.998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.442,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79300000000000004,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81000000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.80800000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85199999999999998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.59199999999999997},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.82399999999999995,"5min":1.071,"15min":1.071},"last":0.77200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574881,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9356,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362484,"statfs":{"total":96636764160,"available":96627183616,"internally_reserved":0,"allocated":659456,"data_stored":300651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.315},{"interface":"front","average":{"1min":0.373,"5min":0.37,"15min":0.37},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.40699999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38,"15min":0.38300000000000001},"min":{"1min":0.26800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.435},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39700000000000002,"15min":0.39900000000000002},"min":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57999999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.46500000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36799999999999999,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.48999999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.58999999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.47999999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81599999999999995,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86299999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.64100000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.65100000000000002,"15min":0.62},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":5.4269999999999996,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.62,"15min":0.622},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.79900000000000004,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.58499999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89500000000000002,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.44800000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.88200000000000001,"5min":1.016,"15min":1.016},"last":0.626}]},{"osd":7,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85799999999999998,"5min":1.105,"15min":1.105},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.505,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.71199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574881,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9076,"kb_used_data":620,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362764,"statfs":{"total":96636764160,"available":96627470336,"internally_reserved":0,"allocated":634880,"data_stored":290659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.36699999999999999,"15min":0.374},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.41699999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38100000000000001,"15min":0.38700000000000001},"min":{"1min":0.248,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51900000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.437}]},{"osd":1,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.39000000000000001,"15min":0.39400000000000002},"min":{"1min":0.26100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51000000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60899999999999999,"5min":1.528,"15min":1.528},"last":0.34000000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.39000000000000001,"15min":0.39600000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69599999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.46899999999999997,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.59699999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.47599999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79000000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.49399999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75600000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.85699999999999998,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52400000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.62,"5min":0.66400000000000003,"15min":0.67100000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.61599999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64300000000000002,"15min":0.64800000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.623,"15min":0.628},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.55700000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574880,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9284,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362556,"statfs":{"total":96636764160,"available":96627257344,"internally_reserved":0,"allocated":651264,"data_stored":298593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.83399999999999996,"15min":0.92000000000000004},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.46300000000000002,"5min":11.965,"15min":11.965},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.87,"15min":0.95799999999999996},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55000000000000004,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.255}]},{"osd":1,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.33900000000000002,"15min":0.33700000000000002},"min":{"1min":0.23999999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.314},{"interface":"front","average":{"1min":0.36599999999999999,"5min":1.0940000000000001,"15min":1.2150000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56799999999999995,"5min":16.988,"15min":16.988},"last":0.32600000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.49399999999999999,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75600000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":1.6020000000000001,"15min":1.7969999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64100000000000001,"5min":27.887,"15min":27.887},"last":0.36799999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.68799999999999994,"15min":0.70899999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.75900000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.65800000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.748,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.72499999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.67900000000000005,"15min":0.69199999999999995},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.745,"5min":1.98,"15min":1.98},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.59699999999999998,"5min":1.1220000000000001,"15min":1.21},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78100000000000003,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.1279999999999999,"15min":1.2170000000000001},"min":{"1min":0.44,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73799999999999999,"5min":11.888,"15min":11.888},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.71799999999999997,"15min":0.73699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.76000000000000001,"5min":1.925,"15min":1.925},"last":0.69499999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77300000000000002,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.498,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.79700000000000004,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.56000000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574881,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9340,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362500,"statfs":{"total":96636764160,"available":96627200000,"internally_reserved":0,"allocated":643072,"data_stored":293112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.748,"5min":0.996,"15min":0.996},"last":0.377},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.90300000000000002,"15min":0.96499999999999997},"min":{"1min":0.41099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.70099999999999996,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.48599999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.34999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72499999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70499999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.46899999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67900000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.375,"5min":0.374,"15min":0.374},"max":{"1min":0.749,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.52000000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.72899999999999998,"15min":0.75900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70399999999999996,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.47699999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71599999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.67500000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33100000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71999999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.626},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.35799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79300000000000004,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.57699999999999996}]},{"osd":6,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59999999999999998,"15min":0.60799999999999998},"min":{"1min":0.35899999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76400000000000001,"5min":1.145,"15min":1.145},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.374,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":1.085,"15min":1.085},"last":0.56799999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.40600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.69699999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574881,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9220,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362620,"statfs":{"total":96636764160,"available":96627322880,"internally_reserved":0,"allocated":651264,"data_stored":299173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.36499999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71199999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73099999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40799999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79300000000000004,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.66100000000000003,"15min":0.67600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.71199999999999997,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.51200000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78300000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73699999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.48699999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.67200000000000004,"15min":0.69099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80400000000000005,"5min":1.042,"15min":1.042},"last":0.625}]},{"osd":4,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.629,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.75900000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.69599999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73399999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72699999999999998,"5min":1.036,"15min":1.036},"last":0.64200000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.68200000000000005,"15min":0.69599999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78300000000000003,"5min":1.329,"15min":1.329},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75800000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.67000000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:11:08.853 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:11:08.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.990+0000 7fc933735700 1 Processor -- start 2023-11-28T11:11:08.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.991+0000 7fc933735700 1 -- start start 2023-11-28T11:11:09.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.991+0000 7fc933735700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc92c15c990 0x7fc92c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.991+0000 7fc933735700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92c0cc310 0x7fc92c0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.991+0000 7fc933735700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc92c0ce340 con 0x7fc92c161090 2023-11-28T11:11:09.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.991+0000 7fc933735700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92c0cebe0 con 0x7fc92c15c990 2023-11-28T11:11:09.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.991+0000 7fc933735700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0cccd0 con 0x7fc92c0cc310 2023-11-28T11:11:09.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc930cd0700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc92c161090 0x7fc92c0ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38634/0 (socket says 172.21.15.28:38634) 2023-11-28T11:11:09.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc930cd0700 1 -- 172.21.15.28:0/1765779151 learned_addr learned my addr 172.21.15.28:0/1765779151 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:09.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc931cd2700 1 --2- 172.21.15.28:0/1765779151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92c0cc310 0x7fc92c0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 -- 172.21.15.28:0/1765779151 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487385986 0 0) 0x7fc92c0ce340 con 0x7fc92c161090 2023-11-28T11:11:09.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc9314d1700 1 --2- 172.21.15.28:0/1765779151 >> v2:172.21.15.59:3300/0 conn(0x7fc92c15c990 0x7fc92c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 -- 172.21.15.28:0/1765779151 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9180035b0 con 0x7fc92c161090 2023-11-28T11:11:09.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 -- 172.21.15.28:0/1765779151 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4019128465 0 0) 0x7fc9180035b0 con 0x7fc92c161090 2023-11-28T11:11:09.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 -- 172.21.15.28:0/1765779151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92c0cc310 msgr2=0x7fc92c0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:09.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 --2- 172.21.15.28:0/1765779151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92c0cc310 0x7fc92c0cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 -- 172.21.15.28:0/1765779151 >> v2:172.21.15.59:3300/0 conn(0x7fc92c15c990 msgr2=0x7fc92c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:09.003 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 --2- 172.21.15.28:0/1765779151 >> v2:172.21.15.59:3300/0 conn(0x7fc92c15c990 0x7fc92c0cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 -- 172.21.15.28:0/1765779151 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c0cb3f0 con 0x7fc92c161090 2023-11-28T11:11:09.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 -- 172.21.15.28:0/1765779151 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fc928001670 con 0x7fc92c161090 2023-11-28T11:11:09.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 -- 172.21.15.28:0/1765779151 >> v1:172.21.15.28:6789/0 conn(0x7fc92c161090 legacy=0x7fc92c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 --2- 172.21.15.28:0/1765779151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc918003a00 0x7fc918003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 --2- 172.21.15.28:0/1765779151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92c0cc310 0x7fc918004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.004 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 --2- 172.21.15.28:0/1765779151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc918004d90 0x7fc918005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 -- 172.21.15.28:0/1765779151 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0cebe0 con 0x7fc918004d90 2023-11-28T11:11:09.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 -- 172.21.15.28:0/1765779151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0cccd0 con 0x7fc918003a00 2023-11-28T11:11:09.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc923fff700 1 -- 172.21.15.28:0/1765779151 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91800bae0 con 0x7fc92c0cc310 2023-11-28T11:11:09.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.992+0000 7fc931cd2700 1 --2- 172.21.15.28:0/1765779151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc918004d90 0x7fc918005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.993+0000 7fc9314d1700 1 --2- 172.21.15.28:0/1765779151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc918003a00 0x7fc918003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.993+0000 7fc930cd0700 1 --2- 172.21.15.28:0/1765779151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92c0cc310 0x7fc918004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.993+0000 7fc931cd2700 1 -- 172.21.15.28:0/1765779151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92c0cc310 msgr2=0x7fc918004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:09.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.993+0000 7fc931cd2700 1 --2- 172.21.15.28:0/1765779151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92c0cc310 0x7fc918004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.993+0000 7fc931cd2700 1 -- 172.21.15.28:0/1765779151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc918003a00 msgr2=0x7fc918003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:09.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.993+0000 7fc931cd2700 1 --2- 172.21.15.28:0/1765779151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc918003a00 0x7fc918003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.006 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.993+0000 7fc931cd2700 1 -- 172.21.15.28:0/1765779151 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9180035b0 con 0x7fc918004d90 2023-11-28T11:11:09.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.993+0000 7fc931cd2700 1 --2- 172.21.15.28:0/1765779151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc918004d90 0x7fc918005200 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc924009fc0 tx=0x7fc918005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7134100e5d7a9fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.993+0000 7fc923fff700 1 -- 172.21.15.28:0/1765779151 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92400ec30 con 0x7fc918004d90 2023-11-28T11:11:09.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.993+0000 7fc923fff700 1 -- 172.21.15.28:0/1765779151 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92400f040 con 0x7fc918004d90 2023-11-28T11:11:09.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.993+0000 7fc923fff700 1 -- 172.21.15.28:0/1765779151 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc924014480 con 0x7fc918004d90 2023-11-28T11:11:09.007 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.994+0000 7fc933735700 1 -- 172.21.15.28:0/1765779151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc918004d90 msgr2=0x7fc918005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.994+0000 7fc933735700 1 --2- 172.21.15.28:0/1765779151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc918004d90 0x7fc918005200 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fc924009fc0 tx=0x7fc918005880 comp rx=0 tx=0).stop 2023-11-28T11:11:09.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.994+0000 7fc931cd2700 1 -- 172.21.15.28:0/1765779151 reap_dead start 2023-11-28T11:11:09.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.994+0000 7fc933735700 1 -- 172.21.15.28:0/1765779151 shutdown_connections 2023-11-28T11:11:09.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.994+0000 7fc933735700 1 -- 172.21.15.28:0/1765779151 >> 172.21.15.28:0/1765779151 conn(0x7fc92c0b1650 msgr2=0x7fc92c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:09.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.994+0000 7fc933735700 1 -- 172.21.15.28:0/1765779151 shutdown_connections 2023-11-28T11:11:09.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.994+0000 7fc933735700 1 -- 172.21.15.28:0/1765779151 wait complete. 2023-11-28T11:11:09.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.995+0000 7fc933735700 1 Processor -- start 2023-11-28T11:11:09.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.995+0000 7fc933735700 1 -- start start 2023-11-28T11:11:09.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc933735700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92c0c80d0 0x7fc92c0c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc933735700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc92c0c8ff0 0x7fc92c15cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc933735700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc92c0c84a0 0x7fc92c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc933735700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0ab8c0 con 0x7fc92c0c8ff0 2023-11-28T11:11:09.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc933735700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0abba0 con 0x7fc92c0c84a0 2023-11-28T11:11:09.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc933735700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92c0abe80 con 0x7fc92c0c80d0 2023-11-28T11:11:09.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc930cd0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc92c0c8ff0 0x7fc92c15cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc930cd0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc92c0c8ff0 0x7fc92c15cd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50588/0 (socket says 172.21.15.28:50588) 2023-11-28T11:11:09.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc930cd0700 1 -- 172.21.15.28:0/3068550002 learned_addr learned my addr 172.21.15.28:0/3068550002 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:09.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc931cd2700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc92c0c84a0 0x7fc92c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc9314d1700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92c0c80d0 0x7fc92c0c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc930cd0700 1 -- 172.21.15.28:0/3068550002 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92c0c80d0 msgr2=0x7fc92c0c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc930cd0700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92c0c80d0 0x7fc92c0c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc930cd0700 1 -- 172.21.15.28:0/3068550002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc92c0c84a0 msgr2=0x7fc92c15d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc930cd0700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc92c0c84a0 0x7fc92c15d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc930cd0700 1 -- 172.21.15.28:0/3068550002 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc92c0ce870 con 0x7fc92c0c8ff0 2023-11-28T11:11:09.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc9314d1700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92c0c80d0 0x7fc92c0c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:09.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc931cd2700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc92c0c84a0 0x7fc92c15d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:09.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.996+0000 7fc930cd0700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc92c0c8ff0 0x7fc92c15cd60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc928000c00 tx=0x7fc9280076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.997+0000 7fc921ffb700 1 -- 172.21.15.28:0/3068550002 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92800f870 con 0x7fc92c0c8ff0 2023-11-28T11:11:09.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.997+0000 7fc921ffb700 1 -- 172.21.15.28:0/3068550002 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc92800f9d0 con 0x7fc92c0c8ff0 2023-11-28T11:11:09.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.997+0000 7fc933735700 1 -- 172.21.15.28:0/3068550002 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92c0ce590 con 0x7fc92c0c8ff0 2023-11-28T11:11:09.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.997+0000 7fc921ffb700 1 -- 172.21.15.28:0/3068550002 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc92800fd80 con 0x7fc92c0c8ff0 2023-11-28T11:11:09.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.997+0000 7fc933735700 1 -- 172.21.15.28:0/3068550002 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc92c0cb080 con 0x7fc92c0c8ff0 2023-11-28T11:11:09.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.998+0000 7fc933735700 1 -- 172.21.15.28:0/3068550002 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92c0675a0 con 0x7fc92c0c8ff0 2023-11-28T11:11:09.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.998+0000 7fc921ffb700 1 -- 172.21.15.28:0/3068550002 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc92801d460 con 0x7fc92c0c8ff0 2023-11-28T11:11:09.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.998+0000 7fc921ffb700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9100682f0 0x7fc91006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.999+0000 7fc921ffb700 1 -- 172.21.15.28:0/3068550002 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc928085fe0 con 0x7fc92c0c8ff0 2023-11-28T11:11:09.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.999+0000 7fc9314d1700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9100682f0 0x7fc91006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:08.999+0000 7fc9314d1700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9100682f0 0x7fc91006a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009c60 tx=0x7fc91c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.003+0000 7fc921ffb700 1 -- 172.21.15.28:0/3068550002 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc9280526a0 con 0x7fc92c0c8ff0 2023-11-28T11:11:09.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.193+0000 7fc933735700 1 -- 172.21.15.28:0/3068550002 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc92c0cf710 con 0x7fc9100682f0 2023-11-28T11:11:09.202 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.201+0000 7fc921ffb700 1 -- 172.21.15.28:0/3068550002 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311428 (secure 0 0 0) 0x7fc92c0cf710 con 0x7fc9100682f0 2023-11-28T11:11:09.206 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.204+0000 7fc933735700 1 -- 172.21.15.28:0/3068550002 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9100682f0 msgr2=0x7fc91006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.204+0000 7fc933735700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9100682f0 0x7fc91006a7a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc91c009c60 tx=0x7fc91c008040 comp rx=0 tx=0).stop 2023-11-28T11:11:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.204+0000 7fc933735700 1 -- 172.21.15.28:0/3068550002 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc92c0c8ff0 msgr2=0x7fc92c15cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.204+0000 7fc933735700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc92c0c8ff0 0x7fc92c15cd60 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fc928000c00 tx=0x7fc9280076a0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.204+0000 7fc933735700 1 -- 172.21.15.28:0/3068550002 shutdown_connections 2023-11-28T11:11:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.204+0000 7fc933735700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9100682f0 0x7fc91006a7a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.204+0000 7fc933735700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc92c0c84a0 0x7fc92c15d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.204+0000 7fc933735700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc92c0c80d0 0x7fc92c0c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.204+0000 7fc933735700 1 --2- 172.21.15.28:0/3068550002 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc92c0c8ff0 0x7fc92c15cd60 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.204+0000 7fc933735700 1 -- 172.21.15.28:0/3068550002 >> 172.21.15.28:0/3068550002 conn(0x7fc92c0b1650 msgr2=0x7fc92c15bef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:09.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.204+0000 7fc933735700 1 -- 172.21.15.28:0/3068550002 shutdown_connections 2023-11-28T11:11:09.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.204+0000 7fc933735700 1 -- 172.21.15.28:0/3068550002 wait complete. 2023-11-28T11:11:09.210 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:11:09.218 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2023-11-28T11:11:09.193922+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":76004,"kb_used_data":6244,"kb_used_omap":0,"kb_used_meta":69696,"kb_avail":754898716,"statfs":{"total":773094113280,"available":773016285184,"internally_reserved":0,"allocated":6393856,"data_stored":3572731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":71368704},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.420104"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905490+0000","last_change":"2023-11-28T11:10:59.788613+0000","last_active":"2023-11-28T11:11:02.905490+0000","last_peered":"2023-11-28T11:11:02.905490+0000","last_clean":"2023-11-28T11:11:02.905490+0000","last_became_active":"2023-11-28T11:10:59.788354+0000","last_became_peered":"2023-11-28T11:10:59.788354+0000","last_unstale":"2023-11-28T11:11:02.905490+0000","last_undegraded":"2023-11-28T11:11:02.905490+0000","last_fullsized":"2023-11-28T11:11:02.905490+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:06:53.730776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905951+0000","last_change":"2023-11-28T11:10:59.802140+0000","last_active":"2023-11-28T11:11:02.905951+0000","last_peered":"2023-11-28T11:11:02.905951+0000","last_clean":"2023-11-28T11:11:02.905951+0000","last_became_active":"2023-11-28T11:10:59.801891+0000","last_became_peered":"2023-11-28T11:10:59.801891+0000","last_unstale":"2023-11-28T11:11:02.905951+0000","last_undegraded":"2023-11-28T11:11:02.905951+0000","last_fullsized":"2023-11-28T11:11:02.905951+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:51:59.306560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906388+0000","last_change":"2023-11-28T11:10:59.799426+0000","last_active":"2023-11-28T11:11:02.906388+0000","last_peered":"2023-11-28T11:11:02.906388+0000","last_clean":"2023-11-28T11:11:02.906388+0000","last_became_active":"2023-11-28T11:10:59.799085+0000","last_became_peered":"2023-11-28T11:10:59.799085+0000","last_unstale":"2023-11-28T11:11:02.906388+0000","last_undegraded":"2023-11-28T11:11:02.906388+0000","last_fullsized":"2023-11-28T11:11:02.906388+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:41:04.478315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784336+0000","last_change":"2023-11-28T11:10:59.791480+0000","last_active":"2023-11-28T11:11:00.784336+0000","last_peered":"2023-11-28T11:11:00.784336+0000","last_clean":"2023-11-28T11:11:00.784336+0000","last_became_active":"2023-11-28T11:10:59.791142+0000","last_became_peered":"2023-11-28T11:10:59.791142+0000","last_unstale":"2023-11-28T11:11:00.784336+0000","last_undegraded":"2023-11-28T11:11:00.784336+0000","last_fullsized":"2023-11-28T11:11:00.784336+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:41:25.021451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905306+0000","last_change":"2023-11-28T11:10:59.789548+0000","last_active":"2023-11-28T11:11:02.905306+0000","last_peered":"2023-11-28T11:11:02.905306+0000","last_clean":"2023-11-28T11:11:02.905306+0000","last_became_active":"2023-11-28T11:10:59.789280+0000","last_became_peered":"2023-11-28T11:10:59.789280+0000","last_unstale":"2023-11-28T11:11:02.905306+0000","last_undegraded":"2023-11-28T11:11:02.905306+0000","last_fullsized":"2023-11-28T11:11:02.905306+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:50:54.142216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906481+0000","last_change":"2023-11-28T11:10:59.798422+0000","last_active":"2023-11-28T11:11:02.906481+0000","last_peered":"2023-11-28T11:11:02.906481+0000","last_clean":"2023-11-28T11:11:02.906481+0000","last_became_active":"2023-11-28T11:10:59.798131+0000","last_became_peered":"2023-11-28T11:10:59.798131+0000","last_unstale":"2023-11-28T11:11:02.906481+0000","last_undegraded":"2023-11-28T11:11:02.906481+0000","last_fullsized":"2023-11-28T11:11:02.906481+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:26.055549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905293+0000","last_change":"2023-11-28T11:10:59.788420+0000","last_active":"2023-11-28T11:11:02.905293+0000","last_peered":"2023-11-28T11:11:02.905293+0000","last_clean":"2023-11-28T11:11:02.905293+0000","last_became_active":"2023-11-28T11:10:59.787673+0000","last_became_peered":"2023-11-28T11:10:59.787673+0000","last_unstale":"2023-11-28T11:11:02.905293+0000","last_undegraded":"2023-11-28T11:11:02.905293+0000","last_fullsized":"2023-11-28T11:11:02.905293+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:35:39.058059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905438+0000","last_change":"2023-11-28T11:10:59.782137+0000","last_active":"2023-11-28T11:11:02.905438+0000","last_peered":"2023-11-28T11:11:02.905438+0000","last_clean":"2023-11-28T11:11:02.905438+0000","last_became_active":"2023-11-28T11:10:59.781878+0000","last_became_peered":"2023-11-28T11:10:59.781878+0000","last_unstale":"2023-11-28T11:11:02.905438+0000","last_undegraded":"2023-11-28T11:11:02.905438+0000","last_fullsized":"2023-11-28T11:11:02.905438+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:39.866743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906413+0000","last_change":"2023-11-28T11:10:59.793042+0000","last_active":"2023-11-28T11:11:02.906413+0000","last_peered":"2023-11-28T11:11:02.906413+0000","last_clean":"2023-11-28T11:11:02.906413+0000","last_became_active":"2023-11-28T11:10:59.792726+0000","last_became_peered":"2023-11-28T11:10:59.792726+0000","last_unstale":"2023-11-28T11:11:02.906413+0000","last_undegraded":"2023-11-28T11:11:02.906413+0000","last_fullsized":"2023-11-28T11:11:02.906413+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:05:35.660459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780564+0000","last_change":"2023-11-28T11:10:59.800651+0000","last_active":"2023-11-28T11:11:00.780564+0000","last_peered":"2023-11-28T11:11:00.780564+0000","last_clean":"2023-11-28T11:11:00.780564+0000","last_became_active":"2023-11-28T11:10:59.800389+0000","last_became_peered":"2023-11-28T11:10:59.800389+0000","last_unstale":"2023-11-28T11:11:00.780564+0000","last_undegraded":"2023-11-28T11:11:00.780564+0000","last_fullsized":"2023-11-28T11:11:00.780564+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:46.598260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780379+0000","last_change":"2023-11-28T11:10:59.796275+0000","last_active":"2023-11-28T11:11:00.780379+0000","last_peered":"2023-11-28T11:11:00.780379+0000","last_clean":"2023-11-28T11:11:00.780379+0000","last_became_active":"2023-11-28T11:10:59.795017+0000","last_became_peered":"2023-11-28T11:10:59.795017+0000","last_unstale":"2023-11-28T11:11:00.780379+0000","last_undegraded":"2023-11-28T11:11:00.780379+0000","last_fullsized":"2023-11-28T11:11:00.780379+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:09:13.076060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784596+0000","last_change":"2023-11-28T11:10:59.803912+0000","last_active":"2023-11-28T11:11:00.784596+0000","last_peered":"2023-11-28T11:11:00.784596+0000","last_clean":"2023-11-28T11:11:00.784596+0000","last_became_active":"2023-11-28T11:10:59.803619+0000","last_became_peered":"2023-11-28T11:10:59.803619+0000","last_unstale":"2023-11-28T11:11:00.784596+0000","last_undegraded":"2023-11-28T11:11:00.784596+0000","last_fullsized":"2023-11-28T11:11:00.784596+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:03.916291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.292124+0000","last_change":"2023-11-28T11:10:59.800886+0000","last_active":"2023-11-28T11:11:02.292124+0000","last_peered":"2023-11-28T11:11:02.292124+0000","last_clean":"2023-11-28T11:11:02.292124+0000","last_became_active":"2023-11-28T11:10:59.800573+0000","last_became_peered":"2023-11-28T11:10:59.800573+0000","last_unstale":"2023-11-28T11:11:02.292124+0000","last_undegraded":"2023-11-28T11:11:02.292124+0000","last_fullsized":"2023-11-28T11:11:02.292124+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:06:44.618142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905494+0000","last_change":"2023-11-28T11:10:59.788195+0000","last_active":"2023-11-28T11:11:02.905494+0000","last_peered":"2023-11-28T11:11:02.905494+0000","last_clean":"2023-11-28T11:11:02.905494+0000","last_became_active":"2023-11-28T11:10:59.787925+0000","last_became_peered":"2023-11-28T11:10:59.787925+0000","last_unstale":"2023-11-28T11:11:02.905494+0000","last_undegraded":"2023-11-28T11:11:02.905494+0000","last_fullsized":"2023-11-28T11:11:02.905494+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:28.106265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905761+0000","last_change":"2023-11-28T11:10:59.801017+0000","last_active":"2023-11-28T11:11:02.905761+0000","last_peered":"2023-11-28T11:11:02.905761+0000","last_clean":"2023-11-28T11:11:02.905761+0000","last_became_active":"2023-11-28T11:10:59.800727+0000","last_became_peered":"2023-11-28T11:10:59.800727+0000","last_unstale":"2023-11-28T11:11:02.905761+0000","last_undegraded":"2023-11-28T11:11:02.905761+0000","last_fullsized":"2023-11-28T11:11:02.905761+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:38:05.209860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906104+0000","last_change":"2023-11-28T11:10:59.796635+0000","last_active":"2023-11-28T11:11:02.906104+0000","last_peered":"2023-11-28T11:11:02.906104+0000","last_clean":"2023-11-28T11:11:02.906104+0000","last_became_active":"2023-11-28T11:10:59.792803+0000","last_became_peered":"2023-11-28T11:10:59.792803+0000","last_unstale":"2023-11-28T11:11:02.906104+0000","last_undegraded":"2023-11-28T11:11:02.906104+0000","last_fullsized":"2023-11-28T11:11:02.906104+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:51:59.306560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905931+0000","last_change":"2023-11-28T11:10:59.791572+0000","last_active":"2023-11-28T11:11:02.905931+0000","last_peered":"2023-11-28T11:11:02.905931+0000","last_clean":"2023-11-28T11:11:02.905931+0000","last_became_active":"2023-11-28T11:10:59.789432+0000","last_became_peered":"2023-11-28T11:10:59.789432+0000","last_unstale":"2023-11-28T11:11:02.905931+0000","last_undegraded":"2023-11-28T11:11:02.905931+0000","last_fullsized":"2023-11-28T11:11:02.905931+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:40:14.482363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906317+0000","last_change":"2023-11-28T11:10:59.799877+0000","last_active":"2023-11-28T11:11:02.906317+0000","last_peered":"2023-11-28T11:11:02.906317+0000","last_clean":"2023-11-28T11:11:02.906317+0000","last_became_active":"2023-11-28T11:10:59.799607+0000","last_became_peered":"2023-11-28T11:10:59.799607+0000","last_unstale":"2023-11-28T11:11:02.906317+0000","last_undegraded":"2023-11-28T11:11:02.906317+0000","last_fullsized":"2023-11-28T11:11:02.906317+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:09:50.383846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905696+0000","last_change":"2023-11-28T11:10:59.788241+0000","last_active":"2023-11-28T11:11:02.905696+0000","last_peered":"2023-11-28T11:11:02.905696+0000","last_clean":"2023-11-28T11:11:02.905696+0000","last_became_active":"2023-11-28T11:10:59.787995+0000","last_became_peered":"2023-11-28T11:10:59.787995+0000","last_unstale":"2023-11-28T11:11:02.905696+0000","last_undegraded":"2023-11-28T11:11:02.905696+0000","last_fullsized":"2023-11-28T11:11:02.905696+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:45.705252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906093+0000","last_change":"2023-11-28T11:10:59.794292+0000","last_active":"2023-11-28T11:11:02.906093+0000","last_peered":"2023-11-28T11:11:02.906093+0000","last_clean":"2023-11-28T11:11:02.906093+0000","last_became_active":"2023-11-28T11:10:59.793249+0000","last_became_peered":"2023-11-28T11:10:59.793249+0000","last_unstale":"2023-11-28T11:11:02.906093+0000","last_undegraded":"2023-11-28T11:11:02.906093+0000","last_fullsized":"2023-11-28T11:11:02.906093+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:05:14.963396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905666+0000","last_change":"2023-11-28T11:10:59.800290+0000","last_active":"2023-11-28T11:11:02.905666+0000","last_peered":"2023-11-28T11:11:02.905666+0000","last_clean":"2023-11-28T11:11:02.905666+0000","last_became_active":"2023-11-28T11:10:59.798617+0000","last_became_peered":"2023-11-28T11:10:59.798617+0000","last_unstale":"2023-11-28T11:11:02.905666+0000","last_undegraded":"2023-11-28T11:11:02.905666+0000","last_fullsized":"2023-11-28T11:11:02.905666+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:47:42.462499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905559+0000","last_change":"2023-11-28T11:10:59.787921+0000","last_active":"2023-11-28T11:11:02.905559+0000","last_peered":"2023-11-28T11:11:02.905559+0000","last_clean":"2023-11-28T11:11:02.905559+0000","last_became_active":"2023-11-28T11:10:59.787453+0000","last_became_peered":"2023-11-28T11:10:59.787453+0000","last_unstale":"2023-11-28T11:11:02.905559+0000","last_undegraded":"2023-11-28T11:11:02.905559+0000","last_fullsized":"2023-11-28T11:11:02.905559+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:56.378053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906247+0000","last_change":"2023-11-28T11:10:59.803707+0000","last_active":"2023-11-28T11:11:02.906247+0000","last_peered":"2023-11-28T11:11:02.906247+0000","last_clean":"2023-11-28T11:11:02.906247+0000","last_became_active":"2023-11-28T11:10:59.803411+0000","last_became_peered":"2023-11-28T11:10:59.803411+0000","last_unstale":"2023-11-28T11:11:02.906247+0000","last_undegraded":"2023-11-28T11:11:02.906247+0000","last_fullsized":"2023-11-28T11:11:02.906247+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:46.079984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.292020+0000","last_change":"2023-11-28T11:10:59.801163+0000","last_active":"2023-11-28T11:11:02.292020+0000","last_peered":"2023-11-28T11:11:02.292020+0000","last_clean":"2023-11-28T11:11:02.292020+0000","last_became_active":"2023-11-28T11:10:59.800902+0000","last_became_peered":"2023-11-28T11:10:59.800902+0000","last_unstale":"2023-11-28T11:11:02.292020+0000","last_undegraded":"2023-11-28T11:11:02.292020+0000","last_fullsized":"2023-11-28T11:11:02.292020+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:52:57.405614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905689+0000","last_change":"2023-11-28T11:10:59.802820+0000","last_active":"2023-11-28T11:11:02.905689+0000","last_peered":"2023-11-28T11:11:02.905689+0000","last_clean":"2023-11-28T11:11:02.905689+0000","last_became_active":"2023-11-28T11:10:59.802559+0000","last_became_peered":"2023-11-28T11:10:59.802559+0000","last_unstale":"2023-11-28T11:11:02.905689+0000","last_undegraded":"2023-11-28T11:11:02.905689+0000","last_fullsized":"2023-11-28T11:11:02.905689+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:19:08.724798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784461+0000","last_change":"2023-11-28T11:10:59.791720+0000","last_active":"2023-11-28T11:11:00.784461+0000","last_peered":"2023-11-28T11:11:00.784461+0000","last_clean":"2023-11-28T11:11:00.784461+0000","last_became_active":"2023-11-28T11:10:59.791413+0000","last_became_peered":"2023-11-28T11:10:59.791413+0000","last_unstale":"2023-11-28T11:11:00.784461+0000","last_undegraded":"2023-11-28T11:11:00.784461+0000","last_fullsized":"2023-11-28T11:11:00.784461+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:41:04.478315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906745+0000","last_change":"2023-11-28T11:10:59.798723+0000","last_active":"2023-11-28T11:11:02.906745+0000","last_peered":"2023-11-28T11:11:02.906745+0000","last_clean":"2023-11-28T11:11:02.906745+0000","last_became_active":"2023-11-28T11:10:59.796726+0000","last_became_peered":"2023-11-28T11:10:59.796726+0000","last_unstale":"2023-11-28T11:11:02.906745+0000","last_undegraded":"2023-11-28T11:11:02.906745+0000","last_fullsized":"2023-11-28T11:11:02.906745+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:58.186048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906436+0000","last_change":"2023-11-28T11:10:59.792848+0000","last_active":"2023-11-28T11:11:02.906436+0000","last_peered":"2023-11-28T11:11:02.906436+0000","last_clean":"2023-11-28T11:11:02.906436+0000","last_became_active":"2023-11-28T11:10:59.792526+0000","last_became_peered":"2023-11-28T11:10:59.792526+0000","last_unstale":"2023-11-28T11:11:02.906436+0000","last_undegraded":"2023-11-28T11:11:02.906436+0000","last_fullsized":"2023-11-28T11:11:02.906436+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:56:00.455574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905887+0000","last_change":"2023-11-28T11:10:59.788209+0000","last_active":"2023-11-28T11:11:02.905887+0000","last_peered":"2023-11-28T11:11:02.905887+0000","last_clean":"2023-11-28T11:11:02.905887+0000","last_became_active":"2023-11-28T11:10:59.787980+0000","last_became_peered":"2023-11-28T11:10:59.787980+0000","last_unstale":"2023-11-28T11:11:02.905887+0000","last_undegraded":"2023-11-28T11:11:02.905887+0000","last_fullsized":"2023-11-28T11:11:02.905887+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:35:10.703217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905973+0000","last_change":"2023-11-28T11:10:59.795807+0000","last_active":"2023-11-28T11:11:02.905973+0000","last_peered":"2023-11-28T11:11:02.905973+0000","last_clean":"2023-11-28T11:11:02.905973+0000","last_became_active":"2023-11-28T11:10:59.795521+0000","last_became_peered":"2023-11-28T11:10:59.795521+0000","last_unstale":"2023-11-28T11:11:02.905973+0000","last_undegraded":"2023-11-28T11:11:02.905973+0000","last_fullsized":"2023-11-28T11:11:02.905973+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:18.802241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905259+0000","last_change":"2023-11-28T11:10:59.802476+0000","last_active":"2023-11-28T11:11:02.905259+0000","last_peered":"2023-11-28T11:11:02.905259+0000","last_clean":"2023-11-28T11:11:02.905259+0000","last_became_active":"2023-11-28T11:10:59.802192+0000","last_became_peered":"2023-11-28T11:10:59.802192+0000","last_unstale":"2023-11-28T11:11:02.905259+0000","last_undegraded":"2023-11-28T11:11:02.905259+0000","last_fullsized":"2023-11-28T11:11:02.905259+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:59:10.861518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906014+0000","last_change":"2023-11-28T11:10:59.796452+0000","last_active":"2023-11-28T11:11:02.906014+0000","last_peered":"2023-11-28T11:11:02.906014+0000","last_clean":"2023-11-28T11:11:02.906014+0000","last_became_active":"2023-11-28T11:10:59.792787+0000","last_became_peered":"2023-11-28T11:10:59.792787+0000","last_unstale":"2023-11-28T11:11:02.906014+0000","last_undegraded":"2023-11-28T11:11:02.906014+0000","last_fullsized":"2023-11-28T11:11:02.906014+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:47:28.106265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906916+0000","last_change":"2023-11-28T11:10:59.796748+0000","last_active":"2023-11-28T11:11:02.906916+0000","last_peered":"2023-11-28T11:11:02.906916+0000","last_clean":"2023-11-28T11:11:02.906916+0000","last_became_active":"2023-11-28T11:10:59.796555+0000","last_became_peered":"2023-11-28T11:10:59.796555+0000","last_unstale":"2023-11-28T11:11:02.906916+0000","last_undegraded":"2023-11-28T11:11:02.906916+0000","last_fullsized":"2023-11-28T11:11:02.906916+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:36:15.362997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"53'4","reported_seq":15,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.907025+0000","last_change":"2023-11-28T11:10:59.365054+0000","last_active":"2023-11-28T11:11:02.907025+0000","last_peered":"2023-11-28T11:11:02.907025+0000","last_clean":"2023-11-28T11:11:02.907025+0000","last_became_active":"2023-11-28T11:10:59.364212+0000","last_became_peered":"2023-11-28T11:10:59.364212+0000","last_unstale":"2023-11-28T11:11:02.907025+0000","last_undegraded":"2023-11-28T11:11:02.907025+0000","last_fullsized":"2023-11-28T11:11:02.907025+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:01:08.672135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780209+0000","last_change":"2023-11-28T11:10:59.800697+0000","last_active":"2023-11-28T11:11:00.780209+0000","last_peered":"2023-11-28T11:11:00.780209+0000","last_clean":"2023-11-28T11:11:00.780209+0000","last_became_active":"2023-11-28T11:10:59.800376+0000","last_became_peered":"2023-11-28T11:10:59.800376+0000","last_unstale":"2023-11-28T11:11:00.780209+0000","last_undegraded":"2023-11-28T11:11:00.780209+0000","last_fullsized":"2023-11-28T11:11:00.780209+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:46.415640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906244+0000","last_change":"2023-11-28T11:10:59.371087+0000","last_active":"2023-11-28T11:11:02.906244+0000","last_peered":"2023-11-28T11:11:02.906244+0000","last_clean":"2023-11-28T11:11:02.906244+0000","last_became_active":"2023-11-28T11:10:59.370756+0000","last_became_peered":"2023-11-28T11:10:59.370756+0000","last_unstale":"2023-11-28T11:11:02.906244+0000","last_undegraded":"2023-11-28T11:11:02.906244+0000","last_fullsized":"2023-11-28T11:11:02.906244+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:41:03.472842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906659+0000","last_change":"2023-11-28T11:10:59.803851+0000","last_active":"2023-11-28T11:11:02.906659+0000","last_peered":"2023-11-28T11:11:02.906659+0000","last_clean":"2023-11-28T11:11:02.906659+0000","last_became_active":"2023-11-28T11:10:59.803504+0000","last_became_peered":"2023-11-28T11:10:59.803504+0000","last_unstale":"2023-11-28T11:11:02.906659+0000","last_undegraded":"2023-11-28T11:11:02.906659+0000","last_fullsized":"2023-11-28T11:11:02.906659+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:00.484044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906918+0000","last_change":"2023-11-28T11:10:59.365025+0000","last_active":"2023-11-28T11:11:02.906918+0000","last_peered":"2023-11-28T11:11:02.906918+0000","last_clean":"2023-11-28T11:11:02.906918+0000","last_became_active":"2023-11-28T11:10:59.364746+0000","last_became_peered":"2023-11-28T11:10:59.364746+0000","last_unstale":"2023-11-28T11:11:02.906918+0000","last_undegraded":"2023-11-28T11:11:02.906918+0000","last_fullsized":"2023-11-28T11:11:02.906918+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:17.796768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906772+0000","last_change":"2023-11-28T11:10:59.798706+0000","last_active":"2023-11-28T11:11:02.906772+0000","last_peered":"2023-11-28T11:11:02.906772+0000","last_clean":"2023-11-28T11:11:02.906772+0000","last_became_active":"2023-11-28T11:10:59.798287+0000","last_became_peered":"2023-11-28T11:10:59.798287+0000","last_unstale":"2023-11-28T11:11:02.906772+0000","last_undegraded":"2023-11-28T11:11:02.906772+0000","last_fullsized":"2023-11-28T11:11:02.906772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:48:56.378053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784575+0000","last_change":"2023-11-28T11:10:59.366716+0000","last_active":"2023-11-28T11:11:00.784575+0000","last_peered":"2023-11-28T11:11:00.784575+0000","last_clean":"2023-11-28T11:11:00.784575+0000","last_became_active":"2023-11-28T11:10:59.366426+0000","last_became_peered":"2023-11-28T11:10:59.366426+0000","last_unstale":"2023-11-28T11:11:00.784575+0000","last_undegraded":"2023-11-28T11:11:00.784575+0000","last_fullsized":"2023-11-28T11:11:00.784575+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:08:45.410168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905821+0000","last_change":"2023-11-28T11:10:59.788013+0000","last_active":"2023-11-28T11:11:02.905821+0000","last_peered":"2023-11-28T11:11:02.905821+0000","last_clean":"2023-11-28T11:11:02.905821+0000","last_became_active":"2023-11-28T11:10:59.787380+0000","last_became_peered":"2023-11-28T11:10:59.787380+0000","last_unstale":"2023-11-28T11:11:02.905821+0000","last_undegraded":"2023-11-28T11:11:02.905821+0000","last_fullsized":"2023-11-28T11:11:02.905821+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:10:58.186048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905452+0000","last_change":"2023-11-28T11:10:59.366526+0000","last_active":"2023-11-28T11:11:02.905452+0000","last_peered":"2023-11-28T11:11:02.905452+0000","last_clean":"2023-11-28T11:11:02.905452+0000","last_became_active":"2023-11-28T11:10:59.366249+0000","last_became_peered":"2023-11-28T11:10:59.366249+0000","last_unstale":"2023-11-28T11:11:02.905452+0000","last_undegraded":"2023-11-28T11:11:02.905452+0000","last_fullsized":"2023-11-28T11:11:02.905452+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:01:08.672135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906005+0000","last_change":"2023-11-28T11:10:59.794676+0000","last_active":"2023-11-28T11:11:02.906005+0000","last_peered":"2023-11-28T11:11:02.906005+0000","last_clean":"2023-11-28T11:11:02.906005+0000","last_became_active":"2023-11-28T11:10:59.793185+0000","last_became_peered":"2023-11-28T11:10:59.793185+0000","last_unstale":"2023-11-28T11:11:02.906005+0000","last_undegraded":"2023-11-28T11:11:02.906005+0000","last_fullsized":"2023-11-28T11:11:02.906005+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:06:17.744141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906517+0000","last_change":"2023-11-28T11:10:59.368381+0000","last_active":"2023-11-28T11:11:02.906517+0000","last_peered":"2023-11-28T11:11:02.906517+0000","last_clean":"2023-11-28T11:11:02.906517+0000","last_became_active":"2023-11-28T11:10:59.367921+0000","last_became_peered":"2023-11-28T11:10:59.367921+0000","last_unstale":"2023-11-28T11:11:02.906517+0000","last_undegraded":"2023-11-28T11:11:02.906517+0000","last_fullsized":"2023-11-28T11:11:02.906517+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:22.959630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.292011+0000","last_change":"2023-11-28T11:10:59.795964+0000","last_active":"2023-11-28T11:11:02.292011+0000","last_peered":"2023-11-28T11:11:02.292011+0000","last_clean":"2023-11-28T11:11:02.292011+0000","last_became_active":"2023-11-28T11:10:59.795666+0000","last_became_peered":"2023-11-28T11:10:59.795666+0000","last_unstale":"2023-11-28T11:11:02.292011+0000","last_undegraded":"2023-11-28T11:11:02.292011+0000","last_fullsized":"2023-11-28T11:11:02.292011+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:00:27.962233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784097+0000","last_change":"2023-11-28T11:10:59.358836+0000","last_active":"2023-11-28T11:11:00.784097+0000","last_peered":"2023-11-28T11:11:00.784097+0000","last_clean":"2023-11-28T11:11:00.784097+0000","last_became_active":"2023-11-28T11:10:59.358559+0000","last_became_peered":"2023-11-28T11:10:59.358559+0000","last_unstale":"2023-11-28T11:11:00.784097+0000","last_undegraded":"2023-11-28T11:11:00.784097+0000","last_fullsized":"2023-11-28T11:11:00.784097+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:28:43.457063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905683+0000","last_change":"2023-11-28T11:10:59.792069+0000","last_active":"2023-11-28T11:11:02.905683+0000","last_peered":"2023-11-28T11:11:02.905683+0000","last_clean":"2023-11-28T11:11:02.905683+0000","last_became_active":"2023-11-28T11:10:59.791286+0000","last_became_peered":"2023-11-28T11:10:59.791286+0000","last_unstale":"2023-11-28T11:11:02.905683+0000","last_undegraded":"2023-11-28T11:11:02.905683+0000","last_fullsized":"2023-11-28T11:11:02.905683+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:16:07.408042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905306+0000","last_change":"2023-11-28T11:10:59.362945+0000","last_active":"2023-11-28T11:11:02.905306+0000","last_peered":"2023-11-28T11:11:02.905306+0000","last_clean":"2023-11-28T11:11:02.905306+0000","last_became_active":"2023-11-28T11:10:59.362418+0000","last_became_peered":"2023-11-28T11:10:59.362418+0000","last_unstale":"2023-11-28T11:11:02.905306+0000","last_undegraded":"2023-11-28T11:11:02.905306+0000","last_fullsized":"2023-11-28T11:11:02.905306+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:12.320362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906363+0000","last_change":"2023-11-28T11:10:59.801036+0000","last_active":"2023-11-28T11:11:02.906363+0000","last_peered":"2023-11-28T11:11:02.906363+0000","last_clean":"2023-11-28T11:11:02.906363+0000","last_became_active":"2023-11-28T11:10:59.800738+0000","last_became_peered":"2023-11-28T11:10:59.800738+0000","last_unstale":"2023-11-28T11:11:02.906363+0000","last_undegraded":"2023-11-28T11:11:02.906363+0000","last_fullsized":"2023-11-28T11:11:02.906363+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:53:40.993953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906283+0000","last_change":"2023-11-28T11:10:59.370351+0000","last_active":"2023-11-28T11:11:02.906283+0000","last_peered":"2023-11-28T11:11:02.906283+0000","last_clean":"2023-11-28T11:11:02.906283+0000","last_became_active":"2023-11-28T11:10:59.370020+0000","last_became_peered":"2023-11-28T11:10:59.370020+0000","last_unstale":"2023-11-28T11:11:02.906283+0000","last_undegraded":"2023-11-28T11:11:02.906283+0000","last_fullsized":"2023-11-28T11:11:02.906283+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:21.232441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906148+0000","last_change":"2023-11-28T11:10:59.801568+0000","last_active":"2023-11-28T11:11:02.906148+0000","last_peered":"2023-11-28T11:11:02.906148+0000","last_clean":"2023-11-28T11:11:02.906148+0000","last_became_active":"2023-11-28T11:10:59.801229+0000","last_became_peered":"2023-11-28T11:10:59.801229+0000","last_unstale":"2023-11-28T11:11:02.906148+0000","last_undegraded":"2023-11-28T11:11:02.906148+0000","last_fullsized":"2023-11-28T11:11:02.906148+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:12:03.916291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906301+0000","last_change":"2023-11-28T11:10:59.365589+0000","last_active":"2023-11-28T11:11:02.906301+0000","last_peered":"2023-11-28T11:11:02.906301+0000","last_clean":"2023-11-28T11:11:02.906301+0000","last_became_active":"2023-11-28T11:10:59.365321+0000","last_became_peered":"2023-11-28T11:10:59.365321+0000","last_unstale":"2023-11-28T11:11:02.906301+0000","last_undegraded":"2023-11-28T11:11:02.906301+0000","last_fullsized":"2023-11-28T11:11:02.906301+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:17.796768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780042+0000","last_change":"2023-11-28T11:10:59.801864+0000","last_active":"2023-11-28T11:11:00.780042+0000","last_peered":"2023-11-28T11:11:00.780042+0000","last_clean":"2023-11-28T11:11:00.780042+0000","last_became_active":"2023-11-28T11:10:59.801546+0000","last_became_peered":"2023-11-28T11:10:59.801546+0000","last_unstale":"2023-11-28T11:11:00.780042+0000","last_undegraded":"2023-11-28T11:11:00.780042+0000","last_fullsized":"2023-11-28T11:11:00.780042+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:49.736618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"53'3","reported_seq":14,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906036+0000","last_change":"2023-11-28T11:10:59.370539+0000","last_active":"2023-11-28T11:11:02.906036+0000","last_peered":"2023-11-28T11:11:02.906036+0000","last_clean":"2023-11-28T11:11:02.906036+0000","last_became_active":"2023-11-28T11:10:59.370111+0000","last_became_peered":"2023-11-28T11:10:59.370111+0000","last_unstale":"2023-11-28T11:11:02.906036+0000","last_undegraded":"2023-11-28T11:11:02.906036+0000","last_fullsized":"2023-11-28T11:11:02.906036+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:12:48.731146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905772+0000","last_change":"2023-11-28T11:10:59.805481+0000","last_active":"2023-11-28T11:11:02.905772+0000","last_peered":"2023-11-28T11:11:02.905772+0000","last_clean":"2023-11-28T11:11:02.905772+0000","last_became_active":"2023-11-28T11:10:59.805201+0000","last_became_peered":"2023-11-28T11:10:59.805201+0000","last_unstale":"2023-11-28T11:11:02.905772+0000","last_undegraded":"2023-11-28T11:11:02.905772+0000","last_fullsized":"2023-11-28T11:11:02.905772+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:53:40.993953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"53'6","reported_seq":17,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905762+0000","last_change":"2023-11-28T11:10:59.365182+0000","last_active":"2023-11-28T11:11:02.905762+0000","last_peered":"2023-11-28T11:11:02.905762+0000","last_clean":"2023-11-28T11:11:02.905762+0000","last_became_active":"2023-11-28T11:10:59.364858+0000","last_became_peered":"2023-11-28T11:10:59.364858+0000","last_unstale":"2023-11-28T11:11:02.905762+0000","last_undegraded":"2023-11-28T11:11:02.905762+0000","last_fullsized":"2023-11-28T11:11:02.905762+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:47:17.796768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784163+0000","last_change":"2023-11-28T11:10:59.794457+0000","last_active":"2023-11-28T11:11:00.784163+0000","last_peered":"2023-11-28T11:11:00.784163+0000","last_clean":"2023-11-28T11:11:00.784163+0000","last_became_active":"2023-11-28T11:10:59.794187+0000","last_became_peered":"2023-11-28T11:10:59.794187+0000","last_unstale":"2023-11-28T11:11:00.784163+0000","last_undegraded":"2023-11-28T11:11:00.784163+0000","last_fullsized":"2023-11-28T11:11:00.784163+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:31:46.079984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906158+0000","last_change":"2023-11-28T11:10:59.366540+0000","last_active":"2023-11-28T11:11:02.906158+0000","last_peered":"2023-11-28T11:11:02.906158+0000","last_clean":"2023-11-28T11:11:02.906158+0000","last_became_active":"2023-11-28T11:10:59.366086+0000","last_became_peered":"2023-11-28T11:10:59.366086+0000","last_unstale":"2023-11-28T11:11:02.906158+0000","last_undegraded":"2023-11-28T11:11:02.906158+0000","last_fullsized":"2023-11-28T11:11:02.906158+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:35.736465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906065+0000","last_change":"2023-11-28T11:10:59.796322+0000","last_active":"2023-11-28T11:11:02.906065+0000","last_peered":"2023-11-28T11:11:02.906065+0000","last_clean":"2023-11-28T11:11:02.906065+0000","last_became_active":"2023-11-28T11:10:59.796041+0000","last_became_peered":"2023-11-28T11:10:59.796041+0000","last_unstale":"2023-11-28T11:11:02.906065+0000","last_undegraded":"2023-11-28T11:11:02.906065+0000","last_fullsized":"2023-11-28T11:11:02.906065+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:41:25.021451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.292124+0000","last_change":"2023-11-28T11:10:59.370497+0000","last_active":"2023-11-28T11:11:02.292124+0000","last_peered":"2023-11-28T11:11:02.292124+0000","last_clean":"2023-11-28T11:11:02.292124+0000","last_became_active":"2023-11-28T11:10:59.370219+0000","last_became_peered":"2023-11-28T11:10:59.370219+0000","last_unstale":"2023-11-28T11:11:02.292124+0000","last_undegraded":"2023-11-28T11:11:02.292124+0000","last_fullsized":"2023-11-28T11:11:02.292124+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:57:16.833452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905218+0000","last_change":"2023-11-28T11:10:59.796459+0000","last_active":"2023-11-28T11:11:02.905218+0000","last_peered":"2023-11-28T11:11:02.905218+0000","last_clean":"2023-11-28T11:11:02.905218+0000","last_became_active":"2023-11-28T11:10:59.796175+0000","last_became_peered":"2023-11-28T11:10:59.796175+0000","last_unstale":"2023-11-28T11:11:02.905218+0000","last_undegraded":"2023-11-28T11:11:02.905218+0000","last_fullsized":"2023-11-28T11:11:02.905218+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:07:26.055549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784167+0000","last_change":"2023-11-28T11:10:59.364716+0000","last_active":"2023-11-28T11:11:00.784167+0000","last_peered":"2023-11-28T11:11:00.784167+0000","last_clean":"2023-11-28T11:11:00.784167+0000","last_became_active":"2023-11-28T11:10:59.364388+0000","last_became_peered":"2023-11-28T11:10:59.364388+0000","last_unstale":"2023-11-28T11:11:00.784167+0000","last_undegraded":"2023-11-28T11:11:00.784167+0000","last_fullsized":"2023-11-28T11:11:00.784167+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:45.592788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906407+0000","last_change":"2023-11-28T11:10:59.793552+0000","last_active":"2023-11-28T11:11:02.906407+0000","last_peered":"2023-11-28T11:11:02.906407+0000","last_clean":"2023-11-28T11:11:02.906407+0000","last_became_active":"2023-11-28T11:10:59.793301+0000","last_became_peered":"2023-11-28T11:10:59.793301+0000","last_unstale":"2023-11-28T11:11:02.906407+0000","last_undegraded":"2023-11-28T11:11:02.906407+0000","last_fullsized":"2023-11-28T11:11:02.906407+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:23.965103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"53'8","reported_seq":19,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905654+0000","last_change":"2023-11-28T11:10:59.371819+0000","last_active":"2023-11-28T11:11:02.905654+0000","last_peered":"2023-11-28T11:11:02.905654+0000","last_clean":"2023-11-28T11:11:02.905654+0000","last_became_active":"2023-11-28T11:10:59.371521+0000","last_became_peered":"2023-11-28T11:10:59.371521+0000","last_unstale":"2023-11-28T11:11:02.905654+0000","last_undegraded":"2023-11-28T11:11:02.905654+0000","last_fullsized":"2023-11-28T11:11:02.905654+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:52:56.400142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784446+0000","last_change":"2023-11-28T11:10:59.791682+0000","last_active":"2023-11-28T11:11:00.784446+0000","last_peered":"2023-11-28T11:11:00.784446+0000","last_clean":"2023-11-28T11:11:00.784446+0000","last_became_active":"2023-11-28T11:10:59.791097+0000","last_became_peered":"2023-11-28T11:10:59.791097+0000","last_unstale":"2023-11-28T11:11:00.784446+0000","last_undegraded":"2023-11-28T11:11:00.784446+0000","last_fullsized":"2023-11-28T11:11:00.784446+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:36.589155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780257+0000","last_change":"2023-11-28T11:10:59.365816+0000","last_active":"2023-11-28T11:11:00.780257+0000","last_peered":"2023-11-28T11:11:00.780257+0000","last_clean":"2023-11-28T11:11:00.780257+0000","last_became_active":"2023-11-28T11:10:59.365556+0000","last_became_peered":"2023-11-28T11:10:59.365556+0000","last_unstale":"2023-11-28T11:11:00.780257+0000","last_undegraded":"2023-11-28T11:11:00.780257+0000","last_fullsized":"2023-11-28T11:11:00.780257+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:38:46.615550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784573+0000","last_change":"2023-11-28T11:10:59.794871+0000","last_active":"2023-11-28T11:11:00.784573+0000","last_peered":"2023-11-28T11:11:00.784573+0000","last_clean":"2023-11-28T11:11:00.784573+0000","last_became_active":"2023-11-28T11:10:59.794570+0000","last_became_peered":"2023-11-28T11:10:59.794570+0000","last_unstale":"2023-11-28T11:11:00.784573+0000","last_undegraded":"2023-11-28T11:11:00.784573+0000","last_fullsized":"2023-11-28T11:11:00.784573+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:47:42.462499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"54'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:03.237375+0000","last_change":"2023-11-28T11:10:59.366239+0000","last_active":"2023-11-28T11:11:03.237375+0000","last_peered":"2023-11-28T11:11:03.237375+0000","last_clean":"2023-11-28T11:11:03.237375+0000","last_became_active":"2023-11-28T11:10:59.365580+0000","last_became_peered":"2023-11-28T11:10:59.365580+0000","last_unstale":"2023-11-28T11:11:03.237375+0000","last_undegraded":"2023-11-28T11:11:03.237375+0000","last_fullsized":"2023-11-28T11:11:03.237375+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:41:03.472842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780106+0000","last_change":"2023-11-28T11:10:59.802145+0000","last_active":"2023-11-28T11:11:00.780106+0000","last_peered":"2023-11-28T11:11:00.780106+0000","last_clean":"2023-11-28T11:11:00.780106+0000","last_became_active":"2023-11-28T11:10:59.801683+0000","last_became_peered":"2023-11-28T11:10:59.801683+0000","last_unstale":"2023-11-28T11:11:00.780106+0000","last_undegraded":"2023-11-28T11:11:00.780106+0000","last_fullsized":"2023-11-28T11:11:00.780106+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:20:22.237914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.292182+0000","last_change":"2023-11-28T11:10:59.364244+0000","last_active":"2023-11-28T11:11:02.292182+0000","last_peered":"2023-11-28T11:11:02.292182+0000","last_clean":"2023-11-28T11:11:02.292182+0000","last_became_active":"2023-11-28T11:10:59.363887+0000","last_became_peered":"2023-11-28T11:10:59.363887+0000","last_unstale":"2023-11-28T11:11:02.292182+0000","last_undegraded":"2023-11-28T11:11:02.292182+0000","last_fullsized":"2023-11-28T11:11:02.292182+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:38:46.615550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906891+0000","last_change":"2023-11-28T11:10:59.798580+0000","last_active":"2023-11-28T11:11:02.906891+0000","last_peered":"2023-11-28T11:11:02.906891+0000","last_clean":"2023-11-28T11:11:02.906891+0000","last_became_active":"2023-11-28T11:10:59.798096+0000","last_became_peered":"2023-11-28T11:10:59.798096+0000","last_unstale":"2023-11-28T11:11:02.906891+0000","last_undegraded":"2023-11-28T11:11:02.906891+0000","last_fullsized":"2023-11-28T11:11:02.906891+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:31:39.866743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784462+0000","last_change":"2023-11-28T11:10:59.359526+0000","last_active":"2023-11-28T11:11:00.784462+0000","last_peered":"2023-11-28T11:11:00.784462+0000","last_clean":"2023-11-28T11:11:00.784462+0000","last_became_active":"2023-11-28T11:10:59.359227+0000","last_became_peered":"2023-11-28T11:10:59.359227+0000","last_unstale":"2023-11-28T11:11:00.784462+0000","last_undegraded":"2023-11-28T11:11:00.784462+0000","last_fullsized":"2023-11-28T11:11:00.784462+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:13:54.036483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905687+0000","last_change":"2023-11-28T11:10:59.800743+0000","last_active":"2023-11-28T11:11:02.905687+0000","last_peered":"2023-11-28T11:11:02.905687+0000","last_clean":"2023-11-28T11:11:02.905687+0000","last_became_active":"2023-11-28T11:10:59.800492+0000","last_became_peered":"2023-11-28T11:10:59.800492+0000","last_unstale":"2023-11-28T11:11:02.905687+0000","last_undegraded":"2023-11-28T11:11:02.905687+0000","last_fullsized":"2023-11-28T11:11:02.905687+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:54:00.484044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.779970+0000","last_change":"2023-11-28T11:10:59.360867+0000","last_active":"2023-11-28T11:11:00.779970+0000","last_peered":"2023-11-28T11:11:00.779970+0000","last_clean":"2023-11-28T11:11:00.779970+0000","last_became_active":"2023-11-28T11:10:59.360582+0000","last_became_peered":"2023-11-28T11:10:59.360582+0000","last_unstale":"2023-11-28T11:11:00.779970+0000","last_undegraded":"2023-11-28T11:11:00.779970+0000","last_fullsized":"2023-11-28T11:11:00.779970+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:25:03.498918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905927+0000","last_change":"2023-11-28T11:10:59.800578+0000","last_active":"2023-11-28T11:11:02.905927+0000","last_peered":"2023-11-28T11:11:02.905927+0000","last_clean":"2023-11-28T11:11:02.905927+0000","last_became_active":"2023-11-28T11:10:59.799028+0000","last_became_peered":"2023-11-28T11:10:59.799028+0000","last_unstale":"2023-11-28T11:11:02.905927+0000","last_undegraded":"2023-11-28T11:11:02.905927+0000","last_fullsized":"2023-11-28T11:11:02.905927+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:36.741937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905606+0000","last_change":"2023-11-28T11:10:59.365099+0000","last_active":"2023-11-28T11:11:02.905606+0000","last_peered":"2023-11-28T11:11:02.905606+0000","last_clean":"2023-11-28T11:11:02.905606+0000","last_became_active":"2023-11-28T11:10:59.364795+0000","last_became_peered":"2023-11-28T11:10:59.364795+0000","last_unstale":"2023-11-28T11:11:02.905606+0000","last_undegraded":"2023-11-28T11:11:02.905606+0000","last_fullsized":"2023-11-28T11:11:02.905606+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:41:03.472842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780687+0000","last_change":"2023-11-28T11:10:59.802156+0000","last_active":"2023-11-28T11:11:00.780687+0000","last_peered":"2023-11-28T11:11:00.780687+0000","last_clean":"2023-11-28T11:11:00.780687+0000","last_became_active":"2023-11-28T11:10:59.801852+0000","last_became_peered":"2023-11-28T11:10:59.801852+0000","last_unstale":"2023-11-28T11:11:00.780687+0000","last_undegraded":"2023-11-28T11:11:00.780687+0000","last_fullsized":"2023-11-28T11:11:00.780687+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:36.892073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.779819+0000","last_change":"2023-11-28T11:10:59.365299+0000","last_active":"2023-11-28T11:11:00.779819+0000","last_peered":"2023-11-28T11:11:00.779819+0000","last_clean":"2023-11-28T11:11:00.779819+0000","last_became_active":"2023-11-28T11:10:59.365006+0000","last_became_peered":"2023-11-28T11:10:59.365006+0000","last_unstale":"2023-11-28T11:11:00.779819+0000","last_undegraded":"2023-11-28T11:11:00.779819+0000","last_fullsized":"2023-11-28T11:11:00.779819+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:01:39.937685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905803+0000","last_change":"2023-11-28T11:10:59.791752+0000","last_active":"2023-11-28T11:11:02.905803+0000","last_peered":"2023-11-28T11:11:02.905803+0000","last_clean":"2023-11-28T11:11:02.905803+0000","last_became_active":"2023-11-28T11:10:59.790973+0000","last_became_peered":"2023-11-28T11:10:59.790973+0000","last_unstale":"2023-11-28T11:11:02.905803+0000","last_undegraded":"2023-11-28T11:11:02.905803+0000","last_fullsized":"2023-11-28T11:11:02.905803+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:59:49.410919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905730+0000","last_change":"2023-11-28T11:10:59.364424+0000","last_active":"2023-11-28T11:11:02.905730+0000","last_peered":"2023-11-28T11:11:02.905730+0000","last_clean":"2023-11-28T11:11:02.905730+0000","last_became_active":"2023-11-28T11:10:59.361681+0000","last_became_peered":"2023-11-28T11:10:59.361681+0000","last_unstale":"2023-11-28T11:11:02.905730+0000","last_undegraded":"2023-11-28T11:11:02.905730+0000","last_fullsized":"2023-11-28T11:11:02.905730+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:12.320362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905917+0000","last_change":"2023-11-28T11:10:59.805162+0000","last_active":"2023-11-28T11:11:02.905917+0000","last_peered":"2023-11-28T11:11:02.905917+0000","last_clean":"2023-11-28T11:11:02.905917+0000","last_became_active":"2023-11-28T11:10:59.804866+0000","last_became_peered":"2023-11-28T11:10:59.804866+0000","last_unstale":"2023-11-28T11:11:02.905917+0000","last_undegraded":"2023-11-28T11:11:02.905917+0000","last_fullsized":"2023-11-28T11:11:02.905917+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:48:36.589155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906802+0000","last_change":"2023-11-28T11:10:59.369331+0000","last_active":"2023-11-28T11:11:02.906802+0000","last_peered":"2023-11-28T11:11:02.906802+0000","last_clean":"2023-11-28T11:11:02.906802+0000","last_became_active":"2023-11-28T11:10:59.369122+0000","last_became_peered":"2023-11-28T11:10:59.369122+0000","last_unstale":"2023-11-28T11:11:02.906802+0000","last_undegraded":"2023-11-28T11:11:02.906802+0000","last_fullsized":"2023-11-28T11:11:02.906802+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:12.320362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":42,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.778121+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:10:59.778121+0000","last_peered":"2023-11-28T11:10:59.778121+0000","last_clean":"2023-11-28T11:10:59.778121+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:10:59.778121+0000","last_undegraded":"2023-11-28T11:10:59.778121+0000","last_fullsized":"2023-11-28T11:10:59.778121+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906171+0000","last_change":"2023-11-28T11:10:59.786892+0000","last_active":"2023-11-28T11:11:02.906171+0000","last_peered":"2023-11-28T11:11:02.906171+0000","last_clean":"2023-11-28T11:11:02.906171+0000","last_became_active":"2023-11-28T11:10:59.786632+0000","last_became_peered":"2023-11-28T11:10:59.786632+0000","last_unstale":"2023-11-28T11:11:02.906171+0000","last_undegraded":"2023-11-28T11:11:02.906171+0000","last_fullsized":"2023-11-28T11:11:02.906171+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:49:03.606113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784692+0000","last_change":"2023-11-28T11:10:59.364836+0000","last_active":"2023-11-28T11:11:00.784692+0000","last_peered":"2023-11-28T11:11:00.784692+0000","last_clean":"2023-11-28T11:11:00.784692+0000","last_became_active":"2023-11-28T11:10:59.364463+0000","last_became_peered":"2023-11-28T11:10:59.364463+0000","last_unstale":"2023-11-28T11:11:00.784692+0000","last_undegraded":"2023-11-28T11:11:00.784692+0000","last_fullsized":"2023-11-28T11:11:00.784692+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:52:35.886600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":42,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.776479+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:10:59.776479+0000","last_peered":"2023-11-28T11:10:59.776479+0000","last_clean":"2023-11-28T11:10:59.776479+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:10:59.776479+0000","last_undegraded":"2023-11-28T11:10:59.776479+0000","last_fullsized":"2023-11-28T11:10:59.776479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906840+0000","last_change":"2023-11-28T11:10:59.792194+0000","last_active":"2023-11-28T11:11:02.906840+0000","last_peered":"2023-11-28T11:11:02.906840+0000","last_clean":"2023-11-28T11:11:02.906840+0000","last_became_active":"2023-11-28T11:10:59.791942+0000","last_became_peered":"2023-11-28T11:10:59.791942+0000","last_unstale":"2023-11-28T11:11:02.906840+0000","last_undegraded":"2023-11-28T11:11:02.906840+0000","last_fullsized":"2023-11-28T11:11:02.906840+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:35:10.703217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.780075+0000","last_change":"2023-11-28T11:10:59.365263+0000","last_active":"2023-11-28T11:11:00.780075+0000","last_peered":"2023-11-28T11:11:00.780075+0000","last_clean":"2023-11-28T11:11:00.780075+0000","last_became_active":"2023-11-28T11:10:59.364962+0000","last_became_peered":"2023-11-28T11:10:59.364962+0000","last_unstale":"2023-11-28T11:11:00.780075+0000","last_undegraded":"2023-11-28T11:11:00.780075+0000","last_fullsized":"2023-11-28T11:11:00.780075+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:57:16.833452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":43,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.781531+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:10:59.781531+0000","last_peered":"2023-11-28T11:10:59.781531+0000","last_clean":"2023-11-28T11:10:59.781531+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:10:59.781531+0000","last_undegraded":"2023-11-28T11:10:59.781531+0000","last_fullsized":"2023-11-28T11:10:59.781531+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":193,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.789300+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:10:59.789300+0000","last_peered":"2023-11-28T11:10:59.789300+0000","last_clean":"2023-11-28T11:10:59.789300+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:10:59.789300+0000","last_undegraded":"2023-11-28T11:10:59.789300+0000","last_fullsized":"2023-11-28T11:10:59.789300+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784636+0000","last_change":"2023-11-28T11:10:59.802124+0000","last_active":"2023-11-28T11:11:00.784636+0000","last_peered":"2023-11-28T11:11:00.784636+0000","last_clean":"2023-11-28T11:11:00.784636+0000","last_became_active":"2023-11-28T11:10:59.801897+0000","last_became_peered":"2023-11-28T11:10:59.801897+0000","last_unstale":"2023-11-28T11:11:00.784636+0000","last_undegraded":"2023-11-28T11:11:00.784636+0000","last_fullsized":"2023-11-28T11:11:00.784636+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:36.741937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906204+0000","last_change":"2023-11-28T11:10:59.364007+0000","last_active":"2023-11-28T11:11:02.906204+0000","last_peered":"2023-11-28T11:11:02.906204+0000","last_clean":"2023-11-28T11:11:02.906204+0000","last_became_active":"2023-11-28T11:10:59.363733+0000","last_became_peered":"2023-11-28T11:10:59.363733+0000","last_unstale":"2023-11-28T11:11:02.906204+0000","last_undegraded":"2023-11-28T11:11:02.906204+0000","last_fullsized":"2023-11-28T11:11:02.906204+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:51:23.407481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":43,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.787257+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:10:59.787257+0000","last_peered":"2023-11-28T11:10:59.787257+0000","last_clean":"2023-11-28T11:10:59.787257+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:10:59.787257+0000","last_undegraded":"2023-11-28T11:10:59.787257+0000","last_fullsized":"2023-11-28T11:10:59.787257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.905628+0000","last_change":"2023-11-28T11:10:59.794401+0000","last_active":"2023-11-28T11:11:02.905628+0000","last_peered":"2023-11-28T11:11:02.905628+0000","last_clean":"2023-11-28T11:11:02.905628+0000","last_became_active":"2023-11-28T11:10:59.794032+0000","last_became_peered":"2023-11-28T11:10:59.794032+0000","last_unstale":"2023-11-28T11:11:02.905628+0000","last_undegraded":"2023-11-28T11:11:02.905628+0000","last_fullsized":"2023-11-28T11:11:02.905628+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:53:00.979070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.292057+0000","last_change":"2023-11-28T11:10:59.364552+0000","last_active":"2023-11-28T11:11:02.292057+0000","last_peered":"2023-11-28T11:11:02.292057+0000","last_clean":"2023-11-28T11:11:02.292057+0000","last_became_active":"2023-11-28T11:10:59.364156+0000","last_became_peered":"2023-11-28T11:10:59.364156+0000","last_unstale":"2023-11-28T11:11:02.292057+0000","last_undegraded":"2023-11-28T11:11:02.292057+0000","last_fullsized":"2023-11-28T11:11:02.292057+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:01:39.937685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":43,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.789181+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:10:59.789181+0000","last_peered":"2023-11-28T11:10:59.789181+0000","last_clean":"2023-11-28T11:10:59.789181+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:10:59.789181+0000","last_undegraded":"2023-11-28T11:10:59.789181+0000","last_fullsized":"2023-11-28T11:10:59.789181+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906014+0000","last_change":"2023-11-28T11:10:59.798700+0000","last_active":"2023-11-28T11:11:02.906014+0000","last_peered":"2023-11-28T11:11:02.906014+0000","last_clean":"2023-11-28T11:11:02.906014+0000","last_became_active":"2023-11-28T11:10:59.798436+0000","last_became_peered":"2023-11-28T11:10:59.798436+0000","last_unstale":"2023-11-28T11:11:02.906014+0000","last_undegraded":"2023-11-28T11:11:02.906014+0000","last_fullsized":"2023-11-28T11:11:02.906014+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:33:52.792611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784096+0000","last_change":"2023-11-28T11:10:59.363284+0000","last_active":"2023-11-28T11:11:00.784096+0000","last_peered":"2023-11-28T11:11:00.784096+0000","last_clean":"2023-11-28T11:11:00.784096+0000","last_became_active":"2023-11-28T11:10:59.362970+0000","last_became_peered":"2023-11-28T11:10:59.362970+0000","last_unstale":"2023-11-28T11:11:00.784096+0000","last_undegraded":"2023-11-28T11:11:00.784096+0000","last_fullsized":"2023-11-28T11:11:00.784096+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:14:29.172511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":46,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.781552+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:10:59.781552+0000","last_peered":"2023-11-28T11:10:59.781552+0000","last_clean":"2023-11-28T11:10:59.781552+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:10:59.781552+0000","last_undegraded":"2023-11-28T11:10:59.781552+0000","last_fullsized":"2023-11-28T11:10:59.781552+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906429+0000","last_change":"2023-11-28T11:10:59.798836+0000","last_active":"2023-11-28T11:11:02.906429+0000","last_peered":"2023-11-28T11:11:02.906429+0000","last_clean":"2023-11-28T11:11:02.906429+0000","last_became_active":"2023-11-28T11:10:59.798528+0000","last_became_peered":"2023-11-28T11:10:59.798528+0000","last_unstale":"2023-11-28T11:11:02.906429+0000","last_undegraded":"2023-11-28T11:11:02.906429+0000","last_fullsized":"2023-11-28T11:11:02.906429+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:23:13.325835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"53'1","reported_seq":12,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.906041+0000","last_change":"2023-11-28T11:10:59.370468+0000","last_active":"2023-11-28T11:11:02.906041+0000","last_peered":"2023-11-28T11:11:02.906041+0000","last_clean":"2023-11-28T11:11:02.906041+0000","last_became_active":"2023-11-28T11:10:59.370130+0000","last_became_peered":"2023-11-28T11:10:59.370130+0000","last_unstale":"2023-11-28T11:11:02.906041+0000","last_undegraded":"2023-11-28T11:11:02.906041+0000","last_fullsized":"2023-11-28T11:11:02.906041+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:22.959630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":41,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.782724+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:10:59.782724+0000","last_peered":"2023-11-28T11:10:59.782724+0000","last_clean":"2023-11-28T11:10:59.782724+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:10:59.782724+0000","last_undegraded":"2023-11-28T11:10:59.782724+0000","last_fullsized":"2023-11-28T11:10:59.782724+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:00.784524+0000","last_change":"2023-11-28T11:10:59.801532+0000","last_active":"2023-11-28T11:11:00.784524+0000","last_peered":"2023-11-28T11:11:00.784524+0000","last_clean":"2023-11-28T11:11:00.784524+0000","last_became_active":"2023-11-28T11:10:59.801211+0000","last_became_peered":"2023-11-28T11:10:59.801211+0000","last_unstale":"2023-11-28T11:11:00.784524+0000","last_undegraded":"2023-11-28T11:11:00.784524+0000","last_fullsized":"2023-11-28T11:11:00.784524+0000","mapping_epoch":52,"log_start":"0'0","ondisk_log_start":"0'0","created":52,"last_epoch_clean":53,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:59.341509+0000","last_clean_scrub_stamp":"2023-11-28T11:10:59.341509+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:23.965103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"53'2","reported_seq":13,"reported_epoch":54,"state":"active+clean","last_fresh":"2023-11-28T11:11:02.907001+0000","last_change":"2023-11-28T11:10:59.366434+0000","last_active":"2023-11-28T11:11:02.907001+0000","last_peered":"2023-11-28T11:11:02.907001+0000","last_clean":"2023-11-28T11:11:02.907001+0000","last_became_active":"2023-11-28T11:10:59.365971+0000","last_became_peered":"2023-11-28T11:10:59.365971+0000","last_unstale":"2023-11-28T11:11:02.907001+0000","last_undegraded":"2023-11-28T11:11:02.907001+0000","last_fullsized":"2023-11-28T11:11:02.907001+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:10:58.336036+0000","last_clean_scrub_stamp":"2023-11-28T11:10:58.336036+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:13:35.736465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":41,"reported_epoch":53,"state":"active+clean","last_fresh":"2023-11-28T11:10:59.784222+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:10:59.784222+0000","last_peered":"2023-11-28T11:10:59.784222+0000","last_clean":"2023-11-28T11:10:59.784222+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:10:59.784222+0000","last_undegraded":"2023-11-28T11:10:59.784222+0000","last_fullsized":"2023-11-28T11:10:59.784222+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10312,"kb_used_data":1216,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361528,"statfs":{"total":96636764160,"available":96626204672,"internally_reserved":0,"allocated":1245184,"data_stored":889065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.621,"15min":0.64700000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.60999999999999999,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.626,"15min":0.64400000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69499999999999995,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.41499999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.67100000000000004,"15min":0.68999999999999995},"min":{"1min":0.34899999999999998,"5min":0.316,"15min":0.316},"max":{"1min":0.879,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.38,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71999999999999997,"5min":1.224,"15min":1.224},"last":0.58399999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.67100000000000004,"15min":0.69299999999999995},"min":{"1min":0.376,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.72399999999999998,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.76000000000000001,"15min":0.78400000000000003},"min":{"1min":0.46400000000000002,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.80700000000000005,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.62}]},{"osd":3,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.68100000000000005,"15min":0.70199999999999996},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74399999999999999,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.75600000000000001,"15min":0.77600000000000002},"min":{"1min":0.47199999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85699999999999998,"5min":3.294,"15min":3.294},"last":0.67200000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.77600000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56100000000000005,"15min":0.56299999999999994},"min":{"1min":0.36199999999999999,"5min":0.317,"15min":0.317},"max":{"1min":0.76100000000000001,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.65600000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64300000000000002,"15min":0.64700000000000002},"min":{"1min":0.39500000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.96299999999999997,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.41099999999999998,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.86399999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68999999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67500000000000004,"15min":0.67900000000000005},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.86899999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63900000000000001,"15min":0.64000000000000001},"min":{"1min":0.41199999999999998,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86499999999999999,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.70199999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574881,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9148,"kb_used_data":628,"kb_used_omap":0,"kb_used_meta":8512,"kb_avail":94362692,"statfs":{"total":96636764160,"available":96627396608,"internally_reserved":0,"allocated":643072,"data_stored":298537,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8716288},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.46000000000000002,"15min":0.45400000000000001},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.72099999999999997,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.432},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.624,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.50700000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.51200000000000001,"15min":0.50700000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.77100000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.497,"15min":0.49299999999999999},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.49299999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.501,"15min":0.498},"min":{"1min":0.34100000000000003,"5min":0.312,"15min":0.312},"max":{"1min":0.753,"5min":0.77400000000000002,"15min":0.77400000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.67100000000000004,"15min":0.68500000000000005},"min":{"1min":0.42999999999999999,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84099999999999997,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.70699999999999996,"15min":0.72399999999999998},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.77900000000000003,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.627},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.68799999999999994,"15min":0.69999999999999996},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.90100000000000002,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.65600000000000003}]},{"osd":4,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.54500000000000004,"15min":0.54600000000000004},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.53400000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.60699999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:10:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.57999999999999996,"15min":0.58499999999999996},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.66900000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574881,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10244,"kb_used_data":1212,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361596,"statfs":{"total":96636764160,"available":96626274304,"internally_reserved":0,"allocated":1241088,"data_stored":888995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.32800000000000001,"15min":0.32700000000000001},"min":{"1min":0.19,"5min":0.19,"15min":0.19},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.35599999999999998,"15min":0.35399999999999998},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.54400000000000004,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.55100000000000005}]},{"osd":2,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.32400000000000001,"15min":0.318},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.436},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38100000000000001,"15min":0.38},"min":{"1min":0.29099999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.58999999999999997,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.49099999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.39400000000000002,"15min":0.38800000000000001},"min":{"1min":0.28100000000000003,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.40999999999999998,"15min":0.40799999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.69999999999999996,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.59199999999999997}]},{"osd":4,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53000000000000003,"15min":0.52700000000000002},"min":{"1min":0.38700000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56699999999999995,"15min":0.56499999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.88200000000000001,"5min":1.014,"15min":1.014},"last":0.75700000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.55600000000000005,"15min":0.54600000000000004},"min":{"1min":0.39900000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.79000000000000004,"5min":0.998,"15min":0.998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.56399999999999995,"15min":0.55800000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.61699999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.442,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.79300000000000004,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.629,"5min":0.59399999999999997,"15min":0.58799999999999997},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.81000000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.67200000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:10:40 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.85199999999999998,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.59599999999999997,"15min":0.59199999999999997},"min":{"1min":0.47799999999999998,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.82399999999999995,"5min":1.071,"15min":1.071},"last":0.68700000000000006}]}]},{"osd":0,"up_from":13,"seq":55834574882,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9356,"kb_used_data":644,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362484,"statfs":{"total":96636764160,"available":96627183616,"internally_reserved":0,"allocated":659456,"data_stored":300651,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.373,"5min":0.37,"15min":0.37},"min":{"1min":0.26300000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.622,"5min":0.622,"15min":0.622},"last":0.52600000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.38,"15min":0.38300000000000001},"min":{"1min":0.26800000000000002,"5min":0.246,"15min":0.246},"max":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39700000000000002,"15min":0.39900000000000002},"min":{"1min":0.28699999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.57999999999999996,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.41599999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36799999999999999,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.48999999999999999,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.30199999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.58999999999999997,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58499999999999996,"15min":0.58299999999999996},"min":{"1min":0.434,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.81599999999999995,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.629},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.41199999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.86299999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.84299999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.65100000000000002,"15min":0.62},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":5.4269999999999996,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.90000000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.62,"15min":0.622},"min":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.47399999999999998},"max":{"1min":0.79900000000000004,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.86899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.61699999999999999,"15min":0.621},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.89500000000000002,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.63100000000000001,"15min":0.63300000000000001},"min":{"1min":0.44800000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.88200000000000001,"5min":1.016,"15min":1.016},"last":0.85999999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.64900000000000002,"15min":0.65100000000000002},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85799999999999998,"5min":1.105,"15min":1.105},"last":0.93400000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63800000000000001,"15min":0.63900000000000001},"min":{"1min":0.505,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.94299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9088,"kb_used_data":632,"kb_used_omap":0,"kb_used_meta":8448,"kb_avail":94362752,"statfs":{"total":96636764160,"available":96627458048,"internally_reserved":0,"allocated":647168,"data_stored":297632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8650752},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.36699999999999999,"15min":0.374},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":0.41699999999999998,"5min":0.55000000000000004,"15min":0.55000000000000004},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38100000000000001,"15min":0.38700000000000001},"min":{"1min":0.248,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":0.51900000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.25600000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36899999999999999,"5min":0.39000000000000001,"15min":0.39400000000000002},"min":{"1min":0.26100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.51000000000000001,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.437,"15min":0.44600000000000001},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60899999999999999,"5min":1.528,"15min":1.528},"last":0.316}]},{"osd":3,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.39000000000000001,"15min":0.39600000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.69599999999999995,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.46899999999999997,"15min":0.47999999999999998},"min":{"1min":0.31,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.59699999999999998,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.38700000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61499999999999999,"15min":0.62},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.79000000000000004,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.53500000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60599999999999998,"15min":0.60999999999999999},"min":{"1min":0.438,"5min":0.438,"15min":0.438},"max":{"1min":0.75600000000000001,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.85699999999999998,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52600000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.503},{"interface":"front","average":{"1min":0.62,"5min":0.66400000000000003,"15min":0.67100000000000004},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.876,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.58699999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:10:43 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.64300000000000002,"15min":0.64800000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.623,"15min":0.628},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.54600000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574881,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9284,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362556,"statfs":{"total":96636764160,"available":96627257344,"internally_reserved":0,"allocated":651264,"data_stored":298593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.83399999999999996,"15min":0.92000000000000004},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.46300000000000002,"5min":11.965,"15min":11.965},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.87,"15min":0.95799999999999996},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55000000000000004,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.311}]},{"osd":1,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.33900000000000002,"15min":0.33700000000000002},"min":{"1min":0.23999999999999999,"5min":0.215,"15min":0.215},"max":{"1min":0.57999999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.36599999999999999,"5min":1.0940000000000001,"15min":1.2150000000000001},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.56799999999999995,"5min":16.988,"15min":16.988},"last":0.33500000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.49399999999999999,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75600000000000001,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":1.6020000000000001,"15min":1.7969999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64100000000000001,"5min":27.887,"15min":27.887},"last":0.38500000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.68799999999999994,"15min":0.70899999999999996},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.75900000000000001,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.65800000000000003,"15min":0.66900000000000004},"min":{"1min":0.44700000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.748,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.59599999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.67900000000000005,"15min":0.69199999999999995},"min":{"1min":0.434,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.745,"5min":1.98,"15min":1.98},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.59699999999999998,"5min":1.1220000000000001,"15min":1.21},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.78100000000000003,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.75700000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":1.1279999999999999,"15min":1.2170000000000001},"min":{"1min":0.44,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.73799999999999999,"5min":11.888,"15min":11.888},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.71799999999999997,"15min":0.73699999999999999},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.76000000000000001,"5min":1.925,"15min":1.925},"last":0.77000000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:10:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.73099999999999998,"15min":0.75},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77300000000000002,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.76600000000000001,"15min":0.78600000000000003},"min":{"1min":0.498,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.79700000000000004,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.78700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574882,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9352,"kb_used_data":640,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362488,"statfs":{"total":96636764160,"available":96627187712,"internally_reserved":0,"allocated":655360,"data_stored":300085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.39700000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.748,"5min":0.996,"15min":0.996},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.90300000000000002,"15min":0.96499999999999997},"min":{"1min":0.41099999999999998,"5min":0.372,"15min":0.372},"max":{"1min":0.70099999999999996,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.34999999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.72499999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57399999999999995,"15min":0.57799999999999996},"min":{"1min":0.375,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.70499999999999996,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.56999999999999995,"15min":0.57399999999999995},"min":{"1min":0.41299999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.67900000000000005,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.375,"5min":0.374,"15min":0.374},"max":{"1min":0.749,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.442}]},{"osd":3,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.72899999999999998,"15min":0.75900000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.70399999999999996,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.623},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63200000000000001,"15min":0.63800000000000001},"min":{"1min":0.47699999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71599999999999997,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.46600000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33100000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.71999999999999997,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.502},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.35799999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.79300000000000004,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.48199999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.59999999999999998,"15min":0.60799999999999998},"min":{"1min":0.35899999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76400000000000001,"5min":1.145,"15min":1.145},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.625,"15min":0.63300000000000001},"min":{"1min":0.374,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.77500000000000002,"5min":1.085,"15min":1.085},"last":0.55400000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:10:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.751,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67900000000000005,"15min":0.68300000000000005},"min":{"1min":0.40600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.85499999999999998,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.51900000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574882,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9220,"kb_used_data":636,"kb_used_omap":0,"kb_used_meta":8576,"kb_avail":94362620,"statfs":{"total":96636764160,"available":96627322880,"internally_reserved":0,"allocated":651264,"data_stored":299173,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8781824},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.36499999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.71199999999999997,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.434},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.34499999999999997,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.73099999999999998,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.66300000000000003,"15min":0.67300000000000004},"min":{"1min":0.41399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79300000000000004,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.66100000000000003,"15min":0.67600000000000005},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.71199999999999997,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.38100000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.78300000000000003,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.53200000000000003,"15min":0.52800000000000002},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.73699999999999999,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.499}]},{"osd":3,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.67200000000000004,"15min":0.69099999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.77400000000000002,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59199999999999997,"15min":0.59099999999999997},"min":{"1min":0.42999999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.80400000000000005,"5min":1.042,"15min":1.042},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.629,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.37},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":0.75900000000000001,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.52200000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.41399999999999998,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.73399999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72699999999999998,"5min":1.036,"15min":1.036},"last":0.56100000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:10:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.68200000000000005,"15min":0.69599999999999995},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.78300000000000003,"5min":1.329,"15min":1.329},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.68200000000000005,"15min":0.69399999999999995},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75800000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.61899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:11:09.223 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:11:09.223 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:11:09.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.389+0000 7fd2be254700 1 Processor -- start 2023-11-28T11:11:09.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.389+0000 7fd2be254700 1 -- start start 2023-11-28T11:11:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2be254700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd2b80b2150 0x7fd2b806f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2be254700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd2b806dd40 0x7fd2b80ca520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2be254700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2b815a860 con 0x7fd2b815a490 2023-11-28T11:11:09.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2be254700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b806b510 con 0x7fd2b806dd40 2023-11-28T11:11:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2be254700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2b80c9a80 con 0x7fd2b80b2150 2023-11-28T11:11:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2b7fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd2b806dd40 0x7fd2b80ca520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2b77fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fd2b815a490 0x7fd2b806d630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38652/0 (socket says 172.21.15.28:38652) 2023-11-28T11:11:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2b77fe700 1 -- 172.21.15.28:0/73362355 learned_addr learned my addr 172.21.15.28:0/73362355 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2b7fff700 1 --2- 172.21.15.28:0/73362355 >> v2:172.21.15.28:3300/0 conn(0x7fd2b806dd40 0x7fd2b80ca520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50598/0 (socket says 172.21.15.28:50598) 2023-11-28T11:11:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2b6ffd700 1 --2- 172.21.15.28:0/73362355 >> v2:172.21.15.59:3300/0 conn(0x7fd2b80b2150 0x7fd2b806f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2b7fff700 1 -- 172.21.15.28:0/73362355 >> v2:172.21.15.59:3300/0 conn(0x7fd2b80b2150 msgr2=0x7fd2b806f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2b7fff700 1 --2- 172.21.15.28:0/73362355 >> v2:172.21.15.59:3300/0 conn(0x7fd2b80b2150 0x7fd2b806f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2b7fff700 1 -- 172.21.15.28:0/73362355 >> v1:172.21.15.28:6789/0 conn(0x7fd2b815a490 legacy=0x7fd2b806d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2b67fc700 1 -- 172.21.15.28:0/73362355 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4062451011 0 0) 0x7fd2b815a860 con 0x7fd2b815a490 2023-11-28T11:11:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2b7fff700 1 -- 172.21.15.28:0/73362355 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b806b230 con 0x7fd2b806dd40 2023-11-28T11:11:09.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2b6ffd700 1 --2- 172.21.15.28:0/73362355 >> v2:172.21.15.59:3300/0 conn(0x7fd2b80b2150 0x7fd2b806f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.390+0000 7fd2b7fff700 1 --2- 172.21.15.28:0/73362355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b806dd40 0x7fd2b80ca520 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80b51b0 tx=0x7fd2b000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d2d85b71acbb6f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.391+0000 7fd2b67fc700 1 -- 172.21.15.28:0/73362355 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000e070 con 0x7fd2b806dd40 2023-11-28T11:11:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.391+0000 7fd2b67fc700 1 -- 172.21.15.28:0/73362355 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b000bbd0 con 0x7fd2b806dd40 2023-11-28T11:11:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.391+0000 7fd2b67fc700 1 -- 172.21.15.28:0/73362355 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b0012430 con 0x7fd2b806dd40 2023-11-28T11:11:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.391+0000 7fd2be254700 1 -- 172.21.15.28:0/73362355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b806dd40 msgr2=0x7fd2b80ca520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.391+0000 7fd2be254700 1 --2- 172.21.15.28:0/73362355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b806dd40 0x7fd2b80ca520 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fd2b80b51b0 tx=0x7fd2b000b330 comp rx=0 tx=0).stop 2023-11-28T11:11:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.391+0000 7fd2be254700 1 -- 172.21.15.28:0/73362355 shutdown_connections 2023-11-28T11:11:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.391+0000 7fd2be254700 1 --2- 172.21.15.28:0/73362355 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b806dd40 0x7fd2b80ca520 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.391+0000 7fd2be254700 1 --2- 172.21.15.28:0/73362355 >> v2:172.21.15.59:3300/0 conn(0x7fd2b80b2150 0x7fd2b806f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.391+0000 7fd2be254700 1 -- 172.21.15.28:0/73362355 >> 172.21.15.28:0/73362355 conn(0x7fd2b80b0df0 msgr2=0x7fd2b80b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.391+0000 7fd2be254700 1 -- 172.21.15.28:0/73362355 shutdown_connections 2023-11-28T11:11:09.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.391+0000 7fd2be254700 1 -- 172.21.15.28:0/73362355 wait complete. 2023-11-28T11:11:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.392+0000 7fd2be254700 1 Processor -- start 2023-11-28T11:11:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.393+0000 7fd2be254700 1 -- start start 2023-11-28T11:11:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.393+0000 7fd2be254700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2b806dd40 0x7fd2b81f15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.393+0000 7fd2be254700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b80b2150 0x7fd2b814eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.393+0000 7fd2be254700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2b815a490 0x7fd2b8149350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.393+0000 7fd2be254700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b81f1af0 con 0x7fd2b80b2150 2023-11-28T11:11:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.393+0000 7fd2be254700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b80ac670 con 0x7fd2b806dd40 2023-11-28T11:11:09.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.393+0000 7fd2be254700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2b80aba60 con 0x7fd2b815a490 2023-11-28T11:11:09.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd2b6ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b80b2150 0x7fd2b814eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd2b6ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b80b2150 0x7fd2b814eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50602/0 (socket says 172.21.15.28:50602) 2023-11-28T11:11:09.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd2b6ffd700 1 -- 172.21.15.28:0/2429139197 learned_addr learned my addr 172.21.15.28:0/2429139197 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:09.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd2b77fe700 1 --2- 172.21.15.28:0/2429139197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2b806dd40 0x7fd2b81f15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd2b7fff700 1 --2- 172.21.15.28:0/2429139197 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2b815a490 0x7fd2b8149350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd2b6ffd700 1 -- 172.21.15.28:0/2429139197 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2b815a490 msgr2=0x7fd2b8149350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd2b6ffd700 1 --2- 172.21.15.28:0/2429139197 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2b815a490 0x7fd2b8149350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd2b6ffd700 1 -- 172.21.15.28:0/2429139197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2b806dd40 msgr2=0x7fd2b81f15b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd2b6ffd700 1 --2- 172.21.15.28:0/2429139197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2b806dd40 0x7fd2b81f15b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd2b6ffd700 1 -- 172.21.15.28:0/2429139197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b0009a70 con 0x7fd2b80b2150 2023-11-28T11:11:09.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd2b6ffd700 1 --2- 172.21.15.28:0/2429139197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b80b2150 0x7fd2b814eea0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd2a800c750 tx=0x7fd2a800ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd29ffff700 1 -- 172.21.15.28:0/2429139197 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8011800 con 0x7fd2b80b2150 2023-11-28T11:11:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd29ffff700 1 -- 172.21.15.28:0/2429139197 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a8011960 con 0x7fd2b80b2150 2023-11-28T11:11:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd2be254700 1 -- 172.21.15.28:0/2429139197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2b80ca000 con 0x7fd2b80b2150 2023-11-28T11:11:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd29ffff700 1 -- 172.21.15.28:0/2429139197 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a8011cd0 con 0x7fd2b80b2150 2023-11-28T11:11:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.394+0000 7fd2be254700 1 -- 172.21.15.28:0/2429139197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2b80c9d20 con 0x7fd2b80b2150 2023-11-28T11:11:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.395+0000 7fd2be254700 1 -- 172.21.15.28:0/2429139197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2b806ae90 con 0x7fd2b80b2150 2023-11-28T11:11:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.396+0000 7fd29ffff700 1 -- 172.21.15.28:0/2429139197 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd2a8020070 con 0x7fd2b80b2150 2023-11-28T11:11:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.396+0000 7fd29ffff700 1 --2- 172.21.15.28:0/2429139197 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd2a40682f0 0x7fd2a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.396+0000 7fd29ffff700 1 -- 172.21.15.28:0/2429139197 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd2a8085a60 con 0x7fd2b80b2150 2023-11-28T11:11:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.396+0000 7fd2b77fe700 1 --2- 172.21.15.28:0/2429139197 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd2a40682f0 0x7fd2a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.397+0000 7fd2b77fe700 1 --2- 172.21.15.28:0/2429139197 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd2a40682f0 0x7fd2a406a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac007fd0 tx=0x7fd2ac007de0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.400+0000 7fd29ffff700 1 -- 172.21.15.28:0/2429139197 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd2a8052180 con 0x7fd2b80b2150 2023-11-28T11:11:09.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.675+0000 7fd2be254700 1 -- 172.21.15.28:0/2429139197 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fd2b80ac7e0 con 0x7fd2b80b2150 2023-11-28T11:11:09.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.678+0000 7fd29ffff700 1 -- 172.21.15.28:0/2429139197 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 184+0+0 (secure 0 0 0) 0x7fd2a80557a0 con 0x7fd2b80b2150 2023-11-28T11:11:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.680+0000 7fd2be254700 1 -- 172.21.15.28:0/2429139197 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd2a40682f0 msgr2=0x7fd2a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.680+0000 7fd2be254700 1 --2- 172.21.15.28:0/2429139197 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd2a40682f0 0x7fd2a406a7a0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fd2ac007fd0 tx=0x7fd2ac007de0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.680+0000 7fd2be254700 1 -- 172.21.15.28:0/2429139197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b80b2150 msgr2=0x7fd2b814eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.682 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.680+0000 7fd2be254700 1 --2- 172.21.15.28:0/2429139197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b80b2150 0x7fd2b814eea0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fd2a800c750 tx=0x7fd2a800ca60 comp rx=0 tx=0).stop 2023-11-28T11:11:09.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.680+0000 7fd2be254700 1 -- 172.21.15.28:0/2429139197 shutdown_connections 2023-11-28T11:11:09.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.680+0000 7fd2be254700 1 --2- 172.21.15.28:0/2429139197 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd2a40682f0 0x7fd2a406a7a0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.680+0000 7fd2be254700 1 --2- 172.21.15.28:0/2429139197 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2b815a490 0x7fd2b8149350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.680+0000 7fd2be254700 1 --2- 172.21.15.28:0/2429139197 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2b806dd40 0x7fd2b81f15b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.680+0000 7fd2be254700 1 --2- 172.21.15.28:0/2429139197 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2b80b2150 0x7fd2b814eea0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.681+0000 7fd2be254700 1 -- 172.21.15.28:0/2429139197 >> 172.21.15.28:0/2429139197 conn(0x7fd2b80b0df0 msgr2=0x7fd2b8159630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:09.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.681+0000 7fd2be254700 1 -- 172.21.15.28:0/2429139197 shutdown_connections 2023-11-28T11:11:09.683 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.681+0000 7fd2be254700 1 -- 172.21.15.28:0/2429139197 wait complete. 2023-11-28T11:11:09.683 INFO:teuthology.orchestra.run.smithi028.stderr:updated caps for client.0 2023-11-28T11:11:09.694 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:09.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.818+0000 7f2f15997700 1 Processor -- start 2023-11-28T11:11:09.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.818+0000 7f2f15997700 1 -- start start 2023-11-28T11:11:09.822 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.819+0000 7f2f15997700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f1015a6c0 0x7f2f100cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.819+0000 7f2f15997700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2f100b2660 0x7f2f100b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.819+0000 7f2f15997700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f100cfb90 con 0x7f2f100d0020 2023-11-28T11:11:09.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.819+0000 7f2f15997700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f100b3020 con 0x7f2f100b2660 2023-11-28T11:11:09.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.819+0000 7f2f15997700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100cb930 con 0x7f2f1015a6c0 2023-11-28T11:11:09.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.819+0000 7f2f0f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2f100b2660 0x7f2f100b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.819+0000 7f2f0effd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f1015a6c0 0x7f2f100cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.819+0000 7f2f0f7fe700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2f100b2660 0x7f2f100b2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:37902/0 (socket says 172.21.15.28:37902) 2023-11-28T11:11:09.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.819+0000 7f2f0f7fe700 1 -- 172.21.15.28:0/3999101685 learned_addr learned my addr 172.21.15.28:0/3999101685 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:09.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.819+0000 7f2f0effd700 1 --2- 172.21.15.28:0/3999101685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f1015a6c0 0x7f2f100cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:56366/0 (socket says 172.21.15.28:56366) 2023-11-28T11:11:09.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.819+0000 7f2f0f7fe700 1 -- 172.21.15.28:0/3999101685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f1015a6c0 msgr2=0x7f2f100cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:09.823 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.819+0000 7f2f0f7fe700 1 --2- 172.21.15.28:0/3999101685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f1015a6c0 0x7f2f100cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.819+0000 7f2f0f7fe700 1 -- 172.21.15.28:0/3999101685 >> v1:172.21.15.59:6789/0 conn(0x7f2f100d0020 legacy=0x7f2f100ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.819+0000 7f2f0f7fe700 1 -- 172.21.15.28:0/3999101685 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f100cf260 con 0x7f2f100b2660 2023-11-28T11:11:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.819+0000 7f2f0f7fe700 1 --2- 172.21.15.28:0/3999101685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f100b2660 0x7f2f100b2ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009e50 tx=0x7f2f0800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9b6784d164e60402 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.820+0000 7f2f0dffb700 1 -- 172.21.15.28:0/3999101685 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0800e070 con 0x7f2f100b2660 2023-11-28T11:11:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.820+0000 7f2f0dffb700 1 -- 172.21.15.28:0/3999101685 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0800b910 con 0x7f2f100b2660 2023-11-28T11:11:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.820+0000 7f2f0dffb700 1 -- 172.21.15.28:0/3999101685 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0800bcc0 con 0x7f2f100b2660 2023-11-28T11:11:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.820+0000 7f2f15997700 1 -- 172.21.15.28:0/3999101685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f100b2660 msgr2=0x7f2f100b2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.820+0000 7f2f15997700 1 --2- 172.21.15.28:0/3999101685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f100b2660 0x7f2f100b2ae0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009e50 tx=0x7f2f0800b330 comp rx=0 tx=0).stop 2023-11-28T11:11:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.820+0000 7f2f15997700 1 -- 172.21.15.28:0/3999101685 shutdown_connections 2023-11-28T11:11:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.820+0000 7f2f15997700 1 --2- 172.21.15.28:0/3999101685 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f100b2660 0x7f2f100b2ae0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.820+0000 7f2f15997700 1 --2- 172.21.15.28:0/3999101685 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f1015a6c0 0x7f2f100cb230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.824 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.820+0000 7f2f15997700 1 -- 172.21.15.28:0/3999101685 >> 172.21.15.28:0/3999101685 conn(0x7f2f100cd660 msgr2=0x7f2f100cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:09.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.820+0000 7f2f15997700 1 -- 172.21.15.28:0/3999101685 shutdown_connections 2023-11-28T11:11:09.825 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.820+0000 7f2f15997700 1 -- 172.21.15.28:0/3999101685 wait complete. 2023-11-28T11:11:09.827 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.822+0000 7f2f15997700 1 Processor -- start 2023-11-28T11:11:09.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.822+0000 7f2f15997700 1 -- start start 2023-11-28T11:11:09.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.823+0000 7f2f15997700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f100b2660 0x7f2f1015bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.823+0000 7f2f15997700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f100d0020 0x7f2f1014f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.823+0000 7f2f15997700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1015a6c0 0x7f2f100af1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.823+0000 7f2f15997700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1015cc40 con 0x7f2f100b2660 2023-11-28T11:11:09.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.823+0000 7f2f15997700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1015c0c0 con 0x7f2f1015a6c0 2023-11-28T11:11:09.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.823+0000 7f2f15997700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f1015c3a0 con 0x7f2f100d0020 2023-11-28T11:11:09.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.823+0000 7f2f0effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f100b2660 0x7f2f1015bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.823+0000 7f2f0effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f100b2660 0x7f2f1015bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50614/0 (socket says 172.21.15.28:50614) 2023-11-28T11:11:09.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.823+0000 7f2f0effd700 1 -- 172.21.15.28:0/349404170 learned_addr learned my addr 172.21.15.28:0/349404170 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:09.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.823+0000 7f2f0f7fe700 1 --2- 172.21.15.28:0/349404170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1015a6c0 0x7f2f100af1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.828 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.823+0000 7f2f0e7fc700 1 --2- 172.21.15.28:0/349404170 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f100d0020 0x7f2f1014f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.823+0000 7f2f0effd700 1 -- 172.21.15.28:0/349404170 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f100d0020 msgr2=0x7f2f1014f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.824+0000 7f2f0effd700 1 --2- 172.21.15.28:0/349404170 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f100d0020 0x7f2f1014f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.824+0000 7f2f0effd700 1 -- 172.21.15.28:0/349404170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1015a6c0 msgr2=0x7f2f100af1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:09.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.824+0000 7f2f0effd700 1 --2- 172.21.15.28:0/349404170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1015a6c0 0x7f2f100af1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:09.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.824+0000 7f2f0effd700 1 -- 172.21.15.28:0/349404170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f08009b30 con 0x7f2f100b2660 2023-11-28T11:11:09.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.824+0000 7f2f0e7fc700 1 --2- 172.21.15.28:0/349404170 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f100d0020 0x7f2f1014f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:09.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.824+0000 7f2f0effd700 1 --2- 172.21.15.28:0/349404170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f100b2660 0x7f2f1015bae0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000e9b0 tx=0x7f2f0000ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.824+0000 7f2ef7fff700 1 -- 172.21.15.28:0/349404170 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f00009710 con 0x7f2f100b2660 2023-11-28T11:11:09.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.824+0000 7f2ef7fff700 1 -- 172.21.15.28:0/349404170 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f0000c850 con 0x7f2f100b2660 2023-11-28T11:11:09.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.824+0000 7f2f15997700 1 -- 172.21.15.28:0/349404170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f100cf820 con 0x7f2f100b2660 2023-11-28T11:11:09.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.824+0000 7f2ef7fff700 1 -- 172.21.15.28:0/349404170 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f0000cbc0 con 0x7f2f100b2660 2023-11-28T11:11:09.829 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.824+0000 7f2f15997700 1 -- 172.21.15.28:0/349404170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f100cf540 con 0x7f2f100b2660 2023-11-28T11:11:09.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.825+0000 7f2f15997700 1 -- 172.21.15.28:0/349404170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f100d0470 con 0x7f2f100b2660 2023-11-28T11:11:09.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.825+0000 7f2ef7fff700 1 -- 172.21.15.28:0/349404170 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2f00019030 con 0x7f2f100b2660 2023-11-28T11:11:09.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.826+0000 7f2ef7fff700 1 --2- 172.21.15.28:0/349404170 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2efc0682f0 0x7f2efc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:09.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.826+0000 7f2ef7fff700 1 -- 172.21.15.28:0/349404170 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2f00085d70 con 0x7f2f100b2660 2023-11-28T11:11:09.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.826+0000 7f2f0e7fc700 1 --2- 172.21.15.28:0/349404170 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2efc0682f0 0x7f2efc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:09.830 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.826+0000 7f2f0e7fc700 1 --2- 172.21.15.28:0/349404170 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2efc0682f0 0x7f2efc06a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2f04000f60 tx=0x7f2f04009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:09.831 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:09.830+0000 7f2ef7fff700 1 -- 172.21.15.28:0/349404170 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2f00052410 con 0x7f2f100b2660 2023-11-28T11:11:10.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.068+0000 7f2f15997700 1 -- 172.21.15.28:0/349404170 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f100675a0 con 0x7f2f100b2660 2023-11-28T11:11:10.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.068+0000 7f2ef7fff700 1 -- 172.21.15.28:0/349404170 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f2f00017090 con 0x7f2f100b2660 2023-11-28T11:11:10.070 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:10.070 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5198,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/284515388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":284515388},{"type":"v1","addr":"172.21.15.59:6835","nonce":284515388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2160385010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2160385010},{"type":"v1","addr":"172.21.15.28:6836","nonce":2160385010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3775145347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3775145347},{"type":"v1","addr":"172.21.15.28:6837","nonce":3775145347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:59.771956+0000","modified":"2023-11-28T11:11:03.109382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6834/3261856585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3261856585},{"type":"v1","addr":"172.21.15.59:6834","nonce":3261856585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:11:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.070+0000 7f2f15997700 1 -- 172.21.15.28:0/349404170 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2efc0682f0 msgr2=0x7f2efc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.070+0000 7f2f15997700 1 --2- 172.21.15.28:0/349404170 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2efc0682f0 0x7f2efc06a7a0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f2f04000f60 tx=0x7f2f04009040 comp rx=0 tx=0).stop 2023-11-28T11:11:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.070+0000 7f2f15997700 1 -- 172.21.15.28:0/349404170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f100b2660 msgr2=0x7f2f1015bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.070+0000 7f2f15997700 1 --2- 172.21.15.28:0/349404170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f100b2660 0x7f2f1015bae0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f2f0000e9b0 tx=0x7f2f0000ecc0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.070+0000 7f2f15997700 1 -- 172.21.15.28:0/349404170 shutdown_connections 2023-11-28T11:11:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.070+0000 7f2f15997700 1 --2- 172.21.15.28:0/349404170 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2efc0682f0 0x7f2efc06a7a0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.070+0000 7f2f15997700 1 --2- 172.21.15.28:0/349404170 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f100b2660 0x7f2f1015bae0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.070+0000 7f2f15997700 1 --2- 172.21.15.28:0/349404170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f1015a6c0 0x7f2f100af1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.070+0000 7f2f15997700 1 --2- 172.21.15.28:0/349404170 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f100d0020 0x7f2f1014f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.070+0000 7f2f15997700 1 -- 172.21.15.28:0/349404170 >> 172.21.15.28:0/349404170 conn(0x7f2f100cd660 msgr2=0x7f2f100cea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:10.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.070+0000 7f2f15997700 1 -- 172.21.15.28:0/349404170 shutdown_connections 2023-11-28T11:11:10.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.070+0000 7f2f15997700 1 -- 172.21.15.28:0/349404170 wait complete. 2023-11-28T11:11:10.073 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 25 2023-11-28T11:11:10.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:10:59.771956+0000', 'modified': '2023-11-28T11:11:03.109382+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6054}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6054': {'gid': 6054, 'name': 'd', 'rank': 0, 'incarnation': 21, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6834/3261856585', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6832', 'nonce': 3261856585}, {'type': 'v1', 'addr': '172.21.15.59:6834', 'nonce': 3261856585}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:11:10.487 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:11:10.487 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T11:11:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.617+0000 7f947e207700 1 Processor -- start 2023-11-28T11:11:10.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.617+0000 7f947e207700 1 -- start start 2023-11-28T11:11:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.618+0000 7f947e207700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f947815c990 0x7f94780cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.618+0000 7f947e207700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94780ce340 con 0x7f94780cc310 2023-11-28T11:11:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.618+0000 7f947e207700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94780cebe0 con 0x7f9478161090 2023-11-28T11:11:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.618+0000 7f947e207700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94780cb3f0 con 0x7f947815c990 2023-11-28T11:11:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.618+0000 7f9477fff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f94780cc310 0x7f94780cc770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38680/0 (socket says 172.21.15.28:38680) 2023-11-28T11:11:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.618+0000 7f9477fff700 1 -- 172.21.15.28:0/3905127417 learned_addr learned my addr 172.21.15.28:0/3905127417 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.618+0000 7f94777fe700 1 --2- 172.21.15.28:0/3905127417 >> v2:172.21.15.59:3300/0 conn(0x7f947815c990 0x7f94780cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.618+0000 7f94767fc700 1 -- 172.21.15.28:0/3905127417 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1245634202 0 0) 0x7f94780ce340 con 0x7f94780cc310 2023-11-28T11:11:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.618+0000 7f94767fc700 1 -- 172.21.15.28:0/3905127417 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94640035b0 con 0x7f94780cc310 2023-11-28T11:11:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94767fc700 1 -- 172.21.15.28:0/3905127417 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3013158935 0 0) 0x7f94640035b0 con 0x7f94780cc310 2023-11-28T11:11:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94767fc700 1 -- 172.21.15.28:0/3905127417 >> v2:172.21.15.59:3300/0 conn(0x7f947815c990 msgr2=0x7f94780cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94767fc700 1 --2- 172.21.15.28:0/3905127417 >> v2:172.21.15.59:3300/0 conn(0x7f947815c990 0x7f94780cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94767fc700 1 -- 172.21.15.28:0/3905127417 >> v1:172.21.15.59:6789/0 conn(0x7f9478161090 legacy=0x7f94780ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94767fc700 1 -- 172.21.15.28:0/3905127417 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780ce590 con 0x7f94780cc310 2023-11-28T11:11:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94767fc700 1 -- 172.21.15.28:0/3905127417 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f94700019f0 con 0x7f94780cc310 2023-11-28T11:11:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94767fc700 1 -- 172.21.15.28:0/3905127417 >> v1:172.21.15.28:6789/0 conn(0x7f94780cc310 legacy=0x7f94780cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94767fc700 1 --2- 172.21.15.28:0/3905127417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9464003a00 0x7f9464003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94767fc700 1 --2- 172.21.15.28:0/3905127417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9464004850 0x7f9464004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94767fc700 1 --2- 172.21.15.28:0/3905127417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9464005320 0x7f946400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94767fc700 1 -- 172.21.15.28:0/3905127417 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94780cb3f0 con 0x7f9464003a00 2023-11-28T11:11:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94767fc700 1 -- 172.21.15.28:0/3905127417 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f946400c250 con 0x7f9464004850 2023-11-28T11:11:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94767fc700 1 -- 172.21.15.28:0/3905127417 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f946400c410 con 0x7f9464005320 2023-11-28T11:11:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94777fe700 1 --2- 172.21.15.28:0/3905127417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9464003a00 0x7f9464003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f9476ffd700 1 --2- 172.21.15.28:0/3905127417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9464004850 0x7f9464004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f9477fff700 1 --2- 172.21.15.28:0/3905127417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9464005320 0x7f946400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94777fe700 1 -- 172.21.15.28:0/3905127417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9464005320 msgr2=0x7f946400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94777fe700 1 --2- 172.21.15.28:0/3905127417 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9464005320 0x7f946400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94777fe700 1 -- 172.21.15.28:0/3905127417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9464004850 msgr2=0x7f9464004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94777fe700 1 --2- 172.21.15.28:0/3905127417 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9464004850 0x7f9464004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f9477fff700 1 -- 172.21.15.28:0/3905127417 reap_dead start 2023-11-28T11:11:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.619+0000 7f94777fe700 1 -- 172.21.15.28:0/3905127417 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94640035b0 con 0x7f9464003a00 2023-11-28T11:11:10.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.620+0000 7f94777fe700 1 --2- 172.21.15.28:0/3905127417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9464003a00 0x7f9464003e10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9460009b60 tx=0x7f9464005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5ddc69f8ec1d058 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.620+0000 7f94767fc700 1 -- 172.21.15.28:0/3905127417 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946000e6c0 con 0x7f9464003a00 2023-11-28T11:11:10.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.620+0000 7f94767fc700 1 -- 172.21.15.28:0/3905127417 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f946000e820 con 0x7f9464003a00 2023-11-28T11:11:10.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.620+0000 7f94767fc700 1 -- 172.21.15.28:0/3905127417 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946000eb90 con 0x7f9464003a00 2023-11-28T11:11:10.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.620+0000 7f947e207700 1 -- 172.21.15.28:0/3905127417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9464003a00 msgr2=0x7f9464003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.620+0000 7f947e207700 1 --2- 172.21.15.28:0/3905127417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9464003a00 0x7f9464003e10 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f9460009b60 tx=0x7f9464005880 comp rx=0 tx=0).stop 2023-11-28T11:11:10.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.620+0000 7f947e207700 1 -- 172.21.15.28:0/3905127417 shutdown_connections 2023-11-28T11:11:10.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.620+0000 7f947e207700 1 --2- 172.21.15.28:0/3905127417 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9464003a00 0x7f9464003e10 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.620+0000 7f947e207700 1 -- 172.21.15.28:0/3905127417 >> 172.21.15.28:0/3905127417 conn(0x7f94780b1650 msgr2=0x7f94780b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:10.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.620+0000 7f947e207700 1 -- 172.21.15.28:0/3905127417 shutdown_connections 2023-11-28T11:11:10.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.620+0000 7f947e207700 1 -- 172.21.15.28:0/3905127417 wait complete. 2023-11-28T11:11:10.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.621+0000 7f947e207700 1 Processor -- start 2023-11-28T11:11:10.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.622+0000 7f947e207700 1 -- start start 2023-11-28T11:11:10.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.622+0000 7f947e207700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9464003a00 0x7f94780c5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.622+0000 7f947e207700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94780c3390 0x7f9478159fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.630 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.622+0000 7f947e207700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94780ad8d0 0x7f94780add30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.622+0000 7f947e207700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94780ae270 con 0x7f9464003a00 2023-11-28T11:11:10.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.622+0000 7f947e207700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94780c4410 con 0x7f94780c3390 2023-11-28T11:11:10.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.622+0000 7f947e207700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94780c3b70 con 0x7f94780ad8d0 2023-11-28T11:11:10.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.622+0000 7f94777fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9464003a00 0x7f94780c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.622+0000 7f94777fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9464003a00 0x7f94780c5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50634/0 (socket says 172.21.15.28:50634) 2023-11-28T11:11:10.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.622+0000 7f94777fe700 1 -- 172.21.15.28:0/3915473304 learned_addr learned my addr 172.21.15.28:0/3915473304 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:10.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.622+0000 7f9477fff700 1 --2- 172.21.15.28:0/3915473304 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94780ad8d0 0x7f94780add30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.623+0000 7f9476ffd700 1 --2- 172.21.15.28:0/3915473304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94780c3390 0x7f9478159fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.623+0000 7f94777fe700 1 -- 172.21.15.28:0/3915473304 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94780ad8d0 msgr2=0x7f94780add30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.623+0000 7f94777fe700 1 --2- 172.21.15.28:0/3915473304 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94780ad8d0 0x7f94780add30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.631 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.623+0000 7f94777fe700 1 -- 172.21.15.28:0/3915473304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94780c3390 msgr2=0x7f9478159fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.623+0000 7f94777fe700 1 --2- 172.21.15.28:0/3915473304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94780c3390 0x7f9478159fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.623+0000 7f94777fe700 1 -- 172.21.15.28:0/3915473304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94780cc770 con 0x7f9464003a00 2023-11-28T11:11:10.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.623+0000 7f94777fe700 1 --2- 172.21.15.28:0/3915473304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9464003a00 0x7f94780c5900 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f946400c220 tx=0x7f94600047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.623+0000 7f946ffff700 1 -- 172.21.15.28:0/3915473304 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f946000e6c0 con 0x7f9464003a00 2023-11-28T11:11:10.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.623+0000 7f946ffff700 1 -- 172.21.15.28:0/3915473304 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9460003ad0 con 0x7f9464003a00 2023-11-28T11:11:10.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.623+0000 7f946ffff700 1 -- 172.21.15.28:0/3915473304 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9460003e80 con 0x7f9464003a00 2023-11-28T11:11:10.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.623+0000 7f947e207700 1 -- 172.21.15.28:0/3915473304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94780ce870 con 0x7f9464003a00 2023-11-28T11:11:10.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.623+0000 7f947e207700 1 -- 172.21.15.28:0/3915473304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94780cc900 con 0x7f9464003a00 2023-11-28T11:11:10.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.625+0000 7f946ffff700 1 -- 172.21.15.28:0/3915473304 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9460019030 con 0x7f9464003a00 2023-11-28T11:11:10.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.625+0000 7f946ffff700 1 --2- 172.21.15.28:0/3915473304 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94580682f0 0x7f945806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.625+0000 7f946ffff700 1 -- 172.21.15.28:0/3915473304 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9460087340 con 0x7f9464003a00 2023-11-28T11:11:10.632 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.625+0000 7f9476ffd700 1 --2- 172.21.15.28:0/3915473304 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94580682f0 0x7f945806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.625+0000 7f9476ffd700 1 --2- 172.21.15.28:0/3915473304 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94580682f0 0x7f945806a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9468000c00 tx=0x7f9468009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.627+0000 7f948a390b80 1 Processor -- start 2023-11-28T11:11:10.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.627+0000 7f948a390b80 1 -- start start 2023-11-28T11:11:10.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.628+0000 7f948a390b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55596810 0x55fe5556ca20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.628+0000 7f948a390b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55594130 0x55fe55596be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.628+0000 7f948a390b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55597120 0x55fe555b2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.628+0000 7f948a390b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55456bc0 con 0x55fe55594130 2023-11-28T11:11:10.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.628+0000 7f948a390b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe554c5350 con 0x55fe55596810 2023-11-28T11:11:10.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.628+0000 7f948a390b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe554be880 con 0x55fe55597120 2023-11-28T11:11:10.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.628+0000 7f9476ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55594130 0x55fe55596be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.628+0000 7f9476ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55594130 0x55fe55596be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50644/0 (socket says 172.21.15.28:50644) 2023-11-28T11:11:10.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.628+0000 7f9476ffd700 1 -- 172.21.15.28:0/624403738 learned_addr learned my addr 172.21.15.28:0/624403738 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:10.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.628+0000 7f9476ffd700 1 -- 172.21.15.28:0/624403738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55597120 msgr2=0x55fe555b2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.628+0000 7f9476ffd700 1 --2- 172.21.15.28:0/624403738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55597120 0x55fe555b2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.628+0000 7f9476ffd700 1 -- 172.21.15.28:0/624403738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55596810 msgr2=0x55fe5556ca20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.628+0000 7f9476ffd700 1 --2- 172.21.15.28:0/624403738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55596810 0x55fe5556ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.628+0000 7f9476ffd700 1 -- 172.21.15.28:0/624403738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe553405e0 con 0x55fe55594130 2023-11-28T11:11:10.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.629+0000 7f9476ffd700 1 --2- 172.21.15.28:0/624403738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55594130 0x55fe55596be0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f94680128b0 tx=0x7f9468015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1133114a9a9c7254 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.629+0000 7f946d7fa700 1 -- 172.21.15.28:0/624403738 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94680156d0 con 0x55fe55594130 2023-11-28T11:11:10.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.629+0000 7f946d7fa700 1 -- 172.21.15.28:0/624403738 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9468015830 con 0x55fe55594130 2023-11-28T11:11:10.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.629+0000 7f946d7fa700 1 -- 172.21.15.28:0/624403738 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468015ba0 con 0x55fe55594130 2023-11-28T11:11:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.629+0000 7f948a390b80 1 -- 172.21.15.28:0/624403738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55594130 msgr2=0x55fe55596be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.629+0000 7f948a390b80 1 --2- 172.21.15.28:0/624403738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55594130 0x55fe55596be0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f94680128b0 tx=0x7f9468015000 comp rx=0 tx=0).stop 2023-11-28T11:11:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.629+0000 7f948a390b80 1 -- 172.21.15.28:0/624403738 shutdown_connections 2023-11-28T11:11:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.629+0000 7f948a390b80 1 --2- 172.21.15.28:0/624403738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55597120 0x55fe555b2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.629+0000 7f948a390b80 1 --2- 172.21.15.28:0/624403738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55596810 0x55fe5556ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.629+0000 7f948a390b80 1 --2- 172.21.15.28:0/624403738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55594130 0x55fe55596be0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.629+0000 7f948a390b80 1 -- 172.21.15.28:0/624403738 >> 172.21.15.28:0/624403738 conn(0x55fe55569d60 msgr2=0x55fe5556a6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.630+0000 7f948a390b80 1 -- 172.21.15.28:0/624403738 shutdown_connections 2023-11-28T11:11:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.630+0000 7f948a390b80 1 -- 172.21.15.28:0/624403738 wait complete. 2023-11-28T11:11:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.630+0000 7f948a390b80 1 -- start start 2023-11-28T11:11:10.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.630+0000 7f948a390b80 1 Processor -- start 2023-11-28T11:11:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f948a390b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55594130 0x55fe555b9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f948a390b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55596810 0x55fe555979d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f948a390b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55597120 0x55fe5558d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f948a390b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe553428d0 con 0x55fe55596810 2023-11-28T11:11:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f948a390b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5558e010 con 0x55fe55597120 2023-11-28T11:11:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f948a390b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe55598220 con 0x55fe55594130 2023-11-28T11:11:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f948a390b80 10 client.0 _ll_register_callbacks cb 0x55fe55364240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f9476ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55596810 0x55fe555979d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f9476ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55596810 0x55fe555979d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50650/0 (socket says 172.21.15.28:50650) 2023-11-28T11:11:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f9476ffd700 1 -- 172.21.15.28:0/1880482262 learned_addr learned my addr 172.21.15.28:0/1880482262 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:10.640 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f9476ffd700 1 -- 172.21.15.28:0/1880482262 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55594130 msgr2=0x55fe555b9780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f9476ffd700 1 --2- 172.21.15.28:0/1880482262 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55594130 0x55fe555b9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f9476ffd700 1 -- 172.21.15.28:0/1880482262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55597120 msgr2=0x55fe5558d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f9476ffd700 1 --2- 172.21.15.28:0/1880482262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55597120 0x55fe5558d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f9476ffd700 1 -- 172.21.15.28:0/1880482262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94780cebe0 con 0x55fe55596810 2023-11-28T11:11:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f9476ffd700 1 --2- 172.21.15.28:0/1880482262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55596810 0x55fe555979d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9468022000 tx=0x7f9468001120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c25782aaf52c9e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.631+0000 7f94567fc700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:11:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.632+0000 7f94567fc700 1 -- 172.21.15.28:0/1880482262 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9468005220 con 0x55fe55596810 2023-11-28T11:11:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.632+0000 7f94567fc700 1 -- 172.21.15.28:0/1880482262 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9468005380 con 0x55fe55596810 2023-11-28T11:11:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.632+0000 7f948a390b80 1 -- 172.21.15.28:0/1880482262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe5548c140 con 0x55fe55596810 2023-11-28T11:11:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.632+0000 7f94567fc700 1 -- 172.21.15.28:0/1880482262 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9468005580 con 0x55fe55596810 2023-11-28T11:11:10.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.632+0000 7f948a390b80 10 client.6228 fetch_fsmap learned FSMap version 25 2023-11-28T11:11:10.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.632+0000 7f948a390b80 1 -- 172.21.15.28:0/1880482262 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55fe552ecd90 con 0x55fe55596810 2023-11-28T11:11:10.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.632+0000 7f94567fc700 1 -- 172.21.15.28:0/1880482262 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fe5548c140 con 0x55fe55596810 2023-11-28T11:11:10.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.632+0000 7f94567fc700 1 -- 172.21.15.28:0/1880482262 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f946802c6e0 con 0x55fe55596810 2023-11-28T11:11:10.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.632+0000 7f948a390b80 10 client.6228 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:11:10.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.632+0000 7f948a390b80 10 client.6228 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:11:10.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.632+0000 7f948a390b80 20 client.6228 populate_metadata read hostname 'smithi028' 2023-11-28T11:11:10.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.632+0000 7f948a390b80 1 --2- 172.21.15.28:0/1880482262 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55fe5558e430 0x55fe555b6080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.632+0000 7f948a390b80 4 client.6228 mds_command: new command op to 6054 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:11:10.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.632+0000 7f948a390b80 1 -- 172.21.15.28:0/1880482262 --> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55fe55342f10 con 0x55fe5558e430 2023-11-28T11:11:10.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.632+0000 7f94777fe700 1 --2- 172.21.15.28:0/1880482262 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55fe5558e430 0x55fe555b6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.633+0000 7f94567fc700 10 client.6228 ms_handle_connect on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:10.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.633+0000 7f94777fe700 1 --2- 172.21.15.28:0/1880482262 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55fe5558e430 0x55fe555b6080 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=160a861cc1abf3a5 server_cookie=30201cd8580bd4bf in_seq=0 out_seq=0 2023-11-28T11:11:10.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.633+0000 7f94567fc700 1 -- 172.21.15.28:0/1880482262 <== mds.0 v2:172.21.15.59:6832/3261856585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55fe55342f10 con 0x55fe5558e430 2023-11-28T11:11:10.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.633+0000 7f94567fc700 10 client.6228 handle_command_reply: tid=0 2023-11-28T11:11:10.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.633+0000 7f948a390b80 1 client.6228 shutdown 2023-11-28T11:11:10.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.634+0000 7f948a390b80 1 -- 172.21.15.28:0/1880482262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55596810 msgr2=0x55fe555979d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.634+0000 7f948a390b80 1 --2- 172.21.15.28:0/1880482262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55596810 0x55fe555979d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f9468022000 tx=0x7f9468001120 comp rx=0 tx=0).stop 2023-11-28T11:11:10.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.634+0000 7f948a390b80 1 -- 172.21.15.28:0/1880482262 shutdown_connections 2023-11-28T11:11:10.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.634+0000 7f948a390b80 1 --2- 172.21.15.28:0/1880482262 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55fe5558e430 0x55fe555b6080 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.634+0000 7f948a390b80 1 --2- 172.21.15.28:0/1880482262 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55597120 0x55fe5558d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.634+0000 7f948a390b80 1 --2- 172.21.15.28:0/1880482262 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55594130 0x55fe555b9780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.634+0000 7f94567fc700 0 client.6228 ms_handle_reset on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:10.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.634+0000 7f948a390b80 1 --2- 172.21.15.28:0/1880482262 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55596810 0x55fe555979d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.634+0000 7f948a390b80 1 -- 172.21.15.28:0/1880482262 >> 172.21.15.28:0/1880482262 conn(0x55fe55569d60 msgr2=0x55fe5556ca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:10.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.634+0000 7f948a390b80 1 -- 172.21.15.28:0/1880482262 shutdown_connections 2023-11-28T11:11:10.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.634+0000 7f948a390b80 1 -- 172.21.15.28:0/1880482262 wait complete. 2023-11-28T11:11:10.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.634+0000 7f948a390b80 20 client.6228 trim_cache size 0 max 16384 2023-11-28T11:11:10.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.651+0000 7f948a390b80 1 Processor -- start 2023-11-28T11:11:10.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.651+0000 7f948a390b80 1 -- start start 2023-11-28T11:11:10.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.652+0000 7f948a390b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55594130 0x55fe555b9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.652+0000 7f948a390b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55596810 0x55fe55565560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.652+0000 7f948a390b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55597120 0x55fe555b6fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.652+0000 7f948a390b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5556b6e0 con 0x55fe55597120 2023-11-28T11:11:10.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.652+0000 7f948a390b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5556b4d0 con 0x55fe55594130 2023-11-28T11:11:10.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.652+0000 7f948a390b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe5558c640 con 0x55fe55596810 2023-11-28T11:11:10.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.652+0000 7f9477fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55597120 0x55fe555b6fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.652+0000 7f9477fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55597120 0x55fe555b6fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50660/0 (socket says 172.21.15.28:50660) 2023-11-28T11:11:10.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.652+0000 7f9477fff700 1 -- 172.21.15.28:0/2426710846 learned_addr learned my addr 172.21.15.28:0/2426710846 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:10.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.653+0000 7f9477fff700 1 -- 172.21.15.28:0/2426710846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55596810 msgr2=0x55fe55565560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.653+0000 7f9477fff700 1 --2- 172.21.15.28:0/2426710846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55596810 0x55fe55565560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.653+0000 7f9477fff700 1 -- 172.21.15.28:0/2426710846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55594130 msgr2=0x55fe555b9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.653+0000 7f9477fff700 1 --2- 172.21.15.28:0/2426710846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55594130 0x55fe555b9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.653+0000 7f9477fff700 1 -- 172.21.15.28:0/2426710846 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55fe55485140 con 0x55fe55597120 2023-11-28T11:11:10.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.653+0000 7f9477fff700 1 --2- 172.21.15.28:0/2426710846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55597120 0x55fe555b6fe0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9478161460 tx=0x7f94700073a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=761c3b3a9679cfce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.653+0000 7f946dffb700 1 -- 172.21.15.28:0/2426710846 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470007a90 con 0x55fe55597120 2023-11-28T11:11:10.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.653+0000 7f946dffb700 1 -- 172.21.15.28:0/2426710846 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9470015440 con 0x55fe55597120 2023-11-28T11:11:10.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.653+0000 7f946dffb700 1 -- 172.21.15.28:0/2426710846 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94700155f0 con 0x55fe55597120 2023-11-28T11:11:10.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.653+0000 7f948a390b80 1 -- 172.21.15.28:0/2426710846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55597120 msgr2=0x55fe555b6fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.653+0000 7f948a390b80 1 --2- 172.21.15.28:0/2426710846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55597120 0x55fe555b6fe0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f9478161460 tx=0x7f94700073a0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.654+0000 7f948a390b80 1 -- 172.21.15.28:0/2426710846 shutdown_connections 2023-11-28T11:11:10.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.654+0000 7f948a390b80 1 --2- 172.21.15.28:0/2426710846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55597120 0x55fe555b6fe0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.654+0000 7f948a390b80 1 --2- 172.21.15.28:0/2426710846 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55594130 0x55fe555b9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.654+0000 7f948a390b80 1 --2- 172.21.15.28:0/2426710846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55596810 0x55fe55565560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.654+0000 7f948a390b80 1 -- 172.21.15.28:0/2426710846 >> 172.21.15.28:0/2426710846 conn(0x55fe55569d60 msgr2=0x55fe5558c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:10.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.654+0000 7f948a390b80 1 -- 172.21.15.28:0/2426710846 shutdown_connections 2023-11-28T11:11:10.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.654+0000 7f948a390b80 1 -- 172.21.15.28:0/2426710846 wait complete. 2023-11-28T11:11:10.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.654+0000 7f948a390b80 1 -- start start 2023-11-28T11:11:10.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.654+0000 7f948a390b80 1 Processor -- start 2023-11-28T11:11:10.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f948a390b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55594130 0x55fe55616fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f948a390b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55596810 0x55fe5556bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f948a390b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55597120 0x55fe55599180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f948a390b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe55617fb0 con 0x55fe55597120 2023-11-28T11:11:10.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f948a390b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55fe5556c000 con 0x55fe55594130 2023-11-28T11:11:10.662 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f948a390b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55fe555b6460 con 0x55fe55596810 2023-11-28T11:11:10.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f948a390b80 10 client.0 _ll_register_callbacks cb 0x55fe55364240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:10.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f9477fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55597120 0x55fe55599180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f9477fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55597120 0x55fe55599180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50666/0 (socket says 172.21.15.28:50666) 2023-11-28T11:11:10.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f9477fff700 1 -- 172.21.15.28:0/712818179 learned_addr learned my addr 172.21.15.28:0/712818179 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:10.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f94777fe700 1 --2- 172.21.15.28:0/712818179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55594130 0x55fe55616fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f9476ffd700 1 --2- 172.21.15.28:0/712818179 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55596810 0x55fe5556bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f9477fff700 1 -- 172.21.15.28:0/712818179 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55596810 msgr2=0x55fe5556bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f9477fff700 1 --2- 172.21.15.28:0/712818179 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55596810 0x55fe5556bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f9477fff700 1 -- 172.21.15.28:0/712818179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55594130 msgr2=0x55fe55616fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f9477fff700 1 --2- 172.21.15.28:0/712818179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55594130 0x55fe55616fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f9477fff700 1 -- 172.21.15.28:0/712818179 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94780ce590 con 0x55fe55597120 2023-11-28T11:11:10.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.655+0000 7f9476ffd700 1 --2- 172.21.15.28:0/712818179 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55596810 0x55fe5556bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:10.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f9477fff700 1 --2- 172.21.15.28:0/712818179 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55597120 0x55fe55599180 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9470018040 tx=0x7f9470000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9d73b065b6f35ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f94567fc700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:11:10.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f94567fc700 1 -- 172.21.15.28:0/712818179 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9470029670 con 0x55fe55597120 2023-11-28T11:11:10.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f94567fc700 1 -- 172.21.15.28:0/712818179 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f94700297d0 con 0x55fe55597120 2023-11-28T11:11:10.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f948a390b80 1 -- 172.21.15.28:0/712818179 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55fe5547c160 con 0x55fe55597120 2023-11-28T11:11:10.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f94567fc700 1 -- 172.21.15.28:0/712818179 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9470007b70 con 0x55fe55597120 2023-11-28T11:11:10.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f948a390b80 10 client.6234 fetch_fsmap learned FSMap version 25 2023-11-28T11:11:10.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f948a390b80 1 -- 172.21.15.28:0/712818179 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55fe55313200 con 0x55fe55597120 2023-11-28T11:11:10.664 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f94567fc700 1 -- 172.21.15.28:0/712818179 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55fe5547c160 con 0x55fe55597120 2023-11-28T11:11:10.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f94567fc700 1 -- 172.21.15.28:0/712818179 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9470012050 con 0x55fe55597120 2023-11-28T11:11:10.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f948a390b80 10 client.6234 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:11:10.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f948a390b80 10 client.6234 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:11:10.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f948a390b80 20 client.6234 populate_metadata read hostname 'smithi028' 2023-11-28T11:11:10.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f948a390b80 1 --2- 172.21.15.28:0/712818179 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55fe555b6fa0 0x55fe555b7370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f948a390b80 4 client.6234 mds_command: new command op to 6054 tid=0[{"prefix": "status"}] 2023-11-28T11:11:10.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f948a390b80 1 -- 172.21.15.28:0/712818179 --> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55fe55466170 con 0x55fe555b6fa0 2023-11-28T11:11:10.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.656+0000 7f94777fe700 1 --2- 172.21.15.28:0/712818179 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55fe555b6fa0 0x55fe555b7370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f94567fc700 10 client.6234 ms_handle_connect on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:10.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f94777fe700 1 --2- 172.21.15.28:0/712818179 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55fe555b6fa0 0x55fe555b7370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11f2618cb7fb6c2c server_cookie=72723ae389292de6 in_seq=0 out_seq=0 2023-11-28T11:11:10.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f94567fc700 1 -- 172.21.15.28:0/712818179 <== mds.0 v2:172.21.15.59:6832/3261856585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55fe55466170 con 0x55fe555b6fa0 2023-11-28T11:11:10.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f94567fc700 10 client.6234 handle_command_reply: tid=0 2023-11-28T11:11:10.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f948a390b80 1 client.6234 shutdown 2023-11-28T11:11:10.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f948a390b80 1 -- 172.21.15.28:0/712818179 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55597120 msgr2=0x55fe55599180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f948a390b80 1 --2- 172.21.15.28:0/712818179 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55597120 0x55fe55599180 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f9470018040 tx=0x7f9470000f40 comp rx=0 tx=0).stop 2023-11-28T11:11:10.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f948a390b80 1 -- 172.21.15.28:0/712818179 shutdown_connections 2023-11-28T11:11:10.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f948a390b80 1 --2- 172.21.15.28:0/712818179 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55fe555b6fa0 0x55fe555b7370 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f948a390b80 1 --2- 172.21.15.28:0/712818179 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55fe55597120 0x55fe55599180 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f94567fc700 0 client.6234 ms_handle_reset on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:10.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f948a390b80 1 --2- 172.21.15.28:0/712818179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fe55594130 0x55fe55616fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f948a390b80 1 --2- 172.21.15.28:0/712818179 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55fe55596810 0x55fe5556bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f948a390b80 1 -- 172.21.15.28:0/712818179 >> 172.21.15.28:0/712818179 conn(0x55fe55569d60 msgr2=0x55fe5558c7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:10.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f948a390b80 1 -- 172.21.15.28:0/712818179 shutdown_connections 2023-11-28T11:11:10.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f948a390b80 1 -- 172.21.15.28:0/712818179 wait complete. 2023-11-28T11:11:10.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.657+0000 7f948a390b80 20 client.6234 trim_cache size 0 max 16384 2023-11-28T11:11:10.667 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:11:10.667 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "6e299adc-7a2a-4cf1-afa2-7ce104b29a51", 2023-11-28T11:11:10.667 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2023-11-28T11:11:10.667 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 6054, 2023-11-28T11:11:10.667 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2023-11-28T11:11:10.667 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2023-11-28T11:11:10.668 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2023-11-28T11:11:10.668 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 10.884890715999999, 2023-11-28T11:11:10.668 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 25, 2023-11-28T11:11:10.668 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 54, 2023-11-28T11:11:10.668 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 54, 2023-11-28T11:11:10.668 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 16.209034093 2023-11-28T11:11:10.668 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:11:10.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.658+0000 7f947e207700 1 -- 172.21.15.28:0/3915473304 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94580682f0 msgr2=0x7f945806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.658+0000 7f947e207700 1 --2- 172.21.15.28:0/3915473304 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94580682f0 0x7f945806a7a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f9468000c00 tx=0x7f9468009040 comp rx=0 tx=0).stop 2023-11-28T11:11:10.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.658+0000 7f947e207700 1 -- 172.21.15.28:0/3915473304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9464003a00 msgr2=0x7f94780c5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.658+0000 7f947e207700 1 --2- 172.21.15.28:0/3915473304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9464003a00 0x7f94780c5900 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f946400c220 tx=0x7f94600047c0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.658+0000 7f947e207700 1 -- 172.21.15.28:0/3915473304 shutdown_connections 2023-11-28T11:11:10.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.658+0000 7f947e207700 1 --2- 172.21.15.28:0/3915473304 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f94580682f0 0x7f945806a7a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.658+0000 7f947e207700 1 --2- 172.21.15.28:0/3915473304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9464003a00 0x7f94780c5900 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.658+0000 7f947e207700 1 --2- 172.21.15.28:0/3915473304 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f94780ad8d0 0x7f94780add30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.658+0000 7f947e207700 1 --2- 172.21.15.28:0/3915473304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f94780c3390 0x7f9478159fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.658+0000 7f947e207700 1 -- 172.21.15.28:0/3915473304 >> 172.21.15.28:0/3915473304 conn(0x7f94780b1650 msgr2=0x7f947806cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:10.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.658+0000 7f947e207700 1 -- 172.21.15.28:0/3915473304 shutdown_connections 2023-11-28T11:11:10.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.658+0000 7f947e207700 1 -- 172.21.15.28:0/3915473304 wait complete. 2023-11-28T11:11:10.672 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:11:10.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.794+0000 7ff9c1e4c700 1 Processor -- start 2023-11-28T11:11:10.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.794+0000 7ff9c1e4c700 1 -- start start 2023-11-28T11:11:10.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.795+0000 7ff9c1e4c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff9bc06b930 0x7ff9bc0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.795+0000 7ff9c1e4c700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9bc0cb570 con 0x7ff9bc0cc4b0 2023-11-28T11:11:10.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.795+0000 7ff9c1e4c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9bc0cc110 con 0x7ff9bc15a5a0 2023-11-28T11:11:10.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.795+0000 7ff9c1e4c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc06bda0 con 0x7ff9bc06b930 2023-11-28T11:11:10.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.795+0000 7ff9bbfff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff9bc0cc4b0 0x7ff9bc0c35f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38732/0 (socket says 172.21.15.28:38732) 2023-11-28T11:11:10.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.795+0000 7ff9bbfff700 1 -- 172.21.15.28:0/2788109831 learned_addr learned my addr 172.21.15.28:0/2788109831 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:10.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9baffd700 1 --2- 172.21.15.28:0/2788109831 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff9bc06b930 0x7ff9bc0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9ba7fc700 1 -- 172.21.15.28:0/2788109831 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737870730 0 0) 0x7ff9bc0cb570 con 0x7ff9bc0cc4b0 2023-11-28T11:11:10.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9ba7fc700 1 -- 172.21.15.28:0/2788109831 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9a80035b0 con 0x7ff9bc0cc4b0 2023-11-28T11:11:10.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9ba7fc700 1 -- 172.21.15.28:0/2788109831 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1586480798 0 0) 0x7ff9a80035b0 con 0x7ff9bc0cc4b0 2023-11-28T11:11:10.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9ba7fc700 1 -- 172.21.15.28:0/2788109831 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff9bc06b930 msgr2=0x7ff9bc0ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9ba7fc700 1 --2- 172.21.15.28:0/2788109831 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff9bc06b930 0x7ff9bc0ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9ba7fc700 1 -- 172.21.15.28:0/2788109831 >> v1:172.21.15.59:6789/0 conn(0x7ff9bc15a5a0 legacy=0x7ff9bc0cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9ba7fc700 1 -- 172.21.15.28:0/2788109831 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc0ca960 con 0x7ff9bc0cc4b0 2023-11-28T11:11:10.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9ba7fc700 1 -- 172.21.15.28:0/2788109831 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7ff9b40019f0 con 0x7ff9bc0cc4b0 2023-11-28T11:11:10.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9ba7fc700 1 -- 172.21.15.28:0/2788109831 >> v1:172.21.15.28:6789/0 conn(0x7ff9bc0cc4b0 legacy=0x7ff9bc0c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9ba7fc700 1 --2- 172.21.15.28:0/2788109831 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff9bc06b930 0x7ff9a8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9ba7fc700 1 --2- 172.21.15.28:0/2788109831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9a8003f70 0x7ff9a8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9ba7fc700 1 --2- 172.21.15.28:0/2788109831 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9a8004d90 0x7ff9a8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9ba7fc700 1 -- 172.21.15.28:0/2788109831 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc06bda0 con 0x7ff9a8004d90 2023-11-28T11:11:10.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9ba7fc700 1 -- 172.21.15.28:0/2788109831 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a800ba40 con 0x7ff9a8003f70 2023-11-28T11:11:10.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9ba7fc700 1 -- 172.21.15.28:0/2788109831 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9a800bc00 con 0x7ff9bc06b930 2023-11-28T11:11:10.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9bbfff700 1 --2- 172.21.15.28:0/2788109831 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9a8004d90 0x7ff9a8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9bb7fe700 1 --2- 172.21.15.28:0/2788109831 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff9bc06b930 0x7ff9a8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.796+0000 7ff9baffd700 1 --2- 172.21.15.28:0/2788109831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9a8003f70 0x7ff9a8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.797+0000 7ff9bbfff700 1 -- 172.21.15.28:0/2788109831 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff9bc06b930 msgr2=0x7ff9a8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.797+0000 7ff9bbfff700 1 --2- 172.21.15.28:0/2788109831 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff9bc06b930 0x7ff9a8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.797+0000 7ff9bbfff700 1 -- 172.21.15.28:0/2788109831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9a8003f70 msgr2=0x7ff9a8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:10.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.797+0000 7ff9bbfff700 1 --2- 172.21.15.28:0/2788109831 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9a8003f70 0x7ff9a8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.797+0000 7ff9bbfff700 1 -- 172.21.15.28:0/2788109831 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a80035b0 con 0x7ff9a8004d90 2023-11-28T11:11:10.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.797+0000 7ff9bbfff700 1 --2- 172.21.15.28:0/2788109831 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9a8004d90 0x7ff9a8005200 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400a830 tx=0x7ff9a8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbbc2f3b1093a5d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.797+0000 7ff9ba7fc700 1 -- 172.21.15.28:0/2788109831 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4012600 con 0x7ff9a8004d90 2023-11-28T11:11:10.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.797+0000 7ff9ba7fc700 1 -- 172.21.15.28:0/2788109831 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9b4012780 con 0x7ff9a8004d90 2023-11-28T11:11:10.805 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.797+0000 7ff9ba7fc700 1 -- 172.21.15.28:0/2788109831 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9b4012b30 con 0x7ff9a8004d90 2023-11-28T11:11:10.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.797+0000 7ff9c1e4c700 1 -- 172.21.15.28:0/2788109831 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9a8004d90 msgr2=0x7ff9a8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.797+0000 7ff9c1e4c700 1 --2- 172.21.15.28:0/2788109831 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9a8004d90 0x7ff9a8005200 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7ff9b400a830 tx=0x7ff9a8005880 comp rx=0 tx=0).stop 2023-11-28T11:11:10.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.797+0000 7ff9bbfff700 1 -- 172.21.15.28:0/2788109831 reap_dead start 2023-11-28T11:11:10.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.797+0000 7ff9c1e4c700 1 -- 172.21.15.28:0/2788109831 shutdown_connections 2023-11-28T11:11:10.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.797+0000 7ff9c1e4c700 1 -- 172.21.15.28:0/2788109831 >> 172.21.15.28:0/2788109831 conn(0x7ff9bc0b1650 msgr2=0x7ff9bc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:10.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.798+0000 7ff9c1e4c700 1 -- 172.21.15.28:0/2788109831 shutdown_connections 2023-11-28T11:11:10.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.798+0000 7ff9c1e4c700 1 -- 172.21.15.28:0/2788109831 wait complete. 2023-11-28T11:11:10.810 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.799+0000 7ff9c1e4c700 1 Processor -- start 2023-11-28T11:11:10.810 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.799+0000 7ff9c1e4c700 1 -- start start 2023-11-28T11:11:10.810 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9c1e4c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9a8004d90 0x7ff9bc15c880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.810 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9c1e4c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9bc14d230 0x7ff9bc15a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9c1e4c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff9bc0b4b20 0x7ff9bc0b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9c1e4c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc0b54d0 con 0x7ff9a8004d90 2023-11-28T11:11:10.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9c1e4c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc15aeb0 con 0x7ff9bc14d230 2023-11-28T11:11:10.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9c1e4c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9bc15b980 con 0x7ff9bc0b4b20 2023-11-28T11:11:10.811 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9bb7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9a8004d90 0x7ff9bc15c880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9bb7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9a8004d90 0x7ff9bc15c880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50682/0 (socket says 172.21.15.28:50682) 2023-11-28T11:11:10.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9bb7fe700 1 -- 172.21.15.28:0/1790603668 learned_addr learned my addr 172.21.15.28:0/1790603668 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:10.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9bbfff700 1 --2- 172.21.15.28:0/1790603668 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff9bc0b4b20 0x7ff9bc0b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9baffd700 1 --2- 172.21.15.28:0/1790603668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9bc14d230 0x7ff9bc15a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9bb7fe700 1 -- 172.21.15.28:0/1790603668 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff9bc0b4b20 msgr2=0x7ff9bc0b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9bb7fe700 1 --2- 172.21.15.28:0/1790603668 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff9bc0b4b20 0x7ff9bc0b4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9bb7fe700 1 -- 172.21.15.28:0/1790603668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9bc14d230 msgr2=0x7ff9bc15a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9bb7fe700 1 --2- 172.21.15.28:0/1790603668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9bc14d230 0x7ff9bc15a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9bb7fe700 1 -- 172.21.15.28:0/1790603668 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9bc0caf20 con 0x7ff9a8004d90 2023-11-28T11:11:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9bb7fe700 1 --2- 172.21.15.28:0/1790603668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9a8004d90 0x7ff9bc15c880 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400f850 tx=0x7ff9a400fc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9b3fff700 1 -- 172.21.15.28:0/1790603668 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a400fe50 con 0x7ff9a8004d90 2023-11-28T11:11:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9b3fff700 1 -- 172.21.15.28:0/1790603668 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a40052e0 con 0x7ff9a8004d90 2023-11-28T11:11:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9b3fff700 1 -- 172.21.15.28:0/1790603668 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a4003680 con 0x7ff9a8004d90 2023-11-28T11:11:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9c1e4c700 1 -- 172.21.15.28:0/1790603668 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9bc0cac40 con 0x7ff9a8004d90 2023-11-28T11:11:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.800+0000 7ff9c1e4c700 1 -- 172.21.15.28:0/1790603668 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9bc0cb200 con 0x7ff9a8004d90 2023-11-28T11:11:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.801+0000 7ff9c1e4c700 1 -- 172.21.15.28:0/1790603668 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9bc0675a0 con 0x7ff9a8004d90 2023-11-28T11:11:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.802+0000 7ff9b3fff700 1 -- 172.21.15.28:0/1790603668 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff9a40037e0 con 0x7ff9a8004d90 2023-11-28T11:11:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.802+0000 7ff9b3fff700 1 --2- 172.21.15.28:0/1790603668 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff99c0682f0 0x7ff99c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.802+0000 7ff9b3fff700 1 -- 172.21.15.28:0/1790603668 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff9a4085e80 con 0x7ff9a8004d90 2023-11-28T11:11:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.802+0000 7ff9baffd700 1 --2- 172.21.15.28:0/1790603668 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff99c0682f0 0x7ff99c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.803+0000 7ff9baffd700 1 --2- 172.21.15.28:0/1790603668 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff99c0682f0 0x7ff99c06a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0b0fc0 tx=0x7ff9ac008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:10.806+0000 7ff9b3fff700 1 -- 172.21.15.28:0/1790603668 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff9a40525a0 con 0x7ff9a8004d90 2023-11-28T11:11:11.046 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:11.045+0000 7ff9c1e4c700 1 -- 172.21.15.28:0/1790603668 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff9bc15b0e0 con 0x7ff9a8004d90 2023-11-28T11:11:11.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:11.045+0000 7ff9b3fff700 1 -- 172.21.15.28:0/1790603668 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v25) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff9a4055bc0 con 0x7ff9a8004d90 2023-11-28T11:11:11.047 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:11:11.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:11.047+0000 7ff9c1e4c700 1 -- 172.21.15.28:0/1790603668 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff99c0682f0 msgr2=0x7ff99c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:11.047+0000 7ff9c1e4c700 1 --2- 172.21.15.28:0/1790603668 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff99c0682f0 0x7ff99c06a7a0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7ff9bc0b0fc0 tx=0x7ff9ac008040 comp rx=0 tx=0).stop 2023-11-28T11:11:11.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:11.047+0000 7ff9c1e4c700 1 -- 172.21.15.28:0/1790603668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9a8004d90 msgr2=0x7ff9bc15c880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:11.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:11.047+0000 7ff9c1e4c700 1 --2- 172.21.15.28:0/1790603668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9a8004d90 0x7ff9bc15c880 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7ff9a400f850 tx=0x7ff9a400fc10 comp rx=0 tx=0).stop 2023-11-28T11:11:11.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:11.047+0000 7ff9c1e4c700 1 -- 172.21.15.28:0/1790603668 shutdown_connections 2023-11-28T11:11:11.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:11.047+0000 7ff9c1e4c700 1 --2- 172.21.15.28:0/1790603668 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff99c0682f0 0x7ff99c06a7a0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:11.047+0000 7ff9c1e4c700 1 --2- 172.21.15.28:0/1790603668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff9a8004d90 0x7ff9bc15c880 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:11.047+0000 7ff9c1e4c700 1 --2- 172.21.15.28:0/1790603668 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff9bc0b4b20 0x7ff9bc0b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:11.047+0000 7ff9c1e4c700 1 --2- 172.21.15.28:0/1790603668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff9bc14d230 0x7ff9bc15a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:11.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:11.047+0000 7ff9c1e4c700 1 -- 172.21.15.28:0/1790603668 >> 172.21.15.28:0/1790603668 conn(0x7ff9bc0b1650 msgr2=0x7ff9bc0d0120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:11.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:11.048+0000 7ff9c1e4c700 1 -- 172.21.15.28:0/1790603668 shutdown_connections 2023-11-28T11:11:11.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:11.048+0000 7ff9c1e4c700 1 -- 172.21.15.28:0/1790603668 wait complete. 2023-11-28T11:11:11.061 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:11:11.061 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:11:11.061 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2023-11-28T11:11:11.061 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:11:11.061 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:11.061 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:11:11.061 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:11:11.061 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:11:11.061 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:11:11.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:11.062 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2023-11-28T11:11:11.079 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:11:11.079 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:11:11.079 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:11:11.080 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:11.080 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2023-11-28T11:11:11.080 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:11.080 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:11:11.080 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:11:11.080 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:11:11.080 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:11:11.080 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:11:11.080 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2023-11-28T11:11:11.080 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T11:11:11.081 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42187sec preferred_lft 42187sec 2023-11-28T11:11:11.081 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2023-11-28T11:11:11.081 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:11.081 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:11:11.081 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2023-11-28T11:11:11.081 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:11:11.081 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 62:ec:54:d3:7a:f0 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:11:11.081 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:11:11.081 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:11.081 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::407f:32ff:fe2d:9b2f/64 scope link 2023-11-28T11:11:11.081 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:11.081 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:11:11.082 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 62:ec:54:d3:7a:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:11:11.082 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::60ec:54ff:fed3:7af0/64 scope link 2023-11-28T11:11:11.082 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:11.082 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:11:11.082 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether c6:1d:6e:49:de:b9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:11:11.082 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::c41d:6eff:fe49:deb9/64 scope link 2023-11-28T11:11:11.082 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:11:11.082 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:11.082 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:11:11.145 INFO:teuthology.orchestra.run.smithi028.stdout:1 2023-11-28T11:11:11.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:11.146 DEBUG:teuthology.orchestra.run.smithi028:> route 2023-11-28T11:11:11.205 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2023-11-28T11:11:11.205 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:11:11.205 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T11:11:11.205 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T11:11:11.205 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:11:11.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:11.206 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:11:11.206 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:11:11.206 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T11:11:11.206 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T11:11:11.206 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T11:11:11.206 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:11:11.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:11.313 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2023-11-28T11:11:11.329 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:11:11.330 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:11:11.330 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:11:11.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:11.330 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2023-11-28T11:11:11.397 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T11:11:11.397 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:11:11.397 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:11.454 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:11:11.455 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:11:11.455 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:11.512 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2023-11-28T11:11:11.580 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:11:11.580 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:11:11.647 INFO:teuthology.orchestra.run.smithi028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:11:11.648 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:11:11.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:11.648 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2023-11-28T11:11:11.705 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T11:11:11.705 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/) 2023-11-28T11:11:11.747 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2023-11-28T11:11:11.774 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:11:11.775 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:11:11.836 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:2023-11-28T11:11:11.835+0000 7fe13e6b15c0 -1 init, newargv = 0x55a95ec908d0 newargc=15 2023-11-28T11:11:11.837 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[125845]: starting ceph client 2023-11-28T11:11:11.841 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[125845]: starting fuse 2023-11-28T11:11:11.844 INFO:teuthology.orchestra.run.smithi028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:11:11.845 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:11:11.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:11.846 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2023-11-28T11:11:11.904 INFO:teuthology.orchestra.run.smithi028.stdout:47 2023-11-28T11:11:11.904 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T11:11:11.904 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:11:11.905 DEBUG:teuthology.orchestra.run.smithi028:> import glob 2023-11-28T11:11:11.905 DEBUG:teuthology.orchestra.run.smithi028:> import re 2023-11-28T11:11:11.905 DEBUG:teuthology.orchestra.run.smithi028:> import os 2023-11-28T11:11:11.905 DEBUG:teuthology.orchestra.run.smithi028:> import subprocess 2023-11-28T11:11:11.905 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:11:11.905 DEBUG:teuthology.orchestra.run.smithi028:> def _find_admin_socket(client_name): 2023-11-28T11:11:11.905 DEBUG:teuthology.orchestra.run.smithi028:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T11:11:11.905 DEBUG:teuthology.orchestra.run.smithi028:> files = glob.glob(asok_path) 2023-11-28T11:11:11.906 DEBUG:teuthology.orchestra.run.smithi028:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T11:11:11.906 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:11:11.907 DEBUG:teuthology.orchestra.run.smithi028:> # Given a non-glob path, it better be there 2023-11-28T11:11:11.907 DEBUG:teuthology.orchestra.run.smithi028:> if "*" not in asok_path: 2023-11-28T11:11:11.907 DEBUG:teuthology.orchestra.run.smithi028:> assert(len(files) == 1) 2023-11-28T11:11:11.907 DEBUG:teuthology.orchestra.run.smithi028:> return files[0] 2023-11-28T11:11:11.908 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:11:11.908 DEBUG:teuthology.orchestra.run.smithi028:> for f in files: 2023-11-28T11:11:11.908 DEBUG:teuthology.orchestra.run.smithi028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T11:11:11.908 DEBUG:teuthology.orchestra.run.smithi028:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T11:11:11.908 DEBUG:teuthology.orchestra.run.smithi028:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T11:11:11.909 DEBUG:teuthology.orchestra.run.smithi028:> contents = proc_f.read() 2023-11-28T11:11:11.909 DEBUG:teuthology.orchestra.run.smithi028:> if mountpoint in contents: 2023-11-28T11:11:11.909 DEBUG:teuthology.orchestra.run.smithi028:> return f 2023-11-28T11:11:11.909 DEBUG:teuthology.orchestra.run.smithi028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T11:11:11.909 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:11:11.910 DEBUG:teuthology.orchestra.run.smithi028:> print(_find_admin_socket("client.0")) 2023-11-28T11:11:11.910 DEBUG:teuthology.orchestra.run.smithi028:> ' 2023-11-28T11:11:12.028 INFO:teuthology.orchestra.run.smithi028.stdout:/var/run/ceph/ceph-client.0.125845.asok 2023-11-28T11:11:12.211 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.125845.asok 2023-11-28T11:11:12.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:12.212 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.125845.asok status 2023-11-28T11:11:12.373 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:11:12.373 INFO:teuthology.orchestra.run.smithi028.stdout: "metadata": { 2023-11-28T11:11:12.374 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T11:11:12.374 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T11:11:12.374 INFO:teuthology.orchestra.run.smithi028.stdout: "entity_id": "0", 2023-11-28T11:11:12.374 INFO:teuthology.orchestra.run.smithi028.stdout: "hostname": "smithi028", 2023-11-28T11:11:12.374 INFO:teuthology.orchestra.run.smithi028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T11:11:12.375 INFO:teuthology.orchestra.run.smithi028.stdout: "pid": "125845", 2023-11-28T11:11:12.375 INFO:teuthology.orchestra.run.smithi028.stdout: "root": "/" 2023-11-28T11:11:12.375 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:12.375 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_count": 0, 2023-11-28T11:11:12.375 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_pinned_count": 0, 2023-11-28T11:11:12.376 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 6246, 2023-11-28T11:11:12.376 INFO:teuthology.orchestra.run.smithi028.stdout: "inst": { 2023-11-28T11:11:12.376 INFO:teuthology.orchestra.run.smithi028.stdout: "name": { 2023-11-28T11:11:12.377 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "client", 2023-11-28T11:11:12.377 INFO:teuthology.orchestra.run.smithi028.stdout: "num": 6246 2023-11-28T11:11:12.377 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:12.377 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2023-11-28T11:11:12.377 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2023-11-28T11:11:12.378 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2023-11-28T11:11:12.378 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1525587081 2023-11-28T11:11:12.378 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:12.378 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:12.378 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2023-11-28T11:11:12.379 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2023-11-28T11:11:12.379 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2023-11-28T11:11:12.379 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1525587081 2023-11-28T11:11:12.380 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:12.380 INFO:teuthology.orchestra.run.smithi028.stdout: "inst_str": "client.6246 192.168.0.1:0/1525587081", 2023-11-28T11:11:12.380 INFO:teuthology.orchestra.run.smithi028.stdout: "addr_str": "192.168.0.1:0/1525587081", 2023-11-28T11:11:12.380 INFO:teuthology.orchestra.run.smithi028.stdout: "inode_count": 1, 2023-11-28T11:11:12.381 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_epoch": 25, 2023-11-28T11:11:12.381 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch": 54, 2023-11-28T11:11:12.381 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch_barrier": 0, 2023-11-28T11:11:12.381 INFO:teuthology.orchestra.run.smithi028.stdout: "blocklisted": false, 2023-11-28T11:11:12.381 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs" 2023-11-28T11:11:12.382 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:11:12.383 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:12.383 DEBUG:teuthology.orchestra.run.smithi028:> stat --file-system '--printf=%T 2023-11-28T11:11:12.383 DEBUG:teuthology.orchestra.run.smithi028:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:12.401 INFO:teuthology.orchestra.run.smithi028.stdout:fuseblk 2023-11-28T11:11:12.401 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:12.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:12.401 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:12.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:12.469 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir parent_a) 2023-11-28T11:11:12.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:12.535 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch parent_a/alpha) 2023-11-28T11:11:12.595 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:11:12.595 DEBUG:teuthology.orchestra.run.smithi028:> import os 2023-11-28T11:11:12.595 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2023-11-28T11:11:12.595 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:11:12.596 DEBUG:teuthology.orchestra.run.smithi028:> print(os.stat("/home/ubuntu/cephtest/mnt.0/parent_a/alpha").st_ino) 2023-11-28T11:11:12.596 DEBUG:teuthology.orchestra.run.smithi028:> ' 2023-11-28T11:11:12.703 INFO:teuthology.orchestra.run.smithi028.stdout:1099511627777 2023-11-28T11:11:12.891 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:13.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.056+0000 7fc1c8cde700 1 Processor -- start 2023-11-28T11:11:13.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.056+0000 7fc1c8cde700 1 -- start start 2023-11-28T11:11:13.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.057+0000 7fc1c8cde700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1c415c990 0x7fc1c40cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.057+0000 7fc1c8cde700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc1c4161090 0x7fc1c40ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.057+0000 7fc1c8cde700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1c40ce340 con 0x7fc1c40caeb0 2023-11-28T11:11:13.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.057+0000 7fc1c8cde700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1c40cebe0 con 0x7fc1c4161090 2023-11-28T11:11:13.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.057+0000 7fc1c8cde700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40cb3f0 con 0x7fc1c415c990 2023-11-28T11:11:13.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.057+0000 7fc1c2d9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc1c40caeb0 0x7fc1c40cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38766/0 (socket says 172.21.15.28:38766) 2023-11-28T11:11:13.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.057+0000 7fc1c2d9d700 1 -- 172.21.15.28:0/2977152400 learned_addr learned my addr 172.21.15.28:0/2977152400 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:13.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c1d9b700 1 --2- 172.21.15.28:0/2977152400 >> v2:172.21.15.59:3300/0 conn(0x7fc1c4161090 0x7fc1c40ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 -- 172.21.15.28:0/2977152400 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441220947 0 0) 0x7fc1c40ce340 con 0x7fc1c40caeb0 2023-11-28T11:11:13.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 -- 172.21.15.28:0/2977152400 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1ac0035b0 con 0x7fc1c40caeb0 2023-11-28T11:11:13.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c259c700 1 --2- 172.21.15.28:0/2977152400 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1c415c990 0x7fc1c40cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 -- 172.21.15.28:0/2977152400 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3542725032 0 0) 0x7fc1ac0035b0 con 0x7fc1c40caeb0 2023-11-28T11:11:13.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 -- 172.21.15.28:0/2977152400 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1c415c990 msgr2=0x7fc1c40cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 --2- 172.21.15.28:0/2977152400 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1c415c990 0x7fc1c40cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 -- 172.21.15.28:0/2977152400 >> v2:172.21.15.59:3300/0 conn(0x7fc1c4161090 msgr2=0x7fc1c40ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 --2- 172.21.15.28:0/2977152400 >> v2:172.21.15.59:3300/0 conn(0x7fc1c4161090 0x7fc1c40ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 -- 172.21.15.28:0/2977152400 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c40ce590 con 0x7fc1c40caeb0 2023-11-28T11:11:13.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 -- 172.21.15.28:0/2977152400 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fc1bc0019f0 con 0x7fc1c40caeb0 2023-11-28T11:11:13.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 -- 172.21.15.28:0/2977152400 >> v1:172.21.15.28:6789/0 conn(0x7fc1c40caeb0 legacy=0x7fc1c40cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 --2- 172.21.15.28:0/2977152400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1ac003a00 0x7fc1ac003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 --2- 172.21.15.28:0/2977152400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1ac004850 0x7fc1ac004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 --2- 172.21.15.28:0/2977152400 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1c415c990 0x7fc1ac005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 -- 172.21.15.28:0/2977152400 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40cebe0 con 0x7fc1ac003a00 2023-11-28T11:11:13.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 -- 172.21.15.28:0/2977152400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c40cb3f0 con 0x7fc1ac004850 2023-11-28T11:11:13.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c159a700 1 -- 172.21.15.28:0/2977152400 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1ac00ba90 con 0x7fc1c415c990 2023-11-28T11:11:13.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c259c700 1 --2- 172.21.15.28:0/2977152400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1ac003a00 0x7fc1ac003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c2d9d700 1 --2- 172.21.15.28:0/2977152400 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1c415c990 0x7fc1ac005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c1d9b700 1 --2- 172.21.15.28:0/2977152400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1ac004850 0x7fc1ac004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c259c700 1 -- 172.21.15.28:0/2977152400 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1c415c990 msgr2=0x7fc1ac005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c259c700 1 --2- 172.21.15.28:0/2977152400 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1c415c990 0x7fc1ac005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c259c700 1 -- 172.21.15.28:0/2977152400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1ac004850 msgr2=0x7fc1ac004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c259c700 1 --2- 172.21.15.28:0/2977152400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1ac004850 0x7fc1ac004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c259c700 1 -- 172.21.15.28:0/2977152400 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1ac0035b0 con 0x7fc1ac003a00 2023-11-28T11:11:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.058+0000 7fc1c259c700 1 --2- 172.21.15.28:0/2977152400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1ac003a00 0x7fc1ac003e10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009b60 tx=0x7fc1ac005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed184913766ddf79 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.059+0000 7fc1c159a700 1 -- 172.21.15.28:0/2977152400 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b800e6c0 con 0x7fc1ac003a00 2023-11-28T11:11:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.059+0000 7fc1c159a700 1 -- 172.21.15.28:0/2977152400 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b800e820 con 0x7fc1ac003a00 2023-11-28T11:11:13.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.059+0000 7fc1c159a700 1 -- 172.21.15.28:0/2977152400 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b800eb90 con 0x7fc1ac003a00 2023-11-28T11:11:13.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.059+0000 7fc1c8cde700 1 -- 172.21.15.28:0/2977152400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1ac003a00 msgr2=0x7fc1ac003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.059+0000 7fc1c8cde700 1 --2- 172.21.15.28:0/2977152400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1ac003a00 0x7fc1ac003e10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fc1b8009b60 tx=0x7fc1ac005880 comp rx=0 tx=0).stop 2023-11-28T11:11:13.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.059+0000 7fc1c2d9d700 1 -- 172.21.15.28:0/2977152400 reap_dead start 2023-11-28T11:11:13.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.059+0000 7fc1c8cde700 1 -- 172.21.15.28:0/2977152400 shutdown_connections 2023-11-28T11:11:13.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.059+0000 7fc1c8cde700 1 -- 172.21.15.28:0/2977152400 >> 172.21.15.28:0/2977152400 conn(0x7fc1c40b1650 msgr2=0x7fc1c40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:13.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.059+0000 7fc1c8cde700 1 -- 172.21.15.28:0/2977152400 shutdown_connections 2023-11-28T11:11:13.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.059+0000 7fc1c8cde700 1 -- 172.21.15.28:0/2977152400 wait complete. 2023-11-28T11:11:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.061+0000 7fc1c8cde700 1 Processor -- start 2023-11-28T11:11:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.061+0000 7fc1c8cde700 1 -- start start 2023-11-28T11:11:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.061+0000 7fc1c8cde700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1ac003a00 0x7fc1c415f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.061+0000 7fc1c8cde700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1c406cd10 0x7fc1c41f1570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.061+0000 7fc1c8cde700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1c406d180 0x7fc1c41f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.061+0000 7fc1c8cde700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c41f1ff0 con 0x7fc1c406cd10 2023-11-28T11:11:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.061+0000 7fc1c8cde700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c414e600 con 0x7fc1ac003a00 2023-11-28T11:11:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.061+0000 7fc1c8cde700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1c414da80 con 0x7fc1c406d180 2023-11-28T11:11:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.061+0000 7fc1c1d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1c406cd10 0x7fc1c41f1570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.061+0000 7fc1c1d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1c406cd10 0x7fc1c41f1570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50718/0 (socket says 172.21.15.28:50718) 2023-11-28T11:11:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.061+0000 7fc1c1d9b700 1 -- 172.21.15.28:0/1994802793 learned_addr learned my addr 172.21.15.28:0/1994802793 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.062+0000 7fc1c2d9d700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1c406d180 0x7fc1c41f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.062+0000 7fc1c259c700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1ac003a00 0x7fc1c415f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.062+0000 7fc1c1d9b700 1 -- 172.21.15.28:0/1994802793 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1c406d180 msgr2=0x7fc1c41f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.062+0000 7fc1c1d9b700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1c406d180 0x7fc1c41f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.062+0000 7fc1c1d9b700 1 -- 172.21.15.28:0/1994802793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1ac003a00 msgr2=0x7fc1c415f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.062+0000 7fc1c1d9b700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1ac003a00 0x7fc1c415f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.062+0000 7fc1c1d9b700 1 -- 172.21.15.28:0/1994802793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1c40cca20 con 0x7fc1c406cd10 2023-11-28T11:11:13.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.062+0000 7fc1c2d9d700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1c406d180 0x7fc1c41f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:13.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.062+0000 7fc1c259c700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1ac003a00 0x7fc1c415f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:13.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.062+0000 7fc1c1d9b700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1c406cd10 0x7fc1c41f1570 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4011a60 tx=0x7fc1b4011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.062+0000 7fc1b2ffd700 1 -- 172.21.15.28:0/1994802793 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b4004240 con 0x7fc1c406cd10 2023-11-28T11:11:13.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.062+0000 7fc1b2ffd700 1 -- 172.21.15.28:0/1994802793 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1b40043a0 con 0x7fc1c406cd10 2023-11-28T11:11:13.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.062+0000 7fc1c8cde700 1 -- 172.21.15.28:0/1994802793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1c40ce870 con 0x7fc1c406cd10 2023-11-28T11:11:13.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.062+0000 7fc1b2ffd700 1 -- 172.21.15.28:0/1994802793 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1b400f790 con 0x7fc1c406cd10 2023-11-28T11:11:13.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.062+0000 7fc1c8cde700 1 -- 172.21.15.28:0/1994802793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1c40ccbb0 con 0x7fc1c406cd10 2023-11-28T11:11:13.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.063+0000 7fc1c8cde700 1 -- 172.21.15.28:0/1994802793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1c40675a0 con 0x7fc1c406cd10 2023-11-28T11:11:13.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.063+0000 7fc1b2ffd700 1 -- 172.21.15.28:0/1994802793 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc1b4025070 con 0x7fc1c406cd10 2023-11-28T11:11:13.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.064+0000 7fc1b2ffd700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc1a40682f0 0x7fc1a406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.064+0000 7fc1b2ffd700 1 -- 172.21.15.28:0/1994802793 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc1b40861a0 con 0x7fc1c406cd10 2023-11-28T11:11:13.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.064+0000 7fc1c259c700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc1a40682f0 0x7fc1a406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.064+0000 7fc1c259c700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc1a40682f0 0x7fc1a406a7a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80090b0 tx=0x7fc1b8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.068+0000 7fc1b2ffd700 1 -- 172.21.15.28:0/1994802793 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc1b40528c0 con 0x7fc1c406cd10 2023-11-28T11:11:13.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.305+0000 7fc1c8cde700 1 -- 172.21.15.28:0/1994802793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc1c40cdc50 con 0x7fc1c406cd10 2023-11-28T11:11:13.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.305+0000 7fc1b2ffd700 1 -- 172.21.15.28:0/1994802793 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4645 (secure 0 0 0) 0x7fc1b4055ee0 con 0x7fc1c406cd10 2023-11-28T11:11:13.307 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:13.307 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5198,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/284515388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":284515388},{"type":"v1","addr":"172.21.15.59:6835","nonce":284515388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2160385010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2160385010},{"type":"v1","addr":"172.21.15.28:6836","nonce":2160385010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3775145347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3775145347},{"type":"v1","addr":"172.21.15.28:6837","nonce":3775145347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:59.771956+0000","modified":"2023-11-28T11:11:03.109382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6834/3261856585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3261856585},{"type":"v1","addr":"172.21.15.59:6834","nonce":3261856585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:11:13.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.308+0000 7fc1c8cde700 1 -- 172.21.15.28:0/1994802793 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc1a40682f0 msgr2=0x7fc1a406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.308+0000 7fc1c8cde700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc1a40682f0 0x7fc1a406a7a0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc1b80090b0 tx=0x7fc1b8008030 comp rx=0 tx=0).stop 2023-11-28T11:11:13.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.308+0000 7fc1c8cde700 1 -- 172.21.15.28:0/1994802793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1c406cd10 msgr2=0x7fc1c41f1570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.308+0000 7fc1c8cde700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1c406cd10 0x7fc1c41f1570 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fc1b4011a60 tx=0x7fc1b4011e20 comp rx=0 tx=0).stop 2023-11-28T11:11:13.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.308+0000 7fc1c8cde700 1 -- 172.21.15.28:0/1994802793 shutdown_connections 2023-11-28T11:11:13.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.308+0000 7fc1c8cde700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc1a40682f0 0x7fc1a406a7a0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.308+0000 7fc1c8cde700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc1c406d180 0x7fc1c41f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.308+0000 7fc1c8cde700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1ac003a00 0x7fc1c415f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.308+0000 7fc1c8cde700 1 --2- 172.21.15.28:0/1994802793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc1c406cd10 0x7fc1c41f1570 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.308+0000 7fc1c8cde700 1 -- 172.21.15.28:0/1994802793 >> 172.21.15.28:0/1994802793 conn(0x7fc1c40b1650 msgr2=0x7fc1c406bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:13.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.308+0000 7fc1c8cde700 1 -- 172.21.15.28:0/1994802793 shutdown_connections 2023-11-28T11:11:13.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.308+0000 7fc1c8cde700 1 -- 172.21.15.28:0/1994802793 wait complete. 2023-11-28T11:11:13.311 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 25 2023-11-28T11:11:13.321 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-28T11:11:13.487 INFO:teuthology.orchestra.run.smithi059.stdout:{"message":"","return_code":0} 2023-11-28T11:11:13.488 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:11:13.488 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:11:13.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.533+0000 7fec327a5080 1 Processor -- start 2023-11-28T11:11:13.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.533+0000 7fec327a5080 1 -- start start 2023-11-28T11:11:13.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.533+0000 7fec327a5080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561c2a6d9e00 0x561c2a630330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.533+0000 7fec327a5080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561c2a6bf280 0x561c2a630870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.533+0000 7fec327a5080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c2a6493a0 0x561c2a6c1640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.533+0000 7fec327a5080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2a53cfe0 con 0x561c2a6bf280 2023-11-28T11:11:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.533+0000 7fec327a5080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2a6c0230 con 0x561c2a6493a0 2023-11-28T11:11:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.533+0000 7fec327a5080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c2a6c0400 con 0x561c2a6d9e00 2023-11-28T11:11:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.533+0000 7fec23fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561c2a6bf280 0x561c2a630870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.533+0000 7fec23fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561c2a6bf280 0x561c2a630870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50738/0 (socket says 172.21.15.28:50738) 2023-11-28T11:11:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.533+0000 7fec23fff700 1 -- 172.21.15.28:0/3696760668 learned_addr learned my addr 172.21.15.28:0/3696760668 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.533+0000 7fec290a9700 1 --2- 172.21.15.28:0/3696760668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c2a6493a0 0x561c2a6c1640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.533+0000 7fec288a8700 1 --2- 172.21.15.28:0/3696760668 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561c2a6d9e00 0x561c2a630330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec23fff700 1 -- 172.21.15.28:0/3696760668 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561c2a6d9e00 msgr2=0x561c2a630330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec23fff700 1 --2- 172.21.15.28:0/3696760668 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561c2a6d9e00 0x561c2a630330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec23fff700 1 -- 172.21.15.28:0/3696760668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c2a6493a0 msgr2=0x561c2a6c1640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec23fff700 1 --2- 172.21.15.28:0/3696760668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c2a6493a0 0x561c2a6c1640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec23fff700 1 -- 172.21.15.28:0/3696760668 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c2a548830 con 0x561c2a6bf280 2023-11-28T11:11:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec23fff700 1 --2- 172.21.15.28:0/3696760668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561c2a6bf280 0x561c2a630870 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fec140097e0 tx=0x7fec14005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e18995349947a59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec22ffd700 1 -- 172.21.15.28:0/3696760668 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1400f040 con 0x561c2a6bf280 2023-11-28T11:11:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec22ffd700 1 -- 172.21.15.28:0/3696760668 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec140088d0 con 0x561c2a6bf280 2023-11-28T11:11:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec22ffd700 1 -- 172.21.15.28:0/3696760668 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec14008c40 con 0x561c2a6bf280 2023-11-28T11:11:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec327a5080 1 -- 172.21.15.28:0/3696760668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561c2a6bf280 msgr2=0x561c2a630870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec327a5080 1 --2- 172.21.15.28:0/3696760668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561c2a6bf280 0x561c2a630870 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fec140097e0 tx=0x7fec14005a00 comp rx=0 tx=0).stop 2023-11-28T11:11:13.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec327a5080 1 -- 172.21.15.28:0/3696760668 shutdown_connections 2023-11-28T11:11:13.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec327a5080 1 --2- 172.21.15.28:0/3696760668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c2a6493a0 0x561c2a6c1640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec327a5080 1 --2- 172.21.15.28:0/3696760668 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561c2a6d9e00 0x561c2a630330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec327a5080 1 --2- 172.21.15.28:0/3696760668 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561c2a6bf280 0x561c2a630870 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.534+0000 7fec327a5080 1 -- 172.21.15.28:0/3696760668 >> 172.21.15.28:0/3696760668 conn(0x561c2a6497e0 msgr2=0x561c2a6313d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:13.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.535+0000 7fec327a5080 1 -- 172.21.15.28:0/3696760668 shutdown_connections 2023-11-28T11:11:13.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.535+0000 7fec327a5080 1 -- 172.21.15.28:0/3696760668 wait complete. 2023-11-28T11:11:13.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.535+0000 7fec327a5080 1 Processor -- start 2023-11-28T11:11:13.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.535+0000 7fec327a5080 1 -- start start 2023-11-28T11:11:13.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec327a5080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561c2a6493a0 0x561c2a6ba7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec327a5080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561c2a6bf280 0x561c2a6d8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec327a5080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c2a6d9e00 0x561c2a6bd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec327a5080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2a6bad30 con 0x561c2a6bf280 2023-11-28T11:11:13.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec327a5080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c2a6d88d0 con 0x561c2a6d9e00 2023-11-28T11:11:13.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec327a5080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x561c2a6d7cc0 con 0x561c2a6493a0 2023-11-28T11:11:13.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec23fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561c2a6bf280 0x561c2a6d8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec23fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561c2a6bf280 0x561c2a6d8d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50744/0 (socket says 172.21.15.28:50744) 2023-11-28T11:11:13.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec23fff700 1 -- 172.21.15.28:0/3435211425 learned_addr learned my addr 172.21.15.28:0/3435211425 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:13.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec288a8700 1 --2- 172.21.15.28:0/3435211425 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561c2a6493a0 0x561c2a6ba7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec290a9700 1 --2- 172.21.15.28:0/3435211425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c2a6d9e00 0x561c2a6bd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec23fff700 1 -- 172.21.15.28:0/3435211425 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561c2a6493a0 msgr2=0x561c2a6ba7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec23fff700 1 --2- 172.21.15.28:0/3435211425 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561c2a6493a0 0x561c2a6ba7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec23fff700 1 -- 172.21.15.28:0/3435211425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c2a6d9e00 msgr2=0x561c2a6bd0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec23fff700 1 --2- 172.21.15.28:0/3435211425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c2a6d9e00 0x561c2a6bd0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.536+0000 7fec23fff700 1 -- 172.21.15.28:0/3435211425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec14009410 con 0x561c2a6bf280 2023-11-28T11:11:13.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.537+0000 7fec23fff700 1 --2- 172.21.15.28:0/3435211425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561c2a6bf280 0x561c2a6d8d60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fec14017040 tx=0x7fec14002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.537+0000 7fec21ffb700 1 -- 172.21.15.28:0/3435211425 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec1400f040 con 0x561c2a6bf280 2023-11-28T11:11:13.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.537+0000 7fec21ffb700 1 -- 172.21.15.28:0/3435211425 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec14023070 con 0x561c2a6bf280 2023-11-28T11:11:13.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.537+0000 7fec327a5080 1 -- 172.21.15.28:0/3435211425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x561c2a555360 con 0x561c2a6bf280 2023-11-28T11:11:13.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.537+0000 7fec21ffb700 1 -- 172.21.15.28:0/3435211425 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec140158f0 con 0x561c2a6bf280 2023-11-28T11:11:13.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.537+0000 7fec327a5080 1 -- 172.21.15.28:0/3435211425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561c2a555120 con 0x561c2a6bf280 2023-11-28T11:11:13.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.538+0000 7fec21ffb700 1 -- 172.21.15.28:0/3435211425 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fec1402e020 con 0x561c2a6bf280 2023-11-28T11:11:13.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.538+0000 7fec21ffb700 1 --2- 172.21.15.28:0/3435211425 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fec0c066c10 0x7fec0c0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.539+0000 7fec21ffb700 1 -- 172.21.15.28:0/3435211425 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fec14086a90 con 0x561c2a6bf280 2023-11-28T11:11:13.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.539+0000 7fec288a8700 1 --2- 172.21.15.28:0/3435211425 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fec0c066c10 0x7fec0c0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.539+0000 7fec327a5080 1 --2- 172.21.15.28:0/3435211425 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x561c2a6d92a0 0x561c2a770f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.539+0000 7fec327a5080 1 -- 172.21.15.28:0/3435211425 --> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54) v8 -- 0x561c2a62bea0 con 0x561c2a6d92a0 2023-11-28T11:11:13.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.539+0000 7fec288a8700 1 --2- 172.21.15.28:0/3435211425 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fec0c066c10 0x7fec0c0690c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fec1c001cf0 tx=0x7fec1c011040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.539+0000 7fec290a9700 1 --2- 172.21.15.28:0/3435211425 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x561c2a6d92a0 0x561c2a770f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.539+0000 7fec290a9700 1 --2- 172.21.15.28:0/3435211425 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x561c2a6d92a0 0x561c2a770f20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.540+0000 7fec290a9700 1 -- 172.21.15.28:0/3435211425 <== osd.5 v2:172.21.15.59:6800/103696 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7fec24018070 con 0x561c2a6d92a0 2023-11-28T11:11:13.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.540+0000 7fec327a5080 1 -- 172.21.15.28:0/3435211425 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x561c2a6d92a0 msgr2=0x561c2a770f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.540+0000 7fec327a5080 1 --2- 172.21.15.28:0/3435211425 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x561c2a6d92a0 0x561c2a770f20 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.540+0000 7fec327a5080 1 -- 172.21.15.28:0/3435211425 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fec0c066c10 msgr2=0x7fec0c0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.540+0000 7fec327a5080 1 --2- 172.21.15.28:0/3435211425 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fec0c066c10 0x7fec0c0690c0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fec1c001cf0 tx=0x7fec1c011040 comp rx=0 tx=0).stop 2023-11-28T11:11:13.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.540+0000 7fec327a5080 1 -- 172.21.15.28:0/3435211425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561c2a6bf280 msgr2=0x561c2a6d8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.540+0000 7fec327a5080 1 --2- 172.21.15.28:0/3435211425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561c2a6bf280 0x561c2a6d8d60 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fec14017040 tx=0x7fec14002fb0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.540+0000 7fec290a9700 1 -- 172.21.15.28:0/3435211425 reap_dead start 2023-11-28T11:11:13.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.540+0000 7fec327a5080 1 -- 172.21.15.28:0/3435211425 shutdown_connections 2023-11-28T11:11:13.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.540+0000 7fec327a5080 1 -- 172.21.15.28:0/3435211425 >> 172.21.15.28:0/3435211425 conn(0x561c2a6497e0 msgr2=0x561c2a630610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:13.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.540+0000 7fec327a5080 1 -- 172.21.15.28:0/3435211425 shutdown_connections 2023-11-28T11:11:13.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.540+0000 7fec327a5080 1 -- 172.21.15.28:0/3435211425 wait complete. 2023-11-28T11:11:13.547 INFO:teuthology.orchestra.run.smithi028.stdout:Yalphaparent_a 2023-11-28T11:11:13.548 DEBUG:teuthology.orchestra.run.smithi028:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:11:13.670 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:11:13.670 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511627777, 2023-11-28T11:11:13.670 INFO:teuthology.orchestra.run.smithi028.stdout: "ancestors": [ 2023-11-28T11:11:13.670 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:13.670 INFO:teuthology.orchestra.run.smithi028.stdout: "dirino": 1099511627776, 2023-11-28T11:11:13.670 INFO:teuthology.orchestra.run.smithi028.stdout: "dname": "alpha", 2023-11-28T11:11:13.670 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 2 2023-11-28T11:11:13.671 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:13.671 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:13.671 INFO:teuthology.orchestra.run.smithi028.stdout: "dirino": 1, 2023-11-28T11:11:13.671 INFO:teuthology.orchestra.run.smithi028.stdout: "dname": "parent_a", 2023-11-28T11:11:13.671 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 4 2023-11-28T11:11:13.671 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:13.671 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2023-11-28T11:11:13.671 INFO:teuthology.orchestra.run.smithi028.stdout: "pool": 9, 2023-11-28T11:11:13.671 INFO:teuthology.orchestra.run.smithi028.stdout: "old_pools": [] 2023-11-28T11:11:13.671 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:11:13.671 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:13.673 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'oh i'"'"'m sorry did i overwrite your xattr?' 2023-11-28T11:11:13.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.717+0000 7f15e104a080 1 Processor -- start 2023-11-28T11:11:13.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.717+0000 7f15e104a080 1 -- start start 2023-11-28T11:11:13.725 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.718+0000 7f15e104a080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15c0002880 0x55e26c0fb190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.725 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.718+0000 7f15e104a080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e26c0fc6f0 0x55e26c0fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.725 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.718+0000 7f15e104a080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55e26c0f4160 0x55e26c101760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.725 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.718+0000 7f15e104a080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26bfd1000 con 0x55e26c0fc6f0 2023-11-28T11:11:13.725 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.718+0000 7f15e104a080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26bf63fe0 con 0x7f15c0002880 2023-11-28T11:11:13.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.718+0000 7f15e104a080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e26c0feff0 con 0x55e26c0f4160 2023-11-28T11:11:13.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.718+0000 7f15d694c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e26c0fc6f0 0x55e26c0fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.718+0000 7f15d694c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e26c0fc6f0 0x55e26c0fbeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50766/0 (socket says 172.21.15.28:50766) 2023-11-28T11:11:13.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.718+0000 7f15d694c700 1 -- 172.21.15.28:0/2747730882 learned_addr learned my addr 172.21.15.28:0/2747730882 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:13.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.718+0000 7f15d694c700 1 -- 172.21.15.28:0/2747730882 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55e26c0f4160 msgr2=0x55e26c101760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.718+0000 7f15d694c700 1 --2- 172.21.15.28:0/2747730882 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55e26c0f4160 0x55e26c101760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.718+0000 7f15d694c700 1 -- 172.21.15.28:0/2747730882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15c0002880 msgr2=0x55e26c0fb190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.718+0000 7f15d694c700 1 --2- 172.21.15.28:0/2747730882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15c0002880 0x55e26c0fb190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.718+0000 7f15d694c700 1 -- 172.21.15.28:0/2747730882 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e26bf7c360 con 0x55e26c0fc6f0 2023-11-28T11:11:13.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.718+0000 7f15d694c700 1 --2- 172.21.15.28:0/2747730882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e26c0fc6f0 0x55e26c0fbeb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f15c800b900 tx=0x7f15c800f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b093a67098df428 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.719+0000 7f15d594a700 1 -- 172.21.15.28:0/2747730882 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c8014040 con 0x55e26c0fc6f0 2023-11-28T11:11:13.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.719+0000 7f15d594a700 1 -- 172.21.15.28:0/2747730882 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c800be30 con 0x55e26c0fc6f0 2023-11-28T11:11:13.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.719+0000 7f15d594a700 1 -- 172.21.15.28:0/2747730882 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c801b670 con 0x55e26c0fc6f0 2023-11-28T11:11:13.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.719+0000 7f15e104a080 1 -- 172.21.15.28:0/2747730882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e26c0fc6f0 msgr2=0x55e26c0fbeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.719+0000 7f15e104a080 1 --2- 172.21.15.28:0/2747730882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e26c0fc6f0 0x55e26c0fbeb0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f15c800b900 tx=0x7f15c800f000 comp rx=0 tx=0).stop 2023-11-28T11:11:13.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.719+0000 7f15e104a080 1 -- 172.21.15.28:0/2747730882 shutdown_connections 2023-11-28T11:11:13.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.719+0000 7f15e104a080 1 --2- 172.21.15.28:0/2747730882 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55e26c0f4160 0x55e26c101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.719+0000 7f15e104a080 1 --2- 172.21.15.28:0/2747730882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15c0002880 0x55e26c0fb190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.719+0000 7f15e104a080 1 --2- 172.21.15.28:0/2747730882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55e26c0fc6f0 0x55e26c0fbeb0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.719+0000 7f15e104a080 1 -- 172.21.15.28:0/2747730882 >> 172.21.15.28:0/2747730882 conn(0x55e26c0707f0 msgr2=0x55e26c0fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:13.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.719+0000 7f15e104a080 1 -- 172.21.15.28:0/2747730882 shutdown_connections 2023-11-28T11:11:13.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.719+0000 7f15e104a080 1 -- 172.21.15.28:0/2747730882 wait complete. 2023-11-28T11:11:13.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.720+0000 7f15e104a080 1 Processor -- start 2023-11-28T11:11:13.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.720+0000 7f15e104a080 1 -- start start 2023-11-28T11:11:13.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.720+0000 7f15e104a080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e26c0f4160 0x55e26c0fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.720+0000 7f15e104a080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55e26c0fc6f0 0x55e26c194cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.720+0000 7f15e104a080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15c0002880 0x55e26c1981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.720+0000 7f15e104a080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26c06d6f0 con 0x7f15c0002880 2023-11-28T11:11:13.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.720+0000 7f15e104a080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e26c06d960 con 0x55e26c0f4160 2023-11-28T11:11:13.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.720+0000 7f15e104a080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55e26c1979d0 con 0x55e26c0fc6f0 2023-11-28T11:11:13.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15d794e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15c0002880 0x55e26c1981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15d794e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15c0002880 0x55e26c1981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50774/0 (socket says 172.21.15.28:50774) 2023-11-28T11:11:13.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15d794e700 1 -- 172.21.15.28:0/808380575 learned_addr learned my addr 172.21.15.28:0/808380575 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:13.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15d714d700 1 --2- 172.21.15.28:0/808380575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e26c0f4160 0x55e26c0fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15d694c700 1 --2- 172.21.15.28:0/808380575 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55e26c0fc6f0 0x55e26c194cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15d794e700 1 -- 172.21.15.28:0/808380575 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55e26c0fc6f0 msgr2=0x55e26c194cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15d794e700 1 --2- 172.21.15.28:0/808380575 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55e26c0fc6f0 0x55e26c194cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15d794e700 1 -- 172.21.15.28:0/808380575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e26c0f4160 msgr2=0x55e26c0fbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15d794e700 1 --2- 172.21.15.28:0/808380575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e26c0f4160 0x55e26c0fbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15d794e700 1 -- 172.21.15.28:0/808380575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15c0001e10 con 0x7f15c0002880 2023-11-28T11:11:13.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15d794e700 1 --2- 172.21.15.28:0/808380575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15c0002880 0x55e26c1981d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x55e26c02a9f0 tx=0x7f15d0003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15d4948700 1 -- 172.21.15.28:0/808380575 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d0003d70 con 0x7f15c0002880 2023-11-28T11:11:13.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15d4948700 1 -- 172.21.15.28:0/808380575 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15d00123f0 con 0x7f15c0002880 2023-11-28T11:11:13.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15d4948700 1 -- 172.21.15.28:0/808380575 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15d00127a0 con 0x7f15c0002880 2023-11-28T11:11:13.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15e104a080 1 -- 172.21.15.28:0/808380575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55e26c072b00 con 0x7f15c0002880 2023-11-28T11:11:13.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.721+0000 7f15e104a080 1 -- 172.21.15.28:0/808380575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e26bf6f830 con 0x7f15c0002880 2023-11-28T11:11:13.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.723+0000 7f15d4948700 1 -- 172.21.15.28:0/808380575 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f15d0012900 con 0x7f15c0002880 2023-11-28T11:11:13.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.723+0000 7f15d4948700 1 --2- 172.21.15.28:0/808380575 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15c4066c10 0x7f15c40690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.723+0000 7f15d4948700 1 -- 172.21.15.28:0/808380575 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f15d00848e0 con 0x7f15c0002880 2023-11-28T11:11:13.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.723+0000 7f15d714d700 1 --2- 172.21.15.28:0/808380575 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15c4066c10 0x7f15c40690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.723+0000 7f15e104a080 1 --2- 172.21.15.28:0/808380575 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x55e26c101c70 0x55e26c0f4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.723+0000 7f15e104a080 1 -- 172.21.15.28:0/808380575 --> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [setxattr parent (40) in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e54) v8 -- 0x55e26c0f4bd0 con 0x55e26c101c70 2023-11-28T11:11:13.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.723+0000 7f15d714d700 1 --2- 172.21.15.28:0/808380575 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15c4066c10 0x7f15c40690c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f15cc005970 tx=0x7f15cc00e040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.723+0000 7f15d694c700 1 --2- 172.21.15.28:0/808380575 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x55e26c101c70 0x55e26c0f4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.724+0000 7f15d694c700 1 --2- 172.21.15.28:0/808380575 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x55e26c101c70 0x55e26c0f4690 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.725+0000 7f15d694c700 1 -- 172.21.15.28:0/808380575 <== osd.5 v2:172.21.15.59:6800/103696 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (40)] v54'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f15c8016070 con 0x55e26c101c70 2023-11-28T11:11:13.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.725+0000 7f15e104a080 1 -- 172.21.15.28:0/808380575 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x55e26c101c70 msgr2=0x55e26c0f4690 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.725+0000 7f15e104a080 1 --2- 172.21.15.28:0/808380575 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x55e26c101c70 0x55e26c0f4690 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.725+0000 7f15e104a080 1 -- 172.21.15.28:0/808380575 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15c4066c10 msgr2=0x7f15c40690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.725+0000 7f15e104a080 1 --2- 172.21.15.28:0/808380575 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15c4066c10 0x7f15c40690c0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f15cc005970 tx=0x7f15cc00e040 comp rx=0 tx=0).stop 2023-11-28T11:11:13.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.725+0000 7f15e104a080 1 -- 172.21.15.28:0/808380575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15c0002880 msgr2=0x55e26c1981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.725+0000 7f15e104a080 1 --2- 172.21.15.28:0/808380575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15c0002880 0x55e26c1981d0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x55e26c02a9f0 tx=0x7f15d0003730 comp rx=0 tx=0).stop 2023-11-28T11:11:13.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.725+0000 7f15d794e700 1 -- 172.21.15.28:0/808380575 reap_dead start 2023-11-28T11:11:13.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.725+0000 7f15e104a080 1 -- 172.21.15.28:0/808380575 shutdown_connections 2023-11-28T11:11:13.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.725+0000 7f15e104a080 1 -- 172.21.15.28:0/808380575 >> 172.21.15.28:0/808380575 conn(0x55e26c0707f0 msgr2=0x55e26c0fcc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:13.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.725+0000 7f15e104a080 1 -- 172.21.15.28:0/808380575 shutdown_connections 2023-11-28T11:11:13.733 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.725+0000 7f15e104a080 1 -- 172.21.15.28:0/808380575 wait complete. 2023-11-28T11:11:13.734 DEBUG:teuthology.orchestra.run.smithi028:> sudo daemon-helper kill ceph --cluster ceph -w 2023-11-28T11:11:13.737 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / repair,recursive 2023-11-28T11:11:13.866 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.864+0000 7fdc23e71700 1 Processor -- start 2023-11-28T11:11:13.866 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.865+0000 7fdc23e71700 1 -- start start 2023-11-28T11:11:13.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc23e71700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdc1c0cb230 0x7fdc1c0ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc23e71700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdc1c0b2660 0x7fdc1c0b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc23e71700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc1c0cfb90 con 0x7fdc1c15a6c0 2023-11-28T11:11:13.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc23e71700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c0b3020 con 0x7fdc1c0cb230 2023-11-28T11:11:13.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc23e71700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc1c0cb680 con 0x7fdc1c0b2660 2023-11-28T11:11:13.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc2140c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdc1c0cb230 0x7fdc1c0ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc21c0d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdc1c15a6c0 0x7fdc1c0d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38838/0 (socket says 172.21.15.28:38838) 2023-11-28T11:11:13.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc21c0d700 1 -- 172.21.15.28:0/4116848885 learned_addr learned my addr 172.21.15.28:0/4116848885 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:13.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc2240e700 1 --2- 172.21.15.28:0/4116848885 >> v2:172.21.15.59:3300/0 conn(0x7fdc1c0b2660 0x7fdc1c0b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc2140c700 1 -- 172.21.15.28:0/4116848885 >> v2:172.21.15.59:3300/0 conn(0x7fdc1c0b2660 msgr2=0x7fdc1c0b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc2140c700 1 --2- 172.21.15.28:0/4116848885 >> v2:172.21.15.59:3300/0 conn(0x7fdc1c0b2660 0x7fdc1c0b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc2140c700 1 -- 172.21.15.28:0/4116848885 >> v1:172.21.15.28:6789/0 conn(0x7fdc1c15a6c0 legacy=0x7fdc1c0d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc20c0b700 1 -- 172.21.15.28:0/4116848885 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087907538 0 0) 0x7fdc1c0cfb90 con 0x7fdc1c15a6c0 2023-11-28T11:11:13.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc2140c700 1 -- 172.21.15.28:0/4116848885 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc1c0cb930 con 0x7fdc1c0cb230 2023-11-28T11:11:13.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc2140c700 1 --2- 172.21.15.28:0/4116848885 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc1c0cb230 0x7fdc1c0ccea0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0cb1c0 tx=0x7fdc0c009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac54b607a9a532a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc20c0b700 1 -- 172.21.15.28:0/4116848885 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c011830 con 0x7fdc1c0cb230 2023-11-28T11:11:13.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc20c0b700 1 -- 172.21.15.28:0/4116848885 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0c011990 con 0x7fdc1c0cb230 2023-11-28T11:11:13.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.866+0000 7fdc20c0b700 1 -- 172.21.15.28:0/4116848885 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c011d20 con 0x7fdc1c0cb230 2023-11-28T11:11:13.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.867+0000 7fdc23e71700 1 -- 172.21.15.28:0/4116848885 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc1c0cb230 msgr2=0x7fdc1c0ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.867+0000 7fdc23e71700 1 --2- 172.21.15.28:0/4116848885 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc1c0cb230 0x7fdc1c0ccea0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fdc1c0cb1c0 tx=0x7fdc0c009b20 comp rx=0 tx=0).stop 2023-11-28T11:11:13.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.867+0000 7fdc23e71700 1 -- 172.21.15.28:0/4116848885 shutdown_connections 2023-11-28T11:11:13.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.867+0000 7fdc23e71700 1 --2- 172.21.15.28:0/4116848885 >> v2:172.21.15.59:3300/0 conn(0x7fdc1c0b2660 0x7fdc1c0b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.867+0000 7fdc23e71700 1 --2- 172.21.15.28:0/4116848885 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc1c0cb230 0x7fdc1c0ccea0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.867+0000 7fdc23e71700 1 -- 172.21.15.28:0/4116848885 >> 172.21.15.28:0/4116848885 conn(0x7fdc1c0cd660 msgr2=0x7fdc1c0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:13.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.867+0000 7fdc23e71700 1 -- 172.21.15.28:0/4116848885 shutdown_connections 2023-11-28T11:11:13.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.867+0000 7fdc23e71700 1 -- 172.21.15.28:0/4116848885 wait complete. 2023-11-28T11:11:13.874 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.868+0000 7fdc23e71700 1 Processor -- start 2023-11-28T11:11:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.868+0000 7fdc23e71700 1 -- start start 2023-11-28T11:11:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.869+0000 7fdc23e71700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc1c0b2660 0x7fdc1c0b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.869+0000 7fdc23e71700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc1c0cb230 0x7fdc1c0c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.869+0000 7fdc23e71700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc1c15a6c0 0x7fdc1c0c81f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.869+0000 7fdc23e71700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c154c00 con 0x7fdc1c15a6c0 2023-11-28T11:11:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.869+0000 7fdc23e71700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c157e10 con 0x7fdc1c0cb230 2023-11-28T11:11:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.869+0000 7fdc23e71700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc1c154e50 con 0x7fdc1c0b2660 2023-11-28T11:11:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.869+0000 7fdc2240e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc1c15a6c0 0x7fdc1c0c81f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.869+0000 7fdc2240e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc1c15a6c0 0x7fdc1c0c81f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50788/0 (socket says 172.21.15.28:50788) 2023-11-28T11:11:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.869+0000 7fdc2240e700 1 -- 172.21.15.28:0/373125710 learned_addr learned my addr 172.21.15.28:0/373125710 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:13.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.870+0000 7fdc2140c700 1 --2- 172.21.15.28:0/373125710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc1c0cb230 0x7fdc1c0c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.870+0000 7fdc21c0d700 1 --2- 172.21.15.28:0/373125710 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc1c0b2660 0x7fdc1c0b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.870+0000 7fdc2240e700 1 -- 172.21.15.28:0/373125710 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc1c0b2660 msgr2=0x7fdc1c0b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.870+0000 7fdc2240e700 1 --2- 172.21.15.28:0/373125710 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc1c0b2660 0x7fdc1c0b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.870+0000 7fdc2240e700 1 -- 172.21.15.28:0/373125710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc1c0cb230 msgr2=0x7fdc1c0c7cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.870+0000 7fdc2240e700 1 --2- 172.21.15.28:0/373125710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc1c0cb230 0x7fdc1c0c7cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.870+0000 7fdc2240e700 1 -- 172.21.15.28:0/373125710 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc0c0094a0 con 0x7fdc1c15a6c0 2023-11-28T11:11:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.870+0000 7fdc2240e700 1 --2- 172.21.15.28:0/373125710 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc1c15a6c0 0x7fdc1c0c81f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdc14007e60 tx=0x7fdc1400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.870+0000 7fdc127fc700 1 -- 172.21.15.28:0/373125710 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc1400ede0 con 0x7fdc1c15a6c0 2023-11-28T11:11:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.870+0000 7fdc127fc700 1 -- 172.21.15.28:0/373125710 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc14014420 con 0x7fdc1c15a6c0 2023-11-28T11:11:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.870+0000 7fdc23e71700 1 -- 172.21.15.28:0/373125710 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc1c0cf540 con 0x7fdc1c15a6c0 2023-11-28T11:11:13.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.870+0000 7fdc127fc700 1 -- 172.21.15.28:0/373125710 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc140147d0 con 0x7fdc1c15a6c0 2023-11-28T11:11:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.870+0000 7fdc23e71700 1 -- 172.21.15.28:0/373125710 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c0cf260 con 0x7fdc1c15a6c0 2023-11-28T11:11:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.872+0000 7fdc127fc700 1 -- 172.21.15.28:0/373125710 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdc14012050 con 0x7fdc1c15a6c0 2023-11-28T11:11:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.873+0000 7fdc127fc700 1 --2- 172.21.15.28:0/373125710 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdc000682f0 0x7fdc0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.873+0000 7fdc127fc700 1 -- 172.21.15.28:0/373125710 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fdc14086470 con 0x7fdc1c15a6c0 2023-11-28T11:11:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.873+0000 7fdc21c0d700 1 --2- 172.21.15.28:0/373125710 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdc000682f0 0x7fdc0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.873+0000 7fdc21c0d700 1 --2- 172.21.15.28:0/373125710 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdc000682f0 0x7fdc0006a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdc18000f60 tx=0x7fdc18007be0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.874+0000 7fdc2fffab80 1 Processor -- start 2023-11-28T11:11:13.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.874+0000 7fdc2fffab80 1 -- start start 2023-11-28T11:11:13.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.875+0000 7fdc2fffab80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ea6af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.875+0000 7fdc2fffab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ece200 0x55c1c1ed0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.875+0000 7fdc2fffab80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ed11f0 0x55c1c1eec840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.875+0000 7fdc2fffab80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c1d90c90 con 0x55c1c1ed08e0 2023-11-28T11:11:13.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.875+0000 7fdc2fffab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c1dff420 con 0x55c1c1ece200 2023-11-28T11:11:13.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.875+0000 7fdc2fffab80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1c1df8950 con 0x55c1c1ed11f0 2023-11-28T11:11:13.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.875+0000 7fdc21c0d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ea6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.875+0000 7fdc21c0d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ea6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50792/0 (socket says 172.21.15.28:50792) 2023-11-28T11:11:13.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.875+0000 7fdc21c0d700 1 -- 172.21.15.28:0/1684980148 learned_addr learned my addr 172.21.15.28:0/1684980148 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:13.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.876+0000 7fdc2240e700 1 --2- 172.21.15.28:0/1684980148 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ed11f0 0x55c1c1eec840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.876+0000 7fdc2140c700 1 --2- 172.21.15.28:0/1684980148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ece200 0x55c1c1ed0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.876+0000 7fdc21c0d700 1 -- 172.21.15.28:0/1684980148 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ed11f0 msgr2=0x55c1c1eec840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.876+0000 7fdc21c0d700 1 --2- 172.21.15.28:0/1684980148 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ed11f0 0x55c1c1eec840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.876+0000 7fdc21c0d700 1 -- 172.21.15.28:0/1684980148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ece200 msgr2=0x55c1c1ed0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.876+0000 7fdc21c0d700 1 --2- 172.21.15.28:0/1684980148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ece200 0x55c1c1ed0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.876+0000 7fdc21c0d700 1 -- 172.21.15.28:0/1684980148 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1c1c7a6b0 con 0x55c1c1ed08e0 2023-11-28T11:11:13.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.876+0000 7fdc21c0d700 1 --2- 172.21.15.28:0/1684980148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ea6af0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdc18015650 tx=0x7fdc18012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fcae92be5f91bbe4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.876+0000 7fdbff7fe700 1 -- 172.21.15.28:0/1684980148 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18015320 con 0x55c1c1ed08e0 2023-11-28T11:11:13.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.876+0000 7fdbff7fe700 1 -- 172.21.15.28:0/1684980148 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc18021450 con 0x55c1c1ed08e0 2023-11-28T11:11:13.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.876+0000 7fdbff7fe700 1 -- 172.21.15.28:0/1684980148 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc180217c0 con 0x55c1c1ed08e0 2023-11-28T11:11:13.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.877+0000 7fdc2fffab80 1 -- 172.21.15.28:0/1684980148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 msgr2=0x55c1c1ea6af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.877+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/1684980148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ea6af0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fdc18015650 tx=0x7fdc18012000 comp rx=0 tx=0).stop 2023-11-28T11:11:13.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.877+0000 7fdc2fffab80 1 -- 172.21.15.28:0/1684980148 shutdown_connections 2023-11-28T11:11:13.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.877+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/1684980148 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ea6af0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.877+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/1684980148 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ed11f0 0x55c1c1eec840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.877+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/1684980148 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ece200 0x55c1c1ed0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.877+0000 7fdc2fffab80 1 -- 172.21.15.28:0/1684980148 >> 172.21.15.28:0/1684980148 conn(0x55c1c1ea3e30 msgr2=0x55c1c1ea47c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:13.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.877+0000 7fdc2fffab80 1 -- 172.21.15.28:0/1684980148 shutdown_connections 2023-11-28T11:11:13.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.877+0000 7fdc2fffab80 1 -- 172.21.15.28:0/1684980148 wait complete. 2023-11-28T11:11:13.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.877+0000 7fdc2fffab80 1 -- start start 2023-11-28T11:11:13.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.877+0000 7fdc2fffab80 1 Processor -- start 2023-11-28T11:11:13.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.878+0000 7fdc2fffab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ece200 0x55c1c1ecfa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.878+0000 7fdc2fffab80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ed1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.878+0000 7fdc2fffab80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ed11f0 0x55c1c1ebf4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.878+0000 7fdc2fffab80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c1c7c9a0 con 0x55c1c1ed08e0 2023-11-28T11:11:13.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.878+0000 7fdc2fffab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c1ed26b0 con 0x55c1c1ece200 2023-11-28T11:11:13.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.878+0000 7fdc2fffab80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1c1ece7a0 con 0x55c1c1ed11f0 2023-11-28T11:11:13.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.878+0000 7fdc2fffab80 10 client.0 _ll_register_callbacks cb 0x55c1c1c9e310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:13.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdc2140c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ed1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdc2140c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ed1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50800/0 (socket says 172.21.15.28:50800) 2023-11-28T11:11:13.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdc2140c700 1 -- 172.21.15.28:0/714118882 learned_addr learned my addr 172.21.15.28:0/714118882 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:13.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdc21c0d700 1 --2- 172.21.15.28:0/714118882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ece200 0x55c1c1ecfa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdc2240e700 1 --2- 172.21.15.28:0/714118882 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ed11f0 0x55c1c1ebf4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdc2140c700 1 -- 172.21.15.28:0/714118882 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ed11f0 msgr2=0x55c1c1ebf4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdc2140c700 1 --2- 172.21.15.28:0/714118882 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ed11f0 0x55c1c1ebf4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdc2140c700 1 -- 172.21.15.28:0/714118882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ece200 msgr2=0x55c1c1ecfa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdc2140c700 1 --2- 172.21.15.28:0/714118882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ece200 0x55c1c1ecfa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdc2140c700 1 -- 172.21.15.28:0/714118882 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1800f040 con 0x55c1c1ed08e0 2023-11-28T11:11:13.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdc2140c700 1 --2- 172.21.15.28:0/714118882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ed1aa0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009b60 tx=0x7fdc0c00e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7134c651babec46c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdbfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:11:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdbfcff9700 1 -- 172.21.15.28:0/714118882 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0c01f040 con 0x55c1c1ed08e0 2023-11-28T11:11:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdbfcff9700 1 -- 172.21.15.28:0/714118882 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fdc0c004450 con 0x55c1c1ed08e0 2023-11-28T11:11:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdc2fffab80 1 -- 172.21.15.28:0/714118882 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1c1dc6210 con 0x55c1c1ed08e0 2023-11-28T11:11:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdbfcff9700 1 -- 172.21.15.28:0/714118882 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc0c0036a0 con 0x55c1c1ed08e0 2023-11-28T11:11:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdc2fffab80 10 client.6285 fetch_fsmap learned FSMap version 25 2023-11-28T11:11:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.879+0000 7fdc2fffab80 1 -- 172.21.15.28:0/714118882 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55c1c1c26e60 con 0x55c1c1ed08e0 2023-11-28T11:11:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.880+0000 7fdbfcff9700 1 -- 172.21.15.28:0/714118882 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c1c1dc6210 con 0x55c1c1ed08e0 2023-11-28T11:11:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.880+0000 7fdbfcff9700 1 -- 172.21.15.28:0/714118882 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fdc0c022030 con 0x55c1c1ed08e0 2023-11-28T11:11:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.880+0000 7fdc2fffab80 10 client.6285 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:11:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.880+0000 7fdc2fffab80 10 client.6285 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:11:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.880+0000 7fdc2fffab80 20 client.6285 populate_metadata read hostname 'smithi028' 2023-11-28T11:11:13.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.880+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/714118882 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55c1c1ec3bd0 0x55c1c1ebfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.880+0000 7fdc2fffab80 4 client.6285 mds_command: new command op to 6054 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:11:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.880+0000 7fdc2fffab80 1 -- 172.21.15.28:0/714118882 --> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c1c1c7cfe0 con 0x55c1c1ec3bd0 2023-11-28T11:11:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.880+0000 7fdc21c0d700 1 --2- 172.21.15.28:0/714118882 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55c1c1ec3bd0 0x55c1c1ebfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.880+0000 7fdbfcff9700 10 client.6285 ms_handle_connect on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.880+0000 7fdc21c0d700 1 --2- 172.21.15.28:0/714118882 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55c1c1ec3bd0 0x55c1c1ebfed0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=253bf11fefb7bcbe server_cookie=dc46c13661562fb1 in_seq=0 out_seq=0 2023-11-28T11:11:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.881+0000 7fdbfcff9700 1 -- 172.21.15.28:0/714118882 <== mds.0 v2:172.21.15.59:6832/3261856585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55c1c1c7cfe0 con 0x55c1c1ec3bd0 2023-11-28T11:11:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.881+0000 7fdbfcff9700 10 client.6285 handle_command_reply: tid=0 2023-11-28T11:11:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.881+0000 7fdc2fffab80 1 client.6285 shutdown 2023-11-28T11:11:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.881+0000 7fdc2fffab80 1 -- 172.21.15.28:0/714118882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 msgr2=0x55c1c1ed1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.881+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/714118882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ed1aa0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fdc0c009b60 tx=0x7fdc0c00e000 comp rx=0 tx=0).stop 2023-11-28T11:11:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.881+0000 7fdc2fffab80 1 -- 172.21.15.28:0/714118882 shutdown_connections 2023-11-28T11:11:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.881+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/714118882 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55c1c1ec3bd0 0x55c1c1ebfed0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.881+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/714118882 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ed11f0 0x55c1c1ebf4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.881+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/714118882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ece200 0x55c1c1ecfa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.881+0000 7fdbfcff9700 0 client.6285 ms_handle_reset on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.881+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/714118882 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ed1aa0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.881+0000 7fdc2fffab80 1 -- 172.21.15.28:0/714118882 >> 172.21.15.28:0/714118882 conn(0x55c1c1ea3e30 msgr2=0x55c1c1ea5300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.881+0000 7fdc2fffab80 1 -- 172.21.15.28:0/714118882 shutdown_connections 2023-11-28T11:11:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.881+0000 7fdc2fffab80 1 -- 172.21.15.28:0/714118882 wait complete. 2023-11-28T11:11:13.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.881+0000 7fdc2fffab80 20 client.6285 trim_cache size 0 max 16384 2023-11-28T11:11:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.887+0000 7fb2a4599700 1 Processor -- start 2023-11-28T11:11:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.887+0000 7fb2a4599700 1 -- start start 2023-11-28T11:11:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.888+0000 7fb2a4599700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb29c0af040 0x7fb29c0cf050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.888+0000 7fb2a4599700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb29c0ca790 0x7fb29c0cabd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.893 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.888+0000 7fb2a4599700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb29c0cebc0 con 0x7fb29c1601b0 2023-11-28T11:11:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.888+0000 7fb2a4599700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb29c0af410 con 0x7fb29c0af040 2023-11-28T11:11:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.888+0000 7fb2a4599700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0cb110 con 0x7fb29c0ca790 2023-11-28T11:11:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.889+0000 7fb2a2b36700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb29c0ca790 0x7fb29c0cabd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.889+0000 7fb2a1b34700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fb29c1601b0 0x7fb29c0cd6d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:49652/0 (socket says 172.21.15.28:49652) 2023-11-28T11:11:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.889+0000 7fb2a2335700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb29c0af040 0x7fb29c0cf050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.889+0000 7fb2a1b34700 1 -- 172.21.15.28:0/1642976557 learned_addr learned my addr 172.21.15.28:0/1642976557 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.889+0000 7fb2a2b36700 1 -- 172.21.15.28:0/1642976557 >> v2:172.21.15.59:3300/0 conn(0x7fb29c0af040 msgr2=0x7fb29c0cf050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.889+0000 7fb2a2b36700 1 --2- 172.21.15.28:0/1642976557 >> v2:172.21.15.59:3300/0 conn(0x7fb29c0af040 0x7fb29c0cf050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.889+0000 7fb2a2b36700 1 -- 172.21.15.28:0/1642976557 >> v1:172.21.15.59:6789/0 conn(0x7fb29c1601b0 legacy=0x7fb29c0cd6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.889+0000 7fb2a2b36700 1 -- 172.21.15.28:0/1642976557 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb29c0cdfb0 con 0x7fb29c0ca790 2023-11-28T11:11:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.889+0000 7fb2a2335700 1 --2- 172.21.15.28:0/1642976557 >> v2:172.21.15.59:3300/0 conn(0x7fb29c0af040 0x7fb29c0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.889+0000 7fb2a2b36700 1 --2- 172.21.15.28:0/1642976557 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb29c0ca790 0x7fb29c0cabd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb294009d90 tx=0x7fb29400b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7c3c65045128487b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.889+0000 7fb2a1333700 1 -- 172.21.15.28:0/1642976557 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb29400e070 con 0x7fb29c0ca790 2023-11-28T11:11:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.889+0000 7fb2a1333700 1 -- 172.21.15.28:0/1642976557 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb29400bb20 con 0x7fb29c0ca790 2023-11-28T11:11:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.890+0000 7fb2a1333700 1 -- 172.21.15.28:0/1642976557 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294012430 con 0x7fb29c0ca790 2023-11-28T11:11:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.890+0000 7fb2a4599700 1 -- 172.21.15.28:0/1642976557 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb29c0ca790 msgr2=0x7fb29c0cabd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.890+0000 7fb2a4599700 1 --2- 172.21.15.28:0/1642976557 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb29c0ca790 0x7fb29c0cabd0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb294009d90 tx=0x7fb29400b330 comp rx=0 tx=0).stop 2023-11-28T11:11:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.890+0000 7fb2a4599700 1 -- 172.21.15.28:0/1642976557 shutdown_connections 2023-11-28T11:11:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.890+0000 7fb2a4599700 1 --2- 172.21.15.28:0/1642976557 >> v2:172.21.15.59:3300/0 conn(0x7fb29c0af040 0x7fb29c0cf050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.890+0000 7fb2a4599700 1 --2- 172.21.15.28:0/1642976557 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb29c0ca790 0x7fb29c0cabd0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.890+0000 7fb2a4599700 1 -- 172.21.15.28:0/1642976557 >> 172.21.15.28:0/1642976557 conn(0x7fb29c0cc580 msgr2=0x7fb29c0cc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:13.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.890+0000 7fb2a4599700 1 -- 172.21.15.28:0/1642976557 shutdown_connections 2023-11-28T11:11:13.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.890+0000 7fb2a4599700 1 -- 172.21.15.28:0/1642976557 wait complete. 2023-11-28T11:11:13.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.891+0000 7fb2a4599700 1 Processor -- start 2023-11-28T11:11:13.896 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.891+0000 7fb2a4599700 1 -- start start 2023-11-28T11:11:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.892+0000 7fb2a4599700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb29c0af040 0x7fb29c1f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.892+0000 7fb2a4599700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb29c0ca790 0x7fb29c1506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.892+0000 7fb2a4599700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb29c1601b0 0x7fb29c0b15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.892+0000 7fb2a4599700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0c6410 con 0x7fb29c1601b0 2023-11-28T11:11:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.892+0000 7fb2a4599700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c1f3620 con 0x7fb29c0af040 2023-11-28T11:11:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.892+0000 7fb2a4599700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c0b1160 con 0x7fb29c0ca790 2023-11-28T11:11:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.892+0000 7fb2a2b36700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb29c1601b0 0x7fb29c0b15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.892+0000 7fb2a2b36700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb29c1601b0 0x7fb29c0b15f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50816/0 (socket says 172.21.15.28:50816) 2023-11-28T11:11:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.892+0000 7fb2a2b36700 1 -- 172.21.15.28:0/428768153 learned_addr learned my addr 172.21.15.28:0/428768153 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.892+0000 7fb2a2335700 1 --2- 172.21.15.28:0/428768153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb29c0af040 0x7fb29c1f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.892+0000 7fb2a1b34700 1 --2- 172.21.15.28:0/428768153 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb29c0ca790 0x7fb29c1506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.893+0000 7fb2a2b36700 1 -- 172.21.15.28:0/428768153 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb29c0ca790 msgr2=0x7fb29c1506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.893+0000 7fb2a2b36700 1 --2- 172.21.15.28:0/428768153 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb29c0ca790 0x7fb29c1506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.893+0000 7fb2a2b36700 1 -- 172.21.15.28:0/428768153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb29c0af040 msgr2=0x7fb29c1f30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.893+0000 7fb2a2b36700 1 --2- 172.21.15.28:0/428768153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb29c0af040 0x7fb29c1f30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.893+0000 7fb2a2b36700 1 -- 172.21.15.28:0/428768153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2980094a0 con 0x7fb29c1601b0 2023-11-28T11:11:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.893+0000 7fb2a1b34700 1 --2- 172.21.15.28:0/428768153 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb29c0ca790 0x7fb29c1506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.893+0000 7fb2a2335700 1 --2- 172.21.15.28:0/428768153 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb29c0af040 0x7fb29c1f30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.893+0000 7fb2a2b36700 1 --2- 172.21.15.28:0/428768153 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb29c1601b0 0x7fb29c0b15f0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7fb294014040 tx=0x7fb294004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.893+0000 7fb28effd700 1 -- 172.21.15.28:0/428768153 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294012430 con 0x7fb29c1601b0 2023-11-28T11:11:13.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.893+0000 7fb28effd700 1 -- 172.21.15.28:0/428768153 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb294020070 con 0x7fb29c1601b0 2023-11-28T11:11:13.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.893+0000 7fb28effd700 1 -- 172.21.15.28:0/428768153 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294003ea0 con 0x7fb29c1601b0 2023-11-28T11:11:13.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.893+0000 7fb2a4599700 1 -- 172.21.15.28:0/428768153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb294009a70 con 0x7fb29c1601b0 2023-11-28T11:11:13.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.893+0000 7fb2a4599700 1 -- 172.21.15.28:0/428768153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c0ce570 con 0x7fb29c1601b0 2023-11-28T11:11:13.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.894+0000 7fb2a4599700 1 -- 172.21.15.28:0/428768153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fb29c0ce290 con 0x7fb29c1601b0 2023-11-28T11:11:13.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.894+0000 7fb28effd700 1 -- 172.21.15.28:0/428768153 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb29402e070 con 0x7fb29c1601b0 2023-11-28T11:11:13.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.895+0000 7fb28effd700 1 --2- 172.21.15.28:0/428768153 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb2800682f0 0x7fb28006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.895+0000 7fb28effd700 1 -- 172.21.15.28:0/428768153 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fb294022070 con 0x7fb29c1601b0 2023-11-28T11:11:13.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.895+0000 7fb2a2335700 1 --2- 172.21.15.28:0/428768153 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb2800682f0 0x7fb28006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.895+0000 7fb28effd700 1 -- 172.21.15.28:0/428768153 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+502 (secure 0 0 0) 0x7fb294088510 con 0x7fb29c1601b0 2023-11-28T11:11:13.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.895+0000 7fb2a4599700 1 -- 172.21.15.28:0/428768153 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({log-info=0+}) v3 -- 0x7fb29c0ce850 con 0x7fb29c1601b0 2023-11-28T11:11:13.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.895+0000 7fb2a2335700 1 --2- 172.21.15.28:0/428768153 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb2800682f0 0x7fb28006a7a0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fb298009dc0 tx=0x7fb298008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.901+0000 7fdc2fffab80 1 Processor -- start 2023-11-28T11:11:13.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.901+0000 7fdc2fffab80 1 -- start start 2023-11-28T11:11:13.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2fffab80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ec3bd0 0x55c1c1ea6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2fffab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ece200 0x55c1c1ec1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2fffab80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ec2290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2fffab80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c1ea0bf0 con 0x55c1c1ed08e0 2023-11-28T11:11:13.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2fffab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c1e9f650 con 0x55c1c1ece200 2023-11-28T11:11:13.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2fffab80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1c1ec0470 con 0x55c1c1ec3bd0 2023-11-28T11:11:13.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2240e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ec2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.907 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2240e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ec2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50822/0 (socket says 172.21.15.28:50822) 2023-11-28T11:11:13.907 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2240e700 1 -- 172.21.15.28:0/1400913003 learned_addr learned my addr 172.21.15.28:0/1400913003 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:13.907 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2140c700 1 --2- 172.21.15.28:0/1400913003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ece200 0x55c1c1ec1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.907 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc21c0d700 1 --2- 172.21.15.28:0/1400913003 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ec3bd0 0x55c1c1ea6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.907 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2240e700 1 -- 172.21.15.28:0/1400913003 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ec3bd0 msgr2=0x55c1c1ea6a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.907 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2240e700 1 --2- 172.21.15.28:0/1400913003 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ec3bd0 0x55c1c1ea6a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.907 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2240e700 1 -- 172.21.15.28:0/1400913003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ece200 msgr2=0x55c1c1ec1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.907 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2240e700 1 --2- 172.21.15.28:0/1400913003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ece200 0x55c1c1ec1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.907 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2240e700 1 -- 172.21.15.28:0/1400913003 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c1c1dbf210 con 0x55c1c1ed08e0 2023-11-28T11:11:13.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc21c0d700 1 --2- 172.21.15.28:0/1400913003 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ec3bd0 0x55c1c1ea6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:13.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdc2240e700 1 --2- 172.21.15.28:0/1400913003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ec2290 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400c040 tx=0x7fdc1405c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f97ba8cfe5250e2b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdbfffff700 1 -- 172.21.15.28:0/1400913003 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14055d40 con 0x55c1c1ed08e0 2023-11-28T11:11:13.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdbfffff700 1 -- 172.21.15.28:0/1400913003 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc14055ea0 con 0x55c1c1ed08e0 2023-11-28T11:11:13.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.902+0000 7fdbfffff700 1 -- 172.21.15.28:0/1400913003 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc14056010 con 0x55c1c1ed08e0 2023-11-28T11:11:13.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.903+0000 7fdc2fffab80 1 -- 172.21.15.28:0/1400913003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 msgr2=0x55c1c1ec2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.903+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/1400913003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ec2290 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fdc1400c040 tx=0x7fdc1405c040 comp rx=0 tx=0).stop 2023-11-28T11:11:13.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.903+0000 7fdc2fffab80 1 -- 172.21.15.28:0/1400913003 shutdown_connections 2023-11-28T11:11:13.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.903+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/1400913003 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ed08e0 0x55c1c1ec2290 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.903+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/1400913003 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ec3bd0 0x55c1c1ea6a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.903+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/1400913003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ece200 0x55c1c1ec1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.903+0000 7fdc2fffab80 1 -- 172.21.15.28:0/1400913003 >> 172.21.15.28:0/1400913003 conn(0x55c1c1ea3e30 msgr2=0x55c1c1ea03a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:13.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.903+0000 7fdc2fffab80 1 -- 172.21.15.28:0/1400913003 shutdown_connections 2023-11-28T11:11:13.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.903+0000 7fdc2fffab80 1 -- 172.21.15.28:0/1400913003 wait complete. 2023-11-28T11:11:13.909 INFO:tasks.ceph.mds.d.smithi059.stderr:2023-11-28T11:11:13.909+0000 7fae49d50700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6246=0-4194304@1} caps={6246=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6246 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x562a8ff59b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:11:13.913 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.904+0000 7fdc2fffab80 1 -- start start 2023-11-28T11:11:13.913 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.904+0000 7fdc2fffab80 1 Processor -- start 2023-11-28T11:11:13.913 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc2fffab80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ec3bd0 0x55c1c1efcf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.913 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc2fffab80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ece200 0x55c1c1eee790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.913 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc2fffab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ed08e0 0x55c1c1eccec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.913 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc2fffab80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c1ecdad0 con 0x55c1c1ec3bd0 2023-11-28T11:11:13.913 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc2fffab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c1c1eedd40 con 0x55c1c1ed08e0 2023-11-28T11:11:13.913 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc2fffab80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c1c1ea54a0 con 0x55c1c1ece200 2023-11-28T11:11:13.914 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc2fffab80 10 client.0 _ll_register_callbacks cb 0x55c1c1c9e310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:13.914 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc21c0d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ec3bd0 0x55c1c1efcf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.914 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc21c0d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ec3bd0 0x55c1c1efcf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50826/0 (socket says 172.21.15.28:50826) 2023-11-28T11:11:13.914 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc21c0d700 1 -- 172.21.15.28:0/3623595207 learned_addr learned my addr 172.21.15.28:0/3623595207 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:13.914 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc2240e700 1 --2- 172.21.15.28:0/3623595207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ed08e0 0x55c1c1eccec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.914 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc2140c700 1 --2- 172.21.15.28:0/3623595207 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ece200 0x55c1c1eee790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.914 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc21c0d700 1 -- 172.21.15.28:0/3623595207 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ece200 msgr2=0x55c1c1eee790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.914 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc21c0d700 1 --2- 172.21.15.28:0/3623595207 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ece200 0x55c1c1eee790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.914 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc21c0d700 1 -- 172.21.15.28:0/3623595207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ed08e0 msgr2=0x55c1c1eccec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:13.914 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc21c0d700 1 --2- 172.21.15.28:0/3623595207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ed08e0 0x55c1c1eccec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc21c0d700 1 -- 172.21.15.28:0/3623595207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc1c0cf260 con 0x55c1c1ec3bd0 2023-11-28T11:11:13.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc21c0d700 1 --2- 172.21.15.28:0/3623595207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ec3bd0 0x55c1c1efcf90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fdc18008fd0 tx=0x7fdc18000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d1796f99b14022c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:13.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdbfcff9700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:11:13.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdbfcff9700 1 -- 172.21.15.28:0/3623595207 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc18028040 con 0x55c1c1ec3bd0 2023-11-28T11:11:13.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdbfcff9700 1 -- 172.21.15.28:0/3623595207 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fdc18022ea0 con 0x55c1c1ec3bd0 2023-11-28T11:11:13.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdc2fffab80 1 -- 172.21.15.28:0/3623595207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c1c1db6230 con 0x55c1c1ec3bd0 2023-11-28T11:11:13.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.905+0000 7fdbfcff9700 1 -- 172.21.15.28:0/3623595207 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdc1802f260 con 0x55c1c1ec3bd0 2023-11-28T11:11:13.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.906+0000 7fdc2fffab80 10 client.6294 fetch_fsmap learned FSMap version 25 2023-11-28T11:11:13.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.906+0000 7fdc2fffab80 1 -- 172.21.15.28:0/3623595207 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x55c1c1c4d2d0 con 0x55c1c1ec3bd0 2023-11-28T11:11:13.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.906+0000 7fdbfcff9700 1 -- 172.21.15.28:0/3623595207 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c1c1db6230 con 0x55c1c1ec3bd0 2023-11-28T11:11:13.916 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.906+0000 7fdbfcff9700 1 -- 172.21.15.28:0/3623595207 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fdc1802f8a0 con 0x55c1c1ec3bd0 2023-11-28T11:11:13.916 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.906+0000 7fdc2fffab80 10 client.6294 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:11:13.916 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.906+0000 7fdc2fffab80 10 client.6294 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:11:13.916 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.906+0000 7fdc2fffab80 20 client.6294 populate_metadata read hostname 'smithi028' 2023-11-28T11:11:13.916 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.906+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/3623595207 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55c1c1ed11f0 0x55c1c1ed3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:13.916 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.906+0000 7fdc2fffab80 4 client.6294 mds_command: new command op to 6054 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}] 2023-11-28T11:11:13.916 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.906+0000 7fdc2fffab80 1 -- 172.21.15.28:0/3623595207 --> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["repair", "recursive"]}) v1 -- 0x55c1c1da0240 con 0x55c1c1ed11f0 2023-11-28T11:11:13.916 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.906+0000 7fdc2140c700 1 --2- 172.21.15.28:0/3623595207 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55c1c1ed11f0 0x55c1c1ed3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:13.916 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.907+0000 7fdbfcff9700 10 client.6294 ms_handle_connect on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:13.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.907+0000 7fdc2140c700 1 --2- 172.21.15.28:0/3623595207 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55c1c1ed11f0 0x55c1c1ed3d40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db53c1d34268b31a server_cookie=19c76cf9806988b7 in_seq=0 out_seq=0 2023-11-28T11:11:13.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.907+0000 7fdbfcff9700 1 -- 172.21.15.28:0/3623595207 <== mds.0 v2:172.21.15.59:6832/3261856585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55c1c1da0240 con 0x55c1c1ed11f0 2023-11-28T11:11:13.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.907+0000 7fdbfcff9700 10 client.6294 handle_command_reply: tid=0 2023-11-28T11:11:13.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.907+0000 7fdc2fffab80 1 client.6294 shutdown 2023-11-28T11:11:13.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc2fffab80 1 -- 172.21.15.28:0/3623595207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ec3bd0 msgr2=0x55c1c1efcf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/3623595207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ec3bd0 0x55c1c1efcf90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fdc18008fd0 tx=0x7fdc18000ed0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc2fffab80 1 -- 172.21.15.28:0/3623595207 shutdown_connections 2023-11-28T11:11:13.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/3623595207 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x55c1c1ed11f0 0x55c1c1ed3d40 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.918 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/3623595207 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55c1c1ec3bd0 0x55c1c1efcf90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.918 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/3623595207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c1c1ed08e0 0x55c1c1eccec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.918 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdbfcff9700 0 client.6294 ms_handle_reset on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:13.918 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc2fffab80 1 --2- 172.21.15.28:0/3623595207 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55c1c1ece200 0x55c1c1eee790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.918 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc2fffab80 1 -- 172.21.15.28:0/3623595207 >> 172.21.15.28:0/3623595207 conn(0x55c1c1ea3e30 msgr2=0x55c1c1ea7a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:13.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc2fffab80 1 -- 172.21.15.28:0/3623595207 shutdown_connections 2023-11-28T11:11:13.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc2fffab80 1 -- 172.21.15.28:0/3623595207 wait complete. 2023-11-28T11:11:13.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc2fffab80 20 client.6294 trim_cache size 0 max 16384 2023-11-28T11:11:13.919 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:11:13.920 INFO:teuthology.orchestra.run.smithi028.stdout: "return_code": 0, 2023-11-28T11:11:13.920 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_tag": "21d7f5a7-2537-46da-8d66-ddc008c3d1e3", 2023-11-28T11:11:13.920 INFO:teuthology.orchestra.run.smithi028.stdout: "mode": "asynchronous" 2023-11-28T11:11:13.920 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:11:13.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc23e71700 1 -- 172.21.15.28:0/373125710 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdc000682f0 msgr2=0x7fdc0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc23e71700 1 --2- 172.21.15.28:0/373125710 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdc000682f0 0x7fdc0006a7a0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fdc18000f60 tx=0x7fdc18007be0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc23e71700 1 -- 172.21.15.28:0/373125710 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc1c15a6c0 msgr2=0x7fdc1c0c81f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:13.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc23e71700 1 --2- 172.21.15.28:0/373125710 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc1c15a6c0 0x7fdc1c0c81f0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fdc14007e60 tx=0x7fdc1400e3c0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc23e71700 1 -- 172.21.15.28:0/373125710 shutdown_connections 2023-11-28T11:11:13.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc23e71700 1 --2- 172.21.15.28:0/373125710 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdc000682f0 0x7fdc0006a7a0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc23e71700 1 --2- 172.21.15.28:0/373125710 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdc1c15a6c0 0x7fdc1c0c81f0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc23e71700 1 --2- 172.21.15.28:0/373125710 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdc1c0b2660 0x7fdc1c0b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc23e71700 1 --2- 172.21.15.28:0/373125710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdc1c0cb230 0x7fdc1c0c7cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:13.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.908+0000 7fdc23e71700 1 -- 172.21.15.28:0/373125710 >> 172.21.15.28:0/373125710 conn(0x7fdc1c0cd660 msgr2=0x7fdc1c0b0620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:13.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.909+0000 7fdc23e71700 1 -- 172.21.15.28:0/373125710 shutdown_connections 2023-11-28T11:11:13.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:13.909+0000 7fdc23e71700 1 -- 172.21.15.28:0/373125710 wait complete. 2023-11-28T11:11:13.923 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T11:11:14.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.048+0000 7f3babc2a700 1 Processor -- start 2023-11-28T11:11:14.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.048+0000 7f3babc2a700 1 -- start start 2023-11-28T11:11:14.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.049+0000 7f3babc2a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3ba40b2d90 0x7f3ba40ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.049+0000 7f3babc2a700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba40cd2a0 con 0x7f3ba40b62d0 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.049+0000 7f3babc2a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ba40d0400 con 0x7f3ba40cb630 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.049+0000 7f3babc2a700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ba40d06e0 con 0x7f3ba40b2d90 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.049+0000 7f3baa1c7700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3ba40b2d90 0x7f3ba40ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.049+0000 7f3ba99c6700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3ba40b62d0 0x7f3ba406f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38892/0 (socket says 172.21.15.28:38892) 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.049+0000 7f3ba99c6700 1 -- 172.21.15.28:0/1918786860 learned_addr learned my addr 172.21.15.28:0/1918786860 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.049+0000 7f3baa1c7700 1 -- 172.21.15.28:0/1918786860 >> v1:172.21.15.59:6789/0 conn(0x7f3ba40cb630 legacy=0x7f3ba40b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.049+0000 7f3baa1c7700 1 -- 172.21.15.28:0/1918786860 >> v1:172.21.15.28:6789/0 conn(0x7f3ba40b62d0 legacy=0x7f3ba406f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.049+0000 7f3baa1c7700 1 -- 172.21.15.28:0/1918786860 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ba40cffb0 con 0x7f3ba40b2d90 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.049+0000 7f3ba89c4700 1 -- 172.21.15.28:0/1918786860 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546071106 0 0) 0x7f3ba40cd2a0 con 0x7f3ba40b62d0 2023-11-28T11:11:14.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.050+0000 7f3baa1c7700 1 --2- 172.21.15.28:0/1918786860 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ba40b2d90 0x7f3ba40ce7d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009d90 tx=0x7f3b9c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938d98ca9c202cee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:14.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.050+0000 7f3ba89c4700 1 -- 172.21.15.28:0/1918786860 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c00e070 con 0x7f3ba40b2d90 2023-11-28T11:11:14.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.050+0000 7f3ba89c4700 1 -- 172.21.15.28:0/1918786860 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9c00bbd0 con 0x7f3ba40b2d90 2023-11-28T11:11:14.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.050+0000 7f3ba89c4700 1 -- 172.21.15.28:0/1918786860 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9c012430 con 0x7f3ba40b2d90 2023-11-28T11:11:14.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.050+0000 7f3babc2a700 1 -- 172.21.15.28:0/1918786860 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ba40b2d90 msgr2=0x7f3ba40ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.050+0000 7f3babc2a700 1 --2- 172.21.15.28:0/1918786860 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ba40b2d90 0x7f3ba40ce7d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f3b9c009d90 tx=0x7f3b9c00b330 comp rx=0 tx=0).stop 2023-11-28T11:11:14.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.050+0000 7f3babc2a700 1 -- 172.21.15.28:0/1918786860 shutdown_connections 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.050+0000 7f3babc2a700 1 --2- 172.21.15.28:0/1918786860 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ba40b2d90 0x7f3ba40ce7d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.050+0000 7f3babc2a700 1 -- 172.21.15.28:0/1918786860 >> 172.21.15.28:0/1918786860 conn(0x7f3ba40cd680 msgr2=0x7f3ba40cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.051+0000 7f3babc2a700 1 -- 172.21.15.28:0/1918786860 shutdown_connections 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.051+0000 7f3babc2a700 1 -- 172.21.15.28:0/1918786860 wait complete. 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.052+0000 7f3babc2a700 1 Processor -- start 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.052+0000 7f3babc2a700 1 -- start start 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.052+0000 7f3babc2a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ba40b2d90 0x7f3ba41ef9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.052+0000 7f3babc2a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ba40b62d0 0x7f3ba40c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.052+0000 7f3babc2a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ba40cb630 0x7f3ba40c69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.052+0000 7f3babc2a700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba41df840 con 0x7f3ba40b2d90 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.052+0000 7f3babc2a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba41f0040 con 0x7f3ba40b62d0 2023-11-28T11:11:14.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.052+0000 7f3babc2a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ba41f0320 con 0x7f3ba40cb630 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.052+0000 7f3ba99c6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ba40b2d90 0x7f3ba41ef9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3ba99c6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ba40b2d90 0x7f3ba41ef9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50840/0 (socket says 172.21.15.28:50840) 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3ba99c6700 1 -- 172.21.15.28:0/2052248680 learned_addr learned my addr 172.21.15.28:0/2052248680 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3ba91c5700 1 --2- 172.21.15.28:0/2052248680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ba40b62d0 0x7f3ba40c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3baa1c7700 1 --2- 172.21.15.28:0/2052248680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ba40cb630 0x7f3ba40c69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3ba99c6700 1 -- 172.21.15.28:0/2052248680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ba40cb630 msgr2=0x7f3ba40c69b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3ba99c6700 1 --2- 172.21.15.28:0/2052248680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ba40cb630 0x7f3ba40c69b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3ba99c6700 1 -- 172.21.15.28:0/2052248680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ba40b62d0 msgr2=0x7f3ba40c9bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3ba99c6700 1 --2- 172.21.15.28:0/2052248680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ba40b62d0 0x7f3ba40c9bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3ba99c6700 1 -- 172.21.15.28:0/2052248680 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b9c009a70 con 0x7f3ba40b2d90 2023-11-28T11:11:14.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3ba91c5700 1 --2- 172.21.15.28:0/2052248680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ba40b62d0 0x7f3ba40c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:14.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3ba99c6700 1 --2- 172.21.15.28:0/2052248680 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ba40b2d90 0x7f3ba41ef9b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400c520 tx=0x7f3b9400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:14.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3b9a7fc700 1 -- 172.21.15.28:0/2052248680 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94004c80 con 0x7f3ba40b2d90 2023-11-28T11:11:14.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3b9a7fc700 1 -- 172.21.15.28:0/2052248680 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b94004e00 con 0x7f3ba40b2d90 2023-11-28T11:11:14.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3babc2a700 1 -- 172.21.15.28:0/2052248680 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ba40cce50 con 0x7f3ba40b2d90 2023-11-28T11:11:14.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3b9a7fc700 1 -- 172.21.15.28:0/2052248680 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b94007310 con 0x7f3ba40b2d90 2023-11-28T11:11:14.059 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.053+0000 7f3babc2a700 1 -- 172.21.15.28:0/2052248680 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba40b66a0 con 0x7f3ba40b2d90 2023-11-28T11:11:14.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.056+0000 7f3b9a7fc700 1 -- 172.21.15.28:0/2052248680 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3b9401a020 con 0x7f3ba40b2d90 2023-11-28T11:11:14.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.056+0000 7f3b9a7fc700 1 --2- 172.21.15.28:0/2052248680 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3b880682f0 0x7f3b8806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.056+0000 7f3ba91c5700 1 --2- 172.21.15.28:0/2052248680 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3b880682f0 0x7f3b8806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.056+0000 7f3b9a7fc700 1 -- 172.21.15.28:0/2052248680 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3b94085ab0 con 0x7f3ba40b2d90 2023-11-28T11:11:14.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.057+0000 7f3ba91c5700 1 --2- 172.21.15.28:0/2052248680 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3b880682f0 0x7f3b8806a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40c7700 tx=0x7f3ba0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:14.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.057+0000 7f3bb7db3b80 1 Processor -- start 2023-11-28T11:11:14.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.058+0000 7f3bb7db3b80 1 -- start start 2023-11-28T11:11:14.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.058+0000 7f3bb7db3b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e6860 0x559c6f2bca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.058+0000 7f3bb7db3b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e4180 0x559c6f2e6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.058+0000 7f3bb7db3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e7170 0x559c6f3027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.058+0000 7f3bb7db3b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6f1a6c10 con 0x559c6f2e6860 2023-11-28T11:11:14.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.058+0000 7f3bb7db3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6f2153a0 con 0x559c6f2e7170 2023-11-28T11:11:14.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.058+0000 7f3bb7db3b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c6f20e8d0 con 0x559c6f2e4180 2023-11-28T11:11:14.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.058+0000 7f3ba99c6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e6860 0x559c6f2bca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.058+0000 7f3ba99c6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e6860 0x559c6f2bca70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50848/0 (socket says 172.21.15.28:50848) 2023-11-28T11:11:14.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.058+0000 7f3ba99c6700 1 -- 172.21.15.28:0/1958657626 learned_addr learned my addr 172.21.15.28:0/1958657626 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:14.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.059+0000 7f3ba99c6700 1 -- 172.21.15.28:0/1958657626 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e4180 msgr2=0x559c6f2e6c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:14.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.059+0000 7f3ba99c6700 1 --2- 172.21.15.28:0/1958657626 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e4180 0x559c6f2e6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.059+0000 7f3ba99c6700 1 -- 172.21.15.28:0/1958657626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e7170 msgr2=0x559c6f3027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:14.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.059+0000 7f3ba99c6700 1 --2- 172.21.15.28:0/1958657626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e7170 0x559c6f3027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.059+0000 7f3ba99c6700 1 -- 172.21.15.28:0/1958657626 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c6f090630 con 0x559c6f2e6860 2023-11-28T11:11:14.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.059+0000 7f3ba99c6700 1 --2- 172.21.15.28:0/1958657626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e6860 0x559c6f2bca70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3b9405d910 tx=0x7f3b9405dec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab47f137533ddd6d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:14.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.059+0000 7f3b877fe700 1 -- 172.21.15.28:0/1958657626 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b940554a0 con 0x559c6f2e6860 2023-11-28T11:11:14.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.059+0000 7f3b877fe700 1 -- 172.21.15.28:0/1958657626 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b9406d070 con 0x559c6f2e6860 2023-11-28T11:11:14.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.059+0000 7f3b877fe700 1 -- 172.21.15.28:0/1958657626 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9405dbd0 con 0x559c6f2e6860 2023-11-28T11:11:14.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.059+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/1958657626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e6860 msgr2=0x559c6f2bca70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.059+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/1958657626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e6860 0x559c6f2bca70 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f3b9405d910 tx=0x7f3b9405dec0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.060+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/1958657626 shutdown_connections 2023-11-28T11:11:14.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.060+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/1958657626 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e6860 0x559c6f2bca70 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.060+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/1958657626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e7170 0x559c6f3027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.060+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/1958657626 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e4180 0x559c6f2e6c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.060+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/1958657626 >> 172.21.15.28:0/1958657626 conn(0x559c6f2b9db0 msgr2=0x559c6f2ba740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:14.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.060+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/1958657626 shutdown_connections 2023-11-28T11:11:14.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.060+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/1958657626 wait complete. 2023-11-28T11:11:14.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.060+0000 7f3bb7db3b80 1 -- start start 2023-11-28T11:11:14.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.060+0000 7f3bb7db3b80 1 Processor -- start 2023-11-28T11:11:14.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.061+0000 7f3bb7db3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e4180 0x559c6f2d8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.061+0000 7f3bb7db3b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e6860 0x559c6f2d69a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.061+0000 7f3bb7db3b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 0x559c6f2e4f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.061+0000 7f3bb7db3b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6f092920 con 0x559c6f2e6860 2023-11-28T11:11:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.061+0000 7f3bb7db3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6f2e5fe0 con 0x559c6f2e4180 2023-11-28T11:11:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.061+0000 7f3bb7db3b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c6f2e7c50 con 0x559c6f2e7170 2023-11-28T11:11:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.061+0000 7f3bb7db3b80 10 client.0 _ll_register_callbacks cb 0x559c6f0b4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.061+0000 7f3ba91c5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e6860 0x559c6f2d69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.061+0000 7f3ba91c5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e6860 0x559c6f2d69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50856/0 (socket says 172.21.15.28:50856) 2023-11-28T11:11:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.061+0000 7f3ba91c5700 1 -- 172.21.15.28:0/4262785926 learned_addr learned my addr 172.21.15.28:0/4262785926 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.061+0000 7f3ba99c6700 1 --2- 172.21.15.28:0/4262785926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e4180 0x559c6f2d8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.061+0000 7f3baa1c7700 1 --2- 172.21.15.28:0/4262785926 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 0x559c6f2e4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.062+0000 7f3ba91c5700 1 -- 172.21.15.28:0/4262785926 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 msgr2=0x559c6f2e4f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:14.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.062+0000 7f3ba91c5700 1 --2- 172.21.15.28:0/4262785926 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 0x559c6f2e4f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.062+0000 7f3ba91c5700 1 -- 172.21.15.28:0/4262785926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e4180 msgr2=0x559c6f2d8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:14.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.062+0000 7f3ba91c5700 1 --2- 172.21.15.28:0/4262785926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e4180 0x559c6f2d8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.062+0000 7f3ba91c5700 1 -- 172.21.15.28:0/4262785926 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ba40b66a0 con 0x559c6f2e6860 2023-11-28T11:11:14.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.062+0000 7f3ba91c5700 1 --2- 172.21.15.28:0/4262785926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e6860 0x559c6f2d69a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0003960 tx=0x7f3ba0003c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae8afe5494558227 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:14.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.062+0000 7f3b84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:11:14.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.062+0000 7f3b84ff9700 1 -- 172.21.15.28:0/4262785926 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ba0013630 con 0x559c6f2e6860 2023-11-28T11:11:14.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.062+0000 7f3b84ff9700 1 -- 172.21.15.28:0/4262785926 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3ba0023d70 con 0x559c6f2e6860 2023-11-28T11:11:14.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.062+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/4262785926 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c6f1dc190 con 0x559c6f2e6860 2023-11-28T11:11:14.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.062+0000 7f3b84ff9700 1 -- 172.21.15.28:0/4262785926 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3ba0013790 con 0x559c6f2e6860 2023-11-28T11:11:14.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.062+0000 7f3bb7db3b80 10 client.6309 fetch_fsmap learned FSMap version 25 2023-11-28T11:11:14.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.062+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/4262785926 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x559c6f03cde0 con 0x559c6f2e6860 2023-11-28T11:11:14.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.062+0000 7f3b84ff9700 1 -- 172.21.15.28:0/4262785926 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559c6f1dc190 con 0x559c6f2e6860 2023-11-28T11:11:14.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.062+0000 7f3b84ff9700 1 -- 172.21.15.28:0/4262785926 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f3ba00138f0 con 0x559c6f2e6860 2023-11-28T11:11:14.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.063+0000 7f3bb7db3b80 10 client.6309 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:11:14.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.063+0000 7f3bb7db3b80 10 client.6309 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:11:14.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.063+0000 7f3bb7db3b80 20 client.6309 populate_metadata read hostname 'smithi028' 2023-11-28T11:11:14.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.063+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/4262785926 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x559c6f2e5990 0x559c6f2f9750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.063+0000 7f3bb7db3b80 4 client.6309 mds_command: new command op to 6054 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:11:14.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.063+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/4262785926 --> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559c6f092f60 con 0x559c6f2e5990 2023-11-28T11:11:14.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.063+0000 7f3ba99c6700 1 --2- 172.21.15.28:0/4262785926 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x559c6f2e5990 0x559c6f2f9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.063+0000 7f3ba99c6700 1 --2- 172.21.15.28:0/4262785926 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x559c6f2e5990 0x559c6f2f9750 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=745cc2af1a719330 server_cookie=d173e78c5b013b73 in_seq=0 out_seq=0 2023-11-28T11:11:14.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.063+0000 7f3b84ff9700 10 client.6309 ms_handle_connect on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:14.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.064+0000 7f3b84ff9700 1 -- 172.21.15.28:0/4262785926 <== mds.0 v2:172.21.15.59:6832/3261856585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x559c6f092f60 con 0x559c6f2e5990 2023-11-28T11:11:14.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.064+0000 7f3b84ff9700 10 client.6309 handle_command_reply: tid=0 2023-11-28T11:11:14.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.064+0000 7f3bb7db3b80 1 client.6309 shutdown 2023-11-28T11:11:14.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.064+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/4262785926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e6860 msgr2=0x559c6f2d69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.064+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/4262785926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e6860 0x559c6f2d69a0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f3ba0003960 tx=0x7f3ba0003c70 comp rx=0 tx=0).stop 2023-11-28T11:11:14.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.064+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/4262785926 shutdown_connections 2023-11-28T11:11:14.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.064+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/4262785926 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x559c6f2e5990 0x559c6f2f9750 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.064+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/4262785926 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 0x559c6f2e4f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.064+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/4262785926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e4180 0x559c6f2d8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.064+0000 7f3b84ff9700 0 client.6309 ms_handle_reset on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:14.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.064+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/4262785926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e6860 0x559c6f2d69a0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.064+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/4262785926 >> 172.21.15.28:0/4262785926 conn(0x559c6f2b9db0 msgr2=0x559c6f2bc060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:14.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.064+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/4262785926 shutdown_connections 2023-11-28T11:11:14.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.064+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/4262785926 wait complete. 2023-11-28T11:11:14.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.064+0000 7f3bb7db3b80 20 client.6309 trim_cache size 0 max 16384 2023-11-28T11:11:14.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.081+0000 7f3bb7db3b80 1 Processor -- start 2023-11-28T11:11:14.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.081+0000 7f3bb7db3b80 1 -- start start 2023-11-28T11:11:14.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.081+0000 7f3bb7db3b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e4180 0x559c6f314030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3bb7db3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e6860 0x559c6f2d4590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3bb7db3b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 0x559c6f2b8040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3bb7db3b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6f2d6200 con 0x559c6f2e4180 2023-11-28T11:11:14.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3bb7db3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6f2b8580 con 0x559c6f2e6860 2023-11-28T11:11:14.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3bb7db3b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c6f2b7390 con 0x559c6f2e7170 2023-11-28T11:11:14.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3ba99c6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e4180 0x559c6f314030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3ba99c6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e4180 0x559c6f314030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50862/0 (socket says 172.21.15.28:50862) 2023-11-28T11:11:14.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3ba99c6700 1 -- 172.21.15.28:0/2187943306 learned_addr learned my addr 172.21.15.28:0/2187943306 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:14.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3baa1c7700 1 --2- 172.21.15.28:0/2187943306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 0x559c6f2b8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3ba91c5700 1 --2- 172.21.15.28:0/2187943306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e6860 0x559c6f2d4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3ba99c6700 1 -- 172.21.15.28:0/2187943306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 msgr2=0x559c6f2b8040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3ba99c6700 1 --2- 172.21.15.28:0/2187943306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 0x559c6f2b8040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3ba99c6700 1 -- 172.21.15.28:0/2187943306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e6860 msgr2=0x559c6f2d4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3ba99c6700 1 --2- 172.21.15.28:0/2187943306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e6860 0x559c6f2d4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3ba99c6700 1 -- 172.21.15.28:0/2187943306 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559c6f1d5190 con 0x559c6f2e4180 2023-11-28T11:11:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3baa1c7700 1 --2- 172.21.15.28:0/2187943306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 0x559c6f2b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:14.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3ba91c5700 1 --2- 172.21.15.28:0/2187943306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e6860 0x559c6f2d4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3ba99c6700 1 --2- 172.21.15.28:0/2187943306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e4180 0x559c6f314030 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3b94055ce0 tx=0x7f3b9401e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fa77372367d13d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3b87fff700 1 -- 172.21.15.28:0/2187943306 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b9405d9b0 con 0x559c6f2e4180 2023-11-28T11:11:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3b87fff700 1 -- 172.21.15.28:0/2187943306 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b94058070 con 0x559c6f2e4180 2023-11-28T11:11:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.082+0000 7f3b87fff700 1 -- 172.21.15.28:0/2187943306 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b940552a0 con 0x559c6f2e4180 2023-11-28T11:11:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.083+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/2187943306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e4180 msgr2=0x559c6f314030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.083+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/2187943306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e4180 0x559c6f314030 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f3b94055ce0 tx=0x7f3b9401e000 comp rx=0 tx=0).stop 2023-11-28T11:11:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.083+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/2187943306 shutdown_connections 2023-11-28T11:11:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.083+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/2187943306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e4180 0x559c6f314030 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.083+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/2187943306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 0x559c6f2b8040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.083+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/2187943306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e6860 0x559c6f2d4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.083+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/2187943306 >> 172.21.15.28:0/2187943306 conn(0x559c6f2b9db0 msgr2=0x559c6f2ba720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.083+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/2187943306 shutdown_connections 2023-11-28T11:11:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.083+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/2187943306 wait complete. 2023-11-28T11:11:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.083+0000 7f3bb7db3b80 1 -- start start 2023-11-28T11:11:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.083+0000 7f3bb7db3b80 1 Processor -- start 2023-11-28T11:11:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.084+0000 7f3bb7db3b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e4180 0x559c6f351310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.084+0000 7f3bb7db3b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e6860 0x559c6f2bd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.084+0000 7f3bb7db3b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 0x559c6f2bdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.084+0000 7f3bb7db3b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6f2b6400 con 0x559c6f2e4180 2023-11-28T11:11:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.084+0000 7f3bb7db3b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x559c6f363580 con 0x559c6f2e6860 2023-11-28T11:11:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.084+0000 7f3bb7db3b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x559c6f2bc050 con 0x559c6f2e7170 2023-11-28T11:11:14.099 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.084+0000 7f3bb7db3b80 10 client.0 _ll_register_callbacks cb 0x559c6f0b4290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.084+0000 7f3ba99c6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e4180 0x559c6f351310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.084+0000 7f3ba99c6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e4180 0x559c6f351310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50868/0 (socket says 172.21.15.28:50868) 2023-11-28T11:11:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.084+0000 7f3ba99c6700 1 -- 172.21.15.28:0/206419718 learned_addr learned my addr 172.21.15.28:0/206419718 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3ba91c5700 1 --2- 172.21.15.28:0/206419718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e6860 0x559c6f2bd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3baa1c7700 1 --2- 172.21.15.28:0/206419718 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 0x559c6f2bdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3ba99c6700 1 -- 172.21.15.28:0/206419718 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 msgr2=0x559c6f2bdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3ba99c6700 1 --2- 172.21.15.28:0/206419718 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 0x559c6f2bdef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3ba99c6700 1 -- 172.21.15.28:0/206419718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e6860 msgr2=0x559c6f2bd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3ba99c6700 1 --2- 172.21.15.28:0/206419718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e6860 0x559c6f2bd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3ba99c6700 1 -- 172.21.15.28:0/206419718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559c6f090630 con 0x559c6f2e4180 2023-11-28T11:11:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3baa1c7700 1 --2- 172.21.15.28:0/206419718 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 0x559c6f2bdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3ba99c6700 1 --2- 172.21.15.28:0/206419718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e4180 0x559c6f351310 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3b94055ab0 tx=0x7f3b9406b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1599570bb5e070e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3b84ff9700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:11:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3b84ff9700 1 -- 172.21.15.28:0/206419718 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b940554a0 con 0x559c6f2e4180 2023-11-28T11:11:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/206419718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559c6f1cc1b0 con 0x559c6f2e4180 2023-11-28T11:11:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3b84ff9700 1 -- 172.21.15.28:0/206419718 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3b9405d9b0 con 0x559c6f2e4180 2023-11-28T11:11:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3b84ff9700 1 -- 172.21.15.28:0/206419718 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3b9405db70 con 0x559c6f2e4180 2023-11-28T11:11:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3bb7db3b80 10 client.6315 fetch_fsmap learned FSMap version 25 2023-11-28T11:11:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/206419718 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x559c6f063250 con 0x559c6f2e4180 2023-11-28T11:11:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3b84ff9700 1 -- 172.21.15.28:0/206419718 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559c6f1cc1b0 con 0x559c6f2e4180 2023-11-28T11:11:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3b84ff9700 1 -- 172.21.15.28:0/206419718 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f3b9407b050 con 0x559c6f2e4180 2023-11-28T11:11:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3bb7db3b80 10 client.6315 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:11:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3bb7db3b80 10 client.6315 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:11:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3bb7db3b80 20 client.6315 populate_metadata read hostname 'smithi028' 2023-11-28T11:11:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/206419718 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x559c6f2bbb50 0x559c6f3623c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3bb7db3b80 4 client.6315 mds_command: new command op to 6054 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:11:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.085+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/206419718 --> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x559c6f1b61c0 con 0x559c6f2bbb50 2023-11-28T11:11:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.086+0000 7f3ba91c5700 1 --2- 172.21.15.28:0/206419718 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x559c6f2bbb50 0x559c6f3623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.086+0000 7f3b84ff9700 10 client.6315 ms_handle_connect on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.086+0000 7f3ba91c5700 1 --2- 172.21.15.28:0/206419718 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x559c6f2bbb50 0x559c6f3623c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1bdb443141476909 server_cookie=9fd5080b36b4a551 in_seq=0 out_seq=0 2023-11-28T11:11:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.086+0000 7f3b84ff9700 1 -- 172.21.15.28:0/206419718 <== mds.0 v2:172.21.15.59:6832/3261856585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+260 (crc 0 0 0) 0x559c6f1b61c0 con 0x559c6f2bbb50 2023-11-28T11:11:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.086+0000 7f3b84ff9700 10 client.6315 handle_command_reply: tid=0 2023-11-28T11:11:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.086+0000 7f3bb7db3b80 1 client.6315 shutdown 2023-11-28T11:11:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.086+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/206419718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e4180 msgr2=0x559c6f351310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.086+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/206419718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e4180 0x559c6f351310 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f3b94055ab0 tx=0x7f3b9406b000 comp rx=0 tx=0).stop 2023-11-28T11:11:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.086+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/206419718 shutdown_connections 2023-11-28T11:11:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.086+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/206419718 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x559c6f2bbb50 0x559c6f3623c0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.086+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/206419718 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x559c6f2e4180 0x559c6f351310 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.086+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/206419718 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x559c6f2e7170 0x559c6f2bdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.086+0000 7f3b84ff9700 0 client.6315 ms_handle_reset on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.086+0000 7f3bb7db3b80 1 --2- 172.21.15.28:0/206419718 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559c6f2e6860 0x559c6f2bd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/206419718 >> 172.21.15.28:0/206419718 conn(0x559c6f2b9db0 msgr2=0x559c6f2d7730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/206419718 shutdown_connections 2023-11-28T11:11:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3bb7db3b80 1 -- 172.21.15.28:0/206419718 wait complete. 2023-11-28T11:11:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3bb7db3b80 20 client.6315 trim_cache size 0 max 16384 2023-11-28T11:11:14.104 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:11:14.104 INFO:teuthology.orchestra.run.smithi028.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T11:11:14.104 INFO:teuthology.orchestra.run.smithi028.stdout: "scrubs": { 2023-11-28T11:11:14.104 INFO:teuthology.orchestra.run.smithi028.stdout: "21d7f5a7-2537-46da-8d66-ddc008c3d1e3": { 2023-11-28T11:11:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/", 2023-11-28T11:11:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: "tag": "21d7f5a7-2537-46da-8d66-ddc008c3d1e3", 2023-11-28T11:11:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: "options": "recursive,repair" 2023-11-28T11:11:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:14.105 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:14.105 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:11:14.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3babc2a700 1 -- 172.21.15.28:0/2052248680 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3b880682f0 msgr2=0x7f3b8806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3babc2a700 1 --2- 172.21.15.28:0/2052248680 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3b880682f0 0x7f3b8806a7a0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f3ba40c7700 tx=0x7f3ba0009040 comp rx=0 tx=0).stop 2023-11-28T11:11:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3babc2a700 1 -- 172.21.15.28:0/2052248680 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ba40b2d90 msgr2=0x7f3ba41ef9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3babc2a700 1 --2- 172.21.15.28:0/2052248680 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ba40b2d90 0x7f3ba41ef9b0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f3b9400c520 tx=0x7f3b9400c8e0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3babc2a700 1 -- 172.21.15.28:0/2052248680 shutdown_connections 2023-11-28T11:11:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3babc2a700 1 --2- 172.21.15.28:0/2052248680 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3b880682f0 0x7f3b8806a7a0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3babc2a700 1 --2- 172.21.15.28:0/2052248680 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ba40b2d90 0x7f3ba41ef9b0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3babc2a700 1 --2- 172.21.15.28:0/2052248680 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ba40cb630 0x7f3ba40c69b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3babc2a700 1 --2- 172.21.15.28:0/2052248680 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ba40b62d0 0x7f3ba40c9bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3babc2a700 1 -- 172.21.15.28:0/2052248680 >> 172.21.15.28:0/2052248680 conn(0x7f3ba40cd680 msgr2=0x7f3ba40cf4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:14.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3babc2a700 1 -- 172.21.15.28:0/2052248680 shutdown_connections 2023-11-28T11:11:14.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:14.087+0000 7f3babc2a700 1 -- 172.21.15.28:0/2052248680 wait complete. 2023-11-28T11:11:14.108 INFO:tasks.cephfs.filesystem:scrub status for tag:21d7f5a7-2537-46da-8d66-ddc008c3d1e3 - {'path': '/', 'tag': '21d7f5a7-2537-46da-8d66-ddc008c3d1e3', 'options': 'recursive,repair'} 2023-11-28T11:11:15.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:15.260+0000 7fb28effd700 1 -- 172.21.15.28:0/428768153 <== mon.0 v2:172.21.15.28:3300/0 7 ==== log(7 entries from seq 1 at 2023-11-28T11:11:13.908617+0000) v1 ==== 1229+0+0 (secure 0 0 0) 0x7fb294088080 con 0x7fb29c1601b0 2023-11-28T11:11:15.262 INFO:teuthology.orchestra.run.smithi028.stdout: cluster: 2023-11-28T11:11:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: id: 6e299adc-7a2a-4cf1-afa2-7ce104b29a51 2023-11-28T11:11:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: health: HEALTH_OK 2023-11-28T11:11:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: services: 2023-11-28T11:11:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: mon: 3 daemons, quorum a,b,c (age 2m) 2023-11-28T11:11:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: mgr: y(active, since 2m), standbys: x 2023-11-28T11:11:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: mds: 1/1 daemons up, 3 standby 2023-11-28T11:11:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2023-11-28T11:11:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:15.263 INFO:teuthology.orchestra.run.smithi028.stdout: data: 2023-11-28T11:11:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: volumes: 1/1 healthy 2023-11-28T11:11:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: pools: 4 pools, 105 pgs 2023-11-28T11:11:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: objects: 24 objects, 579 KiB 2023-11-28T11:11:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: usage: 74 MiB used, 720 GiB / 720 GiB avail 2023-11-28T11:11:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: pgs: 105 active+clean 2023-11-28T11:11:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: io: 2023-11-28T11:11:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: client: 1.2 KiB/s wr, 0 op/s rd, 3 op/s wr 2023-11-28T11:11:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:15.264 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:15.264 INFO:teuthology.orchestra.run.smithi028.stdout:2023-11-28T11:11:13.908617+0000 mds.d [INF] scrub queued for path: / 2023-11-28T11:11:15.264 INFO:teuthology.orchestra.run.smithi028.stdout:2023-11-28T11:11:13.908624+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2023-11-28T11:11:15.265 INFO:teuthology.orchestra.run.smithi028.stdout:2023-11-28T11:11:13.908632+0000 mds.d [INF] scrub summary: active paths [/] 2023-11-28T11:11:15.265 INFO:teuthology.orchestra.run.smithi028.stdout:2023-11-28T11:11:13.909606+0000 mds.d [WRN] bad backtrace on inode 0x10000000001(/parent_a/alpha), rewriting it 2023-11-28T11:11:15.265 INFO:teuthology.orchestra.run.smithi028.stdout:2023-11-28T11:11:13.909618+0000 mds.d [INF] Scrub repaired inode 0x10000000001 (/parent_a/alpha) 2023-11-28T11:11:15.265 INFO:teuthology.orchestra.run.smithi028.stdout:2023-11-28T11:11:13.910148+0000 mds.d [INF] scrub summary: idle+waiting paths [/] 2023-11-28T11:11:15.265 INFO:teuthology.orchestra.run.smithi028.stdout:2023-11-28T11:11:14.452817+0000 mds.d [INF] scrub summary: idle 2023-11-28T11:11:21.207 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:11:21.211 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:11:44.109 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub status 2023-11-28T11:11:44.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.250+0000 7ff1dede5700 1 Processor -- start 2023-11-28T11:11:44.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.250+0000 7ff1dede5700 1 -- start start 2023-11-28T11:11:44.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1dede5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1d80b1d40 0x7ff1d80cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1dede5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff1d815dbf0 0x7ff1d80ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1dede5700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d80ceff0 con 0x7ff1d8159500 2023-11-28T11:11:44.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1dede5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d815e090 con 0x7ff1d815dbf0 2023-11-28T11:11:44.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1dede5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80cede0 con 0x7ff1d80b1d40 2023-11-28T11:11:44.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1d7fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1d80b1d40 0x7ff1d80cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1dd382700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff1d815dbf0 0x7ff1d80ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1dd382700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff1d815dbf0 0x7ff1d80ce8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:38170/0 (socket says 172.21.15.28:38170) 2023-11-28T11:11:44.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1d7fff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1d80b1d40 0x7ff1d80cd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:56636/0 (socket says 172.21.15.28:56636) 2023-11-28T11:11:44.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1dd382700 1 -- 172.21.15.28:0/3265728309 learned_addr learned my addr 172.21.15.28:0/3265728309 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:44.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1dcb81700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7ff1d8159500 0x7ff1d80b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:49720/0 (socket says 172.21.15.28:49720) 2023-11-28T11:11:44.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1dd382700 1 -- 172.21.15.28:0/3265728309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1d80b1d40 msgr2=0x7ff1d80cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:44.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1dd382700 1 --2- 172.21.15.28:0/3265728309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1d80b1d40 0x7ff1d80cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1dd382700 1 -- 172.21.15.28:0/3265728309 >> v1:172.21.15.59:6789/0 conn(0x7ff1d8159500 legacy=0x7ff1d80b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.258 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1dd382700 1 -- 172.21.15.28:0/3265728309 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d80cf2d0 con 0x7ff1d815dbf0 2023-11-28T11:11:44.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.251+0000 7ff1d7fff700 1 --2- 172.21.15.28:0/3265728309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1d80b1d40 0x7ff1d80cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:44.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.252+0000 7ff1dd382700 1 --2- 172.21.15.28:0/3265728309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1d815dbf0 0x7ff1d80ce8a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009e50 tx=0x7ff1d000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47b707af63d85764 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:44.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.252+0000 7ff1d77fe700 1 -- 172.21.15.28:0/3265728309 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000e070 con 0x7ff1d815dbf0 2023-11-28T11:11:44.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.252+0000 7ff1d77fe700 1 -- 172.21.15.28:0/3265728309 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d000b910 con 0x7ff1d815dbf0 2023-11-28T11:11:44.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.252+0000 7ff1d77fe700 1 -- 172.21.15.28:0/3265728309 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d000bcc0 con 0x7ff1d815dbf0 2023-11-28T11:11:44.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.252+0000 7ff1dede5700 1 -- 172.21.15.28:0/3265728309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1d815dbf0 msgr2=0x7ff1d80ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.252+0000 7ff1dede5700 1 --2- 172.21.15.28:0/3265728309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1d815dbf0 0x7ff1d80ce8a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009e50 tx=0x7ff1d000b330 comp rx=0 tx=0).stop 2023-11-28T11:11:44.259 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.252+0000 7ff1dede5700 1 -- 172.21.15.28:0/3265728309 shutdown_connections 2023-11-28T11:11:44.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.252+0000 7ff1dede5700 1 --2- 172.21.15.28:0/3265728309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1d815dbf0 0x7ff1d80ce8a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.252+0000 7ff1dede5700 1 --2- 172.21.15.28:0/3265728309 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1d80b1d40 0x7ff1d80cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.252+0000 7ff1dede5700 1 -- 172.21.15.28:0/3265728309 >> 172.21.15.28:0/3265728309 conn(0x7ff1d80cc630 msgr2=0x7ff1d80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:44.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.253+0000 7ff1dede5700 1 -- 172.21.15.28:0/3265728309 shutdown_connections 2023-11-28T11:11:44.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.253+0000 7ff1dede5700 1 -- 172.21.15.28:0/3265728309 wait complete. 2023-11-28T11:11:44.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.254+0000 7ff1dede5700 1 Processor -- start 2023-11-28T11:11:44.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.254+0000 7ff1dede5700 1 -- start start 2023-11-28T11:11:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.255+0000 7ff1dede5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1d80b1d40 0x7ff1d81f1520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.255+0000 7ff1dede5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1d8159500 0x7ff1d8154170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.255+0000 7ff1dede5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1d815dbf0 0x7ff1d81f0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.255+0000 7ff1dede5700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d81546b0 con 0x7ff1d815dbf0 2023-11-28T11:11:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.255+0000 7ff1dede5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d80c91e0 con 0x7ff1d80b1d40 2023-11-28T11:11:44.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.255+0000 7ff1dede5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d81f1320 con 0x7ff1d8159500 2023-11-28T11:11:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1dd382700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1d815dbf0 0x7ff1d81f0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1dd382700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1d815dbf0 0x7ff1d81f0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50886/0 (socket says 172.21.15.28:50886) 2023-11-28T11:11:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1dd382700 1 -- 172.21.15.28:0/1314636635 learned_addr learned my addr 172.21.15.28:0/1314636635 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1d7fff700 1 --2- 172.21.15.28:0/1314636635 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1d8159500 0x7ff1d8154170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1dcb81700 1 --2- 172.21.15.28:0/1314636635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1d80b1d40 0x7ff1d81f1520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1dd382700 1 -- 172.21.15.28:0/1314636635 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1d8159500 msgr2=0x7ff1d8154170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1dd382700 1 --2- 172.21.15.28:0/1314636635 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1d8159500 0x7ff1d8154170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1dd382700 1 -- 172.21.15.28:0/1314636635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1d80b1d40 msgr2=0x7ff1d81f1520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1dd382700 1 --2- 172.21.15.28:0/1314636635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1d80b1d40 0x7ff1d81f1520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1dd382700 1 -- 172.21.15.28:0/1314636635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1c40094a0 con 0x7ff1d815dbf0 2023-11-28T11:11:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1dd382700 1 --2- 172.21.15.28:0/1314636635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1d815dbf0 0x7ff1d81f0de0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000b680 tx=0x7ff1d0004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:44.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1d57fa700 1 -- 172.21.15.28:0/1314636635 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00099b0 con 0x7ff1d815dbf0 2023-11-28T11:11:44.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1d57fa700 1 -- 172.21.15.28:0/1314636635 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d00092e0 con 0x7ff1d815dbf0 2023-11-28T11:11:44.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1d57fa700 1 -- 172.21.15.28:0/1314636635 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d001c610 con 0x7ff1d815dbf0 2023-11-28T11:11:44.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1dede5700 1 -- 172.21.15.28:0/1314636635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d0009b30 con 0x7ff1d815dbf0 2023-11-28T11:11:44.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.256+0000 7ff1dede5700 1 -- 172.21.15.28:0/1314636635 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d815d3f0 con 0x7ff1d815dbf0 2023-11-28T11:11:44.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.258+0000 7ff1d57fa700 1 -- 172.21.15.28:0/1314636635 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff1d001c770 con 0x7ff1d815dbf0 2023-11-28T11:11:44.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.258+0000 7ff1d57fa700 1 --2- 172.21.15.28:0/1314636635 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1bc0682f0 0x7ff1bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.258+0000 7ff1d57fa700 1 -- 172.21.15.28:0/1314636635 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff1d0087430 con 0x7ff1d815dbf0 2023-11-28T11:11:44.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.258+0000 7ff1dcb81700 1 --2- 172.21.15.28:0/1314636635 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1bc0682f0 0x7ff1bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.258+0000 7ff1dcb81700 1 --2- 172.21.15.28:0/1314636635 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1bc0682f0 0x7ff1bc06a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc000f60 tx=0x7ff1cc007d30 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:44.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.260+0000 7ff1eaf6eb80 1 Processor -- start 2023-11-28T11:11:44.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.260+0000 7ff1eaf6eb80 1 -- start start 2023-11-28T11:11:44.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1eaf6eb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687f860 0x555e66855a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1eaf6eb80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e6687d180 0x555e6687fc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1eaf6eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e66880170 0x555e6689b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1eaf6eb80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e6673fc10 con 0x555e6687d180 2023-11-28T11:11:44.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1eaf6eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e667ae3a0 con 0x555e66880170 2023-11-28T11:11:44.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1eaf6eb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e667a78d0 con 0x555e6687f860 2023-11-28T11:11:44.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1d7fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e6687d180 0x555e6687fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1d7fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e6687d180 0x555e6687fc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50892/0 (socket says 172.21.15.28:50892) 2023-11-28T11:11:44.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1d7fff700 1 -- 172.21.15.28:0/2406066693 learned_addr learned my addr 172.21.15.28:0/2406066693 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1dcb81700 1 --2- 172.21.15.28:0/2406066693 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687f860 0x555e66855a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1dd382700 1 --2- 172.21.15.28:0/2406066693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e66880170 0x555e6689b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1d7fff700 1 -- 172.21.15.28:0/2406066693 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687f860 msgr2=0x555e66855a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1d7fff700 1 --2- 172.21.15.28:0/2406066693 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687f860 0x555e66855a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1d7fff700 1 -- 172.21.15.28:0/2406066693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e66880170 msgr2=0x555e6689b7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1d7fff700 1 --2- 172.21.15.28:0/2406066693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e66880170 0x555e6689b7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1d7fff700 1 -- 172.21.15.28:0/2406066693 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555e66629630 con 0x555e6687d180 2023-11-28T11:11:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.261+0000 7ff1dd382700 1 --2- 172.21.15.28:0/2406066693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e66880170 0x555e6689b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:44.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.262+0000 7ff1d7fff700 1 --2- 172.21.15.28:0/2406066693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e6687d180 0x555e6687fc30 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009d20 tx=0x7ff1c4009210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8206c94f0f2ffd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.262+0000 7ff1c27fc700 1 -- 172.21.15.28:0/2406066693 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c40097c0 con 0x555e6687d180 2023-11-28T11:11:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.262+0000 7ff1c27fc700 1 -- 172.21.15.28:0/2406066693 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1c4011c30 con 0x555e6687d180 2023-11-28T11:11:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.262+0000 7ff1c27fc700 1 -- 172.21.15.28:0/2406066693 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4009e70 con 0x555e6687d180 2023-11-28T11:11:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.263+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/2406066693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e6687d180 msgr2=0x555e6687fc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.263+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/2406066693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e6687d180 0x555e6687fc30 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4009d20 tx=0x7ff1c4009210 comp rx=0 tx=0).stop 2023-11-28T11:11:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.263+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/2406066693 shutdown_connections 2023-11-28T11:11:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.263+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/2406066693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e66880170 0x555e6689b7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.263+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/2406066693 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687f860 0x555e66855a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.263+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/2406066693 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e6687d180 0x555e6687fc30 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.263+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/2406066693 >> 172.21.15.28:0/2406066693 conn(0x555e66852db0 msgr2=0x555e66853740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:44.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.263+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/2406066693 shutdown_connections 2023-11-28T11:11:44.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.263+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/2406066693 wait complete. 2023-11-28T11:11:44.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.264+0000 7ff1eaf6eb80 1 -- start start 2023-11-28T11:11:44.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.264+0000 7ff1eaf6eb80 1 Processor -- start 2023-11-28T11:11:44.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1eaf6eb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687d180 0x555e668946e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1eaf6eb80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e6687f860 0x555e668763a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1eaf6eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e66880170 0x555e668776d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1eaf6eb80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e6662b920 con 0x555e6687f860 2023-11-28T11:11:44.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1eaf6eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e6689f950 con 0x555e66880170 2023-11-28T11:11:44.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1eaf6eb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e668a2620 con 0x555e6687d180 2023-11-28T11:11:44.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1eaf6eb80 10 client.0 _ll_register_callbacks cb 0x555e6664d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:44.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1d7fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e6687f860 0x555e668763a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1d7fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e6687f860 0x555e668763a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50898/0 (socket says 172.21.15.28:50898) 2023-11-28T11:11:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1d7fff700 1 -- 172.21.15.28:0/1117093363 learned_addr learned my addr 172.21.15.28:0/1117093363 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1dcb81700 1 --2- 172.21.15.28:0/1117093363 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687d180 0x555e668946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1dd382700 1 --2- 172.21.15.28:0/1117093363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e66880170 0x555e668776d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1d7fff700 1 -- 172.21.15.28:0/1117093363 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687d180 msgr2=0x555e668946e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1d7fff700 1 --2- 172.21.15.28:0/1117093363 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687d180 0x555e668946e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1d7fff700 1 -- 172.21.15.28:0/1117093363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e66880170 msgr2=0x555e668776d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1d7fff700 1 --2- 172.21.15.28:0/1117093363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e66880170 0x555e668776d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1d7fff700 1 -- 172.21.15.28:0/1117093363 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1c4009920 con 0x555e6687f860 2023-11-28T11:11:44.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1d7fff700 1 --2- 172.21.15.28:0/1117093363 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e6687f860 0x555e668763a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4018000 tx=0x7ff1c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c53dcba0a31358b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:44.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.265+0000 7ff1b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:11:44.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.266+0000 7ff1b77fe700 1 -- 172.21.15.28:0/1117093363 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1c4023040 con 0x555e6687f860 2023-11-28T11:11:44.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.266+0000 7ff1b77fe700 1 -- 172.21.15.28:0/1117093363 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff1c4004220 con 0x555e6687f860 2023-11-28T11:11:44.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.266+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/1117093363 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e66775190 con 0x555e6687f860 2023-11-28T11:11:44.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.266+0000 7ff1b77fe700 1 -- 172.21.15.28:0/1117093363 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1c40043e0 con 0x555e6687f860 2023-11-28T11:11:44.278 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.266+0000 7ff1eaf6eb80 10 client.6324 fetch_fsmap learned FSMap version 25 2023-11-28T11:11:44.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.266+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/1117093363 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x555e665d5de0 con 0x555e6687f860 2023-11-28T11:11:44.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.266+0000 7ff1b77fe700 1 -- 172.21.15.28:0/1117093363 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555e66775190 con 0x555e6687f860 2023-11-28T11:11:44.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.266+0000 7ff1b77fe700 1 -- 172.21.15.28:0/1117093363 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff1c40036a0 con 0x555e6687f860 2023-11-28T11:11:44.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.266+0000 7ff1eaf6eb80 10 client.6324 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:11:44.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.266+0000 7ff1eaf6eb80 10 client.6324 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:11:44.279 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.266+0000 7ff1eaf6eb80 20 client.6324 populate_metadata read hostname 'smithi028' 2023-11-28T11:11:44.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.266+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/1117093363 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x555e66877f70 0x555e6689db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.266+0000 7ff1eaf6eb80 4 client.6324 mds_command: new command op to 6054 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:11:44.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.266+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/1117093363 --> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555e6662bf60 con 0x555e66877f70 2023-11-28T11:11:44.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.266+0000 7ff1dcb81700 1 --2- 172.21.15.28:0/1117093363 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x555e66877f70 0x555e6689db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.280 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.267+0000 7ff1b77fe700 10 client.6324 ms_handle_connect on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:44.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.267+0000 7ff1dcb81700 1 --2- 172.21.15.28:0/1117093363 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x555e66877f70 0x555e6689db80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b707dd7e6ea08b00 server_cookie=4ef71196416c2ef4 in_seq=0 out_seq=0 2023-11-28T11:11:44.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.267+0000 7ff1b77fe700 1 -- 172.21.15.28:0/1117093363 <== mds.0 v2:172.21.15.59:6832/3261856585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x555e6662bf60 con 0x555e66877f70 2023-11-28T11:11:44.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.267+0000 7ff1b77fe700 10 client.6324 handle_command_reply: tid=0 2023-11-28T11:11:44.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.267+0000 7ff1eaf6eb80 1 client.6324 shutdown 2023-11-28T11:11:44.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.268+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/1117093363 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e6687f860 msgr2=0x555e668763a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.281 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.268+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/1117093363 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e6687f860 0x555e668763a0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7ff1c4018000 tx=0x7ff1c4014000 comp rx=0 tx=0).stop 2023-11-28T11:11:44.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.268+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/1117093363 shutdown_connections 2023-11-28T11:11:44.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.268+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/1117093363 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x555e66877f70 0x555e6689db80 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.268+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/1117093363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e66880170 0x555e668776d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.282 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.268+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/1117093363 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687d180 0x555e668946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.268+0000 7ff1b77fe700 0 client.6324 ms_handle_reset on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:44.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.268+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/1117093363 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e6687f860 0x555e668763a0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.268+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/1117093363 >> 172.21.15.28:0/1117093363 conn(0x555e66852db0 msgr2=0x555e66854280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:44.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.268+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/1117093363 shutdown_connections 2023-11-28T11:11:44.283 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.268+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/1117093363 wait complete. 2023-11-28T11:11:44.284 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.268+0000 7ff1eaf6eb80 20 client.6324 trim_cache size 0 max 16384 2023-11-28T11:11:44.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.284+0000 7ff1eaf6eb80 1 Processor -- start 2023-11-28T11:11:44.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.285+0000 7ff1eaf6eb80 1 -- start start 2023-11-28T11:11:44.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.285+0000 7ff1eaf6eb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687d180 0x555e66854df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.285+0000 7ff1eaf6eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e6687f860 0x555e6684e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.285+0000 7ff1eaf6eb80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e66880170 0x555e6684e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.285+0000 7ff1eaf6eb80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e668b4170 con 0x555e66880170 2023-11-28T11:11:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.285+0000 7ff1eaf6eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e66851530 con 0x555e6687f860 2023-11-28T11:11:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.285+0000 7ff1eaf6eb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e6689ea20 con 0x555e6687d180 2023-11-28T11:11:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.285+0000 7ff1dd382700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e66880170 0x555e6684e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.285+0000 7ff1dd382700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e66880170 0x555e6684e9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50908/0 (socket says 172.21.15.28:50908) 2023-11-28T11:11:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.285+0000 7ff1dd382700 1 -- 172.21.15.28:0/1489326302 learned_addr learned my addr 172.21.15.28:0/1489326302 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.286+0000 7ff1dcb81700 1 --2- 172.21.15.28:0/1489326302 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687d180 0x555e66854df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.286+0000 7ff1d7fff700 1 --2- 172.21.15.28:0/1489326302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e6687f860 0x555e6684e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.286+0000 7ff1dd382700 1 -- 172.21.15.28:0/1489326302 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687d180 msgr2=0x555e66854df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:44.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.286+0000 7ff1dd382700 1 --2- 172.21.15.28:0/1489326302 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687d180 0x555e66854df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.286+0000 7ff1dd382700 1 -- 172.21.15.28:0/1489326302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e6687f860 msgr2=0x555e6684e4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:44.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.286+0000 7ff1dd382700 1 --2- 172.21.15.28:0/1489326302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e6687f860 0x555e6684e4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.286+0000 7ff1dd382700 1 -- 172.21.15.28:0/1489326302 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d81eddf0 con 0x555e66880170 2023-11-28T11:11:44.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.286+0000 7ff1dd382700 1 --2- 172.21.15.28:0/1489326302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e66880170 0x555e6684e9e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009680 tx=0x7ff1d0053c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e845311c8c2fca1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:44.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.286+0000 7ff1c2ffd700 1 -- 172.21.15.28:0/1489326302 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0053f20 con 0x555e66880170 2023-11-28T11:11:44.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.286+0000 7ff1c2ffd700 1 -- 172.21.15.28:0/1489326302 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d0054080 con 0x555e66880170 2023-11-28T11:11:44.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.286+0000 7ff1c2ffd700 1 -- 172.21.15.28:0/1489326302 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d00872b0 con 0x555e66880170 2023-11-28T11:11:44.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.287+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/1489326302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e66880170 msgr2=0x555e6684e9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.287+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/1489326302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e66880170 0x555e6684e9e0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7ff1d0009680 tx=0x7ff1d0053c80 comp rx=0 tx=0).stop 2023-11-28T11:11:44.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.287+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/1489326302 shutdown_connections 2023-11-28T11:11:44.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.287+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/1489326302 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e66880170 0x555e6684e9e0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.287+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/1489326302 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687d180 0x555e66854df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.287+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/1489326302 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e6687f860 0x555e6684e4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.287+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/1489326302 >> 172.21.15.28:0/1489326302 conn(0x555e66852db0 msgr2=0x555e668761b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:44.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.287+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/1489326302 shutdown_connections 2023-11-28T11:11:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.287+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/1489326302 wait complete. 2023-11-28T11:11:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.287+0000 7ff1eaf6eb80 1 -- start start 2023-11-28T11:11:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.287+0000 7ff1eaf6eb80 1 Processor -- start 2023-11-28T11:11:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.288+0000 7ff1eaf6eb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687d180 0x555e66878b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.288+0000 7ff1eaf6eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e6687f860 0x555e668ff020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.288+0000 7ff1eaf6eb80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e66880170 0x555e6687d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.288+0000 7ff1eaf6eb80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e668790d0 con 0x555e66880170 2023-11-28T11:11:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.288+0000 7ff1eaf6eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x555e66879530 con 0x555e6687f860 2023-11-28T11:11:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.288+0000 7ff1eaf6eb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x555e6687dde0 con 0x555e6687d180 2023-11-28T11:11:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.288+0000 7ff1eaf6eb80 10 client.0 _ll_register_callbacks cb 0x555e6664d290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:11:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.288+0000 7ff1dd382700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e66880170 0x555e6687d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.288+0000 7ff1dd382700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e66880170 0x555e6687d550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50914/0 (socket says 172.21.15.28:50914) 2023-11-28T11:11:44.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.288+0000 7ff1dd382700 1 -- 172.21.15.28:0/3694088790 learned_addr learned my addr 172.21.15.28:0/3694088790 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:44.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.288+0000 7ff1dcb81700 1 --2- 172.21.15.28:0/3694088790 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687d180 0x555e66878b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.288+0000 7ff1d7fff700 1 --2- 172.21.15.28:0/3694088790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e6687f860 0x555e668ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1dd382700 1 -- 172.21.15.28:0/3694088790 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687d180 msgr2=0x555e66878b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:44.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1dd382700 1 --2- 172.21.15.28:0/3694088790 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687d180 0x555e66878b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1dd382700 1 -- 172.21.15.28:0/3694088790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e6687f860 msgr2=0x555e668ff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:44.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1dd382700 1 --2- 172.21.15.28:0/3694088790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e6687f860 0x555e668ff020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1dd382700 1 -- 172.21.15.28:0/3694088790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d815d3f0 con 0x555e66880170 2023-11-28T11:11:44.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1dd382700 1 --2- 172.21.15.28:0/3694088790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e66880170 0x555e6687d550 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00043d0 tx=0x7ff1d0087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66b46994022ef260 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:44.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1b77fe700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:11:44.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1b77fe700 1 -- 172.21.15.28:0/3694088790 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0053f20 con 0x555e66880170 2023-11-28T11:11:44.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1b77fe700 1 -- 172.21.15.28:0/3694088790 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff1d005f410 con 0x555e66880170 2023-11-28T11:11:44.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/3694088790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555e667651b0 con 0x555e66880170 2023-11-28T11:11:44.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1b77fe700 1 -- 172.21.15.28:0/3694088790 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff1d005f5d0 con 0x555e66880170 2023-11-28T11:11:44.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1eaf6eb80 10 client.6330 fetch_fsmap learned FSMap version 25 2023-11-28T11:11:44.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/3694088790 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=2+}) v3 -- 0x555e665bec60 con 0x555e66880170 2023-11-28T11:11:44.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1b77fe700 1 -- 172.21.15.28:0/3694088790 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x555e667651b0 con 0x555e66880170 2023-11-28T11:11:44.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1b77fe700 1 -- 172.21.15.28:0/3694088790 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 25) v1 ==== 3106+0+0 (secure 0 0 0) 0x7ff1d0058ba0 con 0x555e66880170 2023-11-28T11:11:44.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1eaf6eb80 10 client.6330 fetch_fsmap finished waiting for FSMap version 25 2023-11-28T11:11:44.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1eaf6eb80 10 client.6330 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.d 2023-11-28T11:11:44.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1eaf6eb80 20 client.6330 populate_metadata read hostname 'smithi028' 2023-11-28T11:11:44.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/3694088790 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x555e6689d3c0 0x555e6689d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1eaf6eb80 4 client.6330 mds_command: new command op to 6054 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:11:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.289+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/3694088790 --> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x555e6674f1c0 con 0x555e6689d3c0 2023-11-28T11:11:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.290+0000 7ff1dcb81700 1 --2- 172.21.15.28:0/3694088790 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x555e6689d3c0 0x555e6689d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.290+0000 7ff1b77fe700 10 client.6330 ms_handle_connect on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.290+0000 7ff1dcb81700 1 --2- 172.21.15.28:0/3694088790 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x555e6689d3c0 0x555e6689d7c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe8b49e2db161a4 server_cookie=7e92a6e3a1ccb3fc in_seq=0 out_seq=0 2023-11-28T11:11:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.290+0000 7ff1b77fe700 1 -- 172.21.15.28:0/3694088790 <== mds.0 v2:172.21.15.59:6832/3261856585 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x555e6674f1c0 con 0x555e6689d3c0 2023-11-28T11:11:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.290+0000 7ff1b77fe700 10 client.6330 handle_command_reply: tid=0 2023-11-28T11:11:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.290+0000 7ff1eaf6eb80 1 client.6330 shutdown 2023-11-28T11:11:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.290+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/3694088790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e66880170 msgr2=0x555e6687d550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.290+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/3694088790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e66880170 0x555e6687d550 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7ff1d00043d0 tx=0x7ff1d0087040 comp rx=0 tx=0).stop 2023-11-28T11:11:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.290+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/3694088790 shutdown_connections 2023-11-28T11:11:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.290+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/3694088790 >> [v2:172.21.15.59:6832/3261856585,v1:172.21.15.59:6834/3261856585] conn(0x555e6689d3c0 0x555e6689d7c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.290+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/3694088790 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x555e66880170 0x555e6687d550 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.290+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/3694088790 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x555e6687d180 0x555e66878b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.290+0000 7ff1b77fe700 0 client.6330 ms_handle_reset on v2:172.21.15.59:6832/3261856585 2023-11-28T11:11:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.290+0000 7ff1eaf6eb80 1 --2- 172.21.15.28:0/3694088790 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555e6687f860 0x555e668ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.290+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/3694088790 >> 172.21.15.28:0/3694088790 conn(0x555e66852db0 msgr2=0x555e668938e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.291+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/3694088790 shutdown_connections 2023-11-28T11:11:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.291+0000 7ff1eaf6eb80 1 -- 172.21.15.28:0/3694088790 wait complete. 2023-11-28T11:11:44.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.291+0000 7ff1eaf6eb80 20 client.6330 trim_cache size 0 max 16384 2023-11-28T11:11:44.304 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:11:44.305 INFO:teuthology.orchestra.run.smithi028.stdout: "status": "no active scrubs running", 2023-11-28T11:11:44.305 INFO:teuthology.orchestra.run.smithi028.stdout: "scrubs": {} 2023-11-28T11:11:44.305 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:11:44.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.291+0000 7ff1dede5700 1 -- 172.21.15.28:0/1314636635 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1bc0682f0 msgr2=0x7ff1bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.291+0000 7ff1dede5700 1 --2- 172.21.15.28:0/1314636635 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1bc0682f0 0x7ff1bc06a7a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7ff1cc000f60 tx=0x7ff1cc007d30 comp rx=0 tx=0).stop 2023-11-28T11:11:44.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.291+0000 7ff1dede5700 1 -- 172.21.15.28:0/1314636635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1d815dbf0 msgr2=0x7ff1d81f0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.291+0000 7ff1dede5700 1 --2- 172.21.15.28:0/1314636635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1d815dbf0 0x7ff1d81f0de0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7ff1d000b680 tx=0x7ff1d0004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.291+0000 7ff1dede5700 1 -- 172.21.15.28:0/1314636635 shutdown_connections 2023-11-28T11:11:44.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.291+0000 7ff1dede5700 1 --2- 172.21.15.28:0/1314636635 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1bc0682f0 0x7ff1bc06a7a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.291+0000 7ff1dede5700 1 --2- 172.21.15.28:0/1314636635 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1d815dbf0 0x7ff1d81f0de0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.291+0000 7ff1dede5700 1 --2- 172.21.15.28:0/1314636635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1d80b1d40 0x7ff1d81f1520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.291+0000 7ff1dede5700 1 --2- 172.21.15.28:0/1314636635 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff1d8159500 0x7ff1d8154170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.291+0000 7ff1dede5700 1 -- 172.21.15.28:0/1314636635 >> 172.21.15.28:0/1314636635 conn(0x7ff1d80cc630 msgr2=0x7ff1d80af8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:44.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.291+0000 7ff1dede5700 1 -- 172.21.15.28:0/1314636635 shutdown_connections 2023-11-28T11:11:44.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.291+0000 7ff1dede5700 1 -- 172.21.15.28:0/1314636635 wait complete. 2023-11-28T11:11:44.307 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T11:11:44.409 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:44.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.532+0000 7f6b8387a700 1 Processor -- start 2023-11-28T11:11:44.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.533+0000 7f6b8387a700 1 -- start start 2023-11-28T11:11:44.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.533+0000 7f6b8387a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6b7c161090 0x7f6b7c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.533+0000 7f6b8387a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b7c0caeb0 0x7f6b7c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.533+0000 7f6b8387a700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b7c0ce340 con 0x7f6b7c15c990 2023-11-28T11:11:44.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.533+0000 7f6b8387a700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b7c0cebe0 con 0x7f6b7c161090 2023-11-28T11:11:44.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.533+0000 7f6b8387a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0cf780 con 0x7f6b7c0caeb0 2023-11-28T11:11:44.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.534+0000 7f6b80e15700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6b7c161090 0x7f6b7c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.534+0000 7f6b81616700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6b7c15c990 0x7f6b7c0cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:38976/0 (socket says 172.21.15.28:38976) 2023-11-28T11:11:44.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.534+0000 7f6b81616700 1 -- 172.21.15.28:0/2441369051 learned_addr learned my addr 172.21.15.28:0/2441369051 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:44.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.534+0000 7f6b81e17700 1 --2- 172.21.15.28:0/2441369051 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b7c0caeb0 0x7f6b7c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.534+0000 7f6b80e15700 1 -- 172.21.15.28:0/2441369051 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b7c0caeb0 msgr2=0x7f6b7c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:44.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.534+0000 7f6b80e15700 1 --2- 172.21.15.28:0/2441369051 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b7c0caeb0 0x7f6b7c0cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.534+0000 7f6b80e15700 1 -- 172.21.15.28:0/2441369051 >> v1:172.21.15.28:6789/0 conn(0x7f6b7c15c990 legacy=0x7f6b7c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.534+0000 7f6b80e15700 1 -- 172.21.15.28:0/2441369051 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b7c0cb3f0 con 0x7f6b7c161090 2023-11-28T11:11:44.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.534+0000 7f6b6ffff700 1 -- 172.21.15.28:0/2441369051 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3783872890 0 0) 0x7f6b7c0ce340 con 0x7f6b7c15c990 2023-11-28T11:11:44.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.534+0000 7f6b80e15700 1 --2- 172.21.15.28:0/2441369051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b7c161090 0x7f6b7c0ca970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009840 tx=0x7f6b70009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e294f655b934b06 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:44.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.534+0000 7f6b6ffff700 1 -- 172.21.15.28:0/2441369051 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70011830 con 0x7f6b7c161090 2023-11-28T11:11:44.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.534+0000 7f6b6ffff700 1 -- 172.21.15.28:0/2441369051 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b70011990 con 0x7f6b7c161090 2023-11-28T11:11:44.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.534+0000 7f6b6ffff700 1 -- 172.21.15.28:0/2441369051 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b70011d20 con 0x7f6b7c161090 2023-11-28T11:11:44.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.535+0000 7f6b8387a700 1 -- 172.21.15.28:0/2441369051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b7c161090 msgr2=0x7f6b7c0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.535+0000 7f6b8387a700 1 --2- 172.21.15.28:0/2441369051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b7c161090 0x7f6b7c0ca970 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f6b70009840 tx=0x7f6b70009b50 comp rx=0 tx=0).stop 2023-11-28T11:11:44.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.535+0000 7f6b8387a700 1 -- 172.21.15.28:0/2441369051 shutdown_connections 2023-11-28T11:11:44.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.535+0000 7f6b8387a700 1 --2- 172.21.15.28:0/2441369051 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b7c0caeb0 0x7f6b7c0cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.535+0000 7f6b8387a700 1 --2- 172.21.15.28:0/2441369051 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b7c161090 0x7f6b7c0ca970 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.535+0000 7f6b8387a700 1 -- 172.21.15.28:0/2441369051 >> 172.21.15.28:0/2441369051 conn(0x7f6b7c0b1650 msgr2=0x7f6b7c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:44.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.535+0000 7f6b8387a700 1 -- 172.21.15.28:0/2441369051 shutdown_connections 2023-11-28T11:11:44.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.535+0000 7f6b8387a700 1 -- 172.21.15.28:0/2441369051 wait complete. 2023-11-28T11:11:44.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.536+0000 7f6b8387a700 1 Processor -- start 2023-11-28T11:11:44.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.536+0000 7f6b8387a700 1 -- start start 2023-11-28T11:11:44.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b8387a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0caeb0 0x7f6b7c152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b8387a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b7c15c990 0x7f6b7c1531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b8387a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b7c161090 0x7f6b7c0c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b8387a700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c160a30 con 0x7f6b7c161090 2023-11-28T11:11:44.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b8387a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c0ae270 con 0x7f6b7c0caeb0 2023-11-28T11:11:44.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b8387a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b7c160470 con 0x7f6b7c15c990 2023-11-28T11:11:44.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b81e17700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b7c161090 0x7f6b7c0c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b81e17700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b7c161090 0x7f6b7c0c8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50928/0 (socket says 172.21.15.28:50928) 2023-11-28T11:11:44.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b81e17700 1 -- 172.21.15.28:0/4008550674 learned_addr learned my addr 172.21.15.28:0/4008550674 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:44.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b80e15700 1 --2- 172.21.15.28:0/4008550674 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b7c15c990 0x7f6b7c1531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b81616700 1 --2- 172.21.15.28:0/4008550674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0caeb0 0x7f6b7c152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b81e17700 1 -- 172.21.15.28:0/4008550674 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b7c15c990 msgr2=0x7f6b7c1531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b81e17700 1 --2- 172.21.15.28:0/4008550674 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b7c15c990 0x7f6b7c1531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b81e17700 1 -- 172.21.15.28:0/4008550674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0caeb0 msgr2=0x7f6b7c152c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b81e17700 1 --2- 172.21.15.28:0/4008550674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0caeb0 0x7f6b7c152c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.537+0000 7f6b81e17700 1 -- 172.21.15.28:0/4008550674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b700094a0 con 0x7f6b7c161090 2023-11-28T11:11:44.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.538+0000 7f6b81e17700 1 --2- 172.21.15.28:0/4008550674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b7c161090 0x7f6b7c0c8c60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400efc0 tx=0x7f6b7400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:44.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.538+0000 7f6b6dffb700 1 -- 172.21.15.28:0/4008550674 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b74009ea0 con 0x7f6b7c161090 2023-11-28T11:11:44.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.538+0000 7f6b6dffb700 1 -- 172.21.15.28:0/4008550674 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b7400f040 con 0x7f6b7c161090 2023-11-28T11:11:44.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.538+0000 7f6b8387a700 1 -- 172.21.15.28:0/4008550674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b7c0ce870 con 0x7f6b7c161090 2023-11-28T11:11:44.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.538+0000 7f6b6dffb700 1 -- 172.21.15.28:0/4008550674 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b7400ced0 con 0x7f6b7c161090 2023-11-28T11:11:44.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.538+0000 7f6b8387a700 1 -- 172.21.15.28:0/4008550674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b7c0ce590 con 0x7f6b7c161090 2023-11-28T11:11:44.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.539+0000 7f6b8387a700 1 -- 172.21.15.28:0/4008550674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b7c0cc850 con 0x7f6b7c161090 2023-11-28T11:11:44.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.539+0000 7f6b6dffb700 1 -- 172.21.15.28:0/4008550674 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6b74014480 con 0x7f6b7c161090 2023-11-28T11:11:44.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.540+0000 7f6b6dffb700 1 --2- 172.21.15.28:0/4008550674 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6b600682f0 0x7f6b6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:44.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.540+0000 7f6b6dffb700 1 -- 172.21.15.28:0/4008550674 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6b740866f0 con 0x7f6b7c161090 2023-11-28T11:11:44.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.540+0000 7f6b81616700 1 --2- 172.21.15.28:0/4008550674 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6b600682f0 0x7f6b6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:44.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.540+0000 7f6b81616700 1 --2- 172.21.15.28:0/4008550674 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6b600682f0 0x7f6b6006a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6b78007fd0 tx=0x7f6b78007de0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:44.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.544+0000 7f6b6dffb700 1 -- 172.21.15.28:0/4008550674 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6b74052db0 con 0x7f6b7c161090 2023-11-28T11:11:44.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.784+0000 7f6b8387a700 1 -- 172.21.15.28:0/4008550674 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b7c0675a0 con 0x7f6b7c161090 2023-11-28T11:11:44.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.785+0000 7f6b6dffb700 1 -- 172.21.15.28:0/4008550674 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f6b74057e60 con 0x7f6b7c161090 2023-11-28T11:11:44.786 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:44.787 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5198,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/284515388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":284515388},{"type":"v1","addr":"172.21.15.59:6835","nonce":284515388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2160385010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2160385010},{"type":"v1","addr":"172.21.15.28:6836","nonce":2160385010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3775145347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3775145347},{"type":"v1","addr":"172.21.15.28:6837","nonce":3775145347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:59.771956+0000","modified":"2023-11-28T11:11:03.109382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6834/3261856585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3261856585},{"type":"v1","addr":"172.21.15.59:6834","nonce":3261856585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:11:44.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.787+0000 7f6b8387a700 1 -- 172.21.15.28:0/4008550674 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6b600682f0 msgr2=0x7f6b6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.787+0000 7f6b8387a700 1 --2- 172.21.15.28:0/4008550674 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6b600682f0 0x7f6b6006a7a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f6b78007fd0 tx=0x7f6b78007de0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.787+0000 7f6b8387a700 1 -- 172.21.15.28:0/4008550674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b7c161090 msgr2=0x7f6b7c0c8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:44.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.787+0000 7f6b8387a700 1 --2- 172.21.15.28:0/4008550674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b7c161090 0x7f6b7c0c8c60 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f6b7400efc0 tx=0x7f6b7400c3a0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.787+0000 7f6b8387a700 1 -- 172.21.15.28:0/4008550674 shutdown_connections 2023-11-28T11:11:44.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.787+0000 7f6b8387a700 1 --2- 172.21.15.28:0/4008550674 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6b600682f0 0x7f6b6006a7a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.787+0000 7f6b8387a700 1 --2- 172.21.15.28:0/4008550674 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6b7c161090 0x7f6b7c0c8c60 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.787+0000 7f6b8387a700 1 --2- 172.21.15.28:0/4008550674 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6b7c0caeb0 0x7f6b7c152c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.787+0000 7f6b8387a700 1 --2- 172.21.15.28:0/4008550674 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6b7c15c990 0x7f6b7c1531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:44.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.787+0000 7f6b8387a700 1 -- 172.21.15.28:0/4008550674 >> 172.21.15.28:0/4008550674 conn(0x7f6b7c0b1650 msgr2=0x7f6b7c0ca0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:44.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.787+0000 7f6b8387a700 1 -- 172.21.15.28:0/4008550674 shutdown_connections 2023-11-28T11:11:44.789 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:44.787+0000 7f6b8387a700 1 -- 172.21.15.28:0/4008550674 wait complete. 2023-11-28T11:11:44.790 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 25 2023-11-28T11:11:44.801 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-28T11:11:44.962 INFO:teuthology.orchestra.run.smithi059.stdout:{"message":"","return_code":0} 2023-11-28T11:11:44.962 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:11:44.963 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:11:45.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.006+0000 7fa0238b6080 1 Processor -- start 2023-11-28T11:11:45.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.006+0000 7fa0238b6080 1 -- start start 2023-11-28T11:11:45.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.007+0000 7fa0238b6080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfcd6a3870 0x55dfcd6cd6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.007+0000 7fa0238b6080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dfcd6cfd70 0x55dfcd69f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.007+0000 7fa0238b6080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dfcd7796e0 0x55dfcd69fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.007+0000 7fa0238b6080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcd5dbfe0 con 0x55dfcd6cfd70 2023-11-28T11:11:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.007+0000 7fa0238b6080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcd771080 con 0x55dfcd6a3870 2023-11-28T11:11:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.007+0000 7fa0238b6080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dfcd6ebcd0 con 0x55dfcd7796e0 2023-11-28T11:11:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.007+0000 7fa0191b8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dfcd6cfd70 0x55dfcd69f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.007+0000 7fa0191b8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dfcd6cfd70 0x55dfcd69f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50940/0 (socket says 172.21.15.28:50940) 2023-11-28T11:11:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.007+0000 7fa0191b8700 1 -- 172.21.15.28:0/1538788133 learned_addr learned my addr 172.21.15.28:0/1538788133 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.007+0000 7fa0199b9700 1 --2- 172.21.15.28:0/1538788133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfcd6a3870 0x55dfcd6cd6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.007+0000 7fa01a1ba700 1 --2- 172.21.15.28:0/1538788133 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dfcd7796e0 0x55dfcd69fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa0191b8700 1 -- 172.21.15.28:0/1538788133 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dfcd7796e0 msgr2=0x55dfcd69fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa0191b8700 1 --2- 172.21.15.28:0/1538788133 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dfcd7796e0 0x55dfcd69fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa0191b8700 1 -- 172.21.15.28:0/1538788133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfcd6a3870 msgr2=0x55dfcd6cd6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa0191b8700 1 --2- 172.21.15.28:0/1538788133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfcd6a3870 0x55dfcd6cd6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa0191b8700 1 -- 172.21.15.28:0/1538788133 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa01400a1c0 con 0x55dfcd6cfd70 2023-11-28T11:11:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa0191b8700 1 --2- 172.21.15.28:0/1538788133 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dfcd6cfd70 0x55dfcd69f6e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa00c001cb0 tx=0x7fa00c00b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=131357c1e97ff94c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa00bfff700 1 -- 172.21.15.28:0/1538788133 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c014070 con 0x55dfcd6cfd70 2023-11-28T11:11:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa00bfff700 1 -- 172.21.15.28:0/1538788133 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa00c008030 con 0x55dfcd6cfd70 2023-11-28T11:11:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa00bfff700 1 -- 172.21.15.28:0/1538788133 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa00c00baa0 con 0x55dfcd6cfd70 2023-11-28T11:11:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa0238b6080 1 -- 172.21.15.28:0/1538788133 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dfcd6cfd70 msgr2=0x55dfcd69f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa0238b6080 1 --2- 172.21.15.28:0/1538788133 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dfcd6cfd70 0x55dfcd69f6e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7fa00c001cb0 tx=0x7fa00c00b4c0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa0238b6080 1 -- 172.21.15.28:0/1538788133 shutdown_connections 2023-11-28T11:11:45.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa0238b6080 1 --2- 172.21.15.28:0/1538788133 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dfcd7796e0 0x55dfcd69fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa0238b6080 1 --2- 172.21.15.28:0/1538788133 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfcd6a3870 0x55dfcd6cd6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa0238b6080 1 --2- 172.21.15.28:0/1538788133 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dfcd6cfd70 0x55dfcd69f6e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa0238b6080 1 -- 172.21.15.28:0/1538788133 >> 172.21.15.28:0/1538788133 conn(0x55dfcd6ce300 msgr2=0x55dfcd6d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa0238b6080 1 -- 172.21.15.28:0/1538788133 shutdown_connections 2023-11-28T11:11:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.008+0000 7fa0238b6080 1 -- 172.21.15.28:0/1538788133 wait complete. 2023-11-28T11:11:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.009+0000 7fa0238b6080 1 Processor -- start 2023-11-28T11:11:45.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.009+0000 7fa0238b6080 1 -- start start 2023-11-28T11:11:45.016 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0238b6080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dfcd6a3870 0x55dfcd6cee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.016 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0238b6080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dfcd6cfd70 0x55dfcd6e6390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0238b6080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfcd7796e0 0x55dfcd6e68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0238b6080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcd6a2570 con 0x55dfcd6a3870 2023-11-28T11:11:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0238b6080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55dfcd775da0 con 0x55dfcd7796e0 2023-11-28T11:11:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0238b6080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55dfcd6e5d60 con 0x55dfcd6cfd70 2023-11-28T11:11:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0199b9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dfcd6a3870 0x55dfcd6cee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0199b9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dfcd6a3870 0x55dfcd6cee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50944/0 (socket says 172.21.15.28:50944) 2023-11-28T11:11:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0199b9700 1 -- 172.21.15.28:0/2582326439 learned_addr learned my addr 172.21.15.28:0/2582326439 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0191b8700 1 --2- 172.21.15.28:0/2582326439 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dfcd6cfd70 0x55dfcd6e6390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa01a1ba700 1 --2- 172.21.15.28:0/2582326439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfcd7796e0 0x55dfcd6e68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0199b9700 1 -- 172.21.15.28:0/2582326439 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dfcd6cfd70 msgr2=0x55dfcd6e6390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0199b9700 1 --2- 172.21.15.28:0/2582326439 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dfcd6cfd70 0x55dfcd6e6390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0199b9700 1 -- 172.21.15.28:0/2582326439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfcd7796e0 msgr2=0x55dfcd6e68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0199b9700 1 --2- 172.21.15.28:0/2582326439 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55dfcd7796e0 0x55dfcd6e68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0199b9700 1 -- 172.21.15.28:0/2582326439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa00c00a040 con 0x55dfcd6a3870 2023-11-28T11:11:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0191b8700 1 --2- 172.21.15.28:0/2582326439 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55dfcd6cfd70 0x55dfcd6e6390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.010+0000 7fa0199b9700 1 --2- 172.21.15.28:0/2582326439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dfcd6a3870 0x55dfcd6cee20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa010009fb0 tx=0x7fa0100086e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.011+0000 7fa00affd700 1 -- 172.21.15.28:0/2582326439 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa010011040 con 0x55dfcd6a3870 2023-11-28T11:11:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.011+0000 7fa00affd700 1 -- 172.21.15.28:0/2582326439 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa010016680 con 0x55dfcd6a3870 2023-11-28T11:11:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.011+0000 7fa0238b6080 1 -- 172.21.15.28:0/2582326439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55dfcd5f4120 con 0x55dfcd6a3870 2023-11-28T11:11:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.011+0000 7fa00affd700 1 -- 172.21.15.28:0/2582326439 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100169f0 con 0x55dfcd6a3870 2023-11-28T11:11:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.011+0000 7fa0238b6080 1 -- 172.21.15.28:0/2582326439 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55dfcd5e7830 con 0x55dfcd6a3870 2023-11-28T11:11:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.012+0000 7fa00affd700 1 -- 172.21.15.28:0/2582326439 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa01001d030 con 0x55dfcd6a3870 2023-11-28T11:11:45.018 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.012+0000 7fa00affd700 1 --2- 172.21.15.28:0/2582326439 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ffc066c10 0x7f9ffc0690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.012+0000 7fa00affd700 1 -- 172.21.15.28:0/2582326439 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fa010084420 con 0x55dfcd6a3870 2023-11-28T11:11:45.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.012+0000 7fa0191b8700 1 --2- 172.21.15.28:0/2582326439 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ffc066c10 0x7f9ffc0690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.013+0000 7fa0238b6080 1 --2- 172.21.15.28:0/2582326439 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x55dfcd778d90 0x55dfcd76db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.013+0000 7fa0238b6080 1 -- 172.21.15.28:0/2582326439 --> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] -- osd_op(unknown.0.0:1 9.7 9:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e54) v8 -- 0x55dfcd76e090 con 0x55dfcd778d90 2023-11-28T11:11:45.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.013+0000 7fa0191b8700 1 --2- 172.21.15.28:0/2582326439 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ffc066c10 0x7f9ffc0690c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55dfcd6cdfb0 tx=0x7fa00c009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:45.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.013+0000 7fa01a1ba700 1 --2- 172.21.15.28:0/2582326439 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x55dfcd778d90 0x55dfcd76db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.013+0000 7fa01a1ba700 1 --2- 172.21.15.28:0/2582326439 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x55dfcd778d90 0x55dfcd76db50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:45.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.014+0000 7fa01a1ba700 1 -- 172.21.15.28:0/2582326439 <== osd.5 v2:172.21.15.59:6800/103696 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (95) out=95b] v0'0 uv3 ondisk = 0) v8 ==== 164+0+95 (crc 0 0 0) 0x7fa014017040 con 0x55dfcd778d90 2023-11-28T11:11:45.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.014+0000 7fa0238b6080 1 -- 172.21.15.28:0/2582326439 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x55dfcd778d90 msgr2=0x55dfcd76db50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.014+0000 7fa0238b6080 1 --2- 172.21.15.28:0/2582326439 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x55dfcd778d90 0x55dfcd76db50 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.014+0000 7fa0238b6080 1 -- 172.21.15.28:0/2582326439 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ffc066c10 msgr2=0x7f9ffc0690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.019 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.014+0000 7fa0238b6080 1 --2- 172.21.15.28:0/2582326439 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ffc066c10 0x7f9ffc0690c0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x55dfcd6cdfb0 tx=0x7fa00c009000 comp rx=0 tx=0).stop 2023-11-28T11:11:45.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.014+0000 7fa0238b6080 1 -- 172.21.15.28:0/2582326439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dfcd6a3870 msgr2=0x55dfcd6cee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.014+0000 7fa0238b6080 1 --2- 172.21.15.28:0/2582326439 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55dfcd6a3870 0x55dfcd6cee20 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7fa010009fb0 tx=0x7fa0100086e0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.014+0000 7fa01a1ba700 1 -- 172.21.15.28:0/2582326439 reap_dead start 2023-11-28T11:11:45.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.014+0000 7fa0238b6080 1 -- 172.21.15.28:0/2582326439 shutdown_connections 2023-11-28T11:11:45.020 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.014+0000 7fa0238b6080 1 -- 172.21.15.28:0/2582326439 >> 172.21.15.28:0/2582326439 conn(0x55dfcd6ce300 msgr2=0x55dfcd76fff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:45.021 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.014+0000 7fa0238b6080 1 -- 172.21.15.28:0/2582326439 shutdown_connections 2023-11-28T11:11:45.021 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.014+0000 7fa0238b6080 1 -- 172.21.15.28:0/2582326439 wait complete. 2023-11-28T11:11:45.022 INFO:teuthology.orchestra.run.smithi028.stdout:Yalphaparent_a 2023-11-28T11:11:45.023 DEBUG:teuthology.orchestra.run.smithi028:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:11:45.078 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:11:45.079 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511627777, 2023-11-28T11:11:45.079 INFO:teuthology.orchestra.run.smithi028.stdout: "ancestors": [ 2023-11-28T11:11:45.079 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:45.079 INFO:teuthology.orchestra.run.smithi028.stdout: "dirino": 1099511627776, 2023-11-28T11:11:45.079 INFO:teuthology.orchestra.run.smithi028.stdout: "dname": "alpha", 2023-11-28T11:11:45.079 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 4 2023-11-28T11:11:45.079 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:45.079 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:45.079 INFO:teuthology.orchestra.run.smithi028.stdout: "dirino": 1, 2023-11-28T11:11:45.079 INFO:teuthology.orchestra.run.smithi028.stdout: "dname": "parent_a", 2023-11-28T11:11:45.080 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 7 2023-11-28T11:11:45.080 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:45.080 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2023-11-28T11:11:45.080 INFO:teuthology.orchestra.run.smithi028.stdout: "pool": 9, 2023-11-28T11:11:45.080 INFO:teuthology.orchestra.run.smithi028.stdout: "old_pools": [] 2023-11-28T11:11:45.080 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:11:45.080 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:45.081 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:11:45.081 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:11:45.161 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:11:45.205 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:11:45.205 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:11:45.205 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:45.224 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:11:45.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.386+0000 7ff524a18700 1 Processor -- start 2023-11-28T11:11:45.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.386+0000 7ff524a18700 1 -- start start 2023-11-28T11:11:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.387+0000 7ff524a18700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff5200cc310 0x7ff5200cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.387+0000 7ff524a18700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5200ce340 con 0x7ff52015c990 2023-11-28T11:11:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.387+0000 7ff524a18700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5200cebe0 con 0x7ff520161090 2023-11-28T11:11:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.387+0000 7ff524a18700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5200cf780 con 0x7ff5200cc310 2023-11-28T11:11:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.387+0000 7ff51ed9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff5200cc310 0x7ff5200cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.387+0000 7ff51e59c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff52015c990 0x7ff5200cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39012/0 (socket says 172.21.15.28:39012) 2023-11-28T11:11:45.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.387+0000 7ff51ed9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff5200cc310 0x7ff5200cc770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50958/0 (socket says 172.21.15.28:50958) 2023-11-28T11:11:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.387+0000 7ff51e59c700 1 -- 172.21.15.28:0/4091915053 learned_addr learned my addr 172.21.15.28:0/4091915053 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.387+0000 7ff51ed9d700 1 -- 172.21.15.28:0/4091915053 >> v1:172.21.15.59:6789/0 conn(0x7ff520161090 legacy=0x7ff5200ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.387+0000 7ff51ed9d700 1 -- 172.21.15.28:0/4091915053 >> v1:172.21.15.28:6789/0 conn(0x7ff52015c990 legacy=0x7ff5200cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.387+0000 7ff51d59a700 1 -- 172.21.15.28:0/4091915053 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1078667774 0 0) 0x7ff5200cebe0 con 0x7ff520161090 2023-11-28T11:11:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.387+0000 7ff51ed9d700 1 -- 172.21.15.28:0/4091915053 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5200cccb0 con 0x7ff5200cc310 2023-11-28T11:11:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.387+0000 7ff51ed9d700 1 --2- 172.21.15.28:0/4091915053 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5200cc310 0x7ff5200cc770 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff518009d90 tx=0x7ff51800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c92235d1a57121dc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.388+0000 7ff51d59a700 1 -- 172.21.15.28:0/4091915053 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51800e070 con 0x7ff5200cc310 2023-11-28T11:11:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.388+0000 7ff51d59a700 1 -- 172.21.15.28:0/4091915053 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff51800bad0 con 0x7ff5200cc310 2023-11-28T11:11:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.388+0000 7ff51d59a700 1 -- 172.21.15.28:0/4091915053 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff51800be80 con 0x7ff5200cc310 2023-11-28T11:11:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.388+0000 7ff524a18700 1 -- 172.21.15.28:0/4091915053 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5200cc310 msgr2=0x7ff5200cc770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.388+0000 7ff524a18700 1 --2- 172.21.15.28:0/4091915053 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5200cc310 0x7ff5200cc770 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ff518009d90 tx=0x7ff51800b330 comp rx=0 tx=0).stop 2023-11-28T11:11:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.388+0000 7ff524a18700 1 -- 172.21.15.28:0/4091915053 shutdown_connections 2023-11-28T11:11:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.388+0000 7ff524a18700 1 --2- 172.21.15.28:0/4091915053 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5200cc310 0x7ff5200cc770 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.388+0000 7ff524a18700 1 -- 172.21.15.28:0/4091915053 >> 172.21.15.28:0/4091915053 conn(0x7ff5200b1650 msgr2=0x7ff5200b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.388+0000 7ff524a18700 1 -- 172.21.15.28:0/4091915053 shutdown_connections 2023-11-28T11:11:45.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.388+0000 7ff524a18700 1 -- 172.21.15.28:0/4091915053 wait complete. 2023-11-28T11:11:45.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.390+0000 7ff524a18700 1 Processor -- start 2023-11-28T11:11:45.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.390+0000 7ff524a18700 1 -- start start 2023-11-28T11:11:45.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.391+0000 7ff524a18700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5200cc310 0x7ff5201f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.391+0000 7ff524a18700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff52015c990 0x7ff5200c9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.391+0000 7ff524a18700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff520161090 0x7ff5200ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.391+0000 7ff524a18700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5201f48b0 con 0x7ff5200cc310 2023-11-28T11:11:45.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.391+0000 7ff524a18700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5200c9b70 con 0x7ff52015c990 2023-11-28T11:11:45.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.391+0000 7ff524a18700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5201f3c30 con 0x7ff520161090 2023-11-28T11:11:45.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.391+0000 7ff51e59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5200cc310 0x7ff5201f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.391+0000 7ff51e59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5200cc310 0x7ff5201f3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50960/0 (socket says 172.21.15.28:50960) 2023-11-28T11:11:45.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.391+0000 7ff51e59c700 1 -- 172.21.15.28:0/4282834396 learned_addr learned my addr 172.21.15.28:0/4282834396 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:45.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.391+0000 7ff51dd9b700 1 --2- 172.21.15.28:0/4282834396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff52015c990 0x7ff5200c9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.391+0000 7ff51e59c700 1 -- 172.21.15.28:0/4282834396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff520161090 msgr2=0x7ff5200ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.391+0000 7ff51e59c700 1 --2- 172.21.15.28:0/4282834396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff520161090 0x7ff5200ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.391+0000 7ff51e59c700 1 -- 172.21.15.28:0/4282834396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff52015c990 msgr2=0x7ff5200c9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.391+0000 7ff51e59c700 1 --2- 172.21.15.28:0/4282834396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff52015c990 0x7ff5200c9630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.391+0000 7ff51e59c700 1 -- 172.21.15.28:0/4282834396 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff518009a70 con 0x7ff5200cc310 2023-11-28T11:11:45.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.392+0000 7ff51e59c700 1 --2- 172.21.15.28:0/4282834396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5200cc310 0x7ff5201f3600 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff51000c7b0 tx=0x7ff51000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:45.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.392+0000 7ff50effd700 1 -- 172.21.15.28:0/4282834396 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510004c80 con 0x7ff5200cc310 2023-11-28T11:11:45.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.392+0000 7ff50effd700 1 -- 172.21.15.28:0/4282834396 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff510004de0 con 0x7ff5200cc310 2023-11-28T11:11:45.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.392+0000 7ff50effd700 1 -- 172.21.15.28:0/4282834396 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff510007310 con 0x7ff5200cc310 2023-11-28T11:11:45.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.392+0000 7ff524a18700 1 -- 172.21.15.28:0/4282834396 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5200ce590 con 0x7ff5200cc310 2023-11-28T11:11:45.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.392+0000 7ff524a18700 1 -- 172.21.15.28:0/4282834396 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5200cb3f0 con 0x7ff5200cc310 2023-11-28T11:11:45.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.393+0000 7ff524a18700 1 -- 172.21.15.28:0/4282834396 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5200ce870 con 0x7ff5200cc310 2023-11-28T11:11:45.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.393+0000 7ff50effd700 1 -- 172.21.15.28:0/4282834396 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff510010070 con 0x7ff5200cc310 2023-11-28T11:11:45.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.393+0000 7ff50effd700 1 --2- 172.21.15.28:0/4282834396 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff500068320 0x7ff50006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.394+0000 7ff50effd700 1 -- 172.21.15.28:0/4282834396 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ff510084e00 con 0x7ff5200cc310 2023-11-28T11:11:45.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.394+0000 7ff51dd9b700 1 --2- 172.21.15.28:0/4282834396 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff500068320 0x7ff50006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.398 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.394+0000 7ff51dd9b700 1 --2- 172.21.15.28:0/4282834396 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff500068320 0x7ff50006a7d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff514007fd0 tx=0x7ff514009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:45.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.398+0000 7ff50effd700 1 -- 172.21.15.28:0/4282834396 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff510051520 con 0x7ff5200cc310 2023-11-28T11:11:45.592 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.590+0000 7ff524a18700 1 -- 172.21.15.28:0/4282834396 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff5200675a0 con 0x7ff5200cc310 2023-11-28T11:11:45.593 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.592+0000 7ff50effd700 1 -- 172.21.15.28:0/4282834396 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18074 (secure 0 0 0) 0x7ff510018020 con 0x7ff5200cc310 2023-11-28T11:11:45.593 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:45.594 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":54,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:11:00.773581+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:57.702079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:58.738753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:45.595 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.594+0000 7ff524a18700 1 -- 172.21.15.28:0/4282834396 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff500068320 msgr2=0x7ff50006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.594+0000 7ff524a18700 1 --2- 172.21.15.28:0/4282834396 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff500068320 0x7ff50006a7d0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7ff514007fd0 tx=0x7ff514009040 comp rx=0 tx=0).stop 2023-11-28T11:11:45.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.594+0000 7ff524a18700 1 -- 172.21.15.28:0/4282834396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5200cc310 msgr2=0x7ff5201f3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.594+0000 7ff524a18700 1 --2- 172.21.15.28:0/4282834396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5200cc310 0x7ff5201f3600 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ff51000c7b0 tx=0x7ff51000cb70 comp rx=0 tx=0).stop 2023-11-28T11:11:45.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.594+0000 7ff524a18700 1 -- 172.21.15.28:0/4282834396 shutdown_connections 2023-11-28T11:11:45.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.594+0000 7ff524a18700 1 --2- 172.21.15.28:0/4282834396 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff500068320 0x7ff50006a7d0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.594+0000 7ff524a18700 1 --2- 172.21.15.28:0/4282834396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff5200cc310 0x7ff5201f3600 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.594+0000 7ff524a18700 1 --2- 172.21.15.28:0/4282834396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff520161090 0x7ff5200ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.594+0000 7ff524a18700 1 --2- 172.21.15.28:0/4282834396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff52015c990 0x7ff5200c9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.594+0000 7ff524a18700 1 -- 172.21.15.28:0/4282834396 >> 172.21.15.28:0/4282834396 conn(0x7ff5200b1650 msgr2=0x7ff52015bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:45.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.594+0000 7ff524a18700 1 -- 172.21.15.28:0/4282834396 shutdown_connections 2023-11-28T11:11:45.596 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.594+0000 7ff524a18700 1 -- 172.21.15.28:0/4282834396 wait complete. 2023-11-28T11:11:45.608 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:10:57.702079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:10:58.738753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:11:45.608 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:11:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.734+0000 7f32f0727700 1 Processor -- start 2023-11-28T11:11:45.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.734+0000 7f32f0727700 1 -- start start 2023-11-28T11:11:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.735+0000 7f32f0727700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32e815c4b0 0x7f32e815d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.735+0000 7f32f0727700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32e80c9950 0x7f32e80cced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.735+0000 7f32f0727700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32e80cae80 con 0x7f32e815c880 2023-11-28T11:11:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.735+0000 7f32f0727700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32e815dba0 con 0x7f32e80c9950 2023-11-28T11:11:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.735+0000 7f32f0727700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e80c9de0 con 0x7f32e815c4b0 2023-11-28T11:11:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.735+0000 7f32eecc4700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32e80c9950 0x7f32e80cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.735+0000 7f32edcc2700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f32e815c880 0x7f32e806a1e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39026/0 (socket says 172.21.15.28:39026) 2023-11-28T11:11:45.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.735+0000 7f32edcc2700 1 -- 172.21.15.28:0/1071593863 learned_addr learned my addr 172.21.15.28:0/1071593863 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:45.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.735+0000 7f32ee4c3700 1 --2- 172.21.15.28:0/1071593863 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32e815c4b0 0x7f32e815d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.735+0000 7f32eecc4700 1 -- 172.21.15.28:0/1071593863 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32e815c4b0 msgr2=0x7f32e815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:45.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.735+0000 7f32eecc4700 1 --2- 172.21.15.28:0/1071593863 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32e815c4b0 0x7f32e815d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.735+0000 7f32eecc4700 1 -- 172.21.15.28:0/1071593863 >> v1:172.21.15.28:6789/0 conn(0x7f32e815c880 legacy=0x7f32e806a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.735+0000 7f32eecc4700 1 -- 172.21.15.28:0/1071593863 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e80ca0c0 con 0x7f32e80c9950 2023-11-28T11:11:45.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.735+0000 7f32ee4c3700 1 --2- 172.21.15.28:0/1071593863 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32e815c4b0 0x7f32e815d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:45.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.735+0000 7f32eecc4700 1 --2- 172.21.15.28:0/1071593863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32e80c9950 0x7f32e80cced0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009d90 tx=0x7f32e000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e7dc1d92aa761c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:45.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.736+0000 7f32ed4c1700 1 -- 172.21.15.28:0/1071593863 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e000e070 con 0x7f32e80c9950 2023-11-28T11:11:45.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.736+0000 7f32ed4c1700 1 -- 172.21.15.28:0/1071593863 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e000bbd0 con 0x7f32e80c9950 2023-11-28T11:11:45.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.736+0000 7f32ed4c1700 1 -- 172.21.15.28:0/1071593863 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e0012430 con 0x7f32e80c9950 2023-11-28T11:11:45.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.736+0000 7f32f0727700 1 -- 172.21.15.28:0/1071593863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32e80c9950 msgr2=0x7f32e80cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.736+0000 7f32f0727700 1 --2- 172.21.15.28:0/1071593863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32e80c9950 0x7f32e80cced0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7f32e0009d90 tx=0x7f32e000b330 comp rx=0 tx=0).stop 2023-11-28T11:11:45.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.736+0000 7f32f0727700 1 -- 172.21.15.28:0/1071593863 shutdown_connections 2023-11-28T11:11:45.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.736+0000 7f32f0727700 1 --2- 172.21.15.28:0/1071593863 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32e80c9950 0x7f32e80cced0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.736+0000 7f32f0727700 1 --2- 172.21.15.28:0/1071593863 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32e815c4b0 0x7f32e815d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.736+0000 7f32f0727700 1 -- 172.21.15.28:0/1071593863 >> 172.21.15.28:0/1071593863 conn(0x7f32e80b0df0 msgr2=0x7f32e80b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:45.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.736+0000 7f32f0727700 1 -- 172.21.15.28:0/1071593863 shutdown_connections 2023-11-28T11:11:45.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.736+0000 7f32f0727700 1 -- 172.21.15.28:0/1071593863 wait complete. 2023-11-28T11:11:45.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.737+0000 7f32f0727700 1 Processor -- start 2023-11-28T11:11:45.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.737+0000 7f32f0727700 1 -- start start 2023-11-28T11:11:45.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.738+0000 7f32f0727700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32e80c9950 0x7f32e8148950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.738+0000 7f32f0727700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e815c4b0 0x7f32e80ad2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.738+0000 7f32f0727700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32e815c880 0x7f32e8159fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.738+0000 7f32f0727700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e80c6440 con 0x7f32e80c9950 2023-11-28T11:11:45.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.738+0000 7f32f0727700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32e80adcf0 con 0x7f32e815c4b0 2023-11-28T11:11:45.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.738+0000 7f32f0727700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32e80adfd0 con 0x7f32e815c880 2023-11-28T11:11:45.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.738+0000 7f32ee4c3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32e80c9950 0x7f32e8148950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.738+0000 7f32ee4c3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32e80c9950 0x7f32e8148950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50974/0 (socket says 172.21.15.28:50974) 2023-11-28T11:11:45.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.738+0000 7f32ee4c3700 1 -- 172.21.15.28:0/2773402356 learned_addr learned my addr 172.21.15.28:0/2773402356 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:45.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.739+0000 7f32ee4c3700 1 -- 172.21.15.28:0/2773402356 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32e815c880 msgr2=0x7f32e8159fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.739+0000 7f32ee4c3700 1 --2- 172.21.15.28:0/2773402356 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32e815c880 0x7f32e8159fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.739+0000 7f32edcc2700 1 --2- 172.21.15.28:0/2773402356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e815c4b0 0x7f32e80ad2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.739+0000 7f32ee4c3700 1 -- 172.21.15.28:0/2773402356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e815c4b0 msgr2=0x7f32e80ad2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.739+0000 7f32ee4c3700 1 --2- 172.21.15.28:0/2773402356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e815c4b0 0x7f32e80ad2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.739+0000 7f32ee4c3700 1 -- 172.21.15.28:0/2773402356 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e0009a70 con 0x7f32e80c9950 2023-11-28T11:11:45.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.739+0000 7f32ee4c3700 1 --2- 172.21.15.28:0/2773402356 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32e80c9950 0x7f32e8148950 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f32e400e810 tx=0x7f32e400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:45.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.739+0000 7f32deffd700 1 -- 172.21.15.28:0/2773402356 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4004230 con 0x7f32e80c9950 2023-11-28T11:11:45.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.739+0000 7f32deffd700 1 -- 172.21.15.28:0/2773402356 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e4004390 con 0x7f32e80c9950 2023-11-28T11:11:45.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.739+0000 7f32f0727700 1 -- 172.21.15.28:0/2773402356 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32e80ca680 con 0x7f32e80c9950 2023-11-28T11:11:45.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.739+0000 7f32deffd700 1 -- 172.21.15.28:0/2773402356 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e40036a0 con 0x7f32e80c9950 2023-11-28T11:11:45.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.739+0000 7f32f0727700 1 -- 172.21.15.28:0/2773402356 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32e80ca3a0 con 0x7f32e80c9950 2023-11-28T11:11:45.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.740+0000 7f32f0727700 1 -- 172.21.15.28:0/2773402356 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32e80ca960 con 0x7f32e80c9950 2023-11-28T11:11:45.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.740+0000 7f32deffd700 1 -- 172.21.15.28:0/2773402356 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f32e4022070 con 0x7f32e80c9950 2023-11-28T11:11:45.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.741+0000 7f32deffd700 1 --2- 172.21.15.28:0/2773402356 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32cc0682f0 0x7f32cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:45.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.741+0000 7f32deffd700 1 -- 172.21.15.28:0/2773402356 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f32e4085880 con 0x7f32e80c9950 2023-11-28T11:11:45.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.741+0000 7f32edcc2700 1 --2- 172.21.15.28:0/2773402356 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32cc0682f0 0x7f32cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:45.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.741+0000 7f32edcc2700 1 --2- 172.21.15.28:0/2773402356 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32cc0682f0 0x7f32cc06a7a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f32e80c6160 tx=0x7f32d8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:45.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.745+0000 7f32deffd700 1 -- 172.21.15.28:0/2773402356 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f32e4051f40 con 0x7f32e80c9950 2023-11-28T11:11:45.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.936+0000 7f32f0727700 1 -- 172.21.15.28:0/2773402356 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f32e80c66b0 con 0x7f32e80c9950 2023-11-28T11:11:45.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.936+0000 7f32deffd700 1 -- 172.21.15.28:0/2773402356 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7f32e4055560 con 0x7f32e80c9950 2023-11-28T11:11:45.938 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:11:45.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.938+0000 7f32f0727700 1 -- 172.21.15.28:0/2773402356 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32cc0682f0 msgr2=0x7f32cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.938+0000 7f32f0727700 1 --2- 172.21.15.28:0/2773402356 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32cc0682f0 0x7f32cc06a7a0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f32e80c6160 tx=0x7f32d8009040 comp rx=0 tx=0).stop 2023-11-28T11:11:45.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.938+0000 7f32f0727700 1 -- 172.21.15.28:0/2773402356 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32e80c9950 msgr2=0x7f32e8148950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:45.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.938+0000 7f32f0727700 1 --2- 172.21.15.28:0/2773402356 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32e80c9950 0x7f32e8148950 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7f32e400e810 tx=0x7f32e400ebd0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.938+0000 7f32f0727700 1 -- 172.21.15.28:0/2773402356 shutdown_connections 2023-11-28T11:11:45.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.938+0000 7f32f0727700 1 --2- 172.21.15.28:0/2773402356 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32cc0682f0 0x7f32cc06a7a0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.938+0000 7f32f0727700 1 --2- 172.21.15.28:0/2773402356 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32e80c9950 0x7f32e8148950 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.938+0000 7f32f0727700 1 --2- 172.21.15.28:0/2773402356 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32e815c880 0x7f32e8159fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.938+0000 7f32f0727700 1 --2- 172.21.15.28:0/2773402356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32e815c4b0 0x7f32e80ad2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:45.941 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.938+0000 7f32f0727700 1 -- 172.21.15.28:0/2773402356 >> 172.21.15.28:0/2773402356 conn(0x7f32e80b0df0 msgr2=0x7f32e815b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:45.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.939+0000 7f32f0727700 1 -- 172.21.15.28:0/2773402356 shutdown_connections 2023-11-28T11:11:45.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:45.939+0000 7f32f0727700 1 -- 172.21.15.28:0/2773402356 wait complete. 2023-11-28T11:11:45.952 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:11:46.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.078+0000 7eff79dfe700 1 Processor -- start 2023-11-28T11:11:46.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.078+0000 7eff79dfe700 1 -- start start 2023-11-28T11:11:46.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.079+0000 7eff79dfe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff740b62d0 0x7eff740cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.079+0000 7eff79dfe700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff740cd2a0 con 0x7eff7406f8b0 2023-11-28T11:11:46.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.079+0000 7eff79dfe700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff740d0400 con 0x7eff740b2d90 2023-11-28T11:11:46.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.079+0000 7eff79dfe700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff740d06e0 con 0x7eff740b62d0 2023-11-28T11:11:46.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.079+0000 7eff72ffd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7eff7406f8b0 0x7eff740b2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39042/0 (socket says 172.21.15.28:39042) 2023-11-28T11:11:46.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.079+0000 7eff72ffd700 1 -- 172.21.15.28:0/3537195837 learned_addr learned my addr 172.21.15.28:0/3537195837 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:46.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.079+0000 7eff737fe700 1 --2- 172.21.15.28:0/3537195837 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff740b62d0 0x7eff740cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.079+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 920622933 0 0) 0x7eff740cd2a0 con 0x7eff7406f8b0 2023-11-28T11:11:46.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff600035b0 con 0x7eff7406f8b0 2023-11-28T11:11:46.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133071423 0 0) 0x7eff740d0400 con 0x7eff740b2d90 2023-11-28T11:11:46.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff740cd2a0 con 0x7eff740b2d90 2023-11-28T11:11:46.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2177226184 0 0) 0x7eff600035b0 con 0x7eff7406f8b0 2023-11-28T11:11:46.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff740b62d0 msgr2=0x7eff740cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:46.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 --2- 172.21.15.28:0/3537195837 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff740b62d0 0x7eff740cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 >> v1:172.21.15.59:6789/0 conn(0x7eff740b2d90 legacy=0x7eff740ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff740cffb0 con 0x7eff7406f8b0 2023-11-28T11:11:46.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7eff5c001670 con 0x7eff7406f8b0 2023-11-28T11:11:46.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 >> v1:172.21.15.28:6789/0 conn(0x7eff7406f8b0 legacy=0x7eff740b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 --2- 172.21.15.28:0/3537195837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff60003bc0 0x7eff60003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 --2- 172.21.15.28:0/3537195837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff60004850 0x7eff60004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 --2- 172.21.15.28:0/3537195837 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff740b62d0 0x7eff60005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff740d06e0 con 0x7eff60004850 2023-11-28T11:11:46.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff6000bac0 con 0x7eff60003bc0 2023-11-28T11:11:46.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6000bc30 con 0x7eff740b62d0 2023-11-28T11:11:46.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff72ffd700 1 --2- 172.21.15.28:0/3537195837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff60004850 0x7eff60004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff737fe700 1 --2- 172.21.15.28:0/3537195837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff60003bc0 0x7eff60003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff73fff700 1 --2- 172.21.15.28:0/3537195837 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff740b62d0 0x7eff60005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff72ffd700 1 -- 172.21.15.28:0/3537195837 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff740b62d0 msgr2=0x7eff60005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:46.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff72ffd700 1 --2- 172.21.15.28:0/3537195837 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff740b62d0 0x7eff60005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff72ffd700 1 -- 172.21.15.28:0/3537195837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff60003bc0 msgr2=0x7eff60003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:46.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff72ffd700 1 --2- 172.21.15.28:0/3537195837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff60003bc0 0x7eff60003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.080+0000 7eff72ffd700 1 -- 172.21.15.28:0/3537195837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff600035b0 con 0x7eff60004850 2023-11-28T11:11:46.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.081+0000 7eff72ffd700 1 --2- 172.21.15.28:0/3537195837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff60004850 0x7eff60004cc0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00c240 tx=0x7eff60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4ec43f633158f87 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:46.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.081+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c00cd80 con 0x7eff60004850 2023-11-28T11:11:46.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.081+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c005960 con 0x7eff60004850 2023-11-28T11:11:46.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.081+0000 7eff727fc700 1 -- 172.21.15.28:0/3537195837 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c005d10 con 0x7eff60004850 2023-11-28T11:11:46.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.081+0000 7eff79dfe700 1 -- 172.21.15.28:0/3537195837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff60004850 msgr2=0x7eff60004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.081+0000 7eff79dfe700 1 --2- 172.21.15.28:0/3537195837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff60004850 0x7eff60004cc0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7eff5c00c240 tx=0x7eff60005880 comp rx=0 tx=0).stop 2023-11-28T11:11:46.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.081+0000 7eff73fff700 1 -- 172.21.15.28:0/3537195837 reap_dead start 2023-11-28T11:11:46.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.081+0000 7eff79dfe700 1 -- 172.21.15.28:0/3537195837 shutdown_connections 2023-11-28T11:11:46.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.081+0000 7eff79dfe700 1 -- 172.21.15.28:0/3537195837 >> 172.21.15.28:0/3537195837 conn(0x7eff740cd680 msgr2=0x7eff740cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:46.091 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.081+0000 7eff79dfe700 1 -- 172.21.15.28:0/3537195837 shutdown_connections 2023-11-28T11:11:46.091 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.081+0000 7eff79dfe700 1 -- 172.21.15.28:0/3537195837 wait complete. 2023-11-28T11:11:46.091 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.082+0000 7eff79dfe700 1 Processor -- start 2023-11-28T11:11:46.091 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.083+0000 7eff79dfe700 1 -- start start 2023-11-28T11:11:46.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.083+0000 7eff79dfe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff60004850 0x7eff7415d7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.083+0000 7eff79dfe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff74157ee0 0x7eff740a7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.083+0000 7eff79dfe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff740a84b0 0x7eff7414b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.083+0000 7eff79dfe700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7415dd30 con 0x7eff74157ee0 2023-11-28T11:11:46.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.083+0000 7eff79dfe700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff74157710 con 0x7eff60004850 2023-11-28T11:11:46.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.083+0000 7eff79dfe700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7415c4b0 con 0x7eff740a84b0 2023-11-28T11:11:46.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff72ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff74157ee0 0x7eff740a7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff72ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff74157ee0 0x7eff740a7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:50996/0 (socket says 172.21.15.28:50996) 2023-11-28T11:11:46.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff72ffd700 1 -- 172.21.15.28:0/3376068902 learned_addr learned my addr 172.21.15.28:0/3376068902 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:46.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff737fe700 1 --2- 172.21.15.28:0/3376068902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff60004850 0x7eff7415d7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff73fff700 1 --2- 172.21.15.28:0/3376068902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff740a84b0 0x7eff7414b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff72ffd700 1 -- 172.21.15.28:0/3376068902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff740a84b0 msgr2=0x7eff7414b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff72ffd700 1 --2- 172.21.15.28:0/3376068902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff740a84b0 0x7eff7414b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff72ffd700 1 -- 172.21.15.28:0/3376068902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff60004850 msgr2=0x7eff7415d7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff72ffd700 1 --2- 172.21.15.28:0/3376068902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff60004850 0x7eff7415d7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff72ffd700 1 -- 172.21.15.28:0/3376068902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff7406fd20 con 0x7eff74157ee0 2023-11-28T11:11:46.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff73fff700 1 --2- 172.21.15.28:0/3376068902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff740a84b0 0x7eff7414b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:46.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff72ffd700 1 --2- 172.21.15.28:0/3376068902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff74157ee0 0x7eff740a7f70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7eff60004780 tx=0x7eff740d09c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:46.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff6bfff700 1 -- 172.21.15.28:0/3376068902 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c004870 con 0x7eff74157ee0 2023-11-28T11:11:46.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff6bfff700 1 -- 172.21.15.28:0/3376068902 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5c0049f0 con 0x7eff74157ee0 2023-11-28T11:11:46.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff6bfff700 1 -- 172.21.15.28:0/3376068902 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5c004da0 con 0x7eff74157ee0 2023-11-28T11:11:46.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff79dfe700 1 -- 172.21.15.28:0/3376068902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff740b66a0 con 0x7eff74157ee0 2023-11-28T11:11:46.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.084+0000 7eff79dfe700 1 -- 172.21.15.28:0/3376068902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff740cce50 con 0x7eff74157ee0 2023-11-28T11:11:46.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.085+0000 7eff79dfe700 1 -- 172.21.15.28:0/3376068902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff7415d640 con 0x7eff74157ee0 2023-11-28T11:11:46.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.086+0000 7eff6bfff700 1 -- 172.21.15.28:0/3376068902 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7eff5c02e070 con 0x7eff74157ee0 2023-11-28T11:11:46.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.086+0000 7eff6bfff700 1 --2- 172.21.15.28:0/3376068902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7eff540682f0 0x7eff5406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.086+0000 7eff6bfff700 1 -- 172.21.15.28:0/3376068902 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7eff5c0871e0 con 0x7eff74157ee0 2023-11-28T11:11:46.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.086+0000 7eff737fe700 1 --2- 172.21.15.28:0/3376068902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7eff540682f0 0x7eff5406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.086+0000 7eff737fe700 1 --2- 172.21.15.28:0/3376068902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7eff540682f0 0x7eff5406a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7eff64014370 tx=0x7eff6400b040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:46.098 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.090+0000 7eff6bfff700 1 -- 172.21.15.28:0/3376068902 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7eff5c053880 con 0x7eff74157ee0 2023-11-28T11:11:46.288 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.286+0000 7eff79dfe700 1 -- 172.21.15.28:0/3376068902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7eff740739f0 con 0x7eff74157ee0 2023-11-28T11:11:46.288 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.287+0000 7eff6bfff700 1 -- 172.21.15.28:0/3376068902 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7eff5c056ea0 con 0x7eff74157ee0 2023-11-28T11:11:46.289 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:11:46.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.289+0000 7eff79dfe700 1 -- 172.21.15.28:0/3376068902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7eff540682f0 msgr2=0x7eff5406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.289+0000 7eff79dfe700 1 --2- 172.21.15.28:0/3376068902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7eff540682f0 0x7eff5406a7a0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7eff64014370 tx=0x7eff6400b040 comp rx=0 tx=0).stop 2023-11-28T11:11:46.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.289+0000 7eff79dfe700 1 -- 172.21.15.28:0/3376068902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff74157ee0 msgr2=0x7eff740a7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.289+0000 7eff79dfe700 1 --2- 172.21.15.28:0/3376068902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff74157ee0 0x7eff740a7f70 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7eff60004780 tx=0x7eff740d09c0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.289+0000 7eff79dfe700 1 -- 172.21.15.28:0/3376068902 shutdown_connections 2023-11-28T11:11:46.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.289+0000 7eff79dfe700 1 --2- 172.21.15.28:0/3376068902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7eff540682f0 0x7eff5406a7a0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.289+0000 7eff79dfe700 1 --2- 172.21.15.28:0/3376068902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7eff740a84b0 0x7eff7414b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.289+0000 7eff79dfe700 1 --2- 172.21.15.28:0/3376068902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff60004850 0x7eff7415d7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.289+0000 7eff79dfe700 1 --2- 172.21.15.28:0/3376068902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7eff74157ee0 0x7eff740a7f70 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.289+0000 7eff79dfe700 1 -- 172.21.15.28:0/3376068902 >> 172.21.15.28:0/3376068902 conn(0x7eff740cd680 msgr2=0x7eff740cf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:46.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.289+0000 7eff79dfe700 1 -- 172.21.15.28:0/3376068902 shutdown_connections 2023-11-28T11:11:46.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.289+0000 7eff79dfe700 1 -- 172.21.15.28:0/3376068902 wait complete. 2023-11-28T11:11:46.303 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:11:46.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.428+0000 7fbf51152700 1 Processor -- start 2023-11-28T11:11:46.430 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.428+0000 7fbf51152700 1 -- start start 2023-11-28T11:11:46.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.429+0000 7fbf51152700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbf4c0b1d40 0x7fbf4c0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.429+0000 7fbf51152700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbf4c15dbf0 0x7fbf4c0ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.429+0000 7fbf51152700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf4c0ceff0 con 0x7fbf4c159500 2023-11-28T11:11:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.429+0000 7fbf51152700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf4c15e090 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.429+0000 7fbf51152700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0cede0 con 0x7fbf4c0b1d40 2023-11-28T11:11:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.429+0000 7fbf4b59e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fbf4c15dbf0 0x7fbf4c0ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.429+0000 7fbf4ad9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbf4c159500 0x7fbf4c0b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39060/0 (socket says 172.21.15.28:39060) 2023-11-28T11:11:46.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.429+0000 7fbf4ad9d700 1 -- 172.21.15.28:0/513678979 learned_addr learned my addr 172.21.15.28:0/513678979 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:46.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.430+0000 7fbf4a59c700 1 --2- 172.21.15.28:0/513678979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbf4c0b1d40 0x7fbf4c0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.430+0000 7fbf4b59e700 1 -- 172.21.15.28:0/513678979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbf4c0b1d40 msgr2=0x7fbf4c0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:46.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.430+0000 7fbf4b59e700 1 --2- 172.21.15.28:0/513678979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbf4c0b1d40 0x7fbf4c0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.430+0000 7fbf4b59e700 1 -- 172.21.15.28:0/513678979 >> v1:172.21.15.28:6789/0 conn(0x7fbf4c159500 legacy=0x7fbf4c0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.430+0000 7fbf4b59e700 1 -- 172.21.15.28:0/513678979 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0cf2d0 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.430+0000 7fbf49d9b700 1 -- 172.21.15.28:0/513678979 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 591989707 0 0) 0x7fbf4c0ceff0 con 0x7fbf4c159500 2023-11-28T11:11:46.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.430+0000 7fbf4a59c700 1 --2- 172.21.15.28:0/513678979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbf4c0b1d40 0x7fbf4c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:46.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.430+0000 7fbf4b59e700 1 --2- 172.21.15.28:0/513678979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbf4c15dbf0 0x7fbf4c0ce8a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009d90 tx=0x7fbf4400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4378710022e0744 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:46.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.430+0000 7fbf49d9b700 1 -- 172.21.15.28:0/513678979 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf4400e070 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.430+0000 7fbf49d9b700 1 -- 172.21.15.28:0/513678979 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf4400bbd0 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.430+0000 7fbf49d9b700 1 -- 172.21.15.28:0/513678979 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44012430 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.430+0000 7fbf51152700 1 -- 172.21.15.28:0/513678979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbf4c15dbf0 msgr2=0x7fbf4c0ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.431+0000 7fbf51152700 1 --2- 172.21.15.28:0/513678979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbf4c15dbf0 0x7fbf4c0ce8a0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fbf44009d90 tx=0x7fbf4400b330 comp rx=0 tx=0).stop 2023-11-28T11:11:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.431+0000 7fbf51152700 1 -- 172.21.15.28:0/513678979 shutdown_connections 2023-11-28T11:11:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.431+0000 7fbf51152700 1 --2- 172.21.15.28:0/513678979 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbf4c15dbf0 0x7fbf4c0ce8a0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.431+0000 7fbf51152700 1 --2- 172.21.15.28:0/513678979 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbf4c0b1d40 0x7fbf4c0cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.431+0000 7fbf51152700 1 -- 172.21.15.28:0/513678979 >> 172.21.15.28:0/513678979 conn(0x7fbf4c0cc630 msgr2=0x7fbf4c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.431+0000 7fbf51152700 1 -- 172.21.15.28:0/513678979 shutdown_connections 2023-11-28T11:11:46.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.431+0000 7fbf51152700 1 -- 172.21.15.28:0/513678979 wait complete. 2023-11-28T11:11:46.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.433+0000 7fbf51152700 1 Processor -- start 2023-11-28T11:11:46.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.433+0000 7fbf51152700 1 -- start start 2023-11-28T11:11:46.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf51152700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbf4c0b1d40 0x7fbf4c154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf51152700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf4c159500 0x7fbf4c149e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf51152700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbf4c15dbf0 0x7fbf4c0c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf51152700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0b54a0 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf51152700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c14a390 con 0x7fbf4c159500 2023-11-28T11:11:46.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf51152700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf4c0b4920 con 0x7fbf4c0b1d40 2023-11-28T11:11:46.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf4b59e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbf4c15dbf0 0x7fbf4c0c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf4b59e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbf4c15dbf0 0x7fbf4c0c6c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51012/0 (socket says 172.21.15.28:51012) 2023-11-28T11:11:46.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf4b59e700 1 -- 172.21.15.28:0/1404634902 learned_addr learned my addr 172.21.15.28:0/1404634902 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:46.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf4ad9d700 1 --2- 172.21.15.28:0/1404634902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbf4c0b1d40 0x7fbf4c154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf4a59c700 1 --2- 172.21.15.28:0/1404634902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf4c159500 0x7fbf4c149e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf4b59e700 1 -- 172.21.15.28:0/1404634902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbf4c0b1d40 msgr2=0x7fbf4c154580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf4b59e700 1 --2- 172.21.15.28:0/1404634902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbf4c0b1d40 0x7fbf4c154580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf4b59e700 1 -- 172.21.15.28:0/1404634902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf4c159500 msgr2=0x7fbf4c149e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf4b59e700 1 --2- 172.21.15.28:0/1404634902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf4c159500 0x7fbf4c149e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf4b59e700 1 -- 172.21.15.28:0/1404634902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf44009a70 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf4b59e700 1 --2- 172.21.15.28:0/1404634902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbf4c15dbf0 0x7fbf4c0c6c80 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c066a20 tx=0x7fbf44004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:46.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf437fe700 1 -- 172.21.15.28:0/1404634902 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44020070 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf437fe700 1 -- 172.21.15.28:0/1404634902 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf440098f0 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf51152700 1 -- 172.21.15.28:0/1404634902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf4c15d3f0 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.434+0000 7fbf437fe700 1 -- 172.21.15.28:0/1404634902 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf44003ea0 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.435+0000 7fbf51152700 1 -- 172.21.15.28:0/1404634902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf4c15d110 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.436+0000 7fbf51152700 1 -- 172.21.15.28:0/1404634902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf4c15d6d0 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.436+0000 7fbf437fe700 1 -- 172.21.15.28:0/1404634902 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbf44012bd0 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.436+0000 7fbf437fe700 1 --2- 172.21.15.28:0/1404634902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbf2c0682f0 0x7fbf2c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.436+0000 7fbf437fe700 1 -- 172.21.15.28:0/1404634902 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbf440890d0 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.436+0000 7fbf4ad9d700 1 --2- 172.21.15.28:0/1404634902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbf2c0682f0 0x7fbf2c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.437+0000 7fbf4ad9d700 1 --2- 172.21.15.28:0/1404634902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbf2c0682f0 0x7fbf2c06a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c000f60 tx=0x7fbf3c007e70 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:46.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.440+0000 7fbf437fe700 1 -- 172.21.15.28:0/1404634902 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbf440557f0 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.633 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.632+0000 7fbf51152700 1 -- 172.21.15.28:0/1404634902 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fbf4c0739f0 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.634 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.632+0000 7fbf437fe700 1 -- 172.21.15.28:0/1404634902 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7fbf44012e60 con 0x7fbf4c15dbf0 2023-11-28T11:11:46.634 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:11:46.635 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.634+0000 7fbf51152700 1 -- 172.21.15.28:0/1404634902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbf2c0682f0 msgr2=0x7fbf2c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.634+0000 7fbf51152700 1 --2- 172.21.15.28:0/1404634902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbf2c0682f0 0x7fbf2c06a7a0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c000f60 tx=0x7fbf3c007e70 comp rx=0 tx=0).stop 2023-11-28T11:11:46.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.634+0000 7fbf51152700 1 -- 172.21.15.28:0/1404634902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbf4c15dbf0 msgr2=0x7fbf4c0c6c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.634+0000 7fbf51152700 1 --2- 172.21.15.28:0/1404634902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbf4c15dbf0 0x7fbf4c0c6c80 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c066a20 tx=0x7fbf44004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.634+0000 7fbf51152700 1 -- 172.21.15.28:0/1404634902 shutdown_connections 2023-11-28T11:11:46.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.634+0000 7fbf51152700 1 --2- 172.21.15.28:0/1404634902 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbf2c0682f0 0x7fbf2c06a7a0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.634+0000 7fbf51152700 1 --2- 172.21.15.28:0/1404634902 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbf4c15dbf0 0x7fbf4c0c6c80 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.634+0000 7fbf51152700 1 --2- 172.21.15.28:0/1404634902 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbf4c0b1d40 0x7fbf4c154580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.636 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.634+0000 7fbf51152700 1 --2- 172.21.15.28:0/1404634902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbf4c159500 0x7fbf4c149e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.634+0000 7fbf51152700 1 -- 172.21.15.28:0/1404634902 >> 172.21.15.28:0/1404634902 conn(0x7fbf4c0cc630 msgr2=0x7fbf4c0b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:46.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.634+0000 7fbf51152700 1 -- 172.21.15.28:0/1404634902 shutdown_connections 2023-11-28T11:11:46.637 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.634+0000 7fbf51152700 1 -- 172.21.15.28:0/1404634902 wait complete. 2023-11-28T11:11:46.648 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:11:46.776 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.775+0000 7fbea0211700 1 Processor -- start 2023-11-28T11:11:46.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.775+0000 7fbea0211700 1 -- start start 2023-11-28T11:11:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.776+0000 7fbea0211700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fbe980cd140 0x7fbe9815dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.776+0000 7fbea0211700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe980ceff0 con 0x7fbe980b1d40 2023-11-28T11:11:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.776+0000 7fbea0211700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe980cf2d0 con 0x7fbe98159500 2023-11-28T11:11:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.776+0000 7fbea0211700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe9815d110 con 0x7fbe980cd140 2023-11-28T11:11:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.776+0000 7fbe9d7ac700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fbe980b1d40 0x7fbe9806bb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39076/0 (socket says 172.21.15.28:39076) 2023-11-28T11:11:46.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.776+0000 7fbe9d7ac700 1 -- 172.21.15.28:0/134048025 learned_addr learned my addr 172.21.15.28:0/134048025 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.776+0000 7fbe9e7ae700 1 --2- 172.21.15.28:0/134048025 >> v2:172.21.15.59:3300/0 conn(0x7fbe980cd140 0x7fbe9815dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.776+0000 7fbe9cfab700 1 -- 172.21.15.28:0/134048025 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2013478163 0 0) 0x7fbe980ceff0 con 0x7fbe980b1d40 2023-11-28T11:11:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.776+0000 7fbe9cfab700 1 -- 172.21.15.28:0/134048025 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbe840035b0 con 0x7fbe980b1d40 2023-11-28T11:11:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.776+0000 7fbe9cfab700 1 -- 172.21.15.28:0/134048025 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 44724412 0 0) 0x7fbe840035b0 con 0x7fbe980b1d40 2023-11-28T11:11:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.776+0000 7fbe9e7ae700 1 -- 172.21.15.28:0/134048025 >> v1:172.21.15.59:6789/0 conn(0x7fbe98159500 legacy=0x7fbe980b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.776+0000 7fbe9e7ae700 1 -- 172.21.15.28:0/134048025 >> v1:172.21.15.28:6789/0 conn(0x7fbe980b1d40 legacy=0x7fbe9806bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.777+0000 7fbe9e7ae700 1 -- 172.21.15.28:0/134048025 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe9815d3f0 con 0x7fbe980cd140 2023-11-28T11:11:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.777+0000 7fbe9e7ae700 1 --2- 172.21.15.28:0/134048025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe980cd140 0x7fbe9815dbf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009d90 tx=0x7fbe9000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e0923537bb828108 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.777+0000 7fbe9cfab700 1 -- 172.21.15.28:0/134048025 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9000e070 con 0x7fbe980cd140 2023-11-28T11:11:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.777+0000 7fbe9cfab700 1 -- 172.21.15.28:0/134048025 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe90003a20 con 0x7fbe980cd140 2023-11-28T11:11:46.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.777+0000 7fbe9cfab700 1 -- 172.21.15.28:0/134048025 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe94001670 con 0x7fbe980cd140 2023-11-28T11:11:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.777+0000 7fbea0211700 1 -- 172.21.15.28:0/134048025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe980cd140 msgr2=0x7fbe9815dbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.777+0000 7fbea0211700 1 --2- 172.21.15.28:0/134048025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe980cd140 0x7fbe9815dbf0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fbe90009d90 tx=0x7fbe9000b330 comp rx=0 tx=0).stop 2023-11-28T11:11:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.777+0000 7fbea0211700 1 -- 172.21.15.28:0/134048025 shutdown_connections 2023-11-28T11:11:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.777+0000 7fbea0211700 1 --2- 172.21.15.28:0/134048025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe980cd140 0x7fbe9815dbf0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.777+0000 7fbea0211700 1 -- 172.21.15.28:0/134048025 >> 172.21.15.28:0/134048025 conn(0x7fbe980cc630 msgr2=0x7fbe980cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.778+0000 7fbea0211700 1 -- 172.21.15.28:0/134048025 shutdown_connections 2023-11-28T11:11:46.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.778+0000 7fbea0211700 1 -- 172.21.15.28:0/134048025 wait complete. 2023-11-28T11:11:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.779+0000 7fbea0211700 1 Processor -- start 2023-11-28T11:11:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.779+0000 7fbea0211700 1 -- start start 2023-11-28T11:11:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.780+0000 7fbea0211700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe980b1d40 0x7fbe980b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.780+0000 7fbea0211700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe980cd140 0x7fbe980c9420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.780+0000 7fbea0211700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe98159500 0x7fbe980b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.780+0000 7fbea0211700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe981ef440 con 0x7fbe98159500 2023-11-28T11:11:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.780+0000 7fbea0211700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe981ef720 con 0x7fbe980cd140 2023-11-28T11:11:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.780+0000 7fbea0211700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe981efa00 con 0x7fbe980b1d40 2023-11-28T11:11:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.780+0000 7fbe9e7ae700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe98159500 0x7fbe980b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.780+0000 7fbe9e7ae700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe98159500 0x7fbe980b29e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51026/0 (socket says 172.21.15.28:51026) 2023-11-28T11:11:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.780+0000 7fbe9e7ae700 1 -- 172.21.15.28:0/685659260 learned_addr learned my addr 172.21.15.28:0/685659260 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.780+0000 7fbe9d7ac700 1 --2- 172.21.15.28:0/685659260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe980cd140 0x7fbe980c9420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.780+0000 7fbe9dfad700 1 --2- 172.21.15.28:0/685659260 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe980b1d40 0x7fbe980b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.780+0000 7fbe9e7ae700 1 -- 172.21.15.28:0/685659260 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe980b1d40 msgr2=0x7fbe980b4bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.780+0000 7fbe9e7ae700 1 --2- 172.21.15.28:0/685659260 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe980b1d40 0x7fbe980b4bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.781+0000 7fbe9e7ae700 1 -- 172.21.15.28:0/685659260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe980cd140 msgr2=0x7fbe980c9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.781+0000 7fbe9e7ae700 1 --2- 172.21.15.28:0/685659260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe980cd140 0x7fbe980c9420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.781+0000 7fbe9e7ae700 1 -- 172.21.15.28:0/685659260 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe90009a70 con 0x7fbe98159500 2023-11-28T11:11:46.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.781+0000 7fbe9e7ae700 1 --2- 172.21.15.28:0/685659260 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe98159500 0x7fbe980b29e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbe980cfb80 tx=0x7fbe90004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:46.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.781+0000 7fbe8e7fc700 1 -- 172.21.15.28:0/685659260 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe90004540 con 0x7fbe98159500 2023-11-28T11:11:46.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.781+0000 7fbe8e7fc700 1 -- 172.21.15.28:0/685659260 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe900046a0 con 0x7fbe98159500 2023-11-28T11:11:46.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.781+0000 7fbea0211700 1 -- 172.21.15.28:0/685659260 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe981598d0 con 0x7fbe98159500 2023-11-28T11:11:46.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.781+0000 7fbe8e7fc700 1 -- 172.21.15.28:0/685659260 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9001d470 con 0x7fbe98159500 2023-11-28T11:11:46.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.781+0000 7fbea0211700 1 -- 172.21.15.28:0/685659260 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9815d6d0 con 0x7fbe98159500 2023-11-28T11:11:46.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.782+0000 7fbea0211700 1 -- 172.21.15.28:0/685659260 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe98159a60 con 0x7fbe98159500 2023-11-28T11:11:46.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.782+0000 7fbe8e7fc700 1 -- 172.21.15.28:0/685659260 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fbe90018070 con 0x7fbe98159500 2023-11-28T11:11:46.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.783+0000 7fbe8e7fc700 1 --2- 172.21.15.28:0/685659260 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbe7c0682f0 0x7fbe7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:46.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.783+0000 7fbe8e7fc700 1 -- 172.21.15.28:0/685659260 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fbe900874c0 con 0x7fbe98159500 2023-11-28T11:11:46.786 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.783+0000 7fbe9dfad700 1 --2- 172.21.15.28:0/685659260 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbe7c0682f0 0x7fbe7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:46.787 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.783+0000 7fbe9dfad700 1 --2- 172.21.15.28:0/685659260 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbe7c0682f0 0x7fbe7c06a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbe9806bc20 tx=0x7fbe88009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:46.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.787+0000 7fbe8e7fc700 1 -- 172.21.15.28:0/685659260 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fbe90053be0 con 0x7fbe98159500 2023-11-28T11:11:46.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.978+0000 7fbea0211700 1 -- 172.21.15.28:0/685659260 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbe980739f0 con 0x7fbe98159500 2023-11-28T11:11:46.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.979+0000 7fbe8e7fc700 1 -- 172.21.15.28:0/685659260 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbe90057200 con 0x7fbe98159500 2023-11-28T11:11:46.981 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:11:46.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.981+0000 7fbea0211700 1 -- 172.21.15.28:0/685659260 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbe7c0682f0 msgr2=0x7fbe7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.981+0000 7fbea0211700 1 --2- 172.21.15.28:0/685659260 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbe7c0682f0 0x7fbe7c06a7a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbe9806bc20 tx=0x7fbe88009040 comp rx=0 tx=0).stop 2023-11-28T11:11:46.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.981+0000 7fbea0211700 1 -- 172.21.15.28:0/685659260 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe98159500 msgr2=0x7fbe980b29e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:46.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.981+0000 7fbea0211700 1 --2- 172.21.15.28:0/685659260 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe98159500 0x7fbe980b29e0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fbe980cfb80 tx=0x7fbe90004ad0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.981+0000 7fbea0211700 1 -- 172.21.15.28:0/685659260 shutdown_connections 2023-11-28T11:11:46.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.981+0000 7fbea0211700 1 --2- 172.21.15.28:0/685659260 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fbe7c0682f0 0x7fbe7c06a7a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.981+0000 7fbea0211700 1 --2- 172.21.15.28:0/685659260 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fbe98159500 0x7fbe980b29e0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.981+0000 7fbea0211700 1 --2- 172.21.15.28:0/685659260 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fbe980b1d40 0x7fbe980b4bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.981+0000 7fbea0211700 1 --2- 172.21.15.28:0/685659260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe980cd140 0x7fbe980c9420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:46.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.981+0000 7fbea0211700 1 -- 172.21.15.28:0/685659260 >> 172.21.15.28:0/685659260 conn(0x7fbe980cc630 msgr2=0x7fbe9806c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:46.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.981+0000 7fbea0211700 1 -- 172.21.15.28:0/685659260 shutdown_connections 2023-11-28T11:11:46.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:46.981+0000 7fbea0211700 1 -- 172.21.15.28:0/685659260 wait complete. 2023-11-28T11:11:46.995 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:11:47.122 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.121+0000 7f65078ea700 1 Processor -- start 2023-11-28T11:11:47.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.121+0000 7f65078ea700 1 -- start start 2023-11-28T11:11:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.122+0000 7f65078ea700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f650015c990 0x7f65000cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.122+0000 7f65078ea700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65000ce340 con 0x7f65000cc310 2023-11-28T11:11:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.122+0000 7f65078ea700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65000cebe0 con 0x7f6500161090 2023-11-28T11:11:47.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.122+0000 7f65078ea700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65000cb3f0 con 0x7f650015c990 2023-11-28T11:11:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.122+0000 7f6505686700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f650015c990 0x7f65000cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.122+0000 7f6505e87700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f65000cc310 0x7f65000cc770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39092/0 (socket says 172.21.15.28:39092) 2023-11-28T11:11:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.122+0000 7f6505e87700 1 -- 172.21.15.28:0/3468931014 learned_addr learned my addr 172.21.15.28:0/3468931014 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.122+0000 7f6505686700 1 -- 172.21.15.28:0/3468931014 >> v1:172.21.15.59:6789/0 conn(0x7f6500161090 legacy=0x7f65000ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.122+0000 7f6505686700 1 -- 172.21.15.28:0/3468931014 >> v1:172.21.15.28:6789/0 conn(0x7f65000cc310 legacy=0x7f65000cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.122+0000 7f6505686700 1 -- 172.21.15.28:0/3468931014 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65000ce590 con 0x7f650015c990 2023-11-28T11:11:47.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.122+0000 7f64f7fff700 1 -- 172.21.15.28:0/3468931014 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027635583 0 0) 0x7f65000ce340 con 0x7f65000cc310 2023-11-28T11:11:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.122+0000 7f6505686700 1 --2- 172.21.15.28:0/3468931014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f650015c990 0x7f65000cf070 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009840 tx=0x7f64fc009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4119db0f8c938a5b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.122+0000 7f64f7fff700 1 -- 172.21.15.28:0/3468931014 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc011830 con 0x7f650015c990 2023-11-28T11:11:47.127 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.122+0000 7f64f7fff700 1 -- 172.21.15.28:0/3468931014 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc011990 con 0x7f650015c990 2023-11-28T11:11:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.122+0000 7f64f7fff700 1 -- 172.21.15.28:0/3468931014 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc011d20 con 0x7f650015c990 2023-11-28T11:11:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.123+0000 7f65078ea700 1 -- 172.21.15.28:0/3468931014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f650015c990 msgr2=0x7f65000cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.123+0000 7f65078ea700 1 --2- 172.21.15.28:0/3468931014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f650015c990 0x7f65000cf070 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f64fc009840 tx=0x7f64fc009b50 comp rx=0 tx=0).stop 2023-11-28T11:11:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.123+0000 7f65078ea700 1 -- 172.21.15.28:0/3468931014 shutdown_connections 2023-11-28T11:11:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.123+0000 7f65078ea700 1 --2- 172.21.15.28:0/3468931014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f650015c990 0x7f65000cf070 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.123+0000 7f65078ea700 1 -- 172.21.15.28:0/3468931014 >> 172.21.15.28:0/3468931014 conn(0x7f65000b1650 msgr2=0x7f65000b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.123+0000 7f65078ea700 1 -- 172.21.15.28:0/3468931014 shutdown_connections 2023-11-28T11:11:47.128 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.123+0000 7f65078ea700 1 -- 172.21.15.28:0/3468931014 wait complete. 2023-11-28T11:11:47.131 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.124+0000 7f65078ea700 1 Processor -- start 2023-11-28T11:11:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.124+0000 7f65078ea700 1 -- start start 2023-11-28T11:11:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f65078ea700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65000cc310 0x7f65000ac0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f65078ea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f650015c990 0x7f650006c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f65078ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6500161090 0x7f650006cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f65078ea700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65000c95e0 con 0x7f65000cc310 2023-11-28T11:11:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f65078ea700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65000b5a00 con 0x7f6500161090 2023-11-28T11:11:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f65078ea700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65000b5440 con 0x7f650015c990 2023-11-28T11:11:47.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f6505686700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65000cc310 0x7f65000ac0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f6505686700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65000cc310 0x7f65000ac0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51036/0 (socket says 172.21.15.28:51036) 2023-11-28T11:11:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f6505686700 1 -- 172.21.15.28:0/2135892993 learned_addr learned my addr 172.21.15.28:0/2135892993 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f6505686700 1 -- 172.21.15.28:0/2135892993 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f650015c990 msgr2=0x7f650006c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f6504e85700 1 --2- 172.21.15.28:0/2135892993 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f650015c990 0x7f650006c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f6505e87700 1 --2- 172.21.15.28:0/2135892993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6500161090 0x7f650006cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f6505686700 1 --2- 172.21.15.28:0/2135892993 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f650015c990 0x7f650006c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f6505686700 1 -- 172.21.15.28:0/2135892993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6500161090 msgr2=0x7f650006cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f6505686700 1 --2- 172.21.15.28:0/2135892993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6500161090 0x7f650006cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.125+0000 7f6505686700 1 -- 172.21.15.28:0/2135892993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64fc0094a0 con 0x7f65000cc310 2023-11-28T11:11:47.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.126+0000 7f6505686700 1 --2- 172.21.15.28:0/2135892993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65000cc310 0x7f65000ac0b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f64fc014040 tx=0x7f64fc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:47.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.126+0000 7f64f5ffb700 1 -- 172.21.15.28:0/2135892993 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc01a900 con 0x7f65000cc310 2023-11-28T11:11:47.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.126+0000 7f64f5ffb700 1 -- 172.21.15.28:0/2135892993 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc01aa60 con 0x7f65000cc310 2023-11-28T11:11:47.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.126+0000 7f65078ea700 1 -- 172.21.15.28:0/2135892993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65000cb080 con 0x7f65000cc310 2023-11-28T11:11:47.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.126+0000 7f64f5ffb700 1 -- 172.21.15.28:0/2135892993 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc01add0 con 0x7f65000cc310 2023-11-28T11:11:47.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.126+0000 7f65078ea700 1 -- 172.21.15.28:0/2135892993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65000ce870 con 0x7f65000cc310 2023-11-28T11:11:47.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.127+0000 7f65078ea700 1 -- 172.21.15.28:0/2135892993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65000cf5b0 con 0x7f65000cc310 2023-11-28T11:11:47.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.127+0000 7f64f5ffb700 1 -- 172.21.15.28:0/2135892993 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f64fc02b030 con 0x7f65000cc310 2023-11-28T11:11:47.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.128+0000 7f64f5ffb700 1 --2- 172.21.15.28:0/2135892993 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f64e40682f0 0x7f64e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.128+0000 7f64f5ffb700 1 -- 172.21.15.28:0/2135892993 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f64fc0870f0 con 0x7f65000cc310 2023-11-28T11:11:47.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.128+0000 7f6504e85700 1 --2- 172.21.15.28:0/2135892993 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f64e40682f0 0x7f64e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.128+0000 7f6504e85700 1 --2- 172.21.15.28:0/2135892993 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f64e40682f0 0x7f64e406a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f650006d400 tx=0x7f64f0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:47.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.132+0000 7f64f5ffb700 1 -- 172.21.15.28:0/2135892993 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f64fc0538c0 con 0x7f65000cc310 2023-11-28T11:11:47.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.343+0000 7f65078ea700 1 -- 172.21.15.28:0/2135892993 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f65000675a0 con 0x7f65000cc310 2023-11-28T11:11:47.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.343+0000 7f64f5ffb700 1 -- 172.21.15.28:0/2135892993 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v54) v1 ==== 80+0+0 (secure 0 0 0) 0x7f64fc056ee0 con 0x7f65000cc310 2023-11-28T11:11:47.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.346+0000 7f65078ea700 1 -- 172.21.15.28:0/2135892993 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f64e40682f0 msgr2=0x7f64e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.346+0000 7f65078ea700 1 --2- 172.21.15.28:0/2135892993 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f64e40682f0 0x7f64e406a7a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f650006d400 tx=0x7f64f0009040 comp rx=0 tx=0).stop 2023-11-28T11:11:47.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.346+0000 7f65078ea700 1 -- 172.21.15.28:0/2135892993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65000cc310 msgr2=0x7f65000ac0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.346+0000 7f65078ea700 1 --2- 172.21.15.28:0/2135892993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65000cc310 0x7f65000ac0b0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f64fc014040 tx=0x7f64fc004750 comp rx=0 tx=0).stop 2023-11-28T11:11:47.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.346+0000 7f65078ea700 1 -- 172.21.15.28:0/2135892993 shutdown_connections 2023-11-28T11:11:47.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.346+0000 7f65078ea700 1 --2- 172.21.15.28:0/2135892993 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f64e40682f0 0x7f64e406a7a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.346+0000 7f65078ea700 1 --2- 172.21.15.28:0/2135892993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f65000cc310 0x7f65000ac0b0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.346+0000 7f65078ea700 1 --2- 172.21.15.28:0/2135892993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6500161090 0x7f650006cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.346+0000 7f65078ea700 1 --2- 172.21.15.28:0/2135892993 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f650015c990 0x7f650006c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.346+0000 7f65078ea700 1 -- 172.21.15.28:0/2135892993 >> 172.21.15.28:0/2135892993 conn(0x7f65000b1650 msgr2=0x7f65000b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:47.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.346+0000 7f65078ea700 1 -- 172.21.15.28:0/2135892993 shutdown_connections 2023-11-28T11:11:47.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.346+0000 7f65078ea700 1 -- 172.21.15.28:0/2135892993 wait complete. 2023-11-28T11:11:47.349 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2023-11-28T11:11:47.359 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi028.front.sepia.ceph.com... 2023-11-28T11:11:47.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:47.360 DEBUG:teuthology.orchestra.run.smithi028:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T11:11:47.396 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[125845]: fuse finished with error 0 and tester_r 0 2023-11-28T11:11:47.397 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:11:47.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:47.398 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:11:47.473 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:11:47.473 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:47.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3961 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:11:47.493 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:11:47.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:47.494 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T11:11:47.551 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:11:47.552 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T11:11:47.552 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:11:47.552 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:47.570 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:11:47.571 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:11:47.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:47.571 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T11:11:47.627 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:11:47.627 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T11:11:47.628 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:11:47.628 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:47.648 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3961 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:11:47.648 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:11:47.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:11:47.648 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T11:11:47.707 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:11:47.707 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T11:11:47.707 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:47.834 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.832+0000 7f5858ff9700 1 Processor -- start 2023-11-28T11:11:47.834 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.832+0000 7f5858ff9700 1 -- start start 2023-11-28T11:11:47.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.833+0000 7f5858ff9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f585415aa20 0x7f58540b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.833+0000 7f5858ff9700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58540ca990 con 0x7f58540b5500 2023-11-28T11:11:47.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.833+0000 7f5858ff9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58540caf50 con 0x7f585415a650 2023-11-28T11:11:47.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.833+0000 7f5858ff9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58540aef10 con 0x7f585415aa20 2023-11-28T11:11:47.836 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.833+0000 7f5852d9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f58540b5500 0x7f58540cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39106/0 (socket says 172.21.15.28:39106) 2023-11-28T11:11:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.833+0000 7f5852d9d700 1 -- 172.21.15.28:0/1332137544 learned_addr learned my addr 172.21.15.28:0/1332137544 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.833+0000 7f5851d9b700 1 --2- 172.21.15.28:0/1332137544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f585415aa20 0x7f58540b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.833+0000 7f585159a700 1 -- 172.21.15.28:0/1332137544 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 398864961 0 0) 0x7f58540ca990 con 0x7f58540b5500 2023-11-28T11:11:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.833+0000 7f585159a700 1 -- 172.21.15.28:0/1332137544 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f583c0035b0 con 0x7f58540b5500 2023-11-28T11:11:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.834+0000 7f585159a700 1 -- 172.21.15.28:0/1332137544 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2964603185 0 0) 0x7f58540caf50 con 0x7f585415a650 2023-11-28T11:11:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.834+0000 7f585159a700 1 -- 172.21.15.28:0/1332137544 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58540ca990 con 0x7f585415a650 2023-11-28T11:11:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.834+0000 7f585159a700 1 -- 172.21.15.28:0/1332137544 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1508317952 0 0) 0x7f583c0035b0 con 0x7f58540b5500 2023-11-28T11:11:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.834+0000 7f5851d9b700 1 -- 172.21.15.28:0/1332137544 >> v1:172.21.15.59:6789/0 conn(0x7f585415a650 legacy=0x7f58540ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.834+0000 7f5851d9b700 1 -- 172.21.15.28:0/1332137544 >> v1:172.21.15.28:6789/0 conn(0x7f58540b5500 legacy=0x7f58540cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.834+0000 7f5851d9b700 1 -- 172.21.15.28:0/1332137544 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58540af1f0 con 0x7f585415aa20 2023-11-28T11:11:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.834+0000 7f5851d9b700 1 --2- 172.21.15.28:0/1332137544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f585415aa20 0x7f58540b2950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5844009870 tx=0x7f5844009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24590d2a137af4b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:47.837 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.834+0000 7f585159a700 1 -- 172.21.15.28:0/1332137544 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584c0019f0 con 0x7f585415aa20 2023-11-28T11:11:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.834+0000 7f585159a700 1 -- 172.21.15.28:0/1332137544 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5844004030 con 0x7f585415aa20 2023-11-28T11:11:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.834+0000 7f585159a700 1 -- 172.21.15.28:0/1332137544 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58440043c0 con 0x7f585415aa20 2023-11-28T11:11:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.835+0000 7f5858ff9700 1 -- 172.21.15.28:0/1332137544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f585415aa20 msgr2=0x7f58540b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.835+0000 7f5858ff9700 1 --2- 172.21.15.28:0/1332137544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f585415aa20 0x7f58540b2950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5844009870 tx=0x7f5844009b80 comp rx=0 tx=0).stop 2023-11-28T11:11:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.835+0000 7f5858ff9700 1 -- 172.21.15.28:0/1332137544 shutdown_connections 2023-11-28T11:11:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.835+0000 7f5858ff9700 1 --2- 172.21.15.28:0/1332137544 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f585415aa20 0x7f58540b2950 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.835+0000 7f5858ff9700 1 -- 172.21.15.28:0/1332137544 >> 172.21.15.28:0/1332137544 conn(0x7f58540b1650 msgr2=0x7f58540b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.835+0000 7f5858ff9700 1 -- 172.21.15.28:0/1332137544 shutdown_connections 2023-11-28T11:11:47.838 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.835+0000 7f5858ff9700 1 -- 172.21.15.28:0/1332137544 wait complete. 2023-11-28T11:11:47.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.836+0000 7f5858ff9700 1 Processor -- start 2023-11-28T11:11:47.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.836+0000 7f5858ff9700 1 -- start start 2023-11-28T11:11:47.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f5858ff9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58540b5500 0x7f58540c5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f5858ff9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f585415a650 0x7f58540c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f5858ff9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f585415aa20 0x7f585415bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f5858ff9700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585415b890 con 0x7f58540b5500 2023-11-28T11:11:47.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f5858ff9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58540c6390 con 0x7f585415aa20 2023-11-28T11:11:47.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f5858ff9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585415aef0 con 0x7f585415a650 2023-11-28T11:11:47.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f585259c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58540b5500 0x7f58540c5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f585259c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58540b5500 0x7f58540c5310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51050/0 (socket says 172.21.15.28:51050) 2023-11-28T11:11:47.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f585259c700 1 -- 172.21.15.28:0/946055379 learned_addr learned my addr 172.21.15.28:0/946055379 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:47.842 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f5852d9d700 1 --2- 172.21.15.28:0/946055379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f585415aa20 0x7f585415bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.842 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f5851d9b700 1 --2- 172.21.15.28:0/946055379 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f585415a650 0x7f58540c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.842 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f585259c700 1 -- 172.21.15.28:0/946055379 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f585415a650 msgr2=0x7f58540c35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.842 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f585259c700 1 --2- 172.21.15.28:0/946055379 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f585415a650 0x7f58540c35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.842 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f585259c700 1 -- 172.21.15.28:0/946055379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f585415aa20 msgr2=0x7f585415bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:47.842 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f585259c700 1 --2- 172.21.15.28:0/946055379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f585415aa20 0x7f585415bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:47.842 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f585259c700 1 -- 172.21.15.28:0/946055379 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58440094a0 con 0x7f58540b5500 2023-11-28T11:11:47.842 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.837+0000 7f585259c700 1 --2- 172.21.15.28:0/946055379 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58540b5500 0x7f58540c5310 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f584800c550 tx=0x7f584800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:47.842 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.838+0000 7f5842ffd700 1 -- 172.21.15.28:0/946055379 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58480052d0 con 0x7f58540b5500 2023-11-28T11:11:47.842 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.838+0000 7f5858ff9700 1 -- 172.21.15.28:0/946055379 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58540ccbc0 con 0x7f58540b5500 2023-11-28T11:11:47.842 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.838+0000 7f5842ffd700 1 -- 172.21.15.28:0/946055379 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5848005430 con 0x7f58540b5500 2023-11-28T11:11:47.843 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.838+0000 7f5842ffd700 1 -- 172.21.15.28:0/946055379 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58480038c0 con 0x7f58540b5500 2023-11-28T11:11:47.843 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.838+0000 7f5858ff9700 1 -- 172.21.15.28:0/946055379 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58540cabe0 con 0x7f58540b5500 2023-11-28T11:11:47.843 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.838+0000 7f5858ff9700 1 -- 172.21.15.28:0/946055379 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58540ccd50 con 0x7f58540b5500 2023-11-28T11:11:47.843 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.839+0000 7f5842ffd700 1 -- 172.21.15.28:0/946055379 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5848019030 con 0x7f58540b5500 2023-11-28T11:11:47.843 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.840+0000 7f5842ffd700 1 --2- 172.21.15.28:0/946055379 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f58340682f0 0x7f583406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:47.844 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.840+0000 7f5842ffd700 1 -- 172.21.15.28:0/946055379 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5848085650 con 0x7f58540b5500 2023-11-28T11:11:47.844 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.840+0000 7f5851d9b700 1 --2- 172.21.15.28:0/946055379 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f58340682f0 0x7f583406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:47.844 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.840+0000 7f5851d9b700 1 --2- 172.21.15.28:0/946055379 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f58340682f0 0x7f583406a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f58440097c0 tx=0x7f5844008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:47.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:47.843+0000 7f5842ffd700 1 -- 172.21.15.28:0/946055379 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5848051c80 con 0x7f58540b5500 2023-11-28T11:11:48.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.082+0000 7f5858ff9700 1 -- 172.21.15.28:0/946055379 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f58540675a0 con 0x7f58540b5500 2023-11-28T11:11:48.084 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.082+0000 7f5842ffd700 1 -- 172.21.15.28:0/946055379 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f58480552a0 con 0x7f58540b5500 2023-11-28T11:11:48.084 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:48.084 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5198,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/284515388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":284515388},{"type":"v1","addr":"172.21.15.59:6835","nonce":284515388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2160385010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2160385010},{"type":"v1","addr":"172.21.15.28:6836","nonce":2160385010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3775145347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3775145347},{"type":"v1","addr":"172.21.15.28:6837","nonce":3775145347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:59.771956+0000","modified":"2023-11-28T11:11:03.109382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6834/3261856585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3261856585},{"type":"v1","addr":"172.21.15.59:6834","nonce":3261856585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:11:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.084+0000 7f5858ff9700 1 -- 172.21.15.28:0/946055379 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f58340682f0 msgr2=0x7f583406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.084+0000 7f5858ff9700 1 --2- 172.21.15.28:0/946055379 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f58340682f0 0x7f583406a7a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f58440097c0 tx=0x7f5844008030 comp rx=0 tx=0).stop 2023-11-28T11:11:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.084+0000 7f5858ff9700 1 -- 172.21.15.28:0/946055379 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58540b5500 msgr2=0x7f58540c5310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.084+0000 7f5858ff9700 1 --2- 172.21.15.28:0/946055379 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58540b5500 0x7f58540c5310 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f584800c550 tx=0x7f584800c860 comp rx=0 tx=0).stop 2023-11-28T11:11:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.084+0000 7f5858ff9700 1 -- 172.21.15.28:0/946055379 shutdown_connections 2023-11-28T11:11:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.084+0000 7f5858ff9700 1 --2- 172.21.15.28:0/946055379 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f58340682f0 0x7f583406a7a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.084+0000 7f5858ff9700 1 --2- 172.21.15.28:0/946055379 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f58540b5500 0x7f58540c5310 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.084+0000 7f5858ff9700 1 --2- 172.21.15.28:0/946055379 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f585415aa20 0x7f585415bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.084+0000 7f5858ff9700 1 --2- 172.21.15.28:0/946055379 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f585415a650 0x7f58540c35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.084+0000 7f5858ff9700 1 -- 172.21.15.28:0/946055379 >> 172.21.15.28:0/946055379 conn(0x7f58540b1650 msgr2=0x7f58540cf990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.084+0000 7f5858ff9700 1 -- 172.21.15.28:0/946055379 shutdown_connections 2023-11-28T11:11:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.084+0000 7f5858ff9700 1 -- 172.21.15.28:0/946055379 wait complete. 2023-11-28T11:11:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 25 2023-11-28T11:11:48.098 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:11:48.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.222+0000 7f16a6c6d700 1 Processor -- start 2023-11-28T11:11:48.223 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.222+0000 7f16a6c6d700 1 -- start start 2023-11-28T11:11:48.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f16a6c6d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16a0161090 0x7f16a00ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f16a6c6d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f16a00caeb0 0x7f16a00cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f16a6c6d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16a00ce340 con 0x7f16a015c990 2023-11-28T11:11:48.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f16a6c6d700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16a00cebe0 con 0x7f16a00caeb0 2023-11-28T11:11:48.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f16a6c6d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00cf780 con 0x7f16a0161090 2023-11-28T11:11:48.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f16a520a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f16a00caeb0 0x7f16a00cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.225 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f16a520a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f16a00caeb0 0x7f16a00cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51062/0 (socket says 172.21.15.28:51062) 2023-11-28T11:11:48.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f16a520a700 1 -- 172.21.15.28:0/3975647849 learned_addr learned my addr 172.21.15.28:0/3975647849 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:48.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f169ffff700 1 --2- 172.21.15.28:0/3975647849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16a0161090 0x7f16a00ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f16a520a700 1 -- 172.21.15.28:0/3975647849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16a0161090 msgr2=0x7f16a00ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:48.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f16a520a700 1 --2- 172.21.15.28:0/3975647849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16a0161090 0x7f16a00ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f16a520a700 1 -- 172.21.15.28:0/3975647849 >> v1:172.21.15.59:6789/0 conn(0x7f16a015c990 legacy=0x7f16a00cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f16a520a700 1 -- 172.21.15.28:0/3975647849 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16a00cb3f0 con 0x7f16a00caeb0 2023-11-28T11:11:48.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f16a520a700 1 --2- 172.21.15.28:0/3975647849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16a00caeb0 0x7f16a00cc310 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1698009e50 tx=0x7f169800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16441f6ccbc63500 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:48.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f169f7fe700 1 -- 172.21.15.28:0/3975647849 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800e070 con 0x7f16a00caeb0 2023-11-28T11:11:48.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f169f7fe700 1 -- 172.21.15.28:0/3975647849 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f169800b910 con 0x7f16a00caeb0 2023-11-28T11:11:48.226 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.223+0000 7f169f7fe700 1 -- 172.21.15.28:0/3975647849 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f169800bcc0 con 0x7f16a00caeb0 2023-11-28T11:11:48.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.224+0000 7f16a6c6d700 1 -- 172.21.15.28:0/3975647849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16a00caeb0 msgr2=0x7f16a00cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.224+0000 7f16a6c6d700 1 --2- 172.21.15.28:0/3975647849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16a00caeb0 0x7f16a00cc310 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f1698009e50 tx=0x7f169800b330 comp rx=0 tx=0).stop 2023-11-28T11:11:48.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.224+0000 7f16a6c6d700 1 -- 172.21.15.28:0/3975647849 shutdown_connections 2023-11-28T11:11:48.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.224+0000 7f16a6c6d700 1 --2- 172.21.15.28:0/3975647849 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16a00caeb0 0x7f16a00cc310 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.224+0000 7f16a6c6d700 1 --2- 172.21.15.28:0/3975647849 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16a0161090 0x7f16a00ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.224+0000 7f16a6c6d700 1 -- 172.21.15.28:0/3975647849 >> 172.21.15.28:0/3975647849 conn(0x7f16a00b1650 msgr2=0x7f16a00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:48.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.224+0000 7f16a6c6d700 1 -- 172.21.15.28:0/3975647849 shutdown_connections 2023-11-28T11:11:48.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.224+0000 7f16a6c6d700 1 -- 172.21.15.28:0/3975647849 wait complete. 2023-11-28T11:11:48.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.225+0000 7f16a6c6d700 1 Processor -- start 2023-11-28T11:11:48.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.225+0000 7f16a6c6d700 1 -- start start 2023-11-28T11:11:48.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.226+0000 7f16a6c6d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a00caeb0 0x7f16a015afd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.226+0000 7f16a6c6d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16a015c990 0x7f16a00c5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.226+0000 7f16a6c6d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16a0161090 0x7f16a00ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.226+0000 7f16a6c6d700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a01604a0 con 0x7f16a015c990 2023-11-28T11:11:48.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.226+0000 7f16a6c6d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00b5700 con 0x7f16a00caeb0 2023-11-28T11:11:48.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.226+0000 7f16a6c6d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16a00c4c20 con 0x7f16a0161090 2023-11-28T11:11:48.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.226+0000 7f169ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16a015c990 0x7f16a00c5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.226+0000 7f169ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16a015c990 0x7f16a00c5700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51066/0 (socket says 172.21.15.28:51066) 2023-11-28T11:11:48.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.226+0000 7f169ffff700 1 -- 172.21.15.28:0/3663590411 learned_addr learned my addr 172.21.15.28:0/3663590411 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:48.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.226+0000 7f16a520a700 1 --2- 172.21.15.28:0/3663590411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16a0161090 0x7f16a00ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.226+0000 7f16a4a09700 1 --2- 172.21.15.28:0/3663590411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a00caeb0 0x7f16a015afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.227+0000 7f169ffff700 1 -- 172.21.15.28:0/3663590411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16a0161090 msgr2=0x7f16a00ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.227+0000 7f169ffff700 1 --2- 172.21.15.28:0/3663590411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16a0161090 0x7f16a00ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.227+0000 7f169ffff700 1 -- 172.21.15.28:0/3663590411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a00caeb0 msgr2=0x7f16a015afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.227+0000 7f169ffff700 1 --2- 172.21.15.28:0/3663590411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a00caeb0 0x7f16a015afd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.227+0000 7f169ffff700 1 -- 172.21.15.28:0/3663590411 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1698009b30 con 0x7f16a015c990 2023-11-28T11:11:48.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.227+0000 7f16a4a09700 1 --2- 172.21.15.28:0/3663590411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a00caeb0 0x7f16a015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:48.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.227+0000 7f169ffff700 1 --2- 172.21.15.28:0/3663590411 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16a015c990 0x7f16a00c5700 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f168c00e840 tx=0x7f168c00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:48.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.227+0000 7f169d7fa700 1 -- 172.21.15.28:0/3663590411 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c009590 con 0x7f16a015c990 2023-11-28T11:11:48.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.227+0000 7f169d7fa700 1 -- 172.21.15.28:0/3663590411 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f168c013680 con 0x7f16a015c990 2023-11-28T11:11:48.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.227+0000 7f169d7fa700 1 -- 172.21.15.28:0/3663590411 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f168c013a30 con 0x7f16a015c990 2023-11-28T11:11:48.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.227+0000 7f16a6c6d700 1 -- 172.21.15.28:0/3663590411 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a00ce870 con 0x7f16a015c990 2023-11-28T11:11:48.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.227+0000 7f16a6c6d700 1 -- 172.21.15.28:0/3663590411 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a00ce590 con 0x7f16a015c990 2023-11-28T11:11:48.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.228+0000 7f16a6c6d700 1 -- 172.21.15.28:0/3663590411 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a00cc850 con 0x7f16a015c990 2023-11-28T11:11:48.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.228+0000 7f169d7fa700 1 -- 172.21.15.28:0/3663590411 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f168c019030 con 0x7f16a015c990 2023-11-28T11:11:48.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.229+0000 7f169d7fa700 1 --2- 172.21.15.28:0/3663590411 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f16840682f0 0x7f168406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.229+0000 7f169d7fa700 1 -- 172.21.15.28:0/3663590411 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f168c085bf0 con 0x7f16a015c990 2023-11-28T11:11:48.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.229+0000 7f16a4a09700 1 --2- 172.21.15.28:0/3663590411 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f16840682f0 0x7f168406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.229+0000 7f16a4a09700 1 --2- 172.21.15.28:0/3663590411 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f16840682f0 0x7f168406a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f16a006c410 tx=0x7f1694009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:48.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.233+0000 7f169d7fa700 1 -- 172.21.15.28:0/3663590411 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f168c052310 con 0x7f16a015c990 2023-11-28T11:11:48.431 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.429+0000 7f16a6c6d700 1 -- 172.21.15.28:0/3663590411 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f16a00675a0 con 0x7f16a015c990 2023-11-28T11:11:48.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.431+0000 7f169d7fa700 1 -- 172.21.15.28:0/3663590411 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18074 (secure 0 0 0) 0x7f168c055930 con 0x7f16a015c990 2023-11-28T11:11:48.432 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:48.432 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":54,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:11:00.773581+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:57.702079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:58.738753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:48.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.433+0000 7f16a6c6d700 1 -- 172.21.15.28:0/3663590411 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f16840682f0 msgr2=0x7f168406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.433+0000 7f16a6c6d700 1 --2- 172.21.15.28:0/3663590411 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f16840682f0 0x7f168406a7a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f16a006c410 tx=0x7f1694009040 comp rx=0 tx=0).stop 2023-11-28T11:11:48.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.433+0000 7f16a6c6d700 1 -- 172.21.15.28:0/3663590411 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16a015c990 msgr2=0x7f16a00c5700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.433+0000 7f16a6c6d700 1 --2- 172.21.15.28:0/3663590411 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16a015c990 0x7f16a00c5700 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f168c00e840 tx=0x7f168c00ec00 comp rx=0 tx=0).stop 2023-11-28T11:11:48.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.433+0000 7f16a6c6d700 1 -- 172.21.15.28:0/3663590411 shutdown_connections 2023-11-28T11:11:48.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.433+0000 7f16a6c6d700 1 --2- 172.21.15.28:0/3663590411 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f16840682f0 0x7f168406a7a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.433+0000 7f16a6c6d700 1 --2- 172.21.15.28:0/3663590411 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f16a0161090 0x7f16a00ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.433+0000 7f16a6c6d700 1 --2- 172.21.15.28:0/3663590411 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f16a00caeb0 0x7f16a015afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.433+0000 7f16a6c6d700 1 --2- 172.21.15.28:0/3663590411 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f16a015c990 0x7f16a00c5700 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.433+0000 7f16a6c6d700 1 -- 172.21.15.28:0/3663590411 >> 172.21.15.28:0/3663590411 conn(0x7f16a00b1650 msgr2=0x7f16a00b0e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:48.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.433+0000 7f16a6c6d700 1 -- 172.21.15.28:0/3663590411 shutdown_connections 2023-11-28T11:11:48.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.433+0000 7f16a6c6d700 1 -- 172.21.15.28:0/3663590411 wait complete. 2023-11-28T11:11:48.446 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:10:57.702079+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '53', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:10:58.738753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '54', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:11:48.446 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:11:48.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.572+0000 7fe3ae518700 1 Processor -- start 2023-11-28T11:11:48.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.572+0000 7fe3ae518700 1 -- start start 2023-11-28T11:11:48.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.573+0000 7fe3ae518700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3a80b1d40 0x7fe3a80cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.573+0000 7fe3ae518700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe3a815dbf0 0x7fe3a80ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.573+0000 7fe3ae518700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3a80ceff0 con 0x7fe3a8159500 2023-11-28T11:11:48.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.573+0000 7fe3ae518700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3a815e090 con 0x7fe3a815dbf0 2023-11-28T11:11:48.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.573+0000 7fe3ae518700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a80cede0 con 0x7fe3a80b1d40 2023-11-28T11:11:48.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.573+0000 7fe3acab5700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe3a815dbf0 0x7fe3a80ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.573+0000 7fe3a7fff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe3a8159500 0x7fe3a80b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39130/0 (socket says 172.21.15.28:39130) 2023-11-28T11:11:48.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.573+0000 7fe3a7fff700 1 -- 172.21.15.28:0/2751187172 learned_addr learned my addr 172.21.15.28:0/2751187172 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:48.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.574+0000 7fe3a77fe700 1 --2- 172.21.15.28:0/2751187172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3a80b1d40 0x7fe3a80cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.574+0000 7fe3acab5700 1 -- 172.21.15.28:0/2751187172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3a80b1d40 msgr2=0x7fe3a80cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:48.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.574+0000 7fe3acab5700 1 --2- 172.21.15.28:0/2751187172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3a80b1d40 0x7fe3a80cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.574+0000 7fe3acab5700 1 -- 172.21.15.28:0/2751187172 >> v1:172.21.15.28:6789/0 conn(0x7fe3a8159500 legacy=0x7fe3a80b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.574+0000 7fe3acab5700 1 -- 172.21.15.28:0/2751187172 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a80cf2d0 con 0x7fe3a815dbf0 2023-11-28T11:11:48.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.574+0000 7fe3a6ffd700 1 -- 172.21.15.28:0/2751187172 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198599578 0 0) 0x7fe3a80ceff0 con 0x7fe3a8159500 2023-11-28T11:11:48.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.574+0000 7fe3acab5700 1 --2- 172.21.15.28:0/2751187172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3a815dbf0 0x7fe3a80ce8a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009d90 tx=0x7fe3a000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99adbdacd44ac430 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:48.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.574+0000 7fe3a6ffd700 1 -- 172.21.15.28:0/2751187172 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a000e070 con 0x7fe3a815dbf0 2023-11-28T11:11:48.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.574+0000 7fe3a6ffd700 1 -- 172.21.15.28:0/2751187172 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3a000bbd0 con 0x7fe3a815dbf0 2023-11-28T11:11:48.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.574+0000 7fe3a6ffd700 1 -- 172.21.15.28:0/2751187172 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3a0012430 con 0x7fe3a815dbf0 2023-11-28T11:11:48.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.575+0000 7fe3ae518700 1 -- 172.21.15.28:0/2751187172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3a815dbf0 msgr2=0x7fe3a80ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.575+0000 7fe3ae518700 1 --2- 172.21.15.28:0/2751187172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3a815dbf0 0x7fe3a80ce8a0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe3a0009d90 tx=0x7fe3a000b330 comp rx=0 tx=0).stop 2023-11-28T11:11:48.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.575+0000 7fe3ae518700 1 -- 172.21.15.28:0/2751187172 shutdown_connections 2023-11-28T11:11:48.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.575+0000 7fe3ae518700 1 --2- 172.21.15.28:0/2751187172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3a815dbf0 0x7fe3a80ce8a0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.575+0000 7fe3ae518700 1 --2- 172.21.15.28:0/2751187172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3a80b1d40 0x7fe3a80cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.575+0000 7fe3ae518700 1 -- 172.21.15.28:0/2751187172 >> 172.21.15.28:0/2751187172 conn(0x7fe3a80cc630 msgr2=0x7fe3a80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:48.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.575+0000 7fe3ae518700 1 -- 172.21.15.28:0/2751187172 shutdown_connections 2023-11-28T11:11:48.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.575+0000 7fe3ae518700 1 -- 172.21.15.28:0/2751187172 wait complete. 2023-11-28T11:11:48.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.576+0000 7fe3ae518700 1 Processor -- start 2023-11-28T11:11:48.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.576+0000 7fe3ae518700 1 -- start start 2023-11-28T11:11:48.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3ae518700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3a80b1d40 0x7fe3a81ec5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3ae518700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3a8159500 0x7fe3a8156af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3ae518700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3a815dbf0 0x7fe3a80ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3ae518700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a81574b0 con 0x7fe3a8159500 2023-11-28T11:11:48.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3ae518700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a8157790 con 0x7fe3a815dbf0 2023-11-28T11:11:48.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3ae518700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3a80ad450 con 0x7fe3a80b1d40 2023-11-28T11:11:48.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3a77fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3a8159500 0x7fe3a8156af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3a77fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3a8159500 0x7fe3a8156af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51080/0 (socket says 172.21.15.28:51080) 2023-11-28T11:11:48.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3a77fe700 1 -- 172.21.15.28:0/539203797 learned_addr learned my addr 172.21.15.28:0/539203797 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:48.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3a7fff700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3a80b1d40 0x7fe3a81ec5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3acab5700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3a815dbf0 0x7fe3a80ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3a77fe700 1 -- 172.21.15.28:0/539203797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3a80b1d40 msgr2=0x7fe3a81ec5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3a77fe700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3a80b1d40 0x7fe3a81ec5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3a77fe700 1 -- 172.21.15.28:0/539203797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3a815dbf0 msgr2=0x7fe3a80ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3a77fe700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3a815dbf0 0x7fe3a80ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3a77fe700 1 -- 172.21.15.28:0/539203797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3a0009a70 con 0x7fe3a8159500 2023-11-28T11:11:48.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3a7fff700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3a80b1d40 0x7fe3a81ec5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:48.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3acab5700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3a815dbf0 0x7fe3a80ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:48.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.577+0000 7fe3a77fe700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3a8159500 0x7fe3a8156af0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe39000d800 tx=0x7fe39000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:48.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.578+0000 7fe3a4ff9700 1 -- 172.21.15.28:0/539203797 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3900125f0 con 0x7fe3a8159500 2023-11-28T11:11:48.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.578+0000 7fe3a4ff9700 1 -- 172.21.15.28:0/539203797 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe390012750 con 0x7fe3a8159500 2023-11-28T11:11:48.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.578+0000 7fe3ae518700 1 -- 172.21.15.28:0/539203797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3a815d3f0 con 0x7fe3a8159500 2023-11-28T11:11:48.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.578+0000 7fe3a4ff9700 1 -- 172.21.15.28:0/539203797 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe390012b00 con 0x7fe3a8159500 2023-11-28T11:11:48.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.578+0000 7fe3ae518700 1 -- 172.21.15.28:0/539203797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3a815d110 con 0x7fe3a8159500 2023-11-28T11:11:48.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.579+0000 7fe3ae518700 1 -- 172.21.15.28:0/539203797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3a815d6d0 con 0x7fe3a8159500 2023-11-28T11:11:48.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.579+0000 7fe3a4ff9700 1 -- 172.21.15.28:0/539203797 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe390012c60 con 0x7fe3a8159500 2023-11-28T11:11:48.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.579+0000 7fe3a4ff9700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe3880682f0 0x7fe38806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.579+0000 7fe3a4ff9700 1 -- 172.21.15.28:0/539203797 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe3900860a0 con 0x7fe3a8159500 2023-11-28T11:11:48.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.579+0000 7fe3a7fff700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe3880682f0 0x7fe38806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.580+0000 7fe3a7fff700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe3880682f0 0x7fe38806a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe398007fd0 tx=0x7fe398009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:48.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.584+0000 7fe3a4ff9700 1 -- 172.21.15.28:0/539203797 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe3900527c0 con 0x7fe3a8159500 2023-11-28T11:11:48.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.775+0000 7fe3ae518700 1 -- 172.21.15.28:0/539203797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe3a80739f0 con 0x7fe3a8159500 2023-11-28T11:11:48.777 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.775+0000 7fe3a4ff9700 1 -- 172.21.15.28:0/539203797 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v54) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe390055de0 con 0x7fe3a8159500 2023-11-28T11:11:48.777 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:11:48.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.777+0000 7fe3ae518700 1 -- 172.21.15.28:0/539203797 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe3880682f0 msgr2=0x7fe38806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.779 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.777+0000 7fe3ae518700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe3880682f0 0x7fe38806a7a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fe398007fd0 tx=0x7fe398009040 comp rx=0 tx=0).stop 2023-11-28T11:11:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.778+0000 7fe3ae518700 1 -- 172.21.15.28:0/539203797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3a8159500 msgr2=0x7fe3a8156af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.778+0000 7fe3ae518700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3a8159500 0x7fe3a8156af0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe39000d800 tx=0x7fe39000dbc0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.778+0000 7fe3ae518700 1 -- 172.21.15.28:0/539203797 shutdown_connections 2023-11-28T11:11:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.778+0000 7fe3ae518700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe3880682f0 0x7fe38806a7a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.778+0000 7fe3ae518700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3a815dbf0 0x7fe3a80ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.778+0000 7fe3ae518700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe3a80b1d40 0x7fe3a81ec5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.778+0000 7fe3ae518700 1 --2- 172.21.15.28:0/539203797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe3a8159500 0x7fe3a8156af0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.778+0000 7fe3ae518700 1 -- 172.21.15.28:0/539203797 >> 172.21.15.28:0/539203797 conn(0x7fe3a80cc630 msgr2=0x7fe3a80b13d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.778+0000 7fe3ae518700 1 -- 172.21.15.28:0/539203797 shutdown_connections 2023-11-28T11:11:48.780 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.778+0000 7fe3ae518700 1 -- 172.21.15.28:0/539203797 wait complete. 2023-11-28T11:11:48.791 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:11:48.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.917+0000 7f428de4d700 1 Processor -- start 2023-11-28T11:11:48.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.917+0000 7f428de4d700 1 -- start start 2023-11-28T11:11:48.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.918+0000 7f428de4d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f42880b62d0 0x7f42880cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.918+0000 7f428de4d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42880b2d90 0x7f42880ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.918+0000 7f428de4d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42880cd2a0 con 0x7f428806f8b0 2023-11-28T11:11:48.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.918+0000 7f428de4d700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42880d0400 con 0x7f42880b62d0 2023-11-28T11:11:48.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.918+0000 7f428de4d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42880d06e0 con 0x7f42880b2d90 2023-11-28T11:11:48.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.918+0000 7f42877fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f42880b62d0 0x7f42880cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.918+0000 7f42877fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f42880b62d0 0x7f42880cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51086/0 (socket says 172.21.15.28:51086) 2023-11-28T11:11:48.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.918+0000 7f42877fe700 1 -- 172.21.15.28:0/1941447694 learned_addr learned my addr 172.21.15.28:0/1941447694 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:48.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.919+0000 7f4287fff700 1 --2- 172.21.15.28:0/1941447694 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42880b2d90 0x7f42880ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.919+0000 7f42877fe700 1 -- 172.21.15.28:0/1941447694 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42880b2d90 msgr2=0x7f42880ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:48.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.919+0000 7f42877fe700 1 --2- 172.21.15.28:0/1941447694 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42880b2d90 0x7f42880ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.919+0000 7f42877fe700 1 -- 172.21.15.28:0/1941447694 >> v1:172.21.15.59:6789/0 conn(0x7f428806f8b0 legacy=0x7f42880b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.919+0000 7f42877fe700 1 -- 172.21.15.28:0/1941447694 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42880cffb0 con 0x7f42880b62d0 2023-11-28T11:11:48.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.919+0000 7f42877fe700 1 --2- 172.21.15.28:0/1941447694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42880b62d0 0x7f42880cb630 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f42780098f0 tx=0x7f4278009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cab8f3aabe2a0f37 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:48.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.919+0000 7f42867fc700 1 -- 172.21.15.28:0/1941447694 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4278011830 con 0x7f42880b62d0 2023-11-28T11:11:48.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.919+0000 7f42867fc700 1 -- 172.21.15.28:0/1941447694 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4278011990 con 0x7f42880b62d0 2023-11-28T11:11:48.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.919+0000 7f42867fc700 1 -- 172.21.15.28:0/1941447694 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4278011d20 con 0x7f42880b62d0 2023-11-28T11:11:48.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.920+0000 7f428de4d700 1 -- 172.21.15.28:0/1941447694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42880b62d0 msgr2=0x7f42880cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.920+0000 7f428de4d700 1 --2- 172.21.15.28:0/1941447694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42880b62d0 0x7f42880cb630 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f42780098f0 tx=0x7f4278009c00 comp rx=0 tx=0).stop 2023-11-28T11:11:48.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.920+0000 7f428de4d700 1 -- 172.21.15.28:0/1941447694 shutdown_connections 2023-11-28T11:11:48.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.920+0000 7f428de4d700 1 --2- 172.21.15.28:0/1941447694 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42880b62d0 0x7f42880cb630 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.920+0000 7f428de4d700 1 --2- 172.21.15.28:0/1941447694 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42880b2d90 0x7f42880ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.920+0000 7f428de4d700 1 -- 172.21.15.28:0/1941447694 >> 172.21.15.28:0/1941447694 conn(0x7f42880cd680 msgr2=0x7f42880cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:48.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.920+0000 7f428de4d700 1 -- 172.21.15.28:0/1941447694 shutdown_connections 2023-11-28T11:11:48.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.920+0000 7f428de4d700 1 -- 172.21.15.28:0/1941447694 wait complete. 2023-11-28T11:11:48.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.921+0000 7f428de4d700 1 Processor -- start 2023-11-28T11:11:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.921+0000 7f428de4d700 1 -- start start 2023-11-28T11:11:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.922+0000 7f428de4d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f428806f8b0 0x7f42880c6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.922+0000 7f428de4d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42880b2d90 0x7f4288155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.922+0000 7f428de4d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42880b62d0 0x7f42881f2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.922+0000 7f428de4d700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42881f41e0 con 0x7f42880b2d90 2023-11-28T11:11:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.922+0000 7f428de4d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42880c65c0 con 0x7f428806f8b0 2023-11-28T11:11:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.922+0000 7f428de4d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42881f3660 con 0x7f42880b62d0 2023-11-28T11:11:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.922+0000 7f4286ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42880b2d90 0x7f4288155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.923+0000 7f4286ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42880b2d90 0x7f4288155ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51094/0 (socket says 172.21.15.28:51094) 2023-11-28T11:11:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.923+0000 7f4286ffd700 1 -- 172.21.15.28:0/3803622546 learned_addr learned my addr 172.21.15.28:0/3803622546 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:48.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.923+0000 7f42877fe700 1 --2- 172.21.15.28:0/3803622546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f428806f8b0 0x7f42880c6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.923+0000 7f4287fff700 1 --2- 172.21.15.28:0/3803622546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42880b62d0 0x7f42881f2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.923+0000 7f4286ffd700 1 -- 172.21.15.28:0/3803622546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42880b62d0 msgr2=0x7f42881f2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.923+0000 7f4286ffd700 1 --2- 172.21.15.28:0/3803622546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42880b62d0 0x7f42881f2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.923+0000 7f4286ffd700 1 -- 172.21.15.28:0/3803622546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f428806f8b0 msgr2=0x7f42880c6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.923+0000 7f4286ffd700 1 --2- 172.21.15.28:0/3803622546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f428806f8b0 0x7f42880c6080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.923+0000 7f4286ffd700 1 -- 172.21.15.28:0/3803622546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4278009550 con 0x7f42880b2d90 2023-11-28T11:11:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.923+0000 7f4286ffd700 1 --2- 172.21.15.28:0/3803622546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42880b2d90 0x7f4288155ff0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f427c00c4e0 tx=0x7f427c00c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.923+0000 7f426ffff700 1 -- 172.21.15.28:0/3803622546 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c0052d0 con 0x7f42880b2d90 2023-11-28T11:11:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.923+0000 7f426ffff700 1 -- 172.21.15.28:0/3803622546 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427c005430 con 0x7f42880b2d90 2023-11-28T11:11:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.923+0000 7f428de4d700 1 -- 172.21.15.28:0/3803622546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42880b66a0 con 0x7f42880b2d90 2023-11-28T11:11:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.923+0000 7f426ffff700 1 -- 172.21.15.28:0/3803622546 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427c0038c0 con 0x7f42880b2d90 2023-11-28T11:11:48.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.923+0000 7f428de4d700 1 -- 172.21.15.28:0/3803622546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f428806fd30 con 0x7f42880b2d90 2023-11-28T11:11:48.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.924+0000 7f428de4d700 1 -- 172.21.15.28:0/3803622546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42880cce50 con 0x7f42880b2d90 2023-11-28T11:11:48.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.925+0000 7f426ffff700 1 -- 172.21.15.28:0/3803622546 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f427c019030 con 0x7f42880b2d90 2023-11-28T11:11:48.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.925+0000 7f426ffff700 1 --2- 172.21.15.28:0/3803622546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42740682f0 0x7f427406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:48.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.925+0000 7f426ffff700 1 -- 172.21.15.28:0/3803622546 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f427c085030 con 0x7f42880b2d90 2023-11-28T11:11:48.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.925+0000 7f42877fe700 1 --2- 172.21.15.28:0/3803622546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42740682f0 0x7f427406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:48.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.926+0000 7f42877fe700 1 --2- 172.21.15.28:0/3803622546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42740682f0 0x7f427406a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4278014040 tx=0x7f4278008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:48.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:48.929+0000 7f426ffff700 1 -- 172.21.15.28:0/3803622546 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f427c051750 con 0x7f42880b2d90 2023-11-28T11:11:49.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.121+0000 7f428de4d700 1 -- 172.21.15.28:0/3803622546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f42880739f0 con 0x7f42880b2d90 2023-11-28T11:11:49.123 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.121+0000 7f426ffff700 1 -- 172.21.15.28:0/3803622546 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v54) v1 ==== 93+0+10 (secure 0 0 0) 0x7f427c0170a0 con 0x7f42880b2d90 2023-11-28T11:11:49.123 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:11:49.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.123+0000 7f428de4d700 1 -- 172.21.15.28:0/3803622546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42740682f0 msgr2=0x7f427406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.123+0000 7f428de4d700 1 --2- 172.21.15.28:0/3803622546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42740682f0 0x7f427406a7a0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f4278014040 tx=0x7f4278008030 comp rx=0 tx=0).stop 2023-11-28T11:11:49.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.123+0000 7f428de4d700 1 -- 172.21.15.28:0/3803622546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42880b2d90 msgr2=0x7f4288155ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.123+0000 7f428de4d700 1 --2- 172.21.15.28:0/3803622546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42880b2d90 0x7f4288155ff0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f427c00c4e0 tx=0x7f427c00c7f0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.123+0000 7f428de4d700 1 -- 172.21.15.28:0/3803622546 shutdown_connections 2023-11-28T11:11:49.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.123+0000 7f428de4d700 1 --2- 172.21.15.28:0/3803622546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f42740682f0 0x7f427406a7a0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.125 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.123+0000 7f428de4d700 1 --2- 172.21.15.28:0/3803622546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f42880b62d0 0x7f42881f2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.123+0000 7f428de4d700 1 --2- 172.21.15.28:0/3803622546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f428806f8b0 0x7f42880c6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.123+0000 7f428de4d700 1 --2- 172.21.15.28:0/3803622546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f42880b2d90 0x7f4288155ff0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.123+0000 7f428de4d700 1 -- 172.21.15.28:0/3803622546 >> 172.21.15.28:0/3803622546 conn(0x7f42880cd680 msgr2=0x7f4288065ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:49.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.123+0000 7f428de4d700 1 -- 172.21.15.28:0/3803622546 shutdown_connections 2023-11-28T11:11:49.126 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.123+0000 7f428de4d700 1 -- 172.21.15.28:0/3803622546 wait complete. 2023-11-28T11:11:49.137 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:11:49.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.263+0000 7f6a3b4bc700 1 Processor -- start 2023-11-28T11:11:49.265 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.263+0000 7f6a3b4bc700 1 -- start start 2023-11-28T11:11:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.264+0000 7f6a3b4bc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6a340b1630 0x7f6a340cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.264+0000 7f6a3b4bc700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6a3406afc0 0x7f6a3406b420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.264+0000 7f6a3b4bc700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a340ce550 con 0x7f6a340b1a00 2023-11-28T11:11:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.264+0000 7f6a3b4bc700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a340ce830 con 0x7f6a340b1630 2023-11-28T11:11:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.264+0000 7f6a3b4bc700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a340ceb10 con 0x7f6a3406afc0 2023-11-28T11:11:49.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.264+0000 7f6a39258700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6a340b1630 0x7f6a340cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.264+0000 7f6a39258700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6a340b1630 0x7f6a340cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51100/0 (socket says 172.21.15.28:51100) 2023-11-28T11:11:49.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.264+0000 7f6a39258700 1 -- 172.21.15.28:0/2648800506 learned_addr learned my addr 172.21.15.28:0/2648800506 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:49.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.264+0000 7f6a39a59700 1 --2- 172.21.15.28:0/2648800506 >> v2:172.21.15.59:3300/0 conn(0x7f6a3406afc0 0x7f6a3406b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.264+0000 7f6a39258700 1 -- 172.21.15.28:0/2648800506 >> v2:172.21.15.59:3300/0 conn(0x7f6a3406afc0 msgr2=0x7f6a3406b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:49.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.264+0000 7f6a39258700 1 --2- 172.21.15.28:0/2648800506 >> v2:172.21.15.59:3300/0 conn(0x7f6a3406afc0 0x7f6a3406b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.268 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.264+0000 7f6a39258700 1 -- 172.21.15.28:0/2648800506 >> v1:172.21.15.59:6789/0 conn(0x7f6a340b1a00 legacy=0x7f6a340caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.264+0000 7f6a39258700 1 -- 172.21.15.28:0/2648800506 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a340cedf0 con 0x7f6a340b1630 2023-11-28T11:11:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.264+0000 7f6a39a59700 1 --2- 172.21.15.28:0/2648800506 >> v2:172.21.15.59:3300/0 conn(0x7f6a3406afc0 0x7f6a3406b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.264+0000 7f6a39258700 1 --2- 172.21.15.28:0/2648800506 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a340b1630 0x7f6a340cf5f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6a300098f0 tx=0x7f6a30009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0a2eee3cf5b30d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.265+0000 7f6a2bfff700 1 -- 172.21.15.28:0/2648800506 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3000b860 con 0x7f6a340b1630 2023-11-28T11:11:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.265+0000 7f6a2bfff700 1 -- 172.21.15.28:0/2648800506 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3000b9c0 con 0x7f6a340b1630 2023-11-28T11:11:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.265+0000 7f6a2bfff700 1 -- 172.21.15.28:0/2648800506 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3000bd50 con 0x7f6a340b1630 2023-11-28T11:11:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.265+0000 7f6a3b4bc700 1 -- 172.21.15.28:0/2648800506 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a340b1630 msgr2=0x7f6a340cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.265+0000 7f6a3b4bc700 1 --2- 172.21.15.28:0/2648800506 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a340b1630 0x7f6a340cf5f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f6a300098f0 tx=0x7f6a30009c00 comp rx=0 tx=0).stop 2023-11-28T11:11:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.265+0000 7f6a3b4bc700 1 -- 172.21.15.28:0/2648800506 shutdown_connections 2023-11-28T11:11:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.265+0000 7f6a3b4bc700 1 --2- 172.21.15.28:0/2648800506 >> v2:172.21.15.59:3300/0 conn(0x7f6a3406afc0 0x7f6a3406b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.265+0000 7f6a3b4bc700 1 --2- 172.21.15.28:0/2648800506 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a340b1630 0x7f6a340cf5f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.265+0000 7f6a3b4bc700 1 -- 172.21.15.28:0/2648800506 >> 172.21.15.28:0/2648800506 conn(0x7f6a340cc630 msgr2=0x7f6a340cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:49.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.265+0000 7f6a3b4bc700 1 -- 172.21.15.28:0/2648800506 shutdown_connections 2023-11-28T11:11:49.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.265+0000 7f6a3b4bc700 1 -- 172.21.15.28:0/2648800506 wait complete. 2023-11-28T11:11:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.267+0000 7f6a3b4bc700 1 Processor -- start 2023-11-28T11:11:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.267+0000 7f6a3b4bc700 1 -- start start 2023-11-28T11:11:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.267+0000 7f6a3b4bc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a3406afc0 0x7f6a341f3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.267+0000 7f6a3b4bc700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a340b1630 0x7f6a341e1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.267+0000 7f6a3b4bc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340b1a00 0x7f6a341f50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.267+0000 7f6a3b4bc700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340affc0 con 0x7f6a3406afc0 2023-11-28T11:11:49.272 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.267+0000 7f6a3b4bc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340af690 con 0x7f6a340b1a00 2023-11-28T11:11:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.267+0000 7f6a3b4bc700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a340af900 con 0x7f6a340b1630 2023-11-28T11:11:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.267+0000 7f6a39258700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a3406afc0 0x7f6a341f3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.267+0000 7f6a39258700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a3406afc0 0x7f6a341f3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51104/0 (socket says 172.21.15.28:51104) 2023-11-28T11:11:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.268+0000 7f6a39258700 1 -- 172.21.15.28:0/2527881074 learned_addr learned my addr 172.21.15.28:0/2527881074 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.268+0000 7f6a39a59700 1 --2- 172.21.15.28:0/2527881074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340b1a00 0x7f6a341f50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.268+0000 7f6a38a57700 1 --2- 172.21.15.28:0/2527881074 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a340b1630 0x7f6a341e1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.268+0000 7f6a39258700 1 -- 172.21.15.28:0/2527881074 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a340b1630 msgr2=0x7f6a341e1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.273 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.268+0000 7f6a39258700 1 --2- 172.21.15.28:0/2527881074 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a340b1630 0x7f6a341e1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.268+0000 7f6a39258700 1 -- 172.21.15.28:0/2527881074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340b1a00 msgr2=0x7f6a341f50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.268+0000 7f6a39258700 1 --2- 172.21.15.28:0/2527881074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340b1a00 0x7f6a341f50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.268+0000 7f6a39258700 1 -- 172.21.15.28:0/2527881074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a30009550 con 0x7f6a3406afc0 2023-11-28T11:11:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.268+0000 7f6a39258700 1 --2- 172.21.15.28:0/2527881074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a3406afc0 0x7f6a341f3940 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6a34066a20 tx=0x7f6a30004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.268+0000 7f6a29ffb700 1 -- 172.21.15.28:0/2527881074 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a30004300 con 0x7f6a3406afc0 2023-11-28T11:11:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.268+0000 7f6a29ffb700 1 -- 172.21.15.28:0/2527881074 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3001f070 con 0x7f6a3406afc0 2023-11-28T11:11:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.268+0000 7f6a3b4bc700 1 -- 172.21.15.28:0/2527881074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a340b1ea0 con 0x7f6a3406afc0 2023-11-28T11:11:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.268+0000 7f6a29ffb700 1 -- 172.21.15.28:0/2527881074 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a300036a0 con 0x7f6a3406afc0 2023-11-28T11:11:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.268+0000 7f6a3b4bc700 1 -- 172.21.15.28:0/2527881074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a340cf0d0 con 0x7f6a3406afc0 2023-11-28T11:11:49.274 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.269+0000 7f6a3b4bc700 1 -- 172.21.15.28:0/2527881074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a340cb5b0 con 0x7f6a3406afc0 2023-11-28T11:11:49.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.271+0000 7f6a29ffb700 1 -- 172.21.15.28:0/2527881074 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6a3002a020 con 0x7f6a3406afc0 2023-11-28T11:11:49.275 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.272+0000 7f6a29ffb700 1 --2- 172.21.15.28:0/2527881074 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6a180682f0 0x7f6a1806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.272+0000 7f6a29ffb700 1 -- 172.21.15.28:0/2527881074 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6a30087610 con 0x7f6a3406afc0 2023-11-28T11:11:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.272+0000 7f6a38a57700 1 --2- 172.21.15.28:0/2527881074 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6a180682f0 0x7f6a1806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.276 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.272+0000 7f6a38a57700 1 --2- 172.21.15.28:0/2527881074 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6a180682f0 0x7f6a1806a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6a3415a9c0 tx=0x7f6a24009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:49.277 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.275+0000 7f6a29ffb700 1 -- 172.21.15.28:0/2527881074 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6a30053dd0 con 0x7f6a3406afc0 2023-11-28T11:11:49.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.469+0000 7f6a3b4bc700 1 -- 172.21.15.28:0/2527881074 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6a340b0100 con 0x7f6a3406afc0 2023-11-28T11:11:49.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.470+0000 7f6a29ffb700 1 -- 172.21.15.28:0/2527881074 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v54) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6a340b0100 con 0x7f6a3406afc0 2023-11-28T11:11:49.471 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:11:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.472+0000 7f6a3b4bc700 1 -- 172.21.15.28:0/2527881074 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6a180682f0 msgr2=0x7f6a1806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.472+0000 7f6a3b4bc700 1 --2- 172.21.15.28:0/2527881074 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6a180682f0 0x7f6a1806a7a0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6a3415a9c0 tx=0x7f6a24009040 comp rx=0 tx=0).stop 2023-11-28T11:11:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.472+0000 7f6a3b4bc700 1 -- 172.21.15.28:0/2527881074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a3406afc0 msgr2=0x7f6a341f3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.472+0000 7f6a3b4bc700 1 --2- 172.21.15.28:0/2527881074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a3406afc0 0x7f6a341f3940 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f6a34066a20 tx=0x7f6a30004750 comp rx=0 tx=0).stop 2023-11-28T11:11:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.472+0000 7f6a3b4bc700 1 -- 172.21.15.28:0/2527881074 shutdown_connections 2023-11-28T11:11:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.472+0000 7f6a3b4bc700 1 --2- 172.21.15.28:0/2527881074 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6a180682f0 0x7f6a1806a7a0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.472+0000 7f6a3b4bc700 1 --2- 172.21.15.28:0/2527881074 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6a3406afc0 0x7f6a341f3940 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.472+0000 7f6a3b4bc700 1 --2- 172.21.15.28:0/2527881074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6a340b1a00 0x7f6a341f50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.472+0000 7f6a3b4bc700 1 --2- 172.21.15.28:0/2527881074 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6a340b1630 0x7f6a341e1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.472+0000 7f6a3b4bc700 1 -- 172.21.15.28:0/2527881074 >> 172.21.15.28:0/2527881074 conn(0x7f6a340cc630 msgr2=0x7f6a340ab8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.472+0000 7f6a3b4bc700 1 -- 172.21.15.28:0/2527881074 shutdown_connections 2023-11-28T11:11:49.475 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.472+0000 7f6a3b4bc700 1 -- 172.21.15.28:0/2527881074 wait complete. 2023-11-28T11:11:49.486 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:11:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.612+0000 7fd33e8ea700 1 Processor -- start 2023-11-28T11:11:49.614 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.612+0000 7fd33e8ea700 1 -- start start 2023-11-28T11:11:49.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd33e8ea700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd3380b1630 0x7fd3380cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd33e8ea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3380b1a00 0x7fd3380caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd33e8ea700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd3380cb3e0 0x7fd338160350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd33e8ea700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3380ce550 con 0x7fd3380b1630 2023-11-28T11:11:49.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd33e8ea700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3380ce830 con 0x7fd3380cb3e0 2023-11-28T11:11:49.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd33e8ea700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3380ceb10 con 0x7fd3380b1a00 2023-11-28T11:11:49.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd337fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd3380b1630 0x7fd3380cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd337fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd3380b1630 0x7fd3380cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51112/0 (socket says 172.21.15.28:51112) 2023-11-28T11:11:49.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd337fff700 1 -- 172.21.15.28:0/2967779151 learned_addr learned my addr 172.21.15.28:0/2967779151 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:49.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd3377fe700 1 --2- 172.21.15.28:0/2967779151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3380b1a00 0x7fd3380caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd33ce87700 1 --2- 172.21.15.28:0/2967779151 >> v2:172.21.15.59:3300/0 conn(0x7fd3380cb3e0 0x7fd338160350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd337fff700 1 -- 172.21.15.28:0/2967779151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3380b1a00 msgr2=0x7fd3380caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:49.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd337fff700 1 --2- 172.21.15.28:0/2967779151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3380b1a00 0x7fd3380caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd337fff700 1 -- 172.21.15.28:0/2967779151 >> v2:172.21.15.59:3300/0 conn(0x7fd3380cb3e0 msgr2=0x7fd338160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:49.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd337fff700 1 --2- 172.21.15.28:0/2967779151 >> v2:172.21.15.59:3300/0 conn(0x7fd3380cb3e0 0x7fd338160350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd337fff700 1 -- 172.21.15.28:0/2967779151 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3380cedf0 con 0x7fd3380b1630 2023-11-28T11:11:49.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.613+0000 7fd337fff700 1 --2- 172.21.15.28:0/2967779151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b1630 0x7fd3380cf5f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd33806eb10 tx=0x7fd328009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfbc8691255ed08c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:49.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.614+0000 7fd336ffd700 1 -- 172.21.15.28:0/2967779151 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32800b870 con 0x7fd3380b1630 2023-11-28T11:11:49.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.614+0000 7fd336ffd700 1 -- 172.21.15.28:0/2967779151 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd32800b9d0 con 0x7fd3380b1630 2023-11-28T11:11:49.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.614+0000 7fd336ffd700 1 -- 172.21.15.28:0/2967779151 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd32800bd60 con 0x7fd3380b1630 2023-11-28T11:11:49.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.614+0000 7fd33e8ea700 1 -- 172.21.15.28:0/2967779151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b1630 msgr2=0x7fd3380cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.614+0000 7fd33e8ea700 1 --2- 172.21.15.28:0/2967779151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b1630 0x7fd3380cf5f0 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7fd33806eb10 tx=0x7fd328009bd0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.614+0000 7fd33e8ea700 1 -- 172.21.15.28:0/2967779151 shutdown_connections 2023-11-28T11:11:49.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.614+0000 7fd33e8ea700 1 --2- 172.21.15.28:0/2967779151 >> v2:172.21.15.59:3300/0 conn(0x7fd3380cb3e0 0x7fd338160350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.614+0000 7fd33e8ea700 1 --2- 172.21.15.28:0/2967779151 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b1630 0x7fd3380cf5f0 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.614+0000 7fd33e8ea700 1 --2- 172.21.15.28:0/2967779151 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3380b1a00 0x7fd3380caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.614+0000 7fd33e8ea700 1 -- 172.21.15.28:0/2967779151 >> 172.21.15.28:0/2967779151 conn(0x7fd3380cc630 msgr2=0x7fd3380cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:49.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.614+0000 7fd33e8ea700 1 -- 172.21.15.28:0/2967779151 shutdown_connections 2023-11-28T11:11:49.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.614+0000 7fd33e8ea700 1 -- 172.21.15.28:0/2967779151 wait complete. 2023-11-28T11:11:49.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.615+0000 7fd33e8ea700 1 Processor -- start 2023-11-28T11:11:49.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.616+0000 7fd33e8ea700 1 -- start start 2023-11-28T11:11:49.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.616+0000 7fd33e8ea700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b1630 0x7fd3380b5430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.616+0000 7fd33e8ea700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3380b1a00 0x7fd33814ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.616+0000 7fd33e8ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3380cb3e0 0x7fd3380c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.616+0000 7fd33e8ea700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3380c72d0 con 0x7fd3380b1630 2023-11-28T11:11:49.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.616+0000 7fd33e8ea700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3380c7590 con 0x7fd3380cb3e0 2023-11-28T11:11:49.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.616+0000 7fd33e8ea700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3380c7870 con 0x7fd3380b1a00 2023-11-28T11:11:49.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.616+0000 7fd337fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b1630 0x7fd3380b5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.616+0000 7fd337fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b1630 0x7fd3380b5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51118/0 (socket says 172.21.15.28:51118) 2023-11-28T11:11:49.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.616+0000 7fd337fff700 1 -- 172.21.15.28:0/1094194129 learned_addr learned my addr 172.21.15.28:0/1094194129 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:49.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.617+0000 7fd3377fe700 1 --2- 172.21.15.28:0/1094194129 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3380b1a00 0x7fd33814ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.617+0000 7fd33ce87700 1 --2- 172.21.15.28:0/1094194129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3380cb3e0 0x7fd3380c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.617+0000 7fd337fff700 1 -- 172.21.15.28:0/1094194129 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3380b1a00 msgr2=0x7fd33814ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.617+0000 7fd337fff700 1 --2- 172.21.15.28:0/1094194129 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3380b1a00 0x7fd33814ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.617+0000 7fd337fff700 1 -- 172.21.15.28:0/1094194129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3380cb3e0 msgr2=0x7fd3380c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.617+0000 7fd337fff700 1 --2- 172.21.15.28:0/1094194129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3380cb3e0 0x7fd3380c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.617+0000 7fd337fff700 1 -- 172.21.15.28:0/1094194129 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd328009550 con 0x7fd3380b1630 2023-11-28T11:11:49.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.617+0000 7fd337fff700 1 --2- 172.21.15.28:0/1094194129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b1630 0x7fd3380b5430 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd328000c00 tx=0x7fd328004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:49.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.617+0000 7fd334ff9700 1 -- 172.21.15.28:0/1094194129 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd328004480 con 0x7fd3380b1630 2023-11-28T11:11:49.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.617+0000 7fd334ff9700 1 -- 172.21.15.28:0/1094194129 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3280045e0 con 0x7fd3380b1630 2023-11-28T11:11:49.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.617+0000 7fd33e8ea700 1 -- 172.21.15.28:0/1094194129 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3380b1e70 con 0x7fd3380b1630 2023-11-28T11:11:49.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.617+0000 7fd334ff9700 1 -- 172.21.15.28:0/1094194129 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3280038e0 con 0x7fd3380b1630 2023-11-28T11:11:49.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.617+0000 7fd33e8ea700 1 -- 172.21.15.28:0/1094194129 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3380cf0d0 con 0x7fd3380b1630 2023-11-28T11:11:49.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.618+0000 7fd33e8ea700 1 -- 172.21.15.28:0/1094194129 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3380ce140 con 0x7fd3380b1630 2023-11-28T11:11:49.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.619+0000 7fd334ff9700 1 -- 172.21.15.28:0/1094194129 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd32800f050 con 0x7fd3380b1630 2023-11-28T11:11:49.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.619+0000 7fd334ff9700 1 --2- 172.21.15.28:0/1094194129 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd3240682f0 0x7fd32406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.619+0000 7fd334ff9700 1 -- 172.21.15.28:0/1094194129 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd328087550 con 0x7fd3380b1630 2023-11-28T11:11:49.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.619+0000 7fd3377fe700 1 --2- 172.21.15.28:0/1094194129 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd3240682f0 0x7fd32406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.619+0000 7fd3377fe700 1 --2- 172.21.15.28:0/1094194129 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd3240682f0 0x7fd32406a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd3380c3570 tx=0x7fd32c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:49.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.623+0000 7fd334ff9700 1 -- 172.21.15.28:0/1094194129 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd328053c70 con 0x7fd3380b1630 2023-11-28T11:11:49.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.816+0000 7fd33e8ea700 1 -- 172.21.15.28:0/1094194129 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd3380739f0 con 0x7fd3380b1630 2023-11-28T11:11:49.818 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.816+0000 7fd334ff9700 1 -- 172.21.15.28:0/1094194129 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v54) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd328057290 con 0x7fd3380b1630 2023-11-28T11:11:49.818 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:11:49.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.818+0000 7fd33e8ea700 1 -- 172.21.15.28:0/1094194129 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd3240682f0 msgr2=0x7fd32406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.818+0000 7fd33e8ea700 1 --2- 172.21.15.28:0/1094194129 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd3240682f0 0x7fd32406a7a0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fd3380c3570 tx=0x7fd32c008040 comp rx=0 tx=0).stop 2023-11-28T11:11:49.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.818+0000 7fd33e8ea700 1 -- 172.21.15.28:0/1094194129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b1630 msgr2=0x7fd3380b5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.818+0000 7fd33e8ea700 1 --2- 172.21.15.28:0/1094194129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b1630 0x7fd3380b5430 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7fd328000c00 tx=0x7fd328004750 comp rx=0 tx=0).stop 2023-11-28T11:11:49.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.818+0000 7fd33e8ea700 1 -- 172.21.15.28:0/1094194129 shutdown_connections 2023-11-28T11:11:49.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.818+0000 7fd33e8ea700 1 --2- 172.21.15.28:0/1094194129 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd3240682f0 0x7fd32406a7a0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.818+0000 7fd33e8ea700 1 --2- 172.21.15.28:0/1094194129 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd3380b1630 0x7fd3380b5430 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.818+0000 7fd33e8ea700 1 --2- 172.21.15.28:0/1094194129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd3380cb3e0 0x7fd3380c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.818+0000 7fd33e8ea700 1 --2- 172.21.15.28:0/1094194129 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd3380b1a00 0x7fd33814ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.818+0000 7fd33e8ea700 1 -- 172.21.15.28:0/1094194129 >> 172.21.15.28:0/1094194129 conn(0x7fd3380cc630 msgr2=0x7fd3380cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:49.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.818+0000 7fd33e8ea700 1 -- 172.21.15.28:0/1094194129 shutdown_connections 2023-11-28T11:11:49.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.819+0000 7fd33e8ea700 1 -- 172.21.15.28:0/1094194129 wait complete. 2023-11-28T11:11:49.832 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:49.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.955+0000 7f8c17310700 1 Processor -- start 2023-11-28T11:11:49.957 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.955+0000 7f8c17310700 1 -- start start 2023-11-28T11:11:49.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.956+0000 7f8c17310700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8c1015c560 0x7f8c1006f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.956+0000 7f8c17310700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8c100ceec0 0x7f8c1006c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.956+0000 7f8c17310700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c100b30b0 con 0x7f8c100b2660 2023-11-28T11:11:49.959 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.956+0000 7f8c17310700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c1006c020 con 0x7f8c1015c560 2023-11-28T11:11:49.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.956+0000 7f8c17310700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100d09c0 con 0x7f8c100ceec0 2023-11-28T11:11:49.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.956+0000 7f8c150ac700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f8c1015c560 0x7f8c1006f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.956+0000 7f8c148ab700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8c100b2660 0x7f8c100ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39184/0 (socket says 172.21.15.28:39184) 2023-11-28T11:11:49.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.956+0000 7f8c148ab700 1 -- 172.21.15.28:0/1130958731 learned_addr learned my addr 172.21.15.28:0/1130958731 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:49.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.957+0000 7f8c150ac700 1 -- 172.21.15.28:0/1130958731 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8c100ceec0 msgr2=0x7f8c1006c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:49.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.957+0000 7f8c150ac700 1 --2- 172.21.15.28:0/1130958731 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8c100ceec0 0x7f8c1006c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.957+0000 7f8c150ac700 1 -- 172.21.15.28:0/1130958731 >> v1:172.21.15.28:6789/0 conn(0x7f8c100b2660 legacy=0x7f8c100ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.957+0000 7f8c150ac700 1 -- 172.21.15.28:0/1130958731 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1006fdf0 con 0x7f8c1015c560 2023-11-28T11:11:49.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.957+0000 7f8c150ac700 1 --2- 172.21.15.28:0/1130958731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c1015c560 0x7f8c1006f8b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009840 tx=0x7f8c00009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eb173fd0da85c0b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:49.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.957+0000 7f8c07fff700 1 -- 172.21.15.28:0/1130958731 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00011830 con 0x7f8c1015c560 2023-11-28T11:11:49.960 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.957+0000 7f8c07fff700 1 -- 172.21.15.28:0/1130958731 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c00011990 con 0x7f8c1015c560 2023-11-28T11:11:49.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.957+0000 7f8c07fff700 1 -- 172.21.15.28:0/1130958731 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00011d20 con 0x7f8c1015c560 2023-11-28T11:11:49.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.957+0000 7f8c17310700 1 -- 172.21.15.28:0/1130958731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c1015c560 msgr2=0x7f8c1006f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.957+0000 7f8c17310700 1 --2- 172.21.15.28:0/1130958731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c1015c560 0x7f8c1006f8b0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009840 tx=0x7f8c00009b50 comp rx=0 tx=0).stop 2023-11-28T11:11:49.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.958+0000 7f8c17310700 1 -- 172.21.15.28:0/1130958731 shutdown_connections 2023-11-28T11:11:49.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.958+0000 7f8c17310700 1 --2- 172.21.15.28:0/1130958731 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8c100ceec0 0x7f8c1006c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.958+0000 7f8c17310700 1 --2- 172.21.15.28:0/1130958731 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c1015c560 0x7f8c1006f8b0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.958+0000 7f8c17310700 1 -- 172.21.15.28:0/1130958731 >> 172.21.15.28:0/1130958731 conn(0x7f8c100cd660 msgr2=0x7f8c100cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:49.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.958+0000 7f8c17310700 1 -- 172.21.15.28:0/1130958731 shutdown_connections 2023-11-28T11:11:49.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.958+0000 7f8c17310700 1 -- 172.21.15.28:0/1130958731 wait complete. 2023-11-28T11:11:49.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.959+0000 7f8c17310700 1 Processor -- start 2023-11-28T11:11:49.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.959+0000 7f8c17310700 1 -- start start 2023-11-28T11:11:49.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c17310700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c100b2660 0x7f8c101e8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c17310700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8c100ceec0 0x7f8c100c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c17310700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c1015c560 0x7f8c101545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c17310700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c100c5c20 con 0x7f8c100b2660 2023-11-28T11:11:49.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c17310700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c101e6af0 con 0x7f8c1015c560 2023-11-28T11:11:49.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c17310700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c101e6dd0 con 0x7f8c100ceec0 2023-11-28T11:11:49.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c150ac700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c100b2660 0x7f8c101e8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c150ac700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c100b2660 0x7f8c101e8b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51130/0 (socket says 172.21.15.28:51130) 2023-11-28T11:11:49.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c150ac700 1 -- 172.21.15.28:0/3037336997 learned_addr learned my addr 172.21.15.28:0/3037336997 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:49.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c148ab700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8c100ceec0 0x7f8c100c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c158ad700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c1015c560 0x7f8c101545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c150ac700 1 -- 172.21.15.28:0/3037336997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8c100ceec0 msgr2=0x7f8c100c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c150ac700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8c100ceec0 0x7f8c100c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c150ac700 1 -- 172.21.15.28:0/3037336997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c1015c560 msgr2=0x7f8c101545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:49.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c150ac700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c1015c560 0x7f8c101545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:49.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c150ac700 1 -- 172.21.15.28:0/3037336997 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c000094a0 con 0x7f8c100b2660 2023-11-28T11:11:49.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c148ab700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8c100ceec0 0x7f8c100c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:49.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c158ad700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c1015c560 0x7f8c101545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:49.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c150ac700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c100b2660 0x7f8c101e8b20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8c000090b0 tx=0x7f8c00004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:49.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c05ffb700 1 -- 172.21.15.28:0/3037336997 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c000045f0 con 0x7f8c100b2660 2023-11-28T11:11:49.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c05ffb700 1 -- 172.21.15.28:0/3037336997 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0001e070 con 0x7f8c100b2660 2023-11-28T11:11:49.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c17310700 1 -- 172.21.15.28:0/3037336997 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c100b2dd0 con 0x7f8c100b2660 2023-11-28T11:11:49.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.960+0000 7f8c05ffb700 1 -- 172.21.15.28:0/3037336997 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00003b00 con 0x7f8c100b2660 2023-11-28T11:11:49.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.961+0000 7f8c17310700 1 -- 172.21.15.28:0/3037336997 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c100b2af0 con 0x7f8c100b2660 2023-11-28T11:11:49.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.962+0000 7f8c17310700 1 -- 172.21.15.28:0/3037336997 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1006bcb0 con 0x7f8c100b2660 2023-11-28T11:11:49.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.962+0000 7f8c05ffb700 1 -- 172.21.15.28:0/3037336997 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8c0000f050 con 0x7f8c100b2660 2023-11-28T11:11:49.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.962+0000 7f8c05ffb700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8bf40682f0 0x7f8bf406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:49.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.962+0000 7f8c05ffb700 1 -- 172.21.15.28:0/3037336997 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8c00086fe0 con 0x7f8c100b2660 2023-11-28T11:11:49.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.962+0000 7f8c148ab700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8bf40682f0 0x7f8bf406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:49.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.963+0000 7f8c148ab700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8bf40682f0 0x7f8bf406a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8c100c7b80 tx=0x7f8c0c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:49.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:49.967+0000 7f8c05ffb700 1 -- 172.21.15.28:0/3037336997 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8c00053650 con 0x7f8c100b2660 2023-11-28T11:11:50.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.203+0000 7f8c17310700 1 -- 172.21.15.28:0/3037336997 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8c100675a0 con 0x7f8c100b2660 2023-11-28T11:11:50.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.203+0000 7f8c05ffb700 1 -- 172.21.15.28:0/3037336997 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f8c00015090 con 0x7f8c100b2660 2023-11-28T11:11:50.205 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:50.205 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5198,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/284515388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":284515388},{"type":"v1","addr":"172.21.15.59:6835","nonce":284515388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2160385010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2160385010},{"type":"v1","addr":"172.21.15.28:6836","nonce":2160385010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3775145347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3775145347},{"type":"v1","addr":"172.21.15.28:6837","nonce":3775145347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:59.771956+0000","modified":"2023-11-28T11:11:03.109382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6834/3261856585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3261856585},{"type":"v1","addr":"172.21.15.59:6834","nonce":3261856585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:11:50.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.205+0000 7f8c17310700 1 -- 172.21.15.28:0/3037336997 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8bf40682f0 msgr2=0x7f8bf406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:50.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.205+0000 7f8c17310700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8bf40682f0 0x7f8bf406a7a0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f8c100c7b80 tx=0x7f8c0c009040 comp rx=0 tx=0).stop 2023-11-28T11:11:50.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.205+0000 7f8c17310700 1 -- 172.21.15.28:0/3037336997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c100b2660 msgr2=0x7f8c101e8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:50.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.205+0000 7f8c17310700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c100b2660 0x7f8c101e8b20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f8c000090b0 tx=0x7f8c00004750 comp rx=0 tx=0).stop 2023-11-28T11:11:50.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.205+0000 7f8c17310700 1 -- 172.21.15.28:0/3037336997 shutdown_connections 2023-11-28T11:11:50.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.205+0000 7f8c17310700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8bf40682f0 0x7f8bf406a7a0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.205+0000 7f8c17310700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8c100b2660 0x7f8c101e8b20 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.205+0000 7f8c17310700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c1015c560 0x7f8c101545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.205+0000 7f8c17310700 1 --2- 172.21.15.28:0/3037336997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8c100ceec0 0x7f8c100c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.205+0000 7f8c17310700 1 -- 172.21.15.28:0/3037336997 >> 172.21.15.28:0/3037336997 conn(0x7f8c100cd660 msgr2=0x7f8c100b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:50.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.205+0000 7f8c17310700 1 -- 172.21.15.28:0/3037336997 shutdown_connections 2023-11-28T11:11:50.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.205+0000 7f8c17310700 1 -- 172.21.15.28:0/3037336997 wait complete. 2023-11-28T11:11:50.209 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 25 2023-11-28T11:11:50.219 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:11:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.343+0000 7f417c1fa700 1 Processor -- start 2023-11-28T11:11:50.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.343+0000 7f417c1fa700 1 -- start start 2023-11-28T11:11:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f417c1fa700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f417415c560 0x7f417406f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f417c1fa700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f41740b2660 0x7f41740ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f417c1fa700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41740cf150 con 0x7f41740cecf0 2023-11-28T11:11:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f417c1fa700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41740b30b0 con 0x7f41740b2660 2023-11-28T11:11:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f417c1fa700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f417406c020 con 0x7f417415c560 2023-11-28T11:11:50.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f4179795700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f41740b2660 0x7f41740ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f417a797700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f41740cecf0 0x7f4174160200 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39200/0 (socket says 172.21.15.28:39200) 2023-11-28T11:11:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f417a797700 1 -- 172.21.15.28:0/2122974116 learned_addr learned my addr 172.21.15.28:0/2122974116 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f4179f96700 1 --2- 172.21.15.28:0/2122974116 >> v2:172.21.15.59:3300/0 conn(0x7f417415c560 0x7f417406f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f4179795700 1 -- 172.21.15.28:0/2122974116 >> v2:172.21.15.59:3300/0 conn(0x7f417415c560 msgr2=0x7f417406f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f4179795700 1 --2- 172.21.15.28:0/2122974116 >> v2:172.21.15.59:3300/0 conn(0x7f417415c560 0x7f417406f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f4179795700 1 -- 172.21.15.28:0/2122974116 >> v1:172.21.15.28:6789/0 conn(0x7f41740cecf0 legacy=0x7f4174160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f4179795700 1 -- 172.21.15.28:0/2122974116 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41740d09c0 con 0x7f41740b2660 2023-11-28T11:11:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f4179f96700 1 --2- 172.21.15.28:0/2122974116 >> v2:172.21.15.59:3300/0 conn(0x7f417415c560 0x7f417406f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f4179795700 1 --2- 172.21.15.28:0/2122974116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41740b2660 0x7f41740ce7b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4174066920 tx=0x7f4164009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bbf426cc45315 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f4178f94700 1 -- 172.21.15.28:0/2122974116 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164011830 con 0x7f41740b2660 2023-11-28T11:11:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f4178f94700 1 -- 172.21.15.28:0/2122974116 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4164011990 con 0x7f41740b2660 2023-11-28T11:11:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.344+0000 7f4178f94700 1 -- 172.21.15.28:0/2122974116 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4164011d20 con 0x7f41740b2660 2023-11-28T11:11:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.345+0000 7f417c1fa700 1 -- 172.21.15.28:0/2122974116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41740b2660 msgr2=0x7f41740ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.345+0000 7f417c1fa700 1 --2- 172.21.15.28:0/2122974116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41740b2660 0x7f41740ce7b0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f4174066920 tx=0x7f4164009b20 comp rx=0 tx=0).stop 2023-11-28T11:11:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.345+0000 7f417c1fa700 1 -- 172.21.15.28:0/2122974116 shutdown_connections 2023-11-28T11:11:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.345+0000 7f417c1fa700 1 --2- 172.21.15.28:0/2122974116 >> v2:172.21.15.59:3300/0 conn(0x7f417415c560 0x7f417406f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.345+0000 7f417c1fa700 1 --2- 172.21.15.28:0/2122974116 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41740b2660 0x7f41740ce7b0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.345+0000 7f417c1fa700 1 -- 172.21.15.28:0/2122974116 >> 172.21.15.28:0/2122974116 conn(0x7f41740cd660 msgr2=0x7f41740cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:50.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.345+0000 7f417c1fa700 1 -- 172.21.15.28:0/2122974116 shutdown_connections 2023-11-28T11:11:50.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.345+0000 7f417c1fa700 1 -- 172.21.15.28:0/2122974116 wait complete. 2023-11-28T11:11:50.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.346+0000 7f417c1fa700 1 Processor -- start 2023-11-28T11:11:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.346+0000 7f417c1fa700 1 -- start start 2023-11-28T11:11:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.347+0000 7f417c1fa700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41740cecf0 0x7f41741f2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.347+0000 7f417c1fa700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f417415c560 0x7f41741ef800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.347+0000 7f417c1fa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740c8630 0x7f41741f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.347+0000 7f417c1fa700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41740adb10 con 0x7f41740cecf0 2023-11-28T11:11:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.347+0000 7f417c1fa700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41741f32d0 con 0x7f41740c8630 2023-11-28T11:11:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.347+0000 7f417c1fa700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41741efd40 con 0x7f417415c560 2023-11-28T11:11:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.347+0000 7f4179f96700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41740cecf0 0x7f41741f2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.347+0000 7f4179f96700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41740cecf0 0x7f41741f2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51144/0 (socket says 172.21.15.28:51144) 2023-11-28T11:11:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.347+0000 7f4179f96700 1 -- 172.21.15.28:0/3575731818 learned_addr learned my addr 172.21.15.28:0/3575731818 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.347+0000 7f4179f96700 1 -- 172.21.15.28:0/3575731818 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f417415c560 msgr2=0x7f41741ef800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.347+0000 7f4179f96700 1 --2- 172.21.15.28:0/3575731818 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f417415c560 0x7f41741ef800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.347+0000 7f4179f96700 1 -- 172.21.15.28:0/3575731818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740c8630 msgr2=0x7f41741f34d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.347+0000 7f4179f96700 1 --2- 172.21.15.28:0/3575731818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740c8630 0x7f41741f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.347+0000 7f4179f96700 1 -- 172.21.15.28:0/3575731818 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41640094a0 con 0x7f41740cecf0 2023-11-28T11:11:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.347+0000 7f4179f96700 1 --2- 172.21.15.28:0/3575731818 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41740cecf0 0x7f41741f2d90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f417000c820 tx=0x7f417000cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:50.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.348+0000 7f416a7fc700 1 -- 172.21.15.28:0/3575731818 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170011800 con 0x7f41740cecf0 2023-11-28T11:11:50.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.348+0000 7f416a7fc700 1 -- 172.21.15.28:0/3575731818 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4170011960 con 0x7f41740cecf0 2023-11-28T11:11:50.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.348+0000 7f417c1fa700 1 -- 172.21.15.28:0/3575731818 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41740b2af0 con 0x7f41740cecf0 2023-11-28T11:11:50.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.348+0000 7f416a7fc700 1 -- 172.21.15.28:0/3575731818 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4170011d10 con 0x7f41740cecf0 2023-11-28T11:11:50.355 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.348+0000 7f417c1fa700 1 -- 172.21.15.28:0/3575731818 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f417406fdf0 con 0x7f41740cecf0 2023-11-28T11:11:50.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.349+0000 7f417c1fa700 1 -- 172.21.15.28:0/3575731818 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41740b2dd0 con 0x7f41740cecf0 2023-11-28T11:11:50.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.349+0000 7f416a7fc700 1 -- 172.21.15.28:0/3575731818 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4170021070 con 0x7f41740cecf0 2023-11-28T11:11:50.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.349+0000 7f416a7fc700 1 --2- 172.21.15.28:0/3575731818 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f41580682f0 0x7f415806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:50.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.349+0000 7f416a7fc700 1 -- 172.21.15.28:0/3575731818 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4170085cb0 con 0x7f41740cecf0 2023-11-28T11:11:50.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.349+0000 7f4179795700 1 --2- 172.21.15.28:0/3575731818 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f41580682f0 0x7f415806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:50.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.350+0000 7f4179795700 1 --2- 172.21.15.28:0/3575731818 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f41580682f0 0x7f415806a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f41740cfad0 tx=0x7f4164010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:50.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.354+0000 7f416a7fc700 1 -- 172.21.15.28:0/3575731818 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4170052350 con 0x7f41740cecf0 2023-11-28T11:11:50.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.547+0000 7f417c1fa700 1 -- 172.21.15.28:0/3575731818 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f41740675a0 con 0x7f41740cecf0 2023-11-28T11:11:50.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.548+0000 7f416a7fc700 1 -- 172.21.15.28:0/3575731818 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18074 (secure 0 0 0) 0x7f4170055970 con 0x7f41740cecf0 2023-11-28T11:11:50.550 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:50.550 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":54,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:11:00.773581+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:57.702079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:58.738753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:50.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.550+0000 7f417c1fa700 1 -- 172.21.15.28:0/3575731818 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f41580682f0 msgr2=0x7f415806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:50.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.550+0000 7f417c1fa700 1 --2- 172.21.15.28:0/3575731818 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f41580682f0 0x7f415806a7a0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f41740cfad0 tx=0x7f4164010000 comp rx=0 tx=0).stop 2023-11-28T11:11:50.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.550+0000 7f417c1fa700 1 -- 172.21.15.28:0/3575731818 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41740cecf0 msgr2=0x7f41741f2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.550+0000 7f417c1fa700 1 --2- 172.21.15.28:0/3575731818 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41740cecf0 0x7f41741f2d90 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f417000c820 tx=0x7f417000cbe0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.550+0000 7f417c1fa700 1 -- 172.21.15.28:0/3575731818 shutdown_connections 2023-11-28T11:11:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.550+0000 7f417c1fa700 1 --2- 172.21.15.28:0/3575731818 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f41580682f0 0x7f415806a7a0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.550+0000 7f417c1fa700 1 --2- 172.21.15.28:0/3575731818 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f41740cecf0 0x7f41741f2d90 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.550+0000 7f417c1fa700 1 --2- 172.21.15.28:0/3575731818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f41740c8630 0x7f41741f34d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.550+0000 7f417c1fa700 1 --2- 172.21.15.28:0/3575731818 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f417415c560 0x7f41741ef800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.550+0000 7f417c1fa700 1 -- 172.21.15.28:0/3575731818 >> 172.21.15.28:0/3575731818 conn(0x7f41740cd660 msgr2=0x7f417406eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:50.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.551+0000 7f417c1fa700 1 -- 172.21.15.28:0/3575731818 shutdown_connections 2023-11-28T11:11:50.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.551+0000 7f417c1fa700 1 -- 172.21.15.28:0/3575731818 wait complete. 2023-11-28T11:11:50.564 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:11:50.565 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:50.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.687+0000 7f24fca5a700 1 Processor -- start 2023-11-28T11:11:50.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.687+0000 7f24fca5a700 1 -- start start 2023-11-28T11:11:50.691 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.688+0000 7f24fca5a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f24f815c990 0x7f24f80cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:50.691 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.688+0000 7f24fca5a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24f8161090 0x7f24f80ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.688+0000 7f24fca5a700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f24f80caeb0 0x7f24f80cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.688+0000 7f24fca5a700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f80ce340 con 0x7f24f815c990 2023-11-28T11:11:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.688+0000 7f24fca5a700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24f80cebe0 con 0x7f24f80caeb0 2023-11-28T11:11:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.688+0000 7f24fca5a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80cb3f0 con 0x7f24f8161090 2023-11-28T11:11:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.688+0000 7f24f659c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f24f815c990 0x7f24f80cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.688+0000 7f24f659c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f24f815c990 0x7f24f80cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51152/0 (socket says 172.21.15.28:51152) 2023-11-28T11:11:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.688+0000 7f24f659c700 1 -- 172.21.15.28:0/887825416 learned_addr learned my addr 172.21.15.28:0/887825416 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.688+0000 7f24f6d9d700 1 --2- 172.21.15.28:0/887825416 >> v2:172.21.15.59:3300/0 conn(0x7f24f80caeb0 0x7f24f80cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.688+0000 7f24f5d9b700 1 --2- 172.21.15.28:0/887825416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24f8161090 0x7f24f80ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.689+0000 7f24f659c700 1 -- 172.21.15.28:0/887825416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24f8161090 msgr2=0x7f24f80ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:50.692 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.689+0000 7f24f659c700 1 --2- 172.21.15.28:0/887825416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24f8161090 0x7f24f80ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.689+0000 7f24f659c700 1 -- 172.21.15.28:0/887825416 >> v2:172.21.15.59:3300/0 conn(0x7f24f80caeb0 msgr2=0x7f24f80cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.689+0000 7f24f659c700 1 --2- 172.21.15.28:0/887825416 >> v2:172.21.15.59:3300/0 conn(0x7f24f80caeb0 0x7f24f80cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.689+0000 7f24f659c700 1 -- 172.21.15.28:0/887825416 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24f80ce590 con 0x7f24f815c990 2023-11-28T11:11:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.689+0000 7f24f6d9d700 1 --2- 172.21.15.28:0/887825416 >> v2:172.21.15.59:3300/0 conn(0x7f24f80caeb0 0x7f24f80cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.689+0000 7f24f659c700 1 --2- 172.21.15.28:0/887825416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24f815c990 0x7f24f80cf070 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f24f80ca080 tx=0x7f24ec009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333a435a1f4f3ddc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.689+0000 7f24f559a700 1 -- 172.21.15.28:0/887825416 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec00b870 con 0x7f24f815c990 2023-11-28T11:11:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.689+0000 7f24f559a700 1 -- 172.21.15.28:0/887825416 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24ec00b9d0 con 0x7f24f815c990 2023-11-28T11:11:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.689+0000 7f24f559a700 1 -- 172.21.15.28:0/887825416 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24ec00bd60 con 0x7f24f815c990 2023-11-28T11:11:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.690+0000 7f24fca5a700 1 -- 172.21.15.28:0/887825416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24f815c990 msgr2=0x7f24f80cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.690+0000 7f24fca5a700 1 --2- 172.21.15.28:0/887825416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24f815c990 0x7f24f80cf070 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f24f80ca080 tx=0x7f24ec009bd0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.690+0000 7f24fca5a700 1 -- 172.21.15.28:0/887825416 shutdown_connections 2023-11-28T11:11:50.693 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.690+0000 7f24fca5a700 1 --2- 172.21.15.28:0/887825416 >> v2:172.21.15.59:3300/0 conn(0x7f24f80caeb0 0x7f24f80cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.690+0000 7f24fca5a700 1 --2- 172.21.15.28:0/887825416 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24f815c990 0x7f24f80cf070 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.690+0000 7f24fca5a700 1 --2- 172.21.15.28:0/887825416 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24f8161090 0x7f24f80ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.690+0000 7f24fca5a700 1 -- 172.21.15.28:0/887825416 >> 172.21.15.28:0/887825416 conn(0x7f24f80b1650 msgr2=0x7f24f80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:50.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.690+0000 7f24fca5a700 1 -- 172.21.15.28:0/887825416 shutdown_connections 2023-11-28T11:11:50.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.690+0000 7f24fca5a700 1 -- 172.21.15.28:0/887825416 wait complete. 2023-11-28T11:11:50.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.691+0000 7f24fca5a700 1 Processor -- start 2023-11-28T11:11:50.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.692+0000 7f24fca5a700 1 -- start start 2023-11-28T11:11:50.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.692+0000 7f24fca5a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24f80caeb0 0x7f24f80c5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:50.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.692+0000 7f24fca5a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24f815c990 0x7f24f814ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:50.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.692+0000 7f24fca5a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24f8161090 0x7f24f815a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:50.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.692+0000 7f24fca5a700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f815cd60 con 0x7f24f815c990 2023-11-28T11:11:50.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.692+0000 7f24fca5a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24f80c4d70 con 0x7f24f80caeb0 2023-11-28T11:11:50.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.692+0000 7f24fca5a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24f815cf70 con 0x7f24f8161090 2023-11-28T11:11:50.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24f5d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24f815c990 0x7f24f814ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:50.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24f5d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24f815c990 0x7f24f814ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51160/0 (socket says 172.21.15.28:51160) 2023-11-28T11:11:50.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24f5d9b700 1 -- 172.21.15.28:0/3410908820 learned_addr learned my addr 172.21.15.28:0/3410908820 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:50.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24f659c700 1 --2- 172.21.15.28:0/3410908820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24f80caeb0 0x7f24f80c5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:50.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24f6d9d700 1 --2- 172.21.15.28:0/3410908820 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24f8161090 0x7f24f815a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:50.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24f5d9b700 1 -- 172.21.15.28:0/3410908820 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24f8161090 msgr2=0x7f24f815a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:50.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24f5d9b700 1 --2- 172.21.15.28:0/3410908820 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24f8161090 0x7f24f815a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24f5d9b700 1 -- 172.21.15.28:0/3410908820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24f80caeb0 msgr2=0x7f24f80c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:50.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24f5d9b700 1 --2- 172.21.15.28:0/3410908820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24f80caeb0 0x7f24f80c5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24f5d9b700 1 -- 172.21.15.28:0/3410908820 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24ec009550 con 0x7f24f815c990 2023-11-28T11:11:50.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24f659c700 1 --2- 172.21.15.28:0/3410908820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24f80caeb0 0x7f24f80c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:50.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24f5d9b700 1 --2- 172.21.15.28:0/3410908820 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24f815c990 0x7f24f814ce70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f24e800d800 tx=0x7f24e800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:50.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24e6ffd700 1 -- 172.21.15.28:0/3410908820 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e80125f0 con 0x7f24f815c990 2023-11-28T11:11:50.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24e6ffd700 1 -- 172.21.15.28:0/3410908820 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e8012750 con 0x7f24f815c990 2023-11-28T11:11:50.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24fca5a700 1 -- 172.21.15.28:0/3410908820 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24f80cf5b0 con 0x7f24f815c990 2023-11-28T11:11:50.698 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24e6ffd700 1 -- 172.21.15.28:0/3410908820 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e8012ae0 con 0x7f24f815c990 2023-11-28T11:11:50.699 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.693+0000 7f24fca5a700 1 -- 172.21.15.28:0/3410908820 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24f80ce870 con 0x7f24f815c990 2023-11-28T11:11:50.699 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.694+0000 7f24fca5a700 1 -- 172.21.15.28:0/3410908820 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24f80cf740 con 0x7f24f815c990 2023-11-28T11:11:50.699 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.695+0000 7f24e6ffd700 1 -- 172.21.15.28:0/3410908820 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f24e8012c40 con 0x7f24f815c990 2023-11-28T11:11:50.699 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.695+0000 7f24e6ffd700 1 --2- 172.21.15.28:0/3410908820 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24d80682f0 0x7f24d806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:50.699 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.695+0000 7f24e6ffd700 1 -- 172.21.15.28:0/3410908820 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f24e80862f0 con 0x7f24f815c990 2023-11-28T11:11:50.699 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.695+0000 7f24f659c700 1 --2- 172.21.15.28:0/3410908820 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24d80682f0 0x7f24d806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:50.699 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.696+0000 7f24f659c700 1 --2- 172.21.15.28:0/3410908820 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24d80682f0 0x7f24d806a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f24f80b5710 tx=0x7f24ec008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:50.700 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.699+0000 7f24e6ffd700 1 -- 172.21.15.28:0/3410908820 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f24e8052a10 con 0x7f24f815c990 2023-11-28T11:11:50.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.938+0000 7f24fca5a700 1 -- 172.21.15.28:0/3410908820 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f24f815d460 con 0x7f24f815c990 2023-11-28T11:11:50.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.939+0000 7f24e6ffd700 1 -- 172.21.15.28:0/3410908820 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4645 (secure 0 0 0) 0x7f24e8012ed0 con 0x7f24f815c990 2023-11-28T11:11:50.940 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:50.940 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5198,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/284515388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":284515388},{"type":"v1","addr":"172.21.15.59:6835","nonce":284515388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2160385010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2160385010},{"type":"v1","addr":"172.21.15.28:6836","nonce":2160385010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3775145347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3775145347},{"type":"v1","addr":"172.21.15.28:6837","nonce":3775145347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:59.771956+0000","modified":"2023-11-28T11:11:03.109382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6834/3261856585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3261856585},{"type":"v1","addr":"172.21.15.59:6834","nonce":3261856585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:11:50.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.940+0000 7f24fca5a700 1 -- 172.21.15.28:0/3410908820 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24d80682f0 msgr2=0x7f24d806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:50.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.940+0000 7f24fca5a700 1 --2- 172.21.15.28:0/3410908820 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24d80682f0 0x7f24d806a7a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f24f80b5710 tx=0x7f24ec008030 comp rx=0 tx=0).stop 2023-11-28T11:11:50.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.940+0000 7f24fca5a700 1 -- 172.21.15.28:0/3410908820 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24f815c990 msgr2=0x7f24f814ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:50.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.940+0000 7f24fca5a700 1 --2- 172.21.15.28:0/3410908820 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24f815c990 0x7f24f814ce70 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f24e800d800 tx=0x7f24e800dbc0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.941+0000 7f24fca5a700 1 -- 172.21.15.28:0/3410908820 shutdown_connections 2023-11-28T11:11:50.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.941+0000 7f24fca5a700 1 --2- 172.21.15.28:0/3410908820 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24d80682f0 0x7f24d806a7a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.941+0000 7f24fca5a700 1 --2- 172.21.15.28:0/3410908820 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f24f8161090 0x7f24f815a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.941+0000 7f24fca5a700 1 --2- 172.21.15.28:0/3410908820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24f80caeb0 0x7f24f80c5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.941+0000 7f24fca5a700 1 --2- 172.21.15.28:0/3410908820 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f24f815c990 0x7f24f814ce70 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:50.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.941+0000 7f24fca5a700 1 -- 172.21.15.28:0/3410908820 >> 172.21.15.28:0/3410908820 conn(0x7f24f80b1650 msgr2=0x7f24f80ca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:50.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.941+0000 7f24fca5a700 1 -- 172.21.15.28:0/3410908820 shutdown_connections 2023-11-28T11:11:50.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:50.941+0000 7f24fca5a700 1 -- 172.21.15.28:0/3410908820 wait complete. 2023-11-28T11:11:50.944 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 25 2023-11-28T11:11:50.954 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:51.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.077+0000 7fc474073700 1 Processor -- start 2023-11-28T11:11:51.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.077+0000 7fc474073700 1 -- start start 2023-11-28T11:11:51.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.078+0000 7fc474073700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc46c06e400 0x7fc46c15db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.078+0000 7fc474073700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fc46c0ce890 0x7fc46c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.078+0000 7fc474073700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc46c15e0b0 con 0x7fc46c159590 2023-11-28T11:11:51.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.078+0000 7fc474073700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc46c0cf250 con 0x7fc46c06e400 2023-11-28T11:11:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.078+0000 7fc474073700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc46c15d120 con 0x7fc46c0ce890 2023-11-28T11:11:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.078+0000 7fc47160e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc46c06e400 0x7fc46c15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.078+0000 7fc47160e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fc46c06e400 0x7fc46c15db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51168/0 (socket says 172.21.15.28:51168) 2023-11-28T11:11:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.078+0000 7fc47160e700 1 -- 172.21.15.28:0/1937623993 learned_addr learned my addr 172.21.15.28:0/1937623993 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.079+0000 7fc472610700 1 --2- 172.21.15.28:0/1937623993 >> v2:172.21.15.59:3300/0 conn(0x7fc46c0ce890 0x7fc46c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.079+0000 7fc47160e700 1 -- 172.21.15.28:0/1937623993 >> v2:172.21.15.59:3300/0 conn(0x7fc46c0ce890 msgr2=0x7fc46c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.079+0000 7fc47160e700 1 --2- 172.21.15.28:0/1937623993 >> v2:172.21.15.59:3300/0 conn(0x7fc46c0ce890 0x7fc46c0ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.079+0000 7fc47160e700 1 -- 172.21.15.28:0/1937623993 >> v1:172.21.15.59:6789/0 conn(0x7fc46c159590 legacy=0x7fc46c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.079+0000 7fc47160e700 1 -- 172.21.15.28:0/1937623993 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46c15d6e0 con 0x7fc46c06e400 2023-11-28T11:11:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.079+0000 7fc47160e700 1 --2- 172.21.15.28:0/1937623993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc46c06e400 0x7fc46c15db70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0cd500 tx=0x7fc46800be70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fe22b2db37a8f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:51.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.079+0000 7fc470e0d700 1 -- 172.21.15.28:0/1937623993 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46800dc70 con 0x7fc46c06e400 2023-11-28T11:11:51.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.079+0000 7fc470e0d700 1 -- 172.21.15.28:0/1937623993 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc46800ddd0 con 0x7fc46c06e400 2023-11-28T11:11:51.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.079+0000 7fc470e0d700 1 -- 172.21.15.28:0/1937623993 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468010690 con 0x7fc46c06e400 2023-11-28T11:11:51.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.080+0000 7fc474073700 1 -- 172.21.15.28:0/1937623993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc46c06e400 msgr2=0x7fc46c15db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.080+0000 7fc474073700 1 --2- 172.21.15.28:0/1937623993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc46c06e400 0x7fc46c15db70 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7fc46c0cd500 tx=0x7fc46800be70 comp rx=0 tx=0).stop 2023-11-28T11:11:51.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.080+0000 7fc474073700 1 -- 172.21.15.28:0/1937623993 shutdown_connections 2023-11-28T11:11:51.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.080+0000 7fc474073700 1 --2- 172.21.15.28:0/1937623993 >> v2:172.21.15.59:3300/0 conn(0x7fc46c0ce890 0x7fc46c0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.080+0000 7fc474073700 1 --2- 172.21.15.28:0/1937623993 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc46c06e400 0x7fc46c15db70 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.080+0000 7fc474073700 1 -- 172.21.15.28:0/1937623993 >> 172.21.15.28:0/1937623993 conn(0x7fc46c0b1650 msgr2=0x7fc46c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:51.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.080+0000 7fc474073700 1 -- 172.21.15.28:0/1937623993 shutdown_connections 2023-11-28T11:11:51.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.080+0000 7fc474073700 1 -- 172.21.15.28:0/1937623993 wait complete. 2023-11-28T11:11:51.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.081+0000 7fc474073700 1 Processor -- start 2023-11-28T11:11:51.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.081+0000 7fc474073700 1 -- start start 2023-11-28T11:11:51.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.082+0000 7fc474073700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46c06e400 0x7fc46c0b2e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.082+0000 7fc474073700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc46c0ce890 0x7fc46c1ee9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.082+0000 7fc474073700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc46c159590 0x7fc46c0cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.082+0000 7fc474073700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c1eeee0 con 0x7fc46c0ce890 2023-11-28T11:11:51.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.082+0000 7fc474073700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c1ef280 con 0x7fc46c06e400 2023-11-28T11:11:51.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.082+0000 7fc474073700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46c0b3510 con 0x7fc46c159590 2023-11-28T11:11:51.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.082+0000 7fc47160e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc46c0ce890 0x7fc46c1ee9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.082+0000 7fc47160e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc46c0ce890 0x7fc46c1ee9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51174/0 (socket says 172.21.15.28:51174) 2023-11-28T11:11:51.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.082+0000 7fc47160e700 1 -- 172.21.15.28:0/2782805098 learned_addr learned my addr 172.21.15.28:0/2782805098 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.083+0000 7fc47160e700 1 -- 172.21.15.28:0/2782805098 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc46c159590 msgr2=0x7fc46c0cdef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.083+0000 7fc472610700 1 --2- 172.21.15.28:0/2782805098 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc46c159590 0x7fc46c0cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.083+0000 7fc471e0f700 1 --2- 172.21.15.28:0/2782805098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46c06e400 0x7fc46c0b2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.083+0000 7fc47160e700 1 --2- 172.21.15.28:0/2782805098 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc46c159590 0x7fc46c0cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.083+0000 7fc47160e700 1 -- 172.21.15.28:0/2782805098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46c06e400 msgr2=0x7fc46c0b2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.083+0000 7fc47160e700 1 --2- 172.21.15.28:0/2782805098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46c06e400 0x7fc46c0b2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.083+0000 7fc47160e700 1 -- 172.21.15.28:0/2782805098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc46800b7f0 con 0x7fc46c0ce890 2023-11-28T11:11:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.083+0000 7fc47160e700 1 --2- 172.21.15.28:0/2782805098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc46c0ce890 0x7fc46c1ee9a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc46800beb0 tx=0x7fc468004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.083+0000 7fc4627fc700 1 -- 172.21.15.28:0/2782805098 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc46801f070 con 0x7fc46c0ce890 2023-11-28T11:11:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.083+0000 7fc4627fc700 1 -- 172.21.15.28:0/2782805098 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4680039f0 con 0x7fc46c0ce890 2023-11-28T11:11:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.083+0000 7fc474073700 1 -- 172.21.15.28:0/2782805098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc46c0af240 con 0x7fc46c0ce890 2023-11-28T11:11:51.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.083+0000 7fc4627fc700 1 -- 172.21.15.28:0/2782805098 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc468003dc0 con 0x7fc46c0ce890 2023-11-28T11:11:51.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.083+0000 7fc474073700 1 -- 172.21.15.28:0/2782805098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46c0aef60 con 0x7fc46c0ce890 2023-11-28T11:11:51.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.084+0000 7fc474073700 1 -- 172.21.15.28:0/2782805098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46c15d350 con 0x7fc46c0ce890 2023-11-28T11:11:51.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.084+0000 7fc4627fc700 1 -- 172.21.15.28:0/2782805098 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc468019400 con 0x7fc46c0ce890 2023-11-28T11:11:51.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.085+0000 7fc4627fc700 1 --2- 172.21.15.28:0/2782805098 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc4500682f0 0x7fc45006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.085+0000 7fc4627fc700 1 -- 172.21.15.28:0/2782805098 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc468089380 con 0x7fc46c0ce890 2023-11-28T11:11:51.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.085+0000 7fc471e0f700 1 --2- 172.21.15.28:0/2782805098 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc4500682f0 0x7fc45006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.085+0000 7fc471e0f700 1 --2- 172.21.15.28:0/2782805098 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc4500682f0 0x7fc45006a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc45c007fd0 tx=0x7fc45c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:51.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.089+0000 7fc4627fc700 1 -- 172.21.15.28:0/2782805098 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc468055a40 con 0x7fc46c0ce890 2023-11-28T11:11:51.278 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:11:51.280 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:11:51.329 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.328+0000 7fc474073700 1 -- 172.21.15.28:0/2782805098 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc46c0675a0 con 0x7fc46c0ce890 2023-11-28T11:11:51.330 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.328+0000 7fc4627fc700 1 -- 172.21.15.28:0/2782805098 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+4645 (secure 0 0 0) 0x7fc468059060 con 0x7fc46c0ce890 2023-11-28T11:11:51.330 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:51.331 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":25,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5198,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/284515388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":284515388},{"type":"v1","addr":"172.21.15.59:6835","nonce":284515388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2160385010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2160385010},{"type":"v1","addr":"172.21.15.28:6836","nonce":2160385010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3775145347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3775145347},{"type":"v1","addr":"172.21.15.28:6837","nonce":3775145347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:10:59.771956+0000","modified":"2023-11-28T11:11:03.109382+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6054},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6054":{"gid":6054,"name":"d","rank":0,"incarnation":21,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6834/3261856585","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":3261856585},{"type":"v1","addr":"172.21.15.59:6834","nonce":3261856585}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-11-28T11:11:51.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.330+0000 7fc474073700 1 -- 172.21.15.28:0/2782805098 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc4500682f0 msgr2=0x7fc45006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.330+0000 7fc474073700 1 --2- 172.21.15.28:0/2782805098 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc4500682f0 0x7fc45006a7a0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc45c007fd0 tx=0x7fc45c009040 comp rx=0 tx=0).stop 2023-11-28T11:11:51.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.330+0000 7fc474073700 1 -- 172.21.15.28:0/2782805098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc46c0ce890 msgr2=0x7fc46c1ee9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.330+0000 7fc474073700 1 --2- 172.21.15.28:0/2782805098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc46c0ce890 0x7fc46c1ee9a0 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fc46800beb0 tx=0x7fc468004750 comp rx=0 tx=0).stop 2023-11-28T11:11:51.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.330+0000 7fc474073700 1 -- 172.21.15.28:0/2782805098 shutdown_connections 2023-11-28T11:11:51.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.330+0000 7fc474073700 1 --2- 172.21.15.28:0/2782805098 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc4500682f0 0x7fc45006a7a0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.330+0000 7fc474073700 1 --2- 172.21.15.28:0/2782805098 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc46c159590 0x7fc46c0cdef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.331+0000 7fc474073700 1 --2- 172.21.15.28:0/2782805098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc46c06e400 0x7fc46c0b2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.331+0000 7fc474073700 1 --2- 172.21.15.28:0/2782805098 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc46c0ce890 0x7fc46c1ee9a0 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.331+0000 7fc474073700 1 -- 172.21.15.28:0/2782805098 >> 172.21.15.28:0/2782805098 conn(0x7fc46c0b1650 msgr2=0x7fc46c153250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:51.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.331+0000 7fc474073700 1 -- 172.21.15.28:0/2782805098 shutdown_connections 2023-11-28T11:11:51.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.331+0000 7fc474073700 1 -- 172.21.15.28:0/2782805098 wait complete. 2023-11-28T11:11:51.336 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 25 2023-11-28T11:11:51.345 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:11:51.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.509+0000 7f126d86c700 1 Processor -- start 2023-11-28T11:11:51.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.509+0000 7f126d86c700 1 -- start start 2023-11-28T11:11:51.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f126d86c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f126815c990 0x7f12680cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f126d86c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f1268161090 0x7f12680ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f126d86c700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12680ce340 con 0x7f12680caeb0 2023-11-28T11:11:51.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f126d86c700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12680cebe0 con 0x7f126815c990 2023-11-28T11:11:51.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f126d86c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12680cb3f0 con 0x7f1268161090 2023-11-28T11:11:51.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f1266ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f126815c990 0x7f12680cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f12677fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f12680caeb0 0x7f12680cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39242/0 (socket says 172.21.15.28:39242) 2023-11-28T11:11:51.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f12677fe700 1 -- 172.21.15.28:0/2575201415 learned_addr learned my addr 172.21.15.28:0/2575201415 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:51.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f12667fc700 1 --2- 172.21.15.28:0/2575201415 >> v2:172.21.15.59:3300/0 conn(0x7f1268161090 0x7f12680ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f1266ffd700 1 -- 172.21.15.28:0/2575201415 >> v2:172.21.15.59:3300/0 conn(0x7f1268161090 msgr2=0x7f12680ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:51.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f1266ffd700 1 --2- 172.21.15.28:0/2575201415 >> v2:172.21.15.59:3300/0 conn(0x7f1268161090 0x7f12680ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f1266ffd700 1 -- 172.21.15.28:0/2575201415 >> v1:172.21.15.28:6789/0 conn(0x7f12680caeb0 legacy=0x7f12680cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f1266ffd700 1 -- 172.21.15.28:0/2575201415 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12680ce590 con 0x7f126815c990 2023-11-28T11:11:51.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f1265ffb700 1 -- 172.21.15.28:0/2575201415 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3168071829 0 0) 0x7f12680ce340 con 0x7f12680caeb0 2023-11-28T11:11:51.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f1266ffd700 1 --2- 172.21.15.28:0/2575201415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126815c990 0x7f12680cf070 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f126806ab20 tx=0x7f1258009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cebd68fcaf97e02 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:51.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f1265ffb700 1 -- 172.21.15.28:0/2575201415 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258011830 con 0x7f126815c990 2023-11-28T11:11:51.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f1265ffb700 1 -- 172.21.15.28:0/2575201415 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1258011990 con 0x7f126815c990 2023-11-28T11:11:51.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.510+0000 7f1265ffb700 1 -- 172.21.15.28:0/2575201415 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1258011d20 con 0x7f126815c990 2023-11-28T11:11:51.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.511+0000 7f126d86c700 1 -- 172.21.15.28:0/2575201415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126815c990 msgr2=0x7f12680cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.511+0000 7f126d86c700 1 --2- 172.21.15.28:0/2575201415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126815c990 0x7f12680cf070 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f126806ab20 tx=0x7f1258009b20 comp rx=0 tx=0).stop 2023-11-28T11:11:51.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.511+0000 7f126d86c700 1 -- 172.21.15.28:0/2575201415 shutdown_connections 2023-11-28T11:11:51.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.511+0000 7f126d86c700 1 --2- 172.21.15.28:0/2575201415 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126815c990 0x7f12680cf070 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.511+0000 7f126d86c700 1 --2- 172.21.15.28:0/2575201415 >> v2:172.21.15.59:3300/0 conn(0x7f1268161090 0x7f12680ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.511+0000 7f126d86c700 1 -- 172.21.15.28:0/2575201415 >> 172.21.15.28:0/2575201415 conn(0x7f12680b1650 msgr2=0x7f12680b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:51.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.511+0000 7f126d86c700 1 -- 172.21.15.28:0/2575201415 shutdown_connections 2023-11-28T11:11:51.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.511+0000 7f126d86c700 1 -- 172.21.15.28:0/2575201415 wait complete. 2023-11-28T11:11:51.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.512+0000 7f126d86c700 1 Processor -- start 2023-11-28T11:11:51.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.512+0000 7f126d86c700 1 -- start start 2023-11-28T11:11:51.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.513+0000 7f126d86c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12680caeb0 0x7f1268154f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.513+0000 7f126d86c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126815c990 0x7f1268156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.513+0000 7f126d86c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1268161090 0x7f12680c5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.513+0000 7f126d86c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12680c3b90 con 0x7f126815c990 2023-11-28T11:11:51.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.513+0000 7f126d86c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12680c5db0 con 0x7f12680caeb0 2023-11-28T11:11:51.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.513+0000 7f126d86c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12680c3260 con 0x7f1268161090 2023-11-28T11:11:51.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.513+0000 7f12667fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126815c990 0x7f1268156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.513+0000 7f12667fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126815c990 0x7f1268156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51188/0 (socket says 172.21.15.28:51188) 2023-11-28T11:11:51.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.513+0000 7f12667fc700 1 -- 172.21.15.28:0/2978495032 learned_addr learned my addr 172.21.15.28:0/2978495032 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:51.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.514+0000 7f1266ffd700 1 --2- 172.21.15.28:0/2978495032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12680caeb0 0x7f1268154f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.514+0000 7f12667fc700 1 -- 172.21.15.28:0/2978495032 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1268161090 msgr2=0x7f12680c5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.514+0000 7f12677fe700 1 --2- 172.21.15.28:0/2978495032 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1268161090 0x7f12680c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.514+0000 7f12667fc700 1 --2- 172.21.15.28:0/2978495032 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1268161090 0x7f12680c5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.514+0000 7f12667fc700 1 -- 172.21.15.28:0/2978495032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12680caeb0 msgr2=0x7f1268154f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.514+0000 7f12667fc700 1 --2- 172.21.15.28:0/2978495032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12680caeb0 0x7f1268154f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.514+0000 7f12667fc700 1 -- 172.21.15.28:0/2978495032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12580094a0 con 0x7f126815c990 2023-11-28T11:11:51.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.514+0000 7f1266ffd700 1 --2- 172.21.15.28:0/2978495032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12680caeb0 0x7f1268154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:51.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.514+0000 7f12667fc700 1 --2- 172.21.15.28:0/2978495032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126815c990 0x7f1268156250 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f125c00e810 tx=0x7f125c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:51.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.514+0000 7f124ffff700 1 -- 172.21.15.28:0/2978495032 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c013610 con 0x7f126815c990 2023-11-28T11:11:51.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.514+0000 7f124ffff700 1 -- 172.21.15.28:0/2978495032 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f125c013770 con 0x7f126815c990 2023-11-28T11:11:51.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.514+0000 7f126d86c700 1 -- 172.21.15.28:0/2978495032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12680cf5b0 con 0x7f126815c990 2023-11-28T11:11:51.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.514+0000 7f124ffff700 1 -- 172.21.15.28:0/2978495032 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f125c013b20 con 0x7f126815c990 2023-11-28T11:11:51.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.514+0000 7f126d86c700 1 -- 172.21.15.28:0/2978495032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12680ce870 con 0x7f126815c990 2023-11-28T11:11:51.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.515+0000 7f126d86c700 1 -- 172.21.15.28:0/2978495032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12680cf740 con 0x7f126815c990 2023-11-28T11:11:51.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.515+0000 7f124ffff700 1 -- 172.21.15.28:0/2978495032 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f125c023070 con 0x7f126815c990 2023-11-28T11:11:51.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.516+0000 7f124ffff700 1 --2- 172.21.15.28:0/2978495032 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f12540682f0 0x7f125406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.516+0000 7f124ffff700 1 -- 172.21.15.28:0/2978495032 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f125c085ac0 con 0x7f126815c990 2023-11-28T11:11:51.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.516+0000 7f1266ffd700 1 --2- 172.21.15.28:0/2978495032 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f12540682f0 0x7f125406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.516+0000 7f1266ffd700 1 --2- 172.21.15.28:0/2978495032 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f12540682f0 0x7f125406a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1258009b60 tx=0x7f1258008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:51.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.520+0000 7f124ffff700 1 -- 172.21.15.28:0/2978495032 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f125c052160 con 0x7f126815c990 2023-11-28T11:11:51.716 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.715+0000 7f126d86c700 1 -- 172.21.15.28:0/2978495032 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12680675a0 con 0x7f126815c990 2023-11-28T11:11:51.717 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.716+0000 7f124ffff700 1 -- 172.21.15.28:0/2978495032 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v54) v1 ==== 74+0+18074 (secure 0 0 0) 0x7f125c085020 con 0x7f126815c990 2023-11-28T11:11:51.717 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:51.718 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":54,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:11:00.773581+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:10:57.702079+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-11-28T11:10:58.738753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"54","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:11:51.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.718+0000 7f126d86c700 1 -- 172.21.15.28:0/2978495032 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f12540682f0 msgr2=0x7f125406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.718+0000 7f126d86c700 1 --2- 172.21.15.28:0/2978495032 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f12540682f0 0x7f125406a7a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1258009b60 tx=0x7f1258008030 comp rx=0 tx=0).stop 2023-11-28T11:11:51.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.718+0000 7f126d86c700 1 -- 172.21.15.28:0/2978495032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126815c990 msgr2=0x7f1268156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.718+0000 7f126d86c700 1 --2- 172.21.15.28:0/2978495032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126815c990 0x7f1268156250 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f125c00e810 tx=0x7f125c00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.718+0000 7f126d86c700 1 -- 172.21.15.28:0/2978495032 shutdown_connections 2023-11-28T11:11:51.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.718+0000 7f126d86c700 1 --2- 172.21.15.28:0/2978495032 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f12540682f0 0x7f125406a7a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.718+0000 7f126d86c700 1 --2- 172.21.15.28:0/2978495032 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1268161090 0x7f12680c5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.718+0000 7f126d86c700 1 --2- 172.21.15.28:0/2978495032 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f12680caeb0 0x7f1268154f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.718+0000 7f126d86c700 1 --2- 172.21.15.28:0/2978495032 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f126815c990 0x7f1268156250 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.718+0000 7f126d86c700 1 -- 172.21.15.28:0/2978495032 >> 172.21.15.28:0/2978495032 conn(0x7f12680b1650 msgr2=0x7f12680c7090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:51.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.718+0000 7f126d86c700 1 -- 172.21.15.28:0/2978495032 shutdown_connections 2023-11-28T11:11:51.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.718+0000 7f126d86c700 1 -- 172.21.15.28:0/2978495032 wait complete. 2023-11-28T11:11:51.732 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:11:51.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.856+0000 7f4d8847d700 1 Processor -- start 2023-11-28T11:11:51.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.856+0000 7f4d8847d700 1 -- start start 2023-11-28T11:11:51.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d8847d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d8006e400 0x7f4d8015db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d8847d700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f4d800ce890 0x7f4d800ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.861 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d8847d700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d8015e0b0 con 0x7f4d80159590 2023-11-28T11:11:51.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d8847d700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d800cf250 con 0x7f4d800ce890 2023-11-28T11:11:51.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d8847d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8015d120 con 0x7f4d8006e400 2023-11-28T11:11:51.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d86219700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4d80159590 0x7f4d800ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39252/0 (socket says 172.21.15.28:39252) 2023-11-28T11:11:51.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d86219700 1 -- 172.21.15.28:0/867398256 learned_addr learned my addr 172.21.15.28:0/867398256 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:51.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d86a1a700 1 --2- 172.21.15.28:0/867398256 >> v2:172.21.15.59:3300/0 conn(0x7f4d800ce890 0x7f4d800ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85a18700 1 --2- 172.21.15.28:0/867398256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d8006e400 0x7f4d8015db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 -- 172.21.15.28:0/867398256 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3463223604 0 0) 0x7f4d7c001110 con 0x7f4d80159590 2023-11-28T11:11:51.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 -- 172.21.15.28:0/867398256 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d6c0035b0 con 0x7f4d80159590 2023-11-28T11:11:51.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 -- 172.21.15.28:0/867398256 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3708873955 0 0) 0x7f4d6c0035b0 con 0x7f4d80159590 2023-11-28T11:11:51.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 -- 172.21.15.28:0/867398256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d8006e400 msgr2=0x7f4d8015db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:51.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 --2- 172.21.15.28:0/867398256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d8006e400 0x7f4d8015db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 -- 172.21.15.28:0/867398256 >> v2:172.21.15.59:3300/0 conn(0x7f4d800ce890 msgr2=0x7f4d800ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:51.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 --2- 172.21.15.28:0/867398256 >> v2:172.21.15.59:3300/0 conn(0x7f4d800ce890 0x7f4d800ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 -- 172.21.15.28:0/867398256 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8015d6e0 con 0x7f4d80159590 2023-11-28T11:11:51.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 -- 172.21.15.28:0/867398256 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f4d7c001800 con 0x7f4d80159590 2023-11-28T11:11:51.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 -- 172.21.15.28:0/867398256 >> v1:172.21.15.28:6789/0 conn(0x7f4d80159590 legacy=0x7f4d800ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 --2- 172.21.15.28:0/867398256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d6c003a00 0x7f4d6c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 --2- 172.21.15.28:0/867398256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d8006e400 0x7f4d6c004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 --2- 172.21.15.28:0/867398256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c004d90 0x7f4d6c005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 -- 172.21.15.28:0/867398256 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800cf250 con 0x7f4d6c003a00 2023-11-28T11:11:51.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 -- 172.21.15.28:0/867398256 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d8015d120 con 0x7f4d6c004d90 2023-11-28T11:11:51.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.857+0000 7f4d85217700 1 -- 172.21.15.28:0/867398256 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d6c00ba10 con 0x7f4d8006e400 2023-11-28T11:11:51.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.858+0000 7f4d86219700 1 --2- 172.21.15.28:0/867398256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d6c003a00 0x7f4d6c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.858+0000 7f4d85a18700 1 --2- 172.21.15.28:0/867398256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d8006e400 0x7f4d6c004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.858+0000 7f4d86a1a700 1 --2- 172.21.15.28:0/867398256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c004d90 0x7f4d6c005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.858+0000 7f4d86219700 1 -- 172.21.15.28:0/867398256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d8006e400 msgr2=0x7f4d6c004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:51.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.858+0000 7f4d86219700 1 --2- 172.21.15.28:0/867398256 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d8006e400 0x7f4d6c004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.858+0000 7f4d86219700 1 -- 172.21.15.28:0/867398256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c004d90 msgr2=0x7f4d6c005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:51.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.858+0000 7f4d86219700 1 --2- 172.21.15.28:0/867398256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c004d90 0x7f4d6c005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.858+0000 7f4d86219700 1 -- 172.21.15.28:0/867398256 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6c0035b0 con 0x7f4d6c003a00 2023-11-28T11:11:51.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.858+0000 7f4d86a1a700 1 --2- 172.21.15.28:0/867398256 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c004d90 0x7f4d6c005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:51.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.858+0000 7f4d86219700 1 --2- 172.21.15.28:0/867398256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d6c003a00 0x7f4d6c003e10 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00c240 tx=0x7f4d6c005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4ec6db1687ca2a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:51.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.858+0000 7f4d85217700 1 -- 172.21.15.28:0/867398256 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c007db0 con 0x7f4d6c003a00 2023-11-28T11:11:51.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.858+0000 7f4d85217700 1 -- 172.21.15.28:0/867398256 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7c00cd80 con 0x7f4d6c003a00 2023-11-28T11:11:51.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.858+0000 7f4d85217700 1 -- 172.21.15.28:0/867398256 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7c005b80 con 0x7f4d6c003a00 2023-11-28T11:11:51.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.859+0000 7f4d8847d700 1 -- 172.21.15.28:0/867398256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d6c003a00 msgr2=0x7f4d6c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.859+0000 7f4d8847d700 1 --2- 172.21.15.28:0/867398256 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d6c003a00 0x7f4d6c003e10 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f4d7c00c240 tx=0x7f4d6c005880 comp rx=0 tx=0).stop 2023-11-28T11:11:51.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.859+0000 7f4d86a1a700 1 -- 172.21.15.28:0/867398256 reap_dead start 2023-11-28T11:11:51.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.859+0000 7f4d8847d700 1 -- 172.21.15.28:0/867398256 shutdown_connections 2023-11-28T11:11:51.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.859+0000 7f4d8847d700 1 -- 172.21.15.28:0/867398256 >> 172.21.15.28:0/867398256 conn(0x7f4d800b1650 msgr2=0x7f4d800b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:51.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.859+0000 7f4d8847d700 1 -- 172.21.15.28:0/867398256 shutdown_connections 2023-11-28T11:11:51.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.859+0000 7f4d8847d700 1 -- 172.21.15.28:0/867398256 wait complete. 2023-11-28T11:11:51.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.860+0000 7f4d8847d700 1 Processor -- start 2023-11-28T11:11:51.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.860+0000 7f4d8847d700 1 -- start start 2023-11-28T11:11:51.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d8847d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c003a00 0x7f4d801f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d8847d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d80154a30 0x7f4d80154ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d8847d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d801f37b0 0x7f4d801f3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d8847d700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800cc520 con 0x7f4d80154a30 2023-11-28T11:11:51.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d8847d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800c9a30 con 0x7f4d6c003a00 2023-11-28T11:11:51.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d8847d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d800ab8c0 con 0x7f4d801f37b0 2023-11-28T11:11:51.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d85a18700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d80154a30 0x7f4d80154ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d85a18700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d80154a30 0x7f4d80154ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51208/0 (socket says 172.21.15.28:51208) 2023-11-28T11:11:51.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d85a18700 1 -- 172.21.15.28:0/777840238 learned_addr learned my addr 172.21.15.28:0/777840238 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:51.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d85a18700 1 -- 172.21.15.28:0/777840238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d801f37b0 msgr2=0x7f4d801f3c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d86a1a700 1 --2- 172.21.15.28:0/777840238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d801f37b0 0x7f4d801f3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d86219700 1 --2- 172.21.15.28:0/777840238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c003a00 0x7f4d801f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d85a18700 1 --2- 172.21.15.28:0/777840238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d801f37b0 0x7f4d801f3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d85a18700 1 -- 172.21.15.28:0/777840238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c003a00 msgr2=0x7f4d801f2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:51.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d85a18700 1 --2- 172.21.15.28:0/777840238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c003a00 0x7f4d801f2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:51.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d85a18700 1 -- 172.21.15.28:0/777840238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d800af240 con 0x7f4d80154a30 2023-11-28T11:11:51.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d86219700 1 --2- 172.21.15.28:0/777840238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c003a00 0x7f4d801f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:51.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.861+0000 7f4d85a18700 1 --2- 172.21.15.28:0/777840238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d80154a30 0x7f4d80154ea0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4d70011aa0 tx=0x7f4d70011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:51.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.862+0000 7f4d76ffd700 1 -- 172.21.15.28:0/777840238 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000f980 con 0x7f4d80154a30 2023-11-28T11:11:51.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.862+0000 7f4d76ffd700 1 -- 172.21.15.28:0/777840238 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7000fae0 con 0x7f4d80154a30 2023-11-28T11:11:51.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.862+0000 7f4d8847d700 1 -- 172.21.15.28:0/777840238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d800aef60 con 0x7f4d80154a30 2023-11-28T11:11:51.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.862+0000 7f4d76ffd700 1 -- 172.21.15.28:0/777840238 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7000fe90 con 0x7f4d80154a30 2023-11-28T11:11:51.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.862+0000 7f4d8847d700 1 -- 172.21.15.28:0/777840238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d8015d370 con 0x7f4d80154a30 2023-11-28T11:11:51.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.863+0000 7f4d8847d700 1 -- 172.21.15.28:0/777840238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d800675a0 con 0x7f4d80154a30 2023-11-28T11:11:51.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.863+0000 7f4d76ffd700 1 -- 172.21.15.28:0/777840238 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f4d70014440 con 0x7f4d80154a30 2023-11-28T11:11:51.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.863+0000 7f4d76ffd700 1 --2- 172.21.15.28:0/777840238 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4d640682f0 0x7f4d6406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:51.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.863+0000 7f4d76ffd700 1 -- 172.21.15.28:0/777840238 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f4d700866a0 con 0x7f4d80154a30 2023-11-28T11:11:51.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.863+0000 7f4d86219700 1 --2- 172.21.15.28:0/777840238 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4d640682f0 0x7f4d6406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:51.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.864+0000 7f4d86219700 1 --2- 172.21.15.28:0/777840238 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4d640682f0 0x7f4d6406a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4d800c93e0 tx=0x7f4d7c00a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:51.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:51.868+0000 7f4d76ffd700 1 -- 172.21.15.28:0/777840238 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4d70052e20 con 0x7f4d80154a30 2023-11-28T11:11:52.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.105+0000 7f4d8847d700 1 -- 172.21.15.28:0/777840238 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4d8006c1b0 con 0x7f4d80154a30 2023-11-28T11:11:52.283 INFO:tasks.ceph.mon.a.smithi028.stderr:2023-11-28T11:11:52.282+0000 7fde5abc8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:11:52.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.292+0000 7f4d76ffd700 1 -- 172.21.15.28:0/777840238 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v26) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4d70056440 con 0x7f4d80154a30 2023-11-28T11:11:52.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.294+0000 7f4d8847d700 1 -- 172.21.15.28:0/777840238 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4d640682f0 msgr2=0x7f4d6406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:52.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.294+0000 7f4d8847d700 1 --2- 172.21.15.28:0/777840238 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4d640682f0 0x7f4d6406a7a0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f4d800c93e0 tx=0x7f4d7c00a000 comp rx=0 tx=0).stop 2023-11-28T11:11:52.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.294+0000 7f4d8847d700 1 -- 172.21.15.28:0/777840238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d80154a30 msgr2=0x7f4d80154ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:52.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.294+0000 7f4d8847d700 1 --2- 172.21.15.28:0/777840238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d80154a30 0x7f4d80154ea0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f4d70011aa0 tx=0x7f4d70011e60 comp rx=0 tx=0).stop 2023-11-28T11:11:52.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.294+0000 7f4d8847d700 1 -- 172.21.15.28:0/777840238 shutdown_connections 2023-11-28T11:11:52.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.294+0000 7f4d8847d700 1 --2- 172.21.15.28:0/777840238 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f4d640682f0 0x7f4d6406a7a0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.294+0000 7f4d8847d700 1 --2- 172.21.15.28:0/777840238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f4d801f37b0 0x7f4d801f3c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.294+0000 7f4d8847d700 1 --2- 172.21.15.28:0/777840238 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4d6c003a00 0x7f4d801f2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.294+0000 7f4d8847d700 1 --2- 172.21.15.28:0/777840238 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4d80154a30 0x7f4d80154ea0 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.294+0000 7f4d8847d700 1 -- 172.21.15.28:0/777840238 >> 172.21.15.28:0/777840238 conn(0x7f4d800b1650 msgr2=0x7f4d800b0da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:52.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.294+0000 7f4d8847d700 1 -- 172.21.15.28:0/777840238 shutdown_connections 2023-11-28T11:11:52.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.294+0000 7f4d8847d700 1 -- 172.21.15.28:0/777840238 wait complete. 2023-11-28T11:11:52.299 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:11:52.299 INFO:tasks.ceph.mds.d.smithi059.stderr: -4589> 2023-11-28T11:11:05.341+0000 7fae4d557700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:11:52.300 INFO:tasks.ceph.mds.d.smithi059.stderr: -2974> 2023-11-28T11:11:13.909+0000 7fae49d50700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /parent_a/alpha auth v2 ap=1 DIRTYPARENT s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6246=0-4194304@1} caps={6246=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6246 | request=0 caps=1 dirtyparent=1 dirty=0 authpin=1 scrubqueue=0 0x562a8ff59b80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(9)0x10000000001:[<0x10000000000/alpha v2>,<0x1/parent_a v4>]//[]","error_str":"failed to decode on-disk backtrace (40 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:11:52.309 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:11:52.331 INFO:tasks.ceph.mds.d.smithi059.stdout:starting mds.d at 2023-11-28T11:11:52.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.436+0000 7fceff4a8700 1 Processor -- start 2023-11-28T11:11:52.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.437+0000 7fceff4a8700 1 -- start start 2023-11-28T11:11:52.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.437+0000 7fceff4a8700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcef8159500 0x7fcef80cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:52.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.437+0000 7fceff4a8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcef80b1d40 0x7fcef80ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:52.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.437+0000 7fceff4a8700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcef815e040 con 0x7fcef815dbf0 2023-11-28T11:11:52.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.437+0000 7fceff4a8700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcef80cede0 con 0x7fcef8159500 2023-11-28T11:11:52.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.437+0000 7fceff4a8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef80ceff0 con 0x7fcef80b1d40 2023-11-28T11:11:52.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.437+0000 7fcefca43700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fcef815dbf0 0x7fcef80b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39274/0 (socket says 172.21.15.28:39274) 2023-11-28T11:11:52.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.437+0000 7fcefca43700 1 -- 172.21.15.28:0/1999480509 learned_addr learned my addr 172.21.15.28:0/1999480509 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:52.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.438+0000 7fcefda45700 1 --2- 172.21.15.28:0/1999480509 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcef80b1d40 0x7fcef80ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:52.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.438+0000 7fcefd244700 1 --2- 172.21.15.28:0/1999480509 >> v2:172.21.15.59:3300/0 conn(0x7fcef8159500 0x7fcef80cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:52.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.438+0000 7fceebfff700 1 -- 172.21.15.28:0/1999480509 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1496474589 0 0) 0x7fcef815e040 con 0x7fcef815dbf0 2023-11-28T11:11:52.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.438+0000 7fceebfff700 1 -- 172.21.15.28:0/1999480509 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcee40035b0 con 0x7fcef815dbf0 2023-11-28T11:11:52.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.438+0000 7fcefda45700 1 -- 172.21.15.28:0/1999480509 >> v2:172.21.15.59:3300/0 conn(0x7fcef8159500 msgr2=0x7fcef80cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:52.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.438+0000 7fcefda45700 1 --2- 172.21.15.28:0/1999480509 >> v2:172.21.15.59:3300/0 conn(0x7fcef8159500 0x7fcef80cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.438+0000 7fcefda45700 1 -- 172.21.15.28:0/1999480509 >> v1:172.21.15.28:6789/0 conn(0x7fcef815dbf0 legacy=0x7fcef80b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:52.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.438+0000 7fcefda45700 1 -- 172.21.15.28:0/1999480509 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcef80cf2d0 con 0x7fcef80b1d40 2023-11-28T11:11:52.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.438+0000 7fcefd244700 1 --2- 172.21.15.28:0/1999480509 >> v2:172.21.15.59:3300/0 conn(0x7fcef8159500 0x7fcef80cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:52.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.438+0000 7fcefda45700 1 --2- 172.21.15.28:0/1999480509 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcef80b1d40 0x7fcef80ce8a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009d90 tx=0x7fcef000b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e189892d12f7677d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:52.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.438+0000 7fceebfff700 1 -- 172.21.15.28:0/1999480509 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef000e070 con 0x7fcef80b1d40 2023-11-28T11:11:52.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.438+0000 7fceebfff700 1 -- 172.21.15.28:0/1999480509 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcef000bb20 con 0x7fcef80b1d40 2023-11-28T11:11:52.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.438+0000 7fceebfff700 1 -- 172.21.15.28:0/1999480509 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcef0012430 con 0x7fcef80b1d40 2023-11-28T11:11:52.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.439+0000 7fceff4a8700 1 -- 172.21.15.28:0/1999480509 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcef80b1d40 msgr2=0x7fcef80ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:52.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.439+0000 7fceff4a8700 1 --2- 172.21.15.28:0/1999480509 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcef80b1d40 0x7fcef80ce8a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fcef0009d90 tx=0x7fcef000b330 comp rx=0 tx=0).stop 2023-11-28T11:11:52.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.439+0000 7fceff4a8700 1 -- 172.21.15.28:0/1999480509 shutdown_connections 2023-11-28T11:11:52.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.439+0000 7fceff4a8700 1 --2- 172.21.15.28:0/1999480509 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcef80b1d40 0x7fcef80ce8a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.439+0000 7fceff4a8700 1 --2- 172.21.15.28:0/1999480509 >> v2:172.21.15.59:3300/0 conn(0x7fcef8159500 0x7fcef80cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.439+0000 7fceff4a8700 1 -- 172.21.15.28:0/1999480509 >> 172.21.15.28:0/1999480509 conn(0x7fcef80cc630 msgr2=0x7fcef80cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:52.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.439+0000 7fceff4a8700 1 -- 172.21.15.28:0/1999480509 shutdown_connections 2023-11-28T11:11:52.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.439+0000 7fceff4a8700 1 -- 172.21.15.28:0/1999480509 wait complete. 2023-11-28T11:11:52.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.440+0000 7fceff4a8700 1 Processor -- start 2023-11-28T11:11:52.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.440+0000 7fceff4a8700 1 -- start start 2023-11-28T11:11:52.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fceff4a8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcef80b1d40 0x7fcef8154290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:52.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fceff4a8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef8159500 0x7fcef80ac4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:52.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fceff4a8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcef815dbf0 0x7fcef80ad4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:52.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fceff4a8700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef80ac0b0 con 0x7fcef80b1d40 2023-11-28T11:11:52.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fceff4a8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcef80ae220 con 0x7fcef8159500 2023-11-28T11:11:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fceff4a8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcef80adf40 con 0x7fcef815dbf0 2023-11-28T11:11:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fcefd244700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcef80b1d40 0x7fcef8154290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fcefd244700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcef80b1d40 0x7fcef8154290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51220/0 (socket says 172.21.15.28:51220) 2023-11-28T11:11:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fcefd244700 1 -- 172.21.15.28:0/4158380424 learned_addr learned my addr 172.21.15.28:0/4158380424 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fcefda45700 1 --2- 172.21.15.28:0/4158380424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcef815dbf0 0x7fcef80ad4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fcefca43700 1 --2- 172.21.15.28:0/4158380424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef8159500 0x7fcef80ac4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fcefd244700 1 -- 172.21.15.28:0/4158380424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcef815dbf0 msgr2=0x7fcef80ad4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fcefd244700 1 --2- 172.21.15.28:0/4158380424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcef815dbf0 0x7fcef80ad4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fcefd244700 1 -- 172.21.15.28:0/4158380424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef8159500 msgr2=0x7fcef80ac4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fcefd244700 1 --2- 172.21.15.28:0/4158380424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef8159500 0x7fcef80ac4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fcefd244700 1 -- 172.21.15.28:0/4158380424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec0094a0 con 0x7fcef80b1d40 2023-11-28T11:11:52.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.441+0000 7fcefca43700 1 --2- 172.21.15.28:0/4158380424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef8159500 0x7fcef80ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:52.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.442+0000 7fcefd244700 1 --2- 172.21.15.28:0/4158380424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcef80b1d40 0x7fcef8154290 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fceec00ebe0 tx=0x7fceec009ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:52.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.442+0000 7fcee9ffb700 1 -- 172.21.15.28:0/4158380424 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec00c910 con 0x7fcef80b1d40 2023-11-28T11:11:52.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.442+0000 7fcee9ffb700 1 -- 172.21.15.28:0/4158380424 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec00ca70 con 0x7fcef80b1d40 2023-11-28T11:11:52.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.442+0000 7fceff4a8700 1 -- 172.21.15.28:0/4158380424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcef0009a70 con 0x7fcef80b1d40 2023-11-28T11:11:52.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.442+0000 7fcee9ffb700 1 -- 172.21.15.28:0/4158380424 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec00cde0 con 0x7fcef80b1d40 2023-11-28T11:11:52.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.442+0000 7fceff4a8700 1 -- 172.21.15.28:0/4158380424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcef815d3f0 con 0x7fcef80b1d40 2023-11-28T11:11:52.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.443+0000 7fceff4a8700 1 -- 172.21.15.28:0/4158380424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcef815d110 con 0x7fcef80b1d40 2023-11-28T11:11:52.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.443+0000 7fcee9ffb700 1 -- 172.21.15.28:0/4158380424 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fceec019030 con 0x7fcef80b1d40 2023-11-28T11:11:52.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.444+0000 7fcee9ffb700 1 --2- 172.21.15.28:0/4158380424 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcedc0682f0 0x7fcedc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:52.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.444+0000 7fcee9ffb700 1 -- 172.21.15.28:0/4158380424 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6468+0+0 (secure 0 0 0) 0x7fceec085e90 con 0x7fcef80b1d40 2023-11-28T11:11:52.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.444+0000 7fcefca43700 1 --2- 172.21.15.28:0/4158380424 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcedc0682f0 0x7fcedc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:52.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.444+0000 7fcefca43700 1 --2- 172.21.15.28:0/4158380424 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcedc0682f0 0x7fcedc06a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcef80cb300 tx=0x7fcef4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:52.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.448+0000 7fcee9ffb700 1 -- 172.21.15.28:0/4158380424 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fceec052550 con 0x7fcef80b1d40 2023-11-28T11:11:52.689 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:52.688+0000 7fceff4a8700 1 -- 172.21.15.28:0/4158380424 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fcef815d6d0 con 0x7fcef80b1d40 2023-11-28T11:11:53.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.293+0000 7fcee9ffb700 1 -- 172.21.15.28:0/4158380424 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7fceec05b0a0 con 0x7fcef80b1d40 2023-11-28T11:11:53.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.296+0000 7fceff4a8700 1 -- 172.21.15.28:0/4158380424 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcedc0682f0 msgr2=0x7fcedc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:53.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.296+0000 7fceff4a8700 1 --2- 172.21.15.28:0/4158380424 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcedc0682f0 0x7fcedc06a7a0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fcef80cb300 tx=0x7fcef4009040 comp rx=0 tx=0).stop 2023-11-28T11:11:53.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.296+0000 7fceff4a8700 1 -- 172.21.15.28:0/4158380424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcef80b1d40 msgr2=0x7fcef8154290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:53.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.296+0000 7fceff4a8700 1 --2- 172.21.15.28:0/4158380424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcef80b1d40 0x7fcef8154290 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fceec00ebe0 tx=0x7fceec009ec0 comp rx=0 tx=0).stop 2023-11-28T11:11:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.296+0000 7fceff4a8700 1 -- 172.21.15.28:0/4158380424 shutdown_connections 2023-11-28T11:11:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.296+0000 7fceff4a8700 1 --2- 172.21.15.28:0/4158380424 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcedc0682f0 0x7fcedc06a7a0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.296+0000 7fceff4a8700 1 --2- 172.21.15.28:0/4158380424 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcef80b1d40 0x7fcef8154290 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.296+0000 7fceff4a8700 1 --2- 172.21.15.28:0/4158380424 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcef815dbf0 0x7fcef80ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.296+0000 7fceff4a8700 1 --2- 172.21.15.28:0/4158380424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcef8159500 0x7fcef80ac4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.296+0000 7fceff4a8700 1 -- 172.21.15.28:0/4158380424 >> 172.21.15.28:0/4158380424 conn(0x7fcef80cc630 msgr2=0x7fcef8158150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.297+0000 7fceff4a8700 1 -- 172.21.15.28:0/4158380424 shutdown_connections 2023-11-28T11:11:53.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.297+0000 7fceff4a8700 1 -- 172.21.15.28:0/4158380424 wait complete. 2023-11-28T11:11:53.313 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:11:53.450 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.449+0000 7f1e98fbd700 1 Processor -- start 2023-11-28T11:11:53.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.449+0000 7f1e98fbd700 1 -- start start 2023-11-28T11:11:53.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.450+0000 7f1e98fbd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e94161090 0x7f1e940ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:53.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.450+0000 7f1e98fbd700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e940ce340 con 0x7f1e940caeb0 2023-11-28T11:11:53.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.450+0000 7f1e98fbd700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e940cebe0 con 0x7f1e9415c990 2023-11-28T11:11:53.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.450+0000 7f1e98fbd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940cf780 con 0x7f1e94161090 2023-11-28T11:11:53.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.450+0000 7f1e92d9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f1e940caeb0 0x7f1e940cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39290/0 (socket says 172.21.15.28:39290) 2023-11-28T11:11:53.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.450+0000 7f1e92d9d700 1 -- 172.21.15.28:0/2215644363 learned_addr learned my addr 172.21.15.28:0/2215644363 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:53.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.450+0000 7f1e91d9b700 1 --2- 172.21.15.28:0/2215644363 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e94161090 0x7f1e940ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:53.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.450+0000 7f1e9159a700 1 -- 172.21.15.28:0/2215644363 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 729476327 0 0) 0x7f1e940ce340 con 0x7f1e940caeb0 2023-11-28T11:11:53.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.450+0000 7f1e9159a700 1 -- 172.21.15.28:0/2215644363 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e7c0035b0 con 0x7f1e940caeb0 2023-11-28T11:11:53.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.450+0000 7f1e9159a700 1 -- 172.21.15.28:0/2215644363 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2020972593 0 0) 0x7f1e940cebe0 con 0x7f1e9415c990 2023-11-28T11:11:53.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.450+0000 7f1e9159a700 1 -- 172.21.15.28:0/2215644363 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e940ce340 con 0x7f1e9415c990 2023-11-28T11:11:53.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.450+0000 7f1e91d9b700 1 -- 172.21.15.28:0/2215644363 >> v1:172.21.15.59:6789/0 conn(0x7f1e9415c990 legacy=0x7f1e940cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:53.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.450+0000 7f1e91d9b700 1 -- 172.21.15.28:0/2215644363 >> v1:172.21.15.28:6789/0 conn(0x7f1e940caeb0 legacy=0x7f1e940cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:53.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.450+0000 7f1e91d9b700 1 -- 172.21.15.28:0/2215644363 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e940cb3f0 con 0x7f1e94161090 2023-11-28T11:11:53.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.451+0000 7f1e91d9b700 1 --2- 172.21.15.28:0/2215644363 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e94161090 0x7f1e940ca970 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009870 tx=0x7f1e88009b80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab71ab60399066cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:53.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.451+0000 7f1e9159a700 1 -- 172.21.15.28:0/2215644363 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e88011830 con 0x7f1e94161090 2023-11-28T11:11:53.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.451+0000 7f1e9159a700 1 -- 172.21.15.28:0/2215644363 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e88011990 con 0x7f1e94161090 2023-11-28T11:11:53.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.451+0000 7f1e9159a700 1 -- 172.21.15.28:0/2215644363 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e88011d20 con 0x7f1e94161090 2023-11-28T11:11:53.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.451+0000 7f1e98fbd700 1 -- 172.21.15.28:0/2215644363 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e94161090 msgr2=0x7f1e940ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:53.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.451+0000 7f1e98fbd700 1 --2- 172.21.15.28:0/2215644363 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e94161090 0x7f1e940ca970 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f1e88009870 tx=0x7f1e88009b80 comp rx=0 tx=0).stop 2023-11-28T11:11:53.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.451+0000 7f1e98fbd700 1 -- 172.21.15.28:0/2215644363 shutdown_connections 2023-11-28T11:11:53.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.451+0000 7f1e98fbd700 1 --2- 172.21.15.28:0/2215644363 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e94161090 0x7f1e940ca970 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:53.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.451+0000 7f1e98fbd700 1 -- 172.21.15.28:0/2215644363 >> 172.21.15.28:0/2215644363 conn(0x7f1e940b1650 msgr2=0x7f1e940b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:53.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.451+0000 7f1e98fbd700 1 -- 172.21.15.28:0/2215644363 shutdown_connections 2023-11-28T11:11:53.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.451+0000 7f1e98fbd700 1 -- 172.21.15.28:0/2215644363 wait complete. 2023-11-28T11:11:53.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.453+0000 7f1e98fbd700 1 Processor -- start 2023-11-28T11:11:53.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.453+0000 7f1e98fbd700 1 -- start start 2023-11-28T11:11:53.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.453+0000 7f1e98fbd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e940caeb0 0x7f1e941f34b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:53.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.453+0000 7f1e98fbd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e9415c990 0x7f1e941f62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:53.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.453+0000 7f1e98fbd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e94161090 0x7f1e940c5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:53.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e98fbd700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940ad0c0 con 0x7f1e940caeb0 2023-11-28T11:11:53.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e98fbd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e941f6830 con 0x7f1e94161090 2023-11-28T11:11:53.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e98fbd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e940c3880 con 0x7f1e9415c990 2023-11-28T11:11:53.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e9259c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e940caeb0 0x7f1e941f34b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:53.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e9259c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e940caeb0 0x7f1e941f34b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51234/0 (socket says 172.21.15.28:51234) 2023-11-28T11:11:53.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e9259c700 1 -- 172.21.15.28:0/3140835738 learned_addr learned my addr 172.21.15.28:0/3140835738 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:53.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e91d9b700 1 --2- 172.21.15.28:0/3140835738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e9415c990 0x7f1e941f62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:53.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e92d9d700 1 --2- 172.21.15.28:0/3140835738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e94161090 0x7f1e940c5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:53.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e9259c700 1 -- 172.21.15.28:0/3140835738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e9415c990 msgr2=0x7f1e941f62f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:53.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e9259c700 1 --2- 172.21.15.28:0/3140835738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e9415c990 0x7f1e941f62f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:53.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e9259c700 1 -- 172.21.15.28:0/3140835738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e94161090 msgr2=0x7f1e940c5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:53.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e9259c700 1 --2- 172.21.15.28:0/3140835738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e94161090 0x7f1e940c5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:53.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e9259c700 1 -- 172.21.15.28:0/3140835738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e880094a0 con 0x7f1e940caeb0 2023-11-28T11:11:53.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e9259c700 1 --2- 172.21.15.28:0/3140835738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e940caeb0 0x7f1e941f34b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400c520 tx=0x7f1e8400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:53.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e82ffd700 1 -- 172.21.15.28:0/3140835738 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e840052d0 con 0x7f1e940caeb0 2023-11-28T11:11:53.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e82ffd700 1 -- 172.21.15.28:0/3140835738 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e84005430 con 0x7f1e940caeb0 2023-11-28T11:11:53.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e98fbd700 1 -- 172.21.15.28:0/3140835738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e940ce870 con 0x7f1e940caeb0 2023-11-28T11:11:53.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e82ffd700 1 -- 172.21.15.28:0/3140835738 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e840038c0 con 0x7f1e940caeb0 2023-11-28T11:11:53.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.454+0000 7f1e98fbd700 1 -- 172.21.15.28:0/3140835738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e940ce590 con 0x7f1e940caeb0 2023-11-28T11:11:53.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.455+0000 7f1e98fbd700 1 -- 172.21.15.28:0/3140835738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e940cca20 con 0x7f1e940caeb0 2023-11-28T11:11:53.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.456+0000 7f1e82ffd700 1 -- 172.21.15.28:0/3140835738 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f1e84019030 con 0x7f1e940caeb0 2023-11-28T11:11:53.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.456+0000 7f1e82ffd700 1 --2- 172.21.15.28:0/3140835738 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1e740682f0 0x7f1e7406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:53.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.456+0000 7f1e82ffd700 1 -- 172.21.15.28:0/3140835738 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6468+0+0 (secure 0 0 0) 0x7f1e84085120 con 0x7f1e940caeb0 2023-11-28T11:11:53.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.456+0000 7f1e91d9b700 1 --2- 172.21.15.28:0/3140835738 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1e740682f0 0x7f1e7406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:53.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.457+0000 7f1e91d9b700 1 --2- 172.21.15.28:0/3140835738 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1e740682f0 0x7f1e7406a7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1e941614c0 tx=0x7f1e8801c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:53.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.460+0000 7f1e82ffd700 1 -- 172.21.15.28:0/3140835738 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f1e840517e0 con 0x7f1e940caeb0 2023-11-28T11:11:53.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:53.652+0000 7f1e98fbd700 1 -- 172.21.15.28:0/3140835738 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1e940c3600 con 0x7f1e940caeb0 2023-11-28T11:11:54.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.352+0000 7f1e82ffd700 1 -- 172.21.15.28:0/3140835738 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v56) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1e84054e00 con 0x7f1e940caeb0 2023-11-28T11:11:54.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.354+0000 7f1e98fbd700 1 -- 172.21.15.28:0/3140835738 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1e740682f0 msgr2=0x7f1e7406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:54.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.354+0000 7f1e98fbd700 1 --2- 172.21.15.28:0/3140835738 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1e740682f0 0x7f1e7406a7a0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1e941614c0 tx=0x7f1e8801c040 comp rx=0 tx=0).stop 2023-11-28T11:11:54.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.354+0000 7f1e98fbd700 1 -- 172.21.15.28:0/3140835738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e940caeb0 msgr2=0x7f1e941f34b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.354+0000 7f1e98fbd700 1 --2- 172.21.15.28:0/3140835738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e940caeb0 0x7f1e941f34b0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f1e8400c520 tx=0x7f1e8400c830 comp rx=0 tx=0).stop 2023-11-28T11:11:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.354+0000 7f1e98fbd700 1 -- 172.21.15.28:0/3140835738 shutdown_connections 2023-11-28T11:11:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.354+0000 7f1e98fbd700 1 --2- 172.21.15.28:0/3140835738 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f1e740682f0 0x7f1e7406a7a0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.354+0000 7f1e98fbd700 1 --2- 172.21.15.28:0/3140835738 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f1e940caeb0 0x7f1e941f34b0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.354+0000 7f1e98fbd700 1 --2- 172.21.15.28:0/3140835738 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1e94161090 0x7f1e940c5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.354+0000 7f1e98fbd700 1 --2- 172.21.15.28:0/3140835738 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f1e9415c990 0x7f1e941f62f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.354+0000 7f1e98fbd700 1 -- 172.21.15.28:0/3140835738 >> 172.21.15.28:0/3140835738 conn(0x7f1e940b1650 msgr2=0x7f1e940b0ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.355+0000 7f1e98fbd700 1 -- 172.21.15.28:0/3140835738 shutdown_connections 2023-11-28T11:11:54.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.355+0000 7f1e98fbd700 1 -- 172.21.15.28:0/3140835738 wait complete. 2023-11-28T11:11:54.358 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:11:54.371 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:11:54.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.509+0000 7fde3f5f2700 1 Processor -- start 2023-11-28T11:11:54.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.509+0000 7fde3f5f2700 1 -- start start 2023-11-28T11:11:54.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.510+0000 7fde3f5f2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fde38161090 0x7fde380ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:54.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.510+0000 7fde3f5f2700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde380ce340 con 0x7fde3815c990 2023-11-28T11:11:54.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.510+0000 7fde3f5f2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde380cebe0 con 0x7fde380caeb0 2023-11-28T11:11:54.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.510+0000 7fde3f5f2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde380cf780 con 0x7fde38161090 2023-11-28T11:11:54.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.510+0000 7fde3d38e700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fde3815c990 0x7fde380cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39300/0 (socket says 172.21.15.28:39300) 2023-11-28T11:11:54.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.510+0000 7fde3d38e700 1 -- 172.21.15.28:0/2076785990 learned_addr learned my addr 172.21.15.28:0/2076785990 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:54.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.510+0000 7fde3cb8d700 1 --2- 172.21.15.28:0/2076785990 >> v2:172.21.15.59:3300/0 conn(0x7fde38161090 0x7fde380ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:54.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.510+0000 7fde2ffff700 1 -- 172.21.15.28:0/2076785990 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063546961 0 0) 0x7fde380ce340 con 0x7fde3815c990 2023-11-28T11:11:54.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.510+0000 7fde2ffff700 1 -- 172.21.15.28:0/2076785990 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde240035b0 con 0x7fde3815c990 2023-11-28T11:11:54.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 -- 172.21.15.28:0/2076785990 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3075353881 0 0) 0x7fde240035b0 con 0x7fde3815c990 2023-11-28T11:11:54.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 -- 172.21.15.28:0/2076785990 >> v2:172.21.15.59:3300/0 conn(0x7fde38161090 msgr2=0x7fde380ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:54.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 --2- 172.21.15.28:0/2076785990 >> v2:172.21.15.59:3300/0 conn(0x7fde38161090 0x7fde380ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:54.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 -- 172.21.15.28:0/2076785990 >> v1:172.21.15.59:6789/0 conn(0x7fde380caeb0 legacy=0x7fde380cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:54.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 -- 172.21.15.28:0/2076785990 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde380cb3f0 con 0x7fde3815c990 2023-11-28T11:11:54.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 -- 172.21.15.28:0/2076785990 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fde28001670 con 0x7fde3815c990 2023-11-28T11:11:54.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 -- 172.21.15.28:0/2076785990 >> v1:172.21.15.28:6789/0 conn(0x7fde3815c990 legacy=0x7fde380cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:54.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 --2- 172.21.15.28:0/2076785990 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde24003a00 0x7fde24003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:54.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 --2- 172.21.15.28:0/2076785990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24004850 0x7fde24004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:54.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 --2- 172.21.15.28:0/2076785990 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde24005200 0x7fde2400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:54.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 -- 172.21.15.28:0/2076785990 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde380cf780 con 0x7fde24003a00 2023-11-28T11:11:54.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 -- 172.21.15.28:0/2076785990 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2400c190 con 0x7fde24004850 2023-11-28T11:11:54.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 -- 172.21.15.28:0/2076785990 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2400c350 con 0x7fde24005200 2023-11-28T11:11:54.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde3d38e700 1 --2- 172.21.15.28:0/2076785990 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde24003a00 0x7fde24003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:54.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde3cb8d700 1 --2- 172.21.15.28:0/2076785990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24004850 0x7fde24004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:54.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde3db8f700 1 --2- 172.21.15.28:0/2076785990 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde24005200 0x7fde2400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:54.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde3d38e700 1 -- 172.21.15.28:0/2076785990 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde24005200 msgr2=0x7fde2400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:54.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde3d38e700 1 --2- 172.21.15.28:0/2076785990 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde24005200 0x7fde2400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:54.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde3d38e700 1 -- 172.21.15.28:0/2076785990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24004850 msgr2=0x7fde24004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:54.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde3d38e700 1 --2- 172.21.15.28:0/2076785990 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde24004850 0x7fde24004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:54.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde3d38e700 1 -- 172.21.15.28:0/2076785990 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde240035b0 con 0x7fde24003a00 2023-11-28T11:11:54.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde3db8f700 1 --2- 172.21.15.28:0/2076785990 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde24005200 0x7fde2400ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:54.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde3db8f700 1 -- 172.21.15.28:0/2076785990 reap_dead start 2023-11-28T11:11:54.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde3d38e700 1 --2- 172.21.15.28:0/2076785990 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde24003a00 0x7fde24003dd0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fde28001eb0 tx=0x7fde24005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=652e7e508a1564cd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:54.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 -- 172.21.15.28:0/2076785990 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2800ce00 con 0x7fde24003a00 2023-11-28T11:11:54.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 -- 172.21.15.28:0/2076785990 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde28005960 con 0x7fde24003a00 2023-11-28T11:11:54.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.511+0000 7fde2ffff700 1 -- 172.21.15.28:0/2076785990 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28005d10 con 0x7fde24003a00 2023-11-28T11:11:54.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.512+0000 7fde3f5f2700 1 -- 172.21.15.28:0/2076785990 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde24003a00 msgr2=0x7fde24003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:54.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.512+0000 7fde3f5f2700 1 --2- 172.21.15.28:0/2076785990 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde24003a00 0x7fde24003dd0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fde28001eb0 tx=0x7fde24005880 comp rx=0 tx=0).stop 2023-11-28T11:11:54.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.512+0000 7fde3f5f2700 1 -- 172.21.15.28:0/2076785990 shutdown_connections 2023-11-28T11:11:54.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.512+0000 7fde3f5f2700 1 --2- 172.21.15.28:0/2076785990 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde24003a00 0x7fde24003dd0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:54.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.512+0000 7fde3f5f2700 1 -- 172.21.15.28:0/2076785990 >> 172.21.15.28:0/2076785990 conn(0x7fde380b1650 msgr2=0x7fde380b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:54.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.512+0000 7fde3f5f2700 1 -- 172.21.15.28:0/2076785990 shutdown_connections 2023-11-28T11:11:54.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.512+0000 7fde3f5f2700 1 -- 172.21.15.28:0/2076785990 wait complete. 2023-11-28T11:11:54.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.513+0000 7fde3f5f2700 1 Processor -- start 2023-11-28T11:11:54.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.513+0000 7fde3f5f2700 1 -- start start 2023-11-28T11:11:54.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3f5f2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde24003a00 0x7fde381f1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:54.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3f5f2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde380ad800 0x7fde381f2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:54.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3f5f2700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde380ab8c0 0x7fde380abd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:54.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3f5f2700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde380ac270 con 0x7fde380ab8c0 2023-11-28T11:11:54.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3f5f2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde380adc70 con 0x7fde380ad800 2023-11-28T11:11:54.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3f5f2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde380aeab0 con 0x7fde24003a00 2023-11-28T11:11:54.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3db8f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde380ab8c0 0x7fde380abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:54.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3db8f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde380ab8c0 0x7fde380abd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51258/0 (socket says 172.21.15.28:51258) 2023-11-28T11:11:54.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3db8f700 1 -- 172.21.15.28:0/1273179523 learned_addr learned my addr 172.21.15.28:0/1273179523 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:54.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3cb8d700 1 --2- 172.21.15.28:0/1273179523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde380ad800 0x7fde381f2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:54.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3d38e700 1 --2- 172.21.15.28:0/1273179523 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde24003a00 0x7fde381f1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:54.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3db8f700 1 -- 172.21.15.28:0/1273179523 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde24003a00 msgr2=0x7fde381f1c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:54.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3db8f700 1 --2- 172.21.15.28:0/1273179523 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde24003a00 0x7fde381f1c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:54.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3db8f700 1 -- 172.21.15.28:0/1273179523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde380ad800 msgr2=0x7fde381f2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:54.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3db8f700 1 --2- 172.21.15.28:0/1273179523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde380ad800 0x7fde381f2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:54.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.514+0000 7fde3db8f700 1 -- 172.21.15.28:0/1273179523 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde380ce870 con 0x7fde380ab8c0 2023-11-28T11:11:54.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.515+0000 7fde3db8f700 1 --2- 172.21.15.28:0/1273179523 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde380ab8c0 0x7fde380abd30 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fde38071890 tx=0x7fde3000de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:54.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.515+0000 7fde2dffb700 1 -- 172.21.15.28:0/1273179523 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30003e10 con 0x7fde380ab8c0 2023-11-28T11:11:54.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.515+0000 7fde2dffb700 1 -- 172.21.15.28:0/1273179523 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde30005070 con 0x7fde380ab8c0 2023-11-28T11:11:54.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.515+0000 7fde3f5f2700 1 -- 172.21.15.28:0/1273179523 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde380ce590 con 0x7fde380ab8c0 2023-11-28T11:11:54.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.515+0000 7fde2dffb700 1 -- 172.21.15.28:0/1273179523 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde30005420 con 0x7fde380ab8c0 2023-11-28T11:11:54.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.515+0000 7fde3f5f2700 1 -- 172.21.15.28:0/1273179523 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde380ca970 con 0x7fde380ab8c0 2023-11-28T11:11:54.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.516+0000 7fde3f5f2700 1 -- 172.21.15.28:0/1273179523 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde380739f0 con 0x7fde380ab8c0 2023-11-28T11:11:54.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.516+0000 7fde2dffb700 1 -- 172.21.15.28:0/1273179523 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fde3001a030 con 0x7fde380ab8c0 2023-11-28T11:11:54.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.516+0000 7fde2dffb700 1 --2- 172.21.15.28:0/1273179523 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fde1c0682f0 0x7fde1c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:54.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.517+0000 7fde2dffb700 1 -- 172.21.15.28:0/1273179523 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6030+0+0 (secure 0 0 0) 0x7fde30085640 con 0x7fde380ab8c0 2023-11-28T11:11:54.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.517+0000 7fde3d38e700 1 --2- 172.21.15.28:0/1273179523 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fde1c0682f0 0x7fde1c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:54.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.517+0000 7fde3d38e700 1 --2- 172.21.15.28:0/1273179523 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fde1c0682f0 0x7fde1c06a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fde28000c00 tx=0x7fde28007000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:54.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.521+0000 7fde2dffb700 1 -- 172.21.15.28:0/1273179523 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fde30051ec0 con 0x7fde380ab8c0 2023-11-28T11:11:54.716 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:54.715+0000 7fde3f5f2700 1 -- 172.21.15.28:0/1273179523 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fde380aebf0 con 0x7fde380ab8c0 2023-11-28T11:11:55.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.358+0000 7fde2dffb700 1 -- 172.21.15.28:0/1273179523 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v57) v1 ==== 169+0+0 (secure 0 0 0) 0x7fde300180d0 con 0x7fde380ab8c0 2023-11-28T11:11:55.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.362+0000 7fde3f5f2700 1 -- 172.21.15.28:0/1273179523 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fde1c0682f0 msgr2=0x7fde1c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:55.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.362+0000 7fde3f5f2700 1 --2- 172.21.15.28:0/1273179523 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fde1c0682f0 0x7fde1c06a7a0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fde28000c00 tx=0x7fde28007000 comp rx=0 tx=0).stop 2023-11-28T11:11:55.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.362+0000 7fde3f5f2700 1 -- 172.21.15.28:0/1273179523 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde380ab8c0 msgr2=0x7fde380abd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:55.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.362+0000 7fde3f5f2700 1 --2- 172.21.15.28:0/1273179523 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde380ab8c0 0x7fde380abd30 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fde38071890 tx=0x7fde3000de40 comp rx=0 tx=0).stop 2023-11-28T11:11:55.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.363+0000 7fde3f5f2700 1 -- 172.21.15.28:0/1273179523 shutdown_connections 2023-11-28T11:11:55.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.363+0000 7fde3f5f2700 1 --2- 172.21.15.28:0/1273179523 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fde1c0682f0 0x7fde1c06a7a0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.363+0000 7fde3f5f2700 1 --2- 172.21.15.28:0/1273179523 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fde380ab8c0 0x7fde380abd30 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.363+0000 7fde3f5f2700 1 --2- 172.21.15.28:0/1273179523 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fde24003a00 0x7fde381f1c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.363+0000 7fde3f5f2700 1 --2- 172.21.15.28:0/1273179523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fde380ad800 0x7fde381f2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.363+0000 7fde3f5f2700 1 -- 172.21.15.28:0/1273179523 >> 172.21.15.28:0/1273179523 conn(0x7fde380b1650 msgr2=0x7fde3806c550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:55.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.363+0000 7fde3f5f2700 1 -- 172.21.15.28:0/1273179523 shutdown_connections 2023-11-28T11:11:55.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.363+0000 7fde3f5f2700 1 -- 172.21.15.28:0/1273179523 wait complete. 2023-11-28T11:11:55.366 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' removed 2023-11-28T11:11:55.382 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:11:55.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.514+0000 7f2ff55b5700 1 Processor -- start 2023-11-28T11:11:55.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.514+0000 7f2ff55b5700 1 -- start start 2023-11-28T11:11:55.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.515+0000 7f2ff55b5700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2ff015c640 0x7f2ff00cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:55.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.515+0000 7f2ff55b5700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff00b6350 con 0x7f2ff006bf30 2023-11-28T11:11:55.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.515+0000 7f2ff55b5700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff00d1470 con 0x7f2ff00d2820 2023-11-28T11:11:55.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.515+0000 7f2ff55b5700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff015ca10 con 0x7f2ff015c640 2023-11-28T11:11:55.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.515+0000 7f2feeffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2ff015c640 0x7f2ff00cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:55.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.515+0000 7f2fef7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2ff006bf30 0x7f2ff006c300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39324/0 (socket says 172.21.15.28:39324) 2023-11-28T11:11:55.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.515+0000 7f2fef7fe700 1 -- 172.21.15.28:0/842932914 learned_addr learned my addr 172.21.15.28:0/842932914 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:55.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.515+0000 7f2feeffd700 1 -- 172.21.15.28:0/842932914 >> v1:172.21.15.59:6789/0 conn(0x7f2ff00d2820 legacy=0x7f2ff00b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:55.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.515+0000 7f2feeffd700 1 -- 172.21.15.28:0/842932914 >> v1:172.21.15.28:6789/0 conn(0x7f2ff006bf30 legacy=0x7f2ff006c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:55.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.515+0000 7f2fedffb700 1 -- 172.21.15.28:0/842932914 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612891594 0 0) 0x7f2ff00b6350 con 0x7f2ff006bf30 2023-11-28T11:11:55.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.515+0000 7f2feeffd700 1 -- 172.21.15.28:0/842932914 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff00cd4e0 con 0x7f2ff015c640 2023-11-28T11:11:55.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.515+0000 7f2feeffd700 1 --2- 172.21.15.28:0/842932914 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ff015c640 0x7f2ff00cf040 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009840 tx=0x7f2fe0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c45f9aacafcc5548 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:55.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.515+0000 7f2fedffb700 1 -- 172.21.15.28:0/842932914 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0011830 con 0x7f2ff015c640 2023-11-28T11:11:55.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.515+0000 7f2fedffb700 1 -- 172.21.15.28:0/842932914 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe0011990 con 0x7f2ff015c640 2023-11-28T11:11:55.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.515+0000 7f2fedffb700 1 -- 172.21.15.28:0/842932914 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0011d20 con 0x7f2ff015c640 2023-11-28T11:11:55.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.516+0000 7f2ff55b5700 1 -- 172.21.15.28:0/842932914 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ff015c640 msgr2=0x7f2ff00cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:55.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.516+0000 7f2ff55b5700 1 --2- 172.21.15.28:0/842932914 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ff015c640 0x7f2ff00cf040 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0009840 tx=0x7f2fe0009b50 comp rx=0 tx=0).stop 2023-11-28T11:11:55.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.516+0000 7f2ff55b5700 1 -- 172.21.15.28:0/842932914 shutdown_connections 2023-11-28T11:11:55.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.516+0000 7f2ff55b5700 1 --2- 172.21.15.28:0/842932914 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ff015c640 0x7f2ff00cf040 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.516+0000 7f2ff55b5700 1 -- 172.21.15.28:0/842932914 >> 172.21.15.28:0/842932914 conn(0x7f2ff00d2df0 msgr2=0x7f2ff00d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:55.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.516+0000 7f2ff55b5700 1 -- 172.21.15.28:0/842932914 shutdown_connections 2023-11-28T11:11:55.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.516+0000 7f2ff55b5700 1 -- 172.21.15.28:0/842932914 wait complete. 2023-11-28T11:11:55.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.517+0000 7f2ff55b5700 1 Processor -- start 2023-11-28T11:11:55.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.517+0000 7f2ff55b5700 1 -- start start 2023-11-28T11:11:55.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.518+0000 7f2ff55b5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ff006bf30 0x7f2ff01559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:55.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.518+0000 7f2ff55b5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ff00d2820 0x7f2ff00c7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:55.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.518+0000 7f2ff55b5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ff015c640 0x7f2ff015f220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:55.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.518+0000 7f2ff55b5700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff015d700 con 0x7f2ff006bf30 2023-11-28T11:11:55.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.518+0000 7f2ff55b5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff015cb50 con 0x7f2ff015c640 2023-11-28T11:11:55.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.518+0000 7f2ff55b5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff015cd60 con 0x7f2ff00d2820 2023-11-28T11:11:55.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.518+0000 7f2feeffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ff006bf30 0x7f2ff01559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:55.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.518+0000 7f2feeffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ff006bf30 0x7f2ff01559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51268/0 (socket says 172.21.15.28:51268) 2023-11-28T11:11:55.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.518+0000 7f2feeffd700 1 -- 172.21.15.28:0/2730139636 learned_addr learned my addr 172.21.15.28:0/2730139636 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:55.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.518+0000 7f2fee7fc700 1 --2- 172.21.15.28:0/2730139636 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ff00d2820 0x7f2ff00c7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:55.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.518+0000 7f2fef7fe700 1 --2- 172.21.15.28:0/2730139636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ff015c640 0x7f2ff015f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:55.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.518+0000 7f2feeffd700 1 -- 172.21.15.28:0/2730139636 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ff00d2820 msgr2=0x7f2ff00c7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:55.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.519+0000 7f2feeffd700 1 --2- 172.21.15.28:0/2730139636 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ff00d2820 0x7f2ff00c7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.519+0000 7f2feeffd700 1 -- 172.21.15.28:0/2730139636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ff015c640 msgr2=0x7f2ff015f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:55.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.519+0000 7f2feeffd700 1 --2- 172.21.15.28:0/2730139636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ff015c640 0x7f2ff015f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.519+0000 7f2feeffd700 1 -- 172.21.15.28:0/2730139636 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe00094a0 con 0x7f2ff006bf30 2023-11-28T11:11:55.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.519+0000 7f2fef7fe700 1 --2- 172.21.15.28:0/2730139636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ff015c640 0x7f2ff015f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:11:55.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.519+0000 7f2feeffd700 1 --2- 172.21.15.28:0/2730139636 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ff006bf30 0x7f2ff01559d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0014040 tx=0x7f2fe0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:55.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.519+0000 7f2fd77fe700 1 -- 172.21.15.28:0/2730139636 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe001a900 con 0x7f2ff006bf30 2023-11-28T11:11:55.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.519+0000 7f2fd77fe700 1 -- 172.21.15.28:0/2730139636 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe001aa60 con 0x7f2ff006bf30 2023-11-28T11:11:55.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.519+0000 7f2ff55b5700 1 -- 172.21.15.28:0/2730139636 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff00d1710 con 0x7f2ff006bf30 2023-11-28T11:11:55.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.519+0000 7f2fd77fe700 1 -- 172.21.15.28:0/2730139636 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe001add0 con 0x7f2ff006bf30 2023-11-28T11:11:55.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.519+0000 7f2ff55b5700 1 -- 172.21.15.28:0/2730139636 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff00cd7c0 con 0x7f2ff006bf30 2023-11-28T11:11:55.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.520+0000 7f2ff55b5700 1 -- 172.21.15.28:0/2730139636 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff00cf580 con 0x7f2ff006bf30 2023-11-28T11:11:55.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.520+0000 7f2fd77fe700 1 -- 172.21.15.28:0/2730139636 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2fe002b030 con 0x7f2ff006bf30 2023-11-28T11:11:55.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.521+0000 7f2fd77fe700 1 --2- 172.21.15.28:0/2730139636 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2fdc0682f0 0x7f2fdc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:55.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.521+0000 7f2fee7fc700 1 --2- 172.21.15.28:0/2730139636 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2fdc0682f0 0x7f2fdc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:55.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.521+0000 7f2fd77fe700 1 -- 172.21.15.28:0/2730139636 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f2fe0086f40 con 0x7f2ff006bf30 2023-11-28T11:11:55.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.521+0000 7f2fee7fc700 1 --2- 172.21.15.28:0/2730139636 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2fdc0682f0 0x7f2fdc06a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2ff01f6680 tx=0x7f2fe4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:55.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.525+0000 7f2fd77fe700 1 -- 172.21.15.28:0/2730139636 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2fe0053960 con 0x7f2ff006bf30 2023-11-28T11:11:55.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.720+0000 7f2ff55b5700 1 -- 172.21.15.28:0/2730139636 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2ff015d440 con 0x7f2ff006bf30 2023-11-28T11:11:55.725 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.724+0000 7f2fd77fe700 1 -- 172.21.15.28:0/2730139636 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2fe0056f80 con 0x7f2ff006bf30 2023-11-28T11:11:55.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.726+0000 7f2ff55b5700 1 -- 172.21.15.28:0/2730139636 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2fdc0682f0 msgr2=0x7f2fdc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:55.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.726+0000 7f2ff55b5700 1 --2- 172.21.15.28:0/2730139636 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2fdc0682f0 0x7f2fdc06a7a0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f2ff01f6680 tx=0x7f2fe4009040 comp rx=0 tx=0).stop 2023-11-28T11:11:55.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.726+0000 7f2ff55b5700 1 -- 172.21.15.28:0/2730139636 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ff006bf30 msgr2=0x7f2ff01559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:55.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.726+0000 7f2ff55b5700 1 --2- 172.21.15.28:0/2730139636 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ff006bf30 0x7f2ff01559d0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f2fe0014040 tx=0x7f2fe0004750 comp rx=0 tx=0).stop 2023-11-28T11:11:55.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.726+0000 7f2ff55b5700 1 -- 172.21.15.28:0/2730139636 shutdown_connections 2023-11-28T11:11:55.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.726+0000 7f2ff55b5700 1 --2- 172.21.15.28:0/2730139636 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2fdc0682f0 0x7f2fdc06a7a0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.726+0000 7f2ff55b5700 1 --2- 172.21.15.28:0/2730139636 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ff006bf30 0x7f2ff01559d0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.726+0000 7f2ff55b5700 1 --2- 172.21.15.28:0/2730139636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ff015c640 0x7f2ff015f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.726+0000 7f2ff55b5700 1 --2- 172.21.15.28:0/2730139636 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ff00d2820 0x7f2ff00c7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.726+0000 7f2ff55b5700 1 -- 172.21.15.28:0/2730139636 >> 172.21.15.28:0/2730139636 conn(0x7f2ff00d2df0 msgr2=0x7f2ff006a1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:55.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.726+0000 7f2ff55b5700 1 -- 172.21.15.28:0/2730139636 shutdown_connections 2023-11-28T11:11:55.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.726+0000 7f2ff55b5700 1 -- 172.21.15.28:0/2730139636 wait complete. 2023-11-28T11:11:55.740 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair' 2023-11-28T11:11:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.866+0000 7f99811dd700 1 Processor -- start 2023-11-28T11:11:55.867 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.866+0000 7f99811dd700 1 -- start start 2023-11-28T11:11:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.866+0000 7f99811dd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f997c15c990 0x7f997c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.866+0000 7f99811dd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f997c161090 0x7f997c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f99811dd700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997c0ce340 con 0x7f997c0caeb0 2023-11-28T11:11:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f99811dd700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997c0cebe0 con 0x7f997c161090 2023-11-28T11:11:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f99811dd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997c0cb3f0 con 0x7f997c15c990 2023-11-28T11:11:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f997a59c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f997c161090 0x7f997c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f997a59c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f997c161090 0x7f997c0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51278/0 (socket says 172.21.15.28:51278) 2023-11-28T11:11:55.870 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f997a59c700 1 -- 172.21.15.28:0/3467205146 learned_addr learned my addr 172.21.15.28:0/3467205146 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f997b59e700 1 --1- 172.21.15.28:0/3467205146 >> v1:172.21.15.28:6789/0 conn(0x7f997c0caeb0 0x7f997c0cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39338/0 (socket says 172.21.15.28:39338) 2023-11-28T11:11:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f997ad9d700 1 --2- 172.21.15.28:0/3467205146 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f997c15c990 0x7f997c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f997a59c700 1 -- 172.21.15.28:0/3467205146 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f997c15c990 msgr2=0x7f997c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f997a59c700 1 --2- 172.21.15.28:0/3467205146 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f997c15c990 0x7f997c0cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f997a59c700 1 -- 172.21.15.28:0/3467205146 >> v1:172.21.15.28:6789/0 conn(0x7f997c0caeb0 legacy=0x7f997c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f997a59c700 1 -- 172.21.15.28:0/3467205146 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997c0ce590 con 0x7f997c161090 2023-11-28T11:11:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f9979d9b700 1 -- 172.21.15.28:0/3467205146 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858864658 0 0) 0x7f997c0ce340 con 0x7f997c0caeb0 2023-11-28T11:11:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f997a59c700 1 --2- 172.21.15.28:0/3467205146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f997c161090 0x7f997c0ca970 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f99700098f0 tx=0x7f9970009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2c0f011c6e0e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f9979d9b700 1 -- 172.21.15.28:0/3467205146 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997000b8a0 con 0x7f997c161090 2023-11-28T11:11:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f9979d9b700 1 -- 172.21.15.28:0/3467205146 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f997000ba00 con 0x7f997c161090 2023-11-28T11:11:55.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.867+0000 7f9979d9b700 1 -- 172.21.15.28:0/3467205146 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997000bd90 con 0x7f997c161090 2023-11-28T11:11:55.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.868+0000 7f99811dd700 1 -- 172.21.15.28:0/3467205146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f997c161090 msgr2=0x7f997c0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:55.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.868+0000 7f99811dd700 1 --2- 172.21.15.28:0/3467205146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f997c161090 0x7f997c0ca970 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f99700098f0 tx=0x7f9970009c00 comp rx=0 tx=0).stop 2023-11-28T11:11:55.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.868+0000 7f99811dd700 1 -- 172.21.15.28:0/3467205146 shutdown_connections 2023-11-28T11:11:55.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.868+0000 7f99811dd700 1 --2- 172.21.15.28:0/3467205146 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f997c15c990 0x7f997c0cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.868+0000 7f99811dd700 1 --2- 172.21.15.28:0/3467205146 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f997c161090 0x7f997c0ca970 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.868+0000 7f99811dd700 1 -- 172.21.15.28:0/3467205146 >> 172.21.15.28:0/3467205146 conn(0x7f997c0b1650 msgr2=0x7f997c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:55.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.868+0000 7f99811dd700 1 -- 172.21.15.28:0/3467205146 shutdown_connections 2023-11-28T11:11:55.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.868+0000 7f99811dd700 1 -- 172.21.15.28:0/3467205146 wait complete. 2023-11-28T11:11:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.869+0000 7f99811dd700 1 Processor -- start 2023-11-28T11:11:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.869+0000 7f99811dd700 1 -- start start 2023-11-28T11:11:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.870+0000 7f99811dd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f997c0caeb0 0x7f997c0abb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.870+0000 7f99811dd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f997c15c990 0x7f997c0b3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.870+0000 7f99811dd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f997c161090 0x7f997c0ae580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.870+0000 7f99811dd700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c0c4f90 con 0x7f997c0caeb0 2023-11-28T11:11:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.870+0000 7f99811dd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f997c0c43a0 con 0x7f997c161090 2023-11-28T11:11:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.870+0000 7f99811dd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f997c150db0 con 0x7f997c15c990 2023-11-28T11:11:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.870+0000 7f997ad9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f997c0caeb0 0x7f997c0abb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:55.875 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.870+0000 7f997ad9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f997c0caeb0 0x7f997c0abb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51284/0 (socket says 172.21.15.28:51284) 2023-11-28T11:11:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.870+0000 7f997ad9d700 1 -- 172.21.15.28:0/317717377 learned_addr learned my addr 172.21.15.28:0/317717377 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.871+0000 7f997a59c700 1 --2- 172.21.15.28:0/317717377 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f997c15c990 0x7f997c0b3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.871+0000 7f997b59e700 1 --2- 172.21.15.28:0/317717377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f997c161090 0x7f997c0ae580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.871+0000 7f997ad9d700 1 -- 172.21.15.28:0/317717377 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f997c15c990 msgr2=0x7f997c0b3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.871+0000 7f997ad9d700 1 --2- 172.21.15.28:0/317717377 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f997c15c990 0x7f997c0b3400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.871+0000 7f997ad9d700 1 -- 172.21.15.28:0/317717377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f997c161090 msgr2=0x7f997c0ae580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.871+0000 7f997ad9d700 1 --2- 172.21.15.28:0/317717377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f997c161090 0x7f997c0ae580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.871+0000 7f997ad9d700 1 -- 172.21.15.28:0/317717377 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9970009550 con 0x7f997c0caeb0 2023-11-28T11:11:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.871+0000 7f997ad9d700 1 --2- 172.21.15.28:0/317717377 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f997c0caeb0 0x7f997c0abb30 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f996c00e810 tx=0x7f996c00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.871+0000 7f99637fe700 1 -- 172.21.15.28:0/317717377 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c013610 con 0x7f997c0caeb0 2023-11-28T11:11:55.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.871+0000 7f99637fe700 1 -- 172.21.15.28:0/317717377 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c013770 con 0x7f997c0caeb0 2023-11-28T11:11:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.871+0000 7f99811dd700 1 -- 172.21.15.28:0/317717377 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f997c0cf5b0 con 0x7f997c0caeb0 2023-11-28T11:11:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.871+0000 7f99637fe700 1 -- 172.21.15.28:0/317717377 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c013b20 con 0x7f997c0caeb0 2023-11-28T11:11:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.871+0000 7f99811dd700 1 -- 172.21.15.28:0/317717377 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f997c0ce870 con 0x7f997c0caeb0 2023-11-28T11:11:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.872+0000 7f99811dd700 1 -- 172.21.15.28:0/317717377 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997c0cf740 con 0x7f997c0caeb0 2023-11-28T11:11:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.873+0000 7f99637fe700 1 -- 172.21.15.28:0/317717377 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f996c023070 con 0x7f997c0caeb0 2023-11-28T11:11:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.873+0000 7f99637fe700 1 --2- 172.21.15.28:0/317717377 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f99680682f0 0x7f996806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.873+0000 7f99637fe700 1 -- 172.21.15.28:0/317717377 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f996c085800 con 0x7f997c0caeb0 2023-11-28T11:11:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.873+0000 7f997a59c700 1 --2- 172.21.15.28:0/317717377 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f99680682f0 0x7f996806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:55.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.873+0000 7f997a59c700 1 --2- 172.21.15.28:0/317717377 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f99680682f0 0x7f996806a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f997c1614c0 tx=0x7f9970008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:55.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:55.877+0000 7f99637fe700 1 -- 172.21.15.28:0/317717377 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f996c0521a0 con 0x7f997c0caeb0 2023-11-28T11:11:56.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.151+0000 7f99811dd700 1 -- 172.21.15.28:0/317717377 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]} v 0) v1 -- 0x7f997c0c5200 con 0x7f997c0caeb0 2023-11-28T11:11:56.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.358+0000 7f99637fe700 1 -- 172.21.15.28:0/317717377 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_backtrace_repair"]}]=0 v179) v1 ==== 149+0+0 (secure 0 0 0) 0x7f996c0557c0 con 0x7f997c0caeb0 2023-11-28T11:11:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.360+0000 7f99811dd700 1 -- 172.21.15.28:0/317717377 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f99680682f0 msgr2=0x7f996806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.360+0000 7f99811dd700 1 --2- 172.21.15.28:0/317717377 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f99680682f0 0x7f996806a7a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f997c1614c0 tx=0x7f9970008030 comp rx=0 tx=0).stop 2023-11-28T11:11:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.360+0000 7f99811dd700 1 -- 172.21.15.28:0/317717377 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f997c0caeb0 msgr2=0x7f997c0abb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.360+0000 7f99811dd700 1 --2- 172.21.15.28:0/317717377 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f997c0caeb0 0x7f997c0abb30 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f996c00e810 tx=0x7f996c00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:11:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.360+0000 7f99811dd700 1 -- 172.21.15.28:0/317717377 shutdown_connections 2023-11-28T11:11:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.360+0000 7f99811dd700 1 --2- 172.21.15.28:0/317717377 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f99680682f0 0x7f996806a7a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.360+0000 7f99811dd700 1 --2- 172.21.15.28:0/317717377 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f997c0caeb0 0x7f997c0abb30 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.360+0000 7f99811dd700 1 --2- 172.21.15.28:0/317717377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f997c161090 0x7f997c0ae580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.360+0000 7f99811dd700 1 --2- 172.21.15.28:0/317717377 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f997c15c990 0x7f997c0b3400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:56.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.360+0000 7f99811dd700 1 -- 172.21.15.28:0/317717377 >> 172.21.15.28:0/317717377 conn(0x7f997c0b1650 msgr2=0x7f997c06c650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.360+0000 7f99811dd700 1 -- 172.21.15.28:0/317717377 shutdown_connections 2023-11-28T11:11:56.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.360+0000 7f99811dd700 1 -- 172.21.15.28:0/317717377 wait complete. 2023-11-28T11:11:56.374 INFO:tasks.cephfs_test_runner:That the MDS can repair an inodes backtrace in the data pool ... ok 2023-11-28T11:11:56.374 INFO:tasks.cephfs_test_runner:Starting test: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) Test that the damage health status is cleared 2023-11-28T11:11:56.374 INFO:tasks.cephfs_test_runner:test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:11:56.375 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T11:11:56.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.499+0000 7fe17f556700 1 Processor -- start 2023-11-28T11:11:56.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.499+0000 7fe17f556700 1 -- start start 2023-11-28T11:11:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.500+0000 7fe17f556700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe1780b2660 0x7fe1780ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.500+0000 7fe17f556700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe1780cecf0 0x7fe17806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.500+0000 7fe17f556700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1780b30b0 con 0x7fe17815c560 2023-11-28T11:11:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.500+0000 7fe17f556700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe17806c020 con 0x7fe1780cecf0 2023-11-28T11:11:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.500+0000 7fe17f556700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1780cf110 con 0x7fe1780b2660 2023-11-28T11:11:56.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.500+0000 7fe17daf3700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe1780cecf0 0x7fe17806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.500+0000 7fe17d2f2700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe17815c560 0x7fe17806f8b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39348/0 (socket says 172.21.15.28:39348) 2023-11-28T11:11:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.500+0000 7fe17daf3700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe1780cecf0 0x7fe17806c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51294/0 (socket says 172.21.15.28:51294) 2023-11-28T11:11:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.500+0000 7fe17d2f2700 1 -- 172.21.15.28:0/2841107984 learned_addr learned my addr 172.21.15.28:0/2841107984 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.500+0000 7fe17caf1700 1 --2- 172.21.15.28:0/2841107984 >> v2:172.21.15.59:3300/0 conn(0x7fe1780b2660 0x7fe1780ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.500+0000 7fe17daf3700 1 -- 172.21.15.28:0/2841107984 >> v2:172.21.15.59:3300/0 conn(0x7fe1780b2660 msgr2=0x7fe1780ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.500+0000 7fe17daf3700 1 --2- 172.21.15.28:0/2841107984 >> v2:172.21.15.59:3300/0 conn(0x7fe1780b2660 0x7fe1780ce7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.500+0000 7fe17daf3700 1 -- 172.21.15.28:0/2841107984 >> v1:172.21.15.28:6789/0 conn(0x7fe17815c560 legacy=0x7fe17806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.500+0000 7fe16ffff700 1 -- 172.21.15.28:0/2841107984 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391616397 0 0) 0x7fe1780b30b0 con 0x7fe17815c560 2023-11-28T11:11:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.500+0000 7fe17daf3700 1 -- 172.21.15.28:0/2841107984 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1780d09c0 con 0x7fe1780cecf0 2023-11-28T11:11:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.501+0000 7fe17daf3700 1 --2- 172.21.15.28:0/2841107984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1780cecf0 0x7fe17806c4b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe178071e10 tx=0x7fe17000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c35411747da5b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:56.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.501+0000 7fe16ffff700 1 -- 172.21.15.28:0/2841107984 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe17000e070 con 0x7fe1780cecf0 2023-11-28T11:11:56.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.501+0000 7fe16ffff700 1 -- 172.21.15.28:0/2841107984 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe17000bbd0 con 0x7fe1780cecf0 2023-11-28T11:11:56.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.501+0000 7fe16ffff700 1 -- 172.21.15.28:0/2841107984 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe170012430 con 0x7fe1780cecf0 2023-11-28T11:11:56.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.501+0000 7fe17f556700 1 -- 172.21.15.28:0/2841107984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1780cecf0 msgr2=0x7fe17806c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:56.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.501+0000 7fe17f556700 1 --2- 172.21.15.28:0/2841107984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1780cecf0 0x7fe17806c4b0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7fe178071e10 tx=0x7fe17000b330 comp rx=0 tx=0).stop 2023-11-28T11:11:56.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.501+0000 7fe17f556700 1 -- 172.21.15.28:0/2841107984 shutdown_connections 2023-11-28T11:11:56.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.501+0000 7fe17f556700 1 --2- 172.21.15.28:0/2841107984 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1780cecf0 0x7fe17806c4b0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:56.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.501+0000 7fe17f556700 1 --2- 172.21.15.28:0/2841107984 >> v2:172.21.15.59:3300/0 conn(0x7fe1780b2660 0x7fe1780ce7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:56.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.501+0000 7fe17f556700 1 -- 172.21.15.28:0/2841107984 >> 172.21.15.28:0/2841107984 conn(0x7fe1780cd660 msgr2=0x7fe1780cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:56.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.501+0000 7fe17f556700 1 -- 172.21.15.28:0/2841107984 shutdown_connections 2023-11-28T11:11:56.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.501+0000 7fe17f556700 1 -- 172.21.15.28:0/2841107984 wait complete. 2023-11-28T11:11:56.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.503+0000 7fe17f556700 1 Processor -- start 2023-11-28T11:11:56.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.503+0000 7fe17f556700 1 -- start start 2023-11-28T11:11:56.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.503+0000 7fe17f556700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1780b2660 0x7fe1780ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:56.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.503+0000 7fe17f556700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1780cecf0 0x7fe1780c46d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:56.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.503+0000 7fe17f556700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe17815c560 0x7fe1780c4c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:56.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.503+0000 7fe17f556700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1781f21c0 con 0x7fe1780b2660 2023-11-28T11:11:56.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.503+0000 7fe17f556700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1780c52b0 con 0x7fe1780cecf0 2023-11-28T11:11:56.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.503+0000 7fe17f556700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1780c5590 con 0x7fe17815c560 2023-11-28T11:11:56.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.503+0000 7fe17d2f2700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1780b2660 0x7fe1780ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:56.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.503+0000 7fe17d2f2700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1780b2660 0x7fe1780ae210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51296/0 (socket says 172.21.15.28:51296) 2023-11-28T11:11:56.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.503+0000 7fe17d2f2700 1 -- 172.21.15.28:0/2330337781 learned_addr learned my addr 172.21.15.28:0/2330337781 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:56.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.504+0000 7fe17caf1700 1 --2- 172.21.15.28:0/2330337781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1780cecf0 0x7fe1780c46d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:56.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.504+0000 7fe17daf3700 1 --2- 172.21.15.28:0/2330337781 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe17815c560 0x7fe1780c4c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:56.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.504+0000 7fe17d2f2700 1 -- 172.21.15.28:0/2330337781 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe17815c560 msgr2=0x7fe1780c4c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:56.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.504+0000 7fe17d2f2700 1 --2- 172.21.15.28:0/2330337781 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe17815c560 0x7fe1780c4c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:56.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.504+0000 7fe17d2f2700 1 -- 172.21.15.28:0/2330337781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1780cecf0 msgr2=0x7fe1780c46d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:56.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.504+0000 7fe17d2f2700 1 --2- 172.21.15.28:0/2330337781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1780cecf0 0x7fe1780c46d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:56.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.504+0000 7fe17d2f2700 1 -- 172.21.15.28:0/2330337781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe170009a70 con 0x7fe1780b2660 2023-11-28T11:11:56.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.504+0000 7fe17caf1700 1 --2- 172.21.15.28:0/2330337781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1780cecf0 0x7fe1780c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:56.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.504+0000 7fe17d2f2700 1 --2- 172.21.15.28:0/2330337781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1780b2660 0x7fe1780ae210 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe16800c550 tx=0x7fe16800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:56.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.504+0000 7fe16dffb700 1 -- 172.21.15.28:0/2330337781 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168004c80 con 0x7fe1780b2660 2023-11-28T11:11:56.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.504+0000 7fe16dffb700 1 -- 172.21.15.28:0/2330337781 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe168004e00 con 0x7fe1780b2660 2023-11-28T11:11:56.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.504+0000 7fe17f556700 1 -- 172.21.15.28:0/2330337781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1780b2dd0 con 0x7fe1780b2660 2023-11-28T11:11:56.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.504+0000 7fe16dffb700 1 -- 172.21.15.28:0/2330337781 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe168007310 con 0x7fe1780b2660 2023-11-28T11:11:56.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.504+0000 7fe17f556700 1 -- 172.21.15.28:0/2330337781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1780b2af0 con 0x7fe1780b2660 2023-11-28T11:11:56.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.505+0000 7fe17f556700 1 -- 172.21.15.28:0/2330337781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe17806bcb0 con 0x7fe1780b2660 2023-11-28T11:11:56.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.506+0000 7fe16dffb700 1 -- 172.21.15.28:0/2330337781 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe16801a020 con 0x7fe1780b2660 2023-11-28T11:11:56.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.506+0000 7fe16dffb700 1 --2- 172.21.15.28:0/2330337781 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe15c0682f0 0x7fe15c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:56.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.506+0000 7fe16dffb700 1 -- 172.21.15.28:0/2330337781 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fe168085680 con 0x7fe1780b2660 2023-11-28T11:11:56.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.506+0000 7fe17caf1700 1 --2- 172.21.15.28:0/2330337781 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe15c0682f0 0x7fe15c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:56.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.506+0000 7fe17caf1700 1 --2- 172.21.15.28:0/2330337781 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe15c0682f0 0x7fe15c06a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe1781ef9d0 tx=0x7fe174008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:56.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.510+0000 7fe16dffb700 1 -- 172.21.15.28:0/2330337781 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe1680520a0 con 0x7fe1780b2660 2023-11-28T11:11:56.790 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:56.789+0000 7fe17f556700 1 -- 172.21.15.28:0/2330337781 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fe1780675a0 con 0x7fe1780b2660 2023-11-28T11:11:57.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.359+0000 7fe16dffb700 1 -- 172.21.15.28:0/2330337781 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v180) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe1680556c0 con 0x7fe1780b2660 2023-11-28T11:11:57.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.362+0000 7fe17f556700 1 -- 172.21.15.28:0/2330337781 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe15c0682f0 msgr2=0x7fe15c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:57.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.362+0000 7fe17f556700 1 --2- 172.21.15.28:0/2330337781 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe15c0682f0 0x7fe15c06a7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fe1781ef9d0 tx=0x7fe174008040 comp rx=0 tx=0).stop 2023-11-28T11:11:57.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.362+0000 7fe17f556700 1 -- 172.21.15.28:0/2330337781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1780b2660 msgr2=0x7fe1780ae210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:57.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.362+0000 7fe17f556700 1 --2- 172.21.15.28:0/2330337781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1780b2660 0x7fe1780ae210 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fe16800c550 tx=0x7fe16800c910 comp rx=0 tx=0).stop 2023-11-28T11:11:57.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.362+0000 7fe17f556700 1 -- 172.21.15.28:0/2330337781 shutdown_connections 2023-11-28T11:11:57.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.362+0000 7fe17f556700 1 --2- 172.21.15.28:0/2330337781 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe15c0682f0 0x7fe15c06a7a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.362+0000 7fe17f556700 1 --2- 172.21.15.28:0/2330337781 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe1780b2660 0x7fe1780ae210 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.362+0000 7fe17f556700 1 --2- 172.21.15.28:0/2330337781 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe17815c560 0x7fe1780c4c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.362+0000 7fe17f556700 1 --2- 172.21.15.28:0/2330337781 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe1780cecf0 0x7fe1780c46d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.362+0000 7fe17f556700 1 -- 172.21.15.28:0/2330337781 >> 172.21.15.28:0/2330337781 conn(0x7fe1780cd660 msgr2=0x7fe1780b3a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.362+0000 7fe17f556700 1 -- 172.21.15.28:0/2330337781 shutdown_connections 2023-11-28T11:11:57.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.362+0000 7fe17f556700 1 -- 172.21.15.28:0/2330337781 wait complete. 2023-11-28T11:11:57.379 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T11:11:57.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.513+0000 7f9aedd69700 1 Processor -- start 2023-11-28T11:11:57.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.513+0000 7f9aedd69700 1 -- start start 2023-11-28T11:11:57.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9aedd69700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9ae815c640 0x7f9ae80cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:57.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9aedd69700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9ae80d2820 0x7f9ae80b5c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:57.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9aedd69700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ae806b5b0 con 0x7f9ae806bf30 2023-11-28T11:11:57.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9aedd69700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae80d1810 con 0x7f9ae80d2820 2023-11-28T11:11:57.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9aedd69700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ae815ca10 con 0x7f9ae815c640 2023-11-28T11:11:57.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9ae6ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9ae80d2820 0x7f9ae80b5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:57.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9ae6ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9ae80d2820 0x7f9ae80b5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51306/0 (socket says 172.21.15.28:51306) 2023-11-28T11:11:57.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9ae6ffd700 1 -- 172.21.15.28:0/1519058210 learned_addr learned my addr 172.21.15.28:0/1519058210 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:57.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9ae7fff700 1 --1- 172.21.15.28:0/1519058210 >> v1:172.21.15.28:6789/0 conn(0x7f9ae806bf30 0x7f9ae806c320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39366/0 (socket says 172.21.15.28:39366) 2023-11-28T11:11:57.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9ae77fe700 1 --2- 172.21.15.28:0/1519058210 >> v2:172.21.15.59:3300/0 conn(0x7f9ae815c640 0x7f9ae80cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:57.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9ae6ffd700 1 -- 172.21.15.28:0/1519058210 >> v2:172.21.15.59:3300/0 conn(0x7f9ae815c640 msgr2=0x7f9ae80cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:57.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9ae6ffd700 1 --2- 172.21.15.28:0/1519058210 >> v2:172.21.15.59:3300/0 conn(0x7f9ae815c640 0x7f9ae80cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9ae6ffd700 1 -- 172.21.15.28:0/1519058210 >> v1:172.21.15.28:6789/0 conn(0x7f9ae806bf30 legacy=0x7f9ae806c320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:57.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9ae6ffd700 1 -- 172.21.15.28:0/1519058210 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae80d1470 con 0x7f9ae80d2820 2023-11-28T11:11:57.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9ae67fc700 1 -- 172.21.15.28:0/1519058210 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385628841 0 0) 0x7f9ae806b5b0 con 0x7f9ae806bf30 2023-11-28T11:11:57.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9ae77fe700 1 --2- 172.21.15.28:0/1519058210 >> v2:172.21.15.59:3300/0 conn(0x7f9ae815c640 0x7f9ae80cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:57.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9ae6ffd700 1 --2- 172.21.15.28:0/1519058210 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ae80d2820 0x7f9ae80b5c20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9ae806d420 tx=0x7f9ad0009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85d6ba5991a9f3c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:57.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.515+0000 7f9ae67fc700 1 -- 172.21.15.28:0/1519058210 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000b870 con 0x7f9ae80d2820 2023-11-28T11:11:57.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.516+0000 7f9ae67fc700 1 -- 172.21.15.28:0/1519058210 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad000b9d0 con 0x7f9ae80d2820 2023-11-28T11:11:57.519 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.516+0000 7f9ae67fc700 1 -- 172.21.15.28:0/1519058210 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad000bd60 con 0x7f9ae80d2820 2023-11-28T11:11:57.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.516+0000 7f9aedd69700 1 -- 172.21.15.28:0/1519058210 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ae80d2820 msgr2=0x7f9ae80b5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:57.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.516+0000 7f9aedd69700 1 --2- 172.21.15.28:0/1519058210 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ae80d2820 0x7f9ae80b5c20 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7f9ae806d420 tx=0x7f9ad0009bd0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.516+0000 7f9aedd69700 1 -- 172.21.15.28:0/1519058210 shutdown_connections 2023-11-28T11:11:57.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.516+0000 7f9aedd69700 1 --2- 172.21.15.28:0/1519058210 >> v2:172.21.15.59:3300/0 conn(0x7f9ae815c640 0x7f9ae80cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.516+0000 7f9aedd69700 1 --2- 172.21.15.28:0/1519058210 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ae80d2820 0x7f9ae80b5c20 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.516+0000 7f9aedd69700 1 -- 172.21.15.28:0/1519058210 >> 172.21.15.28:0/1519058210 conn(0x7f9ae80d2df0 msgr2=0x7f9ae80d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:57.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.516+0000 7f9aedd69700 1 -- 172.21.15.28:0/1519058210 shutdown_connections 2023-11-28T11:11:57.520 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.516+0000 7f9aedd69700 1 -- 172.21.15.28:0/1519058210 wait complete. 2023-11-28T11:11:57.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.518+0000 7f9aedd69700 1 Processor -- start 2023-11-28T11:11:57.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.518+0000 7f9aedd69700 1 -- start start 2023-11-28T11:11:57.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9aedd69700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ae806bf30 0x7f9ae80d3430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:57.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9aedd69700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ae80d2820 0x7f9ae815dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:57.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9aedd69700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ae815c640 0x7f9ae80c6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:57.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9aedd69700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae815d710 con 0x7f9ae80d2820 2023-11-28T11:11:57.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9aedd69700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae806a770 con 0x7f9ae806bf30 2023-11-28T11:11:57.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9aedd69700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae806aa50 con 0x7f9ae815c640 2023-11-28T11:11:57.523 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9ae6ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ae80d2820 0x7f9ae815dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:57.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9ae6ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ae80d2820 0x7f9ae815dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51312/0 (socket says 172.21.15.28:51312) 2023-11-28T11:11:57.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9ae6ffd700 1 -- 172.21.15.28:0/4055312205 learned_addr learned my addr 172.21.15.28:0/4055312205 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:57.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9ae77fe700 1 --2- 172.21.15.28:0/4055312205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ae806bf30 0x7f9ae80d3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:57.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9ae7fff700 1 --2- 172.21.15.28:0/4055312205 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ae815c640 0x7f9ae80c6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:57.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9ae6ffd700 1 -- 172.21.15.28:0/4055312205 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ae815c640 msgr2=0x7f9ae80c6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:57.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9ae6ffd700 1 --2- 172.21.15.28:0/4055312205 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ae815c640 0x7f9ae80c6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9ae6ffd700 1 -- 172.21.15.28:0/4055312205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ae806bf30 msgr2=0x7f9ae80d3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:57.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9ae6ffd700 1 --2- 172.21.15.28:0/4055312205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ae806bf30 0x7f9ae80d3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9ae6ffd700 1 -- 172.21.15.28:0/4055312205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ad0009550 con 0x7f9ae80d2820 2023-11-28T11:11:57.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9ae77fe700 1 --2- 172.21.15.28:0/4055312205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ae806bf30 0x7f9ae80d3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:11:57.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.519+0000 7f9ae6ffd700 1 --2- 172.21.15.28:0/4055312205 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ae80d2820 0x7f9ae815dc10 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009c10 tx=0x7f9ad0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:57.524 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.520+0000 7f9adffff700 1 -- 172.21.15.28:0/4055312205 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad0004310 con 0x7f9ae80d2820 2023-11-28T11:11:57.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.520+0000 7f9adffff700 1 -- 172.21.15.28:0/4055312205 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ad001f070 con 0x7f9ae80d2820 2023-11-28T11:11:57.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.520+0000 7f9aedd69700 1 -- 172.21.15.28:0/4055312205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ae80cd7c0 con 0x7f9ae80d2820 2023-11-28T11:11:57.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.520+0000 7f9adffff700 1 -- 172.21.15.28:0/4055312205 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ad00036a0 con 0x7f9ae80d2820 2023-11-28T11:11:57.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.520+0000 7f9aedd69700 1 -- 172.21.15.28:0/4055312205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ae80cd560 con 0x7f9ae80d2820 2023-11-28T11:11:57.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.521+0000 7f9aedd69700 1 -- 172.21.15.28:0/4055312205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ae80cf580 con 0x7f9ae80d2820 2023-11-28T11:11:57.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.521+0000 7f9adffff700 1 -- 172.21.15.28:0/4055312205 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9ad000f050 con 0x7f9ae80d2820 2023-11-28T11:11:57.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.522+0000 7f9adffff700 1 --2- 172.21.15.28:0/4055312205 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ac80682f0 0x7f9ac806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:57.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.522+0000 7f9ae77fe700 1 --2- 172.21.15.28:0/4055312205 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ac80682f0 0x7f9ac806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:57.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.522+0000 7f9adffff700 1 -- 172.21.15.28:0/4055312205 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f9ad0086f60 con 0x7f9ae80d2820 2023-11-28T11:11:57.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.522+0000 7f9ae77fe700 1 --2- 172.21.15.28:0/4055312205 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ac80682f0 0x7f9ac806a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800c960 tx=0x7f9ad8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:57.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.526+0000 7f9adffff700 1 -- 172.21.15.28:0/4055312205 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9ad00b85e0 con 0x7f9ae80d2820 2023-11-28T11:11:57.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.718+0000 7f9aedd69700 1 -- 172.21.15.28:0/4055312205 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f9ae815d900 con 0x7f9ae80d2820 2023-11-28T11:11:57.723 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.722+0000 7f9adffff700 1 -- 172.21.15.28:0/4055312205 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f9ad00290c0 con 0x7f9ae80d2820 2023-11-28T11:11:57.725 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.724+0000 7f9aedd69700 1 -- 172.21.15.28:0/4055312205 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ac80682f0 msgr2=0x7f9ac806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:57.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.724+0000 7f9aedd69700 1 --2- 172.21.15.28:0/4055312205 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ac80682f0 0x7f9ac806a7a0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800c960 tx=0x7f9ad8008040 comp rx=0 tx=0).stop 2023-11-28T11:11:57.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.724+0000 7f9aedd69700 1 -- 172.21.15.28:0/4055312205 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ae80d2820 msgr2=0x7f9ae815dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:57.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.724+0000 7f9aedd69700 1 --2- 172.21.15.28:0/4055312205 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ae80d2820 0x7f9ae815dc10 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009c10 tx=0x7f9ad0004750 comp rx=0 tx=0).stop 2023-11-28T11:11:57.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.724+0000 7f9aedd69700 1 -- 172.21.15.28:0/4055312205 shutdown_connections 2023-11-28T11:11:57.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.724+0000 7f9aedd69700 1 --2- 172.21.15.28:0/4055312205 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ac80682f0 0x7f9ac806a7a0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.724+0000 7f9aedd69700 1 --2- 172.21.15.28:0/4055312205 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9ae815c640 0x7f9ae80c6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.724+0000 7f9aedd69700 1 --2- 172.21.15.28:0/4055312205 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ae806bf30 0x7f9ae80d3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.724+0000 7f9aedd69700 1 --2- 172.21.15.28:0/4055312205 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9ae80d2820 0x7f9ae815dc10 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:57.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.724+0000 7f9aedd69700 1 -- 172.21.15.28:0/4055312205 >> 172.21.15.28:0/4055312205 conn(0x7f9ae80d2df0 msgr2=0x7f9ae80b7c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:57.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.724+0000 7f9aedd69700 1 -- 172.21.15.28:0/4055312205 shutdown_connections 2023-11-28T11:11:57.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:57.724+0000 7f9aedd69700 1 -- 172.21.15.28:0/4055312205 wait complete. 2023-11-28T11:11:57.737 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:11:57.738 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:11:57.777 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:11:57.828 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:11:57.828 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:11:57.828 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:57.847 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:11:57.848 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:11:57.848 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:57.868 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3961 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:11:57.868 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:11:57.868 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:57.905 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:11:57.905 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:11:57.905 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:11:57.927 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3961 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:11:57.927 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T11:11:57.927 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:11:58.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.068+0000 7fa036593700 1 Processor -- start 2023-11-28T11:11:58.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.068+0000 7fa036593700 1 -- start start 2023-11-28T11:11:58.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.069+0000 7fa036593700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa03015c990 0x7fa0300cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.069+0000 7fa036593700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa0300cc310 0x7fa0300cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.069+0000 7fa036593700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0300ce340 con 0x7fa030161090 2023-11-28T11:11:58.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.069+0000 7fa036593700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0300cebe0 con 0x7fa0300cc310 2023-11-28T11:11:58.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.069+0000 7fa036593700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300cccf0 con 0x7fa03015c990 2023-11-28T11:11:58.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.069+0000 7fa02ffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa03015c990 0x7fa0300cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.069+0000 7fa034b30700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa0300cc310 0x7fa0300cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.069+0000 7fa02f7fe700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7fa030161090 0x7fa0300ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:50164/0 (socket says 172.21.15.28:50164) 2023-11-28T11:11:58.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.069+0000 7fa02f7fe700 1 -- 172.21.15.28:0/1389169543 learned_addr learned my addr 172.21.15.28:0/1389169543 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:58.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.069+0000 7fa02effd700 1 -- 172.21.15.28:0/1389169543 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 5493623 0 0) 0x7fa0300ce340 con 0x7fa030161090 2023-11-28T11:11:58.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.069+0000 7fa02effd700 1 -- 172.21.15.28:0/1389169543 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0180035b0 con 0x7fa030161090 2023-11-28T11:11:58.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.070+0000 7fa034b30700 1 -- 172.21.15.28:0/1389169543 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa03015c990 msgr2=0x7fa0300cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:58.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.070+0000 7fa034b30700 1 --2- 172.21.15.28:0/1389169543 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa03015c990 0x7fa0300cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.070+0000 7fa034b30700 1 -- 172.21.15.28:0/1389169543 >> v1:172.21.15.59:6789/0 conn(0x7fa030161090 legacy=0x7fa0300ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.070+0000 7fa034b30700 1 -- 172.21.15.28:0/1389169543 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0300cb3f0 con 0x7fa0300cc310 2023-11-28T11:11:58.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.070+0000 7fa02ffff700 1 --2- 172.21.15.28:0/1389169543 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa03015c990 0x7fa0300cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:58.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.070+0000 7fa034b30700 1 --2- 172.21.15.28:0/1389169543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0300cc310 0x7fa0300cc7b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa028009d90 tx=0x7fa02800b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=94ab0c5580c3b541 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:58.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.070+0000 7fa02effd700 1 -- 172.21.15.28:0/1389169543 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02800e070 con 0x7fa0300cc310 2023-11-28T11:11:58.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.070+0000 7fa02effd700 1 -- 172.21.15.28:0/1389169543 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02800bb20 con 0x7fa0300cc310 2023-11-28T11:11:58.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.070+0000 7fa02effd700 1 -- 172.21.15.28:0/1389169543 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa028012430 con 0x7fa0300cc310 2023-11-28T11:11:58.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.070+0000 7fa036593700 1 -- 172.21.15.28:0/1389169543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0300cc310 msgr2=0x7fa0300cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.070+0000 7fa036593700 1 --2- 172.21.15.28:0/1389169543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0300cc310 0x7fa0300cc7b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa028009d90 tx=0x7fa02800b330 comp rx=0 tx=0).stop 2023-11-28T11:11:58.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.070+0000 7fa036593700 1 -- 172.21.15.28:0/1389169543 shutdown_connections 2023-11-28T11:11:58.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.070+0000 7fa036593700 1 --2- 172.21.15.28:0/1389169543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa0300cc310 0x7fa0300cc7b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.070+0000 7fa036593700 1 --2- 172.21.15.28:0/1389169543 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa03015c990 0x7fa0300cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.071+0000 7fa036593700 1 -- 172.21.15.28:0/1389169543 >> 172.21.15.28:0/1389169543 conn(0x7fa0300b1650 msgr2=0x7fa0300b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:58.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.071+0000 7fa036593700 1 -- 172.21.15.28:0/1389169543 shutdown_connections 2023-11-28T11:11:58.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.071+0000 7fa036593700 1 -- 172.21.15.28:0/1389169543 wait complete. 2023-11-28T11:11:58.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.072+0000 7fa036593700 1 Processor -- start 2023-11-28T11:11:58.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.072+0000 7fa036593700 1 -- start start 2023-11-28T11:11:58.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa036593700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0300cc310 0x7fa0300c5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa036593700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa03015c990 0x7fa0300c3350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa036593700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa030161090 0x7fa0300c3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa036593700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300b3d20 con 0x7fa0300cc310 2023-11-28T11:11:58.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa036593700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300c3dd0 con 0x7fa03015c990 2023-11-28T11:11:58.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa036593700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0300c4be0 con 0x7fa030161090 2023-11-28T11:11:58.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa02ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0300cc310 0x7fa0300c5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa02ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0300cc310 0x7fa0300c5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51324/0 (socket says 172.21.15.28:51324) 2023-11-28T11:11:58.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa02ffff700 1 -- 172.21.15.28:0/3411084793 learned_addr learned my addr 172.21.15.28:0/3411084793 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:58.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa02f7fe700 1 --2- 172.21.15.28:0/3411084793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa03015c990 0x7fa0300c3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa034b30700 1 --2- 172.21.15.28:0/3411084793 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa030161090 0x7fa0300c3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa02ffff700 1 -- 172.21.15.28:0/3411084793 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa030161090 msgr2=0x7fa0300c3890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa02ffff700 1 --2- 172.21.15.28:0/3411084793 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa030161090 0x7fa0300c3890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa02ffff700 1 -- 172.21.15.28:0/3411084793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa03015c990 msgr2=0x7fa0300c3350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa02ffff700 1 --2- 172.21.15.28:0/3411084793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa03015c990 0x7fa0300c3350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa02ffff700 1 -- 172.21.15.28:0/3411084793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0200094a0 con 0x7fa0300cc310 2023-11-28T11:11:58.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa02ffff700 1 --2- 172.21.15.28:0/3411084793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0300cc310 0x7fa0300c5070 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa02000ebe0 tx=0x7fa02000eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:58.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.073+0000 7fa02cff9700 1 -- 172.21.15.28:0/3411084793 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa020009760 con 0x7fa0300cc310 2023-11-28T11:11:58.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.074+0000 7fa02cff9700 1 -- 172.21.15.28:0/3411084793 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa02000c850 con 0x7fa0300cc310 2023-11-28T11:11:58.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.074+0000 7fa036593700 1 -- 172.21.15.28:0/3411084793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa028009a70 con 0x7fa0300cc310 2023-11-28T11:11:58.080 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.074+0000 7fa02cff9700 1 -- 172.21.15.28:0/3411084793 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa02000cbe0 con 0x7fa0300cc310 2023-11-28T11:11:58.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.074+0000 7fa036593700 1 -- 172.21.15.28:0/3411084793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0300ce870 con 0x7fa0300cc310 2023-11-28T11:11:58.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.075+0000 7fa036593700 1 -- 172.21.15.28:0/3411084793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0300ce590 con 0x7fa0300cc310 2023-11-28T11:11:58.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.075+0000 7fa02cff9700 1 -- 172.21.15.28:0/3411084793 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa020019030 con 0x7fa0300cc310 2023-11-28T11:11:58.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.075+0000 7fa02cff9700 1 --2- 172.21.15.28:0/3411084793 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa01c0682f0 0x7fa01c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.075+0000 7fa02cff9700 1 -- 172.21.15.28:0/3411084793 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fa020085990 con 0x7fa0300cc310 2023-11-28T11:11:58.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.075+0000 7fa02f7fe700 1 --2- 172.21.15.28:0/3411084793 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa01c0682f0 0x7fa01c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.081 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.076+0000 7fa02f7fe700 1 --2- 172.21.15.28:0/3411084793 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa01c0682f0 0x7fa01c06a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa0300af1b0 tx=0x7fa024007dc0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:58.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.079+0000 7fa02cff9700 1 -- 172.21.15.28:0/3411084793 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa0200523b0 con 0x7fa0300cc310 2023-11-28T11:11:58.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.319+0000 7fa036593700 1 -- 172.21.15.28:0/3411084793 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0300cb080 con 0x7fa0300cc310 2023-11-28T11:11:58.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.320+0000 7fa02cff9700 1 -- 172.21.15.28:0/3411084793 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) v1 ==== 94+0+3526 (secure 0 0 0) 0x7fa020017de0 con 0x7fa0300cc310 2023-11-28T11:11:58.322 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:58.322 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":27,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5198,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/284515388","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":284515388},{"type":"v1","addr":"172.21.15.59:6835","nonce":284515388}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":5794,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/1480458186","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":1480458186},{"type":"v1","addr":"172.21.15.59:6834","nonce":1480458186}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6045,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2160385010","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2160385010},{"type":"v1","addr":"172.21.15.28:6836","nonce":2160385010}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20},{"gid":6048,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3775145347","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3775145347},{"type":"v1","addr":"172.21.15.28:6837","nonce":3775145347}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":20}],"filesystems":[]} 2023-11-28T11:11:58.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.322+0000 7fa036593700 1 -- 172.21.15.28:0/3411084793 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa01c0682f0 msgr2=0x7fa01c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.322+0000 7fa036593700 1 --2- 172.21.15.28:0/3411084793 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa01c0682f0 0x7fa01c06a7a0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fa0300af1b0 tx=0x7fa024007dc0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.322+0000 7fa036593700 1 -- 172.21.15.28:0/3411084793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0300cc310 msgr2=0x7fa0300c5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.322+0000 7fa036593700 1 --2- 172.21.15.28:0/3411084793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0300cc310 0x7fa0300c5070 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7fa02000ebe0 tx=0x7fa02000eef0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.322+0000 7fa036593700 1 -- 172.21.15.28:0/3411084793 shutdown_connections 2023-11-28T11:11:58.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.322+0000 7fa036593700 1 --2- 172.21.15.28:0/3411084793 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa01c0682f0 0x7fa01c06a7a0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.322+0000 7fa036593700 1 --2- 172.21.15.28:0/3411084793 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa0300cc310 0x7fa0300c5070 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.322+0000 7fa036593700 1 --2- 172.21.15.28:0/3411084793 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa030161090 0x7fa0300c3890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.322+0000 7fa036593700 1 --2- 172.21.15.28:0/3411084793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa03015c990 0x7fa0300c3350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.322+0000 7fa036593700 1 -- 172.21.15.28:0/3411084793 >> 172.21.15.28:0/3411084793 conn(0x7fa0300b1650 msgr2=0x7fa03006b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:58.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.322+0000 7fa036593700 1 -- 172.21.15.28:0/3411084793 shutdown_connections 2023-11-28T11:11:58.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.322+0000 7fa036593700 1 -- 172.21.15.28:0/3411084793 wait complete. 2023-11-28T11:11:58.326 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 27 2023-11-28T11:11:58.336 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T11:11:58.336 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T11:11:58.336 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:11:58.336 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:11:58.336 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T11:11:58.336 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T11:11:58.337 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:11:58.337 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:11:58.337 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T11:11:58.337 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T11:11:58.337 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:11:58.337 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:11:58.337 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T11:11:58.338 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T11:11:58.338 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:11:58.338 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:11:58.345 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:11:58.345 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T11:11:58.346 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:11:58.346 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T11:11:58.352 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:11:58.352 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T11:11:58.353 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:11:58.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T11:11:58.388 INFO:tasks.ceph.mds.a:Started 2023-11-28T11:11:58.388 DEBUG:teuthology.parallel:result is None 2023-11-28T11:11:58.388 INFO:tasks.ceph.mds.c:Started 2023-11-28T11:11:58.389 DEBUG:teuthology.parallel:result is None 2023-11-28T11:11:58.394 INFO:tasks.ceph.mds.b:Started 2023-11-28T11:11:58.395 DEBUG:teuthology.parallel:result is None 2023-11-28T11:11:58.395 INFO:tasks.ceph.mds.d:Started 2023-11-28T11:11:58.395 DEBUG:teuthology.parallel:result is None 2023-11-28T11:11:58.396 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T11:11:58.469 INFO:tasks.ceph.mds.c.smithi028.stdout:starting mds.c at 2023-11-28T11:11:58.469 INFO:tasks.ceph.mds.a.smithi028.stdout:starting mds.a at 2023-11-28T11:11:58.487 INFO:tasks.ceph.mds.b.smithi059.stdout:starting mds.b at 2023-11-28T11:11:58.488 INFO:tasks.ceph.mds.d.smithi059.stdout:starting mds.d at 2023-11-28T11:11:58.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.529+0000 7f7e11a84700 1 Processor -- start 2023-11-28T11:11:58.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.529+0000 7f7e11a84700 1 -- start start 2023-11-28T11:11:58.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e11a84700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7e0c06e400 0x7f7e0c15db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e11a84700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7e0c0ce890 0x7f7e0c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e11a84700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e0c15e0b0 con 0x7f7e0c159590 2023-11-28T11:11:58.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e11a84700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e0c0cf250 con 0x7f7e0c0ce890 2023-11-28T11:11:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e11a84700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c15d120 con 0x7f7e0c06e400 2023-11-28T11:11:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e0b7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7e0c159590 0x7f7e0c0ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39414/0 (socket says 172.21.15.28:39414) 2023-11-28T11:11:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e0b7fe700 1 -- 172.21.15.28:0/2339070312 learned_addr learned my addr 172.21.15.28:0/2339070312 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e0bfff700 1 --2- 172.21.15.28:0/2339070312 >> v2:172.21.15.59:3300/0 conn(0x7f7e0c0ce890 0x7f7e0c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e0affd700 1 --2- 172.21.15.28:0/2339070312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7e0c06e400 0x7f7e0c15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e0a7fc700 1 -- 172.21.15.28:0/2339070312 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877563012 0 0) 0x7f7e00001110 con 0x7f7e0c159590 2023-11-28T11:11:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e0a7fc700 1 -- 172.21.15.28:0/2339070312 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7df40035b0 con 0x7f7e0c159590 2023-11-28T11:11:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e0bfff700 1 -- 172.21.15.28:0/2339070312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7e0c06e400 msgr2=0x7f7e0c15db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e0bfff700 1 --2- 172.21.15.28:0/2339070312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7e0c06e400 0x7f7e0c15db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e0bfff700 1 -- 172.21.15.28:0/2339070312 >> v1:172.21.15.28:6789/0 conn(0x7f7e0c159590 legacy=0x7f7e0c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e0bfff700 1 -- 172.21.15.28:0/2339070312 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e0c15d6e0 con 0x7f7e0c0ce890 2023-11-28T11:11:58.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e0affd700 1 --2- 172.21.15.28:0/2339070312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7e0c06e400 0x7f7e0c15db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:58.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e0bfff700 1 --2- 172.21.15.28:0/2339070312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e0c0ce890 0x7f7e0c0ced10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009d90 tx=0x7f7e0400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7264f073b83a9b1e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:58.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e0a7fc700 1 -- 172.21.15.28:0/2339070312 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e0400e070 con 0x7f7e0c0ce890 2023-11-28T11:11:58.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e0a7fc700 1 -- 172.21.15.28:0/2339070312 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e0400bb20 con 0x7f7e0c0ce890 2023-11-28T11:11:58.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e0a7fc700 1 -- 172.21.15.28:0/2339070312 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e04012430 con 0x7f7e0c0ce890 2023-11-28T11:11:58.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e11a84700 1 -- 172.21.15.28:0/2339070312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e0c0ce890 msgr2=0x7f7e0c0ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e11a84700 1 --2- 172.21.15.28:0/2339070312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e0c0ce890 0x7f7e0c0ced10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7e04009d90 tx=0x7f7e0400b330 comp rx=0 tx=0).stop 2023-11-28T11:11:58.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e11a84700 1 -- 172.21.15.28:0/2339070312 shutdown_connections 2023-11-28T11:11:58.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e11a84700 1 --2- 172.21.15.28:0/2339070312 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e0c0ce890 0x7f7e0c0ced10 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e11a84700 1 --2- 172.21.15.28:0/2339070312 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7e0c06e400 0x7f7e0c15db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e11a84700 1 -- 172.21.15.28:0/2339070312 >> 172.21.15.28:0/2339070312 conn(0x7f7e0c0b1650 msgr2=0x7f7e0c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:58.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e11a84700 1 -- 172.21.15.28:0/2339070312 shutdown_connections 2023-11-28T11:11:58.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.531+0000 7f7e11a84700 1 -- 172.21.15.28:0/2339070312 wait complete. 2023-11-28T11:11:58.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.532+0000 7f7e11a84700 1 Processor -- start 2023-11-28T11:11:58.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.533+0000 7f7e11a84700 1 -- start start 2023-11-28T11:11:58.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.533+0000 7f7e11a84700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7e0c06e400 0x7f7e0c155150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.533+0000 7f7e11a84700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e0c0ce890 0x7f7e0c0b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.533+0000 7f7e11a84700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e0c159590 0x7f7e0c154540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.533+0000 7f7e11a84700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c154a80 con 0x7f7e0c0ce890 2023-11-28T11:11:58.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.533+0000 7f7e11a84700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c154f50 con 0x7f7e0c159590 2023-11-28T11:11:58.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.533+0000 7f7e11a84700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0c0b3050 con 0x7f7e0c06e400 2023-11-28T11:11:58.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7e0affd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e0c0ce890 0x7f7e0c0b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7e0affd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e0c0ce890 0x7f7e0c0b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51364/0 (socket says 172.21.15.28:51364) 2023-11-28T11:11:58.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7e0affd700 1 -- 172.21.15.28:0/3582717910 learned_addr learned my addr 172.21.15.28:0/3582717910 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:58.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7e0bfff700 1 --2- 172.21.15.28:0/3582717910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e0c159590 0x7f7e0c154540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7e0b7fe700 1 --2- 172.21.15.28:0/3582717910 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7e0c06e400 0x7f7e0c155150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7e0affd700 1 -- 172.21.15.28:0/3582717910 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7e0c06e400 msgr2=0x7f7e0c155150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7e0affd700 1 --2- 172.21.15.28:0/3582717910 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7e0c06e400 0x7f7e0c155150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7e0affd700 1 -- 172.21.15.28:0/3582717910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e0c159590 msgr2=0x7f7e0c154540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:58.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7e0affd700 1 --2- 172.21.15.28:0/3582717910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e0c159590 0x7f7e0c154540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:58.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7e0affd700 1 -- 172.21.15.28:0/3582717910 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc0094a0 con 0x7f7e0c0ce890 2023-11-28T11:11:58.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7e0bfff700 1 --2- 172.21.15.28:0/3582717910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e0c159590 0x7f7e0c154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:11:58.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7e0affd700 1 --2- 172.21.15.28:0/3582717910 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e0c0ce890 0x7f7e0c0b2950 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00ebe0 tx=0x7f7dfc00eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:58.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7df3fff700 1 -- 172.21.15.28:0/3582717910 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc00c850 con 0x7f7e0c0ce890 2023-11-28T11:11:58.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7df3fff700 1 -- 172.21.15.28:0/3582717910 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc00c9b0 con 0x7f7e0c0ce890 2023-11-28T11:11:58.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7e11a84700 1 -- 172.21.15.28:0/3582717910 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e04009a70 con 0x7f7e0c0ce890 2023-11-28T11:11:58.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7df3fff700 1 -- 172.21.15.28:0/3582717910 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc00cd40 con 0x7f7e0c0ce890 2023-11-28T11:11:58.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.534+0000 7f7e11a84700 1 -- 172.21.15.28:0/3582717910 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e0c0af240 con 0x7f7e0c0ce890 2023-11-28T11:11:58.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.535+0000 7f7e11a84700 1 -- 172.21.15.28:0/3582717910 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0c0aef60 con 0x7f7e0c0ce890 2023-11-28T11:11:58.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.536+0000 7f7df3fff700 1 -- 172.21.15.28:0/3582717910 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7dfc019030 con 0x7f7e0c0ce890 2023-11-28T11:11:58.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.537+0000 7f7df3fff700 1 --2- 172.21.15.28:0/3582717910 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7df80682f0 0x7f7df806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:58.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.537+0000 7f7df3fff700 1 -- 172.21.15.28:0/3582717910 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f7dfc086240 con 0x7f7e0c0ce890 2023-11-28T11:11:58.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.537+0000 7f7e0b7fe700 1 --2- 172.21.15.28:0/3582717910 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7df80682f0 0x7f7df806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:58.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.537+0000 7f7e0b7fe700 1 --2- 172.21.15.28:0/3582717910 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7df80682f0 0x7f7df806a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7e00007fd0 tx=0x7f7e00007c40 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:58.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.540+0000 7f7df3fff700 1 -- 172.21.15.28:0/3582717910 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7dfc052c10 con 0x7f7e0c0ce890 2023-11-28T11:11:58.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:58.753+0000 7f7e11a84700 1 -- 172.21.15.28:0/3582717910 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f7e0c15d370 con 0x7f7e0c0ce890 2023-11-28T11:11:59.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.517+0000 7f7df3fff700 1 -- 172.21.15.28:0/3582717910 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v62) v1 ==== 97+0+0 (secure 0 0 0) 0x7f7dfc00cea0 con 0x7f7e0c0ce890 2023-11-28T11:11:59.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.520+0000 7f7e11a84700 1 -- 172.21.15.28:0/3582717910 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7df80682f0 msgr2=0x7f7df806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.520+0000 7f7e11a84700 1 --2- 172.21.15.28:0/3582717910 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7df80682f0 0x7f7df806a7a0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f7e00007fd0 tx=0x7f7e00007c40 comp rx=0 tx=0).stop 2023-11-28T11:11:59.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.520+0000 7f7e11a84700 1 -- 172.21.15.28:0/3582717910 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e0c0ce890 msgr2=0x7f7e0c0b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.520+0000 7f7e11a84700 1 --2- 172.21.15.28:0/3582717910 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e0c0ce890 0x7f7e0c0b2950 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc00ebe0 tx=0x7f7dfc00eef0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.520+0000 7f7e11a84700 1 -- 172.21.15.28:0/3582717910 shutdown_connections 2023-11-28T11:11:59.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.520+0000 7f7e11a84700 1 --2- 172.21.15.28:0/3582717910 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7df80682f0 0x7f7df806a7a0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.520+0000 7f7e11a84700 1 --2- 172.21.15.28:0/3582717910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7e0c159590 0x7f7e0c154540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.520+0000 7f7e11a84700 1 --2- 172.21.15.28:0/3582717910 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7e0c06e400 0x7f7e0c155150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.520+0000 7f7e11a84700 1 --2- 172.21.15.28:0/3582717910 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7e0c0ce890 0x7f7e0c0b2950 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.520+0000 7f7e11a84700 1 -- 172.21.15.28:0/3582717910 >> 172.21.15.28:0/3582717910 conn(0x7f7e0c0b1650 msgr2=0x7f7e0c0af480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:59.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.520+0000 7f7e11a84700 1 -- 172.21.15.28:0/3582717910 shutdown_connections 2023-11-28T11:11:59.522 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.520+0000 7f7e11a84700 1 -- 172.21.15.28:0/3582717910 wait complete. 2023-11-28T11:11:59.522 INFO:teuthology.orchestra.run.smithi028.stderr: removed all blocklist entries 2023-11-28T11:11:59.535 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T11:11:59.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.666+0000 7f05c16a6700 1 Processor -- start 2023-11-28T11:11:59.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.667+0000 7f05c16a6700 1 -- start start 2023-11-28T11:11:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.667+0000 7f05c16a6700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f05bc15c890 0x7f05bc0b2660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.667+0000 7f05c16a6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05bc0ceec0 0x7f05bc15d760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.667+0000 7f05c16a6700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05bc15d2d0 con 0x7f05bc160d30 2023-11-28T11:11:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.667+0000 7f05c16a6700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05bc0b2ba0 con 0x7f05bc15c890 2023-11-28T11:11:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.667+0000 7f05c16a6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc15dca0 con 0x7f05bc0ceec0 2023-11-28T11:11:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.668+0000 7f05baffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f05bc15c890 0x7f05bc0b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.668+0000 7f05ba7fc700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f05bc160d30 0x7f05bc0ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39432/0 (socket says 172.21.15.28:39432) 2023-11-28T11:11:59.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.668+0000 7f05ba7fc700 1 -- 172.21.15.28:0/128372300 learned_addr learned my addr 172.21.15.28:0/128372300 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:11:59.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.668+0000 7f05bb7fe700 1 --2- 172.21.15.28:0/128372300 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05bc0ceec0 0x7f05bc15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.668+0000 7f05baffd700 1 -- 172.21.15.28:0/128372300 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05bc0ceec0 msgr2=0x7f05bc15d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:11:59.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.668+0000 7f05baffd700 1 --2- 172.21.15.28:0/128372300 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05bc0ceec0 0x7f05bc15d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.668+0000 7f05baffd700 1 -- 172.21.15.28:0/128372300 >> v1:172.21.15.28:6789/0 conn(0x7f05bc160d30 legacy=0x7f05bc0ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.668+0000 7f05b9ffb700 1 -- 172.21.15.28:0/128372300 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2294434374 0 0) 0x7f05bc15d2d0 con 0x7f05bc160d30 2023-11-28T11:11:59.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.668+0000 7f05baffd700 1 -- 172.21.15.28:0/128372300 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05bc1611a0 con 0x7f05bc15c890 2023-11-28T11:11:59.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.668+0000 7f05baffd700 1 --2- 172.21.15.28:0/128372300 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05bc15c890 0x7f05bc0b2660 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009840 tx=0x7f05ac009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca1d3805605fba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:59.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.668+0000 7f05b9ffb700 1 -- 172.21.15.28:0/128372300 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac011830 con 0x7f05bc15c890 2023-11-28T11:11:59.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.668+0000 7f05b9ffb700 1 -- 172.21.15.28:0/128372300 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac011990 con 0x7f05bc15c890 2023-11-28T11:11:59.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.668+0000 7f05b9ffb700 1 -- 172.21.15.28:0/128372300 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac011d20 con 0x7f05bc15c890 2023-11-28T11:11:59.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.669+0000 7f05c16a6700 1 -- 172.21.15.28:0/128372300 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05bc15c890 msgr2=0x7f05bc0b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.671 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.669+0000 7f05c16a6700 1 --2- 172.21.15.28:0/128372300 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05bc15c890 0x7f05bc0b2660 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f05ac009840 tx=0x7f05ac009b50 comp rx=0 tx=0).stop 2023-11-28T11:11:59.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.669+0000 7f05c16a6700 1 -- 172.21.15.28:0/128372300 shutdown_connections 2023-11-28T11:11:59.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.669+0000 7f05c16a6700 1 --2- 172.21.15.28:0/128372300 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05bc0ceec0 0x7f05bc15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.669+0000 7f05c16a6700 1 --2- 172.21.15.28:0/128372300 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05bc15c890 0x7f05bc0b2660 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.669+0000 7f05c16a6700 1 -- 172.21.15.28:0/128372300 >> 172.21.15.28:0/128372300 conn(0x7f05bc06e9b0 msgr2=0x7f05bc0cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:59.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.669+0000 7f05c16a6700 1 -- 172.21.15.28:0/128372300 shutdown_connections 2023-11-28T11:11:59.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.669+0000 7f05c16a6700 1 -- 172.21.15.28:0/128372300 wait complete. 2023-11-28T11:11:59.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.670+0000 7f05c16a6700 1 Processor -- start 2023-11-28T11:11:59.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.670+0000 7f05c16a6700 1 -- start start 2023-11-28T11:11:59.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05c16a6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05bc0ceec0 0x7f05bc153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05c16a6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05bc15c890 0x7f05bc15f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05c16a6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05bc160d30 0x7f05bc155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05c16a6700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0b5440 con 0x7f05bc160d30 2023-11-28T11:11:59.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05c16a6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0b5860 con 0x7f05bc15c890 2023-11-28T11:11:59.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05c16a6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05bc0b47c0 con 0x7f05bc0ceec0 2023-11-28T11:11:59.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05bb7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05bc160d30 0x7f05bc155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05bb7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05bc160d30 0x7f05bc155940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51384/0 (socket says 172.21.15.28:51384) 2023-11-28T11:11:59.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05bb7fe700 1 -- 172.21.15.28:0/2747809013 learned_addr learned my addr 172.21.15.28:0/2747809013 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:11:59.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05baffd700 1 --2- 172.21.15.28:0/2747809013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05bc0ceec0 0x7f05bc153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05ba7fc700 1 --2- 172.21.15.28:0/2747809013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05bc15c890 0x7f05bc15f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05bb7fe700 1 -- 172.21.15.28:0/2747809013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05bc0ceec0 msgr2=0x7f05bc153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05bb7fe700 1 --2- 172.21.15.28:0/2747809013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05bc0ceec0 0x7f05bc153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05bb7fe700 1 -- 172.21.15.28:0/2747809013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05bc15c890 msgr2=0x7f05bc15f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05bb7fe700 1 --2- 172.21.15.28:0/2747809013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05bc15c890 0x7f05bc15f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.671+0000 7f05bb7fe700 1 -- 172.21.15.28:0/2747809013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05ac0094a0 con 0x7f05bc160d30 2023-11-28T11:11:59.676 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.672+0000 7f05bb7fe700 1 --2- 172.21.15.28:0/2747809013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05bc160d30 0x7f05bc155940 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f05b4007e60 tx=0x7f05b400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:59.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.672+0000 7f05b37fe700 1 -- 172.21.15.28:0/2747809013 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b400ed30 con 0x7f05bc160d30 2023-11-28T11:11:59.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.672+0000 7f05b37fe700 1 -- 172.21.15.28:0/2747809013 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b4014420 con 0x7f05bc160d30 2023-11-28T11:11:59.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.672+0000 7f05c16a6700 1 -- 172.21.15.28:0/2747809013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05bc0b3070 con 0x7f05bc160d30 2023-11-28T11:11:59.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.672+0000 7f05b37fe700 1 -- 172.21.15.28:0/2747809013 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b40147d0 con 0x7f05bc160d30 2023-11-28T11:11:59.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.672+0000 7f05c16a6700 1 -- 172.21.15.28:0/2747809013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05bc0b2d90 con 0x7f05bc160d30 2023-11-28T11:11:59.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.673+0000 7f05c16a6700 1 -- 172.21.15.28:0/2747809013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05bc15cf60 con 0x7f05bc160d30 2023-11-28T11:11:59.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.673+0000 7f05b37fe700 1 -- 172.21.15.28:0/2747809013 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f05b4012050 con 0x7f05bc160d30 2023-11-28T11:11:59.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.674+0000 7f05b37fe700 1 --2- 172.21.15.28:0/2747809013 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f059c0682f0 0x7f059c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:11:59.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.674+0000 7f05b37fe700 1 -- 172.21.15.28:0/2747809013 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f05b4085f30 con 0x7f05bc160d30 2023-11-28T11:11:59.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.674+0000 7f05baffd700 1 --2- 172.21.15.28:0/2747809013 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f059c0682f0 0x7f059c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:11:59.677 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.674+0000 7f05baffd700 1 --2- 172.21.15.28:0/2747809013 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f059c0682f0 0x7f059c06a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0090b0 tx=0x7f05ac008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:11:59.678 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.678+0000 7f05b37fe700 1 -- 172.21.15.28:0/2747809013 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f05b40529a0 con 0x7f05bc160d30 2023-11-28T11:11:59.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.949+0000 7f05c16a6700 1 -- 172.21.15.28:0/2747809013 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f05bc0b5580 con 0x7f05bc160d30 2023-11-28T11:11:59.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.950+0000 7f05b37fe700 1 -- 172.21.15.28:0/2747809013 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5298 (secure 0 0 0) 0x7f05bc0b5580 con 0x7f05bc160d30 2023-11-28T11:11:59.951 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:11:59.951 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:11:59.951 INFO:teuthology.orchestra.run.smithi028.stdout: "auth_dump": [ 2023-11-28T11:11:59.951 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.951 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.a", 2023-11-28T11:11:59.951 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlx1F8HRAAqlv0qwBqCCSPbuXOK00uLQ==", 2023-11-28T11:11:59.952 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.952 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:11:59.952 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2023-11-28T11:11:59.952 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2023-11-28T11:11:59.952 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:11:59.952 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.952 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.953 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.953 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.b", 2023-11-28T11:11:59.953 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlqQd7KhAASvfFg9HJbZLZ48lOZ8rfdg==", 2023-11-28T11:11:59.953 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.953 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:11:59.953 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2023-11-28T11:11:59.953 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2023-11-28T11:11:59.954 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:11:59.954 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.954 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.954 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.954 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.c", 2023-11-28T11:11:59.955 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlz/TyJBAAug4qbJp/xfFm3aHFNMSlBA==", 2023-11-28T11:11:59.955 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.955 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:11:59.955 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2023-11-28T11:11:59.955 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2023-11-28T11:11:59.956 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:11:59.957 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.957 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.957 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.957 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mds.d", 2023-11-28T11:11:59.957 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlJA0pMhAAgmUFSzlTLDsYT7SPRaS4hg==", 2023-11-28T11:11:59.957 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.957 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:11:59.957 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2023-11-28T11:11:59.957 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2023-11-28T11:11:59.957 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:11:59.957 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.957 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.958 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.958 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.0", 2023-11-28T11:11:59.958 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQALymVlGI9xJBAAFY51VVoRxuhhNHGi8tlG5g==", 2023-11-28T11:11:59.958 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.958 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:59.958 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:11:59.958 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:11:59.958 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.958 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.958 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.958 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.1", 2023-11-28T11:11:59.958 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAOymVl6bn5IxAAE02o425v2koATSJg/CKx2Q==", 2023-11-28T11:11:59.959 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.959 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:59.959 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:11:59.959 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:11:59.959 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.959 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.959 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.959 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.2", 2023-11-28T11:11:59.959 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQARymVl0JCmIxAASw1qp+1l8/G+7uP2t2Gfcg==", 2023-11-28T11:11:59.959 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.959 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:59.959 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:11:59.960 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:11:59.960 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.960 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.960 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.960 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.3", 2023-11-28T11:11:59.960 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAUymVlHpKYIxAAeEFLqUE4cWvE29wkIdb4xQ==", 2023-11-28T11:11:59.960 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.960 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:59.960 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:11:59.960 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:11:59.960 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.960 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.961 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.961 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.4", 2023-11-28T11:11:59.961 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAcymVlRw7IDBAACbu1rTbZMEVtwM33xdVPSA==", 2023-11-28T11:11:59.961 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.961 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:59.961 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:11:59.961 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:11:59.961 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.961 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.961 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.961 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.5", 2023-11-28T11:11:59.961 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAfymVlDGXHDBAA5lGng1DglyCM+DnWCQ0iUg==", 2023-11-28T11:11:59.961 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.962 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:59.962 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:11:59.962 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:11:59.962 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.962 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.962 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.962 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.6", 2023-11-28T11:11:59.962 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAiymVlfw+IDBAAX5cXoXUhrY8BX3tZFVyqCA==", 2023-11-28T11:11:59.962 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.962 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:59.963 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:11:59.963 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:11:59.963 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.963 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.963 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.963 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "osd.7", 2023-11-28T11:11:59.963 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAlymVliBSsDBAACyShS6zNLoUAlVFbB5XEag==", 2023-11-28T11:11:59.963 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.963 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow profile osd", 2023-11-28T11:11:59.963 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile osd", 2023-11-28T11:11:59.963 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:11:59.964 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.964 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.964 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.964 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.0", 2023-11-28T11:11:59.964 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVluSHRNhAAmBrJ4shI0iFZ+s0TazdTrw==", 2023-11-28T11:11:59.964 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.964 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:11:59.964 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow r", 2023-11-28T11:11:59.964 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rw pool=cephfs_data" 2023-11-28T11:11:59.964 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.964 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.964 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.964 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.1", 2023-11-28T11:11:59.965 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAKymVl2+otAhAATGAAuzHz3lKdV0enZTr6qA==", 2023-11-28T11:11:59.965 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.965 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:11:59.965 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2023-11-28T11:11:59.965 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2023-11-28T11:11:59.965 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2023-11-28T11:11:59.965 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.965 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.965 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.965 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.2", 2023-11-28T11:11:59.965 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlxqlTOhAAU7U9b//feat8r6WDQNx/Gg==", 2023-11-28T11:11:59.965 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.965 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:11:59.966 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2023-11-28T11:11:59.966 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2023-11-28T11:11:59.966 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2023-11-28T11:11:59.966 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.966 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.966 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.966 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.3", 2023-11-28T11:11:59.966 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAKymVl592XBRAASQZMNs5hbHPm2CYq2ktYdg==", 2023-11-28T11:11:59.966 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.966 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow", 2023-11-28T11:11:59.966 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow r", 2023-11-28T11:11:59.966 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow rw", 2023-11-28T11:11:59.967 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow rwx" 2023-11-28T11:11:59.967 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.967 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.967 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.967 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "client.admin", 2023-11-28T11:11:59.967 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAIymVl5ZzXOhAAt2N6VJgsHFGvT1U6Uee9VA==", 2023-11-28T11:11:59.967 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.967 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2023-11-28T11:11:59.967 INFO:teuthology.orchestra.run.smithi028.stdout: "mgr": "allow *", 2023-11-28T11:11:59.967 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow *", 2023-11-28T11:11:59.967 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:11:59.967 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.968 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.968 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.968 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.x", 2023-11-28T11:11:59.968 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlZrj9GRAAXu1z37jFMZo3YXUJLRB2+w==", 2023-11-28T11:11:59.968 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.968 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2023-11-28T11:11:59.968 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2023-11-28T11:11:59.968 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:11:59.968 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.968 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:11:59.968 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:11:59.968 INFO:teuthology.orchestra.run.smithi028.stdout: "entity": "mgr.y", 2023-11-28T11:11:59.969 INFO:teuthology.orchestra.run.smithi028.stdout: "key": "AQAJymVlyBOJFRAA6cjReBkYUD/3YOUHw7DHpg==", 2023-11-28T11:11:59.969 INFO:teuthology.orchestra.run.smithi028.stdout: "caps": { 2023-11-28T11:11:59.969 INFO:teuthology.orchestra.run.smithi028.stdout: "mds": "allow *", 2023-11-28T11:11:59.969 INFO:teuthology.orchestra.run.smithi028.stdout: "mon": "allow profile mgr", 2023-11-28T11:11:59.969 INFO:teuthology.orchestra.run.smithi028.stdout: "osd": "allow *" 2023-11-28T11:11:59.969 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.969 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:11:59.969 INFO:teuthology.orchestra.run.smithi028.stdout: ] 2023-11-28T11:11:59.969 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:11:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.952+0000 7f05c16a6700 1 -- 172.21.15.28:0/2747809013 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f059c0682f0 msgr2=0x7f059c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.952+0000 7f05c16a6700 1 --2- 172.21.15.28:0/2747809013 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f059c0682f0 0x7f059c06a7a0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f05ac0090b0 tx=0x7f05ac008030 comp rx=0 tx=0).stop 2023-11-28T11:11:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.952+0000 7f05c16a6700 1 -- 172.21.15.28:0/2747809013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05bc160d30 msgr2=0x7f05bc155940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:11:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.952+0000 7f05c16a6700 1 --2- 172.21.15.28:0/2747809013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05bc160d30 0x7f05bc155940 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f05b4007e60 tx=0x7f05b400e3c0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.952+0000 7f05c16a6700 1 -- 172.21.15.28:0/2747809013 shutdown_connections 2023-11-28T11:11:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.952+0000 7f05c16a6700 1 --2- 172.21.15.28:0/2747809013 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f059c0682f0 0x7f059c06a7a0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.952+0000 7f05c16a6700 1 --2- 172.21.15.28:0/2747809013 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f05bc160d30 0x7f05bc155940 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.952+0000 7f05c16a6700 1 --2- 172.21.15.28:0/2747809013 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f05bc0ceec0 0x7f05bc153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.952+0000 7f05c16a6700 1 --2- 172.21.15.28:0/2747809013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05bc15c890 0x7f05bc15f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:11:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.952+0000 7f05c16a6700 1 -- 172.21.15.28:0/2747809013 >> 172.21.15.28:0/2747809013 conn(0x7f05bc06e9b0 msgr2=0x7f05bc0cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:11:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.952+0000 7f05c16a6700 1 -- 172.21.15.28:0/2747809013 shutdown_connections 2023-11-28T11:11:59.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:11:59.952+0000 7f05c16a6700 1 -- 172.21.15.28:0/2747809013 wait complete. 2023-11-28T11:11:59.972 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:12:00.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.096+0000 7f75f1be3700 1 Processor -- start 2023-11-28T11:12:00.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.096+0000 7f75f1be3700 1 -- start start 2023-11-28T11:12:00.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.097+0000 7f75f1be3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75ec06e400 0x7f75ec15db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.097+0000 7f75f1be3700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ec15e0b0 con 0x7f75ec0ce890 2023-11-28T11:12:00.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.097+0000 7f75f1be3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75ec15d120 con 0x7f75ec159590 2023-11-28T11:12:00.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.097+0000 7f75f1be3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec15d6e0 con 0x7f75ec06e400 2023-11-28T11:12:00.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.097+0000 7f75ebfff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f75ec0ce890 0x7f75ec0caf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39450/0 (socket says 172.21.15.28:39450) 2023-11-28T11:12:00.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.097+0000 7f75ebfff700 1 -- 172.21.15.28:0/1036016692 learned_addr learned my addr 172.21.15.28:0/1036016692 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:00.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75eaffd700 1 --2- 172.21.15.28:0/1036016692 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75ec06e400 0x7f75ec15db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ea7fc700 1 -- 172.21.15.28:0/1036016692 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2645260602 0 0) 0x7f75e4001490 con 0x7f75ec0ce890 2023-11-28T11:12:00.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ea7fc700 1 -- 172.21.15.28:0/1036016692 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f75d80035b0 con 0x7f75ec0ce890 2023-11-28T11:12:00.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ea7fc700 1 -- 172.21.15.28:0/1036016692 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4255497200 0 0) 0x7f75d80035b0 con 0x7f75ec0ce890 2023-11-28T11:12:00.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ea7fc700 1 -- 172.21.15.28:0/1036016692 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75ec06e400 msgr2=0x7f75ec15db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:00.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ea7fc700 1 --2- 172.21.15.28:0/1036016692 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75ec06e400 0x7f75ec15db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ea7fc700 1 -- 172.21.15.28:0/1036016692 >> v1:172.21.15.59:6789/0 conn(0x7f75ec159590 legacy=0x7f75ec0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ea7fc700 1 -- 172.21.15.28:0/1036016692 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec0aef60 con 0x7f75ec0ce890 2023-11-28T11:12:00.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ea7fc700 1 -- 172.21.15.28:0/1036016692 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f75e4001b80 con 0x7f75ec0ce890 2023-11-28T11:12:00.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ea7fc700 1 -- 172.21.15.28:0/1036016692 >> v1:172.21.15.28:6789/0 conn(0x7f75ec0ce890 legacy=0x7f75ec0caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ea7fc700 1 --2- 172.21.15.28:0/1036016692 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75ec06e400 0x7f75d8003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ea7fc700 1 --2- 172.21.15.28:0/1036016692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d8003fb0 0x7f75d8004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ea7fc700 1 --2- 172.21.15.28:0/1036016692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75d8004d90 0x7f75d8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ea7fc700 1 -- 172.21.15.28:0/1036016692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec15d6e0 con 0x7f75d8004d90 2023-11-28T11:12:00.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ea7fc700 1 -- 172.21.15.28:0/1036016692 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75d800baf0 con 0x7f75d8003fb0 2023-11-28T11:12:00.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ea7fc700 1 -- 172.21.15.28:0/1036016692 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75d800dec0 con 0x7f75ec06e400 2023-11-28T11:12:00.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75ebfff700 1 --2- 172.21.15.28:0/1036016692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75d8004d90 0x7f75d8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75eb7fe700 1 --2- 172.21.15.28:0/1036016692 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75ec06e400 0x7f75d8003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.098+0000 7f75eaffd700 1 --2- 172.21.15.28:0/1036016692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d8003fb0 0x7f75d8004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75ebfff700 1 -- 172.21.15.28:0/1036016692 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75ec06e400 msgr2=0x7f75d8003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:00.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75ebfff700 1 --2- 172.21.15.28:0/1036016692 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75ec06e400 0x7f75d8003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75ebfff700 1 -- 172.21.15.28:0/1036016692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d8003fb0 msgr2=0x7f75d8004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:00.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75ebfff700 1 --2- 172.21.15.28:0/1036016692 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75d8003fb0 0x7f75d8004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75ebfff700 1 -- 172.21.15.28:0/1036016692 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75d80035b0 con 0x7f75d8004d90 2023-11-28T11:12:00.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75ebfff700 1 --2- 172.21.15.28:0/1036016692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75d8004d90 0x7f75d8005200 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f75e400a7e0 tx=0x7f75d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ff19acafe0d5428 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75ea7fc700 1 -- 172.21.15.28:0/1036016692 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e400a470 con 0x7f75d8004d90 2023-11-28T11:12:00.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75ea7fc700 1 -- 172.21.15.28:0/1036016692 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75e4012620 con 0x7f75d8004d90 2023-11-28T11:12:00.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75ea7fc700 1 -- 172.21.15.28:0/1036016692 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75e40129d0 con 0x7f75d8004d90 2023-11-28T11:12:00.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75f1be3700 1 -- 172.21.15.28:0/1036016692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75d8004d90 msgr2=0x7f75d8005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75f1be3700 1 --2- 172.21.15.28:0/1036016692 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75d8004d90 0x7f75d8005200 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f75e400a7e0 tx=0x7f75d8005880 comp rx=0 tx=0).stop 2023-11-28T11:12:00.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75ebfff700 1 -- 172.21.15.28:0/1036016692 reap_dead start 2023-11-28T11:12:00.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75f1be3700 1 -- 172.21.15.28:0/1036016692 shutdown_connections 2023-11-28T11:12:00.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75f1be3700 1 -- 172.21.15.28:0/1036016692 >> 172.21.15.28:0/1036016692 conn(0x7f75ec0b1650 msgr2=0x7f75ec0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:00.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75f1be3700 1 -- 172.21.15.28:0/1036016692 shutdown_connections 2023-11-28T11:12:00.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.099+0000 7f75f1be3700 1 -- 172.21.15.28:0/1036016692 wait complete. 2023-11-28T11:12:00.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.101+0000 7f75f1be3700 1 Processor -- start 2023-11-28T11:12:00.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.101+0000 7f75f1be3700 1 -- start start 2023-11-28T11:12:00.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75f1be3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75ec15c650 0x7f75ec1f6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75f1be3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75ec0b2950 0x7f75ec1f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75f1be3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75ec1f6220 0x7f75ec0b4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75f1be3700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec1f6690 con 0x7f75ec0b2950 2023-11-28T11:12:00.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75f1be3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec0c4540 con 0x7f75ec15c650 2023-11-28T11:12:00.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75f1be3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec0c4810 con 0x7f75ec1f6220 2023-11-28T11:12:00.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75eaffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75ec0b2950 0x7f75ec1f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75eaffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75ec0b2950 0x7f75ec1f5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51402/0 (socket says 172.21.15.28:51402) 2023-11-28T11:12:00.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75eaffd700 1 -- 172.21.15.28:0/51566817 learned_addr learned my addr 172.21.15.28:0/51566817 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:00.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75ebfff700 1 --2- 172.21.15.28:0/51566817 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75ec1f6220 0x7f75ec0b4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75eb7fe700 1 --2- 172.21.15.28:0/51566817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75ec15c650 0x7f75ec1f6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75eaffd700 1 -- 172.21.15.28:0/51566817 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75ec1f6220 msgr2=0x7f75ec0b4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75eaffd700 1 --2- 172.21.15.28:0/51566817 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75ec1f6220 0x7f75ec0b4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75eaffd700 1 -- 172.21.15.28:0/51566817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75ec15c650 msgr2=0x7f75ec1f6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75eaffd700 1 --2- 172.21.15.28:0/51566817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75ec15c650 0x7f75ec1f6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75eaffd700 1 -- 172.21.15.28:0/51566817 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75ec15d370 con 0x7f75ec0b2950 2023-11-28T11:12:00.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75eb7fe700 1 --2- 172.21.15.28:0/51566817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75ec15c650 0x7f75ec1f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:00.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75eaffd700 1 --2- 172.21.15.28:0/51566817 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75ec0b2950 0x7f75ec1f5ce0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f75dc011aa0 tx=0x7f75dc011db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75e3fff700 1 -- 172.21.15.28:0/51566817 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc00f940 con 0x7f75ec0b2950 2023-11-28T11:12:00.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75e3fff700 1 -- 172.21.15.28:0/51566817 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75dc00faa0 con 0x7f75ec0b2950 2023-11-28T11:12:00.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75f1be3700 1 -- 172.21.15.28:0/51566817 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec0af240 con 0x7f75ec0b2950 2023-11-28T11:12:00.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75e3fff700 1 -- 172.21.15.28:0/51566817 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75dc00fe50 con 0x7f75ec0b2950 2023-11-28T11:12:00.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.102+0000 7f75f1be3700 1 -- 172.21.15.28:0/51566817 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec159960 con 0x7f75ec0b2950 2023-11-28T11:12:00.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.103+0000 7f75f1be3700 1 -- 172.21.15.28:0/51566817 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ec0675a0 con 0x7f75ec0b2950 2023-11-28T11:12:00.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.104+0000 7f75e3fff700 1 -- 172.21.15.28:0/51566817 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f75dc01d400 con 0x7f75ec0b2950 2023-11-28T11:12:00.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.104+0000 7f75e3fff700 1 --2- 172.21.15.28:0/51566817 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75cc0682f0 0x7f75cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.104+0000 7f75e3fff700 1 -- 172.21.15.28:0/51566817 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f75dc086320 con 0x7f75ec0b2950 2023-11-28T11:12:00.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.104+0000 7f75eb7fe700 1 --2- 172.21.15.28:0/51566817 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75cc0682f0 0x7f75cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.105+0000 7f75eb7fe700 1 --2- 172.21.15.28:0/51566817 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75cc0682f0 0x7f75cc06a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f75d4007fd0 tx=0x7f75d4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.108+0000 7f75e3fff700 1 -- 172.21.15.28:0/51566817 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f75dc052ea0 con 0x7f75ec0b2950 2023-11-28T11:12:00.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.302+0000 7f75f1be3700 1 -- 172.21.15.28:0/51566817 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75ec06e160 con 0x7f75ec0b2950 2023-11-28T11:12:00.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.303+0000 7f75e3fff700 1 -- 172.21.15.28:0/51566817 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) v1 ==== 74+0+14366 (secure 0 0 0) 0x7f75dc0564c0 con 0x7f75ec0b2950 2023-11-28T11:12:00.304 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:00.304 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":62,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:11:59.510997+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":52,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:12:00.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.305+0000 7f75f1be3700 1 -- 172.21.15.28:0/51566817 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75cc0682f0 msgr2=0x7f75cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.305+0000 7f75f1be3700 1 --2- 172.21.15.28:0/51566817 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75cc0682f0 0x7f75cc06a7a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f75d4007fd0 tx=0x7f75d4009040 comp rx=0 tx=0).stop 2023-11-28T11:12:00.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.305+0000 7f75f1be3700 1 -- 172.21.15.28:0/51566817 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75ec0b2950 msgr2=0x7f75ec1f5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.305+0000 7f75f1be3700 1 --2- 172.21.15.28:0/51566817 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75ec0b2950 0x7f75ec1f5ce0 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f75dc011aa0 tx=0x7f75dc011db0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.305+0000 7f75f1be3700 1 -- 172.21.15.28:0/51566817 shutdown_connections 2023-11-28T11:12:00.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.305+0000 7f75f1be3700 1 --2- 172.21.15.28:0/51566817 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f75cc0682f0 0x7f75cc06a7a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.305+0000 7f75f1be3700 1 --2- 172.21.15.28:0/51566817 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f75ec1f6220 0x7f75ec0b4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.305+0000 7f75f1be3700 1 --2- 172.21.15.28:0/51566817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f75ec15c650 0x7f75ec1f6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.305+0000 7f75f1be3700 1 --2- 172.21.15.28:0/51566817 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f75ec0b2950 0x7f75ec1f5ce0 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.305+0000 7f75f1be3700 1 -- 172.21.15.28:0/51566817 >> 172.21.15.28:0/51566817 conn(0x7f75ec0b1650 msgr2=0x7f75ec0cf100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:00.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.305+0000 7f75f1be3700 1 -- 172.21.15.28:0/51566817 shutdown_connections 2023-11-28T11:12:00.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.305+0000 7f75f1be3700 1 -- 172.21.15.28:0/51566817 wait complete. 2023-11-28T11:12:00.319 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:12:00.319 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:12:00.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.448+0000 7f5c15ea0700 1 Processor -- start 2023-11-28T11:12:00.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.448+0000 7f5c15ea0700 1 -- start start 2023-11-28T11:12:00.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.449+0000 7f5c15ea0700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5c100b1630 0x7f5c100cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.449+0000 7f5c15ea0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c100b1a00 0x7f5c100caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.449+0000 7f5c15ea0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c100ce550 con 0x7f5c100cb3e0 2023-11-28T11:12:00.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.449+0000 7f5c15ea0700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c100ce830 con 0x7f5c100b1630 2023-11-28T11:12:00.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.449+0000 7f5c15ea0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100ceb10 con 0x7f5c100b1a00 2023-11-28T11:12:00.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.449+0000 7f5c0f7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5c100b1630 0x7f5c100cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.449+0000 7f5c0f7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5c100b1630 0x7f5c100cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51408/0 (socket says 172.21.15.28:51408) 2023-11-28T11:12:00.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.449+0000 7f5c0f7fe700 1 -- 172.21.15.28:0/2192133468 learned_addr learned my addr 172.21.15.28:0/2192133468 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:00.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.450+0000 7f5c0effd700 1 --2- 172.21.15.28:0/2192133468 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c100b1a00 0x7f5c100caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.450+0000 7f5c0f7fe700 1 -- 172.21.15.28:0/2192133468 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c100b1a00 msgr2=0x7f5c100caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:00.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.450+0000 7f5c0f7fe700 1 --2- 172.21.15.28:0/2192133468 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c100b1a00 0x7f5c100caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.450+0000 7f5c0f7fe700 1 -- 172.21.15.28:0/2192133468 >> v1:172.21.15.59:6789/0 conn(0x7f5c100cb3e0 legacy=0x7f5c10160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:00.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.450+0000 7f5c0f7fe700 1 -- 172.21.15.28:0/2192133468 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c100cedf0 con 0x7f5c100b1630 2023-11-28T11:12:00.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.450+0000 7f5c0f7fe700 1 --2- 172.21.15.28:0/2192133468 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c100b1630 0x7f5c100cf5f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5c000098f0 tx=0x7f5c00009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b3cb49510121c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.450+0000 7f5c0e7fc700 1 -- 172.21.15.28:0/2192133468 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c00004230 con 0x7f5c100b1630 2023-11-28T11:12:00.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.450+0000 7f5c0e7fc700 1 -- 172.21.15.28:0/2192133468 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c00004390 con 0x7f5c100b1630 2023-11-28T11:12:00.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.450+0000 7f5c0e7fc700 1 -- 172.21.15.28:0/2192133468 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c000036a0 con 0x7f5c100b1630 2023-11-28T11:12:00.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.450+0000 7f5c15ea0700 1 -- 172.21.15.28:0/2192133468 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c100b1630 msgr2=0x7f5c100cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.450+0000 7f5c15ea0700 1 --2- 172.21.15.28:0/2192133468 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c100b1630 0x7f5c100cf5f0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5c000098f0 tx=0x7f5c00009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:00.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.450+0000 7f5c15ea0700 1 -- 172.21.15.28:0/2192133468 shutdown_connections 2023-11-28T11:12:00.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.450+0000 7f5c15ea0700 1 --2- 172.21.15.28:0/2192133468 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c100b1630 0x7f5c100cf5f0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.451+0000 7f5c15ea0700 1 --2- 172.21.15.28:0/2192133468 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c100b1a00 0x7f5c100caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.451+0000 7f5c15ea0700 1 -- 172.21.15.28:0/2192133468 >> 172.21.15.28:0/2192133468 conn(0x7f5c100cc630 msgr2=0x7f5c100cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:00.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.451+0000 7f5c15ea0700 1 -- 172.21.15.28:0/2192133468 shutdown_connections 2023-11-28T11:12:00.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.451+0000 7f5c15ea0700 1 -- 172.21.15.28:0/2192133468 wait complete. 2023-11-28T11:12:00.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.452+0000 7f5c15ea0700 1 Processor -- start 2023-11-28T11:12:00.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.452+0000 7f5c15ea0700 1 -- start start 2023-11-28T11:12:00.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c15ea0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c100b1630 0x7f5c100c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c15ea0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c100b1a00 0x7f5c100c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c15ea0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c100cb3e0 0x7f5c1015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c15ea0700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100c74b0 con 0x7f5c100cb3e0 2023-11-28T11:12:00.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c15ea0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100c7790 con 0x7f5c100b1630 2023-11-28T11:12:00.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c15ea0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c100c7a70 con 0x7f5c100b1a00 2023-11-28T11:12:00.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c0ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c100cb3e0 0x7f5c1015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c0ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c100cb3e0 0x7f5c1015b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51418/0 (socket says 172.21.15.28:51418) 2023-11-28T11:12:00.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c0ffff700 1 -- 172.21.15.28:0/2299948586 learned_addr learned my addr 172.21.15.28:0/2299948586 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:00.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c0f7fe700 1 --2- 172.21.15.28:0/2299948586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c100b1630 0x7f5c100c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c0effd700 1 --2- 172.21.15.28:0/2299948586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c100b1a00 0x7f5c100c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c0ffff700 1 -- 172.21.15.28:0/2299948586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c100b1a00 msgr2=0x7f5c100c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c0ffff700 1 --2- 172.21.15.28:0/2299948586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c100b1a00 0x7f5c100c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c0ffff700 1 -- 172.21.15.28:0/2299948586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c100b1630 msgr2=0x7f5c100c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c0ffff700 1 --2- 172.21.15.28:0/2299948586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c100b1630 0x7f5c100c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c0ffff700 1 -- 172.21.15.28:0/2299948586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c00009550 con 0x7f5c100cb3e0 2023-11-28T11:12:00.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c0ffff700 1 --2- 172.21.15.28:0/2299948586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c100cb3e0 0x7f5c1015b1c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800b970 tx=0x7f5c0800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.453+0000 7f5c07fff700 1 -- 172.21.15.28:0/2299948586 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800d860 con 0x7f5c100cb3e0 2023-11-28T11:12:00.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.454+0000 7f5c07fff700 1 -- 172.21.15.28:0/2299948586 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c0800d9c0 con 0x7f5c100cb3e0 2023-11-28T11:12:00.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.454+0000 7f5c15ea0700 1 -- 172.21.15.28:0/2299948586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c100b1e70 con 0x7f5c100cb3e0 2023-11-28T11:12:00.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.454+0000 7f5c07fff700 1 -- 172.21.15.28:0/2299948586 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c0800dd70 con 0x7f5c100cb3e0 2023-11-28T11:12:00.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.454+0000 7f5c15ea0700 1 -- 172.21.15.28:0/2299948586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c100cf0d0 con 0x7f5c100cb3e0 2023-11-28T11:12:00.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.454+0000 7f5c15ea0700 1 -- 172.21.15.28:0/2299948586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c100ce140 con 0x7f5c100cb3e0 2023-11-28T11:12:00.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.455+0000 7f5c07fff700 1 -- 172.21.15.28:0/2299948586 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5c0800e430 con 0x7f5c100cb3e0 2023-11-28T11:12:00.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.455+0000 7f5c07fff700 1 --2- 172.21.15.28:0/2299948586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5bf00682f0 0x7f5bf006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.455+0000 7f5c07fff700 1 -- 172.21.15.28:0/2299948586 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f5c08085d40 con 0x7f5c100cb3e0 2023-11-28T11:12:00.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.455+0000 7f5c0f7fe700 1 --2- 172.21.15.28:0/2299948586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5bf00682f0 0x7f5bf006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.456+0000 7f5c0f7fe700 1 --2- 172.21.15.28:0/2299948586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5bf00682f0 0x7f5bf006a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009130 tx=0x7f5c00008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.459+0000 7f5c07fff700 1 -- 172.21.15.28:0/2299948586 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5c08052750 con 0x7f5c100cb3e0 2023-11-28T11:12:00.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.650+0000 7f5c15ea0700 1 -- 172.21.15.28:0/2299948586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5c100739f0 con 0x7f5c100cb3e0 2023-11-28T11:12:00.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.650+0000 7f5c07fff700 1 -- 172.21.15.28:0/2299948586 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v62) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5c08055d70 con 0x7f5c100cb3e0 2023-11-28T11:12:00.651 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:12:00.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.652+0000 7f5c15ea0700 1 -- 172.21.15.28:0/2299948586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5bf00682f0 msgr2=0x7f5bf006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.652+0000 7f5c15ea0700 1 --2- 172.21.15.28:0/2299948586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5bf00682f0 0x7f5bf006a7a0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009130 tx=0x7f5c00008030 comp rx=0 tx=0).stop 2023-11-28T11:12:00.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.652+0000 7f5c15ea0700 1 -- 172.21.15.28:0/2299948586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c100cb3e0 msgr2=0x7f5c1015b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.652+0000 7f5c15ea0700 1 --2- 172.21.15.28:0/2299948586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c100cb3e0 0x7f5c1015b1c0 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f5c0800b970 tx=0x7f5c0800bc80 comp rx=0 tx=0).stop 2023-11-28T11:12:00.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.652+0000 7f5c15ea0700 1 -- 172.21.15.28:0/2299948586 shutdown_connections 2023-11-28T11:12:00.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.652+0000 7f5c15ea0700 1 --2- 172.21.15.28:0/2299948586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5bf00682f0 0x7f5bf006a7a0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.652+0000 7f5c15ea0700 1 --2- 172.21.15.28:0/2299948586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5c100cb3e0 0x7f5c1015b1c0 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.652+0000 7f5c15ea0700 1 --2- 172.21.15.28:0/2299948586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c100b1630 0x7f5c100c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.652+0000 7f5c15ea0700 1 --2- 172.21.15.28:0/2299948586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5c100b1a00 0x7f5c100c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.652+0000 7f5c15ea0700 1 -- 172.21.15.28:0/2299948586 >> 172.21.15.28:0/2299948586 conn(0x7f5c100cc630 msgr2=0x7f5c100afe80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:00.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.652+0000 7f5c15ea0700 1 -- 172.21.15.28:0/2299948586 shutdown_connections 2023-11-28T11:12:00.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.652+0000 7f5c15ea0700 1 -- 172.21.15.28:0/2299948586 wait complete. 2023-11-28T11:12:00.665 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:12:00.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.791+0000 7fcf48644700 1 Processor -- start 2023-11-28T11:12:00.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.791+0000 7fcf48644700 1 -- start start 2023-11-28T11:12:00.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.792+0000 7fcf48644700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf40156bd0 0x7fcf4006dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.794 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.792+0000 7fcf48644700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcf4006e010 0x7fcf40160340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.792+0000 7fcf48644700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fcf40160880 0x7fcf400b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.792+0000 7fcf48644700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf400ca3c0 con 0x7fcf4006e010 2023-11-28T11:12:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.792+0000 7fcf48644700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf4006e480 con 0x7fcf40160880 2023-11-28T11:12:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.792+0000 7fcf48644700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf40156fa0 con 0x7fcf40156bd0 2023-11-28T11:12:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.792+0000 7fcf45bdf700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcf4006e010 0x7fcf40160340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.793+0000 7fcf45bdf700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcf4006e010 0x7fcf40160340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51422/0 (socket says 172.21.15.28:51422) 2023-11-28T11:12:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.793+0000 7fcf45bdf700 1 -- 172.21.15.28:0/3042431735 learned_addr learned my addr 172.21.15.28:0/3042431735 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.793+0000 7fcf46be1700 1 --2- 172.21.15.28:0/3042431735 >> v2:172.21.15.59:3300/0 conn(0x7fcf40160880 0x7fcf400b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.793+0000 7fcf463e0700 1 --2- 172.21.15.28:0/3042431735 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf40156bd0 0x7fcf4006dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.793+0000 7fcf45bdf700 1 -- 172.21.15.28:0/3042431735 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf40156bd0 msgr2=0x7fcf4006dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.793+0000 7fcf45bdf700 1 --2- 172.21.15.28:0/3042431735 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf40156bd0 0x7fcf4006dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.795 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.793+0000 7fcf45bdf700 1 -- 172.21.15.28:0/3042431735 >> v2:172.21.15.59:3300/0 conn(0x7fcf40160880 msgr2=0x7fcf400b1470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.793+0000 7fcf45bdf700 1 --2- 172.21.15.28:0/3042431735 >> v2:172.21.15.59:3300/0 conn(0x7fcf40160880 0x7fcf400b1470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.793+0000 7fcf45bdf700 1 -- 172.21.15.28:0/3042431735 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf400cb390 con 0x7fcf4006e010 2023-11-28T11:12:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.793+0000 7fcf46be1700 1 --2- 172.21.15.28:0/3042431735 >> v2:172.21.15.59:3300/0 conn(0x7fcf40160880 0x7fcf400b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.793+0000 7fcf45bdf700 1 --2- 172.21.15.28:0/3042431735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf4006e010 0x7fcf40160340 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fcf4006eb90 tx=0x7fcf34009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c55d7833e52b89 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.793+0000 7fcf453de700 1 -- 172.21.15.28:0/3042431735 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34011830 con 0x7fcf4006e010 2023-11-28T11:12:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.793+0000 7fcf453de700 1 -- 172.21.15.28:0/3042431735 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf34011990 con 0x7fcf4006e010 2023-11-28T11:12:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.793+0000 7fcf453de700 1 -- 172.21.15.28:0/3042431735 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf34011d20 con 0x7fcf4006e010 2023-11-28T11:12:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.794+0000 7fcf48644700 1 -- 172.21.15.28:0/3042431735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf4006e010 msgr2=0x7fcf40160340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.794+0000 7fcf48644700 1 --2- 172.21.15.28:0/3042431735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf4006e010 0x7fcf40160340 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fcf4006eb90 tx=0x7fcf34009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.794+0000 7fcf48644700 1 -- 172.21.15.28:0/3042431735 shutdown_connections 2023-11-28T11:12:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.794+0000 7fcf48644700 1 --2- 172.21.15.28:0/3042431735 >> v2:172.21.15.59:3300/0 conn(0x7fcf40160880 0x7fcf400b1470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.794+0000 7fcf48644700 1 --2- 172.21.15.28:0/3042431735 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf40156bd0 0x7fcf4006dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.794+0000 7fcf48644700 1 --2- 172.21.15.28:0/3042431735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf4006e010 0x7fcf40160340 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.794+0000 7fcf48644700 1 -- 172.21.15.28:0/3042431735 >> 172.21.15.28:0/3042431735 conn(0x7fcf400cc470 msgr2=0x7fcf400cc870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:00.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.794+0000 7fcf48644700 1 -- 172.21.15.28:0/3042431735 shutdown_connections 2023-11-28T11:12:00.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.794+0000 7fcf48644700 1 -- 172.21.15.28:0/3042431735 wait complete. 2023-11-28T11:12:00.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.795+0000 7fcf48644700 1 Processor -- start 2023-11-28T11:12:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.795+0000 7fcf48644700 1 -- start start 2023-11-28T11:12:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.796+0000 7fcf48644700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf4006e010 0x7fcf4006c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.796+0000 7fcf48644700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf40156bd0 0x7fcf4006cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.796+0000 7fcf48644700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf40160880 0x7fcf400b14f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.796+0000 7fcf48644700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400c39c0 con 0x7fcf4006e010 2023-11-28T11:12:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.796+0000 7fcf48644700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400c3ca0 con 0x7fcf40160880 2023-11-28T11:12:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.796+0000 7fcf48644700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf400c3f80 con 0x7fcf40156bd0 2023-11-28T11:12:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.796+0000 7fcf463e0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf4006e010 0x7fcf4006c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.796+0000 7fcf463e0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf4006e010 0x7fcf4006c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51428/0 (socket says 172.21.15.28:51428) 2023-11-28T11:12:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.796+0000 7fcf463e0700 1 -- 172.21.15.28:0/100908361 learned_addr learned my addr 172.21.15.28:0/100908361 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:00.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.796+0000 7fcf463e0700 1 -- 172.21.15.28:0/100908361 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf40156bd0 msgr2=0x7fcf4006cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.796+0000 7fcf463e0700 1 --2- 172.21.15.28:0/100908361 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf40156bd0 0x7fcf4006cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.796+0000 7fcf463e0700 1 -- 172.21.15.28:0/100908361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf40160880 msgr2=0x7fcf400b14f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.796+0000 7fcf463e0700 1 --2- 172.21.15.28:0/100908361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf40160880 0x7fcf400b14f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.796+0000 7fcf463e0700 1 -- 172.21.15.28:0/100908361 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf34009550 con 0x7fcf4006e010 2023-11-28T11:12:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.796+0000 7fcf463e0700 1 --2- 172.21.15.28:0/100908361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf4006e010 0x7fcf4006c680 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00e850 tx=0x7fcf3c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.797+0000 7fcf32ffd700 1 -- 172.21.15.28:0/100908361 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c0095b0 con 0x7fcf4006e010 2023-11-28T11:12:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.797+0000 7fcf32ffd700 1 -- 172.21.15.28:0/100908361 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf3c00ee50 con 0x7fcf4006e010 2023-11-28T11:12:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.797+0000 7fcf48644700 1 -- 172.21.15.28:0/100908361 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf400ca8b0 con 0x7fcf4006e010 2023-11-28T11:12:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.797+0000 7fcf32ffd700 1 -- 172.21.15.28:0/100908361 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3c00ca70 con 0x7fcf4006e010 2023-11-28T11:12:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.797+0000 7fcf48644700 1 -- 172.21.15.28:0/100908361 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf400ca5d0 con 0x7fcf4006e010 2023-11-28T11:12:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.798+0000 7fcf48644700 1 -- 172.21.15.28:0/100908361 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf400cab90 con 0x7fcf4006e010 2023-11-28T11:12:00.801 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.798+0000 7fcf32ffd700 1 -- 172.21.15.28:0/100908361 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcf3c019030 con 0x7fcf4006e010 2023-11-28T11:12:00.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.798+0000 7fcf32ffd700 1 --2- 172.21.15.28:0/100908361 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcf24068320 0x7fcf2406a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:00.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.798+0000 7fcf32ffd700 1 -- 172.21.15.28:0/100908361 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7fcf3c085a70 con 0x7fcf4006e010 2023-11-28T11:12:00.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.798+0000 7fcf45bdf700 1 --2- 172.21.15.28:0/100908361 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcf24068320 0x7fcf2406a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:00.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.799+0000 7fcf45bdf700 1 --2- 172.21.15.28:0/100908361 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcf24068320 0x7fcf2406a7d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcf34014040 tx=0x7fcf34010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:00.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.803+0000 7fcf32ffd700 1 -- 172.21.15.28:0/100908361 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcf3c052360 con 0x7fcf4006e010 2023-11-28T11:12:00.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.996+0000 7fcf48644700 1 -- 172.21.15.28:0/100908361 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcf400739f0 con 0x7fcf4006e010 2023-11-28T11:12:00.997 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.997+0000 7fcf32ffd700 1 -- 172.21.15.28:0/100908361 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v62) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcf3c0210a0 con 0x7fcf4006e010 2023-11-28T11:12:00.997 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:12:01.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.999+0000 7fcf48644700 1 -- 172.21.15.28:0/100908361 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcf24068320 msgr2=0x7fcf2406a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.999+0000 7fcf48644700 1 --2- 172.21.15.28:0/100908361 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcf24068320 0x7fcf2406a7d0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fcf34014040 tx=0x7fcf34010000 comp rx=0 tx=0).stop 2023-11-28T11:12:01.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.999+0000 7fcf48644700 1 -- 172.21.15.28:0/100908361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf4006e010 msgr2=0x7fcf4006c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.999+0000 7fcf48644700 1 --2- 172.21.15.28:0/100908361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf4006e010 0x7fcf4006c680 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fcf3c00e850 tx=0x7fcf3c00ec10 comp rx=0 tx=0).stop 2023-11-28T11:12:01.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.999+0000 7fcf48644700 1 -- 172.21.15.28:0/100908361 shutdown_connections 2023-11-28T11:12:01.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.999+0000 7fcf48644700 1 --2- 172.21.15.28:0/100908361 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fcf24068320 0x7fcf2406a7d0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.999+0000 7fcf48644700 1 --2- 172.21.15.28:0/100908361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcf4006e010 0x7fcf4006c680 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.999+0000 7fcf48644700 1 --2- 172.21.15.28:0/100908361 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf40160880 0x7fcf400b14f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.999+0000 7fcf48644700 1 --2- 172.21.15.28:0/100908361 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcf40156bd0 0x7fcf4006cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.999+0000 7fcf48644700 1 -- 172.21.15.28:0/100908361 >> 172.21.15.28:0/100908361 conn(0x7fcf400cc470 msgr2=0x7fcf400b4830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:01.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.999+0000 7fcf48644700 1 -- 172.21.15.28:0/100908361 shutdown_connections 2023-11-28T11:12:01.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:00.999+0000 7fcf48644700 1 -- 172.21.15.28:0/100908361 wait complete. 2023-11-28T11:12:01.011 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T11:12:01.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.135+0000 7f73ef870700 1 Processor -- start 2023-11-28T11:12:01.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.135+0000 7f73ef870700 1 -- start start 2023-11-28T11:12:01.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73ef870700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f73e815d1e0 0x7f73e815d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73ef870700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73e806aa00 0x7f73e80cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73ef870700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e806ae80 con 0x7f73e806a560 2023-11-28T11:12:01.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73ef870700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e80cdec0 con 0x7f73e815d1e0 2023-11-28T11:12:01.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73ef870700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80c9cd0 con 0x7f73e806aa00 2023-11-28T11:12:01.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73ed60c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f73e815d1e0 0x7f73e815d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73ece0b700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f73e806a560 0x7f73e80cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39494/0 (socket says 172.21.15.28:39494) 2023-11-28T11:12:01.138 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73ece0b700 1 -- 172.21.15.28:0/3791570272 learned_addr learned my addr 172.21.15.28:0/3791570272 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:01.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73ede0d700 1 --2- 172.21.15.28:0/3791570272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73e806aa00 0x7f73e80cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73ed60c700 1 -- 172.21.15.28:0/3791570272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73e806aa00 msgr2=0x7f73e80cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:01.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73ed60c700 1 --2- 172.21.15.28:0/3791570272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73e806aa00 0x7f73e80cd5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73ed60c700 1 -- 172.21.15.28:0/3791570272 >> v1:172.21.15.28:6789/0 conn(0x7f73e806a560 legacy=0x7f73e80cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73ed60c700 1 -- 172.21.15.28:0/3791570272 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e80c9fb0 con 0x7f73e815d1e0 2023-11-28T11:12:01.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73dffff700 1 -- 172.21.15.28:0/3791570272 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2331946050 0 0) 0x7f73e806ae80 con 0x7f73e806a560 2023-11-28T11:12:01.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73ede0d700 1 --2- 172.21.15.28:0/3791570272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73e806aa00 0x7f73e80cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:01.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.136+0000 7f73ed60c700 1 --2- 172.21.15.28:0/3791570272 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73e815d1e0 0x7f73e815d5b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009840 tx=0x7f73e4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2287cf0d7a372e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:01.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.137+0000 7f73dffff700 1 -- 172.21.15.28:0/3791570272 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e4011830 con 0x7f73e815d1e0 2023-11-28T11:12:01.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.137+0000 7f73dffff700 1 -- 172.21.15.28:0/3791570272 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e4011990 con 0x7f73e815d1e0 2023-11-28T11:12:01.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.137+0000 7f73dffff700 1 -- 172.21.15.28:0/3791570272 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e4011d20 con 0x7f73e815d1e0 2023-11-28T11:12:01.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.137+0000 7f73ef870700 1 -- 172.21.15.28:0/3791570272 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73e815d1e0 msgr2=0x7f73e815d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.137+0000 7f73ef870700 1 --2- 172.21.15.28:0/3791570272 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73e815d1e0 0x7f73e815d5b0 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f73e4009840 tx=0x7f73e4009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:01.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.137+0000 7f73ef870700 1 -- 172.21.15.28:0/3791570272 shutdown_connections 2023-11-28T11:12:01.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.137+0000 7f73ef870700 1 --2- 172.21.15.28:0/3791570272 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73e806aa00 0x7f73e80cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.137+0000 7f73ef870700 1 --2- 172.21.15.28:0/3791570272 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73e815d1e0 0x7f73e815d5b0 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.137+0000 7f73ef870700 1 -- 172.21.15.28:0/3791570272 >> 172.21.15.28:0/3791570272 conn(0x7f73e80b0dd0 msgr2=0x7f73e80b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:01.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.137+0000 7f73ef870700 1 -- 172.21.15.28:0/3791570272 shutdown_connections 2023-11-28T11:12:01.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.137+0000 7f73ef870700 1 -- 172.21.15.28:0/3791570272 wait complete. 2023-11-28T11:12:01.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.138+0000 7f73ef870700 1 Processor -- start 2023-11-28T11:12:01.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.139+0000 7f73ef870700 1 -- start start 2023-11-28T11:12:01.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.139+0000 7f73ef870700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73e806a560 0x7f73e81f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.139+0000 7f73ef870700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73e806aa00 0x7f73e80b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.139+0000 7f73ef870700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73e815d1e0 0x7f73e80b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.139+0000 7f73ef870700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80b4370 con 0x7f73e806a560 2023-11-28T11:12:01.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.139+0000 7f73ef870700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80ac7b0 con 0x7f73e815d1e0 2023-11-28T11:12:01.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.139+0000 7f73ef870700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80ab9f0 con 0x7f73e806aa00 2023-11-28T11:12:01.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.139+0000 7f73ed60c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73e806a560 0x7f73e81f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.139+0000 7f73ed60c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73e806a560 0x7f73e81f18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51440/0 (socket says 172.21.15.28:51440) 2023-11-28T11:12:01.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.139+0000 7f73ed60c700 1 -- 172.21.15.28:0/2002326434 learned_addr learned my addr 172.21.15.28:0/2002326434 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:01.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.140+0000 7f73ece0b700 1 --2- 172.21.15.28:0/2002326434 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73e806aa00 0x7f73e80b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.140+0000 7f73ed60c700 1 -- 172.21.15.28:0/2002326434 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73e806aa00 msgr2=0x7f73e80b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.140+0000 7f73ede0d700 1 --2- 172.21.15.28:0/2002326434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73e815d1e0 0x7f73e80b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.140+0000 7f73ed60c700 1 --2- 172.21.15.28:0/2002326434 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73e806aa00 0x7f73e80b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.140+0000 7f73ed60c700 1 -- 172.21.15.28:0/2002326434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73e815d1e0 msgr2=0x7f73e80b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.140+0000 7f73ed60c700 1 --2- 172.21.15.28:0/2002326434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73e815d1e0 0x7f73e80b3690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.140+0000 7f73ed60c700 1 -- 172.21.15.28:0/2002326434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e40094a0 con 0x7f73e806a560 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.140+0000 7f73ece0b700 1 --2- 172.21.15.28:0/2002326434 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73e806aa00 0x7f73e80b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.140+0000 7f73ed60c700 1 --2- 172.21.15.28:0/2002326434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73e806a560 0x7f73e81f18e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f73e40090b0 tx=0x7f73e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.140+0000 7f73ddffb700 1 -- 172.21.15.28:0/2002326434 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e40038a0 con 0x7f73e806a560 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.140+0000 7f73ddffb700 1 -- 172.21.15.28:0/2002326434 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e401f070 con 0x7f73e806a560 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.140+0000 7f73ef870700 1 -- 172.21.15.28:0/2002326434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e80ca570 con 0x7f73e806a560 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.140+0000 7f73ddffb700 1 -- 172.21.15.28:0/2002326434 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e4003c20 con 0x7f73e806a560 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.140+0000 7f73ef870700 1 -- 172.21.15.28:0/2002326434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e80ca290 con 0x7f73e806a560 2023-11-28T11:12:01.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.141+0000 7f73ef870700 1 -- 172.21.15.28:0/2002326434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e80ca850 con 0x7f73e806a560 2023-11-28T11:12:01.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.142+0000 7f73ddffb700 1 -- 172.21.15.28:0/2002326434 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f73e400f050 con 0x7f73e806a560 2023-11-28T11:12:01.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.142+0000 7f73ddffb700 1 --2- 172.21.15.28:0/2002326434 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f73cc0682f0 0x7f73cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.142+0000 7f73ddffb700 1 -- 172.21.15.28:0/2002326434 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7f73e4021070 con 0x7f73e806a560 2023-11-28T11:12:01.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.142+0000 7f73ece0b700 1 --2- 172.21.15.28:0/2002326434 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f73cc0682f0 0x7f73cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.143+0000 7f73ece0b700 1 --2- 172.21.15.28:0/2002326434 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f73cc0682f0 0x7f73cc06a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f73d8007fd0 tx=0x7f73d8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:01.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.146+0000 7f73ddffb700 1 -- 172.21.15.28:0/2002326434 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f73e4053f20 con 0x7f73e806a560 2023-11-28T11:12:01.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.344+0000 7f73ef870700 1 -- 172.21.15.28:0/2002326434 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f73e80675a0 con 0x7f73e806a560 2023-11-28T11:12:01.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.344+0000 7f73ddffb700 1 -- 172.21.15.28:0/2002326434 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v62) v1 ==== 84+0+126 (secure 0 0 0) 0x7f73e4015090 con 0x7f73e806a560 2023-11-28T11:12:01.345 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:01.345 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2023-11-28T11:12:01.345 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:12:01.345 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 1, 2023-11-28T11:12:01.345 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": "rbd" 2023-11-28T11:12:01.345 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:12:01.345 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:12:01.346 INFO:teuthology.orchestra.run.smithi028.stdout: "poolnum": 2, 2023-11-28T11:12:01.346 INFO:teuthology.orchestra.run.smithi028.stdout: "poolname": ".mgr" 2023-11-28T11:12:01.346 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:12:01.346 INFO:teuthology.orchestra.run.smithi028.stdout:] 2023-11-28T11:12:01.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.346+0000 7f73ef870700 1 -- 172.21.15.28:0/2002326434 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f73cc0682f0 msgr2=0x7f73cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.346+0000 7f73ef870700 1 --2- 172.21.15.28:0/2002326434 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f73cc0682f0 0x7f73cc06a7a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f73d8007fd0 tx=0x7f73d8009040 comp rx=0 tx=0).stop 2023-11-28T11:12:01.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.346+0000 7f73ef870700 1 -- 172.21.15.28:0/2002326434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73e806a560 msgr2=0x7f73e81f18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.346+0000 7f73ef870700 1 --2- 172.21.15.28:0/2002326434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73e806a560 0x7f73e81f18e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f73e40090b0 tx=0x7f73e4004750 comp rx=0 tx=0).stop 2023-11-28T11:12:01.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.347+0000 7f73ef870700 1 -- 172.21.15.28:0/2002326434 shutdown_connections 2023-11-28T11:12:01.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.347+0000 7f73ef870700 1 --2- 172.21.15.28:0/2002326434 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f73cc0682f0 0x7f73cc06a7a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.347+0000 7f73ef870700 1 --2- 172.21.15.28:0/2002326434 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f73e806a560 0x7f73e81f18e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.347+0000 7f73ef870700 1 --2- 172.21.15.28:0/2002326434 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f73e815d1e0 0x7f73e80b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.347+0000 7f73ef870700 1 --2- 172.21.15.28:0/2002326434 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f73e806aa00 0x7f73e80b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.347+0000 7f73ef870700 1 -- 172.21.15.28:0/2002326434 >> 172.21.15.28:0/2002326434 conn(0x7f73e80b0dd0 msgr2=0x7f73e80b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:01.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.347+0000 7f73ef870700 1 -- 172.21.15.28:0/2002326434 shutdown_connections 2023-11-28T11:12:01.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.347+0000 7f73ef870700 1 -- 172.21.15.28:0/2002326434 wait complete. 2023-11-28T11:12:01.359 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T11:12:01.359 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-11-28T11:12:01.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.488+0000 7ff1238fd700 1 Processor -- start 2023-11-28T11:12:01.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.488+0000 7ff1238fd700 1 -- start start 2023-11-28T11:12:01.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.489+0000 7ff1238fd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff11c0cb470 0x7ff11c0ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.489+0000 7ff1238fd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff11c06bb80 0x7ff11c14fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.489+0000 7ff1238fd700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff11c06c140 con 0x7ff11c0cb0a0 2023-11-28T11:12:01.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.489+0000 7ff1238fd700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11c0caad0 con 0x7ff11c0cb470 2023-11-28T11:12:01.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.489+0000 7ff1238fd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff11c0c9ab0 con 0x7ff11c06bb80 2023-11-28T11:12:01.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.489+0000 7ff120e98700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff11c0cb470 0x7ff11c0ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.489+0000 7ff120e98700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff11c0cb470 0x7ff11c0ca590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51450/0 (socket says 172.21.15.28:51450) 2023-11-28T11:12:01.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.489+0000 7ff120e98700 1 -- 172.21.15.28:0/3268219172 learned_addr learned my addr 172.21.15.28:0/3268219172 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:01.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.490+0000 7ff121e9a700 1 --2- 172.21.15.28:0/3268219172 >> v2:172.21.15.59:3300/0 conn(0x7ff11c06bb80 0x7ff11c14fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.490+0000 7ff120e98700 1 -- 172.21.15.28:0/3268219172 >> v2:172.21.15.59:3300/0 conn(0x7ff11c06bb80 msgr2=0x7ff11c14fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:01.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.490+0000 7ff120e98700 1 --2- 172.21.15.28:0/3268219172 >> v2:172.21.15.59:3300/0 conn(0x7ff11c06bb80 0x7ff11c14fea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.490+0000 7ff120e98700 1 -- 172.21.15.28:0/3268219172 >> v1:172.21.15.59:6789/0 conn(0x7ff11c0cb0a0 legacy=0x7ff11c15a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:01.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.490+0000 7ff120e98700 1 -- 172.21.15.28:0/3268219172 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c0c9d90 con 0x7ff11c0cb470 2023-11-28T11:12:01.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.490+0000 7ff120e98700 1 --2- 172.21.15.28:0/3268219172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff11c0cb470 0x7ff11c0ca590 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff11c070fa0 tx=0x7ff118009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4354badfd223044 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:01.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.490+0000 7ff113fff700 1 -- 172.21.15.28:0/3268219172 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff118004230 con 0x7ff11c0cb470 2023-11-28T11:12:01.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.490+0000 7ff113fff700 1 -- 172.21.15.28:0/3268219172 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff118004390 con 0x7ff11c0cb470 2023-11-28T11:12:01.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.490+0000 7ff113fff700 1 -- 172.21.15.28:0/3268219172 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1180036a0 con 0x7ff11c0cb470 2023-11-28T11:12:01.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.491+0000 7ff1238fd700 1 -- 172.21.15.28:0/3268219172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff11c0cb470 msgr2=0x7ff11c0ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.491+0000 7ff1238fd700 1 --2- 172.21.15.28:0/3268219172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff11c0cb470 0x7ff11c0ca590 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7ff11c070fa0 tx=0x7ff118009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.491+0000 7ff1238fd700 1 -- 172.21.15.28:0/3268219172 shutdown_connections 2023-11-28T11:12:01.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.491+0000 7ff1238fd700 1 --2- 172.21.15.28:0/3268219172 >> v2:172.21.15.59:3300/0 conn(0x7ff11c06bb80 0x7ff11c14fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.491+0000 7ff1238fd700 1 --2- 172.21.15.28:0/3268219172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff11c0cb470 0x7ff11c0ca590 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.491+0000 7ff1238fd700 1 -- 172.21.15.28:0/3268219172 >> 172.21.15.28:0/3268219172 conn(0x7ff11c0b0dd0 msgr2=0x7ff11c0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:01.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.491+0000 7ff1238fd700 1 -- 172.21.15.28:0/3268219172 shutdown_connections 2023-11-28T11:12:01.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.491+0000 7ff1238fd700 1 -- 172.21.15.28:0/3268219172 wait complete. 2023-11-28T11:12:01.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.492+0000 7ff1238fd700 1 Processor -- start 2023-11-28T11:12:01.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.492+0000 7ff1238fd700 1 -- start start 2023-11-28T11:12:01.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff1238fd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff11c06bb80 0x7ff11c147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff1238fd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff11c0cb0a0 0x7ff11c155710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff1238fd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff11c0cb470 0x7ff11c155c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff1238fd700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c147c90 con 0x7ff11c0cb470 2023-11-28T11:12:01.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff1238fd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c0b4d90 con 0x7ff11c06bb80 2023-11-28T11:12:01.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff1238fd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff11c1562d0 con 0x7ff11c0cb0a0 2023-11-28T11:12:01.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff121e9a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff11c0cb470 0x7ff11c155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff121e9a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff11c0cb470 0x7ff11c155c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51458/0 (socket says 172.21.15.28:51458) 2023-11-28T11:12:01.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff121e9a700 1 -- 172.21.15.28:0/4095233064 learned_addr learned my addr 172.21.15.28:0/4095233064 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:01.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff121699700 1 --2- 172.21.15.28:0/4095233064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff11c06bb80 0x7ff11c147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff120e98700 1 --2- 172.21.15.28:0/4095233064 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff11c0cb0a0 0x7ff11c155710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff121e9a700 1 -- 172.21.15.28:0/4095233064 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff11c0cb0a0 msgr2=0x7ff11c155710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff121e9a700 1 --2- 172.21.15.28:0/4095233064 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff11c0cb0a0 0x7ff11c155710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff121e9a700 1 -- 172.21.15.28:0/4095233064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff11c06bb80 msgr2=0x7ff11c147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:01.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff121e9a700 1 --2- 172.21.15.28:0/4095233064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff11c06bb80 0x7ff11c147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:01.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff121e9a700 1 -- 172.21.15.28:0/4095233064 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff118009550 con 0x7ff11c0cb470 2023-11-28T11:12:01.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff121e9a700 1 --2- 172.21.15.28:0/4095233064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff11c0cb470 0x7ff11c155c50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff11400cd70 tx=0x7ff11400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:01.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff111ffb700 1 -- 172.21.15.28:0/4095233064 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114003a20 con 0x7ff11c0cb470 2023-11-28T11:12:01.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff111ffb700 1 -- 172.21.15.28:0/4095233064 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff114003b80 con 0x7ff11c0cb470 2023-11-28T11:12:01.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff1238fd700 1 -- 172.21.15.28:0/4095233064 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff11c1503e0 con 0x7ff11c0cb470 2023-11-28T11:12:01.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.493+0000 7ff111ffb700 1 -- 172.21.15.28:0/4095233064 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff114004020 con 0x7ff11c0cb470 2023-11-28T11:12:01.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.494+0000 7ff1238fd700 1 -- 172.21.15.28:0/4095233064 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff11c0ca070 con 0x7ff11c0cb470 2023-11-28T11:12:01.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.495+0000 7ff1238fd700 1 -- 172.21.15.28:0/4095233064 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff11c150570 con 0x7ff11c0cb470 2023-11-28T11:12:01.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.495+0000 7ff111ffb700 1 -- 172.21.15.28:0/4095233064 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff11401c020 con 0x7ff11c0cb470 2023-11-28T11:12:01.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.495+0000 7ff111ffb700 1 --2- 172.21.15.28:0/4095233064 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff100068320 0x7ff10006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:01.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.496+0000 7ff111ffb700 1 -- 172.21.15.28:0/4095233064 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5530+0+0 (secure 0 0 0) 0x7ff114012070 con 0x7ff11c0cb470 2023-11-28T11:12:01.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.496+0000 7ff121699700 1 --2- 172.21.15.28:0/4095233064 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff100068320 0x7ff10006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:01.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.496+0000 7ff121699700 1 --2- 172.21.15.28:0/4095233064 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff100068320 0x7ff10006a7d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0cee80 tx=0x7ff10c007ec0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:01.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.500+0000 7ff111ffb700 1 -- 172.21.15.28:0/4095233064 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff114052b70 con 0x7ff11c0cb470 2023-11-28T11:12:01.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:01.693+0000 7ff1238fd700 1 -- 172.21.15.28:0/4095233064 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff11c0739f0 con 0x7ff11c0cb470 2023-11-28T11:12:02.377 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.377+0000 7ff111ffb700 1 -- 172.21.15.28:0/4095233064 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v63) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff114056190 con 0x7ff11c0cb470 2023-11-28T11:12:02.381 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.381+0000 7ff1238fd700 1 -- 172.21.15.28:0/4095233064 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff100068320 msgr2=0x7ff10006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.381+0000 7ff1238fd700 1 --2- 172.21.15.28:0/4095233064 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff100068320 0x7ff10006a7d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7ff11c0cee80 tx=0x7ff10c007ec0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.381+0000 7ff1238fd700 1 -- 172.21.15.28:0/4095233064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff11c0cb470 msgr2=0x7ff11c155c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.381+0000 7ff1238fd700 1 --2- 172.21.15.28:0/4095233064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff11c0cb470 0x7ff11c155c50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7ff11400cd70 tx=0x7ff11400a570 comp rx=0 tx=0).stop 2023-11-28T11:12:02.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.382+0000 7ff1238fd700 1 -- 172.21.15.28:0/4095233064 shutdown_connections 2023-11-28T11:12:02.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.382+0000 7ff1238fd700 1 --2- 172.21.15.28:0/4095233064 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff100068320 0x7ff10006a7d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.382 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.382+0000 7ff1238fd700 1 --2- 172.21.15.28:0/4095233064 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff11c0cb470 0x7ff11c155c50 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.382+0000 7ff1238fd700 1 --2- 172.21.15.28:0/4095233064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff11c06bb80 0x7ff11c147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.382+0000 7ff1238fd700 1 --2- 172.21.15.28:0/4095233064 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff11c0cb0a0 0x7ff11c155710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.382+0000 7ff1238fd700 1 -- 172.21.15.28:0/4095233064 >> 172.21.15.28:0/4095233064 conn(0x7ff11c0b0dd0 msgr2=0x7ff11c15d3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:02.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.383+0000 7ff1238fd700 1 -- 172.21.15.28:0/4095233064 shutdown_connections 2023-11-28T11:12:02.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.384+0000 7ff1238fd700 1 -- 172.21.15.28:0/4095233064 wait complete. 2023-11-28T11:12:02.386 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' created 2023-11-28T11:12:02.400 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T11:12:02.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.538+0000 7f002345a700 1 Processor -- start 2023-11-28T11:12:02.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.538+0000 7f002345a700 1 -- start start 2023-11-28T11:12:02.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.538+0000 7f002345a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f001c0cd3e0 0x7f001c06a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.538+0000 7f002345a700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f001c06bb90 con 0x7f001c0d0fe0 2023-11-28T11:12:02.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.539+0000 7f002345a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f001c0cfa50 con 0x7f001c15fa10 2023-11-28T11:12:02.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.539+0000 7f002345a700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f001c0cd880 con 0x7f001c0cd3e0 2023-11-28T11:12:02.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.539+0000 7f00209f5700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f001c0cd3e0 0x7f001c06a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.539+0000 7f00219f7700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f001c0d0fe0 0x7f001c0d1420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39524/0 (socket says 172.21.15.28:39524) 2023-11-28T11:12:02.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.539+0000 7f00219f7700 1 -- 172.21.15.28:0/1122814932 learned_addr learned my addr 172.21.15.28:0/1122814932 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:02.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.539+0000 7f00209f5700 1 -- 172.21.15.28:0/1122814932 >> v1:172.21.15.59:6789/0 conn(0x7f001c15fa10 legacy=0x7f001c06a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:02.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.539+0000 7f00209f5700 1 -- 172.21.15.28:0/1122814932 >> v1:172.21.15.28:6789/0 conn(0x7f001c0d0fe0 legacy=0x7f001c0d1420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.539+0000 7f0013fff700 1 -- 172.21.15.28:0/1122814932 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704128442 0 0) 0x7f001c06bb90 con 0x7f001c0d0fe0 2023-11-28T11:12:02.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.539+0000 7f00209f5700 1 -- 172.21.15.28:0/1122814932 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f001c0d0ae0 con 0x7f001c0cd3e0 2023-11-28T11:12:02.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.539+0000 7f00209f5700 1 --2- 172.21.15.28:0/1122814932 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f001c0cd3e0 0x7f001c06a8f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0018009840 tx=0x7f0018009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4507df6afbc427d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:02.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.539+0000 7f0013fff700 1 -- 172.21.15.28:0/1122814932 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0018004230 con 0x7f001c0cd3e0 2023-11-28T11:12:02.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.539+0000 7f0013fff700 1 -- 172.21.15.28:0/1122814932 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0018004390 con 0x7f001c0cd3e0 2023-11-28T11:12:02.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.539+0000 7f0013fff700 1 -- 172.21.15.28:0/1122814932 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00180036a0 con 0x7f001c0cd3e0 2023-11-28T11:12:02.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.540+0000 7f002345a700 1 -- 172.21.15.28:0/1122814932 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f001c0cd3e0 msgr2=0x7f001c06a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.540+0000 7f002345a700 1 --2- 172.21.15.28:0/1122814932 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f001c0cd3e0 0x7f001c06a8f0 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0018009840 tx=0x7f0018009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:02.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.540+0000 7f002345a700 1 -- 172.21.15.28:0/1122814932 shutdown_connections 2023-11-28T11:12:02.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.540+0000 7f002345a700 1 --2- 172.21.15.28:0/1122814932 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f001c0cd3e0 0x7f001c06a8f0 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.540+0000 7f002345a700 1 -- 172.21.15.28:0/1122814932 >> 172.21.15.28:0/1122814932 conn(0x7f001c06ce40 msgr2=0x7f001c0b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:02.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.540+0000 7f002345a700 1 -- 172.21.15.28:0/1122814932 shutdown_connections 2023-11-28T11:12:02.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.540+0000 7f002345a700 1 -- 172.21.15.28:0/1122814932 wait complete. 2023-11-28T11:12:02.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.541+0000 7f002345a700 1 Processor -- start 2023-11-28T11:12:02.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.541+0000 7f002345a700 1 -- start start 2023-11-28T11:12:02.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f002345a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f001c0cd3e0 0x7f001c0cc990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f002345a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f001c0d0fe0 0x7f001c1591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f002345a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f001c15fa10 0x7f001c14c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f002345a700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c15c4f0 con 0x7f001c15fa10 2023-11-28T11:12:02.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f002345a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f001c159730 con 0x7f001c0d0fe0 2023-11-28T11:12:02.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f002345a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001c15bb50 con 0x7f001c0cd3e0 2023-11-28T11:12:02.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f00219f7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f001c15fa10 0x7f001c14c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f00219f7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f001c15fa10 0x7f001c14c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51472/0 (socket says 172.21.15.28:51472) 2023-11-28T11:12:02.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f00219f7700 1 -- 172.21.15.28:0/3562259673 learned_addr learned my addr 172.21.15.28:0/3562259673 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:02.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f00209f5700 1 --2- 172.21.15.28:0/3562259673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f001c0d0fe0 0x7f001c1591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f00211f6700 1 --2- 172.21.15.28:0/3562259673 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f001c0cd3e0 0x7f001c0cc990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f00219f7700 1 -- 172.21.15.28:0/3562259673 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f001c0cd3e0 msgr2=0x7f001c0cc990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f00219f7700 1 --2- 172.21.15.28:0/3562259673 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f001c0cd3e0 0x7f001c0cc990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f00219f7700 1 -- 172.21.15.28:0/3562259673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f001c0d0fe0 msgr2=0x7f001c1591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:02.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f00219f7700 1 --2- 172.21.15.28:0/3562259673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f001c0d0fe0 0x7f001c1591f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:02.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.542+0000 7f00219f7700 1 -- 172.21.15.28:0/3562259673 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00180094a0 con 0x7f001c15fa10 2023-11-28T11:12:02.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.543+0000 7f00209f5700 1 --2- 172.21.15.28:0/3562259673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f001c0d0fe0 0x7f001c1591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:02.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.543+0000 7f00219f7700 1 --2- 172.21.15.28:0/3562259673 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f001c15fa10 0x7f001c14c000 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f001400cca0 tx=0x7f001400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:02.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.543+0000 7f0011ffb700 1 -- 172.21.15.28:0/3562259673 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f001400cea0 con 0x7f001c15fa10 2023-11-28T11:12:02.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.543+0000 7f0011ffb700 1 -- 172.21.15.28:0/3562259673 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f001400ade0 con 0x7f001c15fa10 2023-11-28T11:12:02.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.543+0000 7f0011ffb700 1 -- 172.21.15.28:0/3562259673 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0014007310 con 0x7f001c15fa10 2023-11-28T11:12:02.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.543+0000 7f002345a700 1 -- 172.21.15.28:0/3562259673 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f001c0cff80 con 0x7f001c15fa10 2023-11-28T11:12:02.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.543+0000 7f002345a700 1 -- 172.21.15.28:0/3562259673 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f001c0cfca0 con 0x7f001c15fa10 2023-11-28T11:12:02.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.544+0000 7f002345a700 1 -- 172.21.15.28:0/3562259673 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f001c15fde0 con 0x7f001c15fa10 2023-11-28T11:12:02.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.545+0000 7f0011ffb700 1 -- 172.21.15.28:0/3562259673 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0014010070 con 0x7f001c15fa10 2023-11-28T11:12:02.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.545+0000 7f0011ffb700 1 --2- 172.21.15.28:0/3562259673 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00000682f0 0x7f000006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:02.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.545+0000 7f0011ffb700 1 -- 172.21.15.28:0/3562259673 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 5900+0+0 (secure 0 0 0) 0x7f00140857a0 con 0x7f001c15fa10 2023-11-28T11:12:02.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.545+0000 7f00211f6700 1 --2- 172.21.15.28:0/3562259673 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00000682f0 0x7f000006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:02.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.545+0000 7f00211f6700 1 --2- 172.21.15.28:0/3562259673 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00000682f0 0x7f000006a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f000c003e60 tx=0x7f000c009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:02.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.550+0000 7f0011ffb700 1 -- 172.21.15.28:0/3562259673 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f00140520a0 con 0x7f001c15fa10 2023-11-28T11:12:02.746 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:02.746+0000 7f002345a700 1 -- 172.21.15.28:0/3562259673 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f001c0739f0 con 0x7f001c15fa10 2023-11-28T11:12:03.384 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.384+0000 7f0011ffb700 1 -- 172.21.15.28:0/3562259673 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v64) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0014018020 con 0x7f001c15fa10 2023-11-28T11:12:03.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.389+0000 7f002345a700 1 -- 172.21.15.28:0/3562259673 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00000682f0 msgr2=0x7f000006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.389+0000 7f002345a700 1 --2- 172.21.15.28:0/3562259673 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00000682f0 0x7f000006a7a0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f000c003e60 tx=0x7f000c009040 comp rx=0 tx=0).stop 2023-11-28T11:12:03.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.389+0000 7f002345a700 1 -- 172.21.15.28:0/3562259673 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f001c15fa10 msgr2=0x7f001c14c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.390 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.390+0000 7f002345a700 1 --2- 172.21.15.28:0/3562259673 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f001c15fa10 0x7f001c14c000 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f001400cca0 tx=0x7f001400a340 comp rx=0 tx=0).stop 2023-11-28T11:12:03.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.390+0000 7f002345a700 1 -- 172.21.15.28:0/3562259673 shutdown_connections 2023-11-28T11:12:03.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.390+0000 7f002345a700 1 --2- 172.21.15.28:0/3562259673 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00000682f0 0x7f000006a7a0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.390+0000 7f002345a700 1 --2- 172.21.15.28:0/3562259673 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f001c15fa10 0x7f001c14c000 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.390+0000 7f002345a700 1 --2- 172.21.15.28:0/3562259673 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f001c0cd3e0 0x7f001c0cc990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.390+0000 7f002345a700 1 --2- 172.21.15.28:0/3562259673 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f001c0d0fe0 0x7f001c1591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.390+0000 7f002345a700 1 -- 172.21.15.28:0/3562259673 >> 172.21.15.28:0/3562259673 conn(0x7f001c06ce40 msgr2=0x7f001c15f230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:03.391 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.391+0000 7f002345a700 1 -- 172.21.15.28:0/3562259673 shutdown_connections 2023-11-28T11:12:03.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.394+0000 7f002345a700 1 -- 172.21.15.28:0/3562259673 wait complete. 2023-11-28T11:12:03.404 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' created 2023-11-28T11:12:03.416 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T11:12:03.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.549+0000 7f3aee12c700 1 Processor -- start 2023-11-28T11:12:03.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.549+0000 7f3aee12c700 1 -- start start 2023-11-28T11:12:03.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3aee12c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ae80b62d0 0x7f3ae80cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3aee12c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3ae806f8b0 0x7f3ae80b2680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3aee12c700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae80b3020 con 0x7f3ae80b2bc0 2023-11-28T11:12:03.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3aee12c700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae80cd2a0 con 0x7f3ae806f8b0 2023-11-28T11:12:03.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3aee12c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae80d0400 con 0x7f3ae80b62d0 2023-11-28T11:12:03.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3ae6ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3ae806f8b0 0x7f3ae80b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3ae6ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f3ae806f8b0 0x7f3ae80b2680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51478/0 (socket says 172.21.15.28:51478) 2023-11-28T11:12:03.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3ae6ffd700 1 -- 172.21.15.28:0/711546037 learned_addr learned my addr 172.21.15.28:0/711546037 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:03.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3ae7fff700 1 --1- 172.21.15.28:0/711546037 >> v1:172.21.15.28:6789/0 conn(0x7f3ae80b2bc0 0x7f3ae80ce7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39538/0 (socket says 172.21.15.28:39538) 2023-11-28T11:12:03.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3ae77fe700 1 --2- 172.21.15.28:0/711546037 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ae80b62d0 0x7f3ae80cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3ae6ffd700 1 -- 172.21.15.28:0/711546037 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ae80b62d0 msgr2=0x7f3ae80cb630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:03.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3ae6ffd700 1 --2- 172.21.15.28:0/711546037 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ae80b62d0 0x7f3ae80cb630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3ae6ffd700 1 -- 172.21.15.28:0/711546037 >> v1:172.21.15.28:6789/0 conn(0x7f3ae80b2bc0 legacy=0x7f3ae80ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3ae67fc700 1 -- 172.21.15.28:0/711546037 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3914755595 0 0) 0x7f3ae80b3020 con 0x7f3ae80b2bc0 2023-11-28T11:12:03.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3ae6ffd700 1 -- 172.21.15.28:0/711546037 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae80d06e0 con 0x7f3ae806f8b0 2023-11-28T11:12:03.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.550+0000 7f3ae6ffd700 1 --2- 172.21.15.28:0/711546037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae806f8b0 0x7f3ae80b2680 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00098f0 tx=0x7f3ad0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22d767064bfc80c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:03.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.551+0000 7f3ae67fc700 1 -- 172.21.15.28:0/711546037 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad000b8a0 con 0x7f3ae806f8b0 2023-11-28T11:12:03.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.551+0000 7f3ae67fc700 1 -- 172.21.15.28:0/711546037 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad000ba00 con 0x7f3ae806f8b0 2023-11-28T11:12:03.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.551+0000 7f3ae67fc700 1 -- 172.21.15.28:0/711546037 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad000bd90 con 0x7f3ae806f8b0 2023-11-28T11:12:03.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.551+0000 7f3aee12c700 1 -- 172.21.15.28:0/711546037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae806f8b0 msgr2=0x7f3ae80b2680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.551+0000 7f3aee12c700 1 --2- 172.21.15.28:0/711546037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae806f8b0 0x7f3ae80b2680 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f3ad00098f0 tx=0x7f3ad0009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:03.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.551+0000 7f3aee12c700 1 -- 172.21.15.28:0/711546037 shutdown_connections 2023-11-28T11:12:03.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.551+0000 7f3aee12c700 1 --2- 172.21.15.28:0/711546037 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ae80b62d0 0x7f3ae80cb630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.551+0000 7f3aee12c700 1 --2- 172.21.15.28:0/711546037 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae806f8b0 0x7f3ae80b2680 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.551+0000 7f3aee12c700 1 -- 172.21.15.28:0/711546037 >> 172.21.15.28:0/711546037 conn(0x7f3ae80cd680 msgr2=0x7f3ae80cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:03.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.551+0000 7f3aee12c700 1 -- 172.21.15.28:0/711546037 shutdown_connections 2023-11-28T11:12:03.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.551+0000 7f3aee12c700 1 -- 172.21.15.28:0/711546037 wait complete. 2023-11-28T11:12:03.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.552+0000 7f3aee12c700 1 Processor -- start 2023-11-28T11:12:03.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.553+0000 7f3aee12c700 1 -- start start 2023-11-28T11:12:03.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.553+0000 7f3aee12c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae806f8b0 0x7f3ae80c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.553+0000 7f3aee12c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ae80b2bc0 0x7f3ae81ef950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.553+0000 7f3aee12c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ae80b62d0 0x7f3ae806bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.553+0000 7f3aee12c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae806c250 con 0x7f3ae806f8b0 2023-11-28T11:12:03.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.553+0000 7f3aee12c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae806c4d0 con 0x7f3ae80b2bc0 2023-11-28T11:12:03.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.553+0000 7f3aee12c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae806c7b0 con 0x7f3ae80b62d0 2023-11-28T11:12:03.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.553+0000 7f3ae77fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae806f8b0 0x7f3ae80c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.553+0000 7f3ae77fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae806f8b0 0x7f3ae80c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51482/0 (socket says 172.21.15.28:51482) 2023-11-28T11:12:03.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.553+0000 7f3ae77fe700 1 -- 172.21.15.28:0/2121565266 learned_addr learned my addr 172.21.15.28:0/2121565266 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:03.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.554+0000 7f3ae6ffd700 1 --2- 172.21.15.28:0/2121565266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ae80b2bc0 0x7f3ae81ef950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.554+0000 7f3ae7fff700 1 --2- 172.21.15.28:0/2121565266 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ae80b62d0 0x7f3ae806bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.554+0000 7f3ae77fe700 1 -- 172.21.15.28:0/2121565266 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ae80b62d0 msgr2=0x7f3ae806bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.554+0000 7f3ae77fe700 1 --2- 172.21.15.28:0/2121565266 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ae80b62d0 0x7f3ae806bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.554+0000 7f3ae77fe700 1 -- 172.21.15.28:0/2121565266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ae80b2bc0 msgr2=0x7f3ae81ef950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.554+0000 7f3ae77fe700 1 --2- 172.21.15.28:0/2121565266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ae80b2bc0 0x7f3ae81ef950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.554+0000 7f3ae77fe700 1 -- 172.21.15.28:0/2121565266 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad0009550 con 0x7f3ae806f8b0 2023-11-28T11:12:03.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.554+0000 7f3ae77fe700 1 --2- 172.21.15.28:0/2121565266 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae806f8b0 0x7f3ae80c7fb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800d800 tx=0x7f3ad800db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:03.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.554+0000 7f3adffff700 1 -- 172.21.15.28:0/2121565266 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad80125f0 con 0x7f3ae806f8b0 2023-11-28T11:12:03.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.554+0000 7f3adffff700 1 -- 172.21.15.28:0/2121565266 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad8012750 con 0x7f3ae806f8b0 2023-11-28T11:12:03.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.554+0000 7f3aee12c700 1 -- 172.21.15.28:0/2121565266 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae80b66a0 con 0x7f3ae806f8b0 2023-11-28T11:12:03.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.554+0000 7f3adffff700 1 -- 172.21.15.28:0/2121565266 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8012b00 con 0x7f3ae806f8b0 2023-11-28T11:12:03.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.554+0000 7f3aee12c700 1 -- 172.21.15.28:0/2121565266 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae80cffb0 con 0x7f3ae806f8b0 2023-11-28T11:12:03.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.555+0000 7f3aee12c700 1 -- 172.21.15.28:0/2121565266 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae806fd20 con 0x7f3ae806f8b0 2023-11-28T11:12:03.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.556+0000 7f3adffff700 1 -- 172.21.15.28:0/2121565266 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3ad8012c60 con 0x7f3ae806f8b0 2023-11-28T11:12:03.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.556+0000 7f3adffff700 1 --2- 172.21.15.28:0/2121565266 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ac80682f0 0x7f3ac806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.556+0000 7f3adffff700 1 -- 172.21.15.28:0/2121565266 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6282+0+0 (secure 0 0 0) 0x7f3ad8085f50 con 0x7f3ae806f8b0 2023-11-28T11:12:03.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.556+0000 7f3ae6ffd700 1 --2- 172.21.15.28:0/2121565266 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ac80682f0 0x7f3ac806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.559 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.556+0000 7f3ae6ffd700 1 --2- 172.21.15.28:0/2121565266 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ac80682f0 0x7f3ac806a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009c40 tx=0x7f3ad0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:03.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.561+0000 7f3adffff700 1 -- 172.21.15.28:0/2121565266 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3ad80526d0 con 0x7f3ae806f8b0 2023-11-28T11:12:03.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.799+0000 7f3aee12c700 1 -- 172.21.15.28:0/2121565266 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3ae806d3c0 con 0x7f3ae806f8b0 2023-11-28T11:12:03.806 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.805+0000 7f3adffff700 1 -- 172.21.15.28:0/2121565266 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v29) v1 ==== 174+0+0 (secure 0 0 0) 0x7f3ad8055cf0 con 0x7f3ae806f8b0 2023-11-28T11:12:03.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.815+0000 7f3aee12c700 1 -- 172.21.15.28:0/2121565266 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ac80682f0 msgr2=0x7f3ac806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.815+0000 7f3aee12c700 1 --2- 172.21.15.28:0/2121565266 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ac80682f0 0x7f3ac806a7a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3ad0009c40 tx=0x7f3ad0008030 comp rx=0 tx=0).stop 2023-11-28T11:12:03.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.815+0000 7f3aee12c700 1 -- 172.21.15.28:0/2121565266 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae806f8b0 msgr2=0x7f3ae80c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.815+0000 7f3aee12c700 1 --2- 172.21.15.28:0/2121565266 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae806f8b0 0x7f3ae80c7fb0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f3ad800d800 tx=0x7f3ad800db10 comp rx=0 tx=0).stop 2023-11-28T11:12:03.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.815+0000 7f3aee12c700 1 -- 172.21.15.28:0/2121565266 shutdown_connections 2023-11-28T11:12:03.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.815+0000 7f3aee12c700 1 --2- 172.21.15.28:0/2121565266 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ac80682f0 0x7f3ac806a7a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.815+0000 7f3aee12c700 1 --2- 172.21.15.28:0/2121565266 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3ae806f8b0 0x7f3ae80c7fb0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.815+0000 7f3aee12c700 1 --2- 172.21.15.28:0/2121565266 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3ae80b62d0 0x7f3ae806bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.815+0000 7f3aee12c700 1 --2- 172.21.15.28:0/2121565266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ae80b2bc0 0x7f3ae81ef950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.815+0000 7f3aee12c700 1 -- 172.21.15.28:0/2121565266 >> 172.21.15.28:0/2121565266 conn(0x7f3ae80cd680 msgr2=0x7f3ae80cf560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:03.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.815+0000 7f3aee12c700 1 -- 172.21.15.28:0/2121565266 shutdown_connections 2023-11-28T11:12:03.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.815+0000 7f3aee12c700 1 -- 172.21.15.28:0/2121565266 wait complete. 2023-11-28T11:12:03.823 INFO:teuthology.orchestra.run.smithi028.stderr:new fs with metadata pool 10 and data pool 11 2023-11-28T11:12:03.836 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:12:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.960+0000 7f2a623bf700 1 Processor -- start 2023-11-28T11:12:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.960+0000 7f2a623bf700 1 -- start start 2023-11-28T11:12:03.964 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.961+0000 7f2a623bf700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2a5c15c990 0x7f2a5c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.961+0000 7f2a623bf700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5c0ce340 con 0x7f2a5c0cc310 2023-11-28T11:12:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.961+0000 7f2a623bf700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5c0cebe0 con 0x7f2a5c161090 2023-11-28T11:12:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.961+0000 7f2a623bf700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a5c0cb3f0 con 0x7f2a5c15c990 2023-11-28T11:12:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.961+0000 7f2a6095c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2a5c0cc310 0x7f2a5c0cc770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39552/0 (socket says 172.21.15.28:39552) 2023-11-28T11:12:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.961+0000 7f2a6095c700 1 -- 172.21.15.28:0/2519883398 learned_addr learned my addr 172.21.15.28:0/2519883398 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.961+0000 7f2a5bfff700 1 --2- 172.21.15.28:0/2519883398 >> v2:172.21.15.59:3300/0 conn(0x7f2a5c15c990 0x7f2a5c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.961+0000 7f2a5affd700 1 -- 172.21.15.28:0/2519883398 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865449673 0 0) 0x7f2a5c0ce340 con 0x7f2a5c0cc310 2023-11-28T11:12:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.961+0000 7f2a5affd700 1 -- 172.21.15.28:0/2519883398 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a480035b0 con 0x7f2a5c0cc310 2023-11-28T11:12:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5affd700 1 -- 172.21.15.28:0/2519883398 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2070218993 0 0) 0x7f2a480035b0 con 0x7f2a5c0cc310 2023-11-28T11:12:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5affd700 1 -- 172.21.15.28:0/2519883398 >> v2:172.21.15.59:3300/0 conn(0x7f2a5c15c990 msgr2=0x7f2a5c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5affd700 1 --2- 172.21.15.28:0/2519883398 >> v2:172.21.15.59:3300/0 conn(0x7f2a5c15c990 0x7f2a5c0cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5affd700 1 -- 172.21.15.28:0/2519883398 >> v1:172.21.15.59:6789/0 conn(0x7f2a5c161090 legacy=0x7f2a5c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5affd700 1 -- 172.21.15.28:0/2519883398 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c0ce590 con 0x7f2a5c0cc310 2023-11-28T11:12:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5affd700 1 -- 172.21.15.28:0/2519883398 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f2a540019f0 con 0x7f2a5c0cc310 2023-11-28T11:12:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5affd700 1 -- 172.21.15.28:0/2519883398 >> v1:172.21.15.28:6789/0 conn(0x7f2a5c0cc310 legacy=0x7f2a5c0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5affd700 1 --2- 172.21.15.28:0/2519883398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a48003a00 0x7f2a48003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5affd700 1 --2- 172.21.15.28:0/2519883398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a48004850 0x7f2a48004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5affd700 1 --2- 172.21.15.28:0/2519883398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a48005320 0x7f2a4800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5affd700 1 -- 172.21.15.28:0/2519883398 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0cb3f0 con 0x7f2a48003a00 2023-11-28T11:12:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5affd700 1 -- 172.21.15.28:0/2519883398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4800c250 con 0x7f2a48004850 2023-11-28T11:12:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5affd700 1 -- 172.21.15.28:0/2519883398 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4800c410 con 0x7f2a48005320 2023-11-28T11:12:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5bfff700 1 --2- 172.21.15.28:0/2519883398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a48003a00 0x7f2a48003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5b7fe700 1 --2- 172.21.15.28:0/2519883398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a48004850 0x7f2a48004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a6095c700 1 --2- 172.21.15.28:0/2519883398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a48005320 0x7f2a4800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5bfff700 1 -- 172.21.15.28:0/2519883398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a48005320 msgr2=0x7f2a4800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5bfff700 1 --2- 172.21.15.28:0/2519883398 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a48005320 0x7f2a4800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5bfff700 1 -- 172.21.15.28:0/2519883398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a48004850 msgr2=0x7f2a48004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5bfff700 1 --2- 172.21.15.28:0/2519883398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a48004850 0x7f2a48004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5bfff700 1 -- 172.21.15.28:0/2519883398 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a480035b0 con 0x7f2a48003a00 2023-11-28T11:12:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a6095c700 1 -- 172.21.15.28:0/2519883398 reap_dead start 2023-11-28T11:12:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.962+0000 7f2a5bfff700 1 --2- 172.21.15.28:0/2519883398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a48003a00 0x7f2a48003e10 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009b60 tx=0x7f2a48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96186e0e6363a132 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.963+0000 7f2a5affd700 1 -- 172.21.15.28:0/2519883398 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c00e6c0 con 0x7f2a48003a00 2023-11-28T11:12:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.963+0000 7f2a5affd700 1 -- 172.21.15.28:0/2519883398 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c00e820 con 0x7f2a48003a00 2023-11-28T11:12:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.963+0000 7f2a5affd700 1 -- 172.21.15.28:0/2519883398 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c00eb90 con 0x7f2a48003a00 2023-11-28T11:12:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.963+0000 7f2a623bf700 1 -- 172.21.15.28:0/2519883398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a48003a00 msgr2=0x7f2a48003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.963+0000 7f2a623bf700 1 --2- 172.21.15.28:0/2519883398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a48003a00 0x7f2a48003e10 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c009b60 tx=0x7f2a48005880 comp rx=0 tx=0).stop 2023-11-28T11:12:03.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.963+0000 7f2a623bf700 1 -- 172.21.15.28:0/2519883398 shutdown_connections 2023-11-28T11:12:03.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.963+0000 7f2a623bf700 1 --2- 172.21.15.28:0/2519883398 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a48003a00 0x7f2a48003e10 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.963+0000 7f2a623bf700 1 -- 172.21.15.28:0/2519883398 >> 172.21.15.28:0/2519883398 conn(0x7f2a5c0b1650 msgr2=0x7f2a5c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:03.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.963+0000 7f2a623bf700 1 -- 172.21.15.28:0/2519883398 shutdown_connections 2023-11-28T11:12:03.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.963+0000 7f2a623bf700 1 -- 172.21.15.28:0/2519883398 wait complete. 2023-11-28T11:12:03.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.964+0000 7f2a623bf700 1 Processor -- start 2023-11-28T11:12:03.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.965+0000 7f2a623bf700 1 -- start start 2023-11-28T11:12:03.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.965+0000 7f2a623bf700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a48003a00 0x7f2a5c1508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.965+0000 7f2a623bf700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a5c0b4220 0x7f2a5c0b4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.965+0000 7f2a623bf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5c1f5b80 0x7f2a5c1f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.965+0000 7f2a623bf700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c0b4bd0 con 0x7f2a48003a00 2023-11-28T11:12:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.965+0000 7f2a623bf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c1f25a0 con 0x7f2a5c1f5b80 2023-11-28T11:12:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.965+0000 7f2a623bf700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5c150e10 con 0x7f2a5c0b4220 2023-11-28T11:12:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.965+0000 7f2a5bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a48003a00 0x7f2a5c1508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.965+0000 7f2a5bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a48003a00 0x7f2a5c1508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51502/0 (socket says 172.21.15.28:51502) 2023-11-28T11:12:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.965+0000 7f2a5bfff700 1 -- 172.21.15.28:0/657287228 learned_addr learned my addr 172.21.15.28:0/657287228 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.966+0000 7f2a5b7fe700 1 --2- 172.21.15.28:0/657287228 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a5c0b4220 0x7f2a5c0b4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.966+0000 7f2a6095c700 1 --2- 172.21.15.28:0/657287228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5c1f5b80 0x7f2a5c1f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.966+0000 7f2a5bfff700 1 -- 172.21.15.28:0/657287228 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a5c0b4220 msgr2=0x7f2a5c0b4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.966+0000 7f2a5bfff700 1 --2- 172.21.15.28:0/657287228 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a5c0b4220 0x7f2a5c0b4690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.966+0000 7f2a5bfff700 1 -- 172.21.15.28:0/657287228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5c1f5b80 msgr2=0x7f2a5c1f5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.966+0000 7f2a5bfff700 1 --2- 172.21.15.28:0/657287228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5c1f5b80 0x7f2a5c1f5ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.966+0000 7f2a5bfff700 1 -- 172.21.15.28:0/657287228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5c0cc770 con 0x7f2a48003a00 2023-11-28T11:12:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.966+0000 7f2a6095c700 1 --2- 172.21.15.28:0/657287228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5c1f5b80 0x7f2a5c1f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.966+0000 7f2a5bfff700 1 --2- 172.21.15.28:0/657287228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a48003a00 0x7f2a5c1508d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00e4b0 tx=0x7f2a4c0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.966+0000 7f2a58ff9700 1 -- 172.21.15.28:0/657287228 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c00e6c0 con 0x7f2a48003a00 2023-11-28T11:12:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.966+0000 7f2a58ff9700 1 -- 172.21.15.28:0/657287228 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c003ad0 con 0x7f2a48003a00 2023-11-28T11:12:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.966+0000 7f2a623bf700 1 -- 172.21.15.28:0/657287228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a5c0ce870 con 0x7f2a48003a00 2023-11-28T11:12:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.966+0000 7f2a58ff9700 1 -- 172.21.15.28:0/657287228 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c003e80 con 0x7f2a48003a00 2023-11-28T11:12:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.966+0000 7f2a623bf700 1 -- 172.21.15.28:0/657287228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a5c0cc900 con 0x7f2a48003a00 2023-11-28T11:12:03.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.967+0000 7f2a623bf700 1 -- 172.21.15.28:0/657287228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5c1f3140 con 0x7f2a48003a00 2023-11-28T11:12:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.967+0000 7f2a58ff9700 1 -- 172.21.15.28:0/657287228 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2a4c019030 con 0x7f2a48003a00 2023-11-28T11:12:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.968+0000 7f2a58ff9700 1 --2- 172.21.15.28:0/657287228 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2a3c0682f0 0x7f2a3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.968+0000 7f2a58ff9700 1 -- 172.21.15.28:0/657287228 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2a4c0873b0 con 0x7f2a48003a00 2023-11-28T11:12:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.968+0000 7f2a5b7fe700 1 --2- 172.21.15.28:0/657287228 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2a3c0682f0 0x7f2a3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.968+0000 7f2a5b7fe700 1 --2- 172.21.15.28:0/657287228 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2a3c0682f0 0x7f2a3c06a7a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c1516d0 tx=0x7f2a44009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:03.973 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:03.972+0000 7f2a58ff9700 1 -- 172.21.15.28:0/657287228 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2a4c053ad0 con 0x7f2a48003a00 2023-11-28T11:12:04.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.165+0000 7f2a623bf700 1 -- 172.21.15.28:0/657287228 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2a5c0675a0 con 0x7f2a48003a00 2023-11-28T11:12:04.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.166+0000 7f2a58ff9700 1 -- 172.21.15.28:0/657287228 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18090 (secure 0 0 0) 0x7f2a4c0570f0 con 0x7f2a48003a00 2023-11-28T11:12:04.167 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:04.167 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":65,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:12:03.800571+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:12:01.724396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:12:02.776612+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:12:04.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.168+0000 7f2a623bf700 1 -- 172.21.15.28:0/657287228 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2a3c0682f0 msgr2=0x7f2a3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.168+0000 7f2a623bf700 1 --2- 172.21.15.28:0/657287228 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2a3c0682f0 0x7f2a3c06a7a0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c1516d0 tx=0x7f2a44009040 comp rx=0 tx=0).stop 2023-11-28T11:12:04.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.168+0000 7f2a623bf700 1 -- 172.21.15.28:0/657287228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a48003a00 msgr2=0x7f2a5c1508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.168+0000 7f2a623bf700 1 --2- 172.21.15.28:0/657287228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a48003a00 0x7f2a5c1508d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c00e4b0 tx=0x7f2a4c0047c0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.168+0000 7f2a623bf700 1 -- 172.21.15.28:0/657287228 shutdown_connections 2023-11-28T11:12:04.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.168+0000 7f2a623bf700 1 --2- 172.21.15.28:0/657287228 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2a3c0682f0 0x7f2a3c06a7a0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.168+0000 7f2a623bf700 1 --2- 172.21.15.28:0/657287228 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2a48003a00 0x7f2a5c1508d0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.169 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.168+0000 7f2a623bf700 1 --2- 172.21.15.28:0/657287228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2a5c1f5b80 0x7f2a5c1f5ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.168+0000 7f2a623bf700 1 --2- 172.21.15.28:0/657287228 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2a5c0b4220 0x7f2a5c0b4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.168+0000 7f2a623bf700 1 -- 172.21.15.28:0/657287228 >> 172.21.15.28:0/657287228 conn(0x7f2a5c0b1650 msgr2=0x7f2a5c06d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:04.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.168+0000 7f2a623bf700 1 -- 172.21.15.28:0/657287228 shutdown_connections 2023-11-28T11:12:04.170 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.168+0000 7f2a623bf700 1 -- 172.21.15.28:0/657287228 wait complete. 2023-11-28T11:12:04.182 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:12:04.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.306+0000 7f03effdb700 1 Processor -- start 2023-11-28T11:12:04.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.306+0000 7f03effdb700 1 -- start start 2023-11-28T11:12:04.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.307+0000 7f03effdb700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f03e815a650 0x7f03e80b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.307+0000 7f03effdb700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e80ca990 con 0x7f03e80b5500 2023-11-28T11:12:04.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.307+0000 7f03effdb700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e80caf50 con 0x7f03e815aa20 2023-11-28T11:12:04.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.307+0000 7f03effdb700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e80aef10 con 0x7f03e815a650 2023-11-28T11:12:04.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.307+0000 7f03ee578700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f03e80b5500 0x7f03e80cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39570/0 (socket says 172.21.15.28:39570) 2023-11-28T11:12:04.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.307+0000 7f03ee578700 1 -- 172.21.15.28:0/4277661865 learned_addr learned my addr 172.21.15.28:0/4277661865 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:04.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.307+0000 7f03edd77700 1 --2- 172.21.15.28:0/4277661865 >> v2:172.21.15.59:3300/0 conn(0x7f03e815a650 0x7f03e80b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.307+0000 7f03ecd75700 1 -- 172.21.15.28:0/4277661865 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 61889911 0 0) 0x7f03e80ca990 con 0x7f03e80b5500 2023-11-28T11:12:04.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.307+0000 7f03ecd75700 1 -- 172.21.15.28:0/4277661865 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03d40035b0 con 0x7f03e80b5500 2023-11-28T11:12:04.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.307+0000 7f03edd77700 1 -- 172.21.15.28:0/4277661865 >> v1:172.21.15.59:6789/0 conn(0x7f03e815aa20 legacy=0x7f03e80ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.307+0000 7f03ecd75700 1 -- 172.21.15.28:0/4277661865 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2708723932 0 0) 0x7f03d40035b0 con 0x7f03e80b5500 2023-11-28T11:12:04.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.307+0000 7f03edd77700 1 -- 172.21.15.28:0/4277661865 >> v1:172.21.15.28:6789/0 conn(0x7f03e80b5500 legacy=0x7f03e80cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.310 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.307+0000 7f03edd77700 1 -- 172.21.15.28:0/4277661865 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e80af1f0 con 0x7f03e815a650 2023-11-28T11:12:04.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.308+0000 7f03edd77700 1 --2- 172.21.15.28:0/4277661865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03e815a650 0x7f03e80b2950 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009840 tx=0x7f03e4009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d993df7b901e3bde server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:04.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.308+0000 7f03ecd75700 1 -- 172.21.15.28:0/4277661865 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e00019f0 con 0x7f03e815a650 2023-11-28T11:12:04.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.308+0000 7f03ecd75700 1 -- 172.21.15.28:0/4277661865 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e4004030 con 0x7f03e815a650 2023-11-28T11:12:04.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.308+0000 7f03ecd75700 1 -- 172.21.15.28:0/4277661865 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e40043c0 con 0x7f03e815a650 2023-11-28T11:12:04.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.308+0000 7f03effdb700 1 -- 172.21.15.28:0/4277661865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03e815a650 msgr2=0x7f03e80b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.308+0000 7f03effdb700 1 --2- 172.21.15.28:0/4277661865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03e815a650 0x7f03e80b2950 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009840 tx=0x7f03e4009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:04.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.308+0000 7f03effdb700 1 -- 172.21.15.28:0/4277661865 shutdown_connections 2023-11-28T11:12:04.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.308+0000 7f03effdb700 1 --2- 172.21.15.28:0/4277661865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03e815a650 0x7f03e80b2950 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.308+0000 7f03effdb700 1 -- 172.21.15.28:0/4277661865 >> 172.21.15.28:0/4277661865 conn(0x7f03e80b1650 msgr2=0x7f03e80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:04.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.309+0000 7f03effdb700 1 -- 172.21.15.28:0/4277661865 shutdown_connections 2023-11-28T11:12:04.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.309+0000 7f03effdb700 1 -- 172.21.15.28:0/4277661865 wait complete. 2023-11-28T11:12:04.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.310+0000 7f03effdb700 1 Processor -- start 2023-11-28T11:12:04.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.310+0000 7f03effdb700 1 -- start start 2023-11-28T11:12:04.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.310+0000 7f03effdb700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e80b5500 0x7f03e815ae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.310+0000 7f03effdb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03e815a650 0x7f03e80cb650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03effdb700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03e815aa20 0x7f03e81531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03effdb700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e815b680 con 0x7f03e80b5500 2023-11-28T11:12:04.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03effdb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e80cd670 con 0x7f03e815a650 2023-11-28T11:12:04.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03effdb700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e80ce640 con 0x7f03e815aa20 2023-11-28T11:12:04.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03edd77700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e80b5500 0x7f03e815ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03edd77700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e80b5500 0x7f03e815ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51514/0 (socket says 172.21.15.28:51514) 2023-11-28T11:12:04.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03edd77700 1 -- 172.21.15.28:0/930017586 learned_addr learned my addr 172.21.15.28:0/930017586 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:04.315 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03ed576700 1 --2- 172.21.15.28:0/930017586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03e815a650 0x7f03e80cb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03ee578700 1 --2- 172.21.15.28:0/930017586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03e815aa20 0x7f03e81531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03edd77700 1 -- 172.21.15.28:0/930017586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03e815aa20 msgr2=0x7f03e81531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03edd77700 1 --2- 172.21.15.28:0/930017586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03e815aa20 0x7f03e81531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03edd77700 1 -- 172.21.15.28:0/930017586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03e815a650 msgr2=0x7f03e80cb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03edd77700 1 --2- 172.21.15.28:0/930017586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03e815a650 0x7f03e80cb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03edd77700 1 -- 172.21.15.28:0/930017586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e40094a0 con 0x7f03e80b5500 2023-11-28T11:12:04.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03edd77700 1 --2- 172.21.15.28:0/930017586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e80b5500 0x7f03e815ae50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009b90 tx=0x7f03e4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:04.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03de7fc700 1 -- 172.21.15.28:0/930017586 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e4011630 con 0x7f03e80b5500 2023-11-28T11:12:04.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03de7fc700 1 -- 172.21.15.28:0/930017586 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e401f070 con 0x7f03e80b5500 2023-11-28T11:12:04.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03effdb700 1 -- 172.21.15.28:0/930017586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e80ccbc0 con 0x7f03e80b5500 2023-11-28T11:12:04.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03de7fc700 1 -- 172.21.15.28:0/930017586 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e40119d0 con 0x7f03e80b5500 2023-11-28T11:12:04.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.311+0000 7f03effdb700 1 -- 172.21.15.28:0/930017586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e80cabe0 con 0x7f03e80b5500 2023-11-28T11:12:04.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.312+0000 7f03effdb700 1 -- 172.21.15.28:0/930017586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e80ccd50 con 0x7f03e80b5500 2023-11-28T11:12:04.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.313+0000 7f03de7fc700 1 -- 172.21.15.28:0/930017586 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f03e4017020 con 0x7f03e80b5500 2023-11-28T11:12:04.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.313+0000 7f03de7fc700 1 --2- 172.21.15.28:0/930017586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f03cc0682f0 0x7f03cc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.313+0000 7f03ed576700 1 --2- 172.21.15.28:0/930017586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f03cc0682f0 0x7f03cc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.313+0000 7f03de7fc700 1 -- 172.21.15.28:0/930017586 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f03e4021070 con 0x7f03e80b5500 2023-11-28T11:12:04.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.314+0000 7f03ed576700 1 --2- 172.21.15.28:0/930017586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f03cc0682f0 0x7f03cc06a7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f03e81f5bb0 tx=0x7f03d8009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:04.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.318+0000 7f03de7fc700 1 -- 172.21.15.28:0/930017586 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f03e4053d40 con 0x7f03e80b5500 2023-11-28T11:12:04.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.512+0000 7f03effdb700 1 -- 172.21.15.28:0/930017586 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03e806c310 con 0x7f03e80b5500 2023-11-28T11:12:04.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.513+0000 7f03de7fc700 1 -- 172.21.15.28:0/930017586 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+18090 (secure 0 0 0) 0x7f03e4057360 con 0x7f03e80b5500 2023-11-28T11:12:04.514 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:04.514 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":65,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:12:03.800571+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:12:01.724396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:12:02.776612+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:12:04.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.515+0000 7f03effdb700 1 -- 172.21.15.28:0/930017586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f03cc0682f0 msgr2=0x7f03cc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.515+0000 7f03effdb700 1 --2- 172.21.15.28:0/930017586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f03cc0682f0 0x7f03cc06a7a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f03e81f5bb0 tx=0x7f03d8009040 comp rx=0 tx=0).stop 2023-11-28T11:12:04.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.515+0000 7f03effdb700 1 -- 172.21.15.28:0/930017586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e80b5500 msgr2=0x7f03e815ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.515+0000 7f03effdb700 1 --2- 172.21.15.28:0/930017586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e80b5500 0x7f03e815ae50 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f03e4009b90 tx=0x7f03e4004750 comp rx=0 tx=0).stop 2023-11-28T11:12:04.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.515+0000 7f03effdb700 1 -- 172.21.15.28:0/930017586 shutdown_connections 2023-11-28T11:12:04.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.515+0000 7f03effdb700 1 --2- 172.21.15.28:0/930017586 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f03cc0682f0 0x7f03cc06a7a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.515+0000 7f03effdb700 1 --2- 172.21.15.28:0/930017586 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03e80b5500 0x7f03e815ae50 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.515+0000 7f03effdb700 1 --2- 172.21.15.28:0/930017586 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f03e815aa20 0x7f03e81531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.515+0000 7f03effdb700 1 --2- 172.21.15.28:0/930017586 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f03e815a650 0x7f03e80cb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.515+0000 7f03effdb700 1 -- 172.21.15.28:0/930017586 >> 172.21.15.28:0/930017586 conn(0x7f03e80b1650 msgr2=0x7f03e8155f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:04.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.515+0000 7f03effdb700 1 -- 172.21.15.28:0/930017586 shutdown_connections 2023-11-28T11:12:04.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.515+0000 7f03effdb700 1 -- 172.21.15.28:0/930017586 wait complete. 2023-11-28T11:12:04.528 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T11:12:04.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.655+0000 7f315c6c0700 1 Processor -- start 2023-11-28T11:12:04.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.655+0000 7f315c6c0700 1 -- start start 2023-11-28T11:12:04.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.656+0000 7f315c6c0700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f31540b1630 0x7f31540cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.656+0000 7f315c6c0700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31540ce550 con 0x7f315406afc0 2023-11-28T11:12:04.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.656+0000 7f315c6c0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31540ce830 con 0x7f31540b1a00 2023-11-28T11:12:04.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.656+0000 7f315c6c0700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31540ceb10 con 0x7f31540b1630 2023-11-28T11:12:04.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.656+0000 7f315a45c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f31540b1630 0x7f31540cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.656+0000 7f315ac5d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f315406afc0 0x7f315406b420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39584/0 (socket says 172.21.15.28:39584) 2023-11-28T11:12:04.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.656+0000 7f315a45c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f31540b1630 0x7f31540cf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51522/0 (socket says 172.21.15.28:51522) 2023-11-28T11:12:04.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.656+0000 7f315ac5d700 1 -- 172.21.15.28:0/1786857271 learned_addr learned my addr 172.21.15.28:0/1786857271 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:04.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.657+0000 7f315a45c700 1 -- 172.21.15.28:0/1786857271 >> v1:172.21.15.59:6789/0 conn(0x7f31540b1a00 legacy=0x7f31540caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:04.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.657+0000 7f315a45c700 1 -- 172.21.15.28:0/1786857271 >> v1:172.21.15.28:6789/0 conn(0x7f315406afc0 legacy=0x7f315406b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.657+0000 7f315a45c700 1 -- 172.21.15.28:0/1786857271 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31540cedf0 con 0x7f31540b1630 2023-11-28T11:12:04.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.657+0000 7f315945a700 1 -- 172.21.15.28:0/1786857271 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 451845457 0 0) 0x7f31540ce550 con 0x7f315406afc0 2023-11-28T11:12:04.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.657+0000 7f315a45c700 1 --2- 172.21.15.28:0/1786857271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31540b1630 0x7f31540cf5f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f3150009870 tx=0x7f3150009b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d8f53e0e35ac203 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:04.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.657+0000 7f315945a700 1 -- 172.21.15.28:0/1786857271 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3150004230 con 0x7f31540b1630 2023-11-28T11:12:04.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.657+0000 7f315945a700 1 -- 172.21.15.28:0/1786857271 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3150004390 con 0x7f31540b1630 2023-11-28T11:12:04.660 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.657+0000 7f315945a700 1 -- 172.21.15.28:0/1786857271 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31500036a0 con 0x7f31540b1630 2023-11-28T11:12:04.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.658+0000 7f315c6c0700 1 -- 172.21.15.28:0/1786857271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31540b1630 msgr2=0x7f31540cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.658+0000 7f315c6c0700 1 --2- 172.21.15.28:0/1786857271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31540b1630 0x7f31540cf5f0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f3150009870 tx=0x7f3150009b80 comp rx=0 tx=0).stop 2023-11-28T11:12:04.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.658+0000 7f315c6c0700 1 -- 172.21.15.28:0/1786857271 shutdown_connections 2023-11-28T11:12:04.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.658+0000 7f315c6c0700 1 --2- 172.21.15.28:0/1786857271 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31540b1630 0x7f31540cf5f0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.658+0000 7f315c6c0700 1 -- 172.21.15.28:0/1786857271 >> 172.21.15.28:0/1786857271 conn(0x7f31540cc630 msgr2=0x7f31540cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:04.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.658+0000 7f315c6c0700 1 -- 172.21.15.28:0/1786857271 shutdown_connections 2023-11-28T11:12:04.661 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.658+0000 7f315c6c0700 1 -- 172.21.15.28:0/1786857271 wait complete. 2023-11-28T11:12:04.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.659+0000 7f315c6c0700 1 Processor -- start 2023-11-28T11:12:04.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.659+0000 7f315c6c0700 1 -- start start 2023-11-28T11:12:04.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f315c6c0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f315406afc0 0x7f31540b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f315c6c0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31540b1630 0x7f31540c83a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f315c6c0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f31540b1a00 0x7f31540aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f315c6c0700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31540b45e0 con 0x7f31540b1630 2023-11-28T11:12:04.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f315c6c0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31540c88e0 con 0x7f315406afc0 2023-11-28T11:12:04.665 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f315c6c0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31540b39d0 con 0x7f31540b1a00 2023-11-28T11:12:04.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f3159c5b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31540b1630 0x7f31540c83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f3159c5b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31540b1630 0x7f31540c83a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51530/0 (socket says 172.21.15.28:51530) 2023-11-28T11:12:04.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f3159c5b700 1 -- 172.21.15.28:0/841852872 learned_addr learned my addr 172.21.15.28:0/841852872 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:04.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f315a45c700 1 --2- 172.21.15.28:0/841852872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f315406afc0 0x7f31540b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f315ac5d700 1 --2- 172.21.15.28:0/841852872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f31540b1a00 0x7f31540aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f3159c5b700 1 -- 172.21.15.28:0/841852872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f31540b1a00 msgr2=0x7f31540aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f3159c5b700 1 --2- 172.21.15.28:0/841852872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f31540b1a00 0x7f31540aceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f3159c5b700 1 -- 172.21.15.28:0/841852872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f315406afc0 msgr2=0x7f31540b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:04.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f3159c5b700 1 --2- 172.21.15.28:0/841852872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f315406afc0 0x7f31540b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:04.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f3159c5b700 1 -- 172.21.15.28:0/841852872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31500094a0 con 0x7f31540b1630 2023-11-28T11:12:04.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f315a45c700 1 --2- 172.21.15.28:0/841852872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f315406afc0 0x7f31540b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:04.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f3159c5b700 1 --2- 172.21.15.28:0/841852872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31540b1630 0x7f31540c83a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f314400b4a0 tx=0x7f314400b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:04.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f314affd700 1 -- 172.21.15.28:0/841852872 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3144011830 con 0x7f31540b1630 2023-11-28T11:12:04.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f314affd700 1 -- 172.21.15.28:0/841852872 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3144011990 con 0x7f31540b1630 2023-11-28T11:12:04.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f314affd700 1 -- 172.21.15.28:0/841852872 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3144011d40 con 0x7f31540b1630 2023-11-28T11:12:04.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.660+0000 7f315c6c0700 1 -- 172.21.15.28:0/841852872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31540b1ea0 con 0x7f31540b1630 2023-11-28T11:12:04.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.661+0000 7f315c6c0700 1 -- 172.21.15.28:0/841852872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31540cf0d0 con 0x7f31540b1630 2023-11-28T11:12:04.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.661+0000 7f315c6c0700 1 -- 172.21.15.28:0/841852872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31540cb5b0 con 0x7f31540b1630 2023-11-28T11:12:04.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.662+0000 7f314affd700 1 -- 172.21.15.28:0/841852872 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f314400f050 con 0x7f31540b1630 2023-11-28T11:12:04.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.662+0000 7f314affd700 1 --2- 172.21.15.28:0/841852872 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f31380682f0 0x7f313806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:04.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.662+0000 7f314affd700 1 -- 172.21.15.28:0/841852872 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f3144085a40 con 0x7f31540b1630 2023-11-28T11:12:04.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.662+0000 7f315a45c700 1 --2- 172.21.15.28:0/841852872 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f31380682f0 0x7f313806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:04.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.663+0000 7f315a45c700 1 --2- 172.21.15.28:0/841852872 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f31380682f0 0x7f313806a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3150000c00 tx=0x7f3150008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:04.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.667+0000 7f314affd700 1 -- 172.21.15.28:0/841852872 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3144052010 con 0x7f31540b1630 2023-11-28T11:12:04.907 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:04.907+0000 7f315c6c0700 1 -- 172.21.15.28:0/841852872 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f31540739f0 con 0x7f31540b1630 2023-11-28T11:12:05.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.813+0000 7f314affd700 1 -- 172.21.15.28:0/841852872 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v32) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3144055630 con 0x7f31540b1630 2023-11-28T11:12:05.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.816+0000 7f315c6c0700 1 -- 172.21.15.28:0/841852872 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f31380682f0 msgr2=0x7f313806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:05.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.816+0000 7f315c6c0700 1 --2- 172.21.15.28:0/841852872 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f31380682f0 0x7f313806a7a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f3150000c00 tx=0x7f3150008030 comp rx=0 tx=0).stop 2023-11-28T11:12:05.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.816+0000 7f315c6c0700 1 -- 172.21.15.28:0/841852872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31540b1630 msgr2=0x7f31540c83a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:05.817 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.816+0000 7f315c6c0700 1 --2- 172.21.15.28:0/841852872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31540b1630 0x7f31540c83a0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f314400b4a0 tx=0x7f314400b860 comp rx=0 tx=0).stop 2023-11-28T11:12:05.817 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.816+0000 7f315c6c0700 1 -- 172.21.15.28:0/841852872 shutdown_connections 2023-11-28T11:12:05.817 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.816+0000 7f315c6c0700 1 --2- 172.21.15.28:0/841852872 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f31380682f0 0x7f313806a7a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.817 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.816+0000 7f315c6c0700 1 --2- 172.21.15.28:0/841852872 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f31540b1a00 0x7f31540aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.817 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.816+0000 7f315c6c0700 1 --2- 172.21.15.28:0/841852872 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f315406afc0 0x7f31540b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.817 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.816+0000 7f315c6c0700 1 --2- 172.21.15.28:0/841852872 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f31540b1630 0x7f31540c83a0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.817 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.816+0000 7f315c6c0700 1 -- 172.21.15.28:0/841852872 >> 172.21.15.28:0/841852872 conn(0x7f31540cc630 msgr2=0x7f315406c220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:05.817 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.816+0000 7f315c6c0700 1 -- 172.21.15.28:0/841852872 shutdown_connections 2023-11-28T11:12:05.817 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.816+0000 7f315c6c0700 1 -- 172.21.15.28:0/841852872 wait complete. 2023-11-28T11:12:05.831 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T11:12:05.832 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T11:12:05.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.966+0000 7f810277b700 1 Processor -- start 2023-11-28T11:12:05.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.966+0000 7f810277b700 1 -- start start 2023-11-28T11:12:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.967+0000 7f810277b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80fc0b1630 0x7f80fc0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.967+0000 7f810277b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f80fc06afc0 0x7f80fc06b440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.967+0000 7f810277b700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80fc0ce550 con 0x7f80fc0b1a00 2023-11-28T11:12:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.967+0000 7f810277b700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80fc0ce830 con 0x7f80fc06afc0 2023-11-28T11:12:05.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.967+0000 7f810277b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc0ceb10 con 0x7f80fc0b1630 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.967+0000 7f8100d18700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f80fc06afc0 0x7f80fc06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.967+0000 7f8100d18700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f80fc06afc0 0x7f80fc06b440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51540/0 (socket says 172.21.15.28:51540) 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.967+0000 7f8100d18700 1 -- 172.21.15.28:0/3087374375 learned_addr learned my addr 172.21.15.28:0/3087374375 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.967+0000 7f80fbfff700 1 --2- 172.21.15.28:0/3087374375 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80fc0b1630 0x7f80fc0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.967+0000 7f8100d18700 1 -- 172.21.15.28:0/3087374375 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80fc0b1630 msgr2=0x7f80fc0cf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.967+0000 7f8100d18700 1 --2- 172.21.15.28:0/3087374375 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80fc0b1630 0x7f80fc0cf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.967+0000 7f8100d18700 1 -- 172.21.15.28:0/3087374375 >> v1:172.21.15.59:6789/0 conn(0x7f80fc0b1a00 legacy=0x7f80fc0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.967+0000 7f8100d18700 1 -- 172.21.15.28:0/3087374375 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80fc0cedf0 con 0x7f80fc06afc0 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.967+0000 7f8100d18700 1 --2- 172.21.15.28:0/3087374375 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80fc06afc0 0x7f80fc06b440 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009e50 tx=0x7f80f400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae162c142af9b98a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.968+0000 7f80faffd700 1 -- 172.21.15.28:0/3087374375 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f400e070 con 0x7f80fc06afc0 2023-11-28T11:12:05.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.968+0000 7f80faffd700 1 -- 172.21.15.28:0/3087374375 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80f400b9c0 con 0x7f80fc06afc0 2023-11-28T11:12:05.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.968+0000 7f80faffd700 1 -- 172.21.15.28:0/3087374375 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80f400bd70 con 0x7f80fc06afc0 2023-11-28T11:12:05.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.968+0000 7f810277b700 1 -- 172.21.15.28:0/3087374375 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80fc06afc0 msgr2=0x7f80fc06b440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:05.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.968+0000 7f810277b700 1 --2- 172.21.15.28:0/3087374375 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80fc06afc0 0x7f80fc06b440 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f80f4009e50 tx=0x7f80f400b330 comp rx=0 tx=0).stop 2023-11-28T11:12:05.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.968+0000 7f810277b700 1 -- 172.21.15.28:0/3087374375 shutdown_connections 2023-11-28T11:12:05.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.968+0000 7f810277b700 1 --2- 172.21.15.28:0/3087374375 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80fc06afc0 0x7f80fc06b440 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.968+0000 7f810277b700 1 --2- 172.21.15.28:0/3087374375 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80fc0b1630 0x7f80fc0cf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.968+0000 7f810277b700 1 -- 172.21.15.28:0/3087374375 >> 172.21.15.28:0/3087374375 conn(0x7f80fc0cc630 msgr2=0x7f80fc0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:05.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.968+0000 7f810277b700 1 -- 172.21.15.28:0/3087374375 shutdown_connections 2023-11-28T11:12:05.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.968+0000 7f810277b700 1 -- 172.21.15.28:0/3087374375 wait complete. 2023-11-28T11:12:05.974 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.969+0000 7f810277b700 1 Processor -- start 2023-11-28T11:12:05.974 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.970+0000 7f810277b700 1 -- start start 2023-11-28T11:12:05.974 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.970+0000 7f810277b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80fc06afc0 0x7f80fc157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:05.974 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.970+0000 7f810277b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80fc0b1630 0x7f80fc0c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:05.974 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.970+0000 7f810277b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80fc0b1a00 0x7f80fc0c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.970+0000 7f810277b700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc0c9940 con 0x7f80fc06afc0 2023-11-28T11:12:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.970+0000 7f810277b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc0c5e90 con 0x7f80fc0b1a00 2023-11-28T11:12:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.970+0000 7f810277b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80fc0c5280 con 0x7f80fc0b1630 2023-11-28T11:12:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.970+0000 7f80fbfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80fc06afc0 0x7f80fc157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f80fbfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80fc06afc0 0x7f80fc157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51542/0 (socket says 172.21.15.28:51542) 2023-11-28T11:12:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f80fbfff700 1 -- 172.21.15.28:0/59207956 learned_addr learned my addr 172.21.15.28:0/59207956 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f80fb7fe700 1 --2- 172.21.15.28:0/59207956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80fc0b1630 0x7f80fc0c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f8100d18700 1 --2- 172.21.15.28:0/59207956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80fc0b1a00 0x7f80fc0c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f80fbfff700 1 -- 172.21.15.28:0/59207956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80fc0b1630 msgr2=0x7f80fc0c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f80fbfff700 1 --2- 172.21.15.28:0/59207956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80fc0b1630 0x7f80fc0c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f80fbfff700 1 -- 172.21.15.28:0/59207956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80fc0b1a00 msgr2=0x7f80fc0c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:05.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f80fbfff700 1 --2- 172.21.15.28:0/59207956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80fc0b1a00 0x7f80fc0c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:05.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f80fbfff700 1 -- 172.21.15.28:0/59207956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f4009b30 con 0x7f80fc06afc0 2023-11-28T11:12:05.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f8100d18700 1 --2- 172.21.15.28:0/59207956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80fc0b1a00 0x7f80fc0c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:05.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f80fbfff700 1 --2- 172.21.15.28:0/59207956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80fc06afc0 0x7f80fc157370 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00e810 tx=0x7f80ec00ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:05.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f80f8ff9700 1 -- 172.21.15.28:0/59207956 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec013610 con 0x7f80fc06afc0 2023-11-28T11:12:05.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f80f8ff9700 1 -- 172.21.15.28:0/59207956 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80ec013770 con 0x7f80fc06afc0 2023-11-28T11:12:05.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f810277b700 1 -- 172.21.15.28:0/59207956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80fc0b1e70 con 0x7f80fc06afc0 2023-11-28T11:12:05.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f80f8ff9700 1 -- 172.21.15.28:0/59207956 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80ec013b20 con 0x7f80fc06afc0 2023-11-28T11:12:05.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.971+0000 7f810277b700 1 -- 172.21.15.28:0/59207956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80fc0cf0d0 con 0x7f80fc06afc0 2023-11-28T11:12:05.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.972+0000 7f810277b700 1 -- 172.21.15.28:0/59207956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80fc0cb5b0 con 0x7f80fc06afc0 2023-11-28T11:12:05.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.973+0000 7f80f8ff9700 1 -- 172.21.15.28:0/59207956 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f80ec023070 con 0x7f80fc06afc0 2023-11-28T11:12:05.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.973+0000 7f80f8ff9700 1 --2- 172.21.15.28:0/59207956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80dc0682f0 0x7f80dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:05.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.973+0000 7f80fb7fe700 1 --2- 172.21.15.28:0/59207956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80dc0682f0 0x7f80dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:05.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.973+0000 7f80f8ff9700 1 -- 172.21.15.28:0/59207956 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f80ec085ae0 con 0x7f80fc06afc0 2023-11-28T11:12:05.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.974+0000 7f80fb7fe700 1 --2- 172.21.15.28:0/59207956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80dc0682f0 0x7f80dc06a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0b12c0 tx=0x7f80e4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:05.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:05.978+0000 7f80f8ff9700 1 -- 172.21.15.28:0/59207956 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f80ec052180 con 0x7f80fc06afc0 2023-11-28T11:12:06.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.217+0000 7f810277b700 1 -- 172.21.15.28:0/59207956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f80fc0c6170 con 0x7f80fc06afc0 2023-11-28T11:12:06.816 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.815+0000 7f80f8ff9700 1 -- 172.21.15.28:0/59207956 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v33) v1 ==== 122+0+0 (secure 0 0 0) 0x7f80ec0557a0 con 0x7f80fc06afc0 2023-11-28T11:12:06.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.818+0000 7f810277b700 1 -- 172.21.15.28:0/59207956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80dc0682f0 msgr2=0x7f80dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:06.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.818+0000 7f810277b700 1 --2- 172.21.15.28:0/59207956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80dc0682f0 0x7f80dc06a7a0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f80fc0b12c0 tx=0x7f80e4009040 comp rx=0 tx=0).stop 2023-11-28T11:12:06.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.818+0000 7f810277b700 1 -- 172.21.15.28:0/59207956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80fc06afc0 msgr2=0x7f80fc157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:06.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.818+0000 7f810277b700 1 --2- 172.21.15.28:0/59207956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80fc06afc0 0x7f80fc157370 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f80ec00e810 tx=0x7f80ec00ebd0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.818+0000 7f810277b700 1 -- 172.21.15.28:0/59207956 shutdown_connections 2023-11-28T11:12:06.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.818+0000 7f810277b700 1 --2- 172.21.15.28:0/59207956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80dc0682f0 0x7f80dc06a7a0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.818+0000 7f810277b700 1 --2- 172.21.15.28:0/59207956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80fc06afc0 0x7f80fc157370 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.818+0000 7f810277b700 1 --2- 172.21.15.28:0/59207956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80fc0b1a00 0x7f80fc0c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.818+0000 7f810277b700 1 --2- 172.21.15.28:0/59207956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80fc0b1630 0x7f80fc0c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.819 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.818+0000 7f810277b700 1 -- 172.21.15.28:0/59207956 >> 172.21.15.28:0/59207956 conn(0x7f80fc0cc630 msgr2=0x7f80fc15b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:06.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.818+0000 7f810277b700 1 -- 172.21.15.28:0/59207956 shutdown_connections 2023-11-28T11:12:06.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.818+0000 7f810277b700 1 -- 172.21.15.28:0/59207956 wait complete. 2023-11-28T11:12:06.834 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T11:12:06.834 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:12:06.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.965+0000 7fca0087b700 1 Processor -- start 2023-11-28T11:12:06.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.965+0000 7fca0087b700 1 -- start start 2023-11-28T11:12:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.966+0000 7fca0087b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9fc0ccbc0 0x7fc9fc0ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.966+0000 7fca0087b700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9fc0cb570 con 0x7fc9fc15a5a0 2023-11-28T11:12:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.966+0000 7fca0087b700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9fc0cc110 con 0x7fc9fc06b930 2023-11-28T11:12:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.966+0000 7fca0087b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc06bda0 con 0x7fc9fc0ccbc0 2023-11-28T11:12:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.966+0000 7fc9fa59c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc9fc15a5a0 0x7fc9fc0cba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39608/0 (socket says 172.21.15.28:39608) 2023-11-28T11:12:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.966+0000 7fc9fa59c700 1 -- 172.21.15.28:0/2275102400 learned_addr learned my addr 172.21.15.28:0/2275102400 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:06.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.966+0000 7fc9fad9d700 1 --2- 172.21.15.28:0/2275102400 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9fc0ccbc0 0x7fc9fc0ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.966+0000 7fc9f959a700 1 -- 172.21.15.28:0/2275102400 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632517007 0 0) 0x7fc9fc0cb570 con 0x7fc9fc15a5a0 2023-11-28T11:12:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.966+0000 7fc9f959a700 1 -- 172.21.15.28:0/2275102400 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9e40035b0 con 0x7fc9fc15a5a0 2023-11-28T11:12:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 -- 172.21.15.28:0/2275102400 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 70656742 0 0) 0x7fc9e40035b0 con 0x7fc9fc15a5a0 2023-11-28T11:12:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 -- 172.21.15.28:0/2275102400 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9fc0ccbc0 msgr2=0x7fc9fc0ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 --2- 172.21.15.28:0/2275102400 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9fc0ccbc0 0x7fc9fc0ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 -- 172.21.15.28:0/2275102400 >> v1:172.21.15.59:6789/0 conn(0x7fc9fc06b930 legacy=0x7fc9fc0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 -- 172.21.15.28:0/2275102400 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc0ca960 con 0x7fc9fc15a5a0 2023-11-28T11:12:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 -- 172.21.15.28:0/2275102400 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fc9f0001670 con 0x7fc9fc15a5a0 2023-11-28T11:12:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 -- 172.21.15.28:0/2275102400 >> v1:172.21.15.28:6789/0 conn(0x7fc9fc15a5a0 legacy=0x7fc9fc0cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:06.970 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 --2- 172.21.15.28:0/2275102400 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9fc0ccbc0 0x7fc9e4003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:06.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 --2- 172.21.15.28:0/2275102400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e4003fb0 0x7fc9e4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:06.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 --2- 172.21.15.28:0/2275102400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9e4004d90 0x7fc9e4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:06.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 -- 172.21.15.28:0/2275102400 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc06bda0 con 0x7fc9e4004d90 2023-11-28T11:12:06.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 -- 172.21.15.28:0/2275102400 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e400baf0 con 0x7fc9e4003fb0 2023-11-28T11:12:06.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 -- 172.21.15.28:0/2275102400 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9e400bcb0 con 0x7fc9fc0ccbc0 2023-11-28T11:12:06.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9fad9d700 1 --2- 172.21.15.28:0/2275102400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9e4004d90 0x7fc9e4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:06.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9fa59c700 1 --2- 172.21.15.28:0/2275102400 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9fc0ccbc0 0x7fc9e4003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:06.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f9d9b700 1 --2- 172.21.15.28:0/2275102400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e4003fb0 0x7fc9e4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:06.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9fad9d700 1 -- 172.21.15.28:0/2275102400 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9fc0ccbc0 msgr2=0x7fc9e4003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:06.971 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9fad9d700 1 --2- 172.21.15.28:0/2275102400 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9fc0ccbc0 0x7fc9e4003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9fad9d700 1 -- 172.21.15.28:0/2275102400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e4003fb0 msgr2=0x7fc9e4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:06.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9fad9d700 1 --2- 172.21.15.28:0/2275102400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e4003fb0 0x7fc9e4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9fad9d700 1 -- 172.21.15.28:0/2275102400 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9e40035b0 con 0x7fc9e4004d90 2023-11-28T11:12:06.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9fa59c700 1 --2- 172.21.15.28:0/2275102400 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9fc0ccbc0 0x7fc9e4003a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:06.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9fad9d700 1 --2- 172.21.15.28:0/2275102400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9e4004d90 0x7fc9e4005200 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc071890 tx=0x7fc9e4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d81723444de0c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:06.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 -- 172.21.15.28:0/2275102400 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f400ebd0 con 0x7fc9e4004d90 2023-11-28T11:12:06.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 -- 172.21.15.28:0/2275102400 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f400f040 con 0x7fc9e4004d90 2023-11-28T11:12:06.972 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.967+0000 7fc9f959a700 1 -- 172.21.15.28:0/2275102400 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4014480 con 0x7fc9e4004d90 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.968+0000 7fca0087b700 1 -- 172.21.15.28:0/2275102400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9e4004d90 msgr2=0x7fc9e4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.968+0000 7fca0087b700 1 --2- 172.21.15.28:0/2275102400 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9e4004d90 0x7fc9e4005200 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fc9fc071890 tx=0x7fc9e4005880 comp rx=0 tx=0).stop 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.968+0000 7fc9fad9d700 1 -- 172.21.15.28:0/2275102400 reap_dead start 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.968+0000 7fca0087b700 1 -- 172.21.15.28:0/2275102400 shutdown_connections 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.968+0000 7fca0087b700 1 -- 172.21.15.28:0/2275102400 >> 172.21.15.28:0/2275102400 conn(0x7fc9fc0b1650 msgr2=0x7fc9fc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.968+0000 7fca0087b700 1 -- 172.21.15.28:0/2275102400 shutdown_connections 2023-11-28T11:12:06.973 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.968+0000 7fca0087b700 1 -- 172.21.15.28:0/2275102400 wait complete. 2023-11-28T11:12:06.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.969+0000 7fca0087b700 1 Processor -- start 2023-11-28T11:12:06.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.969+0000 7fca0087b700 1 -- start start 2023-11-28T11:12:06.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.970+0000 7fca0087b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e4004d90 0x7fc9fc0b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:06.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.970+0000 7fca0087b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9fc0b3480 0x7fc9fc0c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:06.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.970+0000 7fca0087b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9fc14ed00 0x7fc9fc15c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:06.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.970+0000 7fca0087b700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc0b50e0 con 0x7fc9fc14ed00 2023-11-28T11:12:06.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.970+0000 7fca0087b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc0b4200 con 0x7fc9e4004d90 2023-11-28T11:12:06.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.970+0000 7fca0087b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9fc0b3c40 con 0x7fc9fc0b3480 2023-11-28T11:12:06.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.970+0000 7fc9fad9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9fc14ed00 0x7fc9fc15c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:06.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.970+0000 7fc9fad9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9fc14ed00 0x7fc9fc15c980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51566/0 (socket says 172.21.15.28:51566) 2023-11-28T11:12:06.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.970+0000 7fc9fad9d700 1 -- 172.21.15.28:0/1384297020 learned_addr learned my addr 172.21.15.28:0/1384297020 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:06.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.971+0000 7fc9fad9d700 1 -- 172.21.15.28:0/1384297020 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9fc0b3480 msgr2=0x7fc9fc0c95e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:06.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.971+0000 7fc9fad9d700 1 --2- 172.21.15.28:0/1384297020 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9fc0b3480 0x7fc9fc0c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.971+0000 7fc9fad9d700 1 -- 172.21.15.28:0/1384297020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e4004d90 msgr2=0x7fc9fc0b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:06.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.971+0000 7fc9fad9d700 1 --2- 172.21.15.28:0/1384297020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e4004d90 0x7fc9fc0b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:06.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.971+0000 7fc9fad9d700 1 -- 172.21.15.28:0/1384297020 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9fc0caf20 con 0x7fc9fc14ed00 2023-11-28T11:12:06.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.971+0000 7fc9fad9d700 1 --2- 172.21.15.28:0/1384297020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9fc14ed00 0x7fc9fc15c980 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4016040 tx=0x7fc9f4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:06.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.971+0000 7fc9eaffd700 1 -- 172.21.15.28:0/1384297020 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4003c20 con 0x7fc9fc14ed00 2023-11-28T11:12:06.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.971+0000 7fc9eaffd700 1 -- 172.21.15.28:0/1384297020 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9f4021070 con 0x7fc9fc14ed00 2023-11-28T11:12:06.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.971+0000 7fca0087b700 1 -- 172.21.15.28:0/1384297020 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9fc0cac40 con 0x7fc9fc14ed00 2023-11-28T11:12:06.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.971+0000 7fc9eaffd700 1 -- 172.21.15.28:0/1384297020 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9f4014690 con 0x7fc9fc14ed00 2023-11-28T11:12:06.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.971+0000 7fca0087b700 1 -- 172.21.15.28:0/1384297020 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9fc0cb200 con 0x7fc9fc14ed00 2023-11-28T11:12:06.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.972+0000 7fca0087b700 1 -- 172.21.15.28:0/1384297020 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9fc0675a0 con 0x7fc9fc14ed00 2023-11-28T11:12:06.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.972+0000 7fc9eaffd700 1 -- 172.21.15.28:0/1384297020 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc9f40147f0 con 0x7fc9fc14ed00 2023-11-28T11:12:06.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.973+0000 7fc9eaffd700 1 --2- 172.21.15.28:0/1384297020 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9dc0682f0 0x7fc9dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:06.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.973+0000 7fc9eaffd700 1 -- 172.21.15.28:0/1384297020 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fc9f4086fe0 con 0x7fc9fc14ed00 2023-11-28T11:12:06.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.973+0000 7fc9fa59c700 1 --2- 172.21.15.28:0/1384297020 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9dc0682f0 0x7fc9dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:06.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.973+0000 7fc9fa59c700 1 --2- 172.21.15.28:0/1384297020 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9dc0682f0 0x7fc9dc06a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0001c90 tx=0x7fc9f0001190 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:06.979 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:06.977+0000 7fc9eaffd700 1 -- 172.21.15.28:0/1384297020 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc9f4053680 con 0x7fc9fc14ed00 2023-11-28T11:12:07.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.216+0000 7fca0087b700 1 -- 172.21.15.28:0/1384297020 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc9fc06b6f0 con 0x7fc9fc14ed00 2023-11-28T11:12:07.217 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.217+0000 7fc9eaffd700 1 -- 172.21.15.28:0/1384297020 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4650 (secure 0 0 0) 0x7fc9f4056ca0 con 0x7fc9fc14ed00 2023-11-28T11:12:07.218 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:07.218 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1564028052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1564028052},{"type":"v1","addr":"172.21.15.59:6835","nonce":1564028052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6558,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2509041002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2509041002},{"type":"v1","addr":"172.21.15.28:6836","nonce":2509041002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6561,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3994419079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3994419079},{"type":"v1","addr":"172.21.15.28:6837","nonce":3994419079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:12:03.800884+0000","modified":"2023-11-28T11:12:06.812004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6834/2406852565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2406852565},{"type":"v1","addr":"172.21.15.59:6834","nonce":2406852565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:12:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.219+0000 7fca0087b700 1 -- 172.21.15.28:0/1384297020 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9dc0682f0 msgr2=0x7fc9dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.219+0000 7fca0087b700 1 --2- 172.21.15.28:0/1384297020 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9dc0682f0 0x7fc9dc06a7a0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0001c90 tx=0x7fc9f0001190 comp rx=0 tx=0).stop 2023-11-28T11:12:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.219+0000 7fca0087b700 1 -- 172.21.15.28:0/1384297020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9fc14ed00 msgr2=0x7fc9fc15c980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.219+0000 7fca0087b700 1 --2- 172.21.15.28:0/1384297020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9fc14ed00 0x7fc9fc15c980 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4016040 tx=0x7fc9f4009750 comp rx=0 tx=0).stop 2023-11-28T11:12:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.219+0000 7fca0087b700 1 -- 172.21.15.28:0/1384297020 shutdown_connections 2023-11-28T11:12:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.219+0000 7fca0087b700 1 --2- 172.21.15.28:0/1384297020 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc9dc0682f0 0x7fc9dc06a7a0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.219+0000 7fca0087b700 1 --2- 172.21.15.28:0/1384297020 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc9fc14ed00 0x7fc9fc15c980 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.219+0000 7fca0087b700 1 --2- 172.21.15.28:0/1384297020 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e4004d90 0x7fc9fc0b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.219+0000 7fca0087b700 1 --2- 172.21.15.28:0/1384297020 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc9fc0b3480 0x7fc9fc0c95e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.220 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.219+0000 7fca0087b700 1 -- 172.21.15.28:0/1384297020 >> 172.21.15.28:0/1384297020 conn(0x7fc9fc0b1650 msgr2=0x7fc9fc157160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:07.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.219+0000 7fca0087b700 1 -- 172.21.15.28:0/1384297020 shutdown_connections 2023-11-28T11:12:07.221 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.219+0000 7fca0087b700 1 -- 172.21.15.28:0/1384297020 wait complete. 2023-11-28T11:12:07.221 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 33 2023-11-28T11:12:07.232 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:12:07.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.356+0000 7f15d7731700 1 Processor -- start 2023-11-28T11:12:07.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.356+0000 7f15d7731700 1 -- start start 2023-11-28T11:12:07.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.357+0000 7f15d7731700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f15d015a6c0 0x7f15d00cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.357+0000 7f15d7731700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d00b2660 0x7f15d00b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.357+0000 7f15d7731700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15d00cfb90 con 0x7f15d00d0020 2023-11-28T11:12:07.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.357+0000 7f15d7731700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15d00b3020 con 0x7f15d015a6c0 2023-11-28T11:12:07.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.357+0000 7f15d7731700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00cb930 con 0x7f15d00b2660 2023-11-28T11:12:07.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.357+0000 7f15d54cd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f15d015a6c0 0x7f15d00cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.357+0000 7f15d54cd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f15d015a6c0 0x7f15d00cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51570/0 (socket says 172.21.15.28:51570) 2023-11-28T11:12:07.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.357+0000 7f15d54cd700 1 -- 172.21.15.28:0/3417615846 learned_addr learned my addr 172.21.15.28:0/3417615846 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:07.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.357+0000 7f15d5cce700 1 --2- 172.21.15.28:0/3417615846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d00b2660 0x7f15d00b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.357+0000 7f15d54cd700 1 -- 172.21.15.28:0/3417615846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d00b2660 msgr2=0x7f15d00b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.357+0000 7f15d54cd700 1 --2- 172.21.15.28:0/3417615846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d00b2660 0x7f15d00b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.357+0000 7f15d54cd700 1 -- 172.21.15.28:0/3417615846 >> v1:172.21.15.59:6789/0 conn(0x7f15d00d0020 legacy=0x7f15d00ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.357+0000 7f15d54cd700 1 -- 172.21.15.28:0/3417615846 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15d00cf260 con 0x7f15d015a6c0 2023-11-28T11:12:07.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.357+0000 7f15d54cd700 1 --2- 172.21.15.28:0/3417615846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15d015a6c0 0x7f15d00cb230 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0098f0 tx=0x7f15cc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=678b16c09adac799 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.357+0000 7f15c7fff700 1 -- 172.21.15.28:0/3417615846 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00b8a0 con 0x7f15d015a6c0 2023-11-28T11:12:07.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.358+0000 7f15c7fff700 1 -- 172.21.15.28:0/3417615846 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15cc00ba00 con 0x7f15d015a6c0 2023-11-28T11:12:07.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.358+0000 7f15c7fff700 1 -- 172.21.15.28:0/3417615846 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15cc00bd90 con 0x7f15d015a6c0 2023-11-28T11:12:07.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.358+0000 7f15d7731700 1 -- 172.21.15.28:0/3417615846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15d015a6c0 msgr2=0x7f15d00cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.358+0000 7f15d7731700 1 --2- 172.21.15.28:0/3417615846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15d015a6c0 0x7f15d00cb230 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f15cc0098f0 tx=0x7f15cc009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:07.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.358+0000 7f15d7731700 1 -- 172.21.15.28:0/3417615846 shutdown_connections 2023-11-28T11:12:07.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.358+0000 7f15d7731700 1 --2- 172.21.15.28:0/3417615846 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15d015a6c0 0x7f15d00cb230 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.358+0000 7f15d7731700 1 --2- 172.21.15.28:0/3417615846 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d00b2660 0x7f15d00b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.358+0000 7f15d7731700 1 -- 172.21.15.28:0/3417615846 >> 172.21.15.28:0/3417615846 conn(0x7f15d00cd660 msgr2=0x7f15d00cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:07.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.358+0000 7f15d7731700 1 -- 172.21.15.28:0/3417615846 shutdown_connections 2023-11-28T11:12:07.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.358+0000 7f15d7731700 1 -- 172.21.15.28:0/3417615846 wait complete. 2023-11-28T11:12:07.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.359+0000 7f15d7731700 1 Processor -- start 2023-11-28T11:12:07.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.359+0000 7f15d7731700 1 -- start start 2023-11-28T11:12:07.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.360+0000 7f15d7731700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d00b2660 0x7f15d00c85a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.360+0000 7f15d7731700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d00d0020 0x7f15d00c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.360+0000 7f15d7731700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15d015a6c0 0x7f15d0153f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.360+0000 7f15d7731700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d0154490 con 0x7f15d015a6c0 2023-11-28T11:12:07.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.360+0000 7f15d7731700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00c8ca0 con 0x7f15d00d0020 2023-11-28T11:12:07.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.360+0000 7f15d7731700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15d00c59f0 con 0x7f15d00b2660 2023-11-28T11:12:07.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.360+0000 7f15d5cce700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15d015a6c0 0x7f15d0153f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.360+0000 7f15d5cce700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15d015a6c0 0x7f15d0153f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51580/0 (socket says 172.21.15.28:51580) 2023-11-28T11:12:07.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.360+0000 7f15d5cce700 1 -- 172.21.15.28:0/882268928 learned_addr learned my addr 172.21.15.28:0/882268928 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:07.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.360+0000 7f15d4ccc700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d00d0020 0x7f15d00c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.360+0000 7f15d54cd700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d00b2660 0x7f15d00c85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.361+0000 7f15d5cce700 1 -- 172.21.15.28:0/882268928 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d00b2660 msgr2=0x7f15d00c85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.361+0000 7f15d5cce700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d00b2660 0x7f15d00c85a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.365 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.361+0000 7f15d5cce700 1 -- 172.21.15.28:0/882268928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d00d0020 msgr2=0x7f15d00c7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.361+0000 7f15d5cce700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d00d0020 0x7f15d00c7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.361+0000 7f15d5cce700 1 -- 172.21.15.28:0/882268928 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15cc009550 con 0x7f15d015a6c0 2023-11-28T11:12:07.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.361+0000 7f15d4ccc700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d00d0020 0x7f15d00c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:07.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.361+0000 7f15d54cd700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d00b2660 0x7f15d00c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:07.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.361+0000 7f15d5cce700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15d015a6c0 0x7f15d0153f50 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f15c8007e40 tx=0x7f15c800e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.361+0000 7f15c5ffb700 1 -- 172.21.15.28:0/882268928 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c800eea0 con 0x7f15d015a6c0 2023-11-28T11:12:07.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.361+0000 7f15c5ffb700 1 -- 172.21.15.28:0/882268928 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15c8014420 con 0x7f15d015a6c0 2023-11-28T11:12:07.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.361+0000 7f15d7731700 1 -- 172.21.15.28:0/882268928 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15d00cf820 con 0x7f15d015a6c0 2023-11-28T11:12:07.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.361+0000 7f15c5ffb700 1 -- 172.21.15.28:0/882268928 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15c80147b0 con 0x7f15d015a6c0 2023-11-28T11:12:07.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.361+0000 7f15d7731700 1 -- 172.21.15.28:0/882268928 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15d00cf540 con 0x7f15d015a6c0 2023-11-28T11:12:07.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.362+0000 7f15d7731700 1 -- 172.21.15.28:0/882268928 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15d00d0470 con 0x7f15d015a6c0 2023-11-28T11:12:07.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.363+0000 7f15c5ffb700 1 -- 172.21.15.28:0/882268928 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f15c8012050 con 0x7f15d015a6c0 2023-11-28T11:12:07.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.363+0000 7f15c5ffb700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15b40682f0 0x7f15b406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.363+0000 7f15c5ffb700 1 -- 172.21.15.28:0/882268928 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f15c8086530 con 0x7f15d015a6c0 2023-11-28T11:12:07.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.363+0000 7f15d54cd700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15b40682f0 0x7f15b406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.363+0000 7f15d54cd700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15b40682f0 0x7f15b406a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f15cc014040 tx=0x7f15cc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.367+0000 7f15c5ffb700 1 -- 172.21.15.28:0/882268928 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f15c8052bc0 con 0x7f15d015a6c0 2023-11-28T11:12:07.561 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.561+0000 7f15d7731700 1 -- 172.21.15.28:0/882268928 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15d00675a0 con 0x7f15d015a6c0 2023-11-28T11:12:07.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.562+0000 7f15c5ffb700 1 -- 172.21.15.28:0/882268928 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+18077 (secure 0 0 0) 0x7f15c80561e0 con 0x7f15d015a6c0 2023-11-28T11:12:07.563 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:07.563 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":66,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:12:04.802683+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:12:01.724396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:12:02.776612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:12:07.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.564+0000 7f15d7731700 1 -- 172.21.15.28:0/882268928 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15b40682f0 msgr2=0x7f15b406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.564+0000 7f15d7731700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15b40682f0 0x7f15b406a7a0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f15cc014040 tx=0x7f15cc008030 comp rx=0 tx=0).stop 2023-11-28T11:12:07.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.564+0000 7f15d7731700 1 -- 172.21.15.28:0/882268928 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15d015a6c0 msgr2=0x7f15d0153f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.564+0000 7f15d7731700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15d015a6c0 0x7f15d0153f50 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7f15c8007e40 tx=0x7f15c800e3c0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.564+0000 7f15d7731700 1 -- 172.21.15.28:0/882268928 shutdown_connections 2023-11-28T11:12:07.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.564+0000 7f15d7731700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f15b40682f0 0x7f15b406a7a0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.564+0000 7f15d7731700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f15d015a6c0 0x7f15d0153f50 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.564+0000 7f15d7731700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f15d00b2660 0x7f15d00c85a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.564+0000 7f15d7731700 1 --2- 172.21.15.28:0/882268928 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15d00d0020 0x7f15d00c7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.564+0000 7f15d7731700 1 -- 172.21.15.28:0/882268928 >> 172.21.15.28:0/882268928 conn(0x7f15d00cd660 msgr2=0x7f15d00d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:07.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.564+0000 7f15d7731700 1 -- 172.21.15.28:0/882268928 shutdown_connections 2023-11-28T11:12:07.566 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.564+0000 7f15d7731700 1 -- 172.21.15.28:0/882268928 wait complete. 2023-11-28T11:12:07.577 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T11:12:07.578 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:12:07.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.702+0000 7f249480b700 1 Processor -- start 2023-11-28T11:12:07.702 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.702+0000 7f249480b700 1 -- start start 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.702+0000 7f249480b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f249006a560 0x7f249006aa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.702+0000 7f249480b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f24900cceb0 0x7f24900cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.702+0000 7f249480b700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24900cdec0 con 0x7f249015d1e0 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.702+0000 7f249480b700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f249006af40 con 0x7f249006a560 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.702+0000 7f249480b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24900c9cd0 con 0x7f24900cceb0 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.703+0000 7f248dd9b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f249006a560 0x7f249006aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.703+0000 7f248e59c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f249015d1e0 0x7f249015d5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39642/0 (socket says 172.21.15.28:39642) 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.703+0000 7f248dd9b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f249006a560 0x7f249006aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51586/0 (socket says 172.21.15.28:51586) 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.703+0000 7f248e59c700 1 -- 172.21.15.28:0/75743909 learned_addr learned my addr 172.21.15.28:0/75743909 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:07.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.703+0000 7f248ed9d700 1 --2- 172.21.15.28:0/75743909 >> v2:172.21.15.59:3300/0 conn(0x7f24900cceb0 0x7f24900cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.703+0000 7f248dd9b700 1 -- 172.21.15.28:0/75743909 >> v2:172.21.15.59:3300/0 conn(0x7f24900cceb0 msgr2=0x7f24900cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:07.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.703+0000 7f248dd9b700 1 --2- 172.21.15.28:0/75743909 >> v2:172.21.15.59:3300/0 conn(0x7f24900cceb0 0x7f24900cd330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.703+0000 7f248dd9b700 1 -- 172.21.15.28:0/75743909 >> v1:172.21.15.28:6789/0 conn(0x7f249015d1e0 legacy=0x7f249015d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.703+0000 7f248dd9b700 1 -- 172.21.15.28:0/75743909 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24900c9fb0 con 0x7f249006a560 2023-11-28T11:12:07.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.703+0000 7f248ed9d700 1 --2- 172.21.15.28:0/75743909 >> v2:172.21.15.59:3300/0 conn(0x7f24900cceb0 0x7f24900cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:07.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.703+0000 7f248dd9b700 1 --2- 172.21.15.28:0/75743909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f249006a560 0x7f249006aa00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f24900c98c0 tx=0x7f2478009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef90d92ddf05192b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.703+0000 7f248d59a700 1 -- 172.21.15.28:0/75743909 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478011830 con 0x7f249006a560 2023-11-28T11:12:07.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.703+0000 7f248d59a700 1 -- 172.21.15.28:0/75743909 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2478011990 con 0x7f249006a560 2023-11-28T11:12:07.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.703+0000 7f248d59a700 1 -- 172.21.15.28:0/75743909 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2478011d20 con 0x7f249006a560 2023-11-28T11:12:07.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.704+0000 7f249480b700 1 -- 172.21.15.28:0/75743909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f249006a560 msgr2=0x7f249006aa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.704+0000 7f249480b700 1 --2- 172.21.15.28:0/75743909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f249006a560 0x7f249006aa00 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f24900c98c0 tx=0x7f2478009b20 comp rx=0 tx=0).stop 2023-11-28T11:12:07.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.704+0000 7f249480b700 1 -- 172.21.15.28:0/75743909 shutdown_connections 2023-11-28T11:12:07.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.704+0000 7f249480b700 1 --2- 172.21.15.28:0/75743909 >> v2:172.21.15.59:3300/0 conn(0x7f24900cceb0 0x7f24900cd330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.704+0000 7f249480b700 1 --2- 172.21.15.28:0/75743909 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f249006a560 0x7f249006aa00 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.704+0000 7f249480b700 1 -- 172.21.15.28:0/75743909 >> 172.21.15.28:0/75743909 conn(0x7f24900b0dd0 msgr2=0x7f24900b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:07.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.704+0000 7f249480b700 1 -- 172.21.15.28:0/75743909 shutdown_connections 2023-11-28T11:12:07.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.704+0000 7f249480b700 1 -- 172.21.15.28:0/75743909 wait complete. 2023-11-28T11:12:07.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.705+0000 7f249480b700 1 Processor -- start 2023-11-28T11:12:07.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.705+0000 7f249480b700 1 -- start start 2023-11-28T11:12:07.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.706+0000 7f249480b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f249006a560 0x7f2490152010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.706+0000 7f249480b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24900cceb0 0x7f249015b200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.706+0000 7f249480b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f249015d1e0 0x7f249015b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.706+0000 7f249480b700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24900b30e0 con 0x7f249006a560 2023-11-28T11:12:07.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.706+0000 7f249480b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24900c5b00 con 0x7f24900cceb0 2023-11-28T11:12:07.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.706+0000 7f249480b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24900b2460 con 0x7f249015d1e0 2023-11-28T11:12:07.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.706+0000 7f248e59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f249006a560 0x7f2490152010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.706+0000 7f248e59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f249006a560 0x7f2490152010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51590/0 (socket says 172.21.15.28:51590) 2023-11-28T11:12:07.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.706+0000 7f248e59c700 1 -- 172.21.15.28:0/543411372 learned_addr learned my addr 172.21.15.28:0/543411372 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:07.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.706+0000 7f248e59c700 1 -- 172.21.15.28:0/543411372 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f249015d1e0 msgr2=0x7f249015b740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.706+0000 7f248e59c700 1 --2- 172.21.15.28:0/543411372 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f249015d1e0 0x7f249015b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.706+0000 7f248e59c700 1 -- 172.21.15.28:0/543411372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24900cceb0 msgr2=0x7f249015b200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.706+0000 7f248e59c700 1 --2- 172.21.15.28:0/543411372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24900cceb0 0x7f249015b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.706+0000 7f248e59c700 1 -- 172.21.15.28:0/543411372 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24780094a0 con 0x7f249006a560 2023-11-28T11:12:07.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.706+0000 7f248e59c700 1 --2- 172.21.15.28:0/543411372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f249006a560 0x7f2490152010 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f248000c490 tx=0x7f248000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.707+0000 7f2486ffd700 1 -- 172.21.15.28:0/543411372 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480005d70 con 0x7f249006a560 2023-11-28T11:12:07.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.707+0000 7f2486ffd700 1 -- 172.21.15.28:0/543411372 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24800070d0 con 0x7f249006a560 2023-11-28T11:12:07.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.707+0000 7f249480b700 1 -- 172.21.15.28:0/543411372 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24900ca570 con 0x7f249006a560 2023-11-28T11:12:07.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.707+0000 7f2486ffd700 1 -- 172.21.15.28:0/543411372 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480007460 con 0x7f249006a560 2023-11-28T11:12:07.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.707+0000 7f249480b700 1 -- 172.21.15.28:0/543411372 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24900ca290 con 0x7f249006a560 2023-11-28T11:12:07.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.708+0000 7f249480b700 1 -- 172.21.15.28:0/543411372 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24900ca850 con 0x7f249006a560 2023-11-28T11:12:07.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.708+0000 7f2486ffd700 1 -- 172.21.15.28:0/543411372 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2480019030 con 0x7f249006a560 2023-11-28T11:12:07.712 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.708+0000 7f2486ffd700 1 --2- 172.21.15.28:0/543411372 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2470068320 0x7f247006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:07.712 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.709+0000 7f2486ffd700 1 -- 172.21.15.28:0/543411372 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f2480085200 con 0x7f249006a560 2023-11-28T11:12:07.712 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.709+0000 7f248dd9b700 1 --2- 172.21.15.28:0/543411372 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2470068320 0x7f247006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:07.712 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.709+0000 7f248dd9b700 1 --2- 172.21.15.28:0/543411372 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2470068320 0x7f247006a7d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f24900c54f0 tx=0x7f2478010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:07.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.713+0000 7f2486ffd700 1 -- 172.21.15.28:0/543411372 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2480051890 con 0x7f249006a560 2023-11-28T11:12:07.902 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.902+0000 7f249480b700 1 -- 172.21.15.28:0/543411372 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f24900675a0 con 0x7f2470068320 2023-11-28T11:12:07.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.910+0000 7f2486ffd700 1 -- 172.21.15.28:0/543411372 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300776 (secure 0 0 0) 0x7f24900675a0 con 0x7f2470068320 2023-11-28T11:12:07.914 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:07.916 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.913+0000 7f249480b700 1 -- 172.21.15.28:0/543411372 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2470068320 msgr2=0x7f247006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.913+0000 7f249480b700 1 --2- 172.21.15.28:0/543411372 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2470068320 0x7f247006a7d0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f24900c54f0 tx=0x7f2478010000 comp rx=0 tx=0).stop 2023-11-28T11:12:07.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.913+0000 7f249480b700 1 -- 172.21.15.28:0/543411372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f249006a560 msgr2=0x7f2490152010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:07.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.913+0000 7f249480b700 1 --2- 172.21.15.28:0/543411372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f249006a560 0x7f2490152010 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f248000c490 tx=0x7f248000c850 comp rx=0 tx=0).stop 2023-11-28T11:12:07.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.913+0000 7f249480b700 1 -- 172.21.15.28:0/543411372 shutdown_connections 2023-11-28T11:12:07.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.913+0000 7f249480b700 1 --2- 172.21.15.28:0/543411372 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2470068320 0x7f247006a7d0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.913+0000 7f249480b700 1 --2- 172.21.15.28:0/543411372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f249006a560 0x7f2490152010 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.913+0000 7f249480b700 1 --2- 172.21.15.28:0/543411372 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f249015d1e0 0x7f249015b740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.913+0000 7f249480b700 1 --2- 172.21.15.28:0/543411372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24900cceb0 0x7f249015b200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:07.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.913+0000 7f249480b700 1 -- 172.21.15.28:0/543411372 >> 172.21.15.28:0/543411372 conn(0x7f24900b0dd0 msgr2=0x7f249006f480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:07.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.913+0000 7f249480b700 1 -- 172.21.15.28:0/543411372 shutdown_connections 2023-11-28T11:12:07.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:07.913+0000 7f249480b700 1 -- 172.21.15.28:0/543411372 wait complete. 2023-11-28T11:12:07.918 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:12:07.926 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2023-11-28T11:12:07.208509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":78584,"kb_used_data":7096,"kb_used_omap":0,"kb_used_meta":71424,"kb_avail":754896136,"statfs":{"total":773094113280,"available":773013643264,"internally_reserved":0,"allocated":7266304,"data_stored":4003357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":73138176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.832676"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.385358+0000","last_change":"2023-11-28T11:12:02.385358+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.385358+0000","last_undegraded":"2023-11-28T11:12:02.385358+0000","last_fullsized":"2023-11-28T11:12:02.385358+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:18.625956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.381881+0000","last_change":"2023-11-28T11:12:02.381881+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.381881+0000","last_undegraded":"2023-11-28T11:12:02.381881+0000","last_fullsized":"2023-11-28T11:12:02.381881+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:32:22.415232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382689+0000","last_change":"2023-11-28T11:12:02.382689+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382689+0000","last_undegraded":"2023-11-28T11:12:02.382689+0000","last_fullsized":"2023-11-28T11:12:02.382689+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:31:31.263698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378143+0000","last_change":"2023-11-28T11:12:02.378143+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378143+0000","last_undegraded":"2023-11-28T11:12:02.378143+0000","last_fullsized":"2023-11-28T11:12:02.378143+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:36.828296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.381713+0000","last_change":"2023-11-28T11:12:02.381713+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.381713+0000","last_undegraded":"2023-11-28T11:12:02.381713+0000","last_fullsized":"2023-11-28T11:12:02.381713+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:07:35.636526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.380101+0000","last_change":"2023-11-28T11:12:02.380101+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.380101+0000","last_undegraded":"2023-11-28T11:12:02.380101+0000","last_fullsized":"2023-11-28T11:12:02.380101+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:48:16.699038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.379058+0000","last_change":"2023-11-28T11:12:02.379058+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.379058+0000","last_undegraded":"2023-11-28T11:12:02.379058+0000","last_fullsized":"2023-11-28T11:12:02.379058+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:51:23.997889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382487+0000","last_change":"2023-11-28T11:12:02.382487+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382487+0000","last_undegraded":"2023-11-28T11:12:02.382487+0000","last_fullsized":"2023-11-28T11:12:02.382487+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:03:13.999980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.383182+0000","last_change":"2023-11-28T11:12:02.383182+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.383182+0000","last_undegraded":"2023-11-28T11:12:02.383182+0000","last_fullsized":"2023-11-28T11:12:02.383182+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:17.899964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378618+0000","last_change":"2023-11-28T11:12:02.378618+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378618+0000","last_undegraded":"2023-11-28T11:12:02.378618+0000","last_fullsized":"2023-11-28T11:12:02.378618+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:38.258754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.381788+0000","last_change":"2023-11-28T11:12:02.381788+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.381788+0000","last_undegraded":"2023-11-28T11:12:02.381788+0000","last_fullsized":"2023-11-28T11:12:02.381788+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:32.442297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378899+0000","last_change":"2023-11-28T11:12:02.378899+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378899+0000","last_undegraded":"2023-11-28T11:12:02.378899+0000","last_fullsized":"2023-11-28T11:12:02.378899+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:04.830939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.379087+0000","last_change":"2023-11-28T11:12:02.379087+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.379087+0000","last_undegraded":"2023-11-28T11:12:02.379087+0000","last_fullsized":"2023-11-28T11:12:02.379087+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:45.516962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.379477+0000","last_change":"2023-11-28T11:12:02.379477+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.379477+0000","last_undegraded":"2023-11-28T11:12:02.379477+0000","last_fullsized":"2023-11-28T11:12:02.379477+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:09:26.168016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.384134+0000","last_change":"2023-11-28T11:12:02.384134+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.384134+0000","last_undegraded":"2023-11-28T11:12:02.384134+0000","last_fullsized":"2023-11-28T11:12:02.384134+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:15:29.533047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382546+0000","last_change":"2023-11-28T11:12:02.382546+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382546+0000","last_undegraded":"2023-11-28T11:12:02.382546+0000","last_fullsized":"2023-11-28T11:12:02.382546+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:29.499469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378504+0000","last_change":"2023-11-28T11:12:02.378504+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378504+0000","last_undegraded":"2023-11-28T11:12:02.378504+0000","last_fullsized":"2023-11-28T11:12:02.378504+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:06.501050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.383656+0000","last_change":"2023-11-28T11:12:02.383656+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.383656+0000","last_undegraded":"2023-11-28T11:12:02.383656+0000","last_fullsized":"2023-11-28T11:12:02.383656+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:26.097220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382595+0000","last_change":"2023-11-28T11:12:02.382595+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382595+0000","last_undegraded":"2023-11-28T11:12:02.382595+0000","last_fullsized":"2023-11-28T11:12:02.382595+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:13:41.696052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382717+0000","last_change":"2023-11-28T11:12:02.382717+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382717+0000","last_undegraded":"2023-11-28T11:12:02.382717+0000","last_fullsized":"2023-11-28T11:12:02.382717+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:53.341867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.383096+0000","last_change":"2023-11-28T11:12:02.383096+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.383096+0000","last_undegraded":"2023-11-28T11:12:02.383096+0000","last_fullsized":"2023-11-28T11:12:02.383096+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:33:57.028903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378632+0000","last_change":"2023-11-28T11:12:02.378632+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378632+0000","last_undegraded":"2023-11-28T11:12:02.378632+0000","last_fullsized":"2023-11-28T11:12:02.378632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:06.501050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382855+0000","last_change":"2023-11-28T11:12:02.382855+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382855+0000","last_undegraded":"2023-11-28T11:12:02.382855+0000","last_fullsized":"2023-11-28T11:12:02.382855+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:39.829848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378141+0000","last_change":"2023-11-28T11:12:02.378141+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378141+0000","last_undegraded":"2023-11-28T11:12:02.378141+0000","last_fullsized":"2023-11-28T11:12:02.378141+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:17:37.393448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382698+0000","last_change":"2023-11-28T11:12:02.382698+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382698+0000","last_undegraded":"2023-11-28T11:12:02.382698+0000","last_fullsized":"2023-11-28T11:12:02.382698+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:04:35.390206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:12:00.406781+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:12:00.406781+0000","last_peered":"2023-11-28T11:12:00.406781+0000","last_clean":"2023-11-28T11:12:00.406781+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:12:00.406781+0000","last_undegraded":"2023-11-28T11:12:00.406781+0000","last_fullsized":"2023-11-28T11:12:00.406781+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.379713+0000","last_change":"2023-11-28T11:12:02.379713+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.379713+0000","last_undegraded":"2023-11-28T11:12:02.379713+0000","last_fullsized":"2023-11-28T11:12:02.379713+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:45.516962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:59.517671+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:11:59.517671+0000","last_peered":"2023-11-28T11:11:59.517671+0000","last_clean":"2023-11-28T11:11:59.517671+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:11:59.517671+0000","last_undegraded":"2023-11-28T11:11:59.517671+0000","last_fullsized":"2023-11-28T11:11:59.517671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.381490+0000","last_change":"2023-11-28T11:12:02.381490+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.381490+0000","last_undegraded":"2023-11-28T11:12:02.381490+0000","last_fullsized":"2023-11-28T11:12:02.381490+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:45.853390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:12:00.313489+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:12:00.313489+0000","last_peered":"2023-11-28T11:12:00.313489+0000","last_clean":"2023-11-28T11:12:00.313489+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:12:00.313489+0000","last_undegraded":"2023-11-28T11:12:00.313489+0000","last_fullsized":"2023-11-28T11:12:00.313489+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":199,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:59.938373+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:11:59.938373+0000","last_peered":"2023-11-28T11:11:59.938373+0000","last_clean":"2023-11-28T11:11:59.938373+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:11:59.938373+0000","last_undegraded":"2023-11-28T11:11:59.938373+0000","last_fullsized":"2023-11-28T11:11:59.938373+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.383536+0000","last_change":"2023-11-28T11:12:02.383536+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.383536+0000","last_undegraded":"2023-11-28T11:12:02.383536+0000","last_fullsized":"2023-11-28T11:12:02.383536+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:18.625956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:59.938370+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:11:59.938370+0000","last_peered":"2023-11-28T11:11:59.938370+0000","last_clean":"2023-11-28T11:11:59.938370+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:11:59.938370+0000","last_undegraded":"2023-11-28T11:11:59.938370+0000","last_fullsized":"2023-11-28T11:11:59.938370+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378164+0000","last_change":"2023-11-28T11:12:02.378164+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378164+0000","last_undegraded":"2023-11-28T11:12:02.378164+0000","last_fullsized":"2023-11-28T11:12:02.378164+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:04.830939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:59.938370+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:11:59.938370+0000","last_peered":"2023-11-28T11:11:59.938370+0000","last_clean":"2023-11-28T11:11:59.938370+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:11:59.938370+0000","last_undegraded":"2023-11-28T11:11:59.938370+0000","last_fullsized":"2023-11-28T11:11:59.938370+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378854+0000","last_change":"2023-11-28T11:12:02.378854+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378854+0000","last_undegraded":"2023-11-28T11:12:02.378854+0000","last_fullsized":"2023-11-28T11:12:02.378854+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:51:23.997889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:59.518746+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:11:59.518746+0000","last_peered":"2023-11-28T11:11:59.518746+0000","last_clean":"2023-11-28T11:11:59.518746+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:11:59.518746+0000","last_undegraded":"2023-11-28T11:11:59.518746+0000","last_fullsized":"2023-11-28T11:11:59.518746+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378579+0000","last_change":"2023-11-28T11:12:02.378579+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378579+0000","last_undegraded":"2023-11-28T11:12:02.378579+0000","last_fullsized":"2023-11-28T11:12:02.378579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:11.744846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:12:00.406696+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:12:00.406696+0000","last_peered":"2023-11-28T11:12:00.406696+0000","last_clean":"2023-11-28T11:12:00.406696+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:12:00.406696+0000","last_undegraded":"2023-11-28T11:12:00.406696+0000","last_fullsized":"2023-11-28T11:12:00.406696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.385277+0000","last_change":"2023-11-28T11:12:02.385277+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.385277+0000","last_undegraded":"2023-11-28T11:12:02.385277+0000","last_fullsized":"2023-11-28T11:12:02.385277+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:26.097220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:12:00.406674+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:12:00.406674+0000","last_peered":"2023-11-28T11:12:00.406674+0000","last_clean":"2023-11-28T11:12:00.406674+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:12:00.406674+0000","last_undegraded":"2023-11-28T11:12:00.406674+0000","last_fullsized":"2023-11-28T11:12:00.406674+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10612,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361228,"statfs":{"total":96636764160,"available":96625897472,"internally_reserved":0,"allocated":1355776,"data_stored":944127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.59599999999999997,"15min":0.63900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76700000000000002,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.81899999999999995,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.627}]},{"osd":1,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.64100000000000001,"15min":0.68000000000000005},"min":{"1min":0.378,"5min":0.316,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.40100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.87,"5min":1.224,"15min":1.224},"last":0.51100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65500000000000003,"15min":0.68700000000000006},"min":{"1min":0.443,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79500000000000004,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.72599999999999998,"15min":0.77300000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79300000000000004,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.54300000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.46899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83299999999999996,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.72499999999999998,"15min":0.76600000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.81499999999999995,"5min":3.294,"15min":3.294},"last":0.60399999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.63300000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55600000000000005,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.79200000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.58299999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.621,"15min":0.64000000000000001},"min":{"1min":0.36199999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73599999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63800000000000001,"15min":0.64700000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78000000000000003,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68400000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86699999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.628,"15min":0.63600000000000001},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9456,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362384,"statfs":{"total":96636764160,"available":96627081216,"internally_reserved":0,"allocated":761856,"data_stored":353759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.46100000000000002},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.745,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.48999999999999999,"15min":0.47899999999999998},"min":{"1min":0.33700000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60099999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53100000000000003,"15min":0.51300000000000001},"min":{"1min":0.41099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86399999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.51500000000000001,"15min":0.499},"min":{"1min":0.32600000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51400000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.38500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66000000000000003,"15min":0.68100000000000005},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84299999999999997,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.54000000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69199999999999995,"15min":0.71899999999999997},"min":{"1min":0.45500000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81299999999999994,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68200000000000005,"15min":0.69799999999999995},"min":{"1min":0.41799999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80900000000000005,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.70199999999999996}]},{"osd":4,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61899999999999999,"15min":0.628},"min":{"1min":0.40200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90300000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84999999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.47199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.873},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.46500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.629}]},{"osd":7,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.54500000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93000000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.81799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10548,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361292,"statfs":{"total":96636764160,"available":96625963008,"internally_reserved":0,"allocated":1355776,"data_stored":944127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33500000000000002,"15min":0.32900000000000001},"min":{"1min":0.20599999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.309},{"interface":"front","average":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.26900000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.57799999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.38100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.33100000000000002,"15min":0.32000000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.47299999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59999999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39500000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.40899999999999997,"15min":0.39300000000000002},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41699999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.60999999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.32900000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.39300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.014},"last":0.626}]},{"osd":5,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.45800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.94799999999999995,"5min":0.998,"15min":0.998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.53300000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94099999999999995,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.018,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.57199999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.443,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96699999999999997,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61499999999999999,"15min":0.59799999999999998},"min":{"1min":0.48699999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89800000000000002,"5min":1.071,"15min":1.071},"last":0.63700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9700,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362140,"statfs":{"total":96636764160,"available":96626831360,"internally_reserved":0,"allocated":749568,"data_stored":347269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.33200000000000002,"15min":0.32800000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.46999999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.379,"15min":0.373},"min":{"1min":0.32200000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.54300000000000004,"5min":0.622,"15min":0.622},"last":0.33100000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.52600000000000002,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53600000000000003,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.28899999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.371,"15min":0.371},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51400000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42999999999999999,"15min":0.42799999999999999},"min":{"1min":0.30299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.56699999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.5}]},{"osd":4,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.48699999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.42399999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84299999999999997,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.74099999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65800000000000003,"15min":0.622},"min":{"1min":0.497,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90000000000000002,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.622,"15min":0.623},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.86899999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.81599999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.623,"15min":0.623},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88700000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.85999999999999999,"5min":1.016,"15min":1.016},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65400000000000003,"15min":0.65300000000000002},"min":{"1min":0.48699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93400000000000005,"5min":1.105,"15min":1.105},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65000000000000002,"15min":0.64300000000000002},"min":{"1min":0.53500000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94299999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.754}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9388,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362452,"statfs":{"total":96636764160,"available":96627150848,"internally_reserved":0,"allocated":757760,"data_stored":352798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.373},"min":{"1min":0.23300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.38500000000000001,"15min":0.38900000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61699999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.31}]},{"osd":1,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64300000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.432,"15min":0.44400000000000001},"min":{"1min":0.29299999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60399999999999998,"5min":1.528,"15min":1.528},"last":0.34599999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38900000000000001,"15min":0.39600000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.59399999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.46700000000000003,"15min":0.47899999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64800000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.437}]},{"osd":4,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81599999999999995,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82799999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56000000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.47899999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88300000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.77600000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.62,"15min":0.622},"min":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.746},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65900000000000003,"15min":0.66900000000000004},"min":{"1min":0.51400000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86399999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.629,"15min":0.63},"min":{"1min":0.52800000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81799999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.80100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9648,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362192,"statfs":{"total":96636764160,"available":96626884608,"internally_reserved":0,"allocated":761856,"data_stored":353759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.71399999999999997,"15min":0.88},"min":{"1min":0.23899999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.53000000000000003,"5min":11.965,"15min":11.965},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.74299999999999999,"15min":0.91500000000000004},"min":{"1min":0.255,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.57399999999999995,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.34699999999999998,"15min":0.34000000000000002},"min":{"1min":0.27100000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.92100000000000004,"15min":1.157},"min":{"1min":0.28499999999999998,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.69499999999999995,"5min":16.988,"15min":16.988},"last":0.43099999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.47799999999999998,"15min":0.5},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64700000000000002,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":1.3149999999999999,"15min":1.7010000000000001},"min":{"1min":0.33800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.629,"5min":27.887,"15min":27.887},"last":0.39500000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67300000000000004,"15min":0.70399999999999996},"min":{"1min":0.46800000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.86499999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.64700000000000002,"15min":0.66500000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89500000000000002,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.48599999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66500000000000004,"15min":0.68700000000000006},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85099999999999998,"5min":1.98,"15min":1.98},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":1,"15min":1.169},"min":{"1min":0.42799999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.59399999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":1.008,"15min":1.177},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82299999999999995,"5min":11.888,"15min":11.888},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.69599999999999995,"15min":0.72999999999999998},"min":{"1min":0.439,"5min":0.437,"15min":0.437},"max":{"1min":0.80900000000000005,"5min":1.925,"15min":1.925},"last":0.50900000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71399999999999997,"15min":0.74399999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91700000000000004,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.746,"15min":0.77900000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93899999999999995,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.57999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9712,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362128,"statfs":{"total":96636764160,"available":96626819072,"internally_reserved":0,"allocated":761856,"data_stored":353759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75900000000000001,"5min":0.996,"15min":0.996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.81699999999999995,"15min":0.93600000000000005},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.752,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.69999999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.40600000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74299999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78700000000000003,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.73799999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.44,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.746,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.71199999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54900000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.69199999999999995,"15min":0.746},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.81999999999999995,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.46600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80100000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.77300000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.33700000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73099999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.755}]},{"osd":6,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.42099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":1.145,"15min":1.145},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.40300000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.747,"5min":1.085,"15min":1.085},"last":0.79900000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.60699999999999998,"15min":0.627},"min":{"1min":0.38500000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70699999999999996,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.66600000000000004,"15min":0.67800000000000005},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80300000000000005,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.83499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9520,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362320,"statfs":{"total":96636764160,"available":96627015680,"internally_reserved":0,"allocated":761856,"data_stored":353759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.503,"15min":0.503},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69899999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.55800000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.82099999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68899999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.64400000000000002,"15min":0.66600000000000004},"min":{"1min":0.42199999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73399999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.64000000000000001,"15min":0.66900000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71699999999999997,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.496}]},{"osd":2,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71999999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47699999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.63400000000000001,"15min":0.67800000000000005},"min":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78700000000000003,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":1.042,"15min":1.042},"last":0.67300000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68100000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.46700000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.73999999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71299999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.47799999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72099999999999997,"5min":1.036,"15min":1.036},"last":0.52600000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66400000000000003,"15min":0.68999999999999995},"min":{"1min":0.39500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.97299999999999998,"5min":1.329,"15min":1.329},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.66600000000000004,"15min":0.68899999999999995},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75800000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.59499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:12:07.932 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:12:08.091 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.090+0000 7fecc3f5f700 1 Processor -- start 2023-11-28T11:12:08.091 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.090+0000 7fecc3f5f700 1 -- start start 2023-11-28T11:12:08.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.091+0000 7fecc3f5f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fecbc15c990 0x7fecbc0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.091+0000 7fecc3f5f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fecbc0cc310 0x7fecbc0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.091+0000 7fecc3f5f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecbc0ce340 con 0x7fecbc161090 2023-11-28T11:12:08.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.091+0000 7fecc3f5f700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecbc0cebe0 con 0x7fecbc15c990 2023-11-28T11:12:08.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.091+0000 7fecc3f5f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecbc0cccb0 con 0x7fecbc0cc310 2023-11-28T11:12:08.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.091+0000 7fecc1cfb700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fecbc15c990 0x7fecbc0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.091+0000 7fecc1cfb700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fecbc15c990 0x7fecbc0cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51598/0 (socket says 172.21.15.28:51598) 2023-11-28T11:12:08.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.091+0000 7fecc1cfb700 1 -- 172.21.15.28:0/2338704610 learned_addr learned my addr 172.21.15.28:0/2338704610 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:08.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.091+0000 7fecc1cfb700 1 -- 172.21.15.28:0/2338704610 >> v2:172.21.15.59:3300/0 conn(0x7fecbc0cc310 msgr2=0x7fecbc0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.091+0000 7fecc1cfb700 1 --2- 172.21.15.28:0/2338704610 >> v2:172.21.15.59:3300/0 conn(0x7fecbc0cc310 0x7fecbc0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.091+0000 7fecc1cfb700 1 -- 172.21.15.28:0/2338704610 >> v1:172.21.15.59:6789/0 conn(0x7fecbc161090 legacy=0x7fecbc0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.091+0000 7fecc1cfb700 1 -- 172.21.15.28:0/2338704610 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc0cb3f0 con 0x7fecbc15c990 2023-11-28T11:12:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.092+0000 7fecc1cfb700 1 --2- 172.21.15.28:0/2338704610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fecbc15c990 0x7fecbc0cf070 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fecbc071a50 tx=0x7fecac009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e76b465a4a062387 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.092+0000 7fecc0cf9700 1 -- 172.21.15.28:0/2338704610 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac004230 con 0x7fecbc15c990 2023-11-28T11:12:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.092+0000 7fecc0cf9700 1 -- 172.21.15.28:0/2338704610 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecac004390 con 0x7fecbc15c990 2023-11-28T11:12:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.092+0000 7fecc0cf9700 1 -- 172.21.15.28:0/2338704610 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecac0036a0 con 0x7fecbc15c990 2023-11-28T11:12:08.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.092+0000 7fecc3f5f700 1 -- 172.21.15.28:0/2338704610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fecbc15c990 msgr2=0x7fecbc0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.092+0000 7fecc3f5f700 1 --2- 172.21.15.28:0/2338704610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fecbc15c990 0x7fecbc0cf070 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fecbc071a50 tx=0x7fecac009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.092+0000 7fecc3f5f700 1 -- 172.21.15.28:0/2338704610 shutdown_connections 2023-11-28T11:12:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.092+0000 7fecc3f5f700 1 --2- 172.21.15.28:0/2338704610 >> v2:172.21.15.59:3300/0 conn(0x7fecbc0cc310 0x7fecbc0cc770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.092+0000 7fecc3f5f700 1 --2- 172.21.15.28:0/2338704610 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fecbc15c990 0x7fecbc0cf070 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.092+0000 7fecc3f5f700 1 -- 172.21.15.28:0/2338704610 >> 172.21.15.28:0/2338704610 conn(0x7fecbc0b1650 msgr2=0x7fecbc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.092+0000 7fecc3f5f700 1 -- 172.21.15.28:0/2338704610 shutdown_connections 2023-11-28T11:12:08.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.092+0000 7fecc3f5f700 1 -- 172.21.15.28:0/2338704610 wait complete. 2023-11-28T11:12:08.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.094+0000 7fecc3f5f700 1 Processor -- start 2023-11-28T11:12:08.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.094+0000 7fecc3f5f700 1 -- start start 2023-11-28T11:12:08.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.094+0000 7fecc3f5f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecbc0cc310 0x7fecbc15e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.094+0000 7fecc3f5f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecbc15c990 0x7fecbc1f2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.094+0000 7fecc3f5f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fecbc161090 0x7fecbc0c3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.094+0000 7fecc3f5f700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc0c4540 con 0x7fecbc161090 2023-11-28T11:12:08.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.094+0000 7fecc3f5f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc0c3690 con 0x7fecbc0cc310 2023-11-28T11:12:08.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.094+0000 7fecc3f5f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc0c38c0 con 0x7fecbc15c990 2023-11-28T11:12:08.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.094+0000 7fecc24fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fecbc161090 0x7fecbc0c3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.094+0000 7fecc24fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fecbc161090 0x7fecbc0c3150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51608/0 (socket says 172.21.15.28:51608) 2023-11-28T11:12:08.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.094+0000 7fecc24fc700 1 -- 172.21.15.28:0/3821629734 learned_addr learned my addr 172.21.15.28:0/3821629734 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:08.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.094+0000 7fecc1cfb700 1 --2- 172.21.15.28:0/3821629734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecbc0cc310 0x7fecbc15e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.095+0000 7fecc14fa700 1 --2- 172.21.15.28:0/3821629734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecbc15c990 0x7fecbc1f2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.095+0000 7fecc24fc700 1 -- 172.21.15.28:0/3821629734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecbc15c990 msgr2=0x7fecbc1f2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.095+0000 7fecc24fc700 1 --2- 172.21.15.28:0/3821629734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecbc15c990 0x7fecbc1f2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.095+0000 7fecc24fc700 1 -- 172.21.15.28:0/3821629734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecbc0cc310 msgr2=0x7fecbc15e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.095+0000 7fecc24fc700 1 --2- 172.21.15.28:0/3821629734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecbc0cc310 0x7fecbc15e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.095+0000 7fecc24fc700 1 -- 172.21.15.28:0/3821629734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecac009550 con 0x7fecbc161090 2023-11-28T11:12:08.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.095+0000 7fecc14fa700 1 --2- 172.21.15.28:0/3821629734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecbc15c990 0x7fecbc1f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:08.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.095+0000 7fecc24fc700 1 --2- 172.21.15.28:0/3821629734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fecbc161090 0x7fecbc0c3150 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fecb400b970 tx=0x7fecb400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.095+0000 7fecb27fc700 1 -- 172.21.15.28:0/3821629734 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb400c930 con 0x7fecbc161090 2023-11-28T11:12:08.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.095+0000 7fecb27fc700 1 -- 172.21.15.28:0/3821629734 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb400cab0 con 0x7fecbc161090 2023-11-28T11:12:08.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.095+0000 7fecc3f5f700 1 -- 172.21.15.28:0/3821629734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecbc0ce870 con 0x7fecbc161090 2023-11-28T11:12:08.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.095+0000 7fecb27fc700 1 -- 172.21.15.28:0/3821629734 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb400ce60 con 0x7fecbc161090 2023-11-28T11:12:08.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.095+0000 7fecc3f5f700 1 -- 172.21.15.28:0/3821629734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc0ce590 con 0x7fecbc161090 2023-11-28T11:12:08.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.096+0000 7fecc3f5f700 1 -- 172.21.15.28:0/3821629734 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecbc0c4680 con 0x7fecbc161090 2023-11-28T11:12:08.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.097+0000 7fecb27fc700 1 -- 172.21.15.28:0/3821629734 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fecb401b030 con 0x7fecbc161090 2023-11-28T11:12:08.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.097+0000 7fecb27fc700 1 --2- 172.21.15.28:0/3821629734 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7feca00682f0 0x7feca006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.097+0000 7fecb27fc700 1 -- 172.21.15.28:0/3821629734 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fecb4085930 con 0x7fecbc161090 2023-11-28T11:12:08.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.097+0000 7fecc1cfb700 1 --2- 172.21.15.28:0/3821629734 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7feca00682f0 0x7feca006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.097+0000 7fecc1cfb700 1 --2- 172.21.15.28:0/3821629734 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7feca00682f0 0x7feca006a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fecac009130 tx=0x7fecac008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.101+0000 7fecb27fc700 1 -- 172.21.15.28:0/3821629734 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fecbc0c4680 con 0x7fecbc161090 2023-11-28T11:12:08.291 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.290+0000 7fecc3f5f700 1 -- 172.21.15.28:0/3821629734 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fecbc0c4680 con 0x7feca00682f0 2023-11-28T11:12:08.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.298+0000 7fecb27fc700 1 -- 172.21.15.28:0/3821629734 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300776 (secure 0 0 0) 0x7fecbc0c4680 con 0x7feca00682f0 2023-11-28T11:12:08.302 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:08.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.302+0000 7fecc3f5f700 1 -- 172.21.15.28:0/3821629734 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7feca00682f0 msgr2=0x7feca006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.302+0000 7fecc3f5f700 1 --2- 172.21.15.28:0/3821629734 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7feca00682f0 0x7feca006a7a0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fecac009130 tx=0x7fecac008030 comp rx=0 tx=0).stop 2023-11-28T11:12:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.302+0000 7fecc3f5f700 1 -- 172.21.15.28:0/3821629734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fecbc161090 msgr2=0x7fecbc0c3150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.302+0000 7fecc3f5f700 1 --2- 172.21.15.28:0/3821629734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fecbc161090 0x7fecbc0c3150 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fecb400b970 tx=0x7fecb400bc80 comp rx=0 tx=0).stop 2023-11-28T11:12:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.302+0000 7fecc3f5f700 1 -- 172.21.15.28:0/3821629734 shutdown_connections 2023-11-28T11:12:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.302+0000 7fecc3f5f700 1 --2- 172.21.15.28:0/3821629734 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7feca00682f0 0x7feca006a7a0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.302+0000 7fecc3f5f700 1 --2- 172.21.15.28:0/3821629734 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fecbc161090 0x7fecbc0c3150 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.302+0000 7fecc3f5f700 1 --2- 172.21.15.28:0/3821629734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fecbc0cc310 0x7fecbc15e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.302+0000 7fecc3f5f700 1 --2- 172.21.15.28:0/3821629734 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fecbc15c990 0x7fecbc1f2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.302+0000 7fecc3f5f700 1 -- 172.21.15.28:0/3821629734 >> 172.21.15.28:0/3821629734 conn(0x7fecbc0b1650 msgr2=0x7fecbc0cfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.302+0000 7fecc3f5f700 1 -- 172.21.15.28:0/3821629734 shutdown_connections 2023-11-28T11:12:08.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.302+0000 7fecc3f5f700 1 -- 172.21.15.28:0/3821629734 wait complete. 2023-11-28T11:12:08.309 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:12:08.315 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2023-11-28T11:12:07.208509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":78584,"kb_used_data":7096,"kb_used_omap":0,"kb_used_meta":71424,"kb_avail":754896136,"statfs":{"total":773094113280,"available":773013643264,"internally_reserved":0,"allocated":7266304,"data_stored":4003357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":73138176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.832676"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.385358+0000","last_change":"2023-11-28T11:12:02.385358+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.385358+0000","last_undegraded":"2023-11-28T11:12:02.385358+0000","last_fullsized":"2023-11-28T11:12:02.385358+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:18.625956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.381881+0000","last_change":"2023-11-28T11:12:02.381881+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.381881+0000","last_undegraded":"2023-11-28T11:12:02.381881+0000","last_fullsized":"2023-11-28T11:12:02.381881+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:32:22.415232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382689+0000","last_change":"2023-11-28T11:12:02.382689+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382689+0000","last_undegraded":"2023-11-28T11:12:02.382689+0000","last_fullsized":"2023-11-28T11:12:02.382689+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:31:31.263698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378143+0000","last_change":"2023-11-28T11:12:02.378143+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378143+0000","last_undegraded":"2023-11-28T11:12:02.378143+0000","last_fullsized":"2023-11-28T11:12:02.378143+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:36.828296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.381713+0000","last_change":"2023-11-28T11:12:02.381713+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.381713+0000","last_undegraded":"2023-11-28T11:12:02.381713+0000","last_fullsized":"2023-11-28T11:12:02.381713+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:07:35.636526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.380101+0000","last_change":"2023-11-28T11:12:02.380101+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.380101+0000","last_undegraded":"2023-11-28T11:12:02.380101+0000","last_fullsized":"2023-11-28T11:12:02.380101+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:48:16.699038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.379058+0000","last_change":"2023-11-28T11:12:02.379058+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.379058+0000","last_undegraded":"2023-11-28T11:12:02.379058+0000","last_fullsized":"2023-11-28T11:12:02.379058+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:51:23.997889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382487+0000","last_change":"2023-11-28T11:12:02.382487+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382487+0000","last_undegraded":"2023-11-28T11:12:02.382487+0000","last_fullsized":"2023-11-28T11:12:02.382487+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:03:13.999980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.383182+0000","last_change":"2023-11-28T11:12:02.383182+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.383182+0000","last_undegraded":"2023-11-28T11:12:02.383182+0000","last_fullsized":"2023-11-28T11:12:02.383182+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:17.899964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378618+0000","last_change":"2023-11-28T11:12:02.378618+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378618+0000","last_undegraded":"2023-11-28T11:12:02.378618+0000","last_fullsized":"2023-11-28T11:12:02.378618+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:38.258754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.381788+0000","last_change":"2023-11-28T11:12:02.381788+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.381788+0000","last_undegraded":"2023-11-28T11:12:02.381788+0000","last_fullsized":"2023-11-28T11:12:02.381788+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:32.442297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378899+0000","last_change":"2023-11-28T11:12:02.378899+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378899+0000","last_undegraded":"2023-11-28T11:12:02.378899+0000","last_fullsized":"2023-11-28T11:12:02.378899+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:04.830939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.379087+0000","last_change":"2023-11-28T11:12:02.379087+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.379087+0000","last_undegraded":"2023-11-28T11:12:02.379087+0000","last_fullsized":"2023-11-28T11:12:02.379087+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:45.516962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.379477+0000","last_change":"2023-11-28T11:12:02.379477+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.379477+0000","last_undegraded":"2023-11-28T11:12:02.379477+0000","last_fullsized":"2023-11-28T11:12:02.379477+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:09:26.168016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.384134+0000","last_change":"2023-11-28T11:12:02.384134+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.384134+0000","last_undegraded":"2023-11-28T11:12:02.384134+0000","last_fullsized":"2023-11-28T11:12:02.384134+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:15:29.533047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382546+0000","last_change":"2023-11-28T11:12:02.382546+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382546+0000","last_undegraded":"2023-11-28T11:12:02.382546+0000","last_fullsized":"2023-11-28T11:12:02.382546+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:29.499469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378504+0000","last_change":"2023-11-28T11:12:02.378504+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378504+0000","last_undegraded":"2023-11-28T11:12:02.378504+0000","last_fullsized":"2023-11-28T11:12:02.378504+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:06.501050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.383656+0000","last_change":"2023-11-28T11:12:02.383656+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.383656+0000","last_undegraded":"2023-11-28T11:12:02.383656+0000","last_fullsized":"2023-11-28T11:12:02.383656+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:26.097220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382595+0000","last_change":"2023-11-28T11:12:02.382595+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382595+0000","last_undegraded":"2023-11-28T11:12:02.382595+0000","last_fullsized":"2023-11-28T11:12:02.382595+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:13:41.696052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382717+0000","last_change":"2023-11-28T11:12:02.382717+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382717+0000","last_undegraded":"2023-11-28T11:12:02.382717+0000","last_fullsized":"2023-11-28T11:12:02.382717+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:53.341867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.383096+0000","last_change":"2023-11-28T11:12:02.383096+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.383096+0000","last_undegraded":"2023-11-28T11:12:02.383096+0000","last_fullsized":"2023-11-28T11:12:02.383096+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:33:57.028903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378632+0000","last_change":"2023-11-28T11:12:02.378632+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378632+0000","last_undegraded":"2023-11-28T11:12:02.378632+0000","last_fullsized":"2023-11-28T11:12:02.378632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:06.501050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382855+0000","last_change":"2023-11-28T11:12:02.382855+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382855+0000","last_undegraded":"2023-11-28T11:12:02.382855+0000","last_fullsized":"2023-11-28T11:12:02.382855+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:39.829848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378141+0000","last_change":"2023-11-28T11:12:02.378141+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378141+0000","last_undegraded":"2023-11-28T11:12:02.378141+0000","last_fullsized":"2023-11-28T11:12:02.378141+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:17:37.393448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382698+0000","last_change":"2023-11-28T11:12:02.382698+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382698+0000","last_undegraded":"2023-11-28T11:12:02.382698+0000","last_fullsized":"2023-11-28T11:12:02.382698+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:04:35.390206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:12:00.406781+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:12:00.406781+0000","last_peered":"2023-11-28T11:12:00.406781+0000","last_clean":"2023-11-28T11:12:00.406781+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:12:00.406781+0000","last_undegraded":"2023-11-28T11:12:00.406781+0000","last_fullsized":"2023-11-28T11:12:00.406781+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.379713+0000","last_change":"2023-11-28T11:12:02.379713+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.379713+0000","last_undegraded":"2023-11-28T11:12:02.379713+0000","last_fullsized":"2023-11-28T11:12:02.379713+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:45.516962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:59.517671+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:11:59.517671+0000","last_peered":"2023-11-28T11:11:59.517671+0000","last_clean":"2023-11-28T11:11:59.517671+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:11:59.517671+0000","last_undegraded":"2023-11-28T11:11:59.517671+0000","last_fullsized":"2023-11-28T11:11:59.517671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.381490+0000","last_change":"2023-11-28T11:12:02.381490+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.381490+0000","last_undegraded":"2023-11-28T11:12:02.381490+0000","last_fullsized":"2023-11-28T11:12:02.381490+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:45.853390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:12:00.313489+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:12:00.313489+0000","last_peered":"2023-11-28T11:12:00.313489+0000","last_clean":"2023-11-28T11:12:00.313489+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:12:00.313489+0000","last_undegraded":"2023-11-28T11:12:00.313489+0000","last_fullsized":"2023-11-28T11:12:00.313489+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":199,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:59.938373+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:11:59.938373+0000","last_peered":"2023-11-28T11:11:59.938373+0000","last_clean":"2023-11-28T11:11:59.938373+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:11:59.938373+0000","last_undegraded":"2023-11-28T11:11:59.938373+0000","last_fullsized":"2023-11-28T11:11:59.938373+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.383536+0000","last_change":"2023-11-28T11:12:02.383536+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.383536+0000","last_undegraded":"2023-11-28T11:12:02.383536+0000","last_fullsized":"2023-11-28T11:12:02.383536+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:18.625956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:59.938370+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:11:59.938370+0000","last_peered":"2023-11-28T11:11:59.938370+0000","last_clean":"2023-11-28T11:11:59.938370+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:11:59.938370+0000","last_undegraded":"2023-11-28T11:11:59.938370+0000","last_fullsized":"2023-11-28T11:11:59.938370+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378164+0000","last_change":"2023-11-28T11:12:02.378164+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378164+0000","last_undegraded":"2023-11-28T11:12:02.378164+0000","last_fullsized":"2023-11-28T11:12:02.378164+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:04.830939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:59.938370+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:11:59.938370+0000","last_peered":"2023-11-28T11:11:59.938370+0000","last_clean":"2023-11-28T11:11:59.938370+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:11:59.938370+0000","last_undegraded":"2023-11-28T11:11:59.938370+0000","last_fullsized":"2023-11-28T11:11:59.938370+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378854+0000","last_change":"2023-11-28T11:12:02.378854+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378854+0000","last_undegraded":"2023-11-28T11:12:02.378854+0000","last_fullsized":"2023-11-28T11:12:02.378854+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:51:23.997889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:59.518746+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:11:59.518746+0000","last_peered":"2023-11-28T11:11:59.518746+0000","last_clean":"2023-11-28T11:11:59.518746+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:11:59.518746+0000","last_undegraded":"2023-11-28T11:11:59.518746+0000","last_fullsized":"2023-11-28T11:11:59.518746+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378579+0000","last_change":"2023-11-28T11:12:02.378579+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378579+0000","last_undegraded":"2023-11-28T11:12:02.378579+0000","last_fullsized":"2023-11-28T11:12:02.378579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:11.744846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:12:00.406696+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:12:00.406696+0000","last_peered":"2023-11-28T11:12:00.406696+0000","last_clean":"2023-11-28T11:12:00.406696+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:12:00.406696+0000","last_undegraded":"2023-11-28T11:12:00.406696+0000","last_fullsized":"2023-11-28T11:12:00.406696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.385277+0000","last_change":"2023-11-28T11:12:02.385277+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.385277+0000","last_undegraded":"2023-11-28T11:12:02.385277+0000","last_fullsized":"2023-11-28T11:12:02.385277+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:26.097220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:12:00.406674+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:12:00.406674+0000","last_peered":"2023-11-28T11:12:00.406674+0000","last_clean":"2023-11-28T11:12:00.406674+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:12:00.406674+0000","last_undegraded":"2023-11-28T11:12:00.406674+0000","last_fullsized":"2023-11-28T11:12:00.406674+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10612,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361228,"statfs":{"total":96636764160,"available":96625897472,"internally_reserved":0,"allocated":1355776,"data_stored":944127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.59599999999999997,"15min":0.63900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76700000000000002,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.81899999999999995,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.627}]},{"osd":1,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.64100000000000001,"15min":0.68000000000000005},"min":{"1min":0.378,"5min":0.316,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.40100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.87,"5min":1.224,"15min":1.224},"last":0.51100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65500000000000003,"15min":0.68700000000000006},"min":{"1min":0.443,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79500000000000004,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.72599999999999998,"15min":0.77300000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79300000000000004,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.54300000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.46899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83299999999999996,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.72499999999999998,"15min":0.76600000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.81499999999999995,"5min":3.294,"15min":3.294},"last":0.60399999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.63300000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55600000000000005,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.79200000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.58299999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.621,"15min":0.64000000000000001},"min":{"1min":0.36199999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73599999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63800000000000001,"15min":0.64700000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78000000000000003,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68400000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86699999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.628,"15min":0.63600000000000001},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9456,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362384,"statfs":{"total":96636764160,"available":96627081216,"internally_reserved":0,"allocated":761856,"data_stored":353759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.46100000000000002},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.745,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.48999999999999999,"15min":0.47899999999999998},"min":{"1min":0.33700000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60099999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53100000000000003,"15min":0.51300000000000001},"min":{"1min":0.41099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86399999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.51500000000000001,"15min":0.499},"min":{"1min":0.32600000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51400000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.38500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66000000000000003,"15min":0.68100000000000005},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84299999999999997,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.54000000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69199999999999995,"15min":0.71899999999999997},"min":{"1min":0.45500000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81299999999999994,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68200000000000005,"15min":0.69799999999999995},"min":{"1min":0.41799999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80900000000000005,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.70199999999999996}]},{"osd":4,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61899999999999999,"15min":0.628},"min":{"1min":0.40200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90300000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84999999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.47199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.873},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.46500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.629}]},{"osd":7,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.54500000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93000000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.81799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10548,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361292,"statfs":{"total":96636764160,"available":96625963008,"internally_reserved":0,"allocated":1355776,"data_stored":944127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33500000000000002,"15min":0.32900000000000001},"min":{"1min":0.20599999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.309},{"interface":"front","average":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.26900000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.57799999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.38100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.33100000000000002,"15min":0.32000000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.47299999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59999999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39500000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.40899999999999997,"15min":0.39300000000000002},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41699999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.60999999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.32900000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.39300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.014},"last":0.626}]},{"osd":5,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.45800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.94799999999999995,"5min":0.998,"15min":0.998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.53300000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94099999999999995,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.018,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.57199999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.443,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96699999999999997,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61499999999999999,"15min":0.59799999999999998},"min":{"1min":0.48699999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89800000000000002,"5min":1.071,"15min":1.071},"last":0.63700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9700,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362140,"statfs":{"total":96636764160,"available":96626831360,"internally_reserved":0,"allocated":749568,"data_stored":347269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.33200000000000002,"15min":0.32800000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.46999999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.379,"15min":0.373},"min":{"1min":0.32200000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.54300000000000004,"5min":0.622,"15min":0.622},"last":0.33100000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.52600000000000002,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53600000000000003,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.28899999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.371,"15min":0.371},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51400000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42999999999999999,"15min":0.42799999999999999},"min":{"1min":0.30299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.56699999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.5}]},{"osd":4,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.48699999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.42399999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84299999999999997,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.74099999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65800000000000003,"15min":0.622},"min":{"1min":0.497,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90000000000000002,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.622,"15min":0.623},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.86899999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.81599999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.623,"15min":0.623},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88700000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.85999999999999999,"5min":1.016,"15min":1.016},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65400000000000003,"15min":0.65300000000000002},"min":{"1min":0.48699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93400000000000005,"5min":1.105,"15min":1.105},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65000000000000002,"15min":0.64300000000000002},"min":{"1min":0.53500000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94299999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.754}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9388,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362452,"statfs":{"total":96636764160,"available":96627150848,"internally_reserved":0,"allocated":757760,"data_stored":352798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.373},"min":{"1min":0.23300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.38500000000000001,"15min":0.38900000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61699999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.31}]},{"osd":1,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64300000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.432,"15min":0.44400000000000001},"min":{"1min":0.29299999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60399999999999998,"5min":1.528,"15min":1.528},"last":0.34599999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38900000000000001,"15min":0.39600000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.59399999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.46700000000000003,"15min":0.47899999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64800000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.437}]},{"osd":4,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81599999999999995,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82799999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56000000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.47899999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88300000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.77600000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.62,"15min":0.622},"min":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.746},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65900000000000003,"15min":0.66900000000000004},"min":{"1min":0.51400000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86399999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.629,"15min":0.63},"min":{"1min":0.52800000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81799999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.80100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9648,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362192,"statfs":{"total":96636764160,"available":96626884608,"internally_reserved":0,"allocated":761856,"data_stored":353759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.71399999999999997,"15min":0.88},"min":{"1min":0.23899999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.53000000000000003,"5min":11.965,"15min":11.965},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.74299999999999999,"15min":0.91500000000000004},"min":{"1min":0.255,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.57399999999999995,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.34699999999999998,"15min":0.34000000000000002},"min":{"1min":0.27100000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.92100000000000004,"15min":1.157},"min":{"1min":0.28499999999999998,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.69499999999999995,"5min":16.988,"15min":16.988},"last":0.43099999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.47799999999999998,"15min":0.5},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64700000000000002,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":1.3149999999999999,"15min":1.7010000000000001},"min":{"1min":0.33800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.629,"5min":27.887,"15min":27.887},"last":0.39500000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67300000000000004,"15min":0.70399999999999996},"min":{"1min":0.46800000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.86499999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.64700000000000002,"15min":0.66500000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89500000000000002,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.48599999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66500000000000004,"15min":0.68700000000000006},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85099999999999998,"5min":1.98,"15min":1.98},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":1,"15min":1.169},"min":{"1min":0.42799999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.59399999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":1.008,"15min":1.177},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82299999999999995,"5min":11.888,"15min":11.888},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.69599999999999995,"15min":0.72999999999999998},"min":{"1min":0.439,"5min":0.437,"15min":0.437},"max":{"1min":0.80900000000000005,"5min":1.925,"15min":1.925},"last":0.50900000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71399999999999997,"15min":0.74399999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91700000000000004,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.746,"15min":0.77900000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93899999999999995,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.57999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9712,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362128,"statfs":{"total":96636764160,"available":96626819072,"internally_reserved":0,"allocated":761856,"data_stored":353759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75900000000000001,"5min":0.996,"15min":0.996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.81699999999999995,"15min":0.93600000000000005},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.752,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.69999999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.40600000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74299999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78700000000000003,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.73799999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.44,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.746,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.71199999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54900000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.69199999999999995,"15min":0.746},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.81999999999999995,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.46600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80100000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.77300000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.33700000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73099999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.755}]},{"osd":6,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.42099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":1.145,"15min":1.145},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.40300000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.747,"5min":1.085,"15min":1.085},"last":0.79900000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.60699999999999998,"15min":0.627},"min":{"1min":0.38500000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70699999999999996,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.66600000000000004,"15min":0.67800000000000005},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80300000000000005,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.83499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9520,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362320,"statfs":{"total":96636764160,"available":96627015680,"internally_reserved":0,"allocated":761856,"data_stored":353759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.503,"15min":0.503},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69899999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.55800000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.82099999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68899999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.64400000000000002,"15min":0.66600000000000004},"min":{"1min":0.42199999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73399999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.64000000000000001,"15min":0.66900000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71699999999999997,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.496}]},{"osd":2,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71999999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47699999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.63400000000000001,"15min":0.67800000000000005},"min":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78700000000000003,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":1.042,"15min":1.042},"last":0.67300000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68100000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.46700000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.73999999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71299999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.47799999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72099999999999997,"5min":1.036,"15min":1.036},"last":0.52600000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66400000000000003,"15min":0.68999999999999995},"min":{"1min":0.39500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.97299999999999998,"5min":1.329,"15min":1.329},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.66600000000000004,"15min":0.68899999999999995},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75800000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.59499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:12:08.320 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:12:08.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.481+0000 7fef23233700 1 Processor -- start 2023-11-28T11:12:08.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.481+0000 7fef23233700 1 -- start start 2023-11-28T11:12:08.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.482+0000 7fef23233700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef1c159590 0x7fef1c06e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.482+0000 7fef23233700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fef1c0ce890 0x7fef1c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.482+0000 7fef23233700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef1c15d120 con 0x7fef1c15db70 2023-11-28T11:12:08.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.482+0000 7fef23233700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef1c0cf250 con 0x7fef1c0ce890 2023-11-28T11:12:08.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.482+0000 7fef23233700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c15d6e0 con 0x7fef1c159590 2023-11-28T11:12:08.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.482+0000 7fef217d0700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fef1c0ce890 0x7fef1c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.482+0000 7fef217d0700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fef1c0ce890 0x7fef1c0ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51616/0 (socket says 172.21.15.28:51616) 2023-11-28T11:12:08.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.482+0000 7fef217d0700 1 -- 172.21.15.28:0/2063880716 learned_addr learned my addr 172.21.15.28:0/2063880716 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:08.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.482+0000 7fef20fcf700 1 --2- 172.21.15.28:0/2063880716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef1c159590 0x7fef1c06e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.482+0000 7fef217d0700 1 -- 172.21.15.28:0/2063880716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef1c159590 msgr2=0x7fef1c06e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.482+0000 7fef217d0700 1 --2- 172.21.15.28:0/2063880716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef1c159590 0x7fef1c06e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.482+0000 7fef217d0700 1 -- 172.21.15.28:0/2063880716 >> v1:172.21.15.59:6789/0 conn(0x7fef1c15db70 legacy=0x7fef1c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.482+0000 7fef217d0700 1 -- 172.21.15.28:0/2063880716 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef1c0aef60 con 0x7fef1c0ce890 2023-11-28T11:12:08.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.482+0000 7fef217d0700 1 --2- 172.21.15.28:0/2063880716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef1c0ce890 0x7fef1c0ced10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fef14009e50 tx=0x7fef1400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba6953f7486cb872 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.482+0000 7fef1b7fe700 1 -- 172.21.15.28:0/2063880716 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1400e070 con 0x7fef1c0ce890 2023-11-28T11:12:08.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.483+0000 7fef1b7fe700 1 -- 172.21.15.28:0/2063880716 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef1400b9c0 con 0x7fef1c0ce890 2023-11-28T11:12:08.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.483+0000 7fef1b7fe700 1 -- 172.21.15.28:0/2063880716 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef1400bd70 con 0x7fef1c0ce890 2023-11-28T11:12:08.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.483+0000 7fef23233700 1 -- 172.21.15.28:0/2063880716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef1c0ce890 msgr2=0x7fef1c0ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.483+0000 7fef23233700 1 --2- 172.21.15.28:0/2063880716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef1c0ce890 0x7fef1c0ced10 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7fef14009e50 tx=0x7fef1400b330 comp rx=0 tx=0).stop 2023-11-28T11:12:08.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.483+0000 7fef23233700 1 -- 172.21.15.28:0/2063880716 shutdown_connections 2023-11-28T11:12:08.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.483+0000 7fef23233700 1 --2- 172.21.15.28:0/2063880716 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef1c0ce890 0x7fef1c0ced10 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.483+0000 7fef23233700 1 --2- 172.21.15.28:0/2063880716 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef1c159590 0x7fef1c06e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.483+0000 7fef23233700 1 -- 172.21.15.28:0/2063880716 >> 172.21.15.28:0/2063880716 conn(0x7fef1c0b1650 msgr2=0x7fef1c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.483+0000 7fef23233700 1 -- 172.21.15.28:0/2063880716 shutdown_connections 2023-11-28T11:12:08.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.483+0000 7fef23233700 1 -- 172.21.15.28:0/2063880716 wait complete. 2023-11-28T11:12:08.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.484+0000 7fef23233700 1 Processor -- start 2023-11-28T11:12:08.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.484+0000 7fef23233700 1 -- start start 2023-11-28T11:12:08.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.485+0000 7fef23233700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef1c0ce890 0x7fef1c148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.485+0000 7fef23233700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef1c159590 0x7fef1c0b2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.485+0000 7fef23233700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef1c15db70 0x7fef1c0b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.485+0000 7fef23233700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c1532c0 con 0x7fef1c159590 2023-11-28T11:12:08.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.485+0000 7fef23233700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c0b3730 con 0x7fef1c0ce890 2023-11-28T11:12:08.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.485+0000 7fef23233700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c0b3a10 con 0x7fef1c15db70 2023-11-28T11:12:08.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.485+0000 7fef1bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef1c159590 0x7fef1c0b2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.485+0000 7fef1bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef1c159590 0x7fef1c0b2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51620/0 (socket says 172.21.15.28:51620) 2023-11-28T11:12:08.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.485+0000 7fef1bfff700 1 -- 172.21.15.28:0/1828934381 learned_addr learned my addr 172.21.15.28:0/1828934381 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:08.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.485+0000 7fef20fcf700 1 --2- 172.21.15.28:0/1828934381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef1c0ce890 0x7fef1c148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.486+0000 7fef217d0700 1 --2- 172.21.15.28:0/1828934381 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef1c15db70 0x7fef1c0b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.486+0000 7fef1bfff700 1 -- 172.21.15.28:0/1828934381 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef1c15db70 msgr2=0x7fef1c0b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.486+0000 7fef1bfff700 1 --2- 172.21.15.28:0/1828934381 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef1c15db70 0x7fef1c0b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.486+0000 7fef1bfff700 1 -- 172.21.15.28:0/1828934381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef1c0ce890 msgr2=0x7fef1c148d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.486+0000 7fef1bfff700 1 --2- 172.21.15.28:0/1828934381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef1c0ce890 0x7fef1c148d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.486+0000 7fef1bfff700 1 -- 172.21.15.28:0/1828934381 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef14009b30 con 0x7fef1c159590 2023-11-28T11:12:08.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.486+0000 7fef20fcf700 1 --2- 172.21.15.28:0/1828934381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef1c0ce890 0x7fef1c148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:08.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.486+0000 7fef1bfff700 1 --2- 172.21.15.28:0/1828934381 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef1c159590 0x7fef1c0b2b20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fef0800e940 tx=0x7fef0800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.486+0000 7fef197fa700 1 -- 172.21.15.28:0/1828934381 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef080052d0 con 0x7fef1c159590 2023-11-28T11:12:08.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.486+0000 7fef197fa700 1 -- 172.21.15.28:0/1828934381 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef08005430 con 0x7fef1c159590 2023-11-28T11:12:08.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.486+0000 7fef23233700 1 -- 172.21.15.28:0/1828934381 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1c15d370 con 0x7fef1c159590 2023-11-28T11:12:08.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.486+0000 7fef197fa700 1 -- 172.21.15.28:0/1828934381 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0800cc30 con 0x7fef1c159590 2023-11-28T11:12:08.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.486+0000 7fef23233700 1 -- 172.21.15.28:0/1828934381 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c0af240 con 0x7fef1c159590 2023-11-28T11:12:08.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.487+0000 7fef23233700 1 -- 172.21.15.28:0/1828934381 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1c15dfe0 con 0x7fef1c159590 2023-11-28T11:12:08.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.487+0000 7fef197fa700 1 -- 172.21.15.28:0/1828934381 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fef08021070 con 0x7fef1c159590 2023-11-28T11:12:08.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.488+0000 7fef197fa700 1 --2- 172.21.15.28:0/1828934381 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fef000682f0 0x7fef0006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.488+0000 7fef197fa700 1 -- 172.21.15.28:0/1828934381 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fef08087e00 con 0x7fef1c159590 2023-11-28T11:12:08.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.488+0000 7fef20fcf700 1 --2- 172.21.15.28:0/1828934381 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fef000682f0 0x7fef0006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.488+0000 7fef20fcf700 1 --2- 172.21.15.28:0/1828934381 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fef000682f0 0x7fef0006a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fef100099f0 tx=0x7fef10008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.492+0000 7fef197fa700 1 -- 172.21.15.28:0/1828934381 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fef08054490 con 0x7fef1c159590 2023-11-28T11:12:08.680 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.679+0000 7fef23233700 1 -- 172.21.15.28:0/1828934381 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fef1c0675a0 con 0x7fef000682f0 2023-11-28T11:12:08.688 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.687+0000 7fef197fa700 1 -- 172.21.15.28:0/1828934381 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+300776 (secure 0 0 0) 0x7fef1c0675a0 con 0x7fef000682f0 2023-11-28T11:12:08.691 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:08.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.691+0000 7fef23233700 1 -- 172.21.15.28:0/1828934381 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fef000682f0 msgr2=0x7fef0006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.691+0000 7fef23233700 1 --2- 172.21.15.28:0/1828934381 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fef000682f0 0x7fef0006a7a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fef100099f0 tx=0x7fef10008040 comp rx=0 tx=0).stop 2023-11-28T11:12:08.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.691+0000 7fef23233700 1 -- 172.21.15.28:0/1828934381 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef1c159590 msgr2=0x7fef1c0b2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.691+0000 7fef23233700 1 --2- 172.21.15.28:0/1828934381 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef1c159590 0x7fef1c0b2b20 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7fef0800e940 tx=0x7fef0800ed00 comp rx=0 tx=0).stop 2023-11-28T11:12:08.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.691+0000 7fef23233700 1 -- 172.21.15.28:0/1828934381 shutdown_connections 2023-11-28T11:12:08.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.691+0000 7fef23233700 1 --2- 172.21.15.28:0/1828934381 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fef000682f0 0x7fef0006a7a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.694 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.691+0000 7fef23233700 1 --2- 172.21.15.28:0/1828934381 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef1c15db70 0x7fef1c0b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.695 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.691+0000 7fef23233700 1 --2- 172.21.15.28:0/1828934381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef1c0ce890 0x7fef1c148d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.695 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.691+0000 7fef23233700 1 --2- 172.21.15.28:0/1828934381 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef1c159590 0x7fef1c0b2b20 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.695 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.691+0000 7fef23233700 1 -- 172.21.15.28:0/1828934381 >> 172.21.15.28:0/1828934381 conn(0x7fef1c0b1650 msgr2=0x7fef1c0cf890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.695 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.691+0000 7fef23233700 1 -- 172.21.15.28:0/1828934381 shutdown_connections 2023-11-28T11:12:08.695 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.691+0000 7fef23233700 1 -- 172.21.15.28:0/1828934381 wait complete. 2023-11-28T11:12:08.695 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:12:08.703 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":164,"stamp":"2023-11-28T11:12:07.208509+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":82,"acting":82,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":22,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":78584,"kb_used_data":7096,"kb_used_omap":0,"kb_used_meta":71424,"kb_avail":754896136,"statfs":{"total":773094113280,"available":773013643264,"internally_reserved":0,"allocated":7266304,"data_stored":4003357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":73138176},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.832676"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.385358+0000","last_change":"2023-11-28T11:12:02.385358+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.385358+0000","last_undegraded":"2023-11-28T11:12:02.385358+0000","last_fullsized":"2023-11-28T11:12:02.385358+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:18.625956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.381881+0000","last_change":"2023-11-28T11:12:02.381881+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.381881+0000","last_undegraded":"2023-11-28T11:12:02.381881+0000","last_fullsized":"2023-11-28T11:12:02.381881+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:32:22.415232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382689+0000","last_change":"2023-11-28T11:12:02.382689+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382689+0000","last_undegraded":"2023-11-28T11:12:02.382689+0000","last_fullsized":"2023-11-28T11:12:02.382689+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:31:31.263698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378143+0000","last_change":"2023-11-28T11:12:02.378143+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378143+0000","last_undegraded":"2023-11-28T11:12:02.378143+0000","last_fullsized":"2023-11-28T11:12:02.378143+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:36.828296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.381713+0000","last_change":"2023-11-28T11:12:02.381713+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.381713+0000","last_undegraded":"2023-11-28T11:12:02.381713+0000","last_fullsized":"2023-11-28T11:12:02.381713+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:07:35.636526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.380101+0000","last_change":"2023-11-28T11:12:02.380101+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.380101+0000","last_undegraded":"2023-11-28T11:12:02.380101+0000","last_fullsized":"2023-11-28T11:12:02.380101+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:48:16.699038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.379058+0000","last_change":"2023-11-28T11:12:02.379058+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.379058+0000","last_undegraded":"2023-11-28T11:12:02.379058+0000","last_fullsized":"2023-11-28T11:12:02.379058+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:51:23.997889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382487+0000","last_change":"2023-11-28T11:12:02.382487+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382487+0000","last_undegraded":"2023-11-28T11:12:02.382487+0000","last_fullsized":"2023-11-28T11:12:02.382487+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:03:13.999980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.383182+0000","last_change":"2023-11-28T11:12:02.383182+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.383182+0000","last_undegraded":"2023-11-28T11:12:02.383182+0000","last_fullsized":"2023-11-28T11:12:02.383182+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:17.899964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378618+0000","last_change":"2023-11-28T11:12:02.378618+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378618+0000","last_undegraded":"2023-11-28T11:12:02.378618+0000","last_fullsized":"2023-11-28T11:12:02.378618+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:38.258754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.381788+0000","last_change":"2023-11-28T11:12:02.381788+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.381788+0000","last_undegraded":"2023-11-28T11:12:02.381788+0000","last_fullsized":"2023-11-28T11:12:02.381788+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:32.442297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378899+0000","last_change":"2023-11-28T11:12:02.378899+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378899+0000","last_undegraded":"2023-11-28T11:12:02.378899+0000","last_fullsized":"2023-11-28T11:12:02.378899+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:04.830939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.379087+0000","last_change":"2023-11-28T11:12:02.379087+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.379087+0000","last_undegraded":"2023-11-28T11:12:02.379087+0000","last_fullsized":"2023-11-28T11:12:02.379087+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:45.516962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.379477+0000","last_change":"2023-11-28T11:12:02.379477+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.379477+0000","last_undegraded":"2023-11-28T11:12:02.379477+0000","last_fullsized":"2023-11-28T11:12:02.379477+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:09:26.168016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.384134+0000","last_change":"2023-11-28T11:12:02.384134+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.384134+0000","last_undegraded":"2023-11-28T11:12:02.384134+0000","last_fullsized":"2023-11-28T11:12:02.384134+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:15:29.533047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382546+0000","last_change":"2023-11-28T11:12:02.382546+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382546+0000","last_undegraded":"2023-11-28T11:12:02.382546+0000","last_fullsized":"2023-11-28T11:12:02.382546+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:29.499469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378504+0000","last_change":"2023-11-28T11:12:02.378504+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378504+0000","last_undegraded":"2023-11-28T11:12:02.378504+0000","last_fullsized":"2023-11-28T11:12:02.378504+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:06.501050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.383656+0000","last_change":"2023-11-28T11:12:02.383656+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.383656+0000","last_undegraded":"2023-11-28T11:12:02.383656+0000","last_fullsized":"2023-11-28T11:12:02.383656+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:26.097220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382595+0000","last_change":"2023-11-28T11:12:02.382595+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382595+0000","last_undegraded":"2023-11-28T11:12:02.382595+0000","last_fullsized":"2023-11-28T11:12:02.382595+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:13:41.696052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382717+0000","last_change":"2023-11-28T11:12:02.382717+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382717+0000","last_undegraded":"2023-11-28T11:12:02.382717+0000","last_fullsized":"2023-11-28T11:12:02.382717+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:53.341867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.383096+0000","last_change":"2023-11-28T11:12:02.383096+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.383096+0000","last_undegraded":"2023-11-28T11:12:02.383096+0000","last_fullsized":"2023-11-28T11:12:02.383096+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:33:57.028903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378632+0000","last_change":"2023-11-28T11:12:02.378632+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378632+0000","last_undegraded":"2023-11-28T11:12:02.378632+0000","last_fullsized":"2023-11-28T11:12:02.378632+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:06.501050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382855+0000","last_change":"2023-11-28T11:12:02.382855+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382855+0000","last_undegraded":"2023-11-28T11:12:02.382855+0000","last_fullsized":"2023-11-28T11:12:02.382855+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:39.829848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378141+0000","last_change":"2023-11-28T11:12:02.378141+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378141+0000","last_undegraded":"2023-11-28T11:12:02.378141+0000","last_fullsized":"2023-11-28T11:12:02.378141+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:17:37.393448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.382698+0000","last_change":"2023-11-28T11:12:02.382698+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.382698+0000","last_undegraded":"2023-11-28T11:12:02.382698+0000","last_fullsized":"2023-11-28T11:12:02.382698+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:04:35.390206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:12:00.406781+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:12:00.406781+0000","last_peered":"2023-11-28T11:12:00.406781+0000","last_clean":"2023-11-28T11:12:00.406781+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:12:00.406781+0000","last_undegraded":"2023-11-28T11:12:00.406781+0000","last_fullsized":"2023-11-28T11:12:00.406781+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.379713+0000","last_change":"2023-11-28T11:12:02.379713+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.379713+0000","last_undegraded":"2023-11-28T11:12:02.379713+0000","last_fullsized":"2023-11-28T11:12:02.379713+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:45.516962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:59.517671+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:11:59.517671+0000","last_peered":"2023-11-28T11:11:59.517671+0000","last_clean":"2023-11-28T11:11:59.517671+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:11:59.517671+0000","last_undegraded":"2023-11-28T11:11:59.517671+0000","last_fullsized":"2023-11-28T11:11:59.517671+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.381490+0000","last_change":"2023-11-28T11:12:02.381490+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.381490+0000","last_undegraded":"2023-11-28T11:12:02.381490+0000","last_fullsized":"2023-11-28T11:12:02.381490+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:45.853390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":50,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:12:00.313489+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:12:00.313489+0000","last_peered":"2023-11-28T11:12:00.313489+0000","last_clean":"2023-11-28T11:12:00.313489+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:12:00.313489+0000","last_undegraded":"2023-11-28T11:12:00.313489+0000","last_fullsized":"2023-11-28T11:12:00.313489+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":199,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:59.938373+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:11:59.938373+0000","last_peered":"2023-11-28T11:11:59.938373+0000","last_clean":"2023-11-28T11:11:59.938373+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:11:59.938373+0000","last_undegraded":"2023-11-28T11:11:59.938373+0000","last_fullsized":"2023-11-28T11:11:59.938373+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.383536+0000","last_change":"2023-11-28T11:12:02.383536+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.383536+0000","last_undegraded":"2023-11-28T11:12:02.383536+0000","last_fullsized":"2023-11-28T11:12:02.383536+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:18.625956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:59.938370+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:11:59.938370+0000","last_peered":"2023-11-28T11:11:59.938370+0000","last_clean":"2023-11-28T11:11:59.938370+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:11:59.938370+0000","last_undegraded":"2023-11-28T11:11:59.938370+0000","last_fullsized":"2023-11-28T11:11:59.938370+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378164+0000","last_change":"2023-11-28T11:12:02.378164+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378164+0000","last_undegraded":"2023-11-28T11:12:02.378164+0000","last_fullsized":"2023-11-28T11:12:02.378164+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:04.830939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:59.938370+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:11:59.938370+0000","last_peered":"2023-11-28T11:11:59.938370+0000","last_clean":"2023-11-28T11:11:59.938370+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:11:59.938370+0000","last_undegraded":"2023-11-28T11:11:59.938370+0000","last_fullsized":"2023-11-28T11:11:59.938370+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378854+0000","last_change":"2023-11-28T11:12:02.378854+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378854+0000","last_undegraded":"2023-11-28T11:12:02.378854+0000","last_fullsized":"2023-11-28T11:12:02.378854+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:51:23.997889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":54,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:11:59.518746+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:11:59.518746+0000","last_peered":"2023-11-28T11:11:59.518746+0000","last_clean":"2023-11-28T11:11:59.518746+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:11:59.518746+0000","last_undegraded":"2023-11-28T11:11:59.518746+0000","last_fullsized":"2023-11-28T11:11:59.518746+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.378579+0000","last_change":"2023-11-28T11:12:02.378579+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.378579+0000","last_undegraded":"2023-11-28T11:12:02.378579+0000","last_fullsized":"2023-11-28T11:12:02.378579+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:11.744846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:12:00.406696+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:12:00.406696+0000","last_peered":"2023-11-28T11:12:00.406696+0000","last_clean":"2023-11-28T11:12:00.406696+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:12:00.406696+0000","last_undegraded":"2023-11-28T11:12:00.406696+0000","last_fullsized":"2023-11-28T11:12:00.406696+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T11:12:03.375781+0000","last_change":"2023-11-28T11:12:03.375781+0000","last_active":"2023-11-28T11:12:03.375781+0000","last_peered":"2023-11-28T11:12:03.375781+0000","last_clean":"2023-11-28T11:12:03.375781+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:03.375781+0000","last_undegraded":"2023-11-28T11:12:03.375781+0000","last_fullsized":"2023-11-28T11:12:03.375781+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":1,"reported_epoch":63,"state":"creating+peering","last_fresh":"2023-11-28T11:12:02.385277+0000","last_change":"2023-11-28T11:12:02.385277+0000","last_active":"2023-11-28T11:12:02.370874+0000","last_peered":"2023-11-28T11:12:02.370874+0000","last_clean":"2023-11-28T11:12:02.370874+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T11:12:02.385277+0000","last_undegraded":"2023-11-28T11:12:02.385277+0000","last_fullsized":"2023-11-28T11:12:02.385277+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:26.097220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":49,"reported_epoch":62,"state":"active+clean","last_fresh":"2023-11-28T11:12:00.406674+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:12:00.406674+0000","last_peered":"2023-11-28T11:12:00.406674+0000","last_clean":"2023-11-28T11:12:00.406674+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:12:00.406674+0000","last_undegraded":"2023-11-28T11:12:00.406674+0000","last_fullsized":"2023-11-28T11:12:00.406674+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10612,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361228,"statfs":{"total":96636764160,"available":96625897472,"internally_reserved":0,"allocated":1355776,"data_stored":944127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.59599999999999997,"15min":0.63900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76700000000000002,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.81899999999999995,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.627}]},{"osd":1,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.64100000000000001,"15min":0.68000000000000005},"min":{"1min":0.378,"5min":0.316,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.40100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.87,"5min":1.224,"15min":1.224},"last":0.51100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65500000000000003,"15min":0.68700000000000006},"min":{"1min":0.443,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79500000000000004,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.628,"5min":0.72599999999999998,"15min":0.77300000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79300000000000004,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.54300000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.46899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83299999999999996,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.72499999999999998,"15min":0.76600000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.81499999999999995,"5min":3.294,"15min":3.294},"last":0.60399999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.63300000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55600000000000005,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.79200000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.58299999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.621,"15min":0.64000000000000001},"min":{"1min":0.36199999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73599999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63800000000000001,"15min":0.64700000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78000000000000003,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68400000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86699999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.628,"15min":0.63600000000000001},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.65100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574892,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9456,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8704,"kb_avail":94362384,"statfs":{"total":96636764160,"available":96627081216,"internally_reserved":0,"allocated":761856,"data_stored":353759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8912896},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.46100000000000002},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.745,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.48999999999999999,"15min":0.47899999999999998},"min":{"1min":0.33700000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.60099999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53100000000000003,"15min":0.51300000000000001},"min":{"1min":0.41099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86399999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.51500000000000001,"15min":0.499},"min":{"1min":0.32600000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51400000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.38500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66000000000000003,"15min":0.68100000000000005},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84299999999999997,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.54000000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69199999999999995,"15min":0.71899999999999997},"min":{"1min":0.45500000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81299999999999994,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68200000000000005,"15min":0.69799999999999995},"min":{"1min":0.41799999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80900000000000005,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.70199999999999996}]},{"osd":4,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61899999999999999,"15min":0.628},"min":{"1min":0.40200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90300000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.84999999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.47199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.873},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.46500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.629}]},{"osd":7,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.54500000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93000000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.81799999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10548,"kb_used_data":1324,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361292,"statfs":{"total":96636764160,"available":96625963008,"internally_reserved":0,"allocated":1355776,"data_stored":944127,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33500000000000002,"15min":0.32900000000000001},"min":{"1min":0.20599999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.309},{"interface":"front","average":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.26900000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.57799999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.38100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.33100000000000002,"15min":0.32000000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.47299999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59999999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.39500000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.40899999999999997,"15min":0.39300000000000002},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41699999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.60999999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.32900000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.39300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.014},"last":0.626}]},{"osd":5,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.45800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.94799999999999995,"5min":0.998,"15min":0.998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.53300000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94099999999999995,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.018,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.57199999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.443,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96699999999999997,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61499999999999999,"15min":0.59799999999999998},"min":{"1min":0.48699999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89800000000000002,"5min":1.071,"15min":1.071},"last":0.63700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9700,"kb_used_data":732,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362140,"statfs":{"total":96636764160,"available":96626831360,"internally_reserved":0,"allocated":749568,"data_stored":347269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.33200000000000002,"15min":0.32800000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.46999999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.379,"15min":0.373},"min":{"1min":0.32200000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.54300000000000004,"5min":0.622,"15min":0.622},"last":0.33100000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.52600000000000002,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53600000000000003,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.28899999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.371,"15min":0.371},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51400000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42999999999999999,"15min":0.42799999999999999},"min":{"1min":0.30299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.56699999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.5}]},{"osd":4,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.48699999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.42399999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84299999999999997,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.74099999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65800000000000003,"15min":0.622},"min":{"1min":0.497,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90000000000000002,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.622,"15min":0.623},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.86899999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.81599999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.623,"15min":0.623},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88700000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.85999999999999999,"5min":1.016,"15min":1.016},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65400000000000003,"15min":0.65300000000000002},"min":{"1min":0.48699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93400000000000005,"5min":1.105,"15min":1.105},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65000000000000002,"15min":0.64300000000000002},"min":{"1min":0.53500000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94299999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.754}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9388,"kb_used_data":740,"kb_used_omap":0,"kb_used_meta":8640,"kb_avail":94362452,"statfs":{"total":96636764160,"available":96627150848,"internally_reserved":0,"allocated":757760,"data_stored":352798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8847360},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.373},"min":{"1min":0.23300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.38500000000000001,"15min":0.38900000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61699999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.31}]},{"osd":1,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64300000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.432,"15min":0.44400000000000001},"min":{"1min":0.29299999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60399999999999998,"5min":1.528,"15min":1.528},"last":0.34599999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38900000000000001,"15min":0.39600000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.59399999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.46700000000000003,"15min":0.47899999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64800000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.437}]},{"osd":4,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81599999999999995,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82799999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56000000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.47899999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88300000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.77600000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.62,"15min":0.622},"min":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.746},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65900000000000003,"15min":0.66900000000000004},"min":{"1min":0.51400000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86399999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.72599999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.629,"15min":0.63},"min":{"1min":0.52800000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81799999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.80100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574892,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9648,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362192,"statfs":{"total":96636764160,"available":96626884608,"internally_reserved":0,"allocated":761856,"data_stored":353759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.71399999999999997,"15min":0.88},"min":{"1min":0.23899999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.53000000000000003,"5min":11.965,"15min":11.965},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.74299999999999999,"15min":0.91500000000000004},"min":{"1min":0.255,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.57399999999999995,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.32300000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.34699999999999998,"15min":0.34000000000000002},"min":{"1min":0.27100000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.92100000000000004,"15min":1.157},"min":{"1min":0.28499999999999998,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.69499999999999995,"5min":16.988,"15min":16.988},"last":0.43099999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.47799999999999998,"15min":0.5},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64700000000000002,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.46200000000000002,"5min":1.3149999999999999,"15min":1.7010000000000001},"min":{"1min":0.33800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.629,"5min":27.887,"15min":27.887},"last":0.39500000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67300000000000004,"15min":0.70399999999999996},"min":{"1min":0.46800000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.86499999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.64700000000000002,"15min":0.66500000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89500000000000002,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.48599999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66500000000000004,"15min":0.68700000000000006},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85099999999999998,"5min":1.98,"15min":1.98},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":1,"15min":1.169},"min":{"1min":0.42799999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.59399999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":1.008,"15min":1.177},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82299999999999995,"5min":11.888,"15min":11.888},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.69599999999999995,"15min":0.72999999999999998},"min":{"1min":0.439,"5min":0.437,"15min":0.437},"max":{"1min":0.80900000000000005,"5min":1.925,"15min":1.925},"last":0.50900000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71399999999999997,"15min":0.74399999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91700000000000004,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.746,"15min":0.77900000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93899999999999995,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.57999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9712,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362128,"statfs":{"total":96636764160,"available":96626819072,"internally_reserved":0,"allocated":761856,"data_stored":353759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75900000000000001,"5min":0.996,"15min":0.996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.81699999999999995,"15min":0.93600000000000005},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.752,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.69999999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.40600000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74299999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78700000000000003,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.73799999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.44,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.746,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.71199999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.54900000000000004}]},{"osd":3,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.69199999999999995,"15min":0.746},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.81999999999999995,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.46600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80100000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.77300000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.33700000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73099999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.755}]},{"osd":6,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.42099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":1.145,"15min":1.145},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.40300000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.747,"5min":1.085,"15min":1.085},"last":0.79900000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.60699999999999998,"15min":0.627},"min":{"1min":0.38500000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70699999999999996,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.66600000000000004,"15min":0.67800000000000005},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80300000000000005,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.83499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9520,"kb_used_data":744,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362320,"statfs":{"total":96636764160,"available":96627015680,"internally_reserved":0,"allocated":761856,"data_stored":353759,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.503,"15min":0.503},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69899999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.55800000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.82099999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.68899999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.64400000000000002,"15min":0.66600000000000004},"min":{"1min":0.42199999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73399999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.64000000000000001,"15min":0.66900000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71699999999999997,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.496}]},{"osd":2,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71999999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47699999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.63400000000000001,"15min":0.67800000000000005},"min":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78700000000000003,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":1.042,"15min":1.042},"last":0.67300000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68100000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.46700000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.73999999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55200000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71299999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.47799999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72099999999999997,"5min":1.036,"15min":1.036},"last":0.52600000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66400000000000003,"15min":0.68999999999999995},"min":{"1min":0.39500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.97299999999999998,"5min":1.329,"15min":1.329},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.623,"5min":0.66600000000000004,"15min":0.68899999999999995},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75800000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.59499999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:12:08.708 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3f is not active+clean 2023-11-28T11:12:08.708 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3e is not active+clean 2023-11-28T11:12:08.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3d is not active+clean 2023-11-28T11:12:08.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3c is not active+clean 2023-11-28T11:12:08.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3b is not active+clean 2023-11-28T11:12:08.709 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.709 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3a is not active+clean 2023-11-28T11:12:08.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.39 is not active+clean 2023-11-28T11:12:08.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.38 is not active+clean 2023-11-28T11:12:08.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.37 is not active+clean 2023-11-28T11:12:08.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.710 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.36 is not active+clean 2023-11-28T11:12:08.710 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.35 is not active+clean 2023-11-28T11:12:08.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.34 is not active+clean 2023-11-28T11:12:08.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.713 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.33 is not active+clean 2023-11-28T11:12:08.713 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.32 is not active+clean 2023-11-28T11:12:08.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.31 is not active+clean 2023-11-28T11:12:08.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.30 is not active+clean 2023-11-28T11:12:08.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2f is not active+clean 2023-11-28T11:12:08.714 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.714 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2e is not active+clean 2023-11-28T11:12:08.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2d is not active+clean 2023-11-28T11:12:08.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2c is not active+clean 2023-11-28T11:12:08.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2b is not active+clean 2023-11-28T11:12:08.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.715 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2a is not active+clean 2023-11-28T11:12:08.715 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.29 is not active+clean 2023-11-28T11:12:08.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.28 is not active+clean 2023-11-28T11:12:08.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.27 is not active+clean 2023-11-28T11:12:08.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.26 is not active+clean 2023-11-28T11:12:08.716 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.716 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.25 is not active+clean 2023-11-28T11:12:08.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.24 is not active+clean 2023-11-28T11:12:08.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.23 is not active+clean 2023-11-28T11:12:08.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.22 is not active+clean 2023-11-28T11:12:08.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.21 is not active+clean 2023-11-28T11:12:08.717 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.717 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.20 is not active+clean 2023-11-28T11:12:08.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1e is not active+clean 2023-11-28T11:12:08.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1f is not active+clean 2023-11-28T11:12:08.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.385358+0000', 'last_change': '2023-11-28T11:12:02.385358+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.385358+0000', 'last_undegraded': '2023-11-28T11:12:02.385358+0000', 'last_fullsized': '2023-11-28T11:12:02.385358+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:50:18.625956+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T11:12:08.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1f is not active+clean 2023-11-28T11:12:08.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1e is not active+clean 2023-11-28T11:12:08.718 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.381881+0000', 'last_change': '2023-11-28T11:12:02.381881+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.381881+0000', 'last_undegraded': '2023-11-28T11:12:02.381881+0000', 'last_fullsized': '2023-11-28T11:12:02.381881+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T11:32:22.415232+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:12:08.718 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1c is not active+clean 2023-11-28T11:12:08.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1d is not active+clean 2023-11-28T11:12:08.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.382689+0000', 'last_change': '2023-11-28T11:12:02.382689+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.382689+0000', 'last_undegraded': '2023-11-28T11:12:02.382689+0000', 'last_fullsized': '2023-11-28T11:12:02.382689+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T12:31:31.263698+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T11:12:08.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1d is not active+clean 2023-11-28T11:12:08.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1c is not active+clean 2023-11-28T11:12:08.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.378143+0000', 'last_change': '2023-11-28T11:12:02.378143+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.378143+0000', 'last_undegraded': '2023-11-28T11:12:02.378143+0000', 'last_fullsized': '2023-11-28T11:12:02.378143+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T13:52:36.828296+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:12:08.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1a is not active+clean 2023-11-28T11:12:08.719 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.719 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1b is not active+clean 2023-11-28T11:12:08.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.381713+0000', 'last_change': '2023-11-28T11:12:02.381713+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.381713+0000', 'last_undegraded': '2023-11-28T11:12:02.381713+0000', 'last_fullsized': '2023-11-28T11:12:02.381713+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:07:35.636526+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T11:12:08.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1b is not active+clean 2023-11-28T11:12:08.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1a is not active+clean 2023-11-28T11:12:08.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.380101+0000', 'last_change': '2023-11-28T11:12:02.380101+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.380101+0000', 'last_undegraded': '2023-11-28T11:12:02.380101+0000', 'last_fullsized': '2023-11-28T11:12:02.380101+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:48:16.699038+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:12:08.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.18 is not active+clean 2023-11-28T11:12:08.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.720 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.19 is not active+clean 2023-11-28T11:12:08.720 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.379058+0000', 'last_change': '2023-11-28T11:12:02.379058+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.379058+0000', 'last_undegraded': '2023-11-28T11:12:02.379058+0000', 'last_fullsized': '2023-11-28T11:12:02.379058+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:51:23.997889+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T11:12:08.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.19 is not active+clean 2023-11-28T11:12:08.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.18 is not active+clean 2023-11-28T11:12:08.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.382487+0000', 'last_change': '2023-11-28T11:12:02.382487+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.382487+0000', 'last_undegraded': '2023-11-28T11:12:02.382487+0000', 'last_fullsized': '2023-11-28T11:12:02.382487+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T12:03:13.999980+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T11:12:08.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.16 is not active+clean 2023-11-28T11:12:08.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.17 is not active+clean 2023-11-28T11:12:08.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.383182+0000', 'last_change': '2023-11-28T11:12:02.383182+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.383182+0000', 'last_undegraded': '2023-11-28T11:12:02.383182+0000', 'last_fullsized': '2023-11-28T11:12:02.383182+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T16:11:17.899964+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T11:12:08.721 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.17 is not active+clean 2023-11-28T11:12:08.721 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.16 is not active+clean 2023-11-28T11:12:08.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.378618+0000', 'last_change': '2023-11-28T11:12:02.378618+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.378618+0000', 'last_undegraded': '2023-11-28T11:12:02.378618+0000', 'last_fullsized': '2023-11-28T11:12:02.378618+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T18:50:38.258754+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:12:08.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.14 is not active+clean 2023-11-28T11:12:08.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.15 is not active+clean 2023-11-28T11:12:08.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.381788+0000', 'last_change': '2023-11-28T11:12:02.381788+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.381788+0000', 'last_undegraded': '2023-11-28T11:12:02.381788+0000', 'last_fullsized': '2023-11-28T11:12:02.381788+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:30:32.442297+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T11:12:08.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.15 is not active+clean 2023-11-28T11:12:08.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.722 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.14 is not active+clean 2023-11-28T11:12:08.722 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.378899+0000', 'last_change': '2023-11-28T11:12:02.378899+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.378899+0000', 'last_undegraded': '2023-11-28T11:12:02.378899+0000', 'last_fullsized': '2023-11-28T11:12:02.378899+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:27:04.830939+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T11:12:08.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.12 is not active+clean 2023-11-28T11:12:08.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.13 is not active+clean 2023-11-28T11:12:08.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.379087+0000', 'last_change': '2023-11-28T11:12:02.379087+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.379087+0000', 'last_undegraded': '2023-11-28T11:12:02.379087+0000', 'last_fullsized': '2023-11-28T11:12:02.379087+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:27:45.516962+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T11:12:08.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.13 is not active+clean 2023-11-28T11:12:08.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.12 is not active+clean 2023-11-28T11:12:08.723 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.379477+0000', 'last_change': '2023-11-28T11:12:02.379477+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.379477+0000', 'last_undegraded': '2023-11-28T11:12:02.379477+0000', 'last_fullsized': '2023-11-28T11:12:02.379477+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T13:09:26.168016+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:12:08.723 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.10 is not active+clean 2023-11-28T11:12:08.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.11 is not active+clean 2023-11-28T11:12:08.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.384134+0000', 'last_change': '2023-11-28T11:12:02.384134+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.384134+0000', 'last_undegraded': '2023-11-28T11:12:02.384134+0000', 'last_fullsized': '2023-11-28T11:12:02.384134+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:15:29.533047+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T11:12:08.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.11 is not active+clean 2023-11-28T11:12:08.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.10 is not active+clean 2023-11-28T11:12:08.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.382546+0000', 'last_change': '2023-11-28T11:12:02.382546+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.382546+0000', 'last_undegraded': '2023-11-28T11:12:02.382546+0000', 'last_fullsized': '2023-11-28T11:12:02.382546+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:17:29.499469+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T11:12:08.724 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.6 is not active+clean 2023-11-28T11:12:08.724 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.7 is not active+clean 2023-11-28T11:12:08.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.378504+0000', 'last_change': '2023-11-28T11:12:02.378504+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.378504+0000', 'last_undegraded': '2023-11-28T11:12:02.378504+0000', 'last_fullsized': '2023-11-28T11:12:02.378504+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T13:10:06.501050+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:12:08.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.7 is not active+clean 2023-11-28T11:12:08.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.6 is not active+clean 2023-11-28T11:12:08.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.383656+0000', 'last_change': '2023-11-28T11:12:02.383656+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.383656+0000', 'last_undegraded': '2023-11-28T11:12:02.383656+0000', 'last_fullsized': '2023-11-28T11:12:02.383656+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:19:26.097220+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T11:12:08.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.4 is not active+clean 2023-11-28T11:12:08.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.725 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.5 is not active+clean 2023-11-28T11:12:08.725 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.382595+0000', 'last_change': '2023-11-28T11:12:02.382595+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.382595+0000', 'last_undegraded': '2023-11-28T11:12:02.382595+0000', 'last_fullsized': '2023-11-28T11:12:02.382595+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T12:13:41.696052+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T11:12:08.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.5 is not active+clean 2023-11-28T11:12:08.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.4 is not active+clean 2023-11-28T11:12:08.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.382717+0000', 'last_change': '2023-11-28T11:12:02.382717+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.382717+0000', 'last_undegraded': '2023-11-28T11:12:02.382717+0000', 'last_fullsized': '2023-11-28T11:12:02.382717+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:31:53.341867+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:12:08.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.2 is not active+clean 2023-11-28T11:12:08.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.3 is not active+clean 2023-11-28T11:12:08.726 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.383096+0000', 'last_change': '2023-11-28T11:12:02.383096+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.383096+0000', 'last_undegraded': '2023-11-28T11:12:02.383096+0000', 'last_fullsized': '2023-11-28T11:12:02.383096+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T16:33:57.028903+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T11:12:08.726 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.3 is not active+clean 2023-11-28T11:12:08.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.2 is not active+clean 2023-11-28T11:12:08.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.378632+0000', 'last_change': '2023-11-28T11:12:02.378632+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.378632+0000', 'last_undegraded': '2023-11-28T11:12:02.378632+0000', 'last_fullsized': '2023-11-28T11:12:02.378632+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T13:10:06.501050+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T11:12:08.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.0 is not active+clean 2023-11-28T11:12:08.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.1 is not active+clean 2023-11-28T11:12:08.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.382855+0000', 'last_change': '2023-11-28T11:12:02.382855+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.382855+0000', 'last_undegraded': '2023-11-28T11:12:02.382855+0000', 'last_fullsized': '2023-11-28T11:12:02.382855+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T20:01:39.829848+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T11:12:08.727 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.1 is not active+clean 2023-11-28T11:12:08.727 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.0 is not active+clean 2023-11-28T11:12:08.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.378141+0000', 'last_change': '2023-11-28T11:12:02.378141+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.378141+0000', 'last_undegraded': '2023-11-28T11:12:02.378141+0000', 'last_fullsized': '2023-11-28T11:12:02.378141+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:17:37.393448+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:12:08.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.f is not active+clean 2023-11-28T11:12:08.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.e is not active+clean 2023-11-28T11:12:08.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.382698+0000', 'last_change': '2023-11-28T11:12:02.382698+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.382698+0000', 'last_undegraded': '2023-11-28T11:12:02.382698+0000', 'last_fullsized': '2023-11-28T11:12:02.382698+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T17:04:35.390206+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T11:12:08.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.e is not active+clean 2023-11-28T11:12:08.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.728 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.f is not active+clean 2023-11-28T11:12:08.728 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.379713+0000', 'last_change': '2023-11-28T11:12:02.379713+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.379713+0000', 'last_undegraded': '2023-11-28T11:12:02.379713+0000', 'last_fullsized': '2023-11-28T11:12:02.379713+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:27:45.516962+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:12:08.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.9 is not active+clean 2023-11-28T11:12:08.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.8 is not active+clean 2023-11-28T11:12:08.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.381490+0000', 'last_change': '2023-11-28T11:12:02.381490+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.381490+0000', 'last_undegraded': '2023-11-28T11:12:02.381490+0000', 'last_fullsized': '2023-11-28T11:12:02.381490+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T18:42:45.853390+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T11:12:08.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.8 is not active+clean 2023-11-28T11:12:08.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.9 is not active+clean 2023-11-28T11:12:08.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.383536+0000', 'last_change': '2023-11-28T11:12:02.383536+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.383536+0000', 'last_undegraded': '2023-11-28T11:12:02.383536+0000', 'last_fullsized': '2023-11-28T11:12:02.383536+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T14:50:18.625956+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T11:12:08.729 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.a is not active+clean 2023-11-28T11:12:08.729 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.b is not active+clean 2023-11-28T11:12:08.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.378164+0000', 'last_change': '2023-11-28T11:12:02.378164+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.378164+0000', 'last_undegraded': '2023-11-28T11:12:02.378164+0000', 'last_fullsized': '2023-11-28T11:12:02.378164+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T21:27:04.830939+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:12:08.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.b is not active+clean 2023-11-28T11:12:08.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.a is not active+clean 2023-11-28T11:12:08.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.378854+0000', 'last_change': '2023-11-28T11:12:02.378854+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.378854+0000', 'last_undegraded': '2023-11-28T11:12:02.378854+0000', 'last_fullsized': '2023-11-28T11:12:02.378854+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T15:51:23.997889+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:12:08.730 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.c is not active+clean 2023-11-28T11:12:08.730 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.d is not active+clean 2023-11-28T11:12:08.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.378579+0000', 'last_change': '2023-11-28T11:12:02.378579+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.378579+0000', 'last_undegraded': '2023-11-28T11:12:02.378579+0000', 'last_fullsized': '2023-11-28T11:12:02.378579+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T23:08:11.744846+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T11:12:08.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 11.d is not active+clean 2023-11-28T11:12:08.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '11.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T11:12:03.375781+0000', 'last_change': '2023-11-28T11:12:03.375781+0000', 'last_active': '2023-11-28T11:12:03.375781+0000', 'last_peered': '2023-11-28T11:12:03.375781+0000', 'last_clean': '2023-11-28T11:12:03.375781+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:03.375781+0000', 'last_undegraded': '2023-11-28T11:12:03.375781+0000', 'last_fullsized': '2023-11-28T11:12:03.375781+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:03.375781+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T11:12:08.731 INFO:tasks.cephfs.filesystem.ceph_manager:PG 10.c is not active+clean 2023-11-28T11:12:08.731 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '10.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 63, 'state': 'creating+peering', 'last_fresh': '2023-11-28T11:12:02.385277+0000', 'last_change': '2023-11-28T11:12:02.385277+0000', 'last_active': '2023-11-28T11:12:02.370874+0000', 'last_peered': '2023-11-28T11:12:02.370874+0000', 'last_clean': '2023-11-28T11:12:02.370874+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T11:12:02.385277+0000', 'last_undegraded': '2023-11-28T11:12:02.385277+0000', 'last_fullsized': '2023-11-28T11:12:02.385277+0000', 'mapping_epoch': 63, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 63, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'last_clean_scrub_stamp': '2023-11-28T11:12:02.370874+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-11-29T19:19:26.097220+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T11:12:08.732 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T11:12:08.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.868+0000 7fd2f3df1700 1 Processor -- start 2023-11-28T11:12:08.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.868+0000 7fd2f3df1700 1 -- start start 2023-11-28T11:12:08.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f3df1700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd2ec0b0d10 0x7fd2ec0b20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f3df1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fd2ec15ca20 0x7fd2ec0b10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f3df1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2ec1602f0 0x7fd2ec160790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f3df1700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ec0721d0 con 0x7fd2ec0b0d10 2023-11-28T11:12:08.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f3df1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2ec0b2620 con 0x7fd2ec15ca20 2023-11-28T11:12:08.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f3df1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec0b1620 con 0x7fd2ec1602f0 2023-11-28T11:12:08.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f1b8d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd2ec0b0d10 0x7fd2ec0b20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f1b8d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fd2ec0b0d10 0x7fd2ec0b20e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51626/0 (socket says 172.21.15.28:51626) 2023-11-28T11:12:08.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f1b8d700 1 -- 172.21.15.28:0/2170688394 learned_addr learned my addr 172.21.15.28:0/2170688394 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:08.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f138c700 1 --2- 172.21.15.28:0/2170688394 >> v2:172.21.15.59:3300/0 conn(0x7fd2ec15ca20 0x7fd2ec0b10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.871 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f238e700 1 --2- 172.21.15.28:0/2170688394 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2ec1602f0 0x7fd2ec160790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f1b8d700 1 -- 172.21.15.28:0/2170688394 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2ec1602f0 msgr2=0x7fd2ec160790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f1b8d700 1 --2- 172.21.15.28:0/2170688394 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2ec1602f0 0x7fd2ec160790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f1b8d700 1 -- 172.21.15.28:0/2170688394 >> v2:172.21.15.59:3300/0 conn(0x7fd2ec15ca20 msgr2=0x7fd2ec0b10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:08.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f1b8d700 1 --2- 172.21.15.28:0/2170688394 >> v2:172.21.15.59:3300/0 conn(0x7fd2ec15ca20 0x7fd2ec0b10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f1b8d700 1 -- 172.21.15.28:0/2170688394 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec071550 con 0x7fd2ec0b0d10 2023-11-28T11:12:08.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.869+0000 7fd2f1b8d700 1 --2- 172.21.15.28:0/2170688394 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2ec0b0d10 0x7fd2ec0b20e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0c9890 tx=0x7fd2e8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a864ef13d386d00e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.870+0000 7fd2f0b8b700 1 -- 172.21.15.28:0/2170688394 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8005e30 con 0x7fd2ec0b0d10 2023-11-28T11:12:08.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.870+0000 7fd2f0b8b700 1 -- 172.21.15.28:0/2170688394 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e8004030 con 0x7fd2ec0b0d10 2023-11-28T11:12:08.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.870+0000 7fd2f0b8b700 1 -- 172.21.15.28:0/2170688394 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80043e0 con 0x7fd2ec0b0d10 2023-11-28T11:12:08.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.870+0000 7fd2f3df1700 1 -- 172.21.15.28:0/2170688394 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2ec0b0d10 msgr2=0x7fd2ec0b20e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.870+0000 7fd2f3df1700 1 --2- 172.21.15.28:0/2170688394 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2ec0b0d10 0x7fd2ec0b20e0 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec0c9890 tx=0x7fd2e8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.870+0000 7fd2f3df1700 1 -- 172.21.15.28:0/2170688394 shutdown_connections 2023-11-28T11:12:08.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.870+0000 7fd2f3df1700 1 --2- 172.21.15.28:0/2170688394 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2ec1602f0 0x7fd2ec160790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.870+0000 7fd2f3df1700 1 --2- 172.21.15.28:0/2170688394 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2ec0b0d10 0x7fd2ec0b20e0 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.870+0000 7fd2f3df1700 1 --2- 172.21.15.28:0/2170688394 >> v2:172.21.15.59:3300/0 conn(0x7fd2ec15ca20 0x7fd2ec0b10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.870+0000 7fd2f3df1700 1 -- 172.21.15.28:0/2170688394 >> 172.21.15.28:0/2170688394 conn(0x7fd2ec0cbd10 msgr2=0x7fd2ec0cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:08.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.870+0000 7fd2f3df1700 1 -- 172.21.15.28:0/2170688394 shutdown_connections 2023-11-28T11:12:08.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.870+0000 7fd2f3df1700 1 -- 172.21.15.28:0/2170688394 wait complete. 2023-11-28T11:12:08.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.871+0000 7fd2f3df1700 1 Processor -- start 2023-11-28T11:12:08.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.872+0000 7fd2f3df1700 1 -- start start 2023-11-28T11:12:08.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.872+0000 7fd2f3df1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2ec0b0d10 0x7fd2ec14fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.872+0000 7fd2f3df1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2ec15ca20 0x7fd2ec0cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.872+0000 7fd2f3df1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2ec1602f0 0x7fd2ec06b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.872+0000 7fd2f3df1700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec15f520 con 0x7fd2ec0b0d10 2023-11-28T11:12:08.876 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.872+0000 7fd2f3df1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec150350 con 0x7fd2ec1602f0 2023-11-28T11:12:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.872+0000 7fd2f3df1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2ec15ec80 con 0x7fd2ec15ca20 2023-11-28T11:12:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.872+0000 7fd2f1b8d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2ec0b0d10 0x7fd2ec14fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.872+0000 7fd2f1b8d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2ec0b0d10 0x7fd2ec14fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51632/0 (socket says 172.21.15.28:51632) 2023-11-28T11:12:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.872+0000 7fd2f1b8d700 1 -- 172.21.15.28:0/1626579068 learned_addr learned my addr 172.21.15.28:0/1626579068 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.872+0000 7fd2f138c700 1 --2- 172.21.15.28:0/1626579068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2ec15ca20 0x7fd2ec0cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.873+0000 7fd2f238e700 1 --2- 172.21.15.28:0/1626579068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2ec1602f0 0x7fd2ec06b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.873+0000 7fd2f1b8d700 1 -- 172.21.15.28:0/1626579068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2ec15ca20 msgr2=0x7fd2ec0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.873+0000 7fd2f1b8d700 1 --2- 172.21.15.28:0/1626579068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2ec15ca20 0x7fd2ec0cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.873+0000 7fd2f1b8d700 1 -- 172.21.15.28:0/1626579068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2ec1602f0 msgr2=0x7fd2ec06b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.873+0000 7fd2f1b8d700 1 --2- 172.21.15.28:0/1626579068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2ec1602f0 0x7fd2ec06b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.873+0000 7fd2f1b8d700 1 -- 172.21.15.28:0/1626579068 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2e8009500 con 0x7fd2ec0b0d10 2023-11-28T11:12:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.873+0000 7fd2f138c700 1 --2- 172.21.15.28:0/1626579068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2ec15ca20 0x7fd2ec0cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.873+0000 7fd2f1b8d700 1 --2- 172.21.15.28:0/1626579068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2ec0b0d10 0x7fd2ec14fe10 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8000c00 tx=0x7fd2e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.873+0000 7fd2e27fc700 1 -- 172.21.15.28:0/1626579068 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8003e70 con 0x7fd2ec0b0d10 2023-11-28T11:12:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.873+0000 7fd2e27fc700 1 -- 172.21.15.28:0/1626579068 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2e8011420 con 0x7fd2ec0b0d10 2023-11-28T11:12:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.873+0000 7fd2f3df1700 1 -- 172.21.15.28:0/1626579068 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2ec071b10 con 0x7fd2ec0b0d10 2023-11-28T11:12:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.873+0000 7fd2e27fc700 1 -- 172.21.15.28:0/1626579068 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80117f0 con 0x7fd2ec0b0d10 2023-11-28T11:12:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.873+0000 7fd2f3df1700 1 -- 172.21.15.28:0/1626579068 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2ec071830 con 0x7fd2ec0b0d10 2023-11-28T11:12:08.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.874+0000 7fd2f3df1700 1 -- 172.21.15.28:0/1626579068 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2ec071df0 con 0x7fd2ec0b0d10 2023-11-28T11:12:08.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.874+0000 7fd2e27fc700 1 -- 172.21.15.28:0/1626579068 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fd2e800f050 con 0x7fd2ec0b0d10 2023-11-28T11:12:08.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.875+0000 7fd2e27fc700 1 --2- 172.21.15.28:0/1626579068 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd2d00682f0 0x7fd2d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:08.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.875+0000 7fd2e27fc700 1 -- 172.21.15.28:0/1626579068 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fd2e80870e0 con 0x7fd2ec0b0d10 2023-11-28T11:12:08.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.875+0000 7fd2f138c700 1 --2- 172.21.15.28:0/1626579068 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd2d00682f0 0x7fd2d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:08.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.875+0000 7fd2f138c700 1 --2- 172.21.15.28:0/1626579068 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd2d00682f0 0x7fd2d006a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec15fef0 tx=0x7fd2dc008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:08.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:08.879+0000 7fd2e27fc700 1 -- 172.21.15.28:0/1626579068 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fd2e8053800 con 0x7fd2ec0b0d10 2023-11-28T11:12:09.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.150+0000 7fd2f3df1700 1 -- 172.21.15.28:0/1626579068 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fd2ec066c60 con 0x7fd2ec0b0d10 2023-11-28T11:12:09.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.150+0000 7fd2e27fc700 1 -- 172.21.15.28:0/1626579068 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1269 (secure 0 0 0) 0x7fd2e8056e20 con 0x7fd2ec0b0d10 2023-11-28T11:12:09.151 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:09.151 INFO:teuthology.orchestra.run.smithi028.stdout:{"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":221,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":66,"num_osds":8,"num_up_osds":8,"osd_up_since":1701169717,"num_in_osds":8,"osd_in_since":1701169709,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":9}],"num_pgs":105,"num_pools":4,"num_objects":2,"data_bytes":590368,"bytes_used":80470016,"bytes_avail":773013643264,"bytes_total":773094113280,"unknown_pgs_ratio":0.60952383279800415,"inactive_pgs_ratio":0.30476191639900208},"fsmap":{"epoch":33,"id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"d","status":"up:active","gid":6573}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":9,"modified":"2023-11-28T11:11:57.206406+0000","services":{"mds":{"daemons":{"summary":"","d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-11-28T11:12:09.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.152+0000 7fd2f3df1700 1 -- 172.21.15.28:0/1626579068 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd2d00682f0 msgr2=0x7fd2d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:09.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.152+0000 7fd2f3df1700 1 --2- 172.21.15.28:0/1626579068 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd2d00682f0 0x7fd2d006a7a0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec15fef0 tx=0x7fd2dc008040 comp rx=0 tx=0).stop 2023-11-28T11:12:09.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.153+0000 7fd2f3df1700 1 -- 172.21.15.28:0/1626579068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2ec0b0d10 msgr2=0x7fd2ec14fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:09.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.153+0000 7fd2f3df1700 1 --2- 172.21.15.28:0/1626579068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2ec0b0d10 0x7fd2ec14fe10 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8000c00 tx=0x7fd2e8004750 comp rx=0 tx=0).stop 2023-11-28T11:12:09.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.153+0000 7fd2f3df1700 1 -- 172.21.15.28:0/1626579068 shutdown_connections 2023-11-28T11:12:09.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.153+0000 7fd2f3df1700 1 --2- 172.21.15.28:0/1626579068 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fd2d00682f0 0x7fd2d006a7a0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:09.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.153+0000 7fd2f3df1700 1 --2- 172.21.15.28:0/1626579068 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fd2ec0b0d10 0x7fd2ec14fe10 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:09.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.153+0000 7fd2f3df1700 1 --2- 172.21.15.28:0/1626579068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd2ec1602f0 0x7fd2ec06b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:09.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.153+0000 7fd2f3df1700 1 --2- 172.21.15.28:0/1626579068 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fd2ec15ca20 0x7fd2ec0cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:09.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.153+0000 7fd2f3df1700 1 -- 172.21.15.28:0/1626579068 >> 172.21.15.28:0/1626579068 conn(0x7fd2ec0cbd10 msgr2=0x7fd2ec0acbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:09.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.153+0000 7fd2f3df1700 1 -- 172.21.15.28:0/1626579068 shutdown_connections 2023-11-28T11:12:09.154 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.153+0000 7fd2f3df1700 1 -- 172.21.15.28:0/1626579068 wait complete. 2023-11-28T11:12:09.165 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T11:12:09.165 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:12:09.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.289+0000 7f8eeb96c700 1 Processor -- start 2023-11-28T11:12:09.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.289+0000 7f8eeb96c700 1 -- start start 2023-11-28T11:12:09.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8eeb96c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8ee40cb470 0x7f8ee40ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:09.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8eeb96c700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee406c140 con 0x7f8ee40cb0a0 2023-11-28T11:12:09.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8eeb96c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ee40c9ab0 con 0x7f8ee406bb80 2023-11-28T11:12:09.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8eeb96c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ee40caad0 con 0x7f8ee40cb470 2023-11-28T11:12:09.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8ee9708700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8ee40cb0a0 0x7f8ee415a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39698/0 (socket says 172.21.15.28:39698) 2023-11-28T11:12:09.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8ee9708700 1 -- 172.21.15.28:0/2335646328 learned_addr learned my addr 172.21.15.28:0/2335646328 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:09.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8ee8f07700 1 --2- 172.21.15.28:0/2335646328 >> v2:172.21.15.59:3300/0 conn(0x7f8ee40cb470 0x7f8ee40ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:09.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8edbfff700 1 -- 172.21.15.28:0/2335646328 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504636804 0 0) 0x7f8ee406c140 con 0x7f8ee40cb0a0 2023-11-28T11:12:09.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8edbfff700 1 -- 172.21.15.28:0/2335646328 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ed00057c0 con 0x7f8ee40cb0a0 2023-11-28T11:12:09.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8edbfff700 1 -- 172.21.15.28:0/2335646328 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1545846243 0 0) 0x7f8ed00057c0 con 0x7f8ee40cb0a0 2023-11-28T11:12:09.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8edbfff700 1 -- 172.21.15.28:0/2335646328 >> v2:172.21.15.59:3300/0 conn(0x7f8ee40cb470 msgr2=0x7f8ee40ca590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:09.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8edbfff700 1 --2- 172.21.15.28:0/2335646328 >> v2:172.21.15.59:3300/0 conn(0x7f8ee40cb470 0x7f8ee40ca590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:09.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8edbfff700 1 -- 172.21.15.28:0/2335646328 >> v1:172.21.15.59:6789/0 conn(0x7f8ee406bb80 legacy=0x7f8ee414fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:09.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8edbfff700 1 -- 172.21.15.28:0/2335646328 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee40c9d90 con 0x7f8ee40cb0a0 2023-11-28T11:12:09.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8edbfff700 1 -- 172.21.15.28:0/2335646328 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f8ed4001670 con 0x7f8ee40cb0a0 2023-11-28T11:12:09.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8edbfff700 1 -- 172.21.15.28:0/2335646328 >> v1:172.21.15.28:6789/0 conn(0x7f8ee40cb0a0 legacy=0x7f8ee415a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:09.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8edbfff700 1 --2- 172.21.15.28:0/2335646328 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ed0005c10 0x7f8ed0006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:09.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8edbfff700 1 --2- 172.21.15.28:0/2335646328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ed0006a60 0x7f8ed0006ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:09.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8edbfff700 1 --2- 172.21.15.28:0/2335646328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0007410 0x7f8ed000dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:09.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8edbfff700 1 -- 172.21.15.28:0/2335646328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed000e2f0 con 0x7f8ed0007410 2023-11-28T11:12:09.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8edbfff700 1 -- 172.21.15.28:0/2335646328 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed000e460 con 0x7f8ed0006a60 2023-11-28T11:12:09.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8edbfff700 1 -- 172.21.15.28:0/2335646328 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed000e620 con 0x7f8ed0005c10 2023-11-28T11:12:09.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.291+0000 7f8ee9f09700 1 --2- 172.21.15.28:0/2335646328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0007410 0x7f8ed000dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:09.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.292+0000 7f8ee8f07700 1 --2- 172.21.15.28:0/2335646328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ed0006a60 0x7f8ed0006ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:09.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.292+0000 7f8ee9708700 1 --2- 172.21.15.28:0/2335646328 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ed0005c10 0x7f8ed0006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:09.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.292+0000 7f8ee9f09700 1 -- 172.21.15.28:0/2335646328 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ed0005c10 msgr2=0x7f8ed0006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:09.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.292+0000 7f8ee9f09700 1 --2- 172.21.15.28:0/2335646328 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ed0005c10 0x7f8ed0006020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:09.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.292+0000 7f8ee9f09700 1 -- 172.21.15.28:0/2335646328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ed0006a60 msgr2=0x7f8ed0006ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:09.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.292+0000 7f8ee9f09700 1 --2- 172.21.15.28:0/2335646328 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ed0006a60 0x7f8ed0006ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:09.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.292+0000 7f8ee9f09700 1 -- 172.21.15.28:0/2335646328 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed00057c0 con 0x7f8ed0007410 2023-11-28T11:12:09.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.292+0000 7f8ee9f09700 1 -- 172.21.15.28:0/2335646328 reap_dead start 2023-11-28T11:12:09.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.292+0000 7f8ee9f09700 1 --2- 172.21.15.28:0/2335646328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0007410 0x7f8ed000dc20 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00c240 tx=0x7f8ed0007a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25039dcd8d0391a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:09.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.292+0000 7f8edbfff700 1 -- 172.21.15.28:0/2335646328 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc001df0 con 0x7f8ed0007410 2023-11-28T11:12:09.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.292+0000 7f8edbfff700 1 -- 172.21.15.28:0/2335646328 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8edc00cdc0 con 0x7f8ed0007410 2023-11-28T11:12:09.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.292+0000 7f8edbfff700 1 -- 172.21.15.28:0/2335646328 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8edc003c40 con 0x7f8ed0007410 2023-11-28T11:12:09.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.293+0000 7f8eeb96c700 1 -- 172.21.15.28:0/2335646328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0007410 msgr2=0x7f8ed000dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:09.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.293+0000 7f8eeb96c700 1 --2- 172.21.15.28:0/2335646328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0007410 0x7f8ed000dc20 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f8edc00c240 tx=0x7f8ed0007a90 comp rx=0 tx=0).stop 2023-11-28T11:12:09.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.293+0000 7f8eeb96c700 1 -- 172.21.15.28:0/2335646328 shutdown_connections 2023-11-28T11:12:09.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.293+0000 7f8eeb96c700 1 --2- 172.21.15.28:0/2335646328 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0007410 0x7f8ed000dc20 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:09.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.293+0000 7f8eeb96c700 1 -- 172.21.15.28:0/2335646328 >> 172.21.15.28:0/2335646328 conn(0x7f8ee40b0dd0 msgr2=0x7f8ee40b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:09.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.293+0000 7f8eeb96c700 1 -- 172.21.15.28:0/2335646328 shutdown_connections 2023-11-28T11:12:09.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.293+0000 7f8eeb96c700 1 -- 172.21.15.28:0/2335646328 wait complete. 2023-11-28T11:12:09.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.294+0000 7f8eeb96c700 1 Processor -- start 2023-11-28T11:12:09.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.294+0000 7f8eeb96c700 1 -- start start 2023-11-28T11:12:09.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8eeb96c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0007410 0x7f8ee40b3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:09.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8eeb96c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ee4153810 0x7f8ee40b3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8eeb96c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ee40b4510 0x7f8ee4158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8eeb96c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee40b3a90 con 0x7f8ed0007410 2023-11-28T11:12:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8eeb96c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee40b30c0 con 0x7f8ee40b4510 2023-11-28T11:12:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8eeb96c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ee40b24b0 con 0x7f8ee4153810 2023-11-28T11:12:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8ee9708700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0007410 0x7f8ee40b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8ee9708700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0007410 0x7f8ee40b3550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51652/0 (socket says 172.21.15.28:51652) 2023-11-28T11:12:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8ee9708700 1 -- 172.21.15.28:0/241379541 learned_addr learned my addr 172.21.15.28:0/241379541 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8ee9f09700 1 --2- 172.21.15.28:0/241379541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ee40b4510 0x7f8ee4158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8ee8f07700 1 --2- 172.21.15.28:0/241379541 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ee4153810 0x7f8ee40b3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8ee9708700 1 -- 172.21.15.28:0/241379541 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ee4153810 msgr2=0x7f8ee40b3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8ee9708700 1 --2- 172.21.15.28:0/241379541 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ee4153810 0x7f8ee40b3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:09.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8ee9708700 1 -- 172.21.15.28:0/241379541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ee40b4510 msgr2=0x7f8ee4158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:09.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8ee9708700 1 --2- 172.21.15.28:0/241379541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ee40b4510 0x7f8ee4158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:09.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8ee9708700 1 -- 172.21.15.28:0/241379541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ee40ca590 con 0x7f8ed0007410 2023-11-28T11:12:09.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8ee9708700 1 --2- 172.21.15.28:0/241379541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0007410 0x7f8ee40b3550 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40072c0 tx=0x7f8ed4007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:09.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8ed9ffb700 1 -- 172.21.15.28:0/241379541 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed40077a0 con 0x7f8ed0007410 2023-11-28T11:12:09.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8ed9ffb700 1 -- 172.21.15.28:0/241379541 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed400f8d0 con 0x7f8ed0007410 2023-11-28T11:12:09.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8eeb96c700 1 -- 172.21.15.28:0/241379541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ee40ca070 con 0x7f8ed0007410 2023-11-28T11:12:09.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.295+0000 7f8ed9ffb700 1 -- 172.21.15.28:0/241379541 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed400fc80 con 0x7f8ed0007410 2023-11-28T11:12:09.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.296+0000 7f8eeb96c700 1 -- 172.21.15.28:0/241379541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ee40ca720 con 0x7f8ed0007410 2023-11-28T11:12:09.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.297+0000 7f8eeb96c700 1 -- 172.21.15.28:0/241379541 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ee40b48e0 con 0x7f8ed0007410 2023-11-28T11:12:09.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.297+0000 7f8ed9ffb700 1 -- 172.21.15.28:0/241379541 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8ed401d460 con 0x7f8ed0007410 2023-11-28T11:12:09.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.297+0000 7f8ed9ffb700 1 --2- 172.21.15.28:0/241379541 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8ec80682f0 0x7f8ec806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:09.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.298+0000 7f8ee8f07700 1 --2- 172.21.15.28:0/241379541 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8ec80682f0 0x7f8ec806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:09.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.298+0000 7f8ed9ffb700 1 -- 172.21.15.28:0/241379541 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f8ed4085f70 con 0x7f8ed0007410 2023-11-28T11:12:09.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.298+0000 7f8ee8f07700 1 --2- 172.21.15.28:0/241379541 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8ec80682f0 0x7f8ec806a7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4159120 tx=0x7f8ee0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:09.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.302+0000 7f8ed9ffb700 1 -- 172.21.15.28:0/241379541 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8ed40526e0 con 0x7f8ed0007410 2023-11-28T11:12:09.372 INFO:tasks.ceph.mds.d.smithi059.stderr:2023-11-28T11:12:09.371+0000 7fa84c78e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:12:09.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.493+0000 7f8eeb96c700 1 -- 172.21.15.28:0/241379541 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ee40b3200 con 0x7f8ec80682f0 2023-11-28T11:12:09.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.502+0000 7f8ed9ffb700 1 -- 172.21.15.28:0/241379541 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311459 (secure 0 0 0) 0x7f8ee0009770 con 0x7f8ec80682f0 2023-11-28T11:12:09.506 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:09.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.505+0000 7f8eeb96c700 1 -- 172.21.15.28:0/241379541 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8ec80682f0 msgr2=0x7f8ec806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:09.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.505+0000 7f8eeb96c700 1 --2- 172.21.15.28:0/241379541 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8ec80682f0 0x7f8ec806a7a0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f8ee4159120 tx=0x7f8ee0008040 comp rx=0 tx=0).stop 2023-11-28T11:12:09.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.505+0000 7f8eeb96c700 1 -- 172.21.15.28:0/241379541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0007410 msgr2=0x7f8ee40b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:09.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.505+0000 7f8eeb96c700 1 --2- 172.21.15.28:0/241379541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0007410 0x7f8ee40b3550 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40072c0 tx=0x7f8ed4007680 comp rx=0 tx=0).stop 2023-11-28T11:12:09.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.505+0000 7f8eeb96c700 1 -- 172.21.15.28:0/241379541 shutdown_connections 2023-11-28T11:12:09.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.505+0000 7f8eeb96c700 1 --2- 172.21.15.28:0/241379541 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8ec80682f0 0x7f8ec806a7a0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:09.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.505+0000 7f8eeb96c700 1 --2- 172.21.15.28:0/241379541 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8ed0007410 0x7f8ee40b3550 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:09.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.505+0000 7f8eeb96c700 1 --2- 172.21.15.28:0/241379541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8ee40b4510 0x7f8ee4158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:09.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.505+0000 7f8eeb96c700 1 --2- 172.21.15.28:0/241379541 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8ee4153810 0x7f8ee40b3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:09.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.505+0000 7f8eeb96c700 1 -- 172.21.15.28:0/241379541 >> 172.21.15.28:0/241379541 conn(0x7f8ee40b0dd0 msgr2=0x7f8ee40aecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:09.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.505+0000 7f8eeb96c700 1 -- 172.21.15.28:0/241379541 shutdown_connections 2023-11-28T11:12:09.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:09.505+0000 7f8eeb96c700 1 -- 172.21.15.28:0/241379541 wait complete. 2023-11-28T11:12:09.510 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:12:09.518 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":165,"stamp":"2023-11-28T11:12:09.208822+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":200,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":80164,"kb_used_data":7460,"kb_used_omap":0,"kb_used_meta":72640,"kb_avail":754894556,"statfs":{"total":773094113280,"available":773012025344,"internally_reserved":0,"allocated":7639040,"data_stored":4185459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":74383360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001356"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115820+0000","last_change":"2023-11-28T11:12:03.835618+0000","last_active":"2023-11-28T11:12:06.115820+0000","last_peered":"2023-11-28T11:12:06.115820+0000","last_clean":"2023-11-28T11:12:06.115820+0000","last_became_active":"2023-11-28T11:12:03.833256+0000","last_became_peered":"2023-11-28T11:12:03.833256+0000","last_unstale":"2023-11-28T11:12:06.115820+0000","last_undegraded":"2023-11-28T11:12:06.115820+0000","last_fullsized":"2023-11-28T11:12:06.115820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:59.308020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.202640+0000","last_change":"2023-11-28T11:12:03.841405+0000","last_active":"2023-11-28T11:12:05.202640+0000","last_peered":"2023-11-28T11:12:05.202640+0000","last_clean":"2023-11-28T11:12:05.202640+0000","last_became_active":"2023-11-28T11:12:03.841128+0000","last_became_peered":"2023-11-28T11:12:03.841128+0000","last_unstale":"2023-11-28T11:12:05.202640+0000","last_undegraded":"2023-11-28T11:12:05.202640+0000","last_fullsized":"2023-11-28T11:12:05.202640+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.904871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114245+0000","last_change":"2023-11-28T11:12:03.816160+0000","last_active":"2023-11-28T11:12:06.114245+0000","last_peered":"2023-11-28T11:12:06.114245+0000","last_clean":"2023-11-28T11:12:06.114245+0000","last_became_active":"2023-11-28T11:12:03.815913+0000","last_became_peered":"2023-11-28T11:12:03.815913+0000","last_unstale":"2023-11-28T11:12:06.114245+0000","last_undegraded":"2023-11-28T11:12:06.114245+0000","last_fullsized":"2023-11-28T11:12:06.114245+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:32.223293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.809974+0000","last_change":"2023-11-28T11:12:03.828381+0000","last_active":"2023-11-28T11:12:04.809974+0000","last_peered":"2023-11-28T11:12:04.809974+0000","last_clean":"2023-11-28T11:12:04.809974+0000","last_became_active":"2023-11-28T11:12:03.827787+0000","last_became_peered":"2023-11-28T11:12:03.827787+0000","last_unstale":"2023-11-28T11:12:04.809974+0000","last_undegraded":"2023-11-28T11:12:04.809974+0000","last_fullsized":"2023-11-28T11:12:04.809974+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:39.132766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348245+0000","last_change":"2023-11-28T11:12:03.830513+0000","last_active":"2023-11-28T11:12:05.348245+0000","last_peered":"2023-11-28T11:12:05.348245+0000","last_clean":"2023-11-28T11:12:05.348245+0000","last_became_active":"2023-11-28T11:12:03.830213+0000","last_became_peered":"2023-11-28T11:12:03.830213+0000","last_unstale":"2023-11-28T11:12:05.348245+0000","last_undegraded":"2023-11-28T11:12:05.348245+0000","last_fullsized":"2023-11-28T11:12:05.348245+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:19.707029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.201989+0000","last_change":"2023-11-28T11:12:03.841348+0000","last_active":"2023-11-28T11:12:05.201989+0000","last_peered":"2023-11-28T11:12:05.201989+0000","last_clean":"2023-11-28T11:12:05.201989+0000","last_became_active":"2023-11-28T11:12:03.841037+0000","last_became_peered":"2023-11-28T11:12:03.841037+0000","last_unstale":"2023-11-28T11:12:05.201989+0000","last_undegraded":"2023-11-28T11:12:05.201989+0000","last_fullsized":"2023-11-28T11:12:05.201989+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:30.504376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.810853+0000","last_change":"2023-11-28T11:12:03.816285+0000","last_active":"2023-11-28T11:12:04.810853+0000","last_peered":"2023-11-28T11:12:04.810853+0000","last_clean":"2023-11-28T11:12:04.810853+0000","last_became_active":"2023-11-28T11:12:03.815963+0000","last_became_peered":"2023-11-28T11:12:03.815963+0000","last_unstale":"2023-11-28T11:12:04.810853+0000","last_undegraded":"2023-11-28T11:12:04.810853+0000","last_fullsized":"2023-11-28T11:12:04.810853+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:09.091995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.201966+0000","last_change":"2023-11-28T11:12:03.830920+0000","last_active":"2023-11-28T11:12:05.201966+0000","last_peered":"2023-11-28T11:12:05.201966+0000","last_clean":"2023-11-28T11:12:05.201966+0000","last_became_active":"2023-11-28T11:12:03.830633+0000","last_became_peered":"2023-11-28T11:12:03.830633+0000","last_unstale":"2023-11-28T11:12:05.201966+0000","last_undegraded":"2023-11-28T11:12:05.201966+0000","last_fullsized":"2023-11-28T11:12:05.201966+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:02.573643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348500+0000","last_change":"2023-11-28T11:12:03.821337+0000","last_active":"2023-11-28T11:12:05.348500+0000","last_peered":"2023-11-28T11:12:05.348500+0000","last_clean":"2023-11-28T11:12:05.348500+0000","last_became_active":"2023-11-28T11:12:03.820702+0000","last_became_peered":"2023-11-28T11:12:03.820702+0000","last_unstale":"2023-11-28T11:12:05.348500+0000","last_undegraded":"2023-11-28T11:12:05.348500+0000","last_fullsized":"2023-11-28T11:12:05.348500+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:05.835846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115635+0000","last_change":"2023-11-28T11:12:03.823397+0000","last_active":"2023-11-28T11:12:06.115635+0000","last_peered":"2023-11-28T11:12:06.115635+0000","last_clean":"2023-11-28T11:12:06.115635+0000","last_became_active":"2023-11-28T11:12:03.823037+0000","last_became_peered":"2023-11-28T11:12:03.823037+0000","last_unstale":"2023-11-28T11:12:06.115635+0000","last_undegraded":"2023-11-28T11:12:06.115635+0000","last_fullsized":"2023-11-28T11:12:06.115635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:33.447204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115464+0000","last_change":"2023-11-28T11:12:03.830195+0000","last_active":"2023-11-28T11:12:06.115464+0000","last_peered":"2023-11-28T11:12:06.115464+0000","last_clean":"2023-11-28T11:12:06.115464+0000","last_became_active":"2023-11-28T11:12:03.829670+0000","last_became_peered":"2023-11-28T11:12:03.829670+0000","last_unstale":"2023-11-28T11:12:06.115464+0000","last_undegraded":"2023-11-28T11:12:06.115464+0000","last_fullsized":"2023-11-28T11:12:06.115464+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:32:23.420139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.810029+0000","last_change":"2023-11-28T11:12:03.824974+0000","last_active":"2023-11-28T11:12:04.810029+0000","last_peered":"2023-11-28T11:12:04.810029+0000","last_clean":"2023-11-28T11:12:04.810029+0000","last_became_active":"2023-11-28T11:12:03.824539+0000","last_became_peered":"2023-11-28T11:12:03.824539+0000","last_unstale":"2023-11-28T11:12:04.810029+0000","last_undegraded":"2023-11-28T11:12:04.810029+0000","last_fullsized":"2023-11-28T11:12:04.810029+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:17.662708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.165156+0000","last_change":"2023-11-28T11:12:03.821038+0000","last_active":"2023-11-28T11:12:05.165156+0000","last_peered":"2023-11-28T11:12:05.165156+0000","last_clean":"2023-11-28T11:12:05.165156+0000","last_became_active":"2023-11-28T11:12:03.820762+0000","last_became_peered":"2023-11-28T11:12:03.820762+0000","last_unstale":"2023-11-28T11:12:05.165156+0000","last_undegraded":"2023-11-28T11:12:05.165156+0000","last_fullsized":"2023-11-28T11:12:05.165156+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:25:33.025411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348295+0000","last_change":"2023-11-28T11:12:03.821353+0000","last_active":"2023-11-28T11:12:05.348295+0000","last_peered":"2023-11-28T11:12:05.348295+0000","last_clean":"2023-11-28T11:12:05.348295+0000","last_became_active":"2023-11-28T11:12:03.821076+0000","last_became_peered":"2023-11-28T11:12:03.821076+0000","last_unstale":"2023-11-28T11:12:05.348295+0000","last_undegraded":"2023-11-28T11:12:05.348295+0000","last_fullsized":"2023-11-28T11:12:05.348295+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:50.253755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.164947+0000","last_change":"2023-11-28T11:12:03.822735+0000","last_active":"2023-11-28T11:12:05.164947+0000","last_peered":"2023-11-28T11:12:05.164947+0000","last_clean":"2023-11-28T11:12:05.164947+0000","last_became_active":"2023-11-28T11:12:03.822440+0000","last_became_peered":"2023-11-28T11:12:03.822440+0000","last_unstale":"2023-11-28T11:12:05.164947+0000","last_undegraded":"2023-11-28T11:12:05.164947+0000","last_fullsized":"2023-11-28T11:12:05.164947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:16.770348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114756+0000","last_change":"2023-11-28T11:12:03.832886+0000","last_active":"2023-11-28T11:12:06.114756+0000","last_peered":"2023-11-28T11:12:06.114756+0000","last_clean":"2023-11-28T11:12:06.114756+0000","last_became_active":"2023-11-28T11:12:03.832554+0000","last_became_peered":"2023-11-28T11:12:03.832554+0000","last_unstale":"2023-11-28T11:12:06.114756+0000","last_undegraded":"2023-11-28T11:12:06.114756+0000","last_fullsized":"2023-11-28T11:12:06.114756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:01.111914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348633+0000","last_change":"2023-11-28T11:12:03.821113+0000","last_active":"2023-11-28T11:12:05.348633+0000","last_peered":"2023-11-28T11:12:05.348633+0000","last_clean":"2023-11-28T11:12:05.348633+0000","last_became_active":"2023-11-28T11:12:03.820374+0000","last_became_peered":"2023-11-28T11:12:03.820374+0000","last_unstale":"2023-11-28T11:12:05.348633+0000","last_undegraded":"2023-11-28T11:12:05.348633+0000","last_fullsized":"2023-11-28T11:12:05.348633+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:17:38.398355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.810095+0000","last_change":"2023-11-28T11:12:03.828247+0000","last_active":"2023-11-28T11:12:04.810095+0000","last_peered":"2023-11-28T11:12:04.810095+0000","last_clean":"2023-11-28T11:12:04.810095+0000","last_became_active":"2023-11-28T11:12:03.827965+0000","last_became_peered":"2023-11-28T11:12:03.827965+0000","last_unstale":"2023-11-28T11:12:04.810095+0000","last_undegraded":"2023-11-28T11:12:04.810095+0000","last_fullsized":"2023-11-28T11:12:04.810095+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:07.059900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.202300+0000","last_change":"2023-11-28T11:12:03.830360+0000","last_active":"2023-11-28T11:12:05.202300+0000","last_peered":"2023-11-28T11:12:05.202300+0000","last_clean":"2023-11-28T11:12:05.202300+0000","last_became_active":"2023-11-28T11:12:03.829992+0000","last_became_peered":"2023-11-28T11:12:03.829992+0000","last_unstale":"2023-11-28T11:12:05.202300+0000","last_undegraded":"2023-11-28T11:12:05.202300+0000","last_fullsized":"2023-11-28T11:12:05.202300+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:39:33.773688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.165118+0000","last_change":"2023-11-28T11:12:03.819529+0000","last_active":"2023-11-28T11:12:05.165118+0000","last_peered":"2023-11-28T11:12:05.165118+0000","last_clean":"2023-11-28T11:12:05.165118+0000","last_became_active":"2023-11-28T11:12:03.819245+0000","last_became_peered":"2023-11-28T11:12:03.819245+0000","last_unstale":"2023-11-28T11:12:05.165118+0000","last_undegraded":"2023-11-28T11:12:05.165118+0000","last_fullsized":"2023-11-28T11:12:05.165118+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:22:37.131304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115400+0000","last_change":"2023-11-28T11:12:03.823568+0000","last_active":"2023-11-28T11:12:06.115400+0000","last_peered":"2023-11-28T11:12:06.115400+0000","last_clean":"2023-11-28T11:12:06.115400+0000","last_became_active":"2023-11-28T11:12:03.823247+0000","last_became_peered":"2023-11-28T11:12:03.823247+0000","last_unstale":"2023-11-28T11:12:06.115400+0000","last_undegraded":"2023-11-28T11:12:06.115400+0000","last_fullsized":"2023-11-28T11:12:06.115400+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:48:55.831494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115050+0000","last_change":"2023-11-28T11:12:03.823483+0000","last_active":"2023-11-28T11:12:06.115050+0000","last_peered":"2023-11-28T11:12:06.115050+0000","last_clean":"2023-11-28T11:12:06.115050+0000","last_became_active":"2023-11-28T11:12:03.822785+0000","last_became_peered":"2023-11-28T11:12:03.822785+0000","last_unstale":"2023-11-28T11:12:06.115050+0000","last_undegraded":"2023-11-28T11:12:06.115050+0000","last_fullsized":"2023-11-28T11:12:06.115050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:46.824986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115031+0000","last_change":"2023-11-28T11:12:03.831161+0000","last_active":"2023-11-28T11:12:06.115031+0000","last_peered":"2023-11-28T11:12:06.115031+0000","last_clean":"2023-11-28T11:12:06.115031+0000","last_became_active":"2023-11-28T11:12:03.830869+0000","last_became_peered":"2023-11-28T11:12:03.830869+0000","last_unstale":"2023-11-28T11:12:06.115031+0000","last_undegraded":"2023-11-28T11:12:06.115031+0000","last_fullsized":"2023-11-28T11:12:06.115031+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:03:44.943498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115029+0000","last_change":"2023-11-28T11:12:03.833613+0000","last_active":"2023-11-28T11:12:06.115029+0000","last_peered":"2023-11-28T11:12:06.115029+0000","last_clean":"2023-11-28T11:12:06.115029+0000","last_became_active":"2023-11-28T11:12:03.832946+0000","last_became_peered":"2023-11-28T11:12:03.832946+0000","last_unstale":"2023-11-28T11:12:06.115029+0000","last_undegraded":"2023-11-28T11:12:06.115029+0000","last_fullsized":"2023-11-28T11:12:06.115029+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:33.447204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115452+0000","last_change":"2023-11-28T11:12:03.831302+0000","last_active":"2023-11-28T11:12:06.115452+0000","last_peered":"2023-11-28T11:12:06.115452+0000","last_clean":"2023-11-28T11:12:06.115452+0000","last_became_active":"2023-11-28T11:12:03.831032+0000","last_became_peered":"2023-11-28T11:12:03.831032+0000","last_unstale":"2023-11-28T11:12:06.115452+0000","last_undegraded":"2023-11-28T11:12:06.115452+0000","last_fullsized":"2023-11-28T11:12:06.115452+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:59.308020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.811227+0000","last_change":"2023-11-28T11:12:03.832151+0000","last_active":"2023-11-28T11:12:04.811227+0000","last_peered":"2023-11-28T11:12:04.811227+0000","last_clean":"2023-11-28T11:12:04.811227+0000","last_became_active":"2023-11-28T11:12:03.831816+0000","last_became_peered":"2023-11-28T11:12:03.831816+0000","last_unstale":"2023-11-28T11:12:04.811227+0000","last_undegraded":"2023-11-28T11:12:04.811227+0000","last_fullsized":"2023-11-28T11:12:04.811227+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:54.346774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114295+0000","last_change":"2023-11-28T11:12:03.826789+0000","last_active":"2023-11-28T11:12:06.114295+0000","last_peered":"2023-11-28T11:12:06.114295+0000","last_clean":"2023-11-28T11:12:06.114295+0000","last_became_active":"2023-11-28T11:12:03.826520+0000","last_became_peered":"2023-11-28T11:12:03.826520+0000","last_unstale":"2023-11-28T11:12:06.114295+0000","last_undegraded":"2023-11-28T11:12:06.114295+0000","last_fullsized":"2023-11-28T11:12:06.114295+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:39.541984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115504+0000","last_change":"2023-11-28T11:12:03.824121+0000","last_active":"2023-11-28T11:12:06.115504+0000","last_peered":"2023-11-28T11:12:06.115504+0000","last_clean":"2023-11-28T11:12:06.115504+0000","last_became_active":"2023-11-28T11:12:03.823702+0000","last_became_peered":"2023-11-28T11:12:03.823702+0000","last_unstale":"2023-11-28T11:12:06.115504+0000","last_undegraded":"2023-11-28T11:12:06.115504+0000","last_fullsized":"2023-11-28T11:12:06.115504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:07:36.641433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114167+0000","last_change":"2023-11-28T11:12:03.818921+0000","last_active":"2023-11-28T11:12:06.114167+0000","last_peered":"2023-11-28T11:12:06.114167+0000","last_clean":"2023-11-28T11:12:06.114167+0000","last_became_active":"2023-11-28T11:12:03.818749+0000","last_became_peered":"2023-11-28T11:12:03.818749+0000","last_unstale":"2023-11-28T11:12:06.114167+0000","last_undegraded":"2023-11-28T11:12:06.114167+0000","last_fullsized":"2023-11-28T11:12:06.114167+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:47.752363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.811030+0000","last_change":"2023-11-28T11:12:03.832151+0000","last_active":"2023-11-28T11:12:04.811030+0000","last_peered":"2023-11-28T11:12:04.811030+0000","last_clean":"2023-11-28T11:12:04.811030+0000","last_became_active":"2023-11-28T11:12:03.831807+0000","last_became_peered":"2023-11-28T11:12:03.831807+0000","last_unstale":"2023-11-28T11:12:04.811030+0000","last_undegraded":"2023-11-28T11:12:04.811030+0000","last_fullsized":"2023-11-28T11:12:04.811030+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:46.858297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.811079+0000","last_change":"2023-11-28T11:12:03.822425+0000","last_active":"2023-11-28T11:12:04.811079+0000","last_peered":"2023-11-28T11:12:04.811079+0000","last_clean":"2023-11-28T11:12:04.811079+0000","last_became_active":"2023-11-28T11:12:03.822117+0000","last_became_peered":"2023-11-28T11:12:03.822117+0000","last_unstale":"2023-11-28T11:12:04.811079+0000","last_undegraded":"2023-11-28T11:12:04.811079+0000","last_fullsized":"2023-11-28T11:12:04.811079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:18:47.403543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.201905+0000","last_change":"2023-11-28T11:12:03.819807+0000","last_active":"2023-11-28T11:12:05.201905+0000","last_peered":"2023-11-28T11:12:05.201905+0000","last_clean":"2023-11-28T11:12:05.201905+0000","last_became_active":"2023-11-28T11:12:03.819519+0000","last_became_peered":"2023-11-28T11:12:03.819519+0000","last_unstale":"2023-11-28T11:12:05.201905+0000","last_undegraded":"2023-11-28T11:12:05.201905+0000","last_fullsized":"2023-11-28T11:12:05.201905+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:19.630863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115907+0000","last_change":"2023-11-28T11:12:03.839270+0000","last_active":"2023-11-28T11:12:06.115907+0000","last_peered":"2023-11-28T11:12:06.115907+0000","last_clean":"2023-11-28T11:12:06.115907+0000","last_became_active":"2023-11-28T11:12:03.839008+0000","last_became_peered":"2023-11-28T11:12:03.839008+0000","last_unstale":"2023-11-28T11:12:06.115907+0000","last_undegraded":"2023-11-28T11:12:06.115907+0000","last_fullsized":"2023-11-28T11:12:06.115907+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:48:55.831494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"65'4","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.838840+0000","last_change":"2023-11-28T11:12:03.405175+0000","last_active":"2023-11-28T11:12:03.838840+0000","last_peered":"2023-11-28T11:12:03.838840+0000","last_clean":"2023-11-28T11:12:03.838840+0000","last_became_active":"2023-11-28T11:12:03.404896+0000","last_became_peered":"2023-11-28T11:12:03.404896+0000","last_unstale":"2023-11-28T11:12:03.838840+0000","last_undegraded":"2023-11-28T11:12:03.838840+0000","last_fullsized":"2023-11-28T11:12:03.838840+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:18.625956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348649+0000","last_change":"2023-11-28T11:12:03.820945+0000","last_active":"2023-11-28T11:12:05.348649+0000","last_peered":"2023-11-28T11:12:05.348649+0000","last_clean":"2023-11-28T11:12:05.348649+0000","last_became_active":"2023-11-28T11:12:03.820261+0000","last_became_peered":"2023-11-28T11:12:03.820261+0000","last_unstale":"2023-11-28T11:12:05.348649+0000","last_undegraded":"2023-11-28T11:12:05.348649+0000","last_fullsized":"2023-11-28T11:12:05.348649+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:37.833203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.821456+0000","last_change":"2023-11-28T11:12:03.401516+0000","last_active":"2023-11-28T11:12:03.821456+0000","last_peered":"2023-11-28T11:12:03.821456+0000","last_clean":"2023-11-28T11:12:03.821456+0000","last_became_active":"2023-11-28T11:12:03.401240+0000","last_became_peered":"2023-11-28T11:12:03.401240+0000","last_unstale":"2023-11-28T11:12:03.821456+0000","last_undegraded":"2023-11-28T11:12:03.821456+0000","last_fullsized":"2023-11-28T11:12:03.821456+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:32:22.415232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.202299+0000","last_change":"2023-11-28T11:12:03.826765+0000","last_active":"2023-11-28T11:12:05.202299+0000","last_peered":"2023-11-28T11:12:05.202299+0000","last_clean":"2023-11-28T11:12:05.202299+0000","last_became_active":"2023-11-28T11:12:03.826472+0000","last_became_peered":"2023-11-28T11:12:03.826472+0000","last_unstale":"2023-11-28T11:12:05.202299+0000","last_undegraded":"2023-11-28T11:12:05.202299+0000","last_fullsized":"2023-11-28T11:12:05.202299+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:04:36.395113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.840681+0000","last_change":"2023-11-28T11:12:03.404430+0000","last_active":"2023-11-28T11:12:03.840681+0000","last_peered":"2023-11-28T11:12:03.840681+0000","last_clean":"2023-11-28T11:12:03.840681+0000","last_became_active":"2023-11-28T11:12:03.404085+0000","last_became_peered":"2023-11-28T11:12:03.404085+0000","last_unstale":"2023-11-28T11:12:03.840681+0000","last_undegraded":"2023-11-28T11:12:03.840681+0000","last_fullsized":"2023-11-28T11:12:03.840681+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:31:31.263698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115523+0000","last_change":"2023-11-28T11:12:03.829985+0000","last_active":"2023-11-28T11:12:06.115523+0000","last_peered":"2023-11-28T11:12:06.115523+0000","last_clean":"2023-11-28T11:12:06.115523+0000","last_became_active":"2023-11-28T11:12:03.829105+0000","last_became_peered":"2023-11-28T11:12:03.829105+0000","last_unstale":"2023-11-28T11:12:06.115523+0000","last_undegraded":"2023-11-28T11:12:06.115523+0000","last_fullsized":"2023-11-28T11:12:06.115523+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:30.549631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.829662+0000","last_change":"2023-11-28T11:12:03.402127+0000","last_active":"2023-11-28T11:12:03.829662+0000","last_peered":"2023-11-28T11:12:03.829662+0000","last_clean":"2023-11-28T11:12:03.829662+0000","last_became_active":"2023-11-28T11:12:03.400646+0000","last_became_peered":"2023-11-28T11:12:03.400646+0000","last_unstale":"2023-11-28T11:12:03.829662+0000","last_undegraded":"2023-11-28T11:12:03.829662+0000","last_fullsized":"2023-11-28T11:12:03.829662+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:36.828296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.811066+0000","last_change":"2023-11-28T11:12:03.817612+0000","last_active":"2023-11-28T11:12:04.811066+0000","last_peered":"2023-11-28T11:12:04.811066+0000","last_clean":"2023-11-28T11:12:04.811066+0000","last_became_active":"2023-11-28T11:12:03.817002+0000","last_became_peered":"2023-11-28T11:12:03.817002+0000","last_unstale":"2023-11-28T11:12:04.811066+0000","last_undegraded":"2023-11-28T11:12:04.811066+0000","last_fullsized":"2023-11-28T11:12:04.811066+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:30.549631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.840145+0000","last_change":"2023-11-28T11:12:03.399675+0000","last_active":"2023-11-28T11:12:03.840145+0000","last_peered":"2023-11-28T11:12:03.840145+0000","last_clean":"2023-11-28T11:12:03.840145+0000","last_became_active":"2023-11-28T11:12:03.399395+0000","last_became_peered":"2023-11-28T11:12:03.399395+0000","last_unstale":"2023-11-28T11:12:03.840145+0000","last_undegraded":"2023-11-28T11:12:03.840145+0000","last_fullsized":"2023-11-28T11:12:03.840145+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:07:35.636526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115515+0000","last_change":"2023-11-28T11:12:03.831136+0000","last_active":"2023-11-28T11:12:06.115515+0000","last_peered":"2023-11-28T11:12:06.115515+0000","last_clean":"2023-11-28T11:12:06.115515+0000","last_became_active":"2023-11-28T11:12:03.830872+0000","last_became_peered":"2023-11-28T11:12:03.830872+0000","last_unstale":"2023-11-28T11:12:06.115515+0000","last_undegraded":"2023-11-28T11:12:06.115515+0000","last_fullsized":"2023-11-28T11:12:06.115515+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:54.346774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.831025+0000","last_change":"2023-11-28T11:12:03.407430+0000","last_active":"2023-11-28T11:12:03.831025+0000","last_peered":"2023-11-28T11:12:03.831025+0000","last_clean":"2023-11-28T11:12:03.831025+0000","last_became_active":"2023-11-28T11:12:03.407106+0000","last_became_peered":"2023-11-28T11:12:03.407106+0000","last_unstale":"2023-11-28T11:12:03.831025+0000","last_undegraded":"2023-11-28T11:12:03.831025+0000","last_fullsized":"2023-11-28T11:12:03.831025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:48:16.699038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348061+0000","last_change":"2023-11-28T11:12:03.833947+0000","last_active":"2023-11-28T11:12:05.348061+0000","last_peered":"2023-11-28T11:12:05.348061+0000","last_clean":"2023-11-28T11:12:05.348061+0000","last_became_active":"2023-11-28T11:12:03.833669+0000","last_became_peered":"2023-11-28T11:12:03.833669+0000","last_unstale":"2023-11-28T11:12:05.348061+0000","last_undegraded":"2023-11-28T11:12:05.348061+0000","last_fullsized":"2023-11-28T11:12:05.348061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:05.997226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.807329+0000","last_change":"2023-11-28T11:12:03.390405+0000","last_active":"2023-11-28T11:12:03.807329+0000","last_peered":"2023-11-28T11:12:03.807329+0000","last_clean":"2023-11-28T11:12:03.807329+0000","last_became_active":"2023-11-28T11:12:03.390092+0000","last_became_peered":"2023-11-28T11:12:03.390092+0000","last_unstale":"2023-11-28T11:12:03.807329+0000","last_undegraded":"2023-11-28T11:12:03.807329+0000","last_fullsized":"2023-11-28T11:12:03.807329+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:51:23.997889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.164957+0000","last_change":"2023-11-28T11:12:03.826749+0000","last_active":"2023-11-28T11:12:05.164957+0000","last_peered":"2023-11-28T11:12:05.164957+0000","last_clean":"2023-11-28T11:12:05.164957+0000","last_became_active":"2023-11-28T11:12:03.826451+0000","last_became_peered":"2023-11-28T11:12:03.826451+0000","last_unstale":"2023-11-28T11:12:05.164957+0000","last_undegraded":"2023-11-28T11:12:05.164957+0000","last_fullsized":"2023-11-28T11:12:05.164957+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:22:12.398418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.809575+0000","last_change":"2023-11-28T11:12:03.392079+0000","last_active":"2023-11-28T11:12:03.809575+0000","last_peered":"2023-11-28T11:12:03.809575+0000","last_clean":"2023-11-28T11:12:03.809575+0000","last_became_active":"2023-11-28T11:12:03.391649+0000","last_became_peered":"2023-11-28T11:12:03.391649+0000","last_unstale":"2023-11-28T11:12:03.809575+0000","last_undegraded":"2023-11-28T11:12:03.809575+0000","last_fullsized":"2023-11-28T11:12:03.809575+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:03:13.999980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.811346+0000","last_change":"2023-11-28T11:12:03.824511+0000","last_active":"2023-11-28T11:12:04.811346+0000","last_peered":"2023-11-28T11:12:04.811346+0000","last_clean":"2023-11-28T11:12:04.811346+0000","last_became_active":"2023-11-28T11:12:03.824178+0000","last_became_peered":"2023-11-28T11:12:03.824178+0000","last_unstale":"2023-11-28T11:12:04.811346+0000","last_undegraded":"2023-11-28T11:12:04.811346+0000","last_fullsized":"2023-11-28T11:12:04.811346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:32:23.420139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.815173+0000","last_change":"2023-11-28T11:12:03.400809+0000","last_active":"2023-11-28T11:12:03.815173+0000","last_peered":"2023-11-28T11:12:03.815173+0000","last_clean":"2023-11-28T11:12:03.815173+0000","last_became_active":"2023-11-28T11:12:03.400503+0000","last_became_peered":"2023-11-28T11:12:03.400503+0000","last_unstale":"2023-11-28T11:12:03.815173+0000","last_undegraded":"2023-11-28T11:12:03.815173+0000","last_fullsized":"2023-11-28T11:12:03.815173+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:17.899964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115847+0000","last_change":"2023-11-28T11:12:03.836342+0000","last_active":"2023-11-28T11:12:06.115847+0000","last_peered":"2023-11-28T11:12:06.115847+0000","last_clean":"2023-11-28T11:12:06.115847+0000","last_became_active":"2023-11-28T11:12:03.836002+0000","last_became_peered":"2023-11-28T11:12:03.836002+0000","last_unstale":"2023-11-28T11:12:06.115847+0000","last_undegraded":"2023-11-28T11:12:06.115847+0000","last_fullsized":"2023-11-28T11:12:06.115847+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:42.823746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.816598+0000","last_change":"2023-11-28T11:12:03.407424+0000","last_active":"2023-11-28T11:12:03.816598+0000","last_peered":"2023-11-28T11:12:03.816598+0000","last_clean":"2023-11-28T11:12:03.816598+0000","last_became_active":"2023-11-28T11:12:03.407085+0000","last_became_peered":"2023-11-28T11:12:03.407085+0000","last_unstale":"2023-11-28T11:12:03.816598+0000","last_undegraded":"2023-11-28T11:12:03.816598+0000","last_fullsized":"2023-11-28T11:12:03.816598+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:38.258754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.202408+0000","last_change":"2023-11-28T11:12:03.842277+0000","last_active":"2023-11-28T11:12:05.202408+0000","last_peered":"2023-11-28T11:12:05.202408+0000","last_clean":"2023-11-28T11:12:05.202408+0000","last_became_active":"2023-11-28T11:12:03.841981+0000","last_became_peered":"2023-11-28T11:12:03.841981+0000","last_unstale":"2023-11-28T11:12:05.202408+0000","last_undegraded":"2023-11-28T11:12:05.202408+0000","last_fullsized":"2023-11-28T11:12:05.202408+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:15:30.537954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"65'3","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.832084+0000","last_change":"2023-11-28T11:12:03.399002+0000","last_active":"2023-11-28T11:12:03.832084+0000","last_peered":"2023-11-28T11:12:03.832084+0000","last_clean":"2023-11-28T11:12:03.832084+0000","last_became_active":"2023-11-28T11:12:03.398692+0000","last_became_peered":"2023-11-28T11:12:03.398692+0000","last_unstale":"2023-11-28T11:12:03.832084+0000","last_undegraded":"2023-11-28T11:12:03.832084+0000","last_fullsized":"2023-11-28T11:12:03.832084+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:32.442297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348502+0000","last_change":"2023-11-28T11:12:03.821916+0000","last_active":"2023-11-28T11:12:05.348502+0000","last_peered":"2023-11-28T11:12:05.348502+0000","last_clean":"2023-11-28T11:12:05.348502+0000","last_became_active":"2023-11-28T11:12:03.821120+0000","last_became_peered":"2023-11-28T11:12:03.821120+0000","last_unstale":"2023-11-28T11:12:05.348502+0000","last_undegraded":"2023-11-28T11:12:05.348502+0000","last_fullsized":"2023-11-28T11:12:05.348502+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:51:25.002796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"65'6","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.841666+0000","last_change":"2023-11-28T11:12:03.402773+0000","last_active":"2023-11-28T11:12:03.841666+0000","last_peered":"2023-11-28T11:12:03.841666+0000","last_clean":"2023-11-28T11:12:03.841666+0000","last_became_active":"2023-11-28T11:12:03.402308+0000","last_became_peered":"2023-11-28T11:12:03.402308+0000","last_unstale":"2023-11-28T11:12:03.841666+0000","last_undegraded":"2023-11-28T11:12:03.841666+0000","last_fullsized":"2023-11-28T11:12:03.841666+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:04.830939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114883+0000","last_change":"2023-11-28T11:12:03.829146+0000","last_active":"2023-11-28T11:12:06.114883+0000","last_peered":"2023-11-28T11:12:06.114883+0000","last_clean":"2023-11-28T11:12:06.114883+0000","last_became_active":"2023-11-28T11:12:03.828841+0000","last_became_peered":"2023-11-28T11:12:03.828841+0000","last_unstale":"2023-11-28T11:12:06.114883+0000","last_undegraded":"2023-11-28T11:12:06.114883+0000","last_fullsized":"2023-11-28T11:12:06.114883+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:46.858297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.836303+0000","last_change":"2023-11-28T11:12:03.400815+0000","last_active":"2023-11-28T11:12:03.836303+0000","last_peered":"2023-11-28T11:12:03.836303+0000","last_clean":"2023-11-28T11:12:03.836303+0000","last_became_active":"2023-11-28T11:12:03.400513+0000","last_became_peered":"2023-11-28T11:12:03.400513+0000","last_unstale":"2023-11-28T11:12:03.836303+0000","last_undegraded":"2023-11-28T11:12:03.836303+0000","last_fullsized":"2023-11-28T11:12:03.836303+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:45.516962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.811229+0000","last_change":"2023-11-28T11:12:03.840762+0000","last_active":"2023-11-28T11:12:04.811229+0000","last_peered":"2023-11-28T11:12:04.811229+0000","last_clean":"2023-11-28T11:12:04.811229+0000","last_became_active":"2023-11-28T11:12:03.840452+0000","last_became_peered":"2023-11-28T11:12:03.840452+0000","last_unstale":"2023-11-28T11:12:04.811229+0000","last_undegraded":"2023-11-28T11:12:04.811229+0000","last_fullsized":"2023-11-28T11:12:04.811229+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:37.484325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.816701+0000","last_change":"2023-11-28T11:12:03.404712+0000","last_active":"2023-11-28T11:12:03.816701+0000","last_peered":"2023-11-28T11:12:03.816701+0000","last_clean":"2023-11-28T11:12:03.816701+0000","last_became_active":"2023-11-28T11:12:03.402929+0000","last_became_peered":"2023-11-28T11:12:03.402929+0000","last_unstale":"2023-11-28T11:12:03.816701+0000","last_undegraded":"2023-11-28T11:12:03.816701+0000","last_fullsized":"2023-11-28T11:12:03.816701+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:09:26.168016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.164911+0000","last_change":"2023-11-28T11:12:03.821050+0000","last_active":"2023-11-28T11:12:05.164911+0000","last_peered":"2023-11-28T11:12:05.164911+0000","last_clean":"2023-11-28T11:12:05.164911+0000","last_became_active":"2023-11-28T11:12:03.820787+0000","last_became_peered":"2023-11-28T11:12:03.820787+0000","last_unstale":"2023-11-28T11:12:05.164911+0000","last_undegraded":"2023-11-28T11:12:05.164911+0000","last_fullsized":"2023-11-28T11:12:05.164911+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:29:36.124745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.810573+0000","last_change":"2023-11-28T11:12:03.397837+0000","last_active":"2023-11-28T11:12:03.810573+0000","last_peered":"2023-11-28T11:12:03.810573+0000","last_clean":"2023-11-28T11:12:03.810573+0000","last_became_active":"2023-11-28T11:12:03.397522+0000","last_became_peered":"2023-11-28T11:12:03.397522+0000","last_unstale":"2023-11-28T11:12:03.810573+0000","last_undegraded":"2023-11-28T11:12:03.810573+0000","last_fullsized":"2023-11-28T11:12:03.810573+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:15:29.533047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348209+0000","last_change":"2023-11-28T11:12:03.821866+0000","last_active":"2023-11-28T11:12:05.348209+0000","last_peered":"2023-11-28T11:12:05.348209+0000","last_clean":"2023-11-28T11:12:05.348209+0000","last_became_active":"2023-11-28T11:12:03.821489+0000","last_became_peered":"2023-11-28T11:12:03.821489+0000","last_unstale":"2023-11-28T11:12:05.348209+0000","last_undegraded":"2023-11-28T11:12:05.348209+0000","last_fullsized":"2023-11-28T11:12:05.348209+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:46.318560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"65'8","reported_seq":19,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.842152+0000","last_change":"2023-11-28T11:12:03.404983+0000","last_active":"2023-11-28T11:12:03.842152+0000","last_peered":"2023-11-28T11:12:03.842152+0000","last_clean":"2023-11-28T11:12:03.842152+0000","last_became_active":"2023-11-28T11:12:03.404687+0000","last_became_peered":"2023-11-28T11:12:03.404687+0000","last_unstale":"2023-11-28T11:12:03.842152+0000","last_undegraded":"2023-11-28T11:12:03.842152+0000","last_fullsized":"2023-11-28T11:12:03.842152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:29.499469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.165224+0000","last_change":"2023-11-28T11:12:03.822004+0000","last_active":"2023-11-28T11:12:05.165224+0000","last_peered":"2023-11-28T11:12:05.165224+0000","last_clean":"2023-11-28T11:12:05.165224+0000","last_became_active":"2023-11-28T11:12:03.821732+0000","last_became_peered":"2023-11-28T11:12:03.821732+0000","last_unstale":"2023-11-28T11:12:05.165224+0000","last_undegraded":"2023-11-28T11:12:05.165224+0000","last_fullsized":"2023-11-28T11:12:05.165224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:17.415503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.827498+0000","last_change":"2023-11-28T11:12:03.403409+0000","last_active":"2023-11-28T11:12:03.827498+0000","last_peered":"2023-11-28T11:12:03.827498+0000","last_clean":"2023-11-28T11:12:03.827498+0000","last_became_active":"2023-11-28T11:12:03.403091+0000","last_became_peered":"2023-11-28T11:12:03.403091+0000","last_unstale":"2023-11-28T11:12:03.827498+0000","last_undegraded":"2023-11-28T11:12:03.827498+0000","last_fullsized":"2023-11-28T11:12:03.827498+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:06.501050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.811240+0000","last_change":"2023-11-28T11:12:03.818491+0000","last_active":"2023-11-28T11:12:04.811240+0000","last_peered":"2023-11-28T11:12:04.811240+0000","last_clean":"2023-11-28T11:12:04.811240+0000","last_became_active":"2023-11-28T11:12:03.818109+0000","last_became_peered":"2023-11-28T11:12:03.818109+0000","last_unstale":"2023-11-28T11:12:04.811240+0000","last_undegraded":"2023-11-28T11:12:04.811240+0000","last_fullsized":"2023-11-28T11:12:04.811240+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:15.857309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.811650+0000","last_change":"2023-11-28T11:12:03.397826+0000","last_active":"2023-11-28T11:12:03.811650+0000","last_peered":"2023-11-28T11:12:03.811650+0000","last_clean":"2023-11-28T11:12:03.811650+0000","last_became_active":"2023-11-28T11:12:03.397512+0000","last_became_peered":"2023-11-28T11:12:03.397512+0000","last_unstale":"2023-11-28T11:12:03.811650+0000","last_undegraded":"2023-11-28T11:12:03.811650+0000","last_fullsized":"2023-11-28T11:12:03.811650+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:26.097220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115266+0000","last_change":"2023-11-28T11:12:03.832920+0000","last_active":"2023-11-28T11:12:06.115266+0000","last_peered":"2023-11-28T11:12:06.115266+0000","last_clean":"2023-11-28T11:12:06.115266+0000","last_became_active":"2023-11-28T11:12:03.832618+0000","last_became_peered":"2023-11-28T11:12:03.832618+0000","last_unstale":"2023-11-28T11:12:06.115266+0000","last_undegraded":"2023-11-28T11:12:06.115266+0000","last_fullsized":"2023-11-28T11:12:06.115266+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:46.824986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.815405+0000","last_change":"2023-11-28T11:12:03.400853+0000","last_active":"2023-11-28T11:12:03.815405+0000","last_peered":"2023-11-28T11:12:03.815405+0000","last_clean":"2023-11-28T11:12:03.815405+0000","last_became_active":"2023-11-28T11:12:03.400559+0000","last_became_peered":"2023-11-28T11:12:03.400559+0000","last_unstale":"2023-11-28T11:12:03.815405+0000","last_undegraded":"2023-11-28T11:12:03.815405+0000","last_fullsized":"2023-11-28T11:12:03.815405+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:13:41.696052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115531+0000","last_change":"2023-11-28T11:12:03.835954+0000","last_active":"2023-11-28T11:12:06.115531+0000","last_peered":"2023-11-28T11:12:06.115531+0000","last_clean":"2023-11-28T11:12:06.115531+0000","last_became_active":"2023-11-28T11:12:03.835554+0000","last_became_peered":"2023-11-28T11:12:03.835554+0000","last_unstale":"2023-11-28T11:12:06.115531+0000","last_undegraded":"2023-11-28T11:12:06.115531+0000","last_fullsized":"2023-11-28T11:12:06.115531+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:37.484325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.804869+0000","last_change":"2023-11-28T11:12:03.394462+0000","last_active":"2023-11-28T11:12:03.804869+0000","last_peered":"2023-11-28T11:12:03.804869+0000","last_clean":"2023-11-28T11:12:03.804869+0000","last_became_active":"2023-11-28T11:12:03.394176+0000","last_became_peered":"2023-11-28T11:12:03.394176+0000","last_unstale":"2023-11-28T11:12:03.804869+0000","last_undegraded":"2023-11-28T11:12:03.804869+0000","last_fullsized":"2023-11-28T11:12:03.804869+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:53.341867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348692+0000","last_change":"2023-11-28T11:12:03.818464+0000","last_active":"2023-11-28T11:12:05.348692+0000","last_peered":"2023-11-28T11:12:05.348692+0000","last_clean":"2023-11-28T11:12:05.348692+0000","last_became_active":"2023-11-28T11:12:03.818173+0000","last_became_peered":"2023-11-28T11:12:03.818173+0000","last_unstale":"2023-11-28T11:12:05.348692+0000","last_undegraded":"2023-11-28T11:12:05.348692+0000","last_fullsized":"2023-11-28T11:12:05.348692+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:10.047361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.814457+0000","last_change":"2023-11-28T11:12:03.401986+0000","last_active":"2023-11-28T11:12:03.814457+0000","last_peered":"2023-11-28T11:12:03.814457+0000","last_clean":"2023-11-28T11:12:03.814457+0000","last_became_active":"2023-11-28T11:12:03.401694+0000","last_became_peered":"2023-11-28T11:12:03.401694+0000","last_unstale":"2023-11-28T11:12:03.814457+0000","last_undegraded":"2023-11-28T11:12:03.814457+0000","last_fullsized":"2023-11-28T11:12:03.814457+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:33:57.028903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.202108+0000","last_change":"2023-11-28T11:12:03.828100+0000","last_active":"2023-11-28T11:12:05.202108+0000","last_peered":"2023-11-28T11:12:05.202108+0000","last_clean":"2023-11-28T11:12:05.202108+0000","last_became_active":"2023-11-28T11:12:03.827844+0000","last_became_peered":"2023-11-28T11:12:03.827844+0000","last_unstale":"2023-11-28T11:12:05.202108+0000","last_undegraded":"2023-11-28T11:12:05.202108+0000","last_fullsized":"2023-11-28T11:12:05.202108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:31:32.268605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.807427+0000","last_change":"2023-11-28T11:12:03.389789+0000","last_active":"2023-11-28T11:12:03.807427+0000","last_peered":"2023-11-28T11:12:03.807427+0000","last_clean":"2023-11-28T11:12:03.807427+0000","last_became_active":"2023-11-28T11:12:03.389497+0000","last_became_peered":"2023-11-28T11:12:03.389497+0000","last_unstale":"2023-11-28T11:12:03.807427+0000","last_undegraded":"2023-11-28T11:12:03.807427+0000","last_fullsized":"2023-11-28T11:12:03.807427+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:06.501050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.809722+0000","last_change":"2023-11-28T11:12:03.816789+0000","last_active":"2023-11-28T11:12:04.809722+0000","last_peered":"2023-11-28T11:12:04.809722+0000","last_clean":"2023-11-28T11:12:04.809722+0000","last_became_active":"2023-11-28T11:12:03.816210+0000","last_became_peered":"2023-11-28T11:12:03.816210+0000","last_unstale":"2023-11-28T11:12:04.809722+0000","last_undegraded":"2023-11-28T11:12:04.809722+0000","last_fullsized":"2023-11-28T11:12:04.809722+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:44.185700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.809965+0000","last_change":"2023-11-28T11:12:03.404990+0000","last_active":"2023-11-28T11:12:03.809965+0000","last_peered":"2023-11-28T11:12:03.809965+0000","last_clean":"2023-11-28T11:12:03.809965+0000","last_became_active":"2023-11-28T11:12:03.404699+0000","last_became_peered":"2023-11-28T11:12:03.404699+0000","last_unstale":"2023-11-28T11:12:03.809965+0000","last_undegraded":"2023-11-28T11:12:03.809965+0000","last_fullsized":"2023-11-28T11:12:03.809965+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:39.829848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.201898+0000","last_change":"2023-11-28T11:12:03.820212+0000","last_active":"2023-11-28T11:12:05.201898+0000","last_peered":"2023-11-28T11:12:05.201898+0000","last_clean":"2023-11-28T11:12:05.201898+0000","last_became_active":"2023-11-28T11:12:03.819889+0000","last_became_peered":"2023-11-28T11:12:03.819889+0000","last_unstale":"2023-11-28T11:12:05.201898+0000","last_undegraded":"2023-11-28T11:12:05.201898+0000","last_fullsized":"2023-11-28T11:12:05.201898+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:39.132766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.815907+0000","last_change":"2023-11-28T11:12:03.405308+0000","last_active":"2023-11-28T11:12:03.815907+0000","last_peered":"2023-11-28T11:12:03.815907+0000","last_clean":"2023-11-28T11:12:03.815907+0000","last_became_active":"2023-11-28T11:12:03.403597+0000","last_became_peered":"2023-11-28T11:12:03.403597+0000","last_unstale":"2023-11-28T11:12:03.815907+0000","last_undegraded":"2023-11-28T11:12:03.815907+0000","last_fullsized":"2023-11-28T11:12:03.815907+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:17:37.393448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115749+0000","last_change":"2023-11-28T11:12:03.833573+0000","last_active":"2023-11-28T11:12:06.115749+0000","last_peered":"2023-11-28T11:12:06.115749+0000","last_clean":"2023-11-28T11:12:06.115749+0000","last_became_active":"2023-11-28T11:12:03.831031+0000","last_became_peered":"2023-11-28T11:12:03.831031+0000","last_unstale":"2023-11-28T11:12:06.115749+0000","last_undegraded":"2023-11-28T11:12:06.115749+0000","last_fullsized":"2023-11-28T11:12:06.115749+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:54.346774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.840502+0000","last_change":"2023-11-28T11:12:03.404206+0000","last_active":"2023-11-28T11:12:03.840502+0000","last_peered":"2023-11-28T11:12:03.840502+0000","last_clean":"2023-11-28T11:12:03.840502+0000","last_became_active":"2023-11-28T11:12:03.403925+0000","last_became_peered":"2023-11-28T11:12:03.403925+0000","last_unstale":"2023-11-28T11:12:03.840502+0000","last_undegraded":"2023-11-28T11:12:03.840502+0000","last_fullsized":"2023-11-28T11:12:03.840502+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:04:35.390206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.814813+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:12:03.814813+0000","last_peered":"2023-11-28T11:12:03.814813+0000","last_clean":"2023-11-28T11:12:03.814813+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:12:03.814813+0000","last_undegraded":"2023-11-28T11:12:03.814813+0000","last_fullsized":"2023-11-28T11:12:03.814813+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115703+0000","last_change":"2023-11-28T11:12:03.833188+0000","last_active":"2023-11-28T11:12:06.115703+0000","last_peered":"2023-11-28T11:12:06.115703+0000","last_clean":"2023-11-28T11:12:06.115703+0000","last_became_active":"2023-11-28T11:12:03.824023+0000","last_became_peered":"2023-11-28T11:12:03.824023+0000","last_unstale":"2023-11-28T11:12:06.115703+0000","last_undegraded":"2023-11-28T11:12:06.115703+0000","last_fullsized":"2023-11-28T11:12:06.115703+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:42.823746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.829562+0000","last_change":"2023-11-28T11:12:03.403355+0000","last_active":"2023-11-28T11:12:03.829562+0000","last_peered":"2023-11-28T11:12:03.829562+0000","last_clean":"2023-11-28T11:12:03.829562+0000","last_became_active":"2023-11-28T11:12:03.402998+0000","last_became_peered":"2023-11-28T11:12:03.402998+0000","last_unstale":"2023-11-28T11:12:03.829562+0000","last_undegraded":"2023-11-28T11:12:03.829562+0000","last_fullsized":"2023-11-28T11:12:03.829562+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:45.516962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.810343+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:12:03.810343+0000","last_peered":"2023-11-28T11:12:03.810343+0000","last_clean":"2023-11-28T11:12:03.810343+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:12:03.810343+0000","last_undegraded":"2023-11-28T11:12:03.810343+0000","last_fullsized":"2023-11-28T11:12:03.810343+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114167+0000","last_change":"2023-11-28T11:12:03.823050+0000","last_active":"2023-11-28T11:12:06.114167+0000","last_peered":"2023-11-28T11:12:06.114167+0000","last_clean":"2023-11-28T11:12:06.114167+0000","last_became_active":"2023-11-28T11:12:03.822548+0000","last_became_peered":"2023-11-28T11:12:03.822548+0000","last_unstale":"2023-11-28T11:12:06.114167+0000","last_undegraded":"2023-11-28T11:12:06.114167+0000","last_fullsized":"2023-11-28T11:12:06.114167+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:40.599096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.804310+0000","last_change":"2023-11-28T11:12:03.396696+0000","last_active":"2023-11-28T11:12:03.804310+0000","last_peered":"2023-11-28T11:12:03.804310+0000","last_clean":"2023-11-28T11:12:03.804310+0000","last_became_active":"2023-11-28T11:12:03.396404+0000","last_became_peered":"2023-11-28T11:12:03.396404+0000","last_unstale":"2023-11-28T11:12:03.804310+0000","last_undegraded":"2023-11-28T11:12:03.804310+0000","last_fullsized":"2023-11-28T11:12:03.804310+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:45.853390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.807693+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:12:03.807693+0000","last_peered":"2023-11-28T11:12:03.807693+0000","last_clean":"2023-11-28T11:12:03.807693+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:12:03.807693+0000","last_undegraded":"2023-11-28T11:12:03.807693+0000","last_fullsized":"2023-11-28T11:12:03.807693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":202,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.817618+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:12:03.817618+0000","last_peered":"2023-11-28T11:12:03.817618+0000","last_clean":"2023-11-28T11:12:03.817618+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:12:03.817618+0000","last_undegraded":"2023-11-28T11:12:03.817618+0000","last_fullsized":"2023-11-28T11:12:03.817618+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114128+0000","last_change":"2023-11-28T11:12:03.820805+0000","last_active":"2023-11-28T11:12:06.114128+0000","last_peered":"2023-11-28T11:12:06.114128+0000","last_clean":"2023-11-28T11:12:06.114128+0000","last_became_active":"2023-11-28T11:12:03.820554+0000","last_became_peered":"2023-11-28T11:12:03.820554+0000","last_unstale":"2023-11-28T11:12:06.114128+0000","last_undegraded":"2023-11-28T11:12:06.114128+0000","last_fullsized":"2023-11-28T11:12:06.114128+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:13.470501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.810690+0000","last_change":"2023-11-28T11:12:03.395950+0000","last_active":"2023-11-28T11:12:03.810690+0000","last_peered":"2023-11-28T11:12:03.810690+0000","last_clean":"2023-11-28T11:12:03.810690+0000","last_became_active":"2023-11-28T11:12:03.395628+0000","last_became_peered":"2023-11-28T11:12:03.395628+0000","last_unstale":"2023-11-28T11:12:03.810690+0000","last_undegraded":"2023-11-28T11:12:03.810690+0000","last_fullsized":"2023-11-28T11:12:03.810690+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:18.625956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.819385+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:12:03.819385+0000","last_peered":"2023-11-28T11:12:03.819385+0000","last_clean":"2023-11-28T11:12:03.819385+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:12:03.819385+0000","last_undegraded":"2023-11-28T11:12:03.819385+0000","last_fullsized":"2023-11-28T11:12:03.819385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114181+0000","last_change":"2023-11-28T11:12:03.828476+0000","last_active":"2023-11-28T11:12:06.114181+0000","last_peered":"2023-11-28T11:12:06.114181+0000","last_clean":"2023-11-28T11:12:06.114181+0000","last_became_active":"2023-11-28T11:12:03.828202+0000","last_became_peered":"2023-11-28T11:12:03.828202+0000","last_unstale":"2023-11-28T11:12:06.114181+0000","last_undegraded":"2023-11-28T11:12:06.114181+0000","last_fullsized":"2023-11-28T11:12:06.114181+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:39:17.000938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.829861+0000","last_change":"2023-11-28T11:12:03.402458+0000","last_active":"2023-11-28T11:12:03.829861+0000","last_peered":"2023-11-28T11:12:03.829861+0000","last_clean":"2023-11-28T11:12:03.829861+0000","last_became_active":"2023-11-28T11:12:03.402102+0000","last_became_peered":"2023-11-28T11:12:03.402102+0000","last_unstale":"2023-11-28T11:12:03.829861+0000","last_undegraded":"2023-11-28T11:12:03.829861+0000","last_fullsized":"2023-11-28T11:12:03.829861+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:04.830939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.817499+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:12:03.817499+0000","last_peered":"2023-11-28T11:12:03.817499+0000","last_clean":"2023-11-28T11:12:03.817499+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:12:03.817499+0000","last_undegraded":"2023-11-28T11:12:03.817499+0000","last_fullsized":"2023-11-28T11:12:03.817499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114246+0000","last_change":"2023-11-28T11:12:03.812718+0000","last_active":"2023-11-28T11:12:06.114246+0000","last_peered":"2023-11-28T11:12:06.114246+0000","last_clean":"2023-11-28T11:12:06.114246+0000","last_became_active":"2023-11-28T11:12:03.812476+0000","last_became_peered":"2023-11-28T11:12:03.812476+0000","last_unstale":"2023-11-28T11:12:06.114246+0000","last_undegraded":"2023-11-28T11:12:06.114246+0000","last_fullsized":"2023-11-28T11:12:06.114246+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:57:06.522652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.816802+0000","last_change":"2023-11-28T11:12:03.407497+0000","last_active":"2023-11-28T11:12:03.816802+0000","last_peered":"2023-11-28T11:12:03.816802+0000","last_clean":"2023-11-28T11:12:03.816802+0000","last_became_active":"2023-11-28T11:12:03.407175+0000","last_became_peered":"2023-11-28T11:12:03.407175+0000","last_unstale":"2023-11-28T11:12:03.816802+0000","last_undegraded":"2023-11-28T11:12:03.816802+0000","last_fullsized":"2023-11-28T11:12:03.816802+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:51:23.997889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":57,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.810804+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:12:03.810804+0000","last_peered":"2023-11-28T11:12:03.810804+0000","last_clean":"2023-11-28T11:12:03.810804+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:12:03.810804+0000","last_undegraded":"2023-11-28T11:12:03.810804+0000","last_fullsized":"2023-11-28T11:12:03.810804+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.810076+0000","last_change":"2023-11-28T11:12:03.824796+0000","last_active":"2023-11-28T11:12:04.810076+0000","last_peered":"2023-11-28T11:12:04.810076+0000","last_clean":"2023-11-28T11:12:04.810076+0000","last_became_active":"2023-11-28T11:12:03.824165+0000","last_became_peered":"2023-11-28T11:12:03.824165+0000","last_unstale":"2023-11-28T11:12:04.810076+0000","last_undegraded":"2023-11-28T11:12:04.810076+0000","last_fullsized":"2023-11-28T11:12:04.810076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:53.999259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.841886+0000","last_change":"2023-11-28T11:12:03.402574+0000","last_active":"2023-11-28T11:12:03.841886+0000","last_peered":"2023-11-28T11:12:03.841886+0000","last_clean":"2023-11-28T11:12:03.841886+0000","last_became_active":"2023-11-28T11:12:03.402149+0000","last_became_peered":"2023-11-28T11:12:03.402149+0000","last_unstale":"2023-11-28T11:12:03.841886+0000","last_undegraded":"2023-11-28T11:12:03.841886+0000","last_fullsized":"2023-11-28T11:12:03.841886+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:11.744846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.816369+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:12:03.816369+0000","last_peered":"2023-11-28T11:12:03.816369+0000","last_clean":"2023-11-28T11:12:03.816369+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:12:03.816369+0000","last_undegraded":"2023-11-28T11:12:03.816369+0000","last_fullsized":"2023-11-28T11:12:03.816369+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348557+0000","last_change":"2023-11-28T11:12:03.824750+0000","last_active":"2023-11-28T11:12:05.348557+0000","last_peered":"2023-11-28T11:12:05.348557+0000","last_clean":"2023-11-28T11:12:05.348557+0000","last_became_active":"2023-11-28T11:12:03.824473+0000","last_became_peered":"2023-11-28T11:12:03.824473+0000","last_unstale":"2023-11-28T11:12:05.348557+0000","last_undegraded":"2023-11-28T11:12:05.348557+0000","last_fullsized":"2023-11-28T11:12:05.348557+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:07.505957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.817486+0000","last_change":"2023-11-28T11:12:03.405919+0000","last_active":"2023-11-28T11:12:03.817486+0000","last_peered":"2023-11-28T11:12:03.817486+0000","last_clean":"2023-11-28T11:12:03.817486+0000","last_became_active":"2023-11-28T11:12:03.405638+0000","last_became_peered":"2023-11-28T11:12:03.405638+0000","last_unstale":"2023-11-28T11:12:03.817486+0000","last_undegraded":"2023-11-28T11:12:03.817486+0000","last_fullsized":"2023-11-28T11:12:03.817486+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:26.097220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.816352+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:12:03.816352+0000","last_peered":"2023-11-28T11:12:03.816352+0000","last_clean":"2023-11-28T11:12:03.816352+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:12:03.816352+0000","last_undegraded":"2023-11-28T11:12:03.816352+0000","last_fullsized":"2023-11-28T11:12:03.816352+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10848,"kb_used_data":1368,"kb_used_omap":0,"kb_used_meta":9472,"kb_avail":94360992,"statfs":{"total":96636764160,"available":96625655808,"internally_reserved":0,"allocated":1400832,"data_stored":966096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9699328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.59599999999999997,"15min":0.63900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76700000000000002,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.81899999999999995,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.54200000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.64100000000000001,"15min":0.68000000000000005},"min":{"1min":0.378,"5min":0.316,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.40100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.87,"5min":1.224,"15min":1.224},"last":0.44900000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65500000000000003,"15min":0.68700000000000006},"min":{"1min":0.443,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79500000000000004,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.72599999999999998,"15min":0.77300000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79300000000000004,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.65800000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.46899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83299999999999996,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.72499999999999998,"15min":0.76600000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.81499999999999995,"5min":3.294,"15min":3.294},"last":0.73199999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.63300000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.625},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55600000000000005,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.79200000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.64200000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.621,"15min":0.64000000000000001},"min":{"1min":0.36199999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73599999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63800000000000001,"15min":0.64700000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78000000000000003,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86699999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.628,"15min":0.63600000000000001},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9692,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362148,"statfs":{"total":96636764160,"available":96626839552,"internally_reserved":0,"allocated":806912,"data_stored":375220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.46100000000000002},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.745,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.48999999999999999,"15min":0.47899999999999998},"min":{"1min":0.33700000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.71799999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53100000000000003,"15min":0.51300000000000001},"min":{"1min":0.41099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86399999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.51500000000000001,"15min":0.499},"min":{"1min":0.32600000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.81799999999999995}]},{"osd":2,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.38500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66000000000000003,"15min":0.68100000000000005},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84299999999999997,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.80200000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69199999999999995,"15min":0.71899999999999997},"min":{"1min":0.45500000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81299999999999994,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.753},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68200000000000005,"15min":0.69799999999999995},"min":{"1min":0.41799999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80900000000000005,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.76300000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61899999999999999,"15min":0.628},"min":{"1min":0.40200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90300000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.53900000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.47199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.46500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.61899999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.54500000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93000000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.69699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10796,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":9408,"kb_avail":94361044,"statfs":{"total":96636764160,"available":96625709056,"internally_reserved":0,"allocated":1413120,"data_stored":967644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9633792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33500000000000002,"15min":0.32900000000000001},"min":{"1min":0.20599999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.26900000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.57799999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.25700000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.33100000000000002,"15min":0.32000000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.47299999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59999999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.32800000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.40899999999999997,"15min":0.39300000000000002},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41699999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.60999999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.39300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.014},"last":0.73199999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.45800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.94799999999999995,"5min":0.998,"15min":0.998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65200000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94099999999999995,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.018,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.60599999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.443,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96699999999999997,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61499999999999999,"15min":0.59799999999999998},"min":{"1min":0.48699999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89800000000000002,"5min":1.071,"15min":1.071},"last":0.76000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9876,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361964,"statfs":{"total":96636764160,"available":96626651136,"internally_reserved":0,"allocated":798720,"data_stored":375128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.33200000000000002,"15min":0.32800000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.46999999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.379,"15min":0.373},"min":{"1min":0.32200000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.54300000000000004,"5min":0.622,"15min":0.622},"last":0.437}]},{"osd":2,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.52600000000000002,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.31},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53600000000000003,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.34899999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.371,"15min":0.371},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51400000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.376},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42999999999999999,"15min":0.42799999999999999},"min":{"1min":0.30299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.56699999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.48699999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.42399999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84299999999999997,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.58099999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65800000000000003,"15min":0.622},"min":{"1min":0.497,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90000000000000002,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.622,"15min":0.623},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.86899999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.69299999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.623,"15min":0.623},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88700000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.85999999999999999,"5min":1.016,"15min":1.016},"last":0.66500000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65400000000000003,"15min":0.65300000000000002},"min":{"1min":0.48699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93400000000000005,"5min":1.105,"15min":1.105},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65000000000000002,"15min":0.64300000000000002},"min":{"1min":0.53500000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94299999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9564,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362276,"statfs":{"total":96636764160,"available":96626970624,"internally_reserved":0,"allocated":806912,"data_stored":375715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.373},"min":{"1min":0.23300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.38500000000000001,"15min":0.38900000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61699999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42199999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64300000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.432,"15min":0.44400000000000001},"min":{"1min":0.29299999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60399999999999998,"5min":1.528,"15min":1.528},"last":0.55900000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38900000000000001,"15min":0.39600000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.59399999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.317},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.46700000000000003,"15min":0.47899999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64800000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.45900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81599999999999995,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82799999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51200000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.47899999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88300000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52500000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.62,"15min":0.622},"min":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65900000000000003,"15min":0.66900000000000004},"min":{"1min":0.51400000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86399999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.69599999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.629,"15min":0.63},"min":{"1min":0.52800000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81799999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.47799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9820,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362020,"statfs":{"total":96636764160,"available":96626708480,"internally_reserved":0,"allocated":806912,"data_stored":375240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.71399999999999997,"15min":0.88},"min":{"1min":0.23899999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.53000000000000003,"5min":11.965,"15min":11.965},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.74299999999999999,"15min":0.91500000000000004},"min":{"1min":0.255,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.57399999999999995,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.32900000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.34699999999999998,"15min":0.34000000000000002},"min":{"1min":0.27100000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.44},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.92100000000000004,"15min":1.157},"min":{"1min":0.28499999999999998,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.69499999999999995,"5min":16.988,"15min":16.988},"last":0.34899999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.47799999999999998,"15min":0.5},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64700000000000002,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":1.3149999999999999,"15min":1.7010000000000001},"min":{"1min":0.33800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.629,"5min":27.887,"15min":27.887},"last":0.40000000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67300000000000004,"15min":0.70399999999999996},"min":{"1min":0.46800000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.86499999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.64700000000000002,"15min":0.66500000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89500000000000002,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66500000000000004,"15min":0.68700000000000006},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85099999999999998,"5min":1.98,"15min":1.98},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":1,"15min":1.169},"min":{"1min":0.42799999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.45100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":1.008,"15min":1.177},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82299999999999995,"5min":11.888,"15min":11.888},"last":0.5},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.69599999999999995,"15min":0.72999999999999998},"min":{"1min":0.439,"5min":0.437,"15min":0.437},"max":{"1min":0.80900000000000005,"5min":1.925,"15min":1.925},"last":0.41399999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71399999999999997,"15min":0.74399999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91700000000000004,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.746,"15min":0.77900000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93899999999999995,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.53500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9880,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361960,"statfs":{"total":96636764160,"available":96626647040,"internally_reserved":0,"allocated":802816,"data_stored":375198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75900000000000001,"5min":0.996,"15min":0.996},"last":0.377},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.81699999999999995,"15min":0.93600000000000005},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.752,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.35299999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.40600000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74299999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78700000000000003,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.44,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.746,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.71199999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.56499999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.69199999999999995,"15min":0.746},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.81999999999999995,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.46600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80100000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.53700000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.33700000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73099999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.54700000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.42099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":1.145,"15min":1.145},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.40300000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.747,"5min":1.085,"15min":1.085},"last":0.52700000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.60699999999999998,"15min":0.627},"min":{"1min":0.38500000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70699999999999996,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.504},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.66600000000000004,"15min":0.67800000000000005},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80300000000000005,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9688,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362152,"statfs":{"total":96636764160,"available":96626843648,"internally_reserved":0,"allocated":802816,"data_stored":375218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.503,"15min":0.503},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69899999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.55800000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.82099999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.44800000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.64400000000000002,"15min":0.66600000000000004},"min":{"1min":0.42199999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73399999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.621},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.64000000000000001,"15min":0.66900000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71699999999999997,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71999999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58399999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.63400000000000001,"15min":0.67800000000000005},"min":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78700000000000003,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":1.042,"15min":1.042},"last":0.502}]},{"osd":4,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68100000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.46700000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.73999999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71299999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.47799999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72099999999999997,"5min":1.036,"15min":1.036},"last":0.48199999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66400000000000003,"15min":0.68999999999999995},"min":{"1min":0.39500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.97299999999999998,"5min":1.329,"15min":1.329},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.66600000000000004,"15min":0.68899999999999995},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75800000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.47099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:12:12.524 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:12:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.661+0000 7f0251f68700 1 Processor -- start 2023-11-28T11:12:12.663 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.662+0000 7f0251f68700 1 -- start start 2023-11-28T11:12:12.666 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.662+0000 7f0251f68700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f024c0cc310 0x7f024c0cc770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.662+0000 7f0251f68700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f024c0ce340 con 0x7f024c15c990 2023-11-28T11:12:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.662+0000 7f0251f68700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f024c0cebe0 con 0x7f024c161090 2023-11-28T11:12:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.662+0000 7f0251f68700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f024c0cf780 con 0x7f024c0cc310 2023-11-28T11:12:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024b7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f024c15c990 0x7f024c0cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39718/0 (socket says 172.21.15.28:39718) 2023-11-28T11:12:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024b7fe700 1 -- 172.21.15.28:0/3959515552 learned_addr learned my addr 172.21.15.28:0/3959515552 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024bfff700 1 --2- 172.21.15.28:0/3959515552 >> v2:172.21.15.59:3300/0 conn(0x7f024c0cc310 0x7f024c0cc770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657133681 0 0) 0x7f024c0ce340 con 0x7f024c15c990 2023-11-28T11:12:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f02340035b0 con 0x7f024c15c990 2023-11-28T11:12:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632429324 0 0) 0x7f024c0cebe0 con 0x7f024c161090 2023-11-28T11:12:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f024c0ce340 con 0x7f024c161090 2023-11-28T11:12:12.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3401068175 0 0) 0x7f02340035b0 con 0x7f024c15c990 2023-11-28T11:12:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 >> v2:172.21.15.59:3300/0 conn(0x7f024c0cc310 msgr2=0x7f024c0cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 --2- 172.21.15.28:0/3959515552 >> v2:172.21.15.59:3300/0 conn(0x7f024c0cc310 0x7f024c0cc770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 >> v1:172.21.15.59:6789/0 conn(0x7f024c161090 legacy=0x7f024c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c0cccb0 con 0x7f024c15c990 2023-11-28T11:12:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f023c001670 con 0x7f024c15c990 2023-11-28T11:12:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 >> v1:172.21.15.28:6789/0 conn(0x7f024c15c990 legacy=0x7f024c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 --2- 172.21.15.28:0/3959515552 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0234003b90 0x7f0234003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 --2- 172.21.15.28:0/3959515552 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0234004850 0x7f0234004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 --2- 172.21.15.28:0/3959515552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0234005270 0x7f023400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c0cf780 con 0x7f0234003b90 2023-11-28T11:12:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023400c1b0 con 0x7f0234005270 2023-11-28T11:12:12.668 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f023400c370 con 0x7f0234004850 2023-11-28T11:12:12.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024b7fe700 1 --2- 172.21.15.28:0/3959515552 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0234003b90 0x7f0234003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024bfff700 1 --2- 172.21.15.28:0/3959515552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0234005270 0x7f023400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.663+0000 7f024affd700 1 --2- 172.21.15.28:0/3959515552 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0234004850 0x7f0234004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.664+0000 7f024b7fe700 1 -- 172.21.15.28:0/3959515552 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0234004850 msgr2=0x7f0234004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:12.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.664+0000 7f024b7fe700 1 --2- 172.21.15.28:0/3959515552 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0234004850 0x7f0234004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.664+0000 7f024b7fe700 1 -- 172.21.15.28:0/3959515552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0234005270 msgr2=0x7f023400ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:12.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.664+0000 7f024b7fe700 1 --2- 172.21.15.28:0/3959515552 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0234005270 0x7f023400ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.664+0000 7f024b7fe700 1 -- 172.21.15.28:0/3959515552 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02340035b0 con 0x7f0234003b90 2023-11-28T11:12:12.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.664+0000 7f024bfff700 1 -- 172.21.15.28:0/3959515552 reap_dead start 2023-11-28T11:12:12.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.664+0000 7f024b7fe700 1 --2- 172.21.15.28:0/3959515552 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0234003b90 0x7f0234003f80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f023c00c210 tx=0x7f0234005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad8a9324935cf540 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:12.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.664+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c00cdb0 con 0x7f0234003b90 2023-11-28T11:12:12.669 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.664+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f023c005960 con 0x7f0234003b90 2023-11-28T11:12:12.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.664+0000 7f024a7fc700 1 -- 172.21.15.28:0/3959515552 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f023c005d10 con 0x7f0234003b90 2023-11-28T11:12:12.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.665+0000 7f0251f68700 1 -- 172.21.15.28:0/3959515552 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0234003b90 msgr2=0x7f0234003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.665+0000 7f0251f68700 1 --2- 172.21.15.28:0/3959515552 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0234003b90 0x7f0234003f80 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f023c00c210 tx=0x7f0234005880 comp rx=0 tx=0).stop 2023-11-28T11:12:12.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.665+0000 7f0251f68700 1 -- 172.21.15.28:0/3959515552 shutdown_connections 2023-11-28T11:12:12.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.665+0000 7f0251f68700 1 --2- 172.21.15.28:0/3959515552 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0234003b90 0x7f0234003f80 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.665+0000 7f0251f68700 1 -- 172.21.15.28:0/3959515552 >> 172.21.15.28:0/3959515552 conn(0x7f024c0b1650 msgr2=0x7f024c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:12.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.665+0000 7f0251f68700 1 -- 172.21.15.28:0/3959515552 shutdown_connections 2023-11-28T11:12:12.670 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.665+0000 7f0251f68700 1 -- 172.21.15.28:0/3959515552 wait complete. 2023-11-28T11:12:12.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.666+0000 7f0251f68700 1 Processor -- start 2023-11-28T11:12:12.672 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.666+0000 7f0251f68700 1 -- start start 2023-11-28T11:12:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f0251f68700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0234003b90 0x7f024c0aeb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f0251f68700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f024c1594f0 0x7f024c06b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f0251f68700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f024c158880 0x7f024c158d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f0251f68700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c0c71e0 con 0x7f024c158880 2023-11-28T11:12:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f0251f68700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f024c159260 con 0x7f0234003b90 2023-11-28T11:12:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f0251f68700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f024c1e7ac0 con 0x7f024c1594f0 2023-11-28T11:12:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f024bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f024c158880 0x7f024c158d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f024bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f024c158880 0x7f024c158d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51676/0 (socket says 172.21.15.28:51676) 2023-11-28T11:12:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f024bfff700 1 -- 172.21.15.28:0/2092924235 learned_addr learned my addr 172.21.15.28:0/2092924235 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f024affd700 1 --2- 172.21.15.28:0/2092924235 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f024c1594f0 0x7f024c06b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.673 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f024b7fe700 1 --2- 172.21.15.28:0/2092924235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0234003b90 0x7f024c0aeb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f024bfff700 1 -- 172.21.15.28:0/2092924235 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f024c1594f0 msgr2=0x7f024c06b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f024bfff700 1 --2- 172.21.15.28:0/2092924235 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f024c1594f0 0x7f024c06b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f024bfff700 1 -- 172.21.15.28:0/2092924235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0234003b90 msgr2=0x7f024c0aeb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f024bfff700 1 --2- 172.21.15.28:0/2092924235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0234003b90 0x7f024c0aeb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f024bfff700 1 -- 172.21.15.28:0/2092924235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f024c0ce590 con 0x7f024c158880 2023-11-28T11:12:12.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.667+0000 7f024bfff700 1 --2- 172.21.15.28:0/2092924235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f024c158880 0x7f024c158d20 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f024c06f710 tx=0x7f0244011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:12.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.668+0000 7f0233fff700 1 -- 172.21.15.28:0/2092924235 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024400e040 con 0x7f024c158880 2023-11-28T11:12:12.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.668+0000 7f0233fff700 1 -- 172.21.15.28:0/2092924235 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024400f040 con 0x7f024c158880 2023-11-28T11:12:12.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.668+0000 7f0251f68700 1 -- 172.21.15.28:0/2092924235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f024c0cb3f0 con 0x7f024c158880 2023-11-28T11:12:12.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.668+0000 7f0233fff700 1 -- 172.21.15.28:0/2092924235 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024400fd50 con 0x7f024c158880 2023-11-28T11:12:12.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.668+0000 7f0251f68700 1 -- 172.21.15.28:0/2092924235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f024c0ce870 con 0x7f024c158880 2023-11-28T11:12:12.674 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.669+0000 7f0251f68700 1 -- 172.21.15.28:0/2092924235 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f024c0675a0 con 0x7f024c158880 2023-11-28T11:12:12.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.669+0000 7f0233fff700 1 -- 172.21.15.28:0/2092924235 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0244015430 con 0x7f024c158880 2023-11-28T11:12:12.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.669+0000 7f0233fff700 1 --2- 172.21.15.28:0/2092924235 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0238068320 0x7f023806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:12.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.669+0000 7f0233fff700 1 -- 172.21.15.28:0/2092924235 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f0244086400 con 0x7f024c158880 2023-11-28T11:12:12.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.669+0000 7f024b7fe700 1 --2- 172.21.15.28:0/2092924235 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0238068320 0x7f023806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:12.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.670+0000 7f024b7fe700 1 --2- 172.21.15.28:0/2092924235 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0238068320 0x7f023806a7d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f023c000c00 tx=0x7f023c00a000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:12.675 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.674+0000 7f0233fff700 1 -- 172.21.15.28:0/2092924235 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0244052ac0 con 0x7f024c158880 2023-11-28T11:12:12.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.862+0000 7f0251f68700 1 -- 172.21.15.28:0/2092924235 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f024c0cd900 con 0x7f0238068320 2023-11-28T11:12:12.872 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.871+0000 7f0233fff700 1 -- 172.21.15.28:0/2092924235 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311459 (secure 0 0 0) 0x7f024c0cd900 con 0x7f0238068320 2023-11-28T11:12:12.876 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.874+0000 7f0251f68700 1 -- 172.21.15.28:0/2092924235 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0238068320 msgr2=0x7f023806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.874+0000 7f0251f68700 1 --2- 172.21.15.28:0/2092924235 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0238068320 0x7f023806a7d0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f023c000c00 tx=0x7f023c00a000 comp rx=0 tx=0).stop 2023-11-28T11:12:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.874+0000 7f0251f68700 1 -- 172.21.15.28:0/2092924235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f024c158880 msgr2=0x7f024c158d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.874+0000 7f0251f68700 1 --2- 172.21.15.28:0/2092924235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f024c158880 0x7f024c158d20 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f024c06f710 tx=0x7f0244011f80 comp rx=0 tx=0).stop 2023-11-28T11:12:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.875+0000 7f0251f68700 1 -- 172.21.15.28:0/2092924235 shutdown_connections 2023-11-28T11:12:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.875+0000 7f0251f68700 1 --2- 172.21.15.28:0/2092924235 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0238068320 0x7f023806a7d0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.875+0000 7f0251f68700 1 --2- 172.21.15.28:0/2092924235 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f024c158880 0x7f024c158d20 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.875+0000 7f0251f68700 1 --2- 172.21.15.28:0/2092924235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0234003b90 0x7f024c0aeb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.875+0000 7f0251f68700 1 --2- 172.21.15.28:0/2092924235 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f024c1594f0 0x7f024c06b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.875+0000 7f0251f68700 1 -- 172.21.15.28:0/2092924235 >> 172.21.15.28:0/2092924235 conn(0x7f024c0b1650 msgr2=0x7f024c15b4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.875+0000 7f0251f68700 1 -- 172.21.15.28:0/2092924235 shutdown_connections 2023-11-28T11:12:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:12.875+0000 7f0251f68700 1 -- 172.21.15.28:0/2092924235 wait complete. 2023-11-28T11:12:12.880 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:12:12.889 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2023-11-28T11:12:11.209321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592698,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":200,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":80164,"kb_used_data":7460,"kb_used_omap":0,"kb_used_meta":72640,"kb_avail":754894556,"statfs":{"total":773094113280,"available":773012025344,"internally_reserved":0,"allocated":7639040,"data_stored":4185459,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":74383360},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.828088"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115820+0000","last_change":"2023-11-28T11:12:03.835618+0000","last_active":"2023-11-28T11:12:06.115820+0000","last_peered":"2023-11-28T11:12:06.115820+0000","last_clean":"2023-11-28T11:12:06.115820+0000","last_became_active":"2023-11-28T11:12:03.833256+0000","last_became_peered":"2023-11-28T11:12:03.833256+0000","last_unstale":"2023-11-28T11:12:06.115820+0000","last_undegraded":"2023-11-28T11:12:06.115820+0000","last_fullsized":"2023-11-28T11:12:06.115820+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:59.308020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.202640+0000","last_change":"2023-11-28T11:12:03.841405+0000","last_active":"2023-11-28T11:12:05.202640+0000","last_peered":"2023-11-28T11:12:05.202640+0000","last_clean":"2023-11-28T11:12:05.202640+0000","last_became_active":"2023-11-28T11:12:03.841128+0000","last_became_peered":"2023-11-28T11:12:03.841128+0000","last_unstale":"2023-11-28T11:12:05.202640+0000","last_undegraded":"2023-11-28T11:12:05.202640+0000","last_fullsized":"2023-11-28T11:12:05.202640+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:18.904871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114245+0000","last_change":"2023-11-28T11:12:03.816160+0000","last_active":"2023-11-28T11:12:06.114245+0000","last_peered":"2023-11-28T11:12:06.114245+0000","last_clean":"2023-11-28T11:12:06.114245+0000","last_became_active":"2023-11-28T11:12:03.815913+0000","last_became_peered":"2023-11-28T11:12:03.815913+0000","last_unstale":"2023-11-28T11:12:06.114245+0000","last_undegraded":"2023-11-28T11:12:06.114245+0000","last_fullsized":"2023-11-28T11:12:06.114245+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:58:32.223293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.809974+0000","last_change":"2023-11-28T11:12:03.828381+0000","last_active":"2023-11-28T11:12:04.809974+0000","last_peered":"2023-11-28T11:12:04.809974+0000","last_clean":"2023-11-28T11:12:04.809974+0000","last_became_active":"2023-11-28T11:12:03.827787+0000","last_became_peered":"2023-11-28T11:12:03.827787+0000","last_unstale":"2023-11-28T11:12:04.809974+0000","last_undegraded":"2023-11-28T11:12:04.809974+0000","last_fullsized":"2023-11-28T11:12:04.809974+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:39.132766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348245+0000","last_change":"2023-11-28T11:12:03.830513+0000","last_active":"2023-11-28T11:12:05.348245+0000","last_peered":"2023-11-28T11:12:05.348245+0000","last_clean":"2023-11-28T11:12:05.348245+0000","last_became_active":"2023-11-28T11:12:03.830213+0000","last_became_peered":"2023-11-28T11:12:03.830213+0000","last_unstale":"2023-11-28T11:12:05.348245+0000","last_undegraded":"2023-11-28T11:12:05.348245+0000","last_fullsized":"2023-11-28T11:12:05.348245+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:38:19.707029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.201989+0000","last_change":"2023-11-28T11:12:03.841348+0000","last_active":"2023-11-28T11:12:05.201989+0000","last_peered":"2023-11-28T11:12:05.201989+0000","last_clean":"2023-11-28T11:12:05.201989+0000","last_became_active":"2023-11-28T11:12:03.841037+0000","last_became_peered":"2023-11-28T11:12:03.841037+0000","last_unstale":"2023-11-28T11:12:05.201989+0000","last_undegraded":"2023-11-28T11:12:05.201989+0000","last_fullsized":"2023-11-28T11:12:05.201989+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:30.504376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.810853+0000","last_change":"2023-11-28T11:12:03.816285+0000","last_active":"2023-11-28T11:12:04.810853+0000","last_peered":"2023-11-28T11:12:04.810853+0000","last_clean":"2023-11-28T11:12:04.810853+0000","last_became_active":"2023-11-28T11:12:03.815963+0000","last_became_peered":"2023-11-28T11:12:03.815963+0000","last_unstale":"2023-11-28T11:12:04.810853+0000","last_undegraded":"2023-11-28T11:12:04.810853+0000","last_fullsized":"2023-11-28T11:12:04.810853+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:56:09.091995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.201966+0000","last_change":"2023-11-28T11:12:03.830920+0000","last_active":"2023-11-28T11:12:05.201966+0000","last_peered":"2023-11-28T11:12:05.201966+0000","last_clean":"2023-11-28T11:12:05.201966+0000","last_became_active":"2023-11-28T11:12:03.830633+0000","last_became_peered":"2023-11-28T11:12:03.830633+0000","last_unstale":"2023-11-28T11:12:05.201966+0000","last_undegraded":"2023-11-28T11:12:05.201966+0000","last_fullsized":"2023-11-28T11:12:05.201966+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:45:02.573643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348500+0000","last_change":"2023-11-28T11:12:03.821337+0000","last_active":"2023-11-28T11:12:05.348500+0000","last_peered":"2023-11-28T11:12:05.348500+0000","last_clean":"2023-11-28T11:12:05.348500+0000","last_became_active":"2023-11-28T11:12:03.820702+0000","last_became_peered":"2023-11-28T11:12:03.820702+0000","last_unstale":"2023-11-28T11:12:05.348500+0000","last_undegraded":"2023-11-28T11:12:05.348500+0000","last_fullsized":"2023-11-28T11:12:05.348500+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:05.835846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115635+0000","last_change":"2023-11-28T11:12:03.823397+0000","last_active":"2023-11-28T11:12:06.115635+0000","last_peered":"2023-11-28T11:12:06.115635+0000","last_clean":"2023-11-28T11:12:06.115635+0000","last_became_active":"2023-11-28T11:12:03.823037+0000","last_became_peered":"2023-11-28T11:12:03.823037+0000","last_unstale":"2023-11-28T11:12:06.115635+0000","last_undegraded":"2023-11-28T11:12:06.115635+0000","last_fullsized":"2023-11-28T11:12:06.115635+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:33.447204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115464+0000","last_change":"2023-11-28T11:12:03.830195+0000","last_active":"2023-11-28T11:12:06.115464+0000","last_peered":"2023-11-28T11:12:06.115464+0000","last_clean":"2023-11-28T11:12:06.115464+0000","last_became_active":"2023-11-28T11:12:03.829670+0000","last_became_peered":"2023-11-28T11:12:03.829670+0000","last_unstale":"2023-11-28T11:12:06.115464+0000","last_undegraded":"2023-11-28T11:12:06.115464+0000","last_fullsized":"2023-11-28T11:12:06.115464+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:32:23.420139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.810029+0000","last_change":"2023-11-28T11:12:03.824974+0000","last_active":"2023-11-28T11:12:04.810029+0000","last_peered":"2023-11-28T11:12:04.810029+0000","last_clean":"2023-11-28T11:12:04.810029+0000","last_became_active":"2023-11-28T11:12:03.824539+0000","last_became_peered":"2023-11-28T11:12:03.824539+0000","last_unstale":"2023-11-28T11:12:04.810029+0000","last_undegraded":"2023-11-28T11:12:04.810029+0000","last_fullsized":"2023-11-28T11:12:04.810029+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:02:17.662708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.165156+0000","last_change":"2023-11-28T11:12:03.821038+0000","last_active":"2023-11-28T11:12:05.165156+0000","last_peered":"2023-11-28T11:12:05.165156+0000","last_clean":"2023-11-28T11:12:05.165156+0000","last_became_active":"2023-11-28T11:12:03.820762+0000","last_became_peered":"2023-11-28T11:12:03.820762+0000","last_unstale":"2023-11-28T11:12:05.165156+0000","last_undegraded":"2023-11-28T11:12:05.165156+0000","last_fullsized":"2023-11-28T11:12:05.165156+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:25:33.025411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348295+0000","last_change":"2023-11-28T11:12:03.821353+0000","last_active":"2023-11-28T11:12:05.348295+0000","last_peered":"2023-11-28T11:12:05.348295+0000","last_clean":"2023-11-28T11:12:05.348295+0000","last_became_active":"2023-11-28T11:12:03.821076+0000","last_became_peered":"2023-11-28T11:12:03.821076+0000","last_unstale":"2023-11-28T11:12:05.348295+0000","last_undegraded":"2023-11-28T11:12:05.348295+0000","last_fullsized":"2023-11-28T11:12:05.348295+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:50.253755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.164947+0000","last_change":"2023-11-28T11:12:03.822735+0000","last_active":"2023-11-28T11:12:05.164947+0000","last_peered":"2023-11-28T11:12:05.164947+0000","last_clean":"2023-11-28T11:12:05.164947+0000","last_became_active":"2023-11-28T11:12:03.822440+0000","last_became_peered":"2023-11-28T11:12:03.822440+0000","last_unstale":"2023-11-28T11:12:05.164947+0000","last_undegraded":"2023-11-28T11:12:05.164947+0000","last_fullsized":"2023-11-28T11:12:05.164947+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:47:16.770348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114756+0000","last_change":"2023-11-28T11:12:03.832886+0000","last_active":"2023-11-28T11:12:06.114756+0000","last_peered":"2023-11-28T11:12:06.114756+0000","last_clean":"2023-11-28T11:12:06.114756+0000","last_became_active":"2023-11-28T11:12:03.832554+0000","last_became_peered":"2023-11-28T11:12:03.832554+0000","last_unstale":"2023-11-28T11:12:06.114756+0000","last_undegraded":"2023-11-28T11:12:06.114756+0000","last_fullsized":"2023-11-28T11:12:06.114756+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:31:01.111914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348633+0000","last_change":"2023-11-28T11:12:03.821113+0000","last_active":"2023-11-28T11:12:05.348633+0000","last_peered":"2023-11-28T11:12:05.348633+0000","last_clean":"2023-11-28T11:12:05.348633+0000","last_became_active":"2023-11-28T11:12:03.820374+0000","last_became_peered":"2023-11-28T11:12:03.820374+0000","last_unstale":"2023-11-28T11:12:05.348633+0000","last_undegraded":"2023-11-28T11:12:05.348633+0000","last_fullsized":"2023-11-28T11:12:05.348633+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:17:38.398355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.810095+0000","last_change":"2023-11-28T11:12:03.828247+0000","last_active":"2023-11-28T11:12:04.810095+0000","last_peered":"2023-11-28T11:12:04.810095+0000","last_clean":"2023-11-28T11:12:04.810095+0000","last_became_active":"2023-11-28T11:12:03.827965+0000","last_became_peered":"2023-11-28T11:12:03.827965+0000","last_unstale":"2023-11-28T11:12:04.810095+0000","last_undegraded":"2023-11-28T11:12:04.810095+0000","last_fullsized":"2023-11-28T11:12:04.810095+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:04:07.059900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.202300+0000","last_change":"2023-11-28T11:12:03.830360+0000","last_active":"2023-11-28T11:12:05.202300+0000","last_peered":"2023-11-28T11:12:05.202300+0000","last_clean":"2023-11-28T11:12:05.202300+0000","last_became_active":"2023-11-28T11:12:03.829992+0000","last_became_peered":"2023-11-28T11:12:03.829992+0000","last_unstale":"2023-11-28T11:12:05.202300+0000","last_undegraded":"2023-11-28T11:12:05.202300+0000","last_fullsized":"2023-11-28T11:12:05.202300+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:39:33.773688+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.165118+0000","last_change":"2023-11-28T11:12:03.819529+0000","last_active":"2023-11-28T11:12:05.165118+0000","last_peered":"2023-11-28T11:12:05.165118+0000","last_clean":"2023-11-28T11:12:05.165118+0000","last_became_active":"2023-11-28T11:12:03.819245+0000","last_became_peered":"2023-11-28T11:12:03.819245+0000","last_unstale":"2023-11-28T11:12:05.165118+0000","last_undegraded":"2023-11-28T11:12:05.165118+0000","last_fullsized":"2023-11-28T11:12:05.165118+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:22:37.131304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115400+0000","last_change":"2023-11-28T11:12:03.823568+0000","last_active":"2023-11-28T11:12:06.115400+0000","last_peered":"2023-11-28T11:12:06.115400+0000","last_clean":"2023-11-28T11:12:06.115400+0000","last_became_active":"2023-11-28T11:12:03.823247+0000","last_became_peered":"2023-11-28T11:12:03.823247+0000","last_unstale":"2023-11-28T11:12:06.115400+0000","last_undegraded":"2023-11-28T11:12:06.115400+0000","last_fullsized":"2023-11-28T11:12:06.115400+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:48:55.831494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115050+0000","last_change":"2023-11-28T11:12:03.823483+0000","last_active":"2023-11-28T11:12:06.115050+0000","last_peered":"2023-11-28T11:12:06.115050+0000","last_clean":"2023-11-28T11:12:06.115050+0000","last_became_active":"2023-11-28T11:12:03.822785+0000","last_became_peered":"2023-11-28T11:12:03.822785+0000","last_unstale":"2023-11-28T11:12:06.115050+0000","last_undegraded":"2023-11-28T11:12:06.115050+0000","last_fullsized":"2023-11-28T11:12:06.115050+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:46.824986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115031+0000","last_change":"2023-11-28T11:12:03.831161+0000","last_active":"2023-11-28T11:12:06.115031+0000","last_peered":"2023-11-28T11:12:06.115031+0000","last_clean":"2023-11-28T11:12:06.115031+0000","last_became_active":"2023-11-28T11:12:03.830869+0000","last_became_peered":"2023-11-28T11:12:03.830869+0000","last_unstale":"2023-11-28T11:12:06.115031+0000","last_undegraded":"2023-11-28T11:12:06.115031+0000","last_fullsized":"2023-11-28T11:12:06.115031+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:03:44.943498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115029+0000","last_change":"2023-11-28T11:12:03.833613+0000","last_active":"2023-11-28T11:12:06.115029+0000","last_peered":"2023-11-28T11:12:06.115029+0000","last_clean":"2023-11-28T11:12:06.115029+0000","last_became_active":"2023-11-28T11:12:03.832946+0000","last_became_peered":"2023-11-28T11:12:03.832946+0000","last_unstale":"2023-11-28T11:12:06.115029+0000","last_undegraded":"2023-11-28T11:12:06.115029+0000","last_fullsized":"2023-11-28T11:12:06.115029+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:33.447204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115452+0000","last_change":"2023-11-28T11:12:03.831302+0000","last_active":"2023-11-28T11:12:06.115452+0000","last_peered":"2023-11-28T11:12:06.115452+0000","last_clean":"2023-11-28T11:12:06.115452+0000","last_became_active":"2023-11-28T11:12:03.831032+0000","last_became_peered":"2023-11-28T11:12:03.831032+0000","last_unstale":"2023-11-28T11:12:06.115452+0000","last_undegraded":"2023-11-28T11:12:06.115452+0000","last_fullsized":"2023-11-28T11:12:06.115452+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:10:59.308020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.811227+0000","last_change":"2023-11-28T11:12:03.832151+0000","last_active":"2023-11-28T11:12:04.811227+0000","last_peered":"2023-11-28T11:12:04.811227+0000","last_clean":"2023-11-28T11:12:04.811227+0000","last_became_active":"2023-11-28T11:12:03.831816+0000","last_became_peered":"2023-11-28T11:12:03.831816+0000","last_unstale":"2023-11-28T11:12:04.811227+0000","last_undegraded":"2023-11-28T11:12:04.811227+0000","last_fullsized":"2023-11-28T11:12:04.811227+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:54.346774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114295+0000","last_change":"2023-11-28T11:12:03.826789+0000","last_active":"2023-11-28T11:12:06.114295+0000","last_peered":"2023-11-28T11:12:06.114295+0000","last_clean":"2023-11-28T11:12:06.114295+0000","last_became_active":"2023-11-28T11:12:03.826520+0000","last_became_peered":"2023-11-28T11:12:03.826520+0000","last_unstale":"2023-11-28T11:12:06.114295+0000","last_undegraded":"2023-11-28T11:12:06.114295+0000","last_fullsized":"2023-11-28T11:12:06.114295+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:49:39.541984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115504+0000","last_change":"2023-11-28T11:12:03.824121+0000","last_active":"2023-11-28T11:12:06.115504+0000","last_peered":"2023-11-28T11:12:06.115504+0000","last_clean":"2023-11-28T11:12:06.115504+0000","last_became_active":"2023-11-28T11:12:03.823702+0000","last_became_peered":"2023-11-28T11:12:03.823702+0000","last_unstale":"2023-11-28T11:12:06.115504+0000","last_undegraded":"2023-11-28T11:12:06.115504+0000","last_fullsized":"2023-11-28T11:12:06.115504+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:07:36.641433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114167+0000","last_change":"2023-11-28T11:12:03.818921+0000","last_active":"2023-11-28T11:12:06.114167+0000","last_peered":"2023-11-28T11:12:06.114167+0000","last_clean":"2023-11-28T11:12:06.114167+0000","last_became_active":"2023-11-28T11:12:03.818749+0000","last_became_peered":"2023-11-28T11:12:03.818749+0000","last_unstale":"2023-11-28T11:12:06.114167+0000","last_undegraded":"2023-11-28T11:12:06.114167+0000","last_fullsized":"2023-11-28T11:12:06.114167+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:57:47.752363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.811030+0000","last_change":"2023-11-28T11:12:03.832151+0000","last_active":"2023-11-28T11:12:04.811030+0000","last_peered":"2023-11-28T11:12:04.811030+0000","last_clean":"2023-11-28T11:12:04.811030+0000","last_became_active":"2023-11-28T11:12:03.831807+0000","last_became_peered":"2023-11-28T11:12:03.831807+0000","last_unstale":"2023-11-28T11:12:04.811030+0000","last_undegraded":"2023-11-28T11:12:04.811030+0000","last_fullsized":"2023-11-28T11:12:04.811030+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:46.858297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.811079+0000","last_change":"2023-11-28T11:12:03.822425+0000","last_active":"2023-11-28T11:12:04.811079+0000","last_peered":"2023-11-28T11:12:04.811079+0000","last_clean":"2023-11-28T11:12:04.811079+0000","last_became_active":"2023-11-28T11:12:03.822117+0000","last_became_peered":"2023-11-28T11:12:03.822117+0000","last_unstale":"2023-11-28T11:12:04.811079+0000","last_undegraded":"2023-11-28T11:12:04.811079+0000","last_fullsized":"2023-11-28T11:12:04.811079+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:18:47.403543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.201905+0000","last_change":"2023-11-28T11:12:03.819807+0000","last_active":"2023-11-28T11:12:05.201905+0000","last_peered":"2023-11-28T11:12:05.201905+0000","last_clean":"2023-11-28T11:12:05.201905+0000","last_became_active":"2023-11-28T11:12:03.819519+0000","last_became_peered":"2023-11-28T11:12:03.819519+0000","last_unstale":"2023-11-28T11:12:05.201905+0000","last_undegraded":"2023-11-28T11:12:05.201905+0000","last_fullsized":"2023-11-28T11:12:05.201905+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:19.630863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115907+0000","last_change":"2023-11-28T11:12:03.839270+0000","last_active":"2023-11-28T11:12:06.115907+0000","last_peered":"2023-11-28T11:12:06.115907+0000","last_clean":"2023-11-28T11:12:06.115907+0000","last_became_active":"2023-11-28T11:12:03.839008+0000","last_became_peered":"2023-11-28T11:12:03.839008+0000","last_unstale":"2023-11-28T11:12:06.115907+0000","last_undegraded":"2023-11-28T11:12:06.115907+0000","last_fullsized":"2023-11-28T11:12:06.115907+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:48:55.831494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"65'4","reported_seq":15,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.838840+0000","last_change":"2023-11-28T11:12:03.405175+0000","last_active":"2023-11-28T11:12:03.838840+0000","last_peered":"2023-11-28T11:12:03.838840+0000","last_clean":"2023-11-28T11:12:03.838840+0000","last_became_active":"2023-11-28T11:12:03.404896+0000","last_became_peered":"2023-11-28T11:12:03.404896+0000","last_unstale":"2023-11-28T11:12:03.838840+0000","last_undegraded":"2023-11-28T11:12:03.838840+0000","last_fullsized":"2023-11-28T11:12:03.838840+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:18.625956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":566,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348649+0000","last_change":"2023-11-28T11:12:03.820945+0000","last_active":"2023-11-28T11:12:05.348649+0000","last_peered":"2023-11-28T11:12:05.348649+0000","last_clean":"2023-11-28T11:12:05.348649+0000","last_became_active":"2023-11-28T11:12:03.820261+0000","last_became_peered":"2023-11-28T11:12:03.820261+0000","last_unstale":"2023-11-28T11:12:05.348649+0000","last_undegraded":"2023-11-28T11:12:05.348649+0000","last_fullsized":"2023-11-28T11:12:05.348649+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:37.833203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.821456+0000","last_change":"2023-11-28T11:12:03.401516+0000","last_active":"2023-11-28T11:12:03.821456+0000","last_peered":"2023-11-28T11:12:03.821456+0000","last_clean":"2023-11-28T11:12:03.821456+0000","last_became_active":"2023-11-28T11:12:03.401240+0000","last_became_peered":"2023-11-28T11:12:03.401240+0000","last_unstale":"2023-11-28T11:12:03.821456+0000","last_undegraded":"2023-11-28T11:12:03.821456+0000","last_fullsized":"2023-11-28T11:12:03.821456+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:32:22.415232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.202299+0000","last_change":"2023-11-28T11:12:03.826765+0000","last_active":"2023-11-28T11:12:05.202299+0000","last_peered":"2023-11-28T11:12:05.202299+0000","last_clean":"2023-11-28T11:12:05.202299+0000","last_became_active":"2023-11-28T11:12:03.826472+0000","last_became_peered":"2023-11-28T11:12:03.826472+0000","last_unstale":"2023-11-28T11:12:05.202299+0000","last_undegraded":"2023-11-28T11:12:05.202299+0000","last_fullsized":"2023-11-28T11:12:05.202299+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:04:36.395113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.840681+0000","last_change":"2023-11-28T11:12:03.404430+0000","last_active":"2023-11-28T11:12:03.840681+0000","last_peered":"2023-11-28T11:12:03.840681+0000","last_clean":"2023-11-28T11:12:03.840681+0000","last_became_active":"2023-11-28T11:12:03.404085+0000","last_became_peered":"2023-11-28T11:12:03.404085+0000","last_unstale":"2023-11-28T11:12:03.840681+0000","last_undegraded":"2023-11-28T11:12:03.840681+0000","last_fullsized":"2023-11-28T11:12:03.840681+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:31:31.263698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115523+0000","last_change":"2023-11-28T11:12:03.829985+0000","last_active":"2023-11-28T11:12:06.115523+0000","last_peered":"2023-11-28T11:12:06.115523+0000","last_clean":"2023-11-28T11:12:06.115523+0000","last_became_active":"2023-11-28T11:12:03.829105+0000","last_became_peered":"2023-11-28T11:12:03.829105+0000","last_unstale":"2023-11-28T11:12:06.115523+0000","last_undegraded":"2023-11-28T11:12:06.115523+0000","last_fullsized":"2023-11-28T11:12:06.115523+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:30.549631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.829662+0000","last_change":"2023-11-28T11:12:03.402127+0000","last_active":"2023-11-28T11:12:03.829662+0000","last_peered":"2023-11-28T11:12:03.829662+0000","last_clean":"2023-11-28T11:12:03.829662+0000","last_became_active":"2023-11-28T11:12:03.400646+0000","last_became_peered":"2023-11-28T11:12:03.400646+0000","last_unstale":"2023-11-28T11:12:03.829662+0000","last_undegraded":"2023-11-28T11:12:03.829662+0000","last_fullsized":"2023-11-28T11:12:03.829662+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:52:36.828296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.811066+0000","last_change":"2023-11-28T11:12:03.817612+0000","last_active":"2023-11-28T11:12:04.811066+0000","last_peered":"2023-11-28T11:12:04.811066+0000","last_clean":"2023-11-28T11:12:04.811066+0000","last_became_active":"2023-11-28T11:12:03.817002+0000","last_became_peered":"2023-11-28T11:12:03.817002+0000","last_unstale":"2023-11-28T11:12:04.811066+0000","last_undegraded":"2023-11-28T11:12:04.811066+0000","last_fullsized":"2023-11-28T11:12:04.811066+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:34:30.549631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.840145+0000","last_change":"2023-11-28T11:12:03.399675+0000","last_active":"2023-11-28T11:12:03.840145+0000","last_peered":"2023-11-28T11:12:03.840145+0000","last_clean":"2023-11-28T11:12:03.840145+0000","last_became_active":"2023-11-28T11:12:03.399395+0000","last_became_peered":"2023-11-28T11:12:03.399395+0000","last_unstale":"2023-11-28T11:12:03.840145+0000","last_undegraded":"2023-11-28T11:12:03.840145+0000","last_fullsized":"2023-11-28T11:12:03.840145+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:07:35.636526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115515+0000","last_change":"2023-11-28T11:12:03.831136+0000","last_active":"2023-11-28T11:12:06.115515+0000","last_peered":"2023-11-28T11:12:06.115515+0000","last_clean":"2023-11-28T11:12:06.115515+0000","last_became_active":"2023-11-28T11:12:03.830872+0000","last_became_peered":"2023-11-28T11:12:03.830872+0000","last_unstale":"2023-11-28T11:12:06.115515+0000","last_undegraded":"2023-11-28T11:12:06.115515+0000","last_fullsized":"2023-11-28T11:12:06.115515+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:54.346774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.831025+0000","last_change":"2023-11-28T11:12:03.407430+0000","last_active":"2023-11-28T11:12:03.831025+0000","last_peered":"2023-11-28T11:12:03.831025+0000","last_clean":"2023-11-28T11:12:03.831025+0000","last_became_active":"2023-11-28T11:12:03.407106+0000","last_became_peered":"2023-11-28T11:12:03.407106+0000","last_unstale":"2023-11-28T11:12:03.831025+0000","last_undegraded":"2023-11-28T11:12:03.831025+0000","last_fullsized":"2023-11-28T11:12:03.831025+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:48:16.699038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348061+0000","last_change":"2023-11-28T11:12:03.833947+0000","last_active":"2023-11-28T11:12:05.348061+0000","last_peered":"2023-11-28T11:12:05.348061+0000","last_clean":"2023-11-28T11:12:05.348061+0000","last_became_active":"2023-11-28T11:12:03.833669+0000","last_became_peered":"2023-11-28T11:12:03.833669+0000","last_unstale":"2023-11-28T11:12:05.348061+0000","last_undegraded":"2023-11-28T11:12:05.348061+0000","last_fullsized":"2023-11-28T11:12:05.348061+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:37:05.997226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.807329+0000","last_change":"2023-11-28T11:12:03.390405+0000","last_active":"2023-11-28T11:12:03.807329+0000","last_peered":"2023-11-28T11:12:03.807329+0000","last_clean":"2023-11-28T11:12:03.807329+0000","last_became_active":"2023-11-28T11:12:03.390092+0000","last_became_peered":"2023-11-28T11:12:03.390092+0000","last_unstale":"2023-11-28T11:12:03.807329+0000","last_undegraded":"2023-11-28T11:12:03.807329+0000","last_fullsized":"2023-11-28T11:12:03.807329+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:51:23.997889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.164957+0000","last_change":"2023-11-28T11:12:03.826749+0000","last_active":"2023-11-28T11:12:05.164957+0000","last_peered":"2023-11-28T11:12:05.164957+0000","last_clean":"2023-11-28T11:12:05.164957+0000","last_became_active":"2023-11-28T11:12:03.826451+0000","last_became_peered":"2023-11-28T11:12:03.826451+0000","last_unstale":"2023-11-28T11:12:05.164957+0000","last_undegraded":"2023-11-28T11:12:05.164957+0000","last_fullsized":"2023-11-28T11:12:05.164957+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:22:12.398418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.809575+0000","last_change":"2023-11-28T11:12:03.392079+0000","last_active":"2023-11-28T11:12:03.809575+0000","last_peered":"2023-11-28T11:12:03.809575+0000","last_clean":"2023-11-28T11:12:03.809575+0000","last_became_active":"2023-11-28T11:12:03.391649+0000","last_became_peered":"2023-11-28T11:12:03.391649+0000","last_unstale":"2023-11-28T11:12:03.809575+0000","last_undegraded":"2023-11-28T11:12:03.809575+0000","last_fullsized":"2023-11-28T11:12:03.809575+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:03:13.999980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.811346+0000","last_change":"2023-11-28T11:12:03.824511+0000","last_active":"2023-11-28T11:12:04.811346+0000","last_peered":"2023-11-28T11:12:04.811346+0000","last_clean":"2023-11-28T11:12:04.811346+0000","last_became_active":"2023-11-28T11:12:03.824178+0000","last_became_peered":"2023-11-28T11:12:03.824178+0000","last_unstale":"2023-11-28T11:12:04.811346+0000","last_undegraded":"2023-11-28T11:12:04.811346+0000","last_fullsized":"2023-11-28T11:12:04.811346+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:32:23.420139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.815173+0000","last_change":"2023-11-28T11:12:03.400809+0000","last_active":"2023-11-28T11:12:03.815173+0000","last_peered":"2023-11-28T11:12:03.815173+0000","last_clean":"2023-11-28T11:12:03.815173+0000","last_became_active":"2023-11-28T11:12:03.400503+0000","last_became_peered":"2023-11-28T11:12:03.400503+0000","last_unstale":"2023-11-28T11:12:03.815173+0000","last_undegraded":"2023-11-28T11:12:03.815173+0000","last_fullsized":"2023-11-28T11:12:03.815173+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:11:17.899964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115847+0000","last_change":"2023-11-28T11:12:03.836342+0000","last_active":"2023-11-28T11:12:06.115847+0000","last_peered":"2023-11-28T11:12:06.115847+0000","last_clean":"2023-11-28T11:12:06.115847+0000","last_became_active":"2023-11-28T11:12:03.836002+0000","last_became_peered":"2023-11-28T11:12:03.836002+0000","last_unstale":"2023-11-28T11:12:06.115847+0000","last_undegraded":"2023-11-28T11:12:06.115847+0000","last_fullsized":"2023-11-28T11:12:06.115847+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:42.823746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.816598+0000","last_change":"2023-11-28T11:12:03.407424+0000","last_active":"2023-11-28T11:12:03.816598+0000","last_peered":"2023-11-28T11:12:03.816598+0000","last_clean":"2023-11-28T11:12:03.816598+0000","last_became_active":"2023-11-28T11:12:03.407085+0000","last_became_peered":"2023-11-28T11:12:03.407085+0000","last_unstale":"2023-11-28T11:12:03.816598+0000","last_undegraded":"2023-11-28T11:12:03.816598+0000","last_fullsized":"2023-11-28T11:12:03.816598+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:50:38.258754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.202408+0000","last_change":"2023-11-28T11:12:03.842277+0000","last_active":"2023-11-28T11:12:05.202408+0000","last_peered":"2023-11-28T11:12:05.202408+0000","last_clean":"2023-11-28T11:12:05.202408+0000","last_became_active":"2023-11-28T11:12:03.841981+0000","last_became_peered":"2023-11-28T11:12:03.841981+0000","last_unstale":"2023-11-28T11:12:05.202408+0000","last_undegraded":"2023-11-28T11:12:05.202408+0000","last_fullsized":"2023-11-28T11:12:05.202408+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:15:30.537954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"65'3","reported_seq":14,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.832084+0000","last_change":"2023-11-28T11:12:03.399002+0000","last_active":"2023-11-28T11:12:03.832084+0000","last_peered":"2023-11-28T11:12:03.832084+0000","last_clean":"2023-11-28T11:12:03.832084+0000","last_became_active":"2023-11-28T11:12:03.398692+0000","last_became_peered":"2023-11-28T11:12:03.398692+0000","last_unstale":"2023-11-28T11:12:03.832084+0000","last_undegraded":"2023-11-28T11:12:03.832084+0000","last_fullsized":"2023-11-28T11:12:03.832084+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:32.442297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348502+0000","last_change":"2023-11-28T11:12:03.821916+0000","last_active":"2023-11-28T11:12:05.348502+0000","last_peered":"2023-11-28T11:12:05.348502+0000","last_clean":"2023-11-28T11:12:05.348502+0000","last_became_active":"2023-11-28T11:12:03.821120+0000","last_became_peered":"2023-11-28T11:12:03.821120+0000","last_unstale":"2023-11-28T11:12:05.348502+0000","last_undegraded":"2023-11-28T11:12:05.348502+0000","last_fullsized":"2023-11-28T11:12:05.348502+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:51:25.002796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"65'6","reported_seq":17,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.841666+0000","last_change":"2023-11-28T11:12:03.402773+0000","last_active":"2023-11-28T11:12:03.841666+0000","last_peered":"2023-11-28T11:12:03.841666+0000","last_clean":"2023-11-28T11:12:03.841666+0000","last_became_active":"2023-11-28T11:12:03.402308+0000","last_became_peered":"2023-11-28T11:12:03.402308+0000","last_unstale":"2023-11-28T11:12:03.841666+0000","last_undegraded":"2023-11-28T11:12:03.841666+0000","last_fullsized":"2023-11-28T11:12:03.841666+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:04.830939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1548,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114883+0000","last_change":"2023-11-28T11:12:03.829146+0000","last_active":"2023-11-28T11:12:06.114883+0000","last_peered":"2023-11-28T11:12:06.114883+0000","last_clean":"2023-11-28T11:12:06.114883+0000","last_became_active":"2023-11-28T11:12:03.828841+0000","last_became_peered":"2023-11-28T11:12:03.828841+0000","last_unstale":"2023-11-28T11:12:06.114883+0000","last_undegraded":"2023-11-28T11:12:06.114883+0000","last_fullsized":"2023-11-28T11:12:06.114883+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:46.858297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.836303+0000","last_change":"2023-11-28T11:12:03.400815+0000","last_active":"2023-11-28T11:12:03.836303+0000","last_peered":"2023-11-28T11:12:03.836303+0000","last_clean":"2023-11-28T11:12:03.836303+0000","last_became_active":"2023-11-28T11:12:03.400513+0000","last_became_peered":"2023-11-28T11:12:03.400513+0000","last_unstale":"2023-11-28T11:12:03.836303+0000","last_undegraded":"2023-11-28T11:12:03.836303+0000","last_fullsized":"2023-11-28T11:12:03.836303+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:45.516962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.811229+0000","last_change":"2023-11-28T11:12:03.840762+0000","last_active":"2023-11-28T11:12:04.811229+0000","last_peered":"2023-11-28T11:12:04.811229+0000","last_clean":"2023-11-28T11:12:04.811229+0000","last_became_active":"2023-11-28T11:12:03.840452+0000","last_became_peered":"2023-11-28T11:12:03.840452+0000","last_unstale":"2023-11-28T11:12:04.811229+0000","last_undegraded":"2023-11-28T11:12:04.811229+0000","last_fullsized":"2023-11-28T11:12:04.811229+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:37.484325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.816701+0000","last_change":"2023-11-28T11:12:03.404712+0000","last_active":"2023-11-28T11:12:03.816701+0000","last_peered":"2023-11-28T11:12:03.816701+0000","last_clean":"2023-11-28T11:12:03.816701+0000","last_became_active":"2023-11-28T11:12:03.402929+0000","last_became_peered":"2023-11-28T11:12:03.402929+0000","last_unstale":"2023-11-28T11:12:03.816701+0000","last_undegraded":"2023-11-28T11:12:03.816701+0000","last_fullsized":"2023-11-28T11:12:03.816701+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:09:26.168016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.164911+0000","last_change":"2023-11-28T11:12:03.821050+0000","last_active":"2023-11-28T11:12:05.164911+0000","last_peered":"2023-11-28T11:12:05.164911+0000","last_clean":"2023-11-28T11:12:05.164911+0000","last_became_active":"2023-11-28T11:12:03.820787+0000","last_became_peered":"2023-11-28T11:12:03.820787+0000","last_unstale":"2023-11-28T11:12:05.164911+0000","last_undegraded":"2023-11-28T11:12:05.164911+0000","last_fullsized":"2023-11-28T11:12:05.164911+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:29:36.124745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.810573+0000","last_change":"2023-11-28T11:12:03.397837+0000","last_active":"2023-11-28T11:12:03.810573+0000","last_peered":"2023-11-28T11:12:03.810573+0000","last_clean":"2023-11-28T11:12:03.810573+0000","last_became_active":"2023-11-28T11:12:03.397522+0000","last_became_peered":"2023-11-28T11:12:03.397522+0000","last_unstale":"2023-11-28T11:12:03.810573+0000","last_undegraded":"2023-11-28T11:12:03.810573+0000","last_fullsized":"2023-11-28T11:12:03.810573+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:15:29.533047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348209+0000","last_change":"2023-11-28T11:12:03.821866+0000","last_active":"2023-11-28T11:12:05.348209+0000","last_peered":"2023-11-28T11:12:05.348209+0000","last_clean":"2023-11-28T11:12:05.348209+0000","last_became_active":"2023-11-28T11:12:03.821489+0000","last_became_peered":"2023-11-28T11:12:03.821489+0000","last_unstale":"2023-11-28T11:12:05.348209+0000","last_undegraded":"2023-11-28T11:12:05.348209+0000","last_fullsized":"2023-11-28T11:12:05.348209+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:06:46.318560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"65'8","reported_seq":19,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.842152+0000","last_change":"2023-11-28T11:12:03.404983+0000","last_active":"2023-11-28T11:12:03.842152+0000","last_peered":"2023-11-28T11:12:03.842152+0000","last_clean":"2023-11-28T11:12:03.842152+0000","last_became_active":"2023-11-28T11:12:03.404687+0000","last_became_peered":"2023-11-28T11:12:03.404687+0000","last_unstale":"2023-11-28T11:12:03.842152+0000","last_undegraded":"2023-11-28T11:12:03.842152+0000","last_fullsized":"2023-11-28T11:12:03.842152+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:17:29.499469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.165224+0000","last_change":"2023-11-28T11:12:03.822004+0000","last_active":"2023-11-28T11:12:05.165224+0000","last_peered":"2023-11-28T11:12:05.165224+0000","last_clean":"2023-11-28T11:12:05.165224+0000","last_became_active":"2023-11-28T11:12:03.821732+0000","last_became_peered":"2023-11-28T11:12:03.821732+0000","last_unstale":"2023-11-28T11:12:05.165224+0000","last_undegraded":"2023-11-28T11:12:05.165224+0000","last_fullsized":"2023-11-28T11:12:05.165224+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:57:17.415503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.827498+0000","last_change":"2023-11-28T11:12:03.403409+0000","last_active":"2023-11-28T11:12:03.827498+0000","last_peered":"2023-11-28T11:12:03.827498+0000","last_clean":"2023-11-28T11:12:03.827498+0000","last_became_active":"2023-11-28T11:12:03.403091+0000","last_became_peered":"2023-11-28T11:12:03.403091+0000","last_unstale":"2023-11-28T11:12:03.827498+0000","last_undegraded":"2023-11-28T11:12:03.827498+0000","last_fullsized":"2023-11-28T11:12:03.827498+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:06.501050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.811240+0000","last_change":"2023-11-28T11:12:03.818491+0000","last_active":"2023-11-28T11:12:04.811240+0000","last_peered":"2023-11-28T11:12:04.811240+0000","last_clean":"2023-11-28T11:12:04.811240+0000","last_became_active":"2023-11-28T11:12:03.818109+0000","last_became_peered":"2023-11-28T11:12:03.818109+0000","last_unstale":"2023-11-28T11:12:04.811240+0000","last_undegraded":"2023-11-28T11:12:04.811240+0000","last_fullsized":"2023-11-28T11:12:04.811240+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:07:15.857309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.811650+0000","last_change":"2023-11-28T11:12:03.397826+0000","last_active":"2023-11-28T11:12:03.811650+0000","last_peered":"2023-11-28T11:12:03.811650+0000","last_clean":"2023-11-28T11:12:03.811650+0000","last_became_active":"2023-11-28T11:12:03.397512+0000","last_became_peered":"2023-11-28T11:12:03.397512+0000","last_unstale":"2023-11-28T11:12:03.811650+0000","last_undegraded":"2023-11-28T11:12:03.811650+0000","last_fullsized":"2023-11-28T11:12:03.811650+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:26.097220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115266+0000","last_change":"2023-11-28T11:12:03.832920+0000","last_active":"2023-11-28T11:12:06.115266+0000","last_peered":"2023-11-28T11:12:06.115266+0000","last_clean":"2023-11-28T11:12:06.115266+0000","last_became_active":"2023-11-28T11:12:03.832618+0000","last_became_peered":"2023-11-28T11:12:03.832618+0000","last_unstale":"2023-11-28T11:12:06.115266+0000","last_undegraded":"2023-11-28T11:12:06.115266+0000","last_fullsized":"2023-11-28T11:12:06.115266+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:40:46.824986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.815405+0000","last_change":"2023-11-28T11:12:03.400853+0000","last_active":"2023-11-28T11:12:03.815405+0000","last_peered":"2023-11-28T11:12:03.815405+0000","last_clean":"2023-11-28T11:12:03.815405+0000","last_became_active":"2023-11-28T11:12:03.400559+0000","last_became_peered":"2023-11-28T11:12:03.400559+0000","last_unstale":"2023-11-28T11:12:03.815405+0000","last_undegraded":"2023-11-28T11:12:03.815405+0000","last_fullsized":"2023-11-28T11:12:03.815405+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:13:41.696052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115531+0000","last_change":"2023-11-28T11:12:03.835954+0000","last_active":"2023-11-28T11:12:06.115531+0000","last_peered":"2023-11-28T11:12:06.115531+0000","last_clean":"2023-11-28T11:12:06.115531+0000","last_became_active":"2023-11-28T11:12:03.835554+0000","last_became_peered":"2023-11-28T11:12:03.835554+0000","last_unstale":"2023-11-28T11:12:06.115531+0000","last_undegraded":"2023-11-28T11:12:06.115531+0000","last_fullsized":"2023-11-28T11:12:06.115531+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:46:37.484325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.804869+0000","last_change":"2023-11-28T11:12:03.394462+0000","last_active":"2023-11-28T11:12:03.804869+0000","last_peered":"2023-11-28T11:12:03.804869+0000","last_clean":"2023-11-28T11:12:03.804869+0000","last_became_active":"2023-11-28T11:12:03.394176+0000","last_became_peered":"2023-11-28T11:12:03.394176+0000","last_unstale":"2023-11-28T11:12:03.804869+0000","last_undegraded":"2023-11-28T11:12:03.804869+0000","last_fullsized":"2023-11-28T11:12:03.804869+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:53.341867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348692+0000","last_change":"2023-11-28T11:12:03.818464+0000","last_active":"2023-11-28T11:12:05.348692+0000","last_peered":"2023-11-28T11:12:05.348692+0000","last_clean":"2023-11-28T11:12:05.348692+0000","last_became_active":"2023-11-28T11:12:03.818173+0000","last_became_peered":"2023-11-28T11:12:03.818173+0000","last_unstale":"2023-11-28T11:12:05.348692+0000","last_undegraded":"2023-11-28T11:12:05.348692+0000","last_fullsized":"2023-11-28T11:12:05.348692+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:32:10.047361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.814457+0000","last_change":"2023-11-28T11:12:03.401986+0000","last_active":"2023-11-28T11:12:03.814457+0000","last_peered":"2023-11-28T11:12:03.814457+0000","last_clean":"2023-11-28T11:12:03.814457+0000","last_became_active":"2023-11-28T11:12:03.401694+0000","last_became_peered":"2023-11-28T11:12:03.401694+0000","last_unstale":"2023-11-28T11:12:03.814457+0000","last_undegraded":"2023-11-28T11:12:03.814457+0000","last_fullsized":"2023-11-28T11:12:03.814457+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:33:57.028903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.202108+0000","last_change":"2023-11-28T11:12:03.828100+0000","last_active":"2023-11-28T11:12:05.202108+0000","last_peered":"2023-11-28T11:12:05.202108+0000","last_clean":"2023-11-28T11:12:05.202108+0000","last_became_active":"2023-11-28T11:12:03.827844+0000","last_became_peered":"2023-11-28T11:12:03.827844+0000","last_unstale":"2023-11-28T11:12:05.202108+0000","last_undegraded":"2023-11-28T11:12:05.202108+0000","last_fullsized":"2023-11-28T11:12:05.202108+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T12:31:32.268605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.807427+0000","last_change":"2023-11-28T11:12:03.389789+0000","last_active":"2023-11-28T11:12:03.807427+0000","last_peered":"2023-11-28T11:12:03.807427+0000","last_clean":"2023-11-28T11:12:03.807427+0000","last_became_active":"2023-11-28T11:12:03.389497+0000","last_became_peered":"2023-11-28T11:12:03.389497+0000","last_unstale":"2023-11-28T11:12:03.807427+0000","last_undegraded":"2023-11-28T11:12:03.807427+0000","last_fullsized":"2023-11-28T11:12:03.807427+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:06.501050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.809722+0000","last_change":"2023-11-28T11:12:03.816789+0000","last_active":"2023-11-28T11:12:04.809722+0000","last_peered":"2023-11-28T11:12:04.809722+0000","last_clean":"2023-11-28T11:12:04.809722+0000","last_became_active":"2023-11-28T11:12:03.816210+0000","last_became_peered":"2023-11-28T11:12:03.816210+0000","last_unstale":"2023-11-28T11:12:04.809722+0000","last_undegraded":"2023-11-28T11:12:04.809722+0000","last_fullsized":"2023-11-28T11:12:04.809722+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:30:44.185700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.809965+0000","last_change":"2023-11-28T11:12:03.404990+0000","last_active":"2023-11-28T11:12:03.809965+0000","last_peered":"2023-11-28T11:12:03.809965+0000","last_clean":"2023-11-28T11:12:03.809965+0000","last_became_active":"2023-11-28T11:12:03.404699+0000","last_became_peered":"2023-11-28T11:12:03.404699+0000","last_unstale":"2023-11-28T11:12:03.809965+0000","last_undegraded":"2023-11-28T11:12:03.809965+0000","last_fullsized":"2023-11-28T11:12:03.809965+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:01:39.829848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.201898+0000","last_change":"2023-11-28T11:12:03.820212+0000","last_active":"2023-11-28T11:12:05.201898+0000","last_peered":"2023-11-28T11:12:05.201898+0000","last_clean":"2023-11-28T11:12:05.201898+0000","last_became_active":"2023-11-28T11:12:03.819889+0000","last_became_peered":"2023-11-28T11:12:03.819889+0000","last_unstale":"2023-11-28T11:12:05.201898+0000","last_undegraded":"2023-11-28T11:12:05.201898+0000","last_fullsized":"2023-11-28T11:12:05.201898+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:26:39.132766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.815907+0000","last_change":"2023-11-28T11:12:03.405308+0000","last_active":"2023-11-28T11:12:03.815907+0000","last_peered":"2023-11-28T11:12:03.815907+0000","last_clean":"2023-11-28T11:12:03.815907+0000","last_became_active":"2023-11-28T11:12:03.403597+0000","last_became_peered":"2023-11-28T11:12:03.403597+0000","last_unstale":"2023-11-28T11:12:03.815907+0000","last_undegraded":"2023-11-28T11:12:03.815907+0000","last_fullsized":"2023-11-28T11:12:03.815907+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:17:37.393448+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115749+0000","last_change":"2023-11-28T11:12:03.833573+0000","last_active":"2023-11-28T11:12:06.115749+0000","last_peered":"2023-11-28T11:12:06.115749+0000","last_clean":"2023-11-28T11:12:06.115749+0000","last_became_active":"2023-11-28T11:12:03.831031+0000","last_became_peered":"2023-11-28T11:12:03.831031+0000","last_unstale":"2023-11-28T11:12:06.115749+0000","last_undegraded":"2023-11-28T11:12:06.115749+0000","last_fullsized":"2023-11-28T11:12:06.115749+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:31:54.346774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.840502+0000","last_change":"2023-11-28T11:12:03.404206+0000","last_active":"2023-11-28T11:12:03.840502+0000","last_peered":"2023-11-28T11:12:03.840502+0000","last_clean":"2023-11-28T11:12:03.840502+0000","last_became_active":"2023-11-28T11:12:03.403925+0000","last_became_peered":"2023-11-28T11:12:03.403925+0000","last_unstale":"2023-11-28T11:12:03.840502+0000","last_undegraded":"2023-11-28T11:12:03.840502+0000","last_fullsized":"2023-11-28T11:12:03.840502+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T17:04:35.390206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.814813+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:12:03.814813+0000","last_peered":"2023-11-28T11:12:03.814813+0000","last_clean":"2023-11-28T11:12:03.814813+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:12:03.814813+0000","last_undegraded":"2023-11-28T11:12:03.814813+0000","last_fullsized":"2023-11-28T11:12:03.814813+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.115703+0000","last_change":"2023-11-28T11:12:03.833188+0000","last_active":"2023-11-28T11:12:06.115703+0000","last_peered":"2023-11-28T11:12:06.115703+0000","last_clean":"2023-11-28T11:12:06.115703+0000","last_became_active":"2023-11-28T11:12:03.824023+0000","last_became_peered":"2023-11-28T11:12:03.824023+0000","last_unstale":"2023-11-28T11:12:06.115703+0000","last_undegraded":"2023-11-28T11:12:06.115703+0000","last_fullsized":"2023-11-28T11:12:06.115703+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:55:42.823746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.829562+0000","last_change":"2023-11-28T11:12:03.403355+0000","last_active":"2023-11-28T11:12:03.829562+0000","last_peered":"2023-11-28T11:12:03.829562+0000","last_clean":"2023-11-28T11:12:03.829562+0000","last_became_active":"2023-11-28T11:12:03.402998+0000","last_became_peered":"2023-11-28T11:12:03.402998+0000","last_unstale":"2023-11-28T11:12:03.829562+0000","last_undegraded":"2023-11-28T11:12:03.829562+0000","last_fullsized":"2023-11-28T11:12:03.829562+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:27:45.516962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.810343+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:12:03.810343+0000","last_peered":"2023-11-28T11:12:03.810343+0000","last_clean":"2023-11-28T11:12:03.810343+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:12:03.810343+0000","last_undegraded":"2023-11-28T11:12:03.810343+0000","last_fullsized":"2023-11-28T11:12:03.810343+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114167+0000","last_change":"2023-11-28T11:12:03.823050+0000","last_active":"2023-11-28T11:12:06.114167+0000","last_peered":"2023-11-28T11:12:06.114167+0000","last_clean":"2023-11-28T11:12:06.114167+0000","last_became_active":"2023-11-28T11:12:03.822548+0000","last_became_peered":"2023-11-28T11:12:03.822548+0000","last_unstale":"2023-11-28T11:12:06.114167+0000","last_undegraded":"2023-11-28T11:12:06.114167+0000","last_fullsized":"2023-11-28T11:12:06.114167+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:25:40.599096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.804310+0000","last_change":"2023-11-28T11:12:03.396696+0000","last_active":"2023-11-28T11:12:03.804310+0000","last_peered":"2023-11-28T11:12:03.804310+0000","last_clean":"2023-11-28T11:12:03.804310+0000","last_became_active":"2023-11-28T11:12:03.396404+0000","last_became_peered":"2023-11-28T11:12:03.396404+0000","last_unstale":"2023-11-28T11:12:03.804310+0000","last_undegraded":"2023-11-28T11:12:03.804310+0000","last_fullsized":"2023-11-28T11:12:03.804310+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:42:45.853390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":53,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.807693+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:12:03.807693+0000","last_peered":"2023-11-28T11:12:03.807693+0000","last_clean":"2023-11-28T11:12:03.807693+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:12:03.807693+0000","last_undegraded":"2023-11-28T11:12:03.807693+0000","last_fullsized":"2023-11-28T11:12:03.807693+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":202,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.817618+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:12:03.817618+0000","last_peered":"2023-11-28T11:12:03.817618+0000","last_clean":"2023-11-28T11:12:03.817618+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:12:03.817618+0000","last_undegraded":"2023-11-28T11:12:03.817618+0000","last_fullsized":"2023-11-28T11:12:03.817618+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114128+0000","last_change":"2023-11-28T11:12:03.820805+0000","last_active":"2023-11-28T11:12:06.114128+0000","last_peered":"2023-11-28T11:12:06.114128+0000","last_clean":"2023-11-28T11:12:06.114128+0000","last_became_active":"2023-11-28T11:12:03.820554+0000","last_became_peered":"2023-11-28T11:12:03.820554+0000","last_unstale":"2023-11-28T11:12:06.114128+0000","last_undegraded":"2023-11-28T11:12:06.114128+0000","last_fullsized":"2023-11-28T11:12:06.114128+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:20:13.470501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.810690+0000","last_change":"2023-11-28T11:12:03.395950+0000","last_active":"2023-11-28T11:12:03.810690+0000","last_peered":"2023-11-28T11:12:03.810690+0000","last_clean":"2023-11-28T11:12:03.810690+0000","last_became_active":"2023-11-28T11:12:03.395628+0000","last_became_peered":"2023-11-28T11:12:03.395628+0000","last_unstale":"2023-11-28T11:12:03.810690+0000","last_undegraded":"2023-11-28T11:12:03.810690+0000","last_fullsized":"2023-11-28T11:12:03.810690+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:50:18.625956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.819385+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:12:03.819385+0000","last_peered":"2023-11-28T11:12:03.819385+0000","last_clean":"2023-11-28T11:12:03.819385+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:12:03.819385+0000","last_undegraded":"2023-11-28T11:12:03.819385+0000","last_fullsized":"2023-11-28T11:12:03.819385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114181+0000","last_change":"2023-11-28T11:12:03.828476+0000","last_active":"2023-11-28T11:12:06.114181+0000","last_peered":"2023-11-28T11:12:06.114181+0000","last_clean":"2023-11-28T11:12:06.114181+0000","last_became_active":"2023-11-28T11:12:03.828202+0000","last_became_peered":"2023-11-28T11:12:03.828202+0000","last_unstale":"2023-11-28T11:12:06.114181+0000","last_undegraded":"2023-11-28T11:12:06.114181+0000","last_fullsized":"2023-11-28T11:12:06.114181+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:39:17.000938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"65'2","reported_seq":13,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.829861+0000","last_change":"2023-11-28T11:12:03.402458+0000","last_active":"2023-11-28T11:12:03.829861+0000","last_peered":"2023-11-28T11:12:03.829861+0000","last_clean":"2023-11-28T11:12:03.829861+0000","last_became_active":"2023-11-28T11:12:03.402102+0000","last_became_peered":"2023-11-28T11:12:03.402102+0000","last_unstale":"2023-11-28T11:12:03.829861+0000","last_undegraded":"2023-11-28T11:12:03.829861+0000","last_fullsized":"2023-11-28T11:12:03.829861+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T21:27:04.830939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.817499+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:12:03.817499+0000","last_peered":"2023-11-28T11:12:03.817499+0000","last_clean":"2023-11-28T11:12:03.817499+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:12:03.817499+0000","last_undegraded":"2023-11-28T11:12:03.817499+0000","last_fullsized":"2023-11-28T11:12:03.817499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:06.114246+0000","last_change":"2023-11-28T11:12:03.812718+0000","last_active":"2023-11-28T11:12:06.114246+0000","last_peered":"2023-11-28T11:12:06.114246+0000","last_clean":"2023-11-28T11:12:06.114246+0000","last_became_active":"2023-11-28T11:12:03.812476+0000","last_became_peered":"2023-11-28T11:12:03.812476+0000","last_unstale":"2023-11-28T11:12:06.114246+0000","last_undegraded":"2023-11-28T11:12:06.114246+0000","last_fullsized":"2023-11-28T11:12:06.114246+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:57:06.522652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.816802+0000","last_change":"2023-11-28T11:12:03.407497+0000","last_active":"2023-11-28T11:12:03.816802+0000","last_peered":"2023-11-28T11:12:03.816802+0000","last_clean":"2023-11-28T11:12:03.816802+0000","last_became_active":"2023-11-28T11:12:03.407175+0000","last_became_peered":"2023-11-28T11:12:03.407175+0000","last_unstale":"2023-11-28T11:12:03.816802+0000","last_undegraded":"2023-11-28T11:12:03.816802+0000","last_fullsized":"2023-11-28T11:12:03.816802+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T15:51:23.997889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":57,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.810804+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:12:03.810804+0000","last_peered":"2023-11-28T11:12:03.810804+0000","last_clean":"2023-11-28T11:12:03.810804+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:12:03.810804+0000","last_undegraded":"2023-11-28T11:12:03.810804+0000","last_fullsized":"2023-11-28T11:12:03.810804+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:04.810076+0000","last_change":"2023-11-28T11:12:03.824796+0000","last_active":"2023-11-28T11:12:04.810076+0000","last_peered":"2023-11-28T11:12:04.810076+0000","last_clean":"2023-11-28T11:12:04.810076+0000","last_became_active":"2023-11-28T11:12:03.824165+0000","last_became_peered":"2023-11-28T11:12:03.824165+0000","last_unstale":"2023-11-28T11:12:04.810076+0000","last_undegraded":"2023-11-28T11:12:04.810076+0000","last_fullsized":"2023-11-28T11:12:04.810076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:46:53.999259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"65'1","reported_seq":12,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.841886+0000","last_change":"2023-11-28T11:12:03.402574+0000","last_active":"2023-11-28T11:12:03.841886+0000","last_peered":"2023-11-28T11:12:03.841886+0000","last_clean":"2023-11-28T11:12:03.841886+0000","last_became_active":"2023-11-28T11:12:03.402149+0000","last_became_peered":"2023-11-28T11:12:03.402149+0000","last_unstale":"2023-11-28T11:12:03.841886+0000","last_undegraded":"2023-11-28T11:12:03.841886+0000","last_fullsized":"2023-11-28T11:12:03.841886+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T23:08:11.744846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.816369+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:12:03.816369+0000","last_peered":"2023-11-28T11:12:03.816369+0000","last_clean":"2023-11-28T11:12:03.816369+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:12:03.816369+0000","last_undegraded":"2023-11-28T11:12:03.816369+0000","last_fullsized":"2023-11-28T11:12:03.816369+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":66,"state":"active+clean","last_fresh":"2023-11-28T11:12:05.348557+0000","last_change":"2023-11-28T11:12:03.824750+0000","last_active":"2023-11-28T11:12:05.348557+0000","last_peered":"2023-11-28T11:12:05.348557+0000","last_clean":"2023-11-28T11:12:05.348557+0000","last_became_active":"2023-11-28T11:12:03.824473+0000","last_became_peered":"2023-11-28T11:12:03.824473+0000","last_unstale":"2023-11-28T11:12:05.348557+0000","last_undegraded":"2023-11-28T11:12:05.348557+0000","last_fullsized":"2023-11-28T11:12:05.348557+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:03.375781+0000","last_clean_scrub_stamp":"2023-11-28T11:12:03.375781+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T13:10:07.505957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.817486+0000","last_change":"2023-11-28T11:12:03.405919+0000","last_active":"2023-11-28T11:12:03.817486+0000","last_peered":"2023-11-28T11:12:03.817486+0000","last_clean":"2023-11-28T11:12:03.817486+0000","last_became_active":"2023-11-28T11:12:03.405638+0000","last_became_peered":"2023-11-28T11:12:03.405638+0000","last_unstale":"2023-11-28T11:12:03.817486+0000","last_undegraded":"2023-11-28T11:12:03.817486+0000","last_fullsized":"2023-11-28T11:12:03.817486+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:12:02.370874+0000","last_clean_scrub_stamp":"2023-11-28T11:12:02.370874+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:19:26.097220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":52,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T11:12:03.816352+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:12:03.816352+0000","last_peered":"2023-11-28T11:12:03.816352+0000","last_clean":"2023-11-28T11:12:03.816352+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:12:03.816352+0000","last_undegraded":"2023-11-28T11:12:03.816352+0000","last_fullsized":"2023-11-28T11:12:03.816352+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2330,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10848,"kb_used_data":1368,"kb_used_omap":0,"kb_used_meta":9472,"kb_avail":94360992,"statfs":{"total":96636764160,"available":96625655808,"internally_reserved":0,"allocated":1400832,"data_stored":966096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9699328},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.59599999999999997,"15min":0.63900000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.76700000000000002,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.60999999999999999,"15min":0.63800000000000001},"min":{"1min":0.40000000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.81899999999999995,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.54200000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.64100000000000001,"15min":0.68000000000000005},"min":{"1min":0.378,"5min":0.316,"15min":0.316},"max":{"1min":0.72299999999999998,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.59399999999999997},"min":{"1min":0.40100000000000002,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.87,"5min":1.224,"15min":1.224},"last":0.44900000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65500000000000003,"15min":0.68700000000000006},"min":{"1min":0.443,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79500000000000004,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.72599999999999998,"15min":0.77300000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79300000000000004,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.65800000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.46899999999999997,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.83299999999999996,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.72499999999999998,"15min":0.76600000000000001},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.81499999999999995,"5min":3.294,"15min":3.294},"last":0.73199999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.35199999999999998,"5min":0.312,"15min":0.312},"max":{"1min":0.63300000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.625},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.55600000000000005,"15min":0.56200000000000006},"min":{"1min":0.34699999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.79200000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.64200000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.621,"15min":0.64000000000000001},"min":{"1min":0.36199999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.73599999999999999,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.63800000000000001,"15min":0.64700000000000002},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.78000000000000003,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.71499999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.66300000000000003,"15min":0.67500000000000004},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.86699999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.628,"15min":0.63600000000000001},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.85599999999999998,"5min":0.89700000000000002,"15min":0.89700000000000002},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574893,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9692,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362148,"statfs":{"total":96636764160,"available":96626839552,"internally_reserved":0,"allocated":806912,"data_stored":375220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.46100000000000002},"min":{"1min":0.29299999999999998,"5min":0.252,"15min":0.252},"max":{"1min":0.745,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.48999999999999999,"15min":0.47899999999999998},"min":{"1min":0.33700000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.751,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.71799999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.53100000000000003,"15min":0.51300000000000001},"min":{"1min":0.41099999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86399999999999999,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.51500000000000001,"15min":0.499},"min":{"1min":0.32600000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.81799999999999995}]},{"osd":2,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.51200000000000001,"15min":0.502},"min":{"1min":0.38500000000000001,"5min":0.312,"15min":0.312},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.66000000000000003,"15min":0.68100000000000005},"min":{"1min":0.40999999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84299999999999997,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.80200000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.69199999999999995,"15min":0.71899999999999997},"min":{"1min":0.45500000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81299999999999994,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.753},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.68200000000000005,"15min":0.69799999999999995},"min":{"1min":0.41799999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.80900000000000005,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.76300000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.54700000000000004,"15min":0.54700000000000004},"min":{"1min":0.36199999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.61899999999999999,"15min":0.628},"min":{"1min":0.40200000000000002,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.90300000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.53900000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.47199999999999998,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.61699999999999999,"15min":0.61499999999999999},"min":{"1min":0.46500000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.86799999999999999,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.61899999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57699999999999996,"15min":0.58399999999999996},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.67700000000000005,"15min":0.67900000000000005},"min":{"1min":0.54500000000000004,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.93000000000000005,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.69699999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10796,"kb_used_data":1380,"kb_used_omap":0,"kb_used_meta":9408,"kb_avail":94361044,"statfs":{"total":96636764160,"available":96625709056,"internally_reserved":0,"allocated":1413120,"data_stored":967644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9633792},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.33500000000000002,"15min":0.32900000000000001},"min":{"1min":0.20599999999999999,"5min":0.19,"15min":0.19},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.371,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.26900000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.57799999999999996,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.25700000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.33100000000000002,"15min":0.32000000000000001},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.47299999999999998,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.38600000000000001,"15min":0.38200000000000001},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.59999999999999998,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.32800000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.40899999999999997,"15min":0.39300000000000002},"min":{"1min":0.29399999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.41699999999999998,"15min":0.41099999999999998},"min":{"1min":0.27900000000000003,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.60999999999999999,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.54900000000000004,"15min":0.53300000000000003},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.57999999999999996,"15min":0.56899999999999995},"min":{"1min":0.39300000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.014},"last":0.73199999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.57899999999999996,"15min":0.55300000000000005},"min":{"1min":0.45800000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.94799999999999995,"5min":0.998,"15min":0.998},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.58199999999999996,"15min":0.56399999999999995},"min":{"1min":0.40200000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89300000000000002,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65200000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94099999999999995,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.60799999999999998,"15min":0.59299999999999997},"min":{"1min":0.433,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.018,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.60599999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.61299999999999999,"15min":0.60299999999999998},"min":{"1min":0.443,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.96699999999999997,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.61499999999999999,"15min":0.59799999999999998},"min":{"1min":0.48699999999999999,"5min":0.45500000000000002,"15min":0.45500000000000002},"max":{"1min":0.89800000000000002,"5min":1.071,"15min":1.071},"last":0.76000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9876,"kb_used_data":780,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361964,"statfs":{"total":96636764160,"available":96626651136,"internally_reserved":0,"allocated":798720,"data_stored":375128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.33200000000000002,"15min":0.32800000000000001},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.46999999999999997,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.379,"15min":0.373},"min":{"1min":0.32200000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.54300000000000004,"5min":0.622,"15min":0.622},"last":0.437}]},{"osd":2,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.254,"5min":0.246,"15min":0.246},"max":{"1min":0.52600000000000002,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.31},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39500000000000002,"15min":0.39800000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.53600000000000003,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.34899999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.371,"15min":0.371},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51400000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.376},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42999999999999999,"15min":0.42799999999999999},"min":{"1min":0.30299999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.56699999999999995,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.39700000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.59299999999999997,"15min":0.58499999999999996},"min":{"1min":0.48699999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78600000000000003,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.61199999999999999,"15min":0.61699999999999999},"min":{"1min":0.42399999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.84299999999999997,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.58099999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65800000000000003,"15min":0.622},"min":{"1min":0.497,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.90000000000000002,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.622,"15min":0.623},"min":{"1min":0.437,"5min":0.437,"15min":0.437},"max":{"1min":0.86899999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.69299999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.623,"15min":0.623},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88700000000000001,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.45000000000000001,"5min":0.432,"15min":0.432},"max":{"1min":0.85999999999999999,"5min":1.016,"15min":1.016},"last":0.66500000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:11:46 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.65400000000000003,"15min":0.65300000000000002},"min":{"1min":0.48699999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.93400000000000005,"5min":1.105,"15min":1.105},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.65000000000000002,"15min":0.64300000000000002},"min":{"1min":0.53500000000000003,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.94299999999999995,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9564,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":8768,"kb_avail":94362276,"statfs":{"total":96636764160,"available":96626970624,"internally_reserved":0,"allocated":806912,"data_stored":375715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":8978432},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.373},"min":{"1min":0.23300000000000001,"5min":0.214,"15min":0.214},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.38500000000000001,"15min":0.38900000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.61699999999999999,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.42199999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39100000000000001,"15min":0.39400000000000002},"min":{"1min":0.23100000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.64300000000000002,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.432,"15min":0.44400000000000001},"min":{"1min":0.29299999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.60399999999999998,"5min":1.528,"15min":1.528},"last":0.55900000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.38900000000000001,"15min":0.39600000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.59399999999999997,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.317},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.46700000000000003,"15min":0.47899999999999998},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.64800000000000002,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.45900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.61599999999999999,"15min":0.62},"min":{"1min":0.48399999999999999,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.81599999999999995,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82799999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.51200000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.60999999999999999,"15min":0.61199999999999999},"min":{"1min":0.47899999999999998,"5min":0.438,"15min":0.438},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66900000000000004,"5min":0.63900000000000001,"15min":0.63700000000000001},"min":{"1min":0.48099999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.88300000000000001,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52500000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.62,"15min":0.622},"min":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65900000000000003,"15min":0.66900000000000004},"min":{"1min":0.51400000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86399999999999999,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.69599999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:11:43 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64600000000000002,"15min":0.64900000000000002},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.63},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.629,"15min":0.63},"min":{"1min":0.52800000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.81799999999999995,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.47799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9820,"kb_used_data":788,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94362020,"statfs":{"total":96636764160,"available":96626708480,"internally_reserved":0,"allocated":806912,"data_stored":375240,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.71399999999999997,"15min":0.88},"min":{"1min":0.23899999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.53000000000000003,"5min":11.965,"15min":11.965},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.74299999999999999,"15min":0.91500000000000004},"min":{"1min":0.255,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.57399999999999995,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.32900000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.34699999999999998,"15min":0.34000000000000002},"min":{"1min":0.27100000000000002,"5min":0.215,"15min":0.215},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.44},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.92100000000000004,"15min":1.157},"min":{"1min":0.28499999999999998,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.69499999999999995,"5min":16.988,"15min":16.988},"last":0.34899999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.47799999999999998,"15min":0.5},"min":{"1min":0.32300000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64700000000000002,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":1.3149999999999999,"15min":1.7010000000000001},"min":{"1min":0.33800000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.629,"5min":27.887,"15min":27.887},"last":0.40000000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67300000000000004,"15min":0.70399999999999996},"min":{"1min":0.46800000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.86499999999999999,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.64700000000000002,"15min":0.66500000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.89500000000000002,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.47099999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66500000000000004,"15min":0.68700000000000006},"min":{"1min":0.439,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.85099999999999998,"5min":1.98,"15min":1.98},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":1,"15min":1.169},"min":{"1min":0.42799999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.45100000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":1.008,"15min":1.177},"min":{"1min":0.47099999999999997,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.82299999999999995,"5min":11.888,"15min":11.888},"last":0.5},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.69599999999999995,"15min":0.72999999999999998},"min":{"1min":0.439,"5min":0.437,"15min":0.437},"max":{"1min":0.80900000000000005,"5min":1.925,"15min":1.925},"last":0.41399999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71399999999999997,"15min":0.74399999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91700000000000004,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.746,"15min":0.77900000000000003},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.93899999999999995,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.53500000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9880,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361960,"statfs":{"total":96636764160,"available":96626647040,"internally_reserved":0,"allocated":802816,"data_stored":375198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53200000000000003,"15min":0.53400000000000003},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75900000000000001,"5min":0.996,"15min":0.996},"last":0.377},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.81699999999999995,"15min":0.93600000000000005},"min":{"1min":0.44,"5min":0.372,"15min":0.372},"max":{"1min":0.752,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.35299999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.40600000000000003,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.74299999999999999,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.57599999999999996,"15min":0.57799999999999996},"min":{"1min":0.34899999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.78700000000000003,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.48499999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.57499999999999996,"15min":0.57599999999999996},"min":{"1min":0.44,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.746,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.42799999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.71199999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.56499999999999995}]},{"osd":3,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.69199999999999995,"15min":0.746},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.81999999999999995,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.63800000000000001},"min":{"1min":0.46600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80100000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.53700000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.33700000000000002,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.69899999999999995,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.56000000000000005,"15min":0.56899999999999995},"min":{"1min":0.36199999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.73099999999999998,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.54700000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.58999999999999997,"15min":0.60399999999999998},"min":{"1min":0.42099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":1.145,"15min":1.145},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.61099999999999999,"15min":0.629},"min":{"1min":0.40300000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.747,"5min":1.085,"15min":1.085},"last":0.52700000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:11:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.60699999999999998,"15min":0.627},"min":{"1min":0.38500000000000001,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70699999999999996,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.504},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.66600000000000004,"15min":0.67800000000000005},"min":{"1min":0.41699999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.80300000000000005,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.66000000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9688,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":8896,"kb_avail":94362152,"statfs":{"total":96636764160,"available":96626843648,"internally_reserved":0,"allocated":802816,"data_stored":375218,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9109504},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.503,"15min":0.503},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.69899999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.54600000000000004,"15min":0.55800000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.82099999999999995,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.44800000000000001}]},{"osd":1,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.64400000000000002,"15min":0.66600000000000004},"min":{"1min":0.42199999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73399999999999999,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.621},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.64000000000000001,"15min":0.66900000000000004},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.71699999999999997,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.40500000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.71999999999999997,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.372,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.58399999999999996}]},{"osd":3,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.63400000000000001,"15min":0.67800000000000005},"min":{"1min":0.38800000000000001,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.78700000000000003,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.58699999999999997,"15min":0.58899999999999997},"min":{"1min":0.42099999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.77000000000000002,"5min":1.042,"15min":1.042},"last":0.502}]},{"osd":4,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.53400000000000003,"15min":0.54400000000000004},"min":{"1min":0.37,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.68100000000000005,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.46700000000000003,"5min":0.314,"15min":0.314},"max":{"1min":0.73999999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.59999999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.39300000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71299999999999997,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.47799999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.72099999999999997,"5min":1.036,"15min":1.036},"last":0.48199999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:11:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.66400000000000003,"15min":0.68999999999999995},"min":{"1min":0.39500000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.97299999999999998,"5min":1.329,"15min":1.329},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.623,"5min":0.66600000000000004,"15min":0.68899999999999995},"min":{"1min":0.437,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.75800000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.47099999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2148,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:12:12.895 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T11:12:12.896 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw pool=cephfs_data' mds allow 2023-11-28T11:12:13.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.059+0000 7fe57031b700 1 Processor -- start 2023-11-28T11:12:13.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.060+0000 7fe57031b700 1 -- start start 2023-11-28T11:12:13.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.060+0000 7fe57031b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe568165670 0x7fe568167a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.060+0000 7fe57031b700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5680ce550 con 0x7fe5680cf5f0 2023-11-28T11:12:13.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe57031b700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5680ce830 con 0x7fe5680b1630 2023-11-28T11:12:13.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe57031b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680ceb10 con 0x7fe568165670 2023-11-28T11:12:13.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d8b6700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe5680cf5f0 0x7fe5680caea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39738/0 (socket says 172.21.15.28:39738) 2023-11-28T11:12:13.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d8b6700 1 -- 172.21.15.28:0/269123609 learned_addr learned my addr 172.21.15.28:0/269123609 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:13.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56e8b8700 1 --2- 172.21.15.28:0/269123609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe568165670 0x7fe568167a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 286094482 0 0) 0x7fe5680ce550 con 0x7fe5680cf5f0 2023-11-28T11:12:13.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5540035b0 con 0x7fe5680cf5f0 2023-11-28T11:12:13.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 627464058 0 0) 0x7fe5680ce830 con 0x7fe5680b1630 2023-11-28T11:12:13.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5680ce550 con 0x7fe5680b1630 2023-11-28T11:12:13.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1858394494 0 0) 0x7fe5540035b0 con 0x7fe5680cf5f0 2023-11-28T11:12:13.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe568165670 msgr2=0x7fe568167a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:13.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 --2- 172.21.15.28:0/269123609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe568165670 0x7fe568167a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 >> v1:172.21.15.59:6789/0 conn(0x7fe5680b1630 legacy=0x7fe5680b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5680cedf0 con 0x7fe5680cf5f0 2023-11-28T11:12:13.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fe564001670 con 0x7fe5680cf5f0 2023-11-28T11:12:13.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 >> v1:172.21.15.28:6789/0 conn(0x7fe5680cf5f0 legacy=0x7fe5680caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 --2- 172.21.15.28:0/269123609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe568165670 0x7fe554003bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 --2- 172.21.15.28:0/269123609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe554004120 0x7fe554004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 --2- 172.21.15.28:0/269123609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe554004d90 0x7fe554005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680ceb10 con 0x7fe554004d90 2023-11-28T11:12:13.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55400bac0 con 0x7fe554004120 2023-11-28T11:12:13.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe55400bc80 con 0x7fe568165670 2023-11-28T11:12:13.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.061+0000 7fe56e8b8700 1 --2- 172.21.15.28:0/269123609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe554004d90 0x7fe554005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.062+0000 7fe56d8b6700 1 --2- 172.21.15.28:0/269123609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe554004120 0x7fe554004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.062+0000 7fe56e0b7700 1 --2- 172.21.15.28:0/269123609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe568165670 0x7fe554003bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.062+0000 7fe56e8b8700 1 -- 172.21.15.28:0/269123609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe568165670 msgr2=0x7fe554003bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.062+0000 7fe56e8b8700 1 --2- 172.21.15.28:0/269123609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe568165670 0x7fe554003bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.062+0000 7fe56e8b8700 1 -- 172.21.15.28:0/269123609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe554004120 msgr2=0x7fe554004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.062+0000 7fe56e8b8700 1 --2- 172.21.15.28:0/269123609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe554004120 0x7fe554004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.062+0000 7fe56e8b8700 1 -- 172.21.15.28:0/269123609 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5540035b0 con 0x7fe554004d90 2023-11-28T11:12:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.062+0000 7fe56e0b7700 1 --2- 172.21.15.28:0/269123609 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe568165670 0x7fe554003bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.062+0000 7fe56e8b8700 1 --2- 172.21.15.28:0/269123609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe554004d90 0x7fe554005200 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe56806eb30 tx=0x7fe554005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eac9340ed3e9df8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.062+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe56000eba0 con 0x7fe554004d90 2023-11-28T11:12:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.062+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe56000f040 con 0x7fe554004d90 2023-11-28T11:12:13.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.062+0000 7fe56d0b5700 1 -- 172.21.15.28:0/269123609 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe560014480 con 0x7fe554004d90 2023-11-28T11:12:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.063+0000 7fe57031b700 1 -- 172.21.15.28:0/269123609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe554004d90 msgr2=0x7fe554005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.063+0000 7fe57031b700 1 --2- 172.21.15.28:0/269123609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe554004d90 0x7fe554005200 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7fe56806eb30 tx=0x7fe554005880 comp rx=0 tx=0).stop 2023-11-28T11:12:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.063+0000 7fe56e8b8700 1 -- 172.21.15.28:0/269123609 reap_dead start 2023-11-28T11:12:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.063+0000 7fe57031b700 1 -- 172.21.15.28:0/269123609 shutdown_connections 2023-11-28T11:12:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.063+0000 7fe57031b700 1 -- 172.21.15.28:0/269123609 >> 172.21.15.28:0/269123609 conn(0x7fe5680cc630 msgr2=0x7fe5680cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.063+0000 7fe57031b700 1 -- 172.21.15.28:0/269123609 shutdown_connections 2023-11-28T11:12:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.063+0000 7fe57031b700 1 -- 172.21.15.28:0/269123609 wait complete. 2023-11-28T11:12:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.064+0000 7fe57031b700 1 Processor -- start 2023-11-28T11:12:13.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.064+0000 7fe57031b700 1 -- start start 2023-11-28T11:12:13.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe57031b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe554004d90 0x7fe5680b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe57031b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe568157800 0x7fe5680b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe57031b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe568150980 0x7fe568150e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe57031b700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680b5800 con 0x7fe554004d90 2023-11-28T11:12:13.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe57031b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680b4e60 con 0x7fe568150980 2023-11-28T11:12:13.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe57031b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5680b5140 con 0x7fe568157800 2023-11-28T11:12:13.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe56e0b7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe554004d90 0x7fe5680b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe56e0b7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe554004d90 0x7fe5680b0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51692/0 (socket says 172.21.15.28:51692) 2023-11-28T11:12:13.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe56e0b7700 1 -- 172.21.15.28:0/3292371236 learned_addr learned my addr 172.21.15.28:0/3292371236 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:13.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe56e0b7700 1 -- 172.21.15.28:0/3292371236 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe568157800 msgr2=0x7fe5680b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe56e0b7700 1 --2- 172.21.15.28:0/3292371236 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe568157800 0x7fe5680b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe56e0b7700 1 -- 172.21.15.28:0/3292371236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe568150980 msgr2=0x7fe568150e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe56e0b7700 1 --2- 172.21.15.28:0/3292371236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe568150980 0x7fe568150e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe56e0b7700 1 -- 172.21.15.28:0/3292371236 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5680cb5b0 con 0x7fe554004d90 2023-11-28T11:12:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe56e0b7700 1 --2- 172.21.15.28:0/3292371236 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe554004d90 0x7fe5680b0640 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe55800e700 tx=0x7fe55800eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.065+0000 7fe55effd700 1 -- 172.21.15.28:0/3292371236 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558017070 con 0x7fe554004d90 2023-11-28T11:12:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.066+0000 7fe55effd700 1 -- 172.21.15.28:0/3292371236 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5580052e0 con 0x7fe554004d90 2023-11-28T11:12:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.066+0000 7fe57031b700 1 -- 172.21.15.28:0/3292371236 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5680cf0d0 con 0x7fe554004d90 2023-11-28T11:12:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.066+0000 7fe55effd700 1 -- 172.21.15.28:0/3292371236 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe558005450 con 0x7fe554004d90 2023-11-28T11:12:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.066+0000 7fe57031b700 1 -- 172.21.15.28:0/3292371236 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5680ce140 con 0x7fe554004d90 2023-11-28T11:12:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.067+0000 7fe57031b700 1 -- 172.21.15.28:0/3292371236 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5680739f0 con 0x7fe554004d90 2023-11-28T11:12:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.067+0000 7fe55effd700 1 -- 172.21.15.28:0/3292371236 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe558006860 con 0x7fe554004d90 2023-11-28T11:12:13.073 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.067+0000 7fe55effd700 1 --2- 172.21.15.28:0/3292371236 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe54c0682f0 0x7fe54c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.068+0000 7fe56d8b6700 1 --2- 172.21.15.28:0/3292371236 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe54c0682f0 0x7fe54c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.068+0000 7fe55effd700 1 -- 172.21.15.28:0/3292371236 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7fe5580857e0 con 0x7fe554004d90 2023-11-28T11:12:13.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.068+0000 7fe56d8b6700 1 --2- 172.21.15.28:0/3292371236 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe54c0682f0 0x7fe54c06a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe5640011e0 tx=0x7fe564001130 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.074 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.072+0000 7fe55effd700 1 -- 172.21.15.28:0/3292371236 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe558051d90 con 0x7fe554004d90 2023-11-28T11:12:13.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.346+0000 7fe57031b700 1 -- 172.21.15.28:0/3292371236 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]} v 0) v1 -- 0x7fe5680cac80 con 0x7fe554004d90 2023-11-28T11:12:13.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.349+0000 7fe55effd700 1 -- 172.21.15.28:0/3292371236 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw pool=cephfs_data", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 184+0+0 (secure 0 0 0) 0x7fe5580553b0 con 0x7fe554004d90 2023-11-28T11:12:13.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.351+0000 7fe57031b700 1 -- 172.21.15.28:0/3292371236 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe54c0682f0 msgr2=0x7fe54c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.351+0000 7fe57031b700 1 --2- 172.21.15.28:0/3292371236 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe54c0682f0 0x7fe54c06a7a0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fe5640011e0 tx=0x7fe564001130 comp rx=0 tx=0).stop 2023-11-28T11:12:13.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.351+0000 7fe57031b700 1 -- 172.21.15.28:0/3292371236 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe554004d90 msgr2=0x7fe5680b0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.351+0000 7fe57031b700 1 --2- 172.21.15.28:0/3292371236 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe554004d90 0x7fe5680b0640 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7fe55800e700 tx=0x7fe55800eac0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.351+0000 7fe57031b700 1 -- 172.21.15.28:0/3292371236 shutdown_connections 2023-11-28T11:12:13.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.351+0000 7fe57031b700 1 --2- 172.21.15.28:0/3292371236 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe54c0682f0 0x7fe54c06a7a0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.351+0000 7fe57031b700 1 --2- 172.21.15.28:0/3292371236 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe554004d90 0x7fe5680b0640 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.351+0000 7fe57031b700 1 --2- 172.21.15.28:0/3292371236 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe568150980 0x7fe568150e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.351+0000 7fe57031b700 1 --2- 172.21.15.28:0/3292371236 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe568157800 0x7fe5680b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.351+0000 7fe57031b700 1 -- 172.21.15.28:0/3292371236 >> 172.21.15.28:0/3292371236 conn(0x7fe5680cc630 msgr2=0x7fe5680cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:13.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.351+0000 7fe57031b700 1 -- 172.21.15.28:0/3292371236 shutdown_connections 2023-11-28T11:12:13.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.351+0000 7fe57031b700 1 -- 172.21.15.28:0/3292371236 wait complete. 2023-11-28T11:12:13.355 INFO:teuthology.orchestra.run.smithi028.stderr:updated caps for client.0 2023-11-28T11:12:13.365 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:12:13.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.489+0000 7f6dd5cb1700 1 Processor -- start 2023-11-28T11:12:13.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.489+0000 7f6dd5cb1700 1 -- start start 2023-11-28T11:12:13.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.489+0000 7f6dd5cb1700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6dd00cb0a0 0x7f6dd00ca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.489+0000 7f6dd5cb1700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd006c140 con 0x7f6dd00cb470 2023-11-28T11:12:13.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.490+0000 7f6dd5cb1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dd00c9ab0 con 0x7f6dd006bb80 2023-11-28T11:12:13.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.490+0000 7f6dd5cb1700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dd00caad0 con 0x7f6dd00cb0a0 2023-11-28T11:12:13.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.490+0000 7f6dcf7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f6dd00cb0a0 0x7f6dd00ca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.490+0000 7f6dceffd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6dd00cb470 0x7f6dd015a3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39760/0 (socket says 172.21.15.28:39760) 2023-11-28T11:12:13.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.490+0000 7f6dceffd700 1 -- 172.21.15.28:0/1286475649 learned_addr learned my addr 172.21.15.28:0/1286475649 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:13.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.490+0000 7f6dcf7fe700 1 -- 172.21.15.28:0/1286475649 >> v1:172.21.15.59:6789/0 conn(0x7f6dd006bb80 legacy=0x7f6dd014fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.490+0000 7f6dcf7fe700 1 -- 172.21.15.28:0/1286475649 >> v1:172.21.15.28:6789/0 conn(0x7f6dd00cb470 legacy=0x7f6dd015a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.490+0000 7f6dce7fc700 1 -- 172.21.15.28:0/1286475649 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4094375523 0 0) 0x7f6dd006c140 con 0x7f6dd00cb470 2023-11-28T11:12:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.490+0000 7f6dcf7fe700 1 -- 172.21.15.28:0/1286475649 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd00c9d90 con 0x7f6dd00cb0a0 2023-11-28T11:12:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.490+0000 7f6dcf7fe700 1 --2- 172.21.15.28:0/1286475649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd00cb0a0 0x7f6dd00ca590 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009840 tx=0x7f6dc0009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6bec74cf33a14c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.490+0000 7f6dce7fc700 1 -- 172.21.15.28:0/1286475649 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0004230 con 0x7f6dd00cb0a0 2023-11-28T11:12:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.490+0000 7f6dce7fc700 1 -- 172.21.15.28:0/1286475649 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc0004390 con 0x7f6dd00cb0a0 2023-11-28T11:12:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.490+0000 7f6dce7fc700 1 -- 172.21.15.28:0/1286475649 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc00036a0 con 0x7f6dd00cb0a0 2023-11-28T11:12:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.491+0000 7f6dd5cb1700 1 -- 172.21.15.28:0/1286475649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd00cb0a0 msgr2=0x7f6dd00ca590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.491+0000 7f6dd5cb1700 1 --2- 172.21.15.28:0/1286475649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd00cb0a0 0x7f6dd00ca590 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f6dc0009840 tx=0x7f6dc0009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.491+0000 7f6dd5cb1700 1 -- 172.21.15.28:0/1286475649 shutdown_connections 2023-11-28T11:12:13.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.491+0000 7f6dd5cb1700 1 --2- 172.21.15.28:0/1286475649 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd00cb0a0 0x7f6dd00ca590 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.491+0000 7f6dd5cb1700 1 -- 172.21.15.28:0/1286475649 >> 172.21.15.28:0/1286475649 conn(0x7f6dd00b0dd0 msgr2=0x7f6dd00b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:13.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.491+0000 7f6dd5cb1700 1 -- 172.21.15.28:0/1286475649 shutdown_connections 2023-11-28T11:12:13.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.491+0000 7f6dd5cb1700 1 -- 172.21.15.28:0/1286475649 wait complete. 2023-11-28T11:12:13.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.492+0000 7f6dd5cb1700 1 Processor -- start 2023-11-28T11:12:13.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.492+0000 7f6dd5cb1700 1 -- start start 2023-11-28T11:12:13.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dd5cb1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd006bb80 0x7f6dd00cd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dd5cb1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dd00cb0a0 0x7f6dd00ce590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dd5cb1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6dd00cb470 0x7f6dd014eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dd5cb1700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd01e4950 con 0x7f6dd006bb80 2023-11-28T11:12:13.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dd5cb1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd00cde20 con 0x7f6dd00cb0a0 2023-11-28T11:12:13.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dd5cb1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dd01e3cd0 con 0x7f6dd00cb470 2023-11-28T11:12:13.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dcf7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd006bb80 0x7f6dd00cd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dcf7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd006bb80 0x7f6dd00cd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51708/0 (socket says 172.21.15.28:51708) 2023-11-28T11:12:13.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dcf7fe700 1 -- 172.21.15.28:0/3083443538 learned_addr learned my addr 172.21.15.28:0/3083443538 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:13.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dcffff700 1 --2- 172.21.15.28:0/3083443538 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6dd00cb470 0x7f6dd014eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dceffd700 1 --2- 172.21.15.28:0/3083443538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dd00cb0a0 0x7f6dd00ce590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dcf7fe700 1 -- 172.21.15.28:0/3083443538 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6dd00cb470 msgr2=0x7f6dd014eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.498 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dcf7fe700 1 --2- 172.21.15.28:0/3083443538 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6dd00cb470 0x7f6dd014eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dcf7fe700 1 -- 172.21.15.28:0/3083443538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dd00cb0a0 msgr2=0x7f6dd00ce590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dcf7fe700 1 --2- 172.21.15.28:0/3083443538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dd00cb0a0 0x7f6dd00ce590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dcf7fe700 1 -- 172.21.15.28:0/3083443538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dc00094a0 con 0x7f6dd006bb80 2023-11-28T11:12:13.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.493+0000 7f6dceffd700 1 --2- 172.21.15.28:0/3083443538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dd00cb0a0 0x7f6dd00ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:13.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.494+0000 7f6dcf7fe700 1 --2- 172.21.15.28:0/3083443538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd006bb80 0x7f6dd00cd8e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00090b0 tx=0x7f6dc0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.494+0000 7f6db7fff700 1 -- 172.21.15.28:0/3083443538 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0011630 con 0x7f6dd006bb80 2023-11-28T11:12:13.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.494+0000 7f6db7fff700 1 -- 172.21.15.28:0/3083443538 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dc001e070 con 0x7f6dd006bb80 2023-11-28T11:12:13.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.494+0000 7f6db7fff700 1 -- 172.21.15.28:0/3083443538 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dc0011990 con 0x7f6dd006bb80 2023-11-28T11:12:13.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.494+0000 7f6dd5cb1700 1 -- 172.21.15.28:0/3083443538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dd01505b0 con 0x7f6dd006bb80 2023-11-28T11:12:13.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.494+0000 7f6dd5cb1700 1 -- 172.21.15.28:0/3083443538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dd00ca070 con 0x7f6dd006bb80 2023-11-28T11:12:13.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.495+0000 7f6dd5cb1700 1 -- 172.21.15.28:0/3083443538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dd0150740 con 0x7f6dd006bb80 2023-11-28T11:12:13.499 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.495+0000 7f6db7fff700 1 -- 172.21.15.28:0/3083443538 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6dc0017020 con 0x7f6dd006bb80 2023-11-28T11:12:13.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.496+0000 7f6db7fff700 1 --2- 172.21.15.28:0/3083443538 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6dbc0682f0 0x7f6dbc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:13.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.496+0000 7f6db7fff700 1 -- 172.21.15.28:0/3083443538 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f6dc00871b0 con 0x7f6dd006bb80 2023-11-28T11:12:13.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.496+0000 7f6dceffd700 1 --2- 172.21.15.28:0/3083443538 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6dbc0682f0 0x7f6dbc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:13.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.496+0000 7f6dceffd700 1 --2- 172.21.15.28:0/3083443538 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6dbc0682f0 0x7f6dbc06a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4000f60 tx=0x7f6dc4009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:13.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.500+0000 7f6db7fff700 1 -- 172.21.15.28:0/3083443538 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6dc00538d0 con 0x7f6dd006bb80 2023-11-28T11:12:13.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.736+0000 7f6dd5cb1700 1 -- 172.21.15.28:0/3083443538 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6dd01e4a90 con 0x7f6dd006bb80 2023-11-28T11:12:13.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.737+0000 7f6db7fff700 1 -- 172.21.15.28:0/3083443538 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f6dc0056ef0 con 0x7f6dd006bb80 2023-11-28T11:12:13.739 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:13.739 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1564028052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1564028052},{"type":"v1","addr":"172.21.15.59:6835","nonce":1564028052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6558,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2509041002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2509041002},{"type":"v1","addr":"172.21.15.28:6836","nonce":2509041002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6561,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3994419079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3994419079},{"type":"v1","addr":"172.21.15.28:6837","nonce":3994419079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:12:03.800884+0000","modified":"2023-11-28T11:12:06.812004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6834/2406852565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2406852565},{"type":"v1","addr":"172.21.15.59:6834","nonce":2406852565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:12:13.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.739+0000 7f6dd5cb1700 1 -- 172.21.15.28:0/3083443538 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6dbc0682f0 msgr2=0x7f6dbc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.739+0000 7f6dd5cb1700 1 --2- 172.21.15.28:0/3083443538 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6dbc0682f0 0x7f6dbc06a7a0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4000f60 tx=0x7f6dc4009040 comp rx=0 tx=0).stop 2023-11-28T11:12:13.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.739+0000 7f6dd5cb1700 1 -- 172.21.15.28:0/3083443538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd006bb80 msgr2=0x7f6dd00cd8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:13.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.739+0000 7f6dd5cb1700 1 --2- 172.21.15.28:0/3083443538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd006bb80 0x7f6dd00cd8e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f6dc00090b0 tx=0x7f6dc0004750 comp rx=0 tx=0).stop 2023-11-28T11:12:13.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.739+0000 7f6dd5cb1700 1 -- 172.21.15.28:0/3083443538 shutdown_connections 2023-11-28T11:12:13.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.739+0000 7f6dd5cb1700 1 --2- 172.21.15.28:0/3083443538 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6dbc0682f0 0x7f6dbc06a7a0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.739+0000 7f6dd5cb1700 1 --2- 172.21.15.28:0/3083443538 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6dd006bb80 0x7f6dd00cd8e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.739+0000 7f6dd5cb1700 1 --2- 172.21.15.28:0/3083443538 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6dd00cb470 0x7f6dd014eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.739+0000 7f6dd5cb1700 1 --2- 172.21.15.28:0/3083443538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6dd00cb0a0 0x7f6dd00ce590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:13.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.739+0000 7f6dd5cb1700 1 -- 172.21.15.28:0/3083443538 >> 172.21.15.28:0/3083443538 conn(0x7f6dd00b0dd0 msgr2=0x7f6dd00b05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:13.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.739+0000 7f6dd5cb1700 1 -- 172.21.15.28:0/3083443538 shutdown_connections 2023-11-28T11:12:13.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:13.739+0000 7f6dd5cb1700 1 -- 172.21.15.28:0/3083443538 wait complete. 2023-11-28T11:12:13.742 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 33 2023-11-28T11:12:14.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 33, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T11:12:03.800884+0000', 'modified': '2023-11-28T11:12:06.812004+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6573}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6573': {'gid': 6573, 'name': 'd', 'rank': 0, 'incarnation': 29, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.59:6834/2406852565', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6832', 'nonce': 2406852565}, {'type': 'v1', 'addr': '172.21.15.59:6834', 'nonce': 2406852565}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-11-28T11:12:14.158 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T11:12:14.159 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T11:12:14.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.295+0000 7f72f4528700 1 Processor -- start 2023-11-28T11:12:14.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.295+0000 7f72f4528700 1 -- start start 2023-11-28T11:12:14.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.296+0000 7f72f4528700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72ec06a560 0x7f72ec06a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.296+0000 7f72f4528700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72ec0cdec0 con 0x7f72ec0cceb0 2023-11-28T11:12:14.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.296+0000 7f72f4528700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72ec0c9cd0 con 0x7f72ec15d1e0 2023-11-28T11:12:14.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.296+0000 7f72f4528700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec06af10 con 0x7f72ec06a560 2023-11-28T11:12:14.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f2ac5700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f72ec0cceb0 0x7f72ec0cd330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39776/0 (socket says 172.21.15.28:39776) 2023-11-28T11:12:14.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f2ac5700 1 -- 172.21.15.28:0/734349461 learned_addr learned my addr 172.21.15.28:0/734349461 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:14.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/734349461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72ec06a560 0x7f72ec06a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.301 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f12c2700 1 -- 172.21.15.28:0/734349461 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2264562295 0 0) 0x7f72ec0cdec0 con 0x7f72ec0cceb0 2023-11-28T11:12:14.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f12c2700 1 -- 172.21.15.28:0/734349461 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72d80035b0 con 0x7f72ec0cceb0 2023-11-28T11:12:14.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f12c2700 1 -- 172.21.15.28:0/734349461 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2377640208 0 0) 0x7f72d80035b0 con 0x7f72ec0cceb0 2023-11-28T11:12:14.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f12c2700 1 -- 172.21.15.28:0/734349461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72ec06a560 msgr2=0x7f72ec06a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f12c2700 1 --2- 172.21.15.28:0/734349461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72ec06a560 0x7f72ec06a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f12c2700 1 -- 172.21.15.28:0/734349461 >> v1:172.21.15.59:6789/0 conn(0x7f72ec15d1e0 legacy=0x7f72ec15d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f12c2700 1 -- 172.21.15.28:0/734349461 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec0c9fb0 con 0x7f72ec0cceb0 2023-11-28T11:12:14.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f12c2700 1 -- 172.21.15.28:0/734349461 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f72e40019f0 con 0x7f72ec0cceb0 2023-11-28T11:12:14.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f12c2700 1 -- 172.21.15.28:0/734349461 >> v1:172.21.15.28:6789/0 conn(0x7f72ec0cceb0 legacy=0x7f72ec0cd330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f12c2700 1 --2- 172.21.15.28:0/734349461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72d8003a30 0x7f72d8003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f12c2700 1 --2- 172.21.15.28:0/734349461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72d80043d0 0x7f72d8004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f12c2700 1 --2- 172.21.15.28:0/734349461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72ec06a560 0x7f72d80050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f12c2700 1 -- 172.21.15.28:0/734349461 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec06af10 con 0x7f72d80043d0 2023-11-28T11:12:14.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f12c2700 1 -- 172.21.15.28:0/734349461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72d800ba10 con 0x7f72d8003a30 2023-11-28T11:12:14.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f12c2700 1 -- 172.21.15.28:0/734349461 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72d800bba0 con 0x7f72ec06a560 2023-11-28T11:12:14.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.297+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/734349461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72d80043d0 0x7f72d8004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.298+0000 7f72f22c4700 1 --2- 172.21.15.28:0/734349461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72d8003a30 0x7f72d8003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.298+0000 7f72f2ac5700 1 --2- 172.21.15.28:0/734349461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72ec06a560 0x7f72d80050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.298+0000 7f72f1ac3700 1 -- 172.21.15.28:0/734349461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72ec06a560 msgr2=0x7f72d80050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.298+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/734349461 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72ec06a560 0x7f72d80050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.298+0000 7f72f1ac3700 1 -- 172.21.15.28:0/734349461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72d8003a30 msgr2=0x7f72d8003e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.305 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.298+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/734349461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72d8003a30 0x7f72d8003e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.298+0000 7f72f1ac3700 1 -- 172.21.15.28:0/734349461 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72d80035b0 con 0x7f72d80043d0 2023-11-28T11:12:14.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.298+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/734349461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72d80043d0 0x7f72d8004890 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009b60 tx=0x7f72d8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a021a0f9ff1ae610 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.298+0000 7f72f12c2700 1 -- 172.21.15.28:0/734349461 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e800e6c0 con 0x7f72d80043d0 2023-11-28T11:12:14.306 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.298+0000 7f72f12c2700 1 -- 172.21.15.28:0/734349461 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e800e820 con 0x7f72d80043d0 2023-11-28T11:12:14.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.298+0000 7f72f12c2700 1 -- 172.21.15.28:0/734349461 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e800eb90 con 0x7f72d80043d0 2023-11-28T11:12:14.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.299+0000 7f72f4528700 1 -- 172.21.15.28:0/734349461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72d80043d0 msgr2=0x7f72d8004890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.299+0000 7f72f4528700 1 --2- 172.21.15.28:0/734349461 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72d80043d0 0x7f72d8004890 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f72e8009b60 tx=0x7f72d8005880 comp rx=0 tx=0).stop 2023-11-28T11:12:14.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.299+0000 7f72f2ac5700 1 -- 172.21.15.28:0/734349461 reap_dead start 2023-11-28T11:12:14.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.299+0000 7f72f4528700 1 -- 172.21.15.28:0/734349461 shutdown_connections 2023-11-28T11:12:14.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.299+0000 7f72f4528700 1 -- 172.21.15.28:0/734349461 >> 172.21.15.28:0/734349461 conn(0x7f72ec0b0dd0 msgr2=0x7f72ec0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.299+0000 7f72f4528700 1 -- 172.21.15.28:0/734349461 shutdown_connections 2023-11-28T11:12:14.307 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.299+0000 7f72f4528700 1 -- 172.21.15.28:0/734349461 wait complete. 2023-11-28T11:12:14.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.300+0000 7f72f4528700 1 Processor -- start 2023-11-28T11:12:14.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.300+0000 7f72f4528700 1 -- start start 2023-11-28T11:12:14.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f4528700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72d80043d0 0x7f72ec155770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f4528700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72ec159df0 0x7f72ec158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f4528700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72ec1f63f0 0x7f72ec06e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f4528700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec159590 con 0x7f72d80043d0 2023-11-28T11:12:14.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f4528700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec1592b0 con 0x7f72ec159df0 2023-11-28T11:12:14.316 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f4528700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec1f5910 con 0x7f72ec1f63f0 2023-11-28T11:12:14.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f22c4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72d80043d0 0x7f72ec155770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f22c4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72d80043d0 0x7f72ec155770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51726/0 (socket says 172.21.15.28:51726) 2023-11-28T11:12:14.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f22c4700 1 -- 172.21.15.28:0/2753516616 learned_addr learned my addr 172.21.15.28:0/2753516616 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:14.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f22c4700 1 -- 172.21.15.28:0/2753516616 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72ec1f63f0 msgr2=0x7f72ec06e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f2ac5700 1 --2- 172.21.15.28:0/2753516616 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72ec1f63f0 0x7f72ec06e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/2753516616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72ec159df0 0x7f72ec158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f22c4700 1 --2- 172.21.15.28:0/2753516616 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72ec1f63f0 0x7f72ec06e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f22c4700 1 -- 172.21.15.28:0/2753516616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72ec159df0 msgr2=0x7f72ec158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f22c4700 1 --2- 172.21.15.28:0/2753516616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72ec159df0 0x7f72ec158940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f22c4700 1 -- 172.21.15.28:0/2753516616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec0ca570 con 0x7f72d80043d0 2023-11-28T11:12:14.317 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.301+0000 7f72f22c4700 1 --2- 172.21.15.28:0/2753516616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72d80043d0 0x7f72ec155770 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00f780 tx=0x7f72dc00fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.302+0000 7f72e2ffd700 1 -- 172.21.15.28:0/2753516616 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0052e0 con 0x7f72d80043d0 2023-11-28T11:12:14.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.302+0000 7f72e2ffd700 1 -- 172.21.15.28:0/2753516616 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc005460 con 0x7f72d80043d0 2023-11-28T11:12:14.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.302+0000 7f72f4528700 1 -- 172.21.15.28:0/2753516616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ec0ca290 con 0x7f72d80043d0 2023-11-28T11:12:14.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.302+0000 7f72e2ffd700 1 -- 172.21.15.28:0/2753516616 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc0038c0 con 0x7f72d80043d0 2023-11-28T11:12:14.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.302+0000 7f72f4528700 1 -- 172.21.15.28:0/2753516616 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec0ca850 con 0x7f72d80043d0 2023-11-28T11:12:14.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.303+0000 7f72e2ffd700 1 -- 172.21.15.28:0/2753516616 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f72dc003a20 con 0x7f72d80043d0 2023-11-28T11:12:14.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.303+0000 7f72e2ffd700 1 --2- 172.21.15.28:0/2753516616 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f72d00682f0 0x7f72d006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.304+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/2753516616 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f72d00682f0 0x7f72d006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.304+0000 7f72e2ffd700 1 -- 172.21.15.28:0/2753516616 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f72dc085e30 con 0x7f72d80043d0 2023-11-28T11:12:14.318 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.304+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/2753516616 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f72d00682f0 0x7f72d006a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0ac940 tx=0x7f72e8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.305+0000 7f73006b1b80 1 Processor -- start 2023-11-28T11:12:14.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.306+0000 7f73006b1b80 1 -- start start 2023-11-28T11:12:14.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f73006b1b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec072fa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f73006b1b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec0757130 0x558ec0759be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f73006b1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec075a120 0x558ec0775770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f73006b1b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec0619bc0 con 0x558ec0759810 2023-11-28T11:12:14.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f73006b1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec0688350 con 0x558ec075a120 2023-11-28T11:12:14.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f73006b1b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ec0681880 con 0x558ec0757130 2023-11-28T11:12:14.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72f22c4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec072fa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72f22c4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec072fa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51734/0 (socket says 172.21.15.28:51734) 2023-11-28T11:12:14.319 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72f22c4700 1 -- 172.21.15.28:0/1462570425 learned_addr learned my addr 172.21.15.28:0/1462570425 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:14.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/1462570425 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec0757130 0x558ec0759be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72f2ac5700 1 --2- 172.21.15.28:0/1462570425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec075a120 0x558ec0775770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72f22c4700 1 -- 172.21.15.28:0/1462570425 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec0757130 msgr2=0x558ec0759be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72f22c4700 1 --2- 172.21.15.28:0/1462570425 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec0757130 0x558ec0759be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72f22c4700 1 -- 172.21.15.28:0/1462570425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec075a120 msgr2=0x558ec0775770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72f22c4700 1 --2- 172.21.15.28:0/1462570425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec075a120 0x558ec0775770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72f22c4700 1 -- 172.21.15.28:0/1462570425 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ec05035e0 con 0x558ec0759810 2023-11-28T11:12:14.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/1462570425 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec0757130 0x558ec0759be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:14.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72f2ac5700 1 --2- 172.21.15.28:0/1462570425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec075a120 0x558ec0775770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:14.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72f22c4700 1 --2- 172.21.15.28:0/1462570425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec072fa20 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f72dc000fb0 tx=0x7f72dc055af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cad13225ad49bba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.320 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72cffff700 1 -- 172.21.15.28:0/1462570425 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc055680 con 0x558ec0759810 2023-11-28T11:12:14.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72cffff700 1 -- 172.21.15.28:0/1462570425 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72dc0557e0 con 0x558ec0759810 2023-11-28T11:12:14.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.307+0000 7f72cffff700 1 -- 172.21.15.28:0/1462570425 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc055950 con 0x558ec0759810 2023-11-28T11:12:14.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.308+0000 7f73006b1b80 1 -- 172.21.15.28:0/1462570425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 msgr2=0x558ec072fa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.308+0000 7f73006b1b80 1 --2- 172.21.15.28:0/1462570425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec072fa20 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7f72dc000fb0 tx=0x7f72dc055af0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.308+0000 7f73006b1b80 1 -- 172.21.15.28:0/1462570425 shutdown_connections 2023-11-28T11:12:14.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.308+0000 7f73006b1b80 1 --2- 172.21.15.28:0/1462570425 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec072fa20 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.308+0000 7f73006b1b80 1 --2- 172.21.15.28:0/1462570425 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec075a120 0x558ec0775770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.308+0000 7f73006b1b80 1 --2- 172.21.15.28:0/1462570425 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec0757130 0x558ec0759be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.308+0000 7f73006b1b80 1 -- 172.21.15.28:0/1462570425 >> 172.21.15.28:0/1462570425 conn(0x558ec072cd60 msgr2=0x558ec072d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.308+0000 7f73006b1b80 1 -- 172.21.15.28:0/1462570425 shutdown_connections 2023-11-28T11:12:14.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.308+0000 7f73006b1b80 1 -- 172.21.15.28:0/1462570425 wait complete. 2023-11-28T11:12:14.321 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.309+0000 7f73006b1b80 1 -- start start 2023-11-28T11:12:14.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.309+0000 7f73006b1b80 1 Processor -- start 2023-11-28T11:12:14.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f73006b1b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0757130 0x558ec074fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f73006b1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec0759810 0x558ec074b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f73006b1b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec075a120 0x558ec076b580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f73006b1b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec05058d0 con 0x558ec0757130 2023-11-28T11:12:14.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f73006b1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec074ff40 con 0x558ec0759810 2023-11-28T11:12:14.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f73006b1b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ec07586f0 con 0x558ec075a120 2023-11-28T11:12:14.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f73006b1b80 10 client.0 _ll_register_callbacks cb 0x558ec0527240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:12:14.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f72f22c4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0757130 0x558ec074fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f72f22c4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0757130 0x558ec074fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51740/0 (socket says 172.21.15.28:51740) 2023-11-28T11:12:14.322 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f72f22c4700 1 -- 172.21.15.28:0/3986768338 learned_addr learned my addr 172.21.15.28:0/3986768338 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:14.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f72f2ac5700 1 --2- 172.21.15.28:0/3986768338 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec075a120 0x558ec076b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/3986768338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec0759810 0x558ec074b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f72f22c4700 1 -- 172.21.15.28:0/3986768338 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec075a120 msgr2=0x558ec076b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f72f22c4700 1 --2- 172.21.15.28:0/3986768338 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec075a120 0x558ec076b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f72f22c4700 1 -- 172.21.15.28:0/3986768338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec0759810 msgr2=0x558ec074b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f72f22c4700 1 --2- 172.21.15.28:0/3986768338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec0759810 0x558ec074b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f72f22c4700 1 -- 172.21.15.28:0/3986768338 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec0ca850 con 0x558ec0757130 2023-11-28T11:12:14.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f72f22c4700 1 --2- 172.21.15.28:0/3986768338 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0757130 0x558ec074fa00 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f72dc056070 tx=0x7f72dc001500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b791c4e0aa0607 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.310+0000 7f72cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:12:14.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.311+0000 7f72cd7fa700 1 -- 172.21.15.28:0/3986768338 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc05df90 con 0x558ec0757130 2023-11-28T11:12:14.323 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.311+0000 7f72cd7fa700 1 -- 172.21.15.28:0/3986768338 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f72dc051f80 con 0x558ec0757130 2023-11-28T11:12:14.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.311+0000 7f73006b1b80 1 -- 172.21.15.28:0/3986768338 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ec064f140 con 0x558ec0757130 2023-11-28T11:12:14.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.311+0000 7f72cd7fa700 1 -- 172.21.15.28:0/3986768338 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72dc052140 con 0x558ec0757130 2023-11-28T11:12:14.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.311+0000 7f73006b1b80 10 client.6747 fetch_fsmap learned FSMap version 33 2023-11-28T11:12:14.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.311+0000 7f73006b1b80 1 -- 172.21.15.28:0/3986768338 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x558ec04afd90 con 0x558ec0757130 2023-11-28T11:12:14.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.311+0000 7f72cd7fa700 1 -- 172.21.15.28:0/3986768338 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558ec064f140 con 0x558ec0757130 2023-11-28T11:12:14.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.311+0000 7f72cd7fa700 1 -- 172.21.15.28:0/3986768338 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f72dc0550f0 con 0x558ec0757130 2023-11-28T11:12:14.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.311+0000 7f73006b1b80 10 client.6747 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:12:14.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.311+0000 7f73006b1b80 10 client.6747 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:12:14.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.311+0000 7f73006b1b80 20 client.6747 populate_metadata read hostname 'smithi028' 2023-11-28T11:12:14.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.311+0000 7f73006b1b80 1 --2- 172.21.15.28:0/3986768338 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x558ec07575e0 0x558ec07579b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.311+0000 7f73006b1b80 4 client.6747 mds_command: new command op to 6573 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:12:14.324 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.311+0000 7f73006b1b80 1 -- 172.21.15.28:0/3986768338 --> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558ec0505f10 con 0x558ec07575e0 2023-11-28T11:12:14.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.311+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/3986768338 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x558ec07575e0 0x558ec07579b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.312+0000 7f72cd7fa700 10 client.6747 ms_handle_connect on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:14.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.312+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/3986768338 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x558ec07575e0 0x558ec07579b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2e149ebfdf5e6e5 server_cookie=d378fa7f892b10c3 in_seq=0 out_seq=0 2023-11-28T11:12:14.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.313+0000 7f72cd7fa700 1 -- 172.21.15.28:0/3986768338 <== mds.0 v2:172.21.15.59:6832/2406852565 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x558ec0505f10 con 0x558ec07575e0 2023-11-28T11:12:14.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.313+0000 7f72cd7fa700 10 client.6747 handle_command_reply: tid=0 2023-11-28T11:12:14.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.313+0000 7f73006b1b80 1 client.6747 shutdown 2023-11-28T11:12:14.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.313+0000 7f73006b1b80 1 -- 172.21.15.28:0/3986768338 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0757130 msgr2=0x558ec074fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.313+0000 7f73006b1b80 1 --2- 172.21.15.28:0/3986768338 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0757130 0x558ec074fa00 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7f72dc056070 tx=0x7f72dc001500 comp rx=0 tx=0).stop 2023-11-28T11:12:14.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.313+0000 7f73006b1b80 1 -- 172.21.15.28:0/3986768338 shutdown_connections 2023-11-28T11:12:14.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.313+0000 7f73006b1b80 1 --2- 172.21.15.28:0/3986768338 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x558ec07575e0 0x558ec07579b0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.313+0000 7f73006b1b80 1 --2- 172.21.15.28:0/3986768338 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0757130 0x558ec074fa00 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.325 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.313+0000 7f73006b1b80 1 --2- 172.21.15.28:0/3986768338 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec075a120 0x558ec076b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.313+0000 7f73006b1b80 1 --2- 172.21.15.28:0/3986768338 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec0759810 0x558ec074b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.313+0000 7f72cd7fa700 0 client.6747 ms_handle_reset on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:14.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.313+0000 7f73006b1b80 1 -- 172.21.15.28:0/3986768338 >> 172.21.15.28:0/3986768338 conn(0x558ec072cd60 msgr2=0x558ec072d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.313+0000 7f73006b1b80 1 -- 172.21.15.28:0/3986768338 shutdown_connections 2023-11-28T11:12:14.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.313+0000 7f73006b1b80 1 -- 172.21.15.28:0/3986768338 wait complete. 2023-11-28T11:12:14.326 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.313+0000 7f73006b1b80 20 client.6747 trim_cache size 0 max 16384 2023-11-28T11:12:14.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.331+0000 7f73006b1b80 1 Processor -- start 2023-11-28T11:12:14.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.331+0000 7f73006b1b80 1 -- start start 2023-11-28T11:12:14.335 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f73006b1b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec0757130 0x558ec072bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f73006b1b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec0756b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f73006b1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec075a120 0x558ec072d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f73006b1b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec07505c0 con 0x558ec0759810 2023-11-28T11:12:14.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f73006b1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec0796b20 con 0x558ec075a120 2023-11-28T11:12:14.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f73006b1b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ec074c850 con 0x558ec0757130 2023-11-28T11:12:14.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f72f1ac3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec0756b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f72f1ac3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec0756b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51750/0 (socket says 172.21.15.28:51750) 2023-11-28T11:12:14.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f72f1ac3700 1 -- 172.21.15.28:0/611240784 learned_addr learned my addr 172.21.15.28:0/611240784 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:14.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f72f22c4700 1 --2- 172.21.15.28:0/611240784 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec0757130 0x558ec072bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f72f2ac5700 1 --2- 172.21.15.28:0/611240784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec075a120 0x558ec072d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f72f1ac3700 1 -- 172.21.15.28:0/611240784 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec0757130 msgr2=0x558ec072bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/611240784 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec0757130 0x558ec072bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f72f1ac3700 1 -- 172.21.15.28:0/611240784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec075a120 msgr2=0x558ec072d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/611240784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec075a120 0x558ec072d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f72f1ac3700 1 -- 172.21.15.28:0/611240784 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558ec0648140 con 0x558ec0759810 2023-11-28T11:12:14.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f72f22c4700 1 --2- 172.21.15.28:0/611240784 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec0757130 0x558ec072bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:14.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/611240784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec0756b50 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f72e80090b0 tx=0x7f72e8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbc2fff40f5bace9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f72e0ff9700 1 -- 172.21.15.28:0/611240784 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e802f040 con 0x558ec0759810 2023-11-28T11:12:14.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f72e0ff9700 1 -- 172.21.15.28:0/611240784 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72e8034420 con 0x558ec0759810 2023-11-28T11:12:14.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.332+0000 7f72e0ff9700 1 -- 172.21.15.28:0/611240784 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8022070 con 0x558ec0759810 2023-11-28T11:12:14.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.333+0000 7f73006b1b80 1 -- 172.21.15.28:0/611240784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 msgr2=0x558ec0756b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.333+0000 7f73006b1b80 1 --2- 172.21.15.28:0/611240784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec0756b50 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f72e80090b0 tx=0x7f72e8013000 comp rx=0 tx=0).stop 2023-11-28T11:12:14.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.333+0000 7f73006b1b80 1 -- 172.21.15.28:0/611240784 shutdown_connections 2023-11-28T11:12:14.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.333+0000 7f73006b1b80 1 --2- 172.21.15.28:0/611240784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec075a120 0x558ec072d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.333+0000 7f73006b1b80 1 --2- 172.21.15.28:0/611240784 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec0757130 0x558ec072bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.333+0000 7f73006b1b80 1 --2- 172.21.15.28:0/611240784 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec0756b50 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.333+0000 7f73006b1b80 1 -- 172.21.15.28:0/611240784 >> 172.21.15.28:0/611240784 conn(0x558ec072cd60 msgr2=0x558ec0728450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.333+0000 7f73006b1b80 1 -- 172.21.15.28:0/611240784 shutdown_connections 2023-11-28T11:12:14.339 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.333+0000 7f73006b1b80 1 -- 172.21.15.28:0/611240784 wait complete. 2023-11-28T11:12:14.339 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.334+0000 7f73006b1b80 1 -- start start 2023-11-28T11:12:14.339 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.334+0000 7f73006b1b80 1 Processor -- start 2023-11-28T11:12:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.334+0000 7f73006b1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec0757130 0x558ec07d9480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.334+0000 7f73006b1b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec075d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.334+0000 7f73006b1b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec075a120 0x558ec075d6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f73006b1b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec078e3f0 con 0x558ec0759810 2023-11-28T11:12:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f73006b1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x558ec07d99c0 con 0x558ec0757130 2023-11-28T11:12:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f73006b1b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x558ec078dec0 con 0x558ec075a120 2023-11-28T11:12:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f73006b1b80 10 client.0 _ll_register_callbacks cb 0x558ec0527240 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:12:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f72f1ac3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec075d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f72f1ac3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec075d1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51756/0 (socket says 172.21.15.28:51756) 2023-11-28T11:12:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f72f1ac3700 1 -- 172.21.15.28:0/4207681456 learned_addr learned my addr 172.21.15.28:0/4207681456 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f72f22c4700 1 --2- 172.21.15.28:0/4207681456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec0757130 0x558ec07d9480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f72f2ac5700 1 --2- 172.21.15.28:0/4207681456 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec075a120 0x558ec075d6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f72f1ac3700 1 -- 172.21.15.28:0/4207681456 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec075a120 msgr2=0x558ec075d6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/4207681456 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec075a120 0x558ec075d6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.343 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f72f1ac3700 1 -- 172.21.15.28:0/4207681456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec0757130 msgr2=0x558ec07d9480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/4207681456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec0757130 0x558ec07d9480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f72f1ac3700 1 -- 172.21.15.28:0/4207681456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72d80035b0 con 0x558ec0759810 2023-11-28T11:12:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f72f2ac5700 1 --2- 172.21.15.28:0/4207681456 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec075a120 0x558ec075d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f72f1ac3700 1 --2- 172.21.15.28:0/4207681456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec075d1b0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f72e8016000 tx=0x7f72e8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4b955ef9498eaf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f72cd7fa700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:12:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f72cd7fa700 1 -- 172.21.15.28:0/4207681456 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72e8022040 con 0x558ec0759810 2023-11-28T11:12:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f72cd7fa700 1 -- 172.21.15.28:0/4207681456 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f72e8004560 con 0x558ec0759810 2023-11-28T11:12:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.335+0000 7f73006b1b80 1 -- 172.21.15.28:0/4207681456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558ec0759fb0 con 0x558ec0759810 2023-11-28T11:12:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.336+0000 7f72cd7fa700 1 -- 172.21.15.28:0/4207681456 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f72e802f040 con 0x558ec0759810 2023-11-28T11:12:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.336+0000 7f73006b1b80 10 client.6753 fetch_fsmap learned FSMap version 33 2023-11-28T11:12:14.344 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.336+0000 7f73006b1b80 1 -- 172.21.15.28:0/4207681456 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x558ec04d6200 con 0x558ec0759810 2023-11-28T11:12:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.336+0000 7f72cd7fa700 1 -- 172.21.15.28:0/4207681456 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558ec0759fb0 con 0x558ec0759810 2023-11-28T11:12:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.336+0000 7f72cd7fa700 1 -- 172.21.15.28:0/4207681456 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f72e80349c0 con 0x558ec0759810 2023-11-28T11:12:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.336+0000 7f73006b1b80 10 client.6753 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:12:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.336+0000 7f73006b1b80 10 client.6753 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:12:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.336+0000 7f73006b1b80 20 client.6753 populate_metadata read hostname 'smithi028' 2023-11-28T11:12:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.336+0000 7f73006b1b80 1 --2- 172.21.15.28:0/4207681456 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x558ec07dcfd0 0x558ec07dd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.336+0000 7f73006b1b80 4 client.6753 mds_command: new command op to 6573 tid=0[{"prefix": "status"}] 2023-11-28T11:12:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.336+0000 7f73006b1b80 1 -- 172.21.15.28:0/4207681456 --> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558ec063f160 con 0x558ec07dcfd0 2023-11-28T11:12:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.336+0000 7f72f22c4700 1 --2- 172.21.15.28:0/4207681456 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x558ec07dcfd0 0x558ec07dd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.336+0000 7f72cd7fa700 10 client.6753 ms_handle_connect on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:14.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.336+0000 7f72f22c4700 1 --2- 172.21.15.28:0/4207681456 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x558ec07dcfd0 0x558ec07dd3a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9bd92fa33067e38 server_cookie=2e5f10bb0d0b39eb in_seq=0 out_seq=0 2023-11-28T11:12:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.337+0000 7f72cd7fa700 1 -- 172.21.15.28:0/4207681456 <== mds.0 v2:172.21.15.59:6832/2406852565 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+319 (crc 0 0 0) 0x558ec063f160 con 0x558ec07dcfd0 2023-11-28T11:12:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.337+0000 7f72cd7fa700 10 client.6753 handle_command_reply: tid=0 2023-11-28T11:12:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.337+0000 7f73006b1b80 1 client.6753 shutdown 2023-11-28T11:12:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.337+0000 7f73006b1b80 1 -- 172.21.15.28:0/4207681456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 msgr2=0x558ec075d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.337+0000 7f73006b1b80 1 --2- 172.21.15.28:0/4207681456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec075d1b0 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f72e8016000 tx=0x7f72e8004750 comp rx=0 tx=0).stop 2023-11-28T11:12:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.337+0000 7f73006b1b80 1 -- 172.21.15.28:0/4207681456 shutdown_connections 2023-11-28T11:12:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.337+0000 7f73006b1b80 1 --2- 172.21.15.28:0/4207681456 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x558ec075a120 0x558ec075d6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.337+0000 7f73006b1b80 1 --2- 172.21.15.28:0/4207681456 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558ec0757130 0x558ec07d9480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.337+0000 7f73006b1b80 1 --2- 172.21.15.28:0/4207681456 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x558ec07dcfd0 0x558ec07dd3a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.337+0000 7f73006b1b80 1 --2- 172.21.15.28:0/4207681456 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x558ec0759810 0x558ec075d1b0 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.337+0000 7f72cd7fa700 0 client.6753 ms_handle_reset on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:14.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.337+0000 7f73006b1b80 1 -- 172.21.15.28:0/4207681456 >> 172.21.15.28:0/4207681456 conn(0x558ec072cd60 msgr2=0x558ec07579b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.337+0000 7f73006b1b80 1 -- 172.21.15.28:0/4207681456 shutdown_connections 2023-11-28T11:12:14.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.337+0000 7f73006b1b80 1 -- 172.21.15.28:0/4207681456 wait complete. 2023-11-28T11:12:14.347 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.337+0000 7f73006b1b80 20 client.6753 trim_cache size 0 max 16384 2023-11-28T11:12:14.347 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:12:14.347 INFO:teuthology.orchestra.run.smithi028.stdout: "cluster_fsid": "6e299adc-7a2a-4cf1-afa2-7ce104b29a51", 2023-11-28T11:12:14.347 INFO:teuthology.orchestra.run.smithi028.stdout: "whoami": 0, 2023-11-28T11:12:14.347 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 6573, 2023-11-28T11:12:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "want_state": "up:active", 2023-11-28T11:12:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "state": "up:active", 2023-11-28T11:12:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs", 2023-11-28T11:12:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "rank_uptime": 10.535497396, 2023-11-28T11:12:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "mdsmap_epoch": 33, 2023-11-28T11:12:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch": 66, 2023-11-28T11:12:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "osdmap_epoch_barrier": 66, 2023-11-28T11:12:14.348 INFO:teuthology.orchestra.run.smithi028.stdout: "uptime": 15.848312774 2023-11-28T11:12:14.348 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:12:14.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.338+0000 7f72f4528700 1 -- 172.21.15.28:0/2753516616 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f72d00682f0 msgr2=0x7f72d006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.338+0000 7f72f4528700 1 --2- 172.21.15.28:0/2753516616 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f72d00682f0 0x7f72d006a7a0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f72ec0ac940 tx=0x7f72e8008030 comp rx=0 tx=0).stop 2023-11-28T11:12:14.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.338+0000 7f72f4528700 1 -- 172.21.15.28:0/2753516616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72d80043d0 msgr2=0x7f72ec155770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.338+0000 7f72f4528700 1 --2- 172.21.15.28:0/2753516616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72d80043d0 0x7f72ec155770 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00f780 tx=0x7f72dc00fb40 comp rx=0 tx=0).stop 2023-11-28T11:12:14.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.338+0000 7f72f4528700 1 -- 172.21.15.28:0/2753516616 shutdown_connections 2023-11-28T11:12:14.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.338+0000 7f72f4528700 1 --2- 172.21.15.28:0/2753516616 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f72d00682f0 0x7f72d006a7a0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.338+0000 7f72f4528700 1 --2- 172.21.15.28:0/2753516616 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f72d80043d0 0x7f72ec155770 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.338+0000 7f72f4528700 1 --2- 172.21.15.28:0/2753516616 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f72ec1f63f0 0x7f72ec06e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.338+0000 7f72f4528700 1 --2- 172.21.15.28:0/2753516616 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f72ec159df0 0x7f72ec158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.338+0000 7f72f4528700 1 -- 172.21.15.28:0/2753516616 >> 172.21.15.28:0/2753516616 conn(0x7f72ec0b0dd0 msgr2=0x7f72ec1563c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.338+0000 7f72f4528700 1 -- 172.21.15.28:0/2753516616 shutdown_connections 2023-11-28T11:12:14.350 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.338+0000 7f72f4528700 1 -- 172.21.15.28:0/2753516616 wait complete. 2023-11-28T11:12:14.351 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-11-28T11:12:14.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.475+0000 7ffbb16b8700 1 Processor -- start 2023-11-28T11:12:14.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.475+0000 7ffbb16b8700 1 -- start start 2023-11-28T11:12:14.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.476+0000 7ffbb16b8700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffbac161090 0x7ffbac0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.476+0000 7ffbb16b8700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbac0ce340 con 0x7ffbac15c990 2023-11-28T11:12:14.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.476+0000 7ffbb16b8700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbac0cebe0 con 0x7ffbac0caeb0 2023-11-28T11:12:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.476+0000 7ffbb16b8700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbac0cf780 con 0x7ffbac161090 2023-11-28T11:12:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.476+0000 7ffbaa7fc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffbac161090 0x7ffbac0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.476+0000 7ffbaa7fc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ffbac161090 0x7ffbac0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51764/0 (socket says 172.21.15.28:51764) 2023-11-28T11:12:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.476+0000 7ffbaaffd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ffbac15c990 0x7ffbac0cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39820/0 (socket says 172.21.15.28:39820) 2023-11-28T11:12:14.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.476+0000 7ffbaa7fc700 1 -- 172.21.15.28:0/2504644022 learned_addr learned my addr 172.21.15.28:0/2504644022 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:14.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.476+0000 7ffbaa7fc700 1 -- 172.21.15.28:0/2504644022 >> v1:172.21.15.59:6789/0 conn(0x7ffbac0caeb0 legacy=0x7ffbac0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:14.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.476+0000 7ffbaa7fc700 1 -- 172.21.15.28:0/2504644022 >> v1:172.21.15.28:6789/0 conn(0x7ffbac15c990 legacy=0x7ffbac0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.476+0000 7ffbaa7fc700 1 -- 172.21.15.28:0/2504644022 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac0cb3f0 con 0x7ffbac161090 2023-11-28T11:12:14.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.476+0000 7ffbaa7fc700 1 --2- 172.21.15.28:0/2504644022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbac161090 0x7ffbac0ca970 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ffb940098f0 tx=0x7ffb94009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08e7af5321e503c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.477+0000 7ffba9ffb700 1 -- 172.21.15.28:0/2504644022 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb94004230 con 0x7ffbac161090 2023-11-28T11:12:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.477+0000 7ffba9ffb700 1 -- 172.21.15.28:0/2504644022 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb94004390 con 0x7ffbac161090 2023-11-28T11:12:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.477+0000 7ffba9ffb700 1 -- 172.21.15.28:0/2504644022 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb940036a0 con 0x7ffbac161090 2023-11-28T11:12:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.477+0000 7ffbb16b8700 1 -- 172.21.15.28:0/2504644022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbac161090 msgr2=0x7ffbac0ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.477+0000 7ffbb16b8700 1 --2- 172.21.15.28:0/2504644022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbac161090 0x7ffbac0ca970 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7ffb940098f0 tx=0x7ffb94009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.477+0000 7ffbb16b8700 1 -- 172.21.15.28:0/2504644022 shutdown_connections 2023-11-28T11:12:14.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.477+0000 7ffbb16b8700 1 --2- 172.21.15.28:0/2504644022 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbac161090 0x7ffbac0ca970 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.477+0000 7ffbb16b8700 1 -- 172.21.15.28:0/2504644022 >> 172.21.15.28:0/2504644022 conn(0x7ffbac0b1650 msgr2=0x7ffbac0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.477+0000 7ffbb16b8700 1 -- 172.21.15.28:0/2504644022 shutdown_connections 2023-11-28T11:12:14.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.477+0000 7ffbb16b8700 1 -- 172.21.15.28:0/2504644022 wait complete. 2023-11-28T11:12:14.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.479+0000 7ffbb16b8700 1 Processor -- start 2023-11-28T11:12:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.479+0000 7ffbb16b8700 1 -- start start 2023-11-28T11:12:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.479+0000 7ffbb16b8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbac0caeb0 0x7ffbac159720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.479+0000 7ffbb16b8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbac15c990 0x7ffbac06c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.479+0000 7ffbb16b8700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffbac161090 0x7ffbac06c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.479+0000 7ffbb16b8700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac0c8ad0 con 0x7ffbac0caeb0 2023-11-28T11:12:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.479+0000 7ffbb16b8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac15a290 con 0x7ffbac15c990 2023-11-28T11:12:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.479+0000 7ffbb16b8700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbac15f950 con 0x7ffbac161090 2023-11-28T11:12:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.479+0000 7ffbaaffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbac0caeb0 0x7ffbac159720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffbaaffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbac0caeb0 0x7ffbac159720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51768/0 (socket says 172.21.15.28:51768) 2023-11-28T11:12:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffbaaffd700 1 -- 172.21.15.28:0/4065388546 learned_addr learned my addr 172.21.15.28:0/4065388546 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffbaa7fc700 1 --2- 172.21.15.28:0/4065388546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbac15c990 0x7ffbac06c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffbaaffd700 1 -- 172.21.15.28:0/4065388546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffbac161090 msgr2=0x7ffbac06c7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffbab7fe700 1 --2- 172.21.15.28:0/4065388546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffbac161090 0x7ffbac06c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffbaaffd700 1 --2- 172.21.15.28:0/4065388546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffbac161090 0x7ffbac06c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffbaaffd700 1 -- 172.21.15.28:0/4065388546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbac15c990 msgr2=0x7ffbac06c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffbaaffd700 1 --2- 172.21.15.28:0/4065388546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbac15c990 0x7ffbac06c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffbaaffd700 1 -- 172.21.15.28:0/4065388546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb94009550 con 0x7ffbac0caeb0 2023-11-28T11:12:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffbaa7fc700 1 --2- 172.21.15.28:0/4065388546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbac15c990 0x7ffbac06c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffbaaffd700 1 --2- 172.21.15.28:0/4065388546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbac0caeb0 0x7ffbac159720 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ffba400cbd0 tx=0x7ffba400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffba37fe700 1 -- 172.21.15.28:0/4065388546 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba400abb0 con 0x7ffbac0caeb0 2023-11-28T11:12:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffba37fe700 1 -- 172.21.15.28:0/4065388546 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffba400ad10 con 0x7ffbac0caeb0 2023-11-28T11:12:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffbb16b8700 1 -- 172.21.15.28:0/4065388546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbac0ce870 con 0x7ffbac0caeb0 2023-11-28T11:12:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffba37fe700 1 -- 172.21.15.28:0/4065388546 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffba4007290 con 0x7ffbac0caeb0 2023-11-28T11:12:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.480+0000 7ffbb16b8700 1 -- 172.21.15.28:0/4065388546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbac0ce590 con 0x7ffbac0caeb0 2023-11-28T11:12:14.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.481+0000 7ffbb16b8700 1 -- 172.21.15.28:0/4065388546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbac0cca20 con 0x7ffbac0caeb0 2023-11-28T11:12:14.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.482+0000 7ffba37fe700 1 -- 172.21.15.28:0/4065388546 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ffba4019030 con 0x7ffbac0caeb0 2023-11-28T11:12:14.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.482+0000 7ffba37fe700 1 --2- 172.21.15.28:0/4065388546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb8c0682f0 0x7ffb8c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:14.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.482+0000 7ffba37fe700 1 -- 172.21.15.28:0/4065388546 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7ffba4085b60 con 0x7ffbac0caeb0 2023-11-28T11:12:14.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.482+0000 7ffbaa7fc700 1 --2- 172.21.15.28:0/4065388546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb8c0682f0 0x7ffb8c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:14.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.483+0000 7ffbaa7fc700 1 --2- 172.21.15.28:0/4065388546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb8c0682f0 0x7ffb8c06a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffbac1f5b80 tx=0x7ffb94008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:14.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.486+0000 7ffba37fe700 1 -- 172.21.15.28:0/4065388546 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ffba4052280 con 0x7ffbac0caeb0 2023-11-28T11:12:14.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.726+0000 7ffbb16b8700 1 -- 172.21.15.28:0/4065388546 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ffbac0c82e0 con 0x7ffbac0caeb0 2023-11-28T11:12:14.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.726+0000 7ffba37fe700 1 -- 172.21.15.28:0/4065388546 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v33) v1 ==== 53+0+73 (secure 0 0 0) 0x7ffbac0c82e0 con 0x7ffbac0caeb0 2023-11-28T11:12:14.728 INFO:teuthology.orchestra.run.smithi028.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T11:12:14.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.729+0000 7ffbb16b8700 1 -- 172.21.15.28:0/4065388546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb8c0682f0 msgr2=0x7ffb8c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.729+0000 7ffbb16b8700 1 --2- 172.21.15.28:0/4065388546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb8c0682f0 0x7ffb8c06a7a0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7ffbac1f5b80 tx=0x7ffb94008030 comp rx=0 tx=0).stop 2023-11-28T11:12:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.729+0000 7ffbb16b8700 1 -- 172.21.15.28:0/4065388546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbac0caeb0 msgr2=0x7ffbac159720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.729+0000 7ffbb16b8700 1 --2- 172.21.15.28:0/4065388546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbac0caeb0 0x7ffbac159720 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7ffba400cbd0 tx=0x7ffba400cf90 comp rx=0 tx=0).stop 2023-11-28T11:12:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.729+0000 7ffbb16b8700 1 -- 172.21.15.28:0/4065388546 shutdown_connections 2023-11-28T11:12:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.729+0000 7ffbb16b8700 1 --2- 172.21.15.28:0/4065388546 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ffb8c0682f0 0x7ffb8c06a7a0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.729+0000 7ffbb16b8700 1 --2- 172.21.15.28:0/4065388546 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ffbac0caeb0 0x7ffbac159720 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.729+0000 7ffbb16b8700 1 --2- 172.21.15.28:0/4065388546 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ffbac161090 0x7ffbac06c7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.729+0000 7ffbb16b8700 1 --2- 172.21.15.28:0/4065388546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffbac15c990 0x7ffbac06c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.729+0000 7ffbb16b8700 1 -- 172.21.15.28:0/4065388546 >> 172.21.15.28:0/4065388546 conn(0x7ffbac0b1650 msgr2=0x7ffbac0b0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:14.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.729+0000 7ffbb16b8700 1 -- 172.21.15.28:0/4065388546 shutdown_connections 2023-11-28T11:12:14.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:14.729+0000 7ffbb16b8700 1 -- 172.21.15.28:0/4065388546 wait complete. 2023-11-28T11:12:14.742 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T11:12:14.742 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T11:12:14.742 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi028.front.sepia.ceph.com 2023-11-28T11:12:14.743 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T11:12:14.743 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T11:12:14.743 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T11:12:14.743 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T11:12:14.743 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T11:12:14.743 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:12:14.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:14.743 DEBUG:teuthology.orchestra.run.smithi028:> ip addr 2023-11-28T11:12:14.761 INFO:teuthology.orchestra.run.smithi028.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T11:12:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T11:12:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T11:12:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:12:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 ::1/128 scope host 2023-11-28T11:12:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:12:14.761 INFO:teuthology.orchestra.run.smithi028.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:12:14.761 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:16 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:12:14.761 INFO:teuthology.orchestra.run.smithi028.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:12:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 0c:c4:7a:6b:ee:17 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:12:14.762 INFO:teuthology.orchestra.run.smithi028.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T11:12:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3c brd ff:ff:ff:ff:ff:ff 2023-11-28T11:12:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: inet 172.21.15.28/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2023-11-28T11:12:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft 42124sec preferred_lft 42124sec 2023-11-28T11:12:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::ae1f:6bff:fea5:ca3c/64 scope link 2023-11-28T11:12:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:12:14.762 INFO:teuthology.orchestra.run.smithi028.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T11:12:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether ac:1f:6b:a5:ca:3d brd ff:ff:ff:ff:ff:ff 2023-11-28T11:12:14.762 INFO:teuthology.orchestra.run.smithi028.stdout:8: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T11:12:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 62:ec:54:d3:7a:f0 brd ff:ff:ff:ff:ff:ff 2023-11-28T11:12:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T11:12:14.762 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:12:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::407f:32ff:fe2d:9b2f/64 scope link 2023-11-28T11:12:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:12:14.763 INFO:teuthology.orchestra.run.smithi028.stdout:9: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:12:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether 62:ec:54:d3:7a:f0 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:12:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::60ec:54ff:fed3:7af0/64 scope link 2023-11-28T11:12:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:12:14.763 INFO:teuthology.orchestra.run.smithi028.stdout:10: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T11:12:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: link/ether c6:1d:6e:49:de:b9 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:12:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: inet6 fe80::c41d:6eff:fe49:deb9/64 scope link 2023-11-28T11:12:14.763 INFO:teuthology.orchestra.run.smithi028.stdout: valid_lft forever preferred_lft forever 2023-11-28T11:12:14.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:14.764 DEBUG:teuthology.orchestra.run.smithi028:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T11:12:14.827 INFO:teuthology.orchestra.run.smithi028.stdout:1 2023-11-28T11:12:14.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:14.829 DEBUG:teuthology.orchestra.run.smithi028:> route 2023-11-28T11:12:14.888 INFO:teuthology.orchestra.run.smithi028.stdout:Kernel IP routing table 2023-11-28T11:12:14.889 INFO:teuthology.orchestra.run.smithi028.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T11:12:14.889 INFO:teuthology.orchestra.run.smithi028.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2023-11-28T11:12:14.889 INFO:teuthology.orchestra.run.smithi028.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2023-11-28T11:12:14.889 INFO:teuthology.orchestra.run.smithi028.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T11:12:14.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:14.889 DEBUG:teuthology.orchestra.run.smithi028:> (cd / && exec bash -c ' 2023-11-28T11:12:14.889 DEBUG:teuthology.orchestra.run.smithi028:> set -e 2023-11-28T11:12:14.890 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-11-28T11:12:14.890 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-11-28T11:12:14.890 DEBUG:teuthology.orchestra.run.smithi028:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-11-28T11:12:14.890 DEBUG:teuthology.orchestra.run.smithi028:> ') 2023-11-28T11:12:14.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:14.996 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2023-11-28T11:12:15.013 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:12:15.014 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:12:15.014 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:12:15.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:15.014 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link set brx.0 up 2023-11-28T11:12:15.082 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-11-28T11:12:15.082 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:12:15.082 DEBUG:teuthology.orchestra.run.smithi028:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-11-28T11:12:15.139 INFO:teuthology.orchestra.run.smithi028.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-11-28T11:12:15.140 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-11-28T11:12:15.140 DEBUG:teuthology.orchestra.run.smithi028:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:12:15.197 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2023-11-28T11:12:15.265 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:12:15.265 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:12:15.332 INFO:teuthology.orchestra.run.smithi028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:12:15.333 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:12:15.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:15.333 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2023-11-28T11:12:15.390 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-11-28T11:12:15.391 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_mountpoint=/) 2023-11-28T11:12:15.435 DEBUG:teuthology.orchestra.run.smithi028:> sudo modprobe fuse 2023-11-28T11:12:15.463 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-11-28T11:12:15.463 DEBUG:teuthology.orchestra.run.smithi028:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-11-28T11:12:15.524 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:2023-11-28T11:12:15.523+0000 7fd0a12ee5c0 -1 init, newargv = 0x562fa2e60770 newargc=15 2023-11-28T11:12:15.525 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[129555]: starting ceph client 2023-11-28T11:12:15.530 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[129555]: starting fuse 2023-11-28T11:12:15.531 INFO:teuthology.orchestra.run.smithi028.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-11-28T11:12:15.533 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-11-28T11:12:15.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:15.534 DEBUG:teuthology.orchestra.run.smithi028:> ls /sys/fs/fuse/connections 2023-11-28T11:12:15.594 INFO:teuthology.orchestra.run.smithi028.stdout:47 2023-11-28T11:12:15.594 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [47] 2023-11-28T11:12:15.594 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:12:15.594 DEBUG:teuthology.orchestra.run.smithi028:> import glob 2023-11-28T11:12:15.595 DEBUG:teuthology.orchestra.run.smithi028:> import re 2023-11-28T11:12:15.595 DEBUG:teuthology.orchestra.run.smithi028:> import os 2023-11-28T11:12:15.595 DEBUG:teuthology.orchestra.run.smithi028:> import subprocess 2023-11-28T11:12:15.595 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:12:15.595 DEBUG:teuthology.orchestra.run.smithi028:> def _find_admin_socket(client_name): 2023-11-28T11:12:15.595 DEBUG:teuthology.orchestra.run.smithi028:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-11-28T11:12:15.595 DEBUG:teuthology.orchestra.run.smithi028:> files = glob.glob(asok_path) 2023-11-28T11:12:15.595 DEBUG:teuthology.orchestra.run.smithi028:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-11-28T11:12:15.595 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:12:15.595 DEBUG:teuthology.orchestra.run.smithi028:> # Given a non-glob path, it better be there 2023-11-28T11:12:15.596 DEBUG:teuthology.orchestra.run.smithi028:> if "*" not in asok_path: 2023-11-28T11:12:15.596 DEBUG:teuthology.orchestra.run.smithi028:> assert(len(files) == 1) 2023-11-28T11:12:15.596 DEBUG:teuthology.orchestra.run.smithi028:> return files[0] 2023-11-28T11:12:15.596 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:12:15.596 DEBUG:teuthology.orchestra.run.smithi028:> for f in files: 2023-11-28T11:12:15.596 DEBUG:teuthology.orchestra.run.smithi028:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-11-28T11:12:15.596 DEBUG:teuthology.orchestra.run.smithi028:> if os.path.exists("/proc/{0}".format(pid)): 2023-11-28T11:12:15.596 DEBUG:teuthology.orchestra.run.smithi028:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-11-28T11:12:15.596 DEBUG:teuthology.orchestra.run.smithi028:> contents = proc_f.read() 2023-11-28T11:12:15.596 DEBUG:teuthology.orchestra.run.smithi028:> if mountpoint in contents: 2023-11-28T11:12:15.596 DEBUG:teuthology.orchestra.run.smithi028:> return f 2023-11-28T11:12:15.596 DEBUG:teuthology.orchestra.run.smithi028:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-11-28T11:12:15.596 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:12:15.597 DEBUG:teuthology.orchestra.run.smithi028:> print(_find_admin_socket("client.0")) 2023-11-28T11:12:15.597 DEBUG:teuthology.orchestra.run.smithi028:> ' 2023-11-28T11:12:15.719 INFO:teuthology.orchestra.run.smithi028.stdout:/var/run/ceph/ceph-client.0.129555.asok 2023-11-28T11:12:15.901 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.129555.asok 2023-11-28T11:12:15.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:15.902 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.129555.asok status 2023-11-28T11:12:16.062 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:12:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: "metadata": { 2023-11-28T11:12:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_sha1": "2f5f6028b893ad41357cd06e827373f0fe2a1232", 2023-11-28T11:12:16.062 INFO:teuthology.orchestra.run.smithi028.stdout: "ceph_version": "ceph version 18.0.0-2697-g2f5f6028 (2f5f6028b893ad41357cd06e827373f0fe2a1232) reef (dev)", 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "entity_id": "0", 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "hostname": "smithi028", 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "pid": "129555", 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "root": "/" 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_count": 0, 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "dentry_pinned_count": 0, 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 6768, 2023-11-28T11:12:16.063 INFO:teuthology.orchestra.run.smithi028.stdout: "inst": { 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "name": { 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "client", 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "num": 6768 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1995202276 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:12:16.064 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": { 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "type": "v1", 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "addr": "192.168.0.1:0", 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "nonce": 1995202276 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "inst_str": "client.6768 192.168.0.1:0/1995202276", 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "addr_str": "192.168.0.1:0/1995202276", 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "inode_count": 1, 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "mds_epoch": 33, 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch": 66, 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "osd_epoch_barrier": 0, 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "blocklisted": false, 2023-11-28T11:12:16.065 INFO:teuthology.orchestra.run.smithi028.stdout: "fs_name": "cephfs" 2023-11-28T11:12:16.066 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:12:16.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:16.072 DEBUG:teuthology.orchestra.run.smithi028:> stat --file-system '--printf=%T 2023-11-28T11:12:16.072 DEBUG:teuthology.orchestra.run.smithi028:> ' -- /home/ubuntu/cephtest/mnt.0 2023-11-28T11:12:16.090 INFO:teuthology.orchestra.run.smithi028.stdout:fuseblk 2023-11-28T11:12:16.090 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-11-28T11:12:16.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:16.090 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T11:12:16.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:16.159 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir dir_test) 2023-11-28T11:12:16.224 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:12:16.224 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest/mnt.0 && exec touch dir_test/file) 2023-11-28T11:12:16.285 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits daemon-helper kill python3 -c ' 2023-11-28T11:12:16.285 DEBUG:teuthology.orchestra.run.smithi028:> import os 2023-11-28T11:12:16.285 DEBUG:teuthology.orchestra.run.smithi028:> import stat 2023-11-28T11:12:16.285 DEBUG:teuthology.orchestra.run.smithi028:> 2023-11-28T11:12:16.285 DEBUG:teuthology.orchestra.run.smithi028:> print(os.stat("/home/ubuntu/cephtest/mnt.0/dir_test/file").st_ino) 2023-11-28T11:12:16.286 DEBUG:teuthology.orchestra.run.smithi028:> ' 2023-11-28T11:12:16.393 INFO:teuthology.orchestra.run.smithi028.stdout:1099511627777 2023-11-28T11:12:16.582 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:12:16.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.745+0000 7f0308490700 1 Processor -- start 2023-11-28T11:12:16.747 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.745+0000 7f0308490700 1 -- start start 2023-11-28T11:12:16.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.746+0000 7f0308490700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f030015c990 0x7f03000cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.746+0000 7f0308490700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0300161090 0x7f03000ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.746+0000 7f0308490700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03000ce340 con 0x7f03000caeb0 2023-11-28T11:12:16.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.746+0000 7f0308490700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03000cebe0 con 0x7f0300161090 2023-11-28T11:12:16.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.746+0000 7f0308490700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03000cb3f0 con 0x7f030015c990 2023-11-28T11:12:16.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.746+0000 7f0305a2b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0300161090 0x7f03000ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.746+0000 7f0305a2b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0300161090 0x7f03000ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51798/0 (socket says 172.21.15.28:51798) 2023-11-28T11:12:16.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.746+0000 7f0305a2b700 1 -- 172.21.15.28:0/2089533889 learned_addr learned my addr 172.21.15.28:0/2089533889 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:16.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.747+0000 7f030622c700 1 --2- 172.21.15.28:0/2089533889 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f030015c990 0x7f03000cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.747+0000 7f0305a2b700 1 -- 172.21.15.28:0/2089533889 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f030015c990 msgr2=0x7f03000cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:16.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.747+0000 7f0305a2b700 1 --2- 172.21.15.28:0/2089533889 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f030015c990 0x7f03000cf070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.747+0000 7f0305a2b700 1 -- 172.21.15.28:0/2089533889 >> v1:172.21.15.59:6789/0 conn(0x7f03000caeb0 legacy=0x7f03000cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:16.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.747+0000 7f0305a2b700 1 -- 172.21.15.28:0/2089533889 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03000ce590 con 0x7f0300161090 2023-11-28T11:12:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.747+0000 7f0305a2b700 1 --2- 172.21.15.28:0/2089533889 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0300161090 0x7f03000ca970 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f02f00098f0 tx=0x7f02f0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=884a4079e477689c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.747+0000 7f030522a700 1 -- 172.21.15.28:0/2089533889 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f0004230 con 0x7f0300161090 2023-11-28T11:12:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.747+0000 7f030522a700 1 -- 172.21.15.28:0/2089533889 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02f0004390 con 0x7f0300161090 2023-11-28T11:12:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.747+0000 7f030522a700 1 -- 172.21.15.28:0/2089533889 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02f00036a0 con 0x7f0300161090 2023-11-28T11:12:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.748+0000 7f0308490700 1 -- 172.21.15.28:0/2089533889 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0300161090 msgr2=0x7f03000ca970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.748+0000 7f0308490700 1 --2- 172.21.15.28:0/2089533889 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0300161090 0x7f03000ca970 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f02f00098f0 tx=0x7f02f0009c00 comp rx=0 tx=0).stop 2023-11-28T11:12:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.748+0000 7f0308490700 1 -- 172.21.15.28:0/2089533889 shutdown_connections 2023-11-28T11:12:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.748+0000 7f0308490700 1 --2- 172.21.15.28:0/2089533889 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f030015c990 0x7f03000cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.748+0000 7f0308490700 1 --2- 172.21.15.28:0/2089533889 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0300161090 0x7f03000ca970 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.748+0000 7f0308490700 1 -- 172.21.15.28:0/2089533889 >> 172.21.15.28:0/2089533889 conn(0x7f03000b1650 msgr2=0x7f03000b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:16.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.748+0000 7f0308490700 1 -- 172.21.15.28:0/2089533889 shutdown_connections 2023-11-28T11:12:16.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.748+0000 7f0308490700 1 -- 172.21.15.28:0/2089533889 wait complete. 2023-11-28T11:12:16.756 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.749+0000 7f0308490700 1 Processor -- start 2023-11-28T11:12:16.756 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.749+0000 7f0308490700 1 -- start start 2023-11-28T11:12:16.756 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f0308490700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03000caeb0 0x7f03001f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.756 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f0308490700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f030015c990 0x7f03000c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f0308490700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0300161090 0x7f03000b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f0308490700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03000c3ed0 con 0x7f03000caeb0 2023-11-28T11:12:16.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f0308490700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03001f2ef0 con 0x7f030015c990 2023-11-28T11:12:16.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f0308490700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f030015f7c0 con 0x7f0300161090 2023-11-28T11:12:16.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f030622c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03000caeb0 0x7f03001f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f030622c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03000caeb0 0x7f03001f29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51802/0 (socket says 172.21.15.28:51802) 2023-11-28T11:12:16.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f030622c700 1 -- 172.21.15.28:0/1888137577 learned_addr learned my addr 172.21.15.28:0/1888137577 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:16.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f0305a2b700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f030015c990 0x7f03000c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f0306a2d700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0300161090 0x7f03000b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f030622c700 1 -- 172.21.15.28:0/1888137577 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0300161090 msgr2=0x7f03000b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f030622c700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0300161090 0x7f03000b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f030622c700 1 -- 172.21.15.28:0/1888137577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f030015c990 msgr2=0x7f03000c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:16.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f030622c700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f030015c990 0x7f03000c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:16.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f030622c700 1 -- 172.21.15.28:0/1888137577 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02f0009550 con 0x7f03000caeb0 2023-11-28T11:12:16.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f0305a2b700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f030015c990 0x7f03000c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:16.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f0306a2d700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0300161090 0x7f03000b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:16.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f030622c700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03000caeb0 0x7f03001f29b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00c7f0 tx=0x7f02fc00cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:16.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f02f6ffd700 1 -- 172.21.15.28:0/1888137577 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc011800 con 0x7f03000caeb0 2023-11-28T11:12:16.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f02f6ffd700 1 -- 172.21.15.28:0/1888137577 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02fc011960 con 0x7f03000caeb0 2023-11-28T11:12:16.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.750+0000 7f0308490700 1 -- 172.21.15.28:0/1888137577 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03000cf5b0 con 0x7f03000caeb0 2023-11-28T11:12:16.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.751+0000 7f02f6ffd700 1 -- 172.21.15.28:0/1888137577 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02fc011cf0 con 0x7f03000caeb0 2023-11-28T11:12:16.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.751+0000 7f0308490700 1 -- 172.21.15.28:0/1888137577 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03000ce870 con 0x7f03000caeb0 2023-11-28T11:12:16.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.752+0000 7f0308490700 1 -- 172.21.15.28:0/1888137577 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03000cf740 con 0x7f03000caeb0 2023-11-28T11:12:16.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.752+0000 7f02f6ffd700 1 -- 172.21.15.28:0/1888137577 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f02fc021070 con 0x7f03000caeb0 2023-11-28T11:12:16.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.752+0000 7f02f6ffd700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f02e40682f0 0x7f02e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:16.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.752+0000 7f02f6ffd700 1 -- 172.21.15.28:0/1888137577 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f02fc085c20 con 0x7f03000caeb0 2023-11-28T11:12:16.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.753+0000 7f0305a2b700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f02e40682f0 0x7f02e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:16.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.753+0000 7f0305a2b700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f02e40682f0 0x7f02e406a7a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f03001614c0 tx=0x7f02f0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:16.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.757+0000 7f02f6ffd700 1 -- 172.21.15.28:0/1888137577 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f02fc0522c0 con 0x7f03000caeb0 2023-11-28T11:12:16.998 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.997+0000 7f0308490700 1 -- 172.21.15.28:0/1888137577 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f03000675a0 con 0x7f03000caeb0 2023-11-28T11:12:16.999 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.997+0000 7f02f6ffd700 1 -- 172.21.15.28:0/1888137577 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f02fc0558e0 con 0x7f03000caeb0 2023-11-28T11:12:16.999 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:16.999 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":33,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1564028052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1564028052},{"type":"v1","addr":"172.21.15.59:6835","nonce":1564028052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6558,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2509041002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2509041002},{"type":"v1","addr":"172.21.15.28:6836","nonce":2509041002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6561,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3994419079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3994419079},{"type":"v1","addr":"172.21.15.28:6837","nonce":3994419079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:12:03.800884+0000","modified":"2023-11-28T11:12:06.812004+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":3,"addr":"172.21.15.59:6834/2406852565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2406852565},{"type":"v1","addr":"172.21.15.59:6834","nonce":2406852565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:12:17.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.999+0000 7f0308490700 1 -- 172.21.15.28:0/1888137577 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f02e40682f0 msgr2=0x7f02e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.999+0000 7f0308490700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f02e40682f0 0x7f02e406a7a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f03001614c0 tx=0x7f02f0008030 comp rx=0 tx=0).stop 2023-11-28T11:12:17.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.999+0000 7f0308490700 1 -- 172.21.15.28:0/1888137577 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03000caeb0 msgr2=0x7f03001f29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.999+0000 7f0308490700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03000caeb0 0x7f03001f29b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f02fc00c7f0 tx=0x7f02fc00cbb0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.999+0000 7f0308490700 1 -- 172.21.15.28:0/1888137577 shutdown_connections 2023-11-28T11:12:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.999+0000 7f0308490700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f02e40682f0 0x7f02e406a7a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.999+0000 7f0308490700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f03000caeb0 0x7f03001f29b0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.999+0000 7f0308490700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0300161090 0x7f03000b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.999+0000 7f0308490700 1 --2- 172.21.15.28:0/1888137577 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f030015c990 0x7f03000c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:16.999+0000 7f0308490700 1 -- 172.21.15.28:0/1888137577 >> 172.21.15.28:0/1888137577 conn(0x7f03000b1650 msgr2=0x7f03000ca0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.000+0000 7f0308490700 1 -- 172.21.15.28:0/1888137577 shutdown_connections 2023-11-28T11:12:17.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.000+0000 7f0308490700 1 -- 172.21.15.28:0/1888137577 wait complete. 2023-11-28T11:12:17.003 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 33 2023-11-28T11:12:17.013 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-11-28T11:12:17.176 INFO:teuthology.orchestra.run.smithi059.stdout:{"message":"","return_code":0} 2023-11-28T11:12:17.176 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-11-28T11:12:17.177 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data getxattr 10000000001.00000000 parent 2023-11-28T11:12:17.222 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.221+0000 7f7d51723080 1 Processor -- start 2023-11-28T11:12:17.222 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.221+0000 7f7d51723080 1 -- start start 2023-11-28T11:12:17.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.221+0000 7f7d51723080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563519706bc0 0x563519751c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d51723080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5635197db720 0x563519750000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d51723080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635197de410 0x563519750540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d51723080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x563519640fe0 con 0x563519706bc0 2023-11-28T11:12:17.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d51723080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635197331c0 con 0x5635197de410 2023-11-28T11:12:17.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d51723080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x5635197dace0 con 0x5635197db720 2023-11-28T11:12:17.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d47826700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563519706bc0 0x563519751c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d47826700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563519706bc0 0x563519751c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51816/0 (socket says 172.21.15.28:51816) 2023-11-28T11:12:17.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d47826700 1 -- 172.21.15.28:0/2420363618 learned_addr learned my addr 172.21.15.28:0/2420363618 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:17.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d47025700 1 --2- 172.21.15.28:0/2420363618 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5635197db720 0x563519750000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d48027700 1 --2- 172.21.15.28:0/2420363618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635197de410 0x563519750540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d47826700 1 -- 172.21.15.28:0/2420363618 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5635197db720 msgr2=0x563519750000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d47826700 1 --2- 172.21.15.28:0/2420363618 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5635197db720 0x563519750000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d47826700 1 -- 172.21.15.28:0/2420363618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635197de410 msgr2=0x563519750540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d47826700 1 --2- 172.21.15.28:0/2420363618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635197de410 0x563519750540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d47826700 1 -- 172.21.15.28:0/2420363618 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d4000a1c0 con 0x563519706bc0 2023-11-28T11:12:17.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d47826700 1 --2- 172.21.15.28:0/2420363618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563519706bc0 0x563519751c40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f7d38001d80 tx=0x7f7d3800b4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=326a0adad64863 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d46023700 1 -- 172.21.15.28:0/2420363618 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d38014070 con 0x563519706bc0 2023-11-28T11:12:17.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d46023700 1 -- 172.21.15.28:0/2420363618 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d38008030 con 0x563519706bc0 2023-11-28T11:12:17.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.222+0000 7f7d46023700 1 -- 172.21.15.28:0/2420363618 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3800bcc0 con 0x563519706bc0 2023-11-28T11:12:17.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.223+0000 7f7d51723080 1 -- 172.21.15.28:0/2420363618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563519706bc0 msgr2=0x563519751c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.223+0000 7f7d51723080 1 --2- 172.21.15.28:0/2420363618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563519706bc0 0x563519751c40 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f7d38001d80 tx=0x7f7d3800b4c0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.223+0000 7f7d51723080 1 -- 172.21.15.28:0/2420363618 shutdown_connections 2023-11-28T11:12:17.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.223+0000 7f7d51723080 1 --2- 172.21.15.28:0/2420363618 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x563519706bc0 0x563519751c40 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.223+0000 7f7d51723080 1 --2- 172.21.15.28:0/2420363618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5635197de410 0x563519750540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.223+0000 7f7d51723080 1 --2- 172.21.15.28:0/2420363618 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5635197db720 0x563519750000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.223+0000 7f7d51723080 1 -- 172.21.15.28:0/2420363618 >> 172.21.15.28:0/2420363618 conn(0x563519733300 msgr2=0x5635197045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.230 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.223+0000 7f7d51723080 1 -- 172.21.15.28:0/2420363618 shutdown_connections 2023-11-28T11:12:17.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.223+0000 7f7d51723080 1 -- 172.21.15.28:0/2420363618 wait complete. 2023-11-28T11:12:17.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.223+0000 7f7d51723080 1 Processor -- start 2023-11-28T11:12:17.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.224+0000 7f7d51723080 1 -- start start 2023-11-28T11:12:17.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.224+0000 7f7d51723080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563519706bc0 0x5635197d9f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.224+0000 7f7d51723080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635197db720 0x5635197d5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.224+0000 7f7d51723080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5635197de410 0x5635197df560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.224+0000 7f7d51723080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635197da440 con 0x5635197db720 2023-11-28T11:12:17.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.224+0000 7f7d51723080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x56351986e9d0 con 0x563519706bc0 2023-11-28T11:12:17.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.224+0000 7f7d51723080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x56351986ef90 con 0x5635197de410 2023-11-28T11:12:17.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.224+0000 7f7d47025700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635197db720 0x5635197d5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.224+0000 7f7d47025700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635197db720 0x5635197d5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51824/0 (socket says 172.21.15.28:51824) 2023-11-28T11:12:17.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.224+0000 7f7d47025700 1 -- 172.21.15.28:0/206948232 learned_addr learned my addr 172.21.15.28:0/206948232 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:17.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.224+0000 7f7d48027700 1 --2- 172.21.15.28:0/206948232 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5635197de410 0x5635197df560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.224+0000 7f7d47826700 1 --2- 172.21.15.28:0/206948232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563519706bc0 0x5635197d9f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.225+0000 7f7d47025700 1 -- 172.21.15.28:0/206948232 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5635197de410 msgr2=0x5635197df560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.225+0000 7f7d47025700 1 --2- 172.21.15.28:0/206948232 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5635197de410 0x5635197df560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.225+0000 7f7d47025700 1 -- 172.21.15.28:0/206948232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563519706bc0 msgr2=0x5635197d9f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.225+0000 7f7d47025700 1 --2- 172.21.15.28:0/206948232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563519706bc0 0x5635197d9f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.225+0000 7f7d47025700 1 -- 172.21.15.28:0/206948232 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3800a040 con 0x5635197db720 2023-11-28T11:12:17.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.225+0000 7f7d47025700 1 --2- 172.21.15.28:0/206948232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635197db720 0x5635197d5c30 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c008400 tx=0x7f7d3c008710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.225+0000 7f7d45021700 1 -- 172.21.15.28:0/206948232 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c011040 con 0x5635197db720 2023-11-28T11:12:17.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.225+0000 7f7d45021700 1 -- 172.21.15.28:0/206948232 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d3c015620 con 0x5635197db720 2023-11-28T11:12:17.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.225+0000 7f7d45021700 1 -- 172.21.15.28:0/206948232 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c015990 con 0x5635197db720 2023-11-28T11:12:17.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.225+0000 7f7d51723080 1 -- 172.21.15.28:0/206948232 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x563519659120 con 0x5635197db720 2023-11-28T11:12:17.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.225+0000 7f7d51723080 1 -- 172.21.15.28:0/206948232 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56351964c830 con 0x5635197db720 2023-11-28T11:12:17.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.226+0000 7f7d45021700 1 -- 172.21.15.28:0/206948232 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7d3c01c030 con 0x5635197db720 2023-11-28T11:12:17.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.227+0000 7f7d45021700 1 --2- 172.21.15.28:0/206948232 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7d34066c10 0x7f7d340690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.227+0000 7f7d45021700 1 -- 172.21.15.28:0/206948232 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f7d3c0841d0 con 0x5635197db720 2023-11-28T11:12:17.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.227+0000 7f7d47826700 1 --2- 172.21.15.28:0/206948232 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7d34066c10 0x7f7d340690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.227+0000 7f7d51723080 1 --2- 172.21.15.28:0/206948232 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x5635197dced0 0x5635197dd2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.227+0000 7f7d51723080 1 -- 172.21.15.28:0/206948232 --> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] -- osd_op(unknown.0.0:1 11.7 11:e2e07930:::10000000001.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66) v8 -- 0x5635197e01e0 con 0x5635197dced0 2023-11-28T11:12:17.236 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.227+0000 7f7d47826700 1 --2- 172.21.15.28:0/206948232 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7d34066c10 0x7f7d340690c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800b500 tx=0x7f7d38009000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.236 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.227+0000 7f7d48027700 1 --2- 172.21.15.28:0/206948232 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x5635197dced0 0x5635197dd2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.236 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.228+0000 7f7d48027700 1 --2- 172.21.15.28:0/206948232 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x5635197dced0 0x5635197dd2d0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.236 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.228+0000 7f7d48027700 1 -- 172.21.15.28:0/206948232 <== osd.5 v2:172.21.15.59:6800/103696 1 ==== osd_op_reply(1 10000000001.00000000 [getxattr (94) out=94b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+94 (crc 0 0 0) 0x7f7d40017040 con 0x5635197dced0 2023-11-28T11:12:17.236 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.228+0000 7f7d51723080 1 -- 172.21.15.28:0/206948232 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x5635197dced0 msgr2=0x5635197dd2d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.236 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.228+0000 7f7d51723080 1 --2- 172.21.15.28:0/206948232 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x5635197dced0 0x5635197dd2d0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.236 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.228+0000 7f7d51723080 1 -- 172.21.15.28:0/206948232 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7d34066c10 msgr2=0x7f7d340690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.237 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.228+0000 7f7d51723080 1 --2- 172.21.15.28:0/206948232 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7d34066c10 0x7f7d340690c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f7d3800b500 tx=0x7f7d38009000 comp rx=0 tx=0).stop 2023-11-28T11:12:17.237 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.228+0000 7f7d51723080 1 -- 172.21.15.28:0/206948232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635197db720 msgr2=0x5635197d5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.237 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.228+0000 7f7d51723080 1 --2- 172.21.15.28:0/206948232 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5635197db720 0x5635197d5c30 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c008400 tx=0x7f7d3c008710 comp rx=0 tx=0).stop 2023-11-28T11:12:17.237 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.228+0000 7f7d48027700 1 -- 172.21.15.28:0/206948232 reap_dead start 2023-11-28T11:12:17.237 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.228+0000 7f7d51723080 1 -- 172.21.15.28:0/206948232 shutdown_connections 2023-11-28T11:12:17.237 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.228+0000 7f7d51723080 1 -- 172.21.15.28:0/206948232 >> 172.21.15.28:0/206948232 conn(0x563519733300 msgr2=0x56351974ef60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.237 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.228+0000 7f7d51723080 1 -- 172.21.15.28:0/206948232 shutdown_connections 2023-11-28T11:12:17.237 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.228+0000 7f7d51723080 1 -- 172.21.15.28:0/206948232 wait complete. 2023-11-28T11:12:17.238 INFO:teuthology.orchestra.run.smithi028.stdout:Xfiledir_test 2023-11-28T11:12:17.238 DEBUG:teuthology.orchestra.run.smithi028:> ceph-dencoder type inode_backtrace_t import - decode dump_json 2023-11-28T11:12:17.295 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:12:17.295 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511627777, 2023-11-28T11:12:17.295 INFO:teuthology.orchestra.run.smithi028.stdout: "ancestors": [ 2023-11-28T11:12:17.295 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:12:17.295 INFO:teuthology.orchestra.run.smithi028.stdout: "dirino": 1099511627776, 2023-11-28T11:12:17.295 INFO:teuthology.orchestra.run.smithi028.stdout: "dname": "file", 2023-11-28T11:12:17.295 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 2 2023-11-28T11:12:17.295 INFO:teuthology.orchestra.run.smithi028.stdout: }, 2023-11-28T11:12:17.295 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:12:17.296 INFO:teuthology.orchestra.run.smithi028.stdout: "dirino": 1, 2023-11-28T11:12:17.296 INFO:teuthology.orchestra.run.smithi028.stdout: "dname": "dir_test", 2023-11-28T11:12:17.296 INFO:teuthology.orchestra.run.smithi028.stdout: "version": 4 2023-11-28T11:12:17.296 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:12:17.296 INFO:teuthology.orchestra.run.smithi028.stdout: ], 2023-11-28T11:12:17.296 INFO:teuthology.orchestra.run.smithi028.stdout: "pool": 11, 2023-11-28T11:12:17.296 INFO:teuthology.orchestra.run.smithi028.stdout: "old_pools": [] 2023-11-28T11:12:17.296 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:12:17.296 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:17.297 DEBUG:teuthology.orchestra.run.smithi028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph --pool cephfs_data setxattr 10000000001.00000000 parent 'The backtrace is corrupted' 2023-11-28T11:12:17.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.381+0000 7f9bc4898080 1 Processor -- start 2023-11-28T11:12:17.383 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.382+0000 7f9bc4898080 1 -- start start 2023-11-28T11:12:17.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.382+0000 7f9bc4898080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5585cf37fb10 0x5585cf40d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.382+0000 7f9bc4898080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5585cf424a60 0x5585cf427300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.382+0000 7f9bc4898080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585cf424e30 0x5585cf3793f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.382+0000 7f9bc4898080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585cf2f7000 con 0x5585cf37fb10 2023-11-28T11:12:17.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.382+0000 7f9bc4898080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585cf289fe0 con 0x5585cf424e30 2023-11-28T11:12:17.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.382+0000 7f9bc4898080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585cf37b860 con 0x5585cf424a60 2023-11-28T11:12:17.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.382+0000 7f9bba99b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5585cf37fb10 0x5585cf40d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.382+0000 7f9bba99b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5585cf37fb10 0x5585cf40d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51838/0 (socket says 172.21.15.28:51838) 2023-11-28T11:12:17.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.382+0000 7f9bba99b700 1 -- 172.21.15.28:0/1063211997 learned_addr learned my addr 172.21.15.28:0/1063211997 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:17.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bbb19c700 1 --2- 172.21.15.28:0/1063211997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585cf424e30 0x5585cf3793f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bba19a700 1 --2- 172.21.15.28:0/1063211997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5585cf424a60 0x5585cf427300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bba99b700 1 -- 172.21.15.28:0/1063211997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5585cf424a60 msgr2=0x5585cf427300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bba99b700 1 --2- 172.21.15.28:0/1063211997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5585cf424a60 0x5585cf427300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bba99b700 1 -- 172.21.15.28:0/1063211997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585cf424e30 msgr2=0x5585cf3793f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bba99b700 1 --2- 172.21.15.28:0/1063211997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585cf424e30 0x5585cf3793f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.387 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bba99b700 1 -- 172.21.15.28:0/1063211997 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bb0005d20 con 0x5585cf37fb10 2023-11-28T11:12:17.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bba99b700 1 --2- 172.21.15.28:0/1063211997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5585cf37fb10 0x5585cf40d7d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000c010 tx=0x7f9bb000d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e8d2f7ef44aa72 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bb9198700 1 -- 172.21.15.28:0/1063211997 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb000a220 con 0x5585cf37fb10 2023-11-28T11:12:17.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bb9198700 1 -- 172.21.15.28:0/1063211997 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb0008960 con 0x5585cf37fb10 2023-11-28T11:12:17.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bb9198700 1 -- 172.21.15.28:0/1063211997 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb0008cf0 con 0x5585cf37fb10 2023-11-28T11:12:17.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bc4898080 1 -- 172.21.15.28:0/1063211997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5585cf37fb10 msgr2=0x5585cf40d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bc4898080 1 --2- 172.21.15.28:0/1063211997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5585cf37fb10 0x5585cf40d7d0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f9bb000c010 tx=0x7f9bb000d000 comp rx=0 tx=0).stop 2023-11-28T11:12:17.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bc4898080 1 -- 172.21.15.28:0/1063211997 shutdown_connections 2023-11-28T11:12:17.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bc4898080 1 --2- 172.21.15.28:0/1063211997 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5585cf37fb10 0x5585cf40d7d0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bc4898080 1 --2- 172.21.15.28:0/1063211997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585cf424e30 0x5585cf3793f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.388 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bc4898080 1 --2- 172.21.15.28:0/1063211997 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5585cf424a60 0x5585cf427300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.383+0000 7f9bc4898080 1 -- 172.21.15.28:0/1063211997 >> 172.21.15.28:0/1063211997 conn(0x5585cf3967f0 msgr2=0x5585cf423d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.384+0000 7f9bc4898080 1 -- 172.21.15.28:0/1063211997 shutdown_connections 2023-11-28T11:12:17.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.384+0000 7f9bc4898080 1 -- 172.21.15.28:0/1063211997 wait complete. 2023-11-28T11:12:17.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.384+0000 7f9bc4898080 1 Processor -- start 2023-11-28T11:12:17.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.384+0000 7f9bc4898080 1 -- start start 2023-11-28T11:12:17.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.385+0000 7f9bc4898080 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585cf37fb10 0x5585cf409d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.385+0000 7f9bc4898080 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5585cf424a60 0x5585cf40c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.385+0000 7f9bc4898080 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5585cf424e30 0x5585cf4218f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.385+0000 7f9bc4898080 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585cf40b6a0 con 0x5585cf424a60 2023-11-28T11:12:17.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.385+0000 7f9bc4898080 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x5585cf37ac00 con 0x5585cf37fb10 2023-11-28T11:12:17.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.385+0000 7f9bc4898080 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x5585cf379990 con 0x5585cf424e30 2023-11-28T11:12:17.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9bba19a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5585cf424a60 0x5585cf40c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9bba19a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5585cf424a60 0x5585cf40c0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51846/0 (socket says 172.21.15.28:51846) 2023-11-28T11:12:17.393 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9bba19a700 1 -- 172.21.15.28:0/2241111915 learned_addr learned my addr 172.21.15.28:0/2241111915 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:17.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9bba99b700 1 --2- 172.21.15.28:0/2241111915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585cf37fb10 0x5585cf409d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9bbb19c700 1 --2- 172.21.15.28:0/2241111915 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5585cf424e30 0x5585cf4218f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9bba19a700 1 -- 172.21.15.28:0/2241111915 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5585cf424e30 msgr2=0x5585cf4218f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9bba19a700 1 --2- 172.21.15.28:0/2241111915 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5585cf424e30 0x5585cf4218f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9bba19a700 1 -- 172.21.15.28:0/2241111915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585cf37fb10 msgr2=0x5585cf409d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9bba19a700 1 --2- 172.21.15.28:0/2241111915 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5585cf37fb10 0x5585cf409d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9bba19a700 1 -- 172.21.15.28:0/2241111915 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5585cf2a2120 con 0x5585cf424a60 2023-11-28T11:12:17.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9bbb19c700 1 --2- 172.21.15.28:0/2241111915 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x5585cf424e30 0x5585cf4218f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:17.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9bba19a700 1 --2- 172.21.15.28:0/2241111915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5585cf424a60 0x5585cf40c0b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00df00 tx=0x7f9bac00b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9ba3fff700 1 -- 172.21.15.28:0/2241111915 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac00bdd0 con 0x5585cf424a60 2023-11-28T11:12:17.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9ba3fff700 1 -- 172.21.15.28:0/2241111915 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac010410 con 0x5585cf424a60 2023-11-28T11:12:17.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9bc4898080 1 -- 172.21.15.28:0/2241111915 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5585cf295830 con 0x5585cf424a60 2023-11-28T11:12:17.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9ba3fff700 1 -- 172.21.15.28:0/2241111915 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac0107c0 con 0x5585cf424a60 2023-11-28T11:12:17.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.386+0000 7f9bc4898080 1 -- 172.21.15.28:0/2241111915 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5585cf2a2360 con 0x5585cf424a60 2023-11-28T11:12:17.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.387+0000 7f9ba3fff700 1 -- 172.21.15.28:0/2241111915 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9bac020070 con 0x5585cf424a60 2023-11-28T11:12:17.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.388+0000 7f9ba3fff700 1 --2- 172.21.15.28:0/2241111915 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ba8066c10 0x7f9ba80690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.388+0000 7f9ba3fff700 1 -- 172.21.15.28:0/2241111915 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9bac00a040 con 0x5585cf424a60 2023-11-28T11:12:17.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.388+0000 7f9bba99b700 1 --2- 172.21.15.28:0/2241111915 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ba8066c10 0x7f9ba80690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.388+0000 7f9bc4898080 1 --2- 172.21.15.28:0/2241111915 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x5585cf34e1c0 0x5585cf4252f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.395 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.388+0000 7f9bc4898080 1 -- 172.21.15.28:0/2241111915 --> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] -- osd_op(unknown.0.0:1 11.7 11:e2e07930:::10000000001.00000000:head [setxattr parent (26) in=32b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e66) v8 -- 0x5585cf421e30 con 0x5585cf34e1c0 2023-11-28T11:12:17.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.388+0000 7f9bba99b700 1 --2- 172.21.15.28:0/2241111915 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ba8066c10 0x7f9ba80690c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x5585cf394e20 tx=0x7f9bb0008000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.388+0000 7f9bbb19c700 1 --2- 172.21.15.28:0/2241111915 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x5585cf34e1c0 0x5585cf4252f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.389+0000 7f9bbb19c700 1 --2- 172.21.15.28:0/2241111915 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x5585cf34e1c0 0x5585cf4252f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.390+0000 7f9bbb19c700 1 -- 172.21.15.28:0/2241111915 <== osd.5 v2:172.21.15.59:6800/103696 1 ==== osd_op_reply(1 10000000001.00000000 [setxattr (26)] v66'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f9bb4001580 con 0x5585cf34e1c0 2023-11-28T11:12:17.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.390+0000 7f9bc4898080 1 -- 172.21.15.28:0/2241111915 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x5585cf34e1c0 msgr2=0x5585cf4252f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.390+0000 7f9bc4898080 1 --2- 172.21.15.28:0/2241111915 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x5585cf34e1c0 0x5585cf4252f0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.390+0000 7f9bc4898080 1 -- 172.21.15.28:0/2241111915 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ba8066c10 msgr2=0x7f9ba80690c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.390+0000 7f9bc4898080 1 --2- 172.21.15.28:0/2241111915 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9ba8066c10 0x7f9ba80690c0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x5585cf394e20 tx=0x7f9bb0008000 comp rx=0 tx=0).stop 2023-11-28T11:12:17.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.390+0000 7f9bc4898080 1 -- 172.21.15.28:0/2241111915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5585cf424a60 msgr2=0x5585cf40c0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.390+0000 7f9bc4898080 1 --2- 172.21.15.28:0/2241111915 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x5585cf424a60 0x5585cf40c0b0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00df00 tx=0x7f9bac00b4f0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.390+0000 7f9bbb19c700 1 -- 172.21.15.28:0/2241111915 reap_dead start 2023-11-28T11:12:17.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.390+0000 7f9bc4898080 1 -- 172.21.15.28:0/2241111915 shutdown_connections 2023-11-28T11:12:17.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.390+0000 7f9bc4898080 1 -- 172.21.15.28:0/2241111915 >> 172.21.15.28:0/2241111915 conn(0x5585cf3967f0 msgr2=0x5585cf423d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.390+0000 7f9bc4898080 1 -- 172.21.15.28:0/2241111915 shutdown_connections 2023-11-28T11:12:17.397 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.390+0000 7f9bc4898080 1 -- 172.21.15.28:0/2241111915 wait complete. 2023-11-28T11:12:17.398 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub start / recursive 2023-11-28T11:12:17.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.525+0000 7f5e30f77700 1 Processor -- start 2023-11-28T11:12:17.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.525+0000 7f5e30f77700 1 -- start start 2023-11-28T11:12:17.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e30f77700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e2c15c990 0x7f5e2c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e30f77700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5e2c161090 0x7f5e2c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e30f77700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5e2c0caeb0 0x7f5e2c0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.529 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e30f77700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e2c0ce340 con 0x7f5e2c0caeb0 2023-11-28T11:12:17.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e30f77700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e2c0cebe0 con 0x7f5e2c161090 2023-11-28T11:12:17.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e30f77700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0cb3f0 con 0x7f5e2c15c990 2023-11-28T11:12:17.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e2ad9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5e2c0caeb0 0x7f5e2c0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.530 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e2ad9d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5e2c0caeb0 0x7f5e2c0cc310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51858/0 (socket says 172.21.15.28:51858) 2023-11-28T11:12:17.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e2ad9d700 1 -- 172.21.15.28:0/898003367 learned_addr learned my addr 172.21.15.28:0/898003367 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:17.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e2a59c700 1 --2- 172.21.15.28:0/898003367 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e2c15c990 0x7f5e2c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/898003367 >> v2:172.21.15.59:3300/0 conn(0x7f5e2c161090 0x7f5e2c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e2ad9d700 1 -- 172.21.15.28:0/898003367 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e2c15c990 msgr2=0x7f5e2c0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e2ad9d700 1 --2- 172.21.15.28:0/898003367 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e2c15c990 0x7f5e2c0cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e2ad9d700 1 -- 172.21.15.28:0/898003367 >> v2:172.21.15.59:3300/0 conn(0x7f5e2c161090 msgr2=0x7f5e2c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e2ad9d700 1 --2- 172.21.15.28:0/898003367 >> v2:172.21.15.59:3300/0 conn(0x7f5e2c161090 0x7f5e2c0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.526+0000 7f5e2ad9d700 1 -- 172.21.15.28:0/898003367 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c0ce590 con 0x7f5e2c0caeb0 2023-11-28T11:12:17.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.527+0000 7f5e2ad9d700 1 --2- 172.21.15.28:0/898003367 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e2c0caeb0 0x7f5e2c0cc310 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c06f850 tx=0x7f5e2400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85a1f9c683e099f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.527+0000 7f5e2959a700 1 -- 172.21.15.28:0/898003367 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400e070 con 0x7f5e2c0caeb0 2023-11-28T11:12:17.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.527+0000 7f5e2959a700 1 -- 172.21.15.28:0/898003367 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2400b9c0 con 0x7f5e2c0caeb0 2023-11-28T11:12:17.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.527+0000 7f5e2959a700 1 -- 172.21.15.28:0/898003367 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2400bd70 con 0x7f5e2c0caeb0 2023-11-28T11:12:17.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.527+0000 7f5e30f77700 1 -- 172.21.15.28:0/898003367 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e2c0caeb0 msgr2=0x7f5e2c0cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.527+0000 7f5e30f77700 1 --2- 172.21.15.28:0/898003367 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e2c0caeb0 0x7f5e2c0cc310 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c06f850 tx=0x7f5e2400b330 comp rx=0 tx=0).stop 2023-11-28T11:12:17.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.527+0000 7f5e30f77700 1 -- 172.21.15.28:0/898003367 shutdown_connections 2023-11-28T11:12:17.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.527+0000 7f5e30f77700 1 --2- 172.21.15.28:0/898003367 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e2c0caeb0 0x7f5e2c0cc310 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.527+0000 7f5e30f77700 1 --2- 172.21.15.28:0/898003367 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e2c15c990 0x7f5e2c0cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.527+0000 7f5e30f77700 1 --2- 172.21.15.28:0/898003367 >> v2:172.21.15.59:3300/0 conn(0x7f5e2c161090 0x7f5e2c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.527+0000 7f5e30f77700 1 -- 172.21.15.28:0/898003367 >> 172.21.15.28:0/898003367 conn(0x7f5e2c0b1650 msgr2=0x7f5e2c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.527+0000 7f5e30f77700 1 -- 172.21.15.28:0/898003367 shutdown_connections 2023-11-28T11:12:17.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.527+0000 7f5e30f77700 1 -- 172.21.15.28:0/898003367 wait complete. 2023-11-28T11:12:17.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.529+0000 7f5e30f77700 1 Processor -- start 2023-11-28T11:12:17.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.529+0000 7f5e30f77700 1 -- start start 2023-11-28T11:12:17.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e30f77700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e2c0caeb0 0x7f5e2c0c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e30f77700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e2c15c990 0x7f5e2c0c89d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e30f77700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e2c161090 0x7f5e2c148a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e30f77700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0c7ad0 con 0x7f5e2c0caeb0 2023-11-28T11:12:17.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e30f77700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0c6ec0 con 0x7f5e2c15c990 2023-11-28T11:12:17.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e30f77700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e2c0c71a0 con 0x7f5e2c161090 2023-11-28T11:12:17.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e2a59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e2c0caeb0 0x7f5e2c0c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e2a59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e2c0caeb0 0x7f5e2c0c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51860/0 (socket says 172.21.15.28:51860) 2023-11-28T11:12:17.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e2a59c700 1 -- 172.21.15.28:0/2025182365 learned_addr learned my addr 172.21.15.28:0/2025182365 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:17.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/2025182365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e2c15c990 0x7f5e2c0c89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e2ad9d700 1 --2- 172.21.15.28:0/2025182365 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e2c161090 0x7f5e2c148a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e2a59c700 1 -- 172.21.15.28:0/2025182365 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e2c161090 msgr2=0x7f5e2c148a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e2a59c700 1 --2- 172.21.15.28:0/2025182365 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e2c161090 0x7f5e2c148a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e2a59c700 1 -- 172.21.15.28:0/2025182365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e2c15c990 msgr2=0x7f5e2c0c89d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e2a59c700 1 --2- 172.21.15.28:0/2025182365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e2c15c990 0x7f5e2c0c89d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e2a59c700 1 -- 172.21.15.28:0/2025182365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e24009b30 con 0x7f5e2c0caeb0 2023-11-28T11:12:17.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e2a59c700 1 --2- 172.21.15.28:0/2025182365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e2c0caeb0 0x7f5e2c0c55d0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00e810 tx=0x7f5e1c00eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e1affd700 1 -- 172.21.15.28:0/2025182365 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c00c070 con 0x7f5e2c0caeb0 2023-11-28T11:12:17.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e1affd700 1 -- 172.21.15.28:0/2025182365 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e1c013610 con 0x7f5e2c0caeb0 2023-11-28T11:12:17.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e30f77700 1 -- 172.21.15.28:0/2025182365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e2c0cf5b0 con 0x7f5e2c0caeb0 2023-11-28T11:12:17.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e1affd700 1 -- 172.21.15.28:0/2025182365 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c0139a0 con 0x7f5e2c0caeb0 2023-11-28T11:12:17.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.530+0000 7f5e30f77700 1 -- 172.21.15.28:0/2025182365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2c0ce870 con 0x7f5e2c0caeb0 2023-11-28T11:12:17.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.532+0000 7f5e1affd700 1 -- 172.21.15.28:0/2025182365 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5e1c023070 con 0x7f5e2c0caeb0 2023-11-28T11:12:17.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.532+0000 7f5e1affd700 1 --2- 172.21.15.28:0/2025182365 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5e0c0682f0 0x7f5e0c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.532+0000 7f5e1affd700 1 -- 172.21.15.28:0/2025182365 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5e1c085ab0 con 0x7f5e2c0caeb0 2023-11-28T11:12:17.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.532+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/2025182365 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5e0c0682f0 0x7f5e0c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.533+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/2025182365 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5e0c0682f0 0x7f5e0c06a7a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0abfa0 tx=0x7f5e20008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.534+0000 7f5e3d100b80 1 Processor -- start 2023-11-28T11:12:17.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.535+0000 7f5e3d100b80 1 -- start start 2023-11-28T11:12:17.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.535+0000 7f5e3d100b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd28c0 0x55aac2ca8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.535+0000 7f5e3d100b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd01e0 0x55aac2cd2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.535+0000 7f5e3d100b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2cee820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.535+0000 7f5e3d100b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac2b92c70 con 0x55aac2cd01e0 2023-11-28T11:12:17.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.535+0000 7f5e3d100b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac2c01400 con 0x55aac2cd31d0 2023-11-28T11:12:17.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.535+0000 7f5e3d100b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aac2bfa930 con 0x55aac2cd28c0 2023-11-28T11:12:17.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.535+0000 7f5e29d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd01e0 0x55aac2cd2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.535+0000 7f5e29d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd01e0 0x55aac2cd2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51870/0 (socket says 172.21.15.28:51870) 2023-11-28T11:12:17.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.535+0000 7f5e29d9b700 1 -- 172.21.15.28:0/556560066 learned_addr learned my addr 172.21.15.28:0/556560066 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:17.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.536+0000 7f5e2ad9d700 1 --2- 172.21.15.28:0/556560066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2cee820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.536+0000 7f5e2a59c700 1 --2- 172.21.15.28:0/556560066 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd28c0 0x55aac2ca8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.536+0000 7f5e29d9b700 1 -- 172.21.15.28:0/556560066 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd28c0 msgr2=0x55aac2ca8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.536+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/556560066 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd28c0 0x55aac2ca8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.536+0000 7f5e29d9b700 1 -- 172.21.15.28:0/556560066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 msgr2=0x55aac2cee820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.536+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/556560066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2cee820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.536+0000 7f5e29d9b700 1 -- 172.21.15.28:0/556560066 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aac2a7c690 con 0x55aac2cd01e0 2023-11-28T11:12:17.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.536+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/556560066 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd01e0 0x55aac2cd2c90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c1614c0 tx=0x7f5e20013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff7b3dc9ea421296 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.536+0000 7f5e0bfff700 1 -- 172.21.15.28:0/556560066 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20016690 con 0x55aac2cd01e0 2023-11-28T11:12:17.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.536+0000 7f5e0bfff700 1 -- 172.21.15.28:0/556560066 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2001a070 con 0x55aac2cd01e0 2023-11-28T11:12:17.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.536+0000 7f5e0bfff700 1 -- 172.21.15.28:0/556560066 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20016a10 con 0x55aac2cd01e0 2023-11-28T11:12:17.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.537+0000 7f5e3d100b80 1 -- 172.21.15.28:0/556560066 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd01e0 msgr2=0x55aac2cd2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.537+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/556560066 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd01e0 0x55aac2cd2c90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c1614c0 tx=0x7f5e20013000 comp rx=0 tx=0).stop 2023-11-28T11:12:17.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.537+0000 7f5e3d100b80 1 -- 172.21.15.28:0/556560066 shutdown_connections 2023-11-28T11:12:17.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.537+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/556560066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2cee820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.537+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/556560066 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd28c0 0x55aac2ca8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.537+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/556560066 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd01e0 0x55aac2cd2c90 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.537+0000 7f5e3d100b80 1 -- 172.21.15.28:0/556560066 >> 172.21.15.28:0/556560066 conn(0x55aac2ca5e10 msgr2=0x55aac2ca67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.537+0000 7f5e3d100b80 1 -- 172.21.15.28:0/556560066 shutdown_connections 2023-11-28T11:12:17.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.537+0000 7f5e3d100b80 1 -- 172.21.15.28:0/556560066 wait complete. 2023-11-28T11:12:17.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.537+0000 7f5e3d100b80 1 -- start start 2023-11-28T11:12:17.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.538+0000 7f5e3d100b80 1 Processor -- start 2023-11-28T11:12:17.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.538+0000 7f5e3d100b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 0x55aac2cc1360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.538+0000 7f5e3d100b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2cd6520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.538+0000 7f5e3d100b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2cc1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e3d100b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac2a7e980 con 0x55aac2cd28c0 2023-11-28T11:12:17.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e3d100b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac2cc18a0 con 0x55aac2cd31d0 2023-11-28T11:12:17.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e3d100b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aac2cd6a60 con 0x55aac2cd01e0 2023-11-28T11:12:17.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e3d100b80 10 client.0 _ll_register_callbacks cb 0x55aac2aa02f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:12:17.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e29d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2cd6520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e2a59c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 0x55aac2cc1360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e2ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2cc1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e29d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2cd6520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51876/0 (socket says 172.21.15.28:51876) 2023-11-28T11:12:17.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e29d9b700 1 -- 172.21.15.28:0/3081135969 learned_addr learned my addr 172.21.15.28:0/3081135969 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:17.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e2a59c700 1 -- 172.21.15.28:0/3081135969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 msgr2=0x55aac2cc1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e2a59c700 1 --2- 172.21.15.28:0/3081135969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2cc1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e2a59c700 1 -- 172.21.15.28:0/3081135969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 msgr2=0x55aac2cd6520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e2a59c700 1 --2- 172.21.15.28:0/3081135969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2cd6520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e2a59c700 1 -- 172.21.15.28:0/3081135969 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e2000e040 con 0x55aac2cd01e0 2023-11-28T11:12:17.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/3081135969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2cd6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:17.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e2ad9d700 1 --2- 172.21.15.28:0/3081135969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2cc1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:17.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e2a59c700 1 --2- 172.21.15.28:0/3081135969 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 0x55aac2cc1360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0522b0 tx=0x7f5e1c0555d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b9d5b44d130c5111 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e097fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3301/0 2023-11-28T11:12:17.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e097fa700 1 -- 172.21.15.28:0/3081135969 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e1c05dd10 con 0x55aac2cd01e0 2023-11-28T11:12:17.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3081135969 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aac2bc81f0 con 0x55aac2cd01e0 2023-11-28T11:12:17.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e097fa700 1 -- 172.21.15.28:0/3081135969 <== mon.2 v2:172.21.15.59:3301/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5e1c058040 con 0x55aac2cd01e0 2023-11-28T11:12:17.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e097fa700 1 -- 172.21.15.28:0/3081135969 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e1c051bf0 con 0x55aac2cd01e0 2023-11-28T11:12:17.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e3d100b80 10 client.5615 fetch_fsmap learned FSMap version 33 2023-11-28T11:12:17.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.539+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3081135969 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55aac2a28e40 con 0x55aac2cd01e0 2023-11-28T11:12:17.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.540+0000 7f5e097fa700 1 -- 172.21.15.28:0/3081135969 <== mon.2 v2:172.21.15.59:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aac2bc81f0 con 0x55aac2cd01e0 2023-11-28T11:12:17.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.540+0000 7f5e097fa700 1 -- 172.21.15.28:0/3081135969 <== mon.2 v2:172.21.15.59:3301/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5e1c051d50 con 0x55aac2cd01e0 2023-11-28T11:12:17.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.540+0000 7f5e3d100b80 10 client.5615 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:12:17.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.540+0000 7f5e3d100b80 10 client.5615 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2023-11-28T11:12:17.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.540+0000 7f5e3d100b80 20 client.5615 populate_metadata read hostname 'smithi028' 2023-11-28T11:12:17.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.540+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/3081135969 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55aac2ce50d0 0x55aac2ce54a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.540+0000 7f5e3d100b80 4 client.5615 mds_command: new command op to 6573 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:12:17.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.540+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3081135969 --> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55aac2a7efc0 con 0x55aac2ce50d0 2023-11-28T11:12:17.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.540+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/3081135969 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55aac2ce50d0 0x55aac2ce54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.540+0000 7f5e097fa700 10 client.5615 ms_handle_connect on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:17.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.540+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/3081135969 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55aac2ce50d0 0x55aac2ce54a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e8113561e359deb server_cookie=d38f6ec90d2a4a99 in_seq=0 out_seq=0 2023-11-28T11:12:17.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.541+0000 7f5e097fa700 1 -- 172.21.15.28:0/3081135969 <== mds.0 v2:172.21.15.59:6832/2406852565 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55aac2a7efc0 con 0x55aac2ce50d0 2023-11-28T11:12:17.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.541+0000 7f5e097fa700 10 client.5615 handle_command_reply: tid=0 2023-11-28T11:12:17.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.541+0000 7f5e3d100b80 1 client.5615 shutdown 2023-11-28T11:12:17.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.541+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3081135969 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 msgr2=0x55aac2cc1360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.541+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/3081135969 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 0x55aac2cc1360 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c0522b0 tx=0x7f5e1c0555d0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.541+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3081135969 shutdown_connections 2023-11-28T11:12:17.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.541+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/3081135969 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2cc1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.541+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/3081135969 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 0x55aac2cc1360 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.541+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/3081135969 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55aac2ce50d0 0x55aac2ce54a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.541+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/3081135969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2cd6520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.541+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3081135969 >> 172.21.15.28:0/3081135969 conn(0x55aac2ca5e10 msgr2=0x55aac2ca72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.541+0000 7f5e097fa700 0 client.5615 ms_handle_reset on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:17.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.541+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3081135969 shutdown_connections 2023-11-28T11:12:17.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.541+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3081135969 wait complete. 2023-11-28T11:12:17.554 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.541+0000 7f5e3d100b80 20 client.5615 trim_cache size 0 max 16384 2023-11-28T11:12:17.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.558+0000 7f5e3d100b80 1 Processor -- start 2023-11-28T11:12:17.560 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.558+0000 7f5e3d100b80 1 -- start start 2023-11-28T11:12:17.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e3d100b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 0x55aac2ca7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e3d100b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2ca3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.562 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e3d100b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2cd6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e3d100b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac2cc0970 con 0x55aac2cd28c0 2023-11-28T11:12:17.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e3d100b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac2c9c280 con 0x55aac2cd31d0 2023-11-28T11:12:17.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e3d100b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aac2ce5fd0 con 0x55aac2cd01e0 2023-11-28T11:12:17.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e29d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2ca3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e29d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2ca3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51884/0 (socket says 172.21.15.28:51884) 2023-11-28T11:12:17.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e29d9b700 1 -- 172.21.15.28:0/9613007 learned_addr learned my addr 172.21.15.28:0/9613007 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:17.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e29d9b700 1 -- 172.21.15.28:0/9613007 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 msgr2=0x55aac2ca7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e2a59c700 1 --2- 172.21.15.28:0/9613007 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 0x55aac2ca7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/9613007 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 0x55aac2ca7810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.563 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e2ad9d700 1 --2- 172.21.15.28:0/9613007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2cd6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e29d9b700 1 -- 172.21.15.28:0/9613007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 msgr2=0x55aac2cd6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/9613007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2cd6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e29d9b700 1 -- 172.21.15.28:0/9613007 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55aac2bc11f0 con 0x55aac2cd28c0 2023-11-28T11:12:17.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/9613007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2ca3000 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x55aac2ca4180 tx=0x7f5e20015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=530c3476c0616b2f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.559+0000 7f5e18ff9700 1 -- 172.21.15.28:0/9613007 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20031040 con 0x55aac2cd28c0 2023-11-28T11:12:17.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.560+0000 7f5e18ff9700 1 -- 172.21.15.28:0/9613007 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2001a040 con 0x55aac2cd28c0 2023-11-28T11:12:17.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.560+0000 7f5e18ff9700 1 -- 172.21.15.28:0/9613007 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e200169a0 con 0x55aac2cd28c0 2023-11-28T11:12:17.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.560+0000 7f5e3d100b80 1 -- 172.21.15.28:0/9613007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 msgr2=0x55aac2ca3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.560+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/9613007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2ca3000 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x55aac2ca4180 tx=0x7f5e20015000 comp rx=0 tx=0).stop 2023-11-28T11:12:17.564 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.560+0000 7f5e3d100b80 1 -- 172.21.15.28:0/9613007 shutdown_connections 2023-11-28T11:12:17.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.560+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/9613007 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2cd6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.560+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/9613007 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 0x55aac2ca7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.560+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/9613007 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2ca3000 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.560+0000 7f5e3d100b80 1 -- 172.21.15.28:0/9613007 >> 172.21.15.28:0/9613007 conn(0x55aac2ca5e10 msgr2=0x55aac2ca8b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.560+0000 7f5e3d100b80 1 -- 172.21.15.28:0/9613007 shutdown_connections 2023-11-28T11:12:17.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.560+0000 7f5e3d100b80 1 -- 172.21.15.28:0/9613007 wait complete. 2023-11-28T11:12:17.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.561+0000 7f5e3d100b80 1 -- start start 2023-11-28T11:12:17.565 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.561+0000 7f5e3d100b80 1 Processor -- start 2023-11-28T11:12:17.566 INFO:tasks.ceph.mds.d.smithi059.stderr:2023-11-28T11:12:17.566+0000 7fa848f87700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6768=0-4194304@1} caps={6768=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6768 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x565370f6db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:12:17.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e3d100b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 0x55aac2cd6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e3d100b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2ca97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e3d100b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2ca9d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.569 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e3d100b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac2ccec30 con 0x55aac2cd28c0 2023-11-28T11:12:17.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e3d100b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55aac2cd67a0 con 0x55aac2cd31d0 2023-11-28T11:12:17.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e3d100b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55aac2cd6550 con 0x55aac2cd01e0 2023-11-28T11:12:17.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e3d100b80 10 client.0 _ll_register_callbacks cb 0x55aac2aa02f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:12:17.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e29d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2ca97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e29d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2ca97e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51890/0 (socket says 172.21.15.28:51890) 2023-11-28T11:12:17.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e29d9b700 1 -- 172.21.15.28:0/3592790827 learned_addr learned my addr 172.21.15.28:0/3592790827 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:17.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e2a59c700 1 --2- 172.21.15.28:0/3592790827 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 0x55aac2cd6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e2ad9d700 1 --2- 172.21.15.28:0/3592790827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2ca9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e29d9b700 1 -- 172.21.15.28:0/3592790827 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 msgr2=0x55aac2cd6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/3592790827 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 0x55aac2cd6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.570 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e29d9b700 1 -- 172.21.15.28:0/3592790827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 msgr2=0x55aac2ca9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/3592790827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2ca9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e29d9b700 1 -- 172.21.15.28:0/3592790827 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55aac2a7c690 con 0x55aac2cd28c0 2023-11-28T11:12:17.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e29d9b700 1 --2- 172.21.15.28:0/3592790827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2ca97e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f5e20006f10 tx=0x7f5e200047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d26873264894e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e097fa700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:12:17.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e097fa700 1 -- 172.21.15.28:0/3592790827 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e20034290 con 0x55aac2cd28c0 2023-11-28T11:12:17.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e097fa700 1 -- 172.21.15.28:0/3592790827 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f5e200343f0 con 0x55aac2cd28c0 2023-11-28T11:12:17.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.562+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3592790827 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55aac2bb8210 con 0x55aac2cd28c0 2023-11-28T11:12:17.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.563+0000 7f5e097fa700 1 -- 172.21.15.28:0/3592790827 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e20005a20 con 0x55aac2cd28c0 2023-11-28T11:12:17.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.563+0000 7f5e3d100b80 10 client.6813 fetch_fsmap learned FSMap version 33 2023-11-28T11:12:17.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.563+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3592790827 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x55aac2a4f2b0 con 0x55aac2cd28c0 2023-11-28T11:12:17.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.563+0000 7f5e097fa700 1 -- 172.21.15.28:0/3592790827 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55aac2bb8210 con 0x55aac2cd28c0 2023-11-28T11:12:17.571 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.563+0000 7f5e097fa700 1 -- 172.21.15.28:0/3592790827 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f5e2003a020 con 0x55aac2cd28c0 2023-11-28T11:12:17.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.563+0000 7f5e3d100b80 10 client.6813 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:12:17.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.563+0000 7f5e3d100b80 10 client.6813 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2023-11-28T11:12:17.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.563+0000 7f5e3d100b80 20 client.6813 populate_metadata read hostname 'smithi028' 2023-11-28T11:12:17.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.563+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/3592790827 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55aac2d00140 0x55aac2ccd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.563+0000 7f5e3d100b80 4 client.6813 mds_command: new command op to 6573 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}] 2023-11-28T11:12:17.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.563+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3592790827 --> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["recursive"]}) v1 -- 0x55aac2ba2220 con 0x55aac2d00140 2023-11-28T11:12:17.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.563+0000 7f5e2a59c700 1 --2- 172.21.15.28:0/3592790827 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55aac2d00140 0x55aac2ccd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.563+0000 7f5e097fa700 10 client.6813 ms_handle_connect on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:17.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.563+0000 7f5e2a59c700 1 --2- 172.21.15.28:0/3592790827 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55aac2d00140 0x55aac2ccd8f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69188c5d70ee1f93 server_cookie=8247c79063e2ffb in_seq=0 out_seq=0 2023-11-28T11:12:17.572 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.564+0000 7f5e097fa700 1 -- 172.21.15.28:0/3592790827 <== mds.0 v2:172.21.15.59:6832/2406852565 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x55aac2ba2220 con 0x55aac2d00140 2023-11-28T11:12:17.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.564+0000 7f5e097fa700 10 client.6813 handle_command_reply: tid=0 2023-11-28T11:12:17.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.564+0000 7f5e3d100b80 1 client.6813 shutdown 2023-11-28T11:12:17.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3592790827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 msgr2=0x55aac2ca97e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/3592790827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2ca97e0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f5e20006f10 tx=0x7f5e200047c0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3592790827 shutdown_connections 2023-11-28T11:12:17.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/3592790827 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aac2cd31d0 0x55aac2ca9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/3592790827 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55aac2cd01e0 0x55aac2cd6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/3592790827 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55aac2d00140 0x55aac2ccd8f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e3d100b80 1 --2- 172.21.15.28:0/3592790827 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55aac2cd28c0 0x55aac2ca97e0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e097fa700 0 client.6813 ms_handle_reset on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:17.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3592790827 >> 172.21.15.28:0/3592790827 conn(0x55aac2ca5e10 msgr2=0x55aac2ca35e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3592790827 shutdown_connections 2023-11-28T11:12:17.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e3d100b80 1 -- 172.21.15.28:0/3592790827 wait complete. 2023-11-28T11:12:17.574 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e3d100b80 20 client.6813 trim_cache size 0 max 16384 2023-11-28T11:12:17.574 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:12:17.574 INFO:teuthology.orchestra.run.smithi028.stdout: "return_code": 0, 2023-11-28T11:12:17.574 INFO:teuthology.orchestra.run.smithi028.stdout: "scrub_tag": "fa253638-ba69-480f-bef8-47a08e461703", 2023-11-28T11:12:17.574 INFO:teuthology.orchestra.run.smithi028.stdout: "mode": "asynchronous" 2023-11-28T11:12:17.575 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:12:17.575 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e30f77700 1 -- 172.21.15.28:0/2025182365 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5e0c0682f0 msgr2=0x7f5e0c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e30f77700 1 --2- 172.21.15.28:0/2025182365 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5e0c0682f0 0x7f5e0c06a7a0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c0abfa0 tx=0x7f5e20008040 comp rx=0 tx=0).stop 2023-11-28T11:12:17.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e30f77700 1 -- 172.21.15.28:0/2025182365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e2c0caeb0 msgr2=0x7f5e2c0c55d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.565+0000 7f5e30f77700 1 --2- 172.21.15.28:0/2025182365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e2c0caeb0 0x7f5e2c0c55d0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c00e810 tx=0x7f5e1c00eb20 comp rx=0 tx=0).stop 2023-11-28T11:12:17.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.566+0000 7f5e30f77700 1 -- 172.21.15.28:0/2025182365 shutdown_connections 2023-11-28T11:12:17.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.566+0000 7f5e30f77700 1 --2- 172.21.15.28:0/2025182365 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5e0c0682f0 0x7f5e0c06a7a0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.566+0000 7f5e30f77700 1 --2- 172.21.15.28:0/2025182365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5e2c0caeb0 0x7f5e2c0c55d0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.566+0000 7f5e30f77700 1 --2- 172.21.15.28:0/2025182365 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5e2c161090 0x7f5e2c148a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.566+0000 7f5e30f77700 1 --2- 172.21.15.28:0/2025182365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5e2c15c990 0x7f5e2c0c89d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.566+0000 7f5e30f77700 1 -- 172.21.15.28:0/2025182365 >> 172.21.15.28:0/2025182365 conn(0x7f5e2c0b1650 msgr2=0x7f5e2c06b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.566+0000 7f5e30f77700 1 -- 172.21.15.28:0/2025182365 shutdown_connections 2023-11-28T11:12:17.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.566+0000 7f5e30f77700 1 -- 172.21.15.28:0/2025182365 wait complete. 2023-11-28T11:12:17.579 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T11:12:17.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.704+0000 7f9c66299700 1 Processor -- start 2023-11-28T11:12:17.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.704+0000 7f9c66299700 1 -- start start 2023-11-28T11:12:17.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.705+0000 7f9c66299700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9c600cf5f0 0x7f9c600caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.705+0000 7f9c66299700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9c600cb3e0 0x7f9c6006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.705+0000 7f9c66299700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c600ce550 con 0x7f9c600b1630 2023-11-28T11:12:17.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.705+0000 7f9c66299700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c600ce830 con 0x7f9c600cf5f0 2023-11-28T11:12:17.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.705+0000 7f9c66299700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c600ceb10 con 0x7f9c600cb3e0 2023-11-28T11:12:17.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.705+0000 7f9c5f7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9c600cf5f0 0x7f9c600caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.705+0000 7f9c5ffff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9c600b1630 0x7f9c600b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:39954/0 (socket says 172.21.15.28:39954) 2023-11-28T11:12:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.705+0000 7f9c5ffff700 1 -- 172.21.15.28:0/910249323 learned_addr learned my addr 172.21.15.28:0/910249323 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.705+0000 7f9c64836700 1 --2- 172.21.15.28:0/910249323 >> v2:172.21.15.59:3300/0 conn(0x7f9c600cb3e0 0x7f9c6006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.705+0000 7f9c5f7fe700 1 -- 172.21.15.28:0/910249323 >> v2:172.21.15.59:3300/0 conn(0x7f9c600cb3e0 msgr2=0x7f9c6006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.705+0000 7f9c5f7fe700 1 --2- 172.21.15.28:0/910249323 >> v2:172.21.15.59:3300/0 conn(0x7f9c600cb3e0 0x7f9c6006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.705+0000 7f9c5f7fe700 1 -- 172.21.15.28:0/910249323 >> v1:172.21.15.28:6789/0 conn(0x7f9c600b1630 legacy=0x7f9c600b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.705+0000 7f9c5f7fe700 1 -- 172.21.15.28:0/910249323 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c600cedf0 con 0x7f9c600cf5f0 2023-11-28T11:12:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.705+0000 7f9c5effd700 1 -- 172.21.15.28:0/910249323 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385980443 0 0) 0x7f9c600ce550 con 0x7f9c600b1630 2023-11-28T11:12:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.705+0000 7f9c5f7fe700 1 --2- 172.21.15.28:0/910249323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c600cf5f0 0x7f9c600caea0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009840 tx=0x7f9c50009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c04c616eac05c39 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.706+0000 7f9c5effd700 1 -- 172.21.15.28:0/910249323 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50011830 con 0x7f9c600cf5f0 2023-11-28T11:12:17.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.706+0000 7f9c5effd700 1 -- 172.21.15.28:0/910249323 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c50011990 con 0x7f9c600cf5f0 2023-11-28T11:12:17.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.706+0000 7f9c5effd700 1 -- 172.21.15.28:0/910249323 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50011d20 con 0x7f9c600cf5f0 2023-11-28T11:12:17.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.706+0000 7f9c66299700 1 -- 172.21.15.28:0/910249323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c600cf5f0 msgr2=0x7f9c600caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.706+0000 7f9c66299700 1 --2- 172.21.15.28:0/910249323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c600cf5f0 0x7f9c600caea0 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009840 tx=0x7f9c50009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:17.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.706+0000 7f9c66299700 1 -- 172.21.15.28:0/910249323 shutdown_connections 2023-11-28T11:12:17.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.706+0000 7f9c66299700 1 --2- 172.21.15.28:0/910249323 >> v2:172.21.15.59:3300/0 conn(0x7f9c600cb3e0 0x7f9c6006afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.706+0000 7f9c66299700 1 --2- 172.21.15.28:0/910249323 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c600cf5f0 0x7f9c600caea0 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.706+0000 7f9c66299700 1 -- 172.21.15.28:0/910249323 >> 172.21.15.28:0/910249323 conn(0x7f9c600cc630 msgr2=0x7f9c600cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.706+0000 7f9c66299700 1 -- 172.21.15.28:0/910249323 shutdown_connections 2023-11-28T11:12:17.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.706+0000 7f9c66299700 1 -- 172.21.15.28:0/910249323 wait complete. 2023-11-28T11:12:17.712 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.708+0000 7f9c66299700 1 Processor -- start 2023-11-28T11:12:17.712 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.708+0000 7f9c66299700 1 -- start start 2023-11-28T11:12:17.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c66299700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c600b1630 0x7f9c600c8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c66299700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c600cb3e0 0x7f9c6015bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c66299700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c600cf5f0 0x7f9c600b4bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c66299700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c601f2860 con 0x7f9c600cb3e0 2023-11-28T11:12:17.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c66299700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c601f2b40 con 0x7f9c600cf5f0 2023-11-28T11:12:17.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c66299700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c601f2e20 con 0x7f9c600b1630 2023-11-28T11:12:17.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c5f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c600cb3e0 0x7f9c6015bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c5f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c600cb3e0 0x7f9c6015bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51904/0 (socket says 172.21.15.28:51904) 2023-11-28T11:12:17.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c5f7fe700 1 -- 172.21.15.28:0/3677063209 learned_addr learned my addr 172.21.15.28:0/3677063209 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:17.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c64836700 1 --2- 172.21.15.28:0/3677063209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c600cf5f0 0x7f9c600b4bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c5ffff700 1 --2- 172.21.15.28:0/3677063209 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c600b1630 0x7f9c600c8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c5f7fe700 1 -- 172.21.15.28:0/3677063209 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c600b1630 msgr2=0x7f9c600c8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c5f7fe700 1 --2- 172.21.15.28:0/3677063209 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c600b1630 0x7f9c600c8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c5f7fe700 1 -- 172.21.15.28:0/3677063209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c600cf5f0 msgr2=0x7f9c600b4bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c5f7fe700 1 --2- 172.21.15.28:0/3677063209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c600cf5f0 0x7f9c600b4bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c5f7fe700 1 -- 172.21.15.28:0/3677063209 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c500094a0 con 0x7f9c600cb3e0 2023-11-28T11:12:17.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c5ffff700 1 --2- 172.21.15.28:0/3677063209 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c600b1630 0x7f9c600c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:17.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.709+0000 7f9c5f7fe700 1 --2- 172.21.15.28:0/3677063209 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c600cb3e0 0x7f9c6015bf70 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009b90 tx=0x7f9c50004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.710+0000 7f9c5cff9700 1 -- 172.21.15.28:0/3677063209 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c500036a0 con 0x7f9c600cb3e0 2023-11-28T11:12:17.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.710+0000 7f9c5cff9700 1 -- 172.21.15.28:0/3677063209 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5001f070 con 0x7f9c600cb3e0 2023-11-28T11:12:17.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.710+0000 7f9c66299700 1 -- 172.21.15.28:0/3677063209 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c600ce140 con 0x7f9c600cb3e0 2023-11-28T11:12:17.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.710+0000 7f9c5cff9700 1 -- 172.21.15.28:0/3677063209 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c50003a20 con 0x7f9c600cb3e0 2023-11-28T11:12:17.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.710+0000 7f9c66299700 1 -- 172.21.15.28:0/3677063209 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c600cf0d0 con 0x7f9c600cb3e0 2023-11-28T11:12:17.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.711+0000 7f9c5cff9700 1 -- 172.21.15.28:0/3677063209 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9c5000f050 con 0x7f9c600cb3e0 2023-11-28T11:12:17.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.712+0000 7f9c5cff9700 1 --2- 172.21.15.28:0/3677063209 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9c4c0682f0 0x7f9c4c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.712+0000 7f9c5cff9700 1 -- 172.21.15.28:0/3677063209 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9c50021070 con 0x7f9c600cb3e0 2023-11-28T11:12:17.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.712+0000 7f9c5ffff700 1 --2- 172.21.15.28:0/3677063209 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9c4c0682f0 0x7f9c4c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.712+0000 7f9c5ffff700 1 --2- 172.21.15.28:0/3677063209 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9c4c0682f0 0x7f9c4c06a7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9c54000f60 tx=0x7f9c54007ca0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.714+0000 7f9c72422b80 1 Processor -- start 2023-11-28T11:12:17.718 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.714+0000 7f9c72422b80 1 -- start start 2023-11-28T11:12:17.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.714+0000 7f9c72422b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a330a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.714+0000 7f9c72422b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a358180 0x557b0a35ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.714+0000 7f9c72422b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a35b170 0x557b0a3767c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.714+0000 7f9c72422b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0a21ac10 con 0x557b0a35a860 2023-11-28T11:12:17.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.714+0000 7f9c72422b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0a2893a0 con 0x557b0a358180 2023-11-28T11:12:17.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.714+0000 7f9c72422b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b0a2828d0 con 0x557b0a35b170 2023-11-28T11:12:17.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.714+0000 7f9c5ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a330a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.714+0000 7f9c5ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a330a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51910/0 (socket says 172.21.15.28:51910) 2023-11-28T11:12:17.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.714+0000 7f9c5ffff700 1 -- 172.21.15.28:0/3931851581 learned_addr learned my addr 172.21.15.28:0/3931851581 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:17.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.715+0000 7f9c64836700 1 --2- 172.21.15.28:0/3931851581 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a35b170 0x557b0a3767c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.719 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.715+0000 7f9c5f7fe700 1 --2- 172.21.15.28:0/3931851581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a358180 0x557b0a35ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.715+0000 7f9c5ffff700 1 -- 172.21.15.28:0/3931851581 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a35b170 msgr2=0x557b0a3767c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.715+0000 7f9c5ffff700 1 --2- 172.21.15.28:0/3931851581 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a35b170 0x557b0a3767c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.715+0000 7f9c5ffff700 1 -- 172.21.15.28:0/3931851581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a358180 msgr2=0x557b0a35ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.715+0000 7f9c5ffff700 1 --2- 172.21.15.28:0/3931851581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a358180 0x557b0a35ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.715+0000 7f9c5ffff700 1 -- 172.21.15.28:0/3931851581 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b0a104630 con 0x557b0a35a860 2023-11-28T11:12:17.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.715+0000 7f9c5f7fe700 1 --2- 172.21.15.28:0/3931851581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a358180 0x557b0a35ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:17.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.715+0000 7f9c5ffff700 1 --2- 172.21.15.28:0/3931851581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a330a70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9c540156d0 tx=0x7f9c54012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80092eae358cd72c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.720 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.715+0000 7f9c45ffb700 1 -- 172.21.15.28:0/3931851581 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54015e70 con 0x557b0a35a860 2023-11-28T11:12:17.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.715+0000 7f9c45ffb700 1 -- 172.21.15.28:0/3931851581 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c54015320 con 0x557b0a35a860 2023-11-28T11:12:17.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.715+0000 7f9c45ffb700 1 -- 172.21.15.28:0/3931851581 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c54004a70 con 0x557b0a35a860 2023-11-28T11:12:17.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.716+0000 7f9c72422b80 1 -- 172.21.15.28:0/3931851581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 msgr2=0x557b0a330a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.716+0000 7f9c72422b80 1 --2- 172.21.15.28:0/3931851581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a330a70 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f9c540156d0 tx=0x7f9c54012000 comp rx=0 tx=0).stop 2023-11-28T11:12:17.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.716+0000 7f9c72422b80 1 -- 172.21.15.28:0/3931851581 shutdown_connections 2023-11-28T11:12:17.721 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.716+0000 7f9c72422b80 1 --2- 172.21.15.28:0/3931851581 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a330a70 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.716+0000 7f9c72422b80 1 --2- 172.21.15.28:0/3931851581 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a35b170 0x557b0a3767c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.716+0000 7f9c72422b80 1 --2- 172.21.15.28:0/3931851581 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a358180 0x557b0a35ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.716+0000 7f9c72422b80 1 -- 172.21.15.28:0/3931851581 >> 172.21.15.28:0/3931851581 conn(0x557b0a32ddb0 msgr2=0x557b0a32e740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.716+0000 7f9c72422b80 1 -- 172.21.15.28:0/3931851581 shutdown_connections 2023-11-28T11:12:17.722 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.716+0000 7f9c72422b80 1 -- 172.21.15.28:0/3931851581 wait complete. 2023-11-28T11:12:17.723 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.716+0000 7f9c72422b80 1 -- start start 2023-11-28T11:12:17.723 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.717+0000 7f9c72422b80 1 Processor -- start 2023-11-28T11:12:17.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.717+0000 7f9c72422b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a358180 0x557b0a34b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.726 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.717+0000 7f9c72422b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a35a860 0x557b0a3599b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.717+0000 7f9c72422b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35b170 0x557b0a350580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.717+0000 7f9c72422b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0a106920 con 0x557b0a35b170 2023-11-28T11:12:17.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.717+0000 7f9c72422b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0a34f070 con 0x557b0a358180 2023-11-28T11:12:17.727 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.717+0000 7f9c72422b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b0a35d420 con 0x557b0a35a860 2023-11-28T11:12:17.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c72422b80 10 client.0 _ll_register_callbacks cb 0x557b0a128290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:12:17.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c64836700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35b170 0x557b0a350580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c5ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a358180 0x557b0a34b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c64836700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35b170 0x557b0a350580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51920/0 (socket says 172.21.15.28:51920) 2023-11-28T11:12:17.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c64836700 1 -- 172.21.15.28:0/1325624306 learned_addr learned my addr 172.21.15.28:0/1325624306 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:17.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c5f7fe700 1 --2- 172.21.15.28:0/1325624306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a35a860 0x557b0a3599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c64836700 1 -- 172.21.15.28:0/1325624306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a35a860 msgr2=0x557b0a3599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c64836700 1 --2- 172.21.15.28:0/1325624306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a35a860 0x557b0a3599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.728 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c64836700 1 -- 172.21.15.28:0/1325624306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a358180 msgr2=0x557b0a34b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c64836700 1 --2- 172.21.15.28:0/1325624306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a358180 0x557b0a34b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c64836700 1 -- 172.21.15.28:0/1325624306 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c5400f040 con 0x557b0a35b170 2023-11-28T11:12:17.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c5ffff700 1 --2- 172.21.15.28:0/1325624306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a358180 0x557b0a34b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:17.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c64836700 1 --2- 172.21.15.28:0/1325624306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35b170 0x557b0a350580 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9c58014db0 tx=0x7f9c58012580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=221733aeba27e235 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c3effd700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:12:17.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c3effd700 1 -- 172.21.15.28:0/1325624306 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5800e040 con 0x557b0a35b170 2023-11-28T11:12:17.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c3effd700 1 -- 172.21.15.28:0/1325624306 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9c5801a070 con 0x557b0a35b170 2023-11-28T11:12:17.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c72422b80 1 -- 172.21.15.28:0/1325624306 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b0a250190 con 0x557b0a35b170 2023-11-28T11:12:17.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c3effd700 1 -- 172.21.15.28:0/1325624306 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c58003e30 con 0x557b0a35b170 2023-11-28T11:12:17.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c72422b80 10 client.6828 fetch_fsmap learned FSMap version 33 2023-11-28T11:12:17.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.718+0000 7f9c72422b80 1 -- 172.21.15.28:0/1325624306 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x557b0a0b0de0 con 0x557b0a35b170 2023-11-28T11:12:17.729 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.719+0000 7f9c3effd700 1 -- 172.21.15.28:0/1325624306 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557b0a250190 con 0x557b0a35b170 2023-11-28T11:12:17.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.719+0000 7f9c3effd700 1 -- 172.21.15.28:0/1325624306 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9c58012a60 con 0x557b0a35b170 2023-11-28T11:12:17.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.719+0000 7f9c72422b80 10 client.6828 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:12:17.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.719+0000 7f9c72422b80 10 client.6828 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2023-11-28T11:12:17.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.719+0000 7f9c72422b80 20 client.6828 populate_metadata read hostname 'smithi028' 2023-11-28T11:12:17.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.719+0000 7f9c72422b80 1 --2- 172.21.15.28:0/1325624306 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x557b0a34d0f0 0x557b0a35c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.719+0000 7f9c72422b80 4 client.6828 mds_command: new command op to 6573 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:12:17.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.719+0000 7f9c72422b80 1 -- 172.21.15.28:0/1325624306 --> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557b0a106f60 con 0x557b0a34d0f0 2023-11-28T11:12:17.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.719+0000 7f9c5ffff700 1 --2- 172.21.15.28:0/1325624306 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x557b0a34d0f0 0x557b0a35c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.719+0000 7f9c3effd700 10 client.6828 ms_handle_connect on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:17.730 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.719+0000 7f9c5ffff700 1 --2- 172.21.15.28:0/1325624306 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x557b0a34d0f0 0x557b0a35c8a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7217f2193c0b3d0 server_cookie=7b90b17dc2a88f6a in_seq=0 out_seq=0 2023-11-28T11:12:17.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.720+0000 7f9c3effd700 1 -- 172.21.15.28:0/1325624306 <== mds.0 v2:172.21.15.59:6832/2406852565 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x557b0a106f60 con 0x557b0a34d0f0 2023-11-28T11:12:17.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.720+0000 7f9c3effd700 10 client.6828 handle_command_reply: tid=0 2023-11-28T11:12:17.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.720+0000 7f9c72422b80 1 client.6828 shutdown 2023-11-28T11:12:17.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.720+0000 7f9c72422b80 1 -- 172.21.15.28:0/1325624306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35b170 msgr2=0x557b0a350580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.720+0000 7f9c72422b80 1 --2- 172.21.15.28:0/1325624306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35b170 0x557b0a350580 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f9c58014db0 tx=0x7f9c58012580 comp rx=0 tx=0).stop 2023-11-28T11:12:17.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.720+0000 7f9c72422b80 1 -- 172.21.15.28:0/1325624306 shutdown_connections 2023-11-28T11:12:17.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.720+0000 7f9c72422b80 1 --2- 172.21.15.28:0/1325624306 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x557b0a34d0f0 0x557b0a35c8a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.720+0000 7f9c72422b80 1 --2- 172.21.15.28:0/1325624306 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35b170 0x557b0a350580 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.720+0000 7f9c72422b80 1 --2- 172.21.15.28:0/1325624306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a358180 0x557b0a34b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.720+0000 7f9c3effd700 0 client.6828 ms_handle_reset on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:17.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.720+0000 7f9c72422b80 1 --2- 172.21.15.28:0/1325624306 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a35a860 0x557b0a3599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.720+0000 7f9c72422b80 1 -- 172.21.15.28:0/1325624306 >> 172.21.15.28:0/1325624306 conn(0x557b0a32ddb0 msgr2=0x557b0a32f280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.720+0000 7f9c72422b80 1 -- 172.21.15.28:0/1325624306 shutdown_connections 2023-11-28T11:12:17.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.720+0000 7f9c72422b80 1 -- 172.21.15.28:0/1325624306 wait complete. 2023-11-28T11:12:17.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.720+0000 7f9c72422b80 20 client.6828 trim_cache size 0 max 16384 2023-11-28T11:12:17.738 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.737+0000 7f9c72422b80 1 Processor -- start 2023-11-28T11:12:17.739 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.737+0000 7f9c72422b80 1 -- start start 2023-11-28T11:12:17.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c72422b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a358180 0x557b0a34e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c72422b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a34ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c72422b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a35b170 0x557b0a35c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c72422b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0a34afb0 con 0x557b0a35a860 2023-11-28T11:12:17.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c72422b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0a35eb70 con 0x557b0a35b170 2023-11-28T11:12:17.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c72422b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b0a325a40 con 0x557b0a358180 2023-11-28T11:12:17.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c5f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a34ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c5f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a34ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51926/0 (socket says 172.21.15.28:51926) 2023-11-28T11:12:17.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c5f7fe700 1 -- 172.21.15.28:0/668007830 learned_addr learned my addr 172.21.15.28:0/668007830 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:17.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c5f7fe700 1 -- 172.21.15.28:0/668007830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a358180 msgr2=0x557b0a34e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c5ffff700 1 --2- 172.21.15.28:0/668007830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a358180 0x557b0a34e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c64836700 1 --2- 172.21.15.28:0/668007830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a35b170 0x557b0a35c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.742 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c5f7fe700 1 --2- 172.21.15.28:0/668007830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a358180 0x557b0a34e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c5f7fe700 1 -- 172.21.15.28:0/668007830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a35b170 msgr2=0x557b0a35c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c5f7fe700 1 --2- 172.21.15.28:0/668007830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a35b170 0x557b0a35c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c5f7fe700 1 -- 172.21.15.28:0/668007830 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557b0a249190 con 0x557b0a35a860 2023-11-28T11:12:17.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c5ffff700 1 --2- 172.21.15.28:0/668007830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a358180 0x557b0a34e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:17.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c64836700 1 --2- 172.21.15.28:0/668007830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a35b170 0x557b0a35c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:17.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c5f7fe700 1 --2- 172.21.15.28:0/668007830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a34ea90 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9c50053e60 tx=0x7f9c50006d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=712bb01691be2655 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c467fc700 1 -- 172.21.15.28:0/668007830 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5005f9f0 con 0x557b0a35a860 2023-11-28T11:12:17.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c467fc700 1 -- 172.21.15.28:0/668007830 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5006d070 con 0x557b0a35a860 2023-11-28T11:12:17.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.738+0000 7f9c467fc700 1 -- 172.21.15.28:0/668007830 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5005f760 con 0x557b0a35a860 2023-11-28T11:12:17.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.739+0000 7f9c72422b80 1 -- 172.21.15.28:0/668007830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 msgr2=0x557b0a34ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.739+0000 7f9c72422b80 1 --2- 172.21.15.28:0/668007830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a34ea90 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f9c50053e60 tx=0x7f9c50006d60 comp rx=0 tx=0).stop 2023-11-28T11:12:17.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.739+0000 7f9c72422b80 1 -- 172.21.15.28:0/668007830 shutdown_connections 2023-11-28T11:12:17.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.739+0000 7f9c72422b80 1 --2- 172.21.15.28:0/668007830 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a35b170 0x557b0a35c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.739+0000 7f9c72422b80 1 --2- 172.21.15.28:0/668007830 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a358180 0x557b0a34e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.739+0000 7f9c72422b80 1 --2- 172.21.15.28:0/668007830 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a34ea90 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.739+0000 7f9c72422b80 1 -- 172.21.15.28:0/668007830 >> 172.21.15.28:0/668007830 conn(0x557b0a32ddb0 msgr2=0x557b0a32f280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.739+0000 7f9c72422b80 1 -- 172.21.15.28:0/668007830 shutdown_connections 2023-11-28T11:12:17.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.739+0000 7f9c72422b80 1 -- 172.21.15.28:0/668007830 wait complete. 2023-11-28T11:12:17.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.739+0000 7f9c72422b80 1 -- start start 2023-11-28T11:12:17.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.740+0000 7f9c72422b80 1 Processor -- start 2023-11-28T11:12:17.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.740+0000 7f9c72422b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a358180 0x557b0a3de5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.740+0000 7f9c72422b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a35df70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.740+0000 7f9c72422b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a35b170 0x557b0a3d9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.740+0000 7f9c72422b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0a3de1c0 con 0x557b0a35a860 2023-11-28T11:12:17.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.740+0000 7f9c72422b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x557b0a3da3b0 con 0x557b0a35b170 2023-11-28T11:12:17.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.740+0000 7f9c72422b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x557b0a35c880 con 0x557b0a358180 2023-11-28T11:12:17.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.740+0000 7f9c72422b80 10 client.0 _ll_register_callbacks cb 0x557b0a128290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:12:17.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c5f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a35df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c5f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a35df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51932/0 (socket says 172.21.15.28:51932) 2023-11-28T11:12:17.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c5ffff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a358180 0x557b0a3de5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c5f7fe700 1 -- 172.21.15.28:0/224936763 learned_addr learned my addr 172.21.15.28:0/224936763 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:17.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c64836700 1 --2- 172.21.15.28:0/224936763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a35b170 0x557b0a3d9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c5f7fe700 1 -- 172.21.15.28:0/224936763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a358180 msgr2=0x557b0a3de5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c5f7fe700 1 --2- 172.21.15.28:0/224936763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a358180 0x557b0a3de5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c5f7fe700 1 -- 172.21.15.28:0/224936763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a35b170 msgr2=0x557b0a3d9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:17.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c5f7fe700 1 --2- 172.21.15.28:0/224936763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a35b170 0x557b0a3d9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c5f7fe700 1 -- 172.21.15.28:0/224936763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c600cf0d0 con 0x557b0a35a860 2023-11-28T11:12:17.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c5f7fe700 1 --2- 172.21.15.28:0/224936763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a35df70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9c50053e60 tx=0x7f9c500046e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13747b883ae2e491 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:17.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c3effd700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:12:17.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c3effd700 1 -- 172.21.15.28:0/224936763 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5005f760 con 0x557b0a35a860 2023-11-28T11:12:17.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c3effd700 1 -- 172.21.15.28:0/224936763 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(66..66 src has 1..66) v4 ==== 6382+0+0 (secure 0 0 0) 0x7f9c5005f9f0 con 0x557b0a35a860 2023-11-28T11:12:17.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c72422b80 1 -- 172.21.15.28:0/224936763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557b0a2401b0 con 0x557b0a35a860 2023-11-28T11:12:17.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c3effd700 1 -- 172.21.15.28:0/224936763 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9c50003b80 con 0x557b0a35a860 2023-11-28T11:12:17.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c72422b80 10 client.6834 fetch_fsmap learned FSMap version 33 2023-11-28T11:12:17.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c72422b80 1 -- 172.21.15.28:0/224936763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=2+}) v3 -- 0x557b0a0d7250 con 0x557b0a35a860 2023-11-28T11:12:17.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.741+0000 7f9c3effd700 1 -- 172.21.15.28:0/224936763 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557b0a2401b0 con 0x557b0a35a860 2023-11-28T11:12:17.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c3effd700 1 -- 172.21.15.28:0/224936763 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 33) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f9c5006a020 con 0x557b0a35a860 2023-11-28T11:12:17.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c72422b80 10 client.6834 fetch_fsmap finished waiting for FSMap version 33 2023-11-28T11:12:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c72422b80 10 client.6834 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2023-11-28T11:12:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c72422b80 20 client.6834 populate_metadata read hostname 'smithi028' 2023-11-28T11:12:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c72422b80 1 --2- 172.21.15.28:0/224936763 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x557b0a35bb60 0x557b0a35bf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c72422b80 4 client.6834 mds_command: new command op to 6573 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:12:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c72422b80 1 -- 172.21.15.28:0/224936763 --> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x557b0a22a1c0 con 0x557b0a35bb60 2023-11-28T11:12:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c5ffff700 1 --2- 172.21.15.28:0/224936763 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x557b0a35bb60 0x557b0a35bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c3effd700 10 client.6834 ms_handle_connect on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c5ffff700 1 --2- 172.21.15.28:0/224936763 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x557b0a35bb60 0x557b0a35bf60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd791c3b5e77fef4 server_cookie=652220c432a374a3 in_seq=0 out_seq=0 2023-11-28T11:12:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c3effd700 1 -- 172.21.15.28:0/224936763 <== mds.0 v2:172.21.15.59:6832/2406852565 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+253 (crc 0 0 0) 0x557b0a22a1c0 con 0x557b0a35bb60 2023-11-28T11:12:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c3effd700 10 client.6834 handle_command_reply: tid=0 2023-11-28T11:12:17.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c72422b80 1 client.6834 shutdown 2023-11-28T11:12:17.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c72422b80 1 -- 172.21.15.28:0/224936763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 msgr2=0x557b0a35df70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c72422b80 1 --2- 172.21.15.28:0/224936763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a35df70 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f9c50053e60 tx=0x7f9c500046e0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c72422b80 1 -- 172.21.15.28:0/224936763 shutdown_connections 2023-11-28T11:12:17.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c72422b80 1 --2- 172.21.15.28:0/224936763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x557b0a35b170 0x557b0a3d9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c72422b80 1 --2- 172.21.15.28:0/224936763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x557b0a358180 0x557b0a3de5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c72422b80 1 --2- 172.21.15.28:0/224936763 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x557b0a35bb60 0x557b0a35bf60 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.742+0000 7f9c72422b80 1 --2- 172.21.15.28:0/224936763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x557b0a35a860 0x557b0a35df70 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c3effd700 0 client.6834 ms_handle_reset on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:17.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c72422b80 1 -- 172.21.15.28:0/224936763 >> 172.21.15.28:0/224936763 conn(0x557b0a32ddb0 msgr2=0x557b0a34a580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c72422b80 1 -- 172.21.15.28:0/224936763 shutdown_connections 2023-11-28T11:12:17.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c72422b80 1 -- 172.21.15.28:0/224936763 wait complete. 2023-11-28T11:12:17.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c72422b80 20 client.6834 trim_cache size 0 max 16384 2023-11-28T11:12:17.755 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:12:17.755 INFO:teuthology.orchestra.run.smithi028.stdout: "status": "IDLE (waiting for more scrubs)", 2023-11-28T11:12:17.755 INFO:teuthology.orchestra.run.smithi028.stdout: "scrubs": { 2023-11-28T11:12:17.755 INFO:teuthology.orchestra.run.smithi028.stdout: "fa253638-ba69-480f-bef8-47a08e461703": { 2023-11-28T11:12:17.755 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/", 2023-11-28T11:12:17.755 INFO:teuthology.orchestra.run.smithi028.stdout: "tag": "fa253638-ba69-480f-bef8-47a08e461703", 2023-11-28T11:12:17.756 INFO:teuthology.orchestra.run.smithi028.stdout: "options": "recursive" 2023-11-28T11:12:17.756 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:12:17.756 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:12:17.756 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:12:17.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c66299700 1 -- 172.21.15.28:0/3677063209 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9c4c0682f0 msgr2=0x7f9c4c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c66299700 1 --2- 172.21.15.28:0/3677063209 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9c4c0682f0 0x7f9c4c06a7a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f9c54000f60 tx=0x7f9c54007ca0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c66299700 1 -- 172.21.15.28:0/3677063209 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c600cb3e0 msgr2=0x7f9c6015bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:17.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c66299700 1 --2- 172.21.15.28:0/3677063209 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c600cb3e0 0x7f9c6015bf70 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f9c50009b90 tx=0x7f9c50004750 comp rx=0 tx=0).stop 2023-11-28T11:12:17.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c66299700 1 -- 172.21.15.28:0/3677063209 shutdown_connections 2023-11-28T11:12:17.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c66299700 1 --2- 172.21.15.28:0/3677063209 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9c4c0682f0 0x7f9c4c06a7a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c66299700 1 --2- 172.21.15.28:0/3677063209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c600cf5f0 0x7f9c600b4bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c66299700 1 --2- 172.21.15.28:0/3677063209 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9c600b1630 0x7f9c600c8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c66299700 1 --2- 172.21.15.28:0/3677063209 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9c600cb3e0 0x7f9c6015bf70 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:17.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c66299700 1 -- 172.21.15.28:0/3677063209 >> 172.21.15.28:0/3677063209 conn(0x7f9c600cc630 msgr2=0x7f9c600cca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:17.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c66299700 1 -- 172.21.15.28:0/3677063209 shutdown_connections 2023-11-28T11:12:17.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:17.743+0000 7f9c66299700 1 -- 172.21.15.28:0/3677063209 wait complete. 2023-11-28T11:12:17.758 INFO:tasks.cephfs.filesystem:scrub status for tag:fa253638-ba69-480f-bef8-47a08e461703 - {'path': '/', 'tag': 'fa253638-ba69-480f-bef8-47a08e461703', 'options': 'recursive'} 2023-11-28T11:12:20.279 INFO:tasks.ceph.mon.a.smithi028.stderr:2023-11-28T11:12:20.277+0000 7fde5abc8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 MDSs report damaged metadata (MDS_DAMAGE) 2023-11-28T11:12:21.377 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:12:21.380 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:12:47.759 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.4:0 scrub status 2023-11-28T11:12:47.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.899+0000 7f2f532d6700 1 Processor -- start 2023-11-28T11:12:47.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.899+0000 7f2f532d6700 1 -- start start 2023-11-28T11:12:47.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.900+0000 7f2f532d6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f4c0b1630 0x7f2f4c0cf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.900+0000 7f2f532d6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2f4c0b1a00 0x7f2f4c0caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.900+0000 7f2f532d6700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f4c0ce550 con 0x7f2f4c0cb3e0 2023-11-28T11:12:47.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.900+0000 7f2f532d6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f4c0ce830 con 0x7f2f4c0b1a00 2023-11-28T11:12:47.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.900+0000 7f2f532d6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c0ceb10 con 0x7f2f4c0b1630 2023-11-28T11:12:47.904 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.901+0000 7f2f51072700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f4c0b1630 0x7f2f4c0cf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.901+0000 7f2f50871700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f2f4c0b1a00 0x7f2f4c0caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.901+0000 7f2f51873700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f2f4c0cb3e0 0x7f2f4c160350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:50784/0 (socket says 172.21.15.28:50784) 2023-11-28T11:12:47.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.901+0000 7f2f51873700 1 -- 172.21.15.28:0/593387845 learned_addr learned my addr 172.21.15.28:0/593387845 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:47.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.901+0000 7f2f43fff700 1 -- 172.21.15.28:0/593387845 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3091756946 0 0) 0x7f2f4c0ce550 con 0x7f2f4c0cb3e0 2023-11-28T11:12:47.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.901+0000 7f2f43fff700 1 -- 172.21.15.28:0/593387845 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2f380035b0 con 0x7f2f4c0cb3e0 2023-11-28T11:12:47.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.901+0000 7f2f51072700 1 -- 172.21.15.28:0/593387845 >> v2:172.21.15.59:3300/0 conn(0x7f2f4c0b1a00 msgr2=0x7f2f4c0caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.901+0000 7f2f51072700 1 --2- 172.21.15.28:0/593387845 >> v2:172.21.15.59:3300/0 conn(0x7f2f4c0b1a00 0x7f2f4c0caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.901+0000 7f2f51072700 1 -- 172.21.15.28:0/593387845 >> v1:172.21.15.59:6789/0 conn(0x7f2f4c0cb3e0 legacy=0x7f2f4c160350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.901+0000 7f2f51072700 1 -- 172.21.15.28:0/593387845 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f4c0cedf0 con 0x7f2f4c0b1630 2023-11-28T11:12:47.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.901+0000 7f2f50871700 1 --2- 172.21.15.28:0/593387845 >> v2:172.21.15.59:3300/0 conn(0x7f2f4c0b1a00 0x7f2f4c0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:47.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.901+0000 7f2f51072700 1 --2- 172.21.15.28:0/593387845 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f4c0b1630 0x7f2f4c0cf5f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009840 tx=0x7f2f3c009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=936c751b1d7b10 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.902+0000 7f2f43fff700 1 -- 172.21.15.28:0/593387845 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c011830 con 0x7f2f4c0b1630 2023-11-28T11:12:47.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.902+0000 7f2f43fff700 1 -- 172.21.15.28:0/593387845 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c011990 con 0x7f2f4c0b1630 2023-11-28T11:12:47.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.902+0000 7f2f43fff700 1 -- 172.21.15.28:0/593387845 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c011d20 con 0x7f2f4c0b1630 2023-11-28T11:12:47.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.902+0000 7f2f532d6700 1 -- 172.21.15.28:0/593387845 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f4c0b1630 msgr2=0x7f2f4c0cf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.902+0000 7f2f532d6700 1 --2- 172.21.15.28:0/593387845 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f4c0b1630 0x7f2f4c0cf5f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c009840 tx=0x7f2f3c009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:47.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.902+0000 7f2f532d6700 1 -- 172.21.15.28:0/593387845 shutdown_connections 2023-11-28T11:12:47.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.902+0000 7f2f532d6700 1 --2- 172.21.15.28:0/593387845 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f4c0b1630 0x7f2f4c0cf5f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.902+0000 7f2f532d6700 1 --2- 172.21.15.28:0/593387845 >> v2:172.21.15.59:3300/0 conn(0x7f2f4c0b1a00 0x7f2f4c0caea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.906 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.902+0000 7f2f532d6700 1 -- 172.21.15.28:0/593387845 >> 172.21.15.28:0/593387845 conn(0x7f2f4c0cc630 msgr2=0x7f2f4c0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:47.907 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.902+0000 7f2f532d6700 1 -- 172.21.15.28:0/593387845 shutdown_connections 2023-11-28T11:12:47.907 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.902+0000 7f2f532d6700 1 -- 172.21.15.28:0/593387845 wait complete. 2023-11-28T11:12:47.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.903+0000 7f2f532d6700 1 Processor -- start 2023-11-28T11:12:47.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.903+0000 7f2f532d6700 1 -- start start 2023-11-28T11:12:47.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.904+0000 7f2f532d6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f4c0b1630 0x7f2f4c0b52d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.904+0000 7f2f532d6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4c0b1a00 0x7f2f4c0c55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.904+0000 7f2f532d6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f4c0cb3e0 0x7f2f4c0c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.904+0000 7f2f532d6700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c15c610 con 0x7f2f4c0b1630 2023-11-28T11:12:47.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.904+0000 7f2f532d6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c0b33f0 con 0x7f2f4c0b1a00 2023-11-28T11:12:47.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.904+0000 7f2f532d6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f4c0b3f70 con 0x7f2f4c0cb3e0 2023-11-28T11:12:47.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.904+0000 7f2f51072700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f4c0b1630 0x7f2f4c0b52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.904+0000 7f2f51072700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f4c0b1630 0x7f2f4c0b52d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51942/0 (socket says 172.21.15.28:51942) 2023-11-28T11:12:47.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.904+0000 7f2f51072700 1 -- 172.21.15.28:0/2688194735 learned_addr learned my addr 172.21.15.28:0/2688194735 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:47.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.905+0000 7f2f50871700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4c0b1a00 0x7f2f4c0c55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.905+0000 7f2f51873700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f4c0cb3e0 0x7f2f4c0c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.905+0000 7f2f51072700 1 -- 172.21.15.28:0/2688194735 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f4c0cb3e0 msgr2=0x7f2f4c0c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.905+0000 7f2f51072700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f4c0cb3e0 0x7f2f4c0c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.905+0000 7f2f51072700 1 -- 172.21.15.28:0/2688194735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4c0b1a00 msgr2=0x7f2f4c0c55f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.905+0000 7f2f51072700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4c0b1a00 0x7f2f4c0c55f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.905+0000 7f2f51072700 1 -- 172.21.15.28:0/2688194735 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f480094a0 con 0x7f2f4c0b1630 2023-11-28T11:12:47.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.905+0000 7f2f51873700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f4c0cb3e0 0x7f2f4c0c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:47.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.905+0000 7f2f50871700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4c0b1a00 0x7f2f4c0c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:47.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.905+0000 7f2f51072700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f4c0b1630 0x7f2f4c0b52d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0090b0 tx=0x7f2f3c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.905+0000 7f2f41ffb700 1 -- 172.21.15.28:0/2688194735 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c0036a0 con 0x7f2f4c0b1630 2023-11-28T11:12:47.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.905+0000 7f2f532d6700 1 -- 172.21.15.28:0/2688194735 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f3c0094a0 con 0x7f2f4c0b1630 2023-11-28T11:12:47.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.905+0000 7f2f41ffb700 1 -- 172.21.15.28:0/2688194735 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c01f070 con 0x7f2f4c0b1630 2023-11-28T11:12:47.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.905+0000 7f2f41ffb700 1 -- 172.21.15.28:0/2688194735 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c003a00 con 0x7f2f4c0b1630 2023-11-28T11:12:47.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.905+0000 7f2f532d6700 1 -- 172.21.15.28:0/2688194735 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f4c0b1e70 con 0x7f2f4c0b1630 2023-11-28T11:12:47.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.907+0000 7f2f41ffb700 1 -- 172.21.15.28:0/2688194735 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2f3c00f050 con 0x7f2f4c0b1630 2023-11-28T11:12:47.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.907+0000 7f2f41ffb700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2f30068320 0x7f2f3006a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.907+0000 7f2f41ffb700 1 -- 172.21.15.28:0/2688194735 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f2f3c021070 con 0x7f2f4c0b1630 2023-11-28T11:12:47.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.907+0000 7f2f50871700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2f30068320 0x7f2f3006a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.908+0000 7f2f50871700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2f30068320 0x7f2f3006a7d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c1f5a10 tx=0x7f2f48008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.909+0000 7f2f5f45fb80 1 Processor -- start 2023-11-28T11:12:47.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.909+0000 7f2f5f45fb80 1 -- start start 2023-11-28T11:12:47.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f5f45fb80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c57860 0x55a269c2da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f5f45fb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c55180 0x55a269c57c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f5f45fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c58170 0x55a269c737c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f5f45fb80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a269b17c10 con 0x55a269c57860 2023-11-28T11:12:47.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f5f45fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a269b863a0 con 0x55a269c58170 2023-11-28T11:12:47.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f5f45fb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a269b7f8d0 con 0x55a269c55180 2023-11-28T11:12:47.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f51072700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c57860 0x55a269c2da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f51072700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c57860 0x55a269c2da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51950/0 (socket says 172.21.15.28:51950) 2023-11-28T11:12:47.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f51072700 1 -- 172.21.15.28:0/1469533178 learned_addr learned my addr 172.21.15.28:0/1469533178 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:47.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f50871700 1 --2- 172.21.15.28:0/1469533178 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c55180 0x55a269c57c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f51873700 1 --2- 172.21.15.28:0/1469533178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c58170 0x55a269c737c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f51072700 1 -- 172.21.15.28:0/1469533178 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c55180 msgr2=0x55a269c57c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f51072700 1 --2- 172.21.15.28:0/1469533178 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c55180 0x55a269c57c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f51072700 1 -- 172.21.15.28:0/1469533178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c58170 msgr2=0x55a269c737c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f51072700 1 --2- 172.21.15.28:0/1469533178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c58170 0x55a269c737c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f51072700 1 -- 172.21.15.28:0/1469533178 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a269a01630 con 0x55a269c57860 2023-11-28T11:12:47.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.910+0000 7f2f51072700 1 --2- 172.21.15.28:0/1469533178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c57860 0x55a269c2da70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c053f90 tx=0x7f2f3c053f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fbfb34be62b72d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.911+0000 7f2f36ffd700 1 -- 172.21.15.28:0/1469533178 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c055c90 con 0x55a269c57860 2023-11-28T11:12:47.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.911+0000 7f2f36ffd700 1 -- 172.21.15.28:0/1469533178 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f3c056f80 con 0x55a269c57860 2023-11-28T11:12:47.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.911+0000 7f2f36ffd700 1 -- 172.21.15.28:0/1469533178 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c05f690 con 0x55a269c57860 2023-11-28T11:12:47.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.911+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/1469533178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c57860 msgr2=0x55a269c2da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.911+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/1469533178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c57860 0x55a269c2da70 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c053f90 tx=0x7f2f3c053f00 comp rx=0 tx=0).stop 2023-11-28T11:12:47.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.911+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/1469533178 shutdown_connections 2023-11-28T11:12:47.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.911+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/1469533178 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c57860 0x55a269c2da70 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.911+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/1469533178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c58170 0x55a269c737c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.911+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/1469533178 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c55180 0x55a269c57c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.911+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/1469533178 >> 172.21.15.28:0/1469533178 conn(0x55a269c2adb0 msgr2=0x55a269c2b740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:47.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.911+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/1469533178 shutdown_connections 2023-11-28T11:12:47.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.911+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/1469533178 wait complete. 2023-11-28T11:12:47.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.912+0000 7f2f5f45fb80 1 -- start start 2023-11-28T11:12:47.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.912+0000 7f2f5f45fb80 1 Processor -- start 2023-11-28T11:12:47.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f5f45fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c55180 0x55a269c4e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f5f45fb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c57860 0x55a269c56640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f5f45fb80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c58170 0x55a269c56b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f5f45fb80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a269a03920 con 0x55a269c58170 2023-11-28T11:12:47.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f5f45fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a269c795d0 con 0x55a269c55180 2023-11-28T11:12:47.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f5f45fb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a269c6c180 con 0x55a269c57860 2023-11-28T11:12:47.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f5f45fb80 10 client.0 _ll_register_callbacks cb 0x55a269a25290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:12:47.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f51873700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c58170 0x55a269c56b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f51873700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c58170 0x55a269c56b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51960/0 (socket says 172.21.15.28:51960) 2023-11-28T11:12:47.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f51873700 1 -- 172.21.15.28:0/1944222575 learned_addr learned my addr 172.21.15.28:0/1944222575 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:47.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f50871700 1 --2- 172.21.15.28:0/1944222575 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c57860 0x55a269c56640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f51072700 1 --2- 172.21.15.28:0/1944222575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c55180 0x55a269c4e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f51873700 1 -- 172.21.15.28:0/1944222575 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c57860 msgr2=0x55a269c56640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f51873700 1 --2- 172.21.15.28:0/1944222575 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c57860 0x55a269c56640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f51873700 1 -- 172.21.15.28:0/1944222575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c55180 msgr2=0x55a269c4e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f51873700 1 --2- 172.21.15.28:0/1944222575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c55180 0x55a269c4e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.913+0000 7f2f51873700 1 -- 172.21.15.28:0/1944222575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f3c085e70 con 0x55a269c58170 2023-11-28T11:12:47.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f51873700 1 --2- 172.21.15.28:0/1944222575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c58170 0x55a269c56b80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f2f440075b0 tx=0x7f2f440078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3d8b93b83725714 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f2bfff700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:12:47.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f2bfff700 1 -- 172.21.15.28:0/1944222575 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f44010670 con 0x55a269c58170 2023-11-28T11:12:47.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f2bfff700 1 -- 172.21.15.28:0/1944222575 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f2f440107d0 con 0x55a269c58170 2023-11-28T11:12:47.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/1944222575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a269b4d190 con 0x55a269c58170 2023-11-28T11:12:47.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f2bfff700 1 -- 172.21.15.28:0/1944222575 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f44010990 con 0x55a269c58170 2023-11-28T11:12:47.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f5f45fb80 10 client.6843 fetch_fsmap learned FSMap version 34 2023-11-28T11:12:47.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/1944222575 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55a2699adde0 con 0x55a269c58170 2023-11-28T11:12:47.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f2bfff700 1 -- 172.21.15.28:0/1944222575 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a269b4d190 con 0x55a269c58170 2023-11-28T11:12:47.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f2bfff700 1 -- 172.21.15.28:0/1944222575 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f2f44010af0 con 0x55a269c58170 2023-11-28T11:12:47.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f5f45fb80 10 client.6843 fetch_fsmap finished waiting for FSMap version 34 2023-11-28T11:12:47.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f5f45fb80 10 client.6843 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2023-11-28T11:12:47.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f5f45fb80 20 client.6843 populate_metadata read hostname 'smithi028' 2023-11-28T11:12:47.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/1944222575 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55a269c2d9e0 0x55a269c4ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f5f45fb80 4 client.6843 mds_command: new command op to 6573 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:12:47.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.914+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/1944222575 --> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a269a03f60 con 0x55a269c2d9e0 2023-11-28T11:12:47.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.915+0000 7f2f51072700 1 --2- 172.21.15.28:0/1944222575 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55a269c2d9e0 0x55a269c4ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.915+0000 7f2f2bfff700 10 client.6843 ms_handle_connect on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:47.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.915+0000 7f2f51072700 1 --2- 172.21.15.28:0/1944222575 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55a269c2d9e0 0x55a269c4ec70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5fb392ac0df81421 server_cookie=12435e3340ade9c6 in_seq=0 out_seq=0 2023-11-28T11:12:47.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.916+0000 7f2f2bfff700 1 -- 172.21.15.28:0/1944222575 <== mds.0 v2:172.21.15.59:6832/2406852565 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x55a269a03f60 con 0x55a269c2d9e0 2023-11-28T11:12:47.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.916+0000 7f2f2bfff700 10 client.6843 handle_command_reply: tid=0 2023-11-28T11:12:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.916+0000 7f2f5f45fb80 1 client.6843 shutdown 2023-11-28T11:12:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.916+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/1944222575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c58170 msgr2=0x55a269c56b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.916+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/1944222575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c58170 0x55a269c56b80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f2f440075b0 tx=0x7f2f440078c0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.916+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/1944222575 shutdown_connections 2023-11-28T11:12:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.916+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/1944222575 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55a269c2d9e0 0x55a269c4ec70 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.916+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/1944222575 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c58170 0x55a269c56b80 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.916+0000 7f2f2bfff700 0 client.6843 ms_handle_reset on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.916+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/1944222575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c55180 0x55a269c4e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.916+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/1944222575 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c57860 0x55a269c56640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.916+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/1944222575 >> 172.21.15.28:0/1944222575 conn(0x55a269c2adb0 msgr2=0x55a269c26f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:47.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.916+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/1944222575 shutdown_connections 2023-11-28T11:12:47.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.916+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/1944222575 wait complete. 2023-11-28T11:12:47.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.916+0000 7f2f5f45fb80 20 client.6843 trim_cache size 0 max 16384 2023-11-28T11:12:47.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.933+0000 7f2f5f45fb80 1 Processor -- start 2023-11-28T11:12:47.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.933+0000 7f2f5f45fb80 1 -- start start 2023-11-28T11:12:47.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.933+0000 7f2f5f45fb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c55180 0x55a269c84d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.933+0000 7f2f5f45fb80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c57860 0x55a269c2ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.933+0000 7f2f5f45fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c58170 0x55a269c2cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.933+0000 7f2f5f45fb80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a269c5b090 con 0x55a269c57860 2023-11-28T11:12:47.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.933+0000 7f2f5f45fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a269c54a70 con 0x55a269c58170 2023-11-28T11:12:47.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.933+0000 7f2f5f45fb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a269c54850 con 0x55a269c55180 2023-11-28T11:12:47.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.934+0000 7f2f50871700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c57860 0x55a269c2ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.934+0000 7f2f50871700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c57860 0x55a269c2ea50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51966/0 (socket says 172.21.15.28:51966) 2023-11-28T11:12:47.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.934+0000 7f2f50871700 1 -- 172.21.15.28:0/3485919253 learned_addr learned my addr 172.21.15.28:0/3485919253 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:47.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.934+0000 7f2f51072700 1 --2- 172.21.15.28:0/3485919253 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c55180 0x55a269c84d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.934+0000 7f2f51873700 1 --2- 172.21.15.28:0/3485919253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c58170 0x55a269c2cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.934+0000 7f2f50871700 1 -- 172.21.15.28:0/3485919253 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c55180 msgr2=0x55a269c84d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.934+0000 7f2f50871700 1 --2- 172.21.15.28:0/3485919253 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c55180 0x55a269c84d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.934+0000 7f2f50871700 1 -- 172.21.15.28:0/3485919253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c58170 msgr2=0x55a269c2cb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.934+0000 7f2f50871700 1 --2- 172.21.15.28:0/3485919253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c58170 0x55a269c2cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.934+0000 7f2f50871700 1 -- 172.21.15.28:0/3485919253 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a269b46190 con 0x55a269c57860 2023-11-28T11:12:47.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.934+0000 7f2f51072700 1 --2- 172.21.15.28:0/3485919253 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c55180 0x55a269c84d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:47.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.934+0000 7f2f50871700 1 --2- 172.21.15.28:0/3485919253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c57860 0x55a269c2ea50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2f48011e50 tx=0x7f2f48014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a926943b2303c4c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.934+0000 7f2f377fe700 1 -- 172.21.15.28:0/3485919253 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f48015c60 con 0x55a269c57860 2023-11-28T11:12:47.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.934+0000 7f2f377fe700 1 -- 172.21.15.28:0/3485919253 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f48015dc0 con 0x55a269c57860 2023-11-28T11:12:47.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.934+0000 7f2f377fe700 1 -- 172.21.15.28:0/3485919253 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f4802c620 con 0x55a269c57860 2023-11-28T11:12:47.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.935+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/3485919253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c57860 msgr2=0x55a269c2ea50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.935+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/3485919253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c57860 0x55a269c2ea50 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f2f48011e50 tx=0x7f2f48014000 comp rx=0 tx=0).stop 2023-11-28T11:12:47.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.935+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/3485919253 shutdown_connections 2023-11-28T11:12:47.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.935+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/3485919253 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c58170 0x55a269c2cb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.935+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/3485919253 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c55180 0x55a269c84d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.935+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/3485919253 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c57860 0x55a269c2ea50 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.935+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/3485919253 >> 172.21.15.28:0/3485919253 conn(0x55a269c2adb0 msgr2=0x55a269c29150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:47.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.935+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/3485919253 shutdown_connections 2023-11-28T11:12:47.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.935+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/3485919253 wait complete. 2023-11-28T11:12:47.939 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.935+0000 7f2f5f45fb80 1 -- start start 2023-11-28T11:12:47.940 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.935+0000 7f2f5f45fb80 1 Processor -- start 2023-11-28T11:12:47.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.936+0000 7f2f5f45fb80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c55180 0x55a269cd7310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.936+0000 7f2f5f45fb80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c57860 0x55a269c4ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.936+0000 7f2f5f45fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c58170 0x55a269c4f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.936+0000 7f2f5f45fb80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a269c763b0 con 0x55a269c55180 2023-11-28T11:12:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.936+0000 7f2f5f45fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a269c264a0 con 0x55a269c58170 2023-11-28T11:12:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.936+0000 7f2f5f45fb80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a269c7a570 con 0x55a269c57860 2023-11-28T11:12:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.936+0000 7f2f5f45fb80 10 client.0 _ll_register_callbacks cb 0x55a269a25290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:12:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.936+0000 7f2f51072700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c55180 0x55a269cd7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.936+0000 7f2f51072700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c55180 0x55a269cd7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51970/0 (socket says 172.21.15.28:51970) 2023-11-28T11:12:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.936+0000 7f2f51072700 1 -- 172.21.15.28:0/2141633920 learned_addr learned my addr 172.21.15.28:0/2141633920 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.936+0000 7f2f50871700 1 --2- 172.21.15.28:0/2141633920 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c57860 0x55a269c4ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.944 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.936+0000 7f2f51873700 1 --2- 172.21.15.28:0/2141633920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c58170 0x55a269c4f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f51072700 1 -- 172.21.15.28:0/2141633920 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c57860 msgr2=0x55a269c4ec70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f51072700 1 --2- 172.21.15.28:0/2141633920 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c57860 0x55a269c4ec70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f51072700 1 -- 172.21.15.28:0/2141633920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c58170 msgr2=0x55a269c4f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f51072700 1 --2- 172.21.15.28:0/2141633920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c58170 0x55a269c4f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f51072700 1 -- 172.21.15.28:0/2141633920 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f48011ad0 con 0x55a269c55180 2023-11-28T11:12:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f51072700 1 --2- 172.21.15.28:0/2141633920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c55180 0x55a269cd7310 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c05fce0 tx=0x7f2f3c026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3e9965fe98dc3d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f2bfff700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:12:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f2bfff700 1 -- 172.21.15.28:0/2141633920 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f3c063070 con 0x55a269c55180 2023-11-28T11:12:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f2bfff700 1 -- 172.21.15.28:0/2141633920 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f2f3c057190 con 0x55a269c55180 2023-11-28T11:12:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/2141633920 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a269b3d1b0 con 0x55a269c55180 2023-11-28T11:12:47.945 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f2bfff700 1 -- 172.21.15.28:0/2141633920 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2f3c074040 con 0x55a269c55180 2023-11-28T11:12:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f5f45fb80 10 client.6849 fetch_fsmap learned FSMap version 34 2023-11-28T11:12:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/2141633920 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x55a2699d4250 con 0x55a269c55180 2023-11-28T11:12:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f2bfff700 1 -- 172.21.15.28:0/2141633920 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a269b3d1b0 con 0x55a269c55180 2023-11-28T11:12:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f2bfff700 1 -- 172.21.15.28:0/2141633920 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7f2f3c05f690 con 0x55a269c55180 2023-11-28T11:12:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f5f45fb80 10 client.6849 fetch_fsmap finished waiting for FSMap version 34 2023-11-28T11:12:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f5f45fb80 10 client.6849 resolve_mds: resolved 4:0 to role '4:0' aka daemon mds.d 2023-11-28T11:12:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f5f45fb80 20 client.6849 populate_metadata read hostname 'smithi028' 2023-11-28T11:12:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/2141633920 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55a269c2d340 0x55a269c26bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f5f45fb80 4 client.6849 mds_command: new command op to 6573 tid=0[{"prefix": "scrub status"}] 2023-11-28T11:12:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.937+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/2141633920 --> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] -- command(tid 0: {"prefix": "scrub status"}) v1 -- 0x55a269b271c0 con 0x55a269c2d340 2023-11-28T11:12:47.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.938+0000 7f2f50871700 1 --2- 172.21.15.28:0/2141633920 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55a269c2d340 0x55a269c26bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.938+0000 7f2f2bfff700 10 client.6849 ms_handle_connect on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.938+0000 7f2f50871700 1 --2- 172.21.15.28:0/2141633920 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55a269c2d340 0x55a269c26bd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2e4f68c805c8284 server_cookie=503be469d7339d8a in_seq=0 out_seq=0 2023-11-28T11:12:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.938+0000 7f2f2bfff700 1 -- 172.21.15.28:0/2141633920 <== mds.0 v2:172.21.15.59:6832/2406852565 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+63 (crc 0 0 0) 0x55a269b271c0 con 0x55a269c2d340 2023-11-28T11:12:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.938+0000 7f2f2bfff700 10 client.6849 handle_command_reply: tid=0 2023-11-28T11:12:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.938+0000 7f2f5f45fb80 1 client.6849 shutdown 2023-11-28T11:12:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.938+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/2141633920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c55180 msgr2=0x55a269cd7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.938+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/2141633920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c55180 0x55a269cd7310 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c05fce0 tx=0x7f2f3c026000 comp rx=0 tx=0).stop 2023-11-28T11:12:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.938+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/2141633920 shutdown_connections 2023-11-28T11:12:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.938+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/2141633920 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x55a269c2d340 0x55a269c26bd0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.938+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/2141633920 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x55a269c55180 0x55a269cd7310 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.938+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/2141633920 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a269c58170 0x55a269c4f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.938+0000 7f2f2bfff700 0 client.6849 ms_handle_reset on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:47.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.938+0000 7f2f5f45fb80 1 --2- 172.21.15.28:0/2141633920 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x55a269c57860 0x55a269c4ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.938+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/2141633920 >> 172.21.15.28:0/2141633920 conn(0x55a269c2adb0 msgr2=0x55a269c6b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:47.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.939+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/2141633920 shutdown_connections 2023-11-28T11:12:47.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.939+0000 7f2f5f45fb80 1 -- 172.21.15.28:0/2141633920 wait complete. 2023-11-28T11:12:47.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.939+0000 7f2f5f45fb80 20 client.6849 trim_cache size 0 max 16384 2023-11-28T11:12:47.948 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:12:47.948 INFO:teuthology.orchestra.run.smithi028.stdout: "status": "no active scrubs running", 2023-11-28T11:12:47.949 INFO:teuthology.orchestra.run.smithi028.stdout: "scrubs": {} 2023-11-28T11:12:47.949 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:12:47.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.939+0000 7f2f532d6700 1 -- 172.21.15.28:0/2688194735 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2f30068320 msgr2=0x7f2f3006a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.939+0000 7f2f532d6700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2f30068320 0x7f2f3006a7d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2f4c1f5a10 tx=0x7f2f48008040 comp rx=0 tx=0).stop 2023-11-28T11:12:47.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.939+0000 7f2f532d6700 1 -- 172.21.15.28:0/2688194735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f4c0b1630 msgr2=0x7f2f4c0b52d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:47.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.939+0000 7f2f532d6700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f4c0b1630 0x7f2f4c0b52d0 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f2f3c0090b0 tx=0x7f2f3c004750 comp rx=0 tx=0).stop 2023-11-28T11:12:47.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.939+0000 7f2f532d6700 1 -- 172.21.15.28:0/2688194735 shutdown_connections 2023-11-28T11:12:47.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.939+0000 7f2f532d6700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2f30068320 0x7f2f3006a7d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.939+0000 7f2f532d6700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2f4c0b1630 0x7f2f4c0b52d0 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.939+0000 7f2f532d6700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2f4c0cb3e0 0x7f2f4c0c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.939+0000 7f2f532d6700 1 --2- 172.21.15.28:0/2688194735 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4c0b1a00 0x7f2f4c0c55f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:47.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.939+0000 7f2f532d6700 1 -- 172.21.15.28:0/2688194735 >> 172.21.15.28:0/2688194735 conn(0x7f2f4c0cc630 msgr2=0x7f2f4c0c8c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:47.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.939+0000 7f2f532d6700 1 -- 172.21.15.28:0/2688194735 shutdown_connections 2023-11-28T11:12:47.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:47.939+0000 7f2f532d6700 1 -- 172.21.15.28:0/2688194735 wait complete. 2023-11-28T11:12:47.952 INFO:tasks.cephfs.filesystem:all active scrubs completed 2023-11-28T11:12:47.953 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:12:48.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.076+0000 7f7ab285e700 1 Processor -- start 2023-11-28T11:12:48.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.076+0000 7f7ab285e700 1 -- start start 2023-11-28T11:12:48.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.077+0000 7f7ab285e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7aac15a230 0x7f7aac15d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.077+0000 7f7ab285e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7aac0cd4a0 0x7f7aac0b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.077+0000 7f7ab285e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7aac0d1a30 con 0x7f7aac0b4e10 2023-11-28T11:12:48.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.077+0000 7f7ab285e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7aac15a600 con 0x7f7aac15a230 2023-11-28T11:12:48.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.077+0000 7f7ab285e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac0b5290 con 0x7f7aac0cd4a0 2023-11-28T11:12:48.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.077+0000 7f7aab7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7aac0cd4a0 0x7f7aac0b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.077+0000 7f7aabfff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7aac15a230 0x7f7aac15d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.077+0000 7f7ab0dfb700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f7aac0b4e10 0x7f7aac160110 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:50826/0 (socket says 172.21.15.28:50826) 2023-11-28T11:12:48.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.077+0000 7f7ab0dfb700 1 -- 172.21.15.28:0/3931236519 learned_addr learned my addr 172.21.15.28:0/3931236519 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.078+0000 7f7aabfff700 1 -- 172.21.15.28:0/3931236519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7aac0cd4a0 msgr2=0x7f7aac0b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.078+0000 7f7aabfff700 1 --2- 172.21.15.28:0/3931236519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7aac0cd4a0 0x7f7aac0b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.078+0000 7f7aabfff700 1 -- 172.21.15.28:0/3931236519 >> v1:172.21.15.59:6789/0 conn(0x7f7aac0b4e10 legacy=0x7f7aac160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.078+0000 7f7aabfff700 1 -- 172.21.15.28:0/3931236519 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac06abb0 con 0x7f7aac15a230 2023-11-28T11:12:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.078+0000 7f7aaaffd700 1 -- 172.21.15.28:0/3931236519 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4196994859 0 0) 0x7f7aac0d1a30 con 0x7f7aac0b4e10 2023-11-28T11:12:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.078+0000 7f7aab7fe700 1 --2- 172.21.15.28:0/3931236519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7aac0cd4a0 0x7f7aac0b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.078+0000 7f7aabfff700 1 --2- 172.21.15.28:0/3931236519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac15a230 0x7f7aac15d6b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009840 tx=0x7f7a9c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=459e21148aa45b6f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.078+0000 7f7aaaffd700 1 -- 172.21.15.28:0/3931236519 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c011830 con 0x7f7aac15a230 2023-11-28T11:12:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.078+0000 7f7aaaffd700 1 -- 172.21.15.28:0/3931236519 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c011990 con 0x7f7aac15a230 2023-11-28T11:12:48.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.078+0000 7f7aaaffd700 1 -- 172.21.15.28:0/3931236519 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c011d20 con 0x7f7aac15a230 2023-11-28T11:12:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.079+0000 7f7ab285e700 1 -- 172.21.15.28:0/3931236519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac15a230 msgr2=0x7f7aac15d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.079+0000 7f7ab285e700 1 --2- 172.21.15.28:0/3931236519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac15a230 0x7f7aac15d6b0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c009840 tx=0x7f7a9c009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.079+0000 7f7ab285e700 1 -- 172.21.15.28:0/3931236519 shutdown_connections 2023-11-28T11:12:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.079+0000 7f7ab285e700 1 --2- 172.21.15.28:0/3931236519 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac15a230 0x7f7aac15d6b0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.079+0000 7f7ab285e700 1 --2- 172.21.15.28:0/3931236519 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7aac0cd4a0 0x7f7aac0b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.079+0000 7f7ab285e700 1 -- 172.21.15.28:0/3931236519 >> 172.21.15.28:0/3931236519 conn(0x7f7aac0cf8d0 msgr2=0x7f7aac0cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.079+0000 7f7ab285e700 1 -- 172.21.15.28:0/3931236519 shutdown_connections 2023-11-28T11:12:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.079+0000 7f7ab285e700 1 -- 172.21.15.28:0/3931236519 wait complete. 2023-11-28T11:12:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.080+0000 7f7ab285e700 1 Processor -- start 2023-11-28T11:12:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.080+0000 7f7ab285e700 1 -- start start 2023-11-28T11:12:48.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7ab285e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7aac0b4e10 0x7f7aac0cc590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7ab285e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac0cd4a0 0x7f7aac14c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7ab285e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7aac15a230 0x7f7aac0a9720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7ab285e700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac0cb4f0 con 0x7f7aac0b4e10 2023-11-28T11:12:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7ab285e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac0cb7d0 con 0x7f7aac0cd4a0 2023-11-28T11:12:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7ab285e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac0cbab0 con 0x7f7aac15a230 2023-11-28T11:12:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7aabfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7aac0b4e10 0x7f7aac0cc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7aabfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7aac0b4e10 0x7f7aac0cc590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:51984/0 (socket says 172.21.15.28:51984) 2023-11-28T11:12:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7aabfff700 1 -- 172.21.15.28:0/493483219 learned_addr learned my addr 172.21.15.28:0/493483219 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7ab0dfb700 1 --2- 172.21.15.28:0/493483219 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7aac15a230 0x7f7aac0a9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7aab7fe700 1 --2- 172.21.15.28:0/493483219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac0cd4a0 0x7f7aac14c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7aabfff700 1 -- 172.21.15.28:0/493483219 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7aac15a230 msgr2=0x7f7aac0a9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7aabfff700 1 --2- 172.21.15.28:0/493483219 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7aac15a230 0x7f7aac0a9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7aabfff700 1 -- 172.21.15.28:0/493483219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac0cd4a0 msgr2=0x7f7aac14c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7aabfff700 1 --2- 172.21.15.28:0/493483219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac0cd4a0 0x7f7aac14c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.081+0000 7f7aabfff700 1 -- 172.21.15.28:0/493483219 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aa00094a0 con 0x7f7aac0b4e10 2023-11-28T11:12:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.082+0000 7f7aabfff700 1 --2- 172.21.15.28:0/493483219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7aac0b4e10 0x7f7aac0cc590 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c014040 tx=0x7f7a9c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.082+0000 7f7aa8ff9700 1 -- 172.21.15.28:0/493483219 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c0036a0 con 0x7f7aac0b4e10 2023-11-28T11:12:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.082+0000 7f7aa8ff9700 1 -- 172.21.15.28:0/493483219 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a9c01e070 con 0x7f7aac0b4e10 2023-11-28T11:12:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.082+0000 7f7ab285e700 1 -- 172.21.15.28:0/493483219 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9c0094a0 con 0x7f7aac0b4e10 2023-11-28T11:12:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.082+0000 7f7aa8ff9700 1 -- 172.21.15.28:0/493483219 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a9c003a40 con 0x7f7aac0b4e10 2023-11-28T11:12:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.082+0000 7f7ab285e700 1 -- 172.21.15.28:0/493483219 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aac06a560 con 0x7f7aac0b4e10 2023-11-28T11:12:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.083+0000 7f7ab285e700 1 -- 172.21.15.28:0/493483219 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aac06a280 con 0x7f7aac0b4e10 2023-11-28T11:12:48.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.083+0000 7f7aa8ff9700 1 -- 172.21.15.28:0/493483219 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7a9c00f050 con 0x7f7aac0b4e10 2023-11-28T11:12:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.084+0000 7f7aa8ff9700 1 --2- 172.21.15.28:0/493483219 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7a980682f0 0x7f7a9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.084+0000 7f7aab7fe700 1 --2- 172.21.15.28:0/493483219 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7a980682f0 0x7f7a9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.084+0000 7f7aa8ff9700 1 -- 172.21.15.28:0/493483219 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f7a9c086d80 con 0x7f7aac0b4e10 2023-11-28T11:12:48.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.084+0000 7f7aab7fe700 1 --2- 172.21.15.28:0/493483219 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7a980682f0 0x7f7a9806a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7aac15fbc0 tx=0x7f7aa0009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.090 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.088+0000 7f7aa8ff9700 1 -- 172.21.15.28:0/493483219 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7a9c0533e0 con 0x7f7aac0b4e10 2023-11-28T11:12:48.328 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.327+0000 7f7ab285e700 1 -- 172.21.15.28:0/493483219 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7aac06a840 con 0x7f7aac0b4e10 2023-11-28T11:12:48.329 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.327+0000 7f7aa8ff9700 1 -- 172.21.15.28:0/493483219 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f7a9c020090 con 0x7f7aac0b4e10 2023-11-28T11:12:48.329 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:48.329 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1564028052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1564028052},{"type":"v1","addr":"172.21.15.59:6835","nonce":1564028052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6558,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2509041002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2509041002},{"type":"v1","addr":"172.21.15.28:6836","nonce":2509041002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6561,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3994419079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3994419079},{"type":"v1","addr":"172.21.15.28:6837","nonce":3994419079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:12:03.800884+0000","modified":"2023-11-28T11:12:20.278498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.59:6834/2406852565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2406852565},{"type":"v1","addr":"172.21.15.59:6834","nonce":2406852565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:12:48.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.330+0000 7f7ab285e700 1 -- 172.21.15.28:0/493483219 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7a980682f0 msgr2=0x7f7a9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.331 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.330+0000 7f7ab285e700 1 --2- 172.21.15.28:0/493483219 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7a980682f0 0x7f7a9806a7a0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f7aac15fbc0 tx=0x7f7aa0009210 comp rx=0 tx=0).stop 2023-11-28T11:12:48.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.330+0000 7f7ab285e700 1 -- 172.21.15.28:0/493483219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7aac0b4e10 msgr2=0x7f7aac0cc590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.330+0000 7f7ab285e700 1 --2- 172.21.15.28:0/493483219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7aac0b4e10 0x7f7aac0cc590 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c014040 tx=0x7f7a9c004750 comp rx=0 tx=0).stop 2023-11-28T11:12:48.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.330+0000 7f7ab285e700 1 -- 172.21.15.28:0/493483219 shutdown_connections 2023-11-28T11:12:48.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.330+0000 7f7ab285e700 1 --2- 172.21.15.28:0/493483219 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7a980682f0 0x7f7a9806a7a0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.330+0000 7f7ab285e700 1 --2- 172.21.15.28:0/493483219 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7aac0b4e10 0x7f7aac0cc590 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.330+0000 7f7ab285e700 1 --2- 172.21.15.28:0/493483219 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7aac15a230 0x7f7aac0a9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.330+0000 7f7ab285e700 1 --2- 172.21.15.28:0/493483219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac0cd4a0 0x7f7aac14c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.330+0000 7f7ab285e700 1 -- 172.21.15.28:0/493483219 >> 172.21.15.28:0/493483219 conn(0x7f7aac0cf8d0 msgr2=0x7f7aac0d17e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.330+0000 7f7ab285e700 1 -- 172.21.15.28:0/493483219 shutdown_connections 2023-11-28T11:12:48.332 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.330+0000 7f7ab285e700 1 -- 172.21.15.28:0/493483219 wait complete. 2023-11-28T11:12:48.333 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 34 2023-11-28T11:12:48.343 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d damage ls --format=json-pretty 2023-11-28T11:12:48.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.469+0000 7fafc5c5f700 1 Processor -- start 2023-11-28T11:12:48.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.469+0000 7fafc5c5f700 1 -- start start 2023-11-28T11:12:48.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.470+0000 7fafc5c5f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafc015fa10 0x7fafc00cd3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.470+0000 7fafc5c5f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fafc006a1e0 0x7fafc006a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.470+0000 7fafc5c5f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc006bb90 con 0x7fafc006ab90 2023-11-28T11:12:48.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.470+0000 7fafc5c5f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc00cd920 con 0x7fafc006a1e0 2023-11-28T11:12:48.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.470+0000 7fafc5c5f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00cfa50 con 0x7fafc015fa10 2023-11-28T11:12:48.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.470+0000 7fafbeffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fafc006a1e0 0x7fafc006a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.470+0000 7fafbf7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafc015fa10 0x7fafc00cd3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.470+0000 7fafbeffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fafc006a1e0 0x7fafc006a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:39282/0 (socket says 172.21.15.28:39282) 2023-11-28T11:12:48.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.470+0000 7fafbeffd700 1 -- 172.21.15.28:0/4105108489 learned_addr learned my addr 172.21.15.28:0/4105108489 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:48.473 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.470+0000 7fafbf7fe700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafc015fa10 0x7fafc00cd3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:57750/0 (socket says 172.21.15.28:57750) 2023-11-28T11:12:48.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.471+0000 7fafbf7fe700 1 -- 172.21.15.28:0/4105108489 >> v2:172.21.15.59:3300/0 conn(0x7fafc006a1e0 msgr2=0x7fafc006a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:48.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.471+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/4105108489 >> v2:172.21.15.59:3300/0 conn(0x7fafc006a1e0 0x7fafc006a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.471+0000 7fafbf7fe700 1 -- 172.21.15.28:0/4105108489 >> v1:172.21.15.59:6789/0 conn(0x7fafc006ab90 legacy=0x7fafc01600c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.471+0000 7fafbf7fe700 1 -- 172.21.15.28:0/4105108489 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc006b010 con 0x7fafc015fa10 2023-11-28T11:12:48.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.471+0000 7fafbeffd700 1 --2- 172.21.15.28:0/4105108489 >> v2:172.21.15.59:3300/0 conn(0x7fafc006a1e0 0x7fafc006a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:12:48.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.471+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/4105108489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafc015fa10 0x7fafc00cd3e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009840 tx=0x7fafb0009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9529ee53934dea9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.471+0000 7fafbe7fc700 1 -- 172.21.15.28:0/4105108489 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0011830 con 0x7fafc015fa10 2023-11-28T11:12:48.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.471+0000 7fafbe7fc700 1 -- 172.21.15.28:0/4105108489 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb0011990 con 0x7fafc015fa10 2023-11-28T11:12:48.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.471+0000 7fafbe7fc700 1 -- 172.21.15.28:0/4105108489 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0011d20 con 0x7fafc015fa10 2023-11-28T11:12:48.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.472+0000 7fafc5c5f700 1 -- 172.21.15.28:0/4105108489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafc015fa10 msgr2=0x7fafc00cd3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.472+0000 7fafc5c5f700 1 --2- 172.21.15.28:0/4105108489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafc015fa10 0x7fafc00cd3e0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009840 tx=0x7fafb0009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:48.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.472+0000 7fafc5c5f700 1 -- 172.21.15.28:0/4105108489 shutdown_connections 2023-11-28T11:12:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.472+0000 7fafc5c5f700 1 --2- 172.21.15.28:0/4105108489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafc015fa10 0x7fafc00cd3e0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.472+0000 7fafc5c5f700 1 --2- 172.21.15.28:0/4105108489 >> v2:172.21.15.59:3300/0 conn(0x7fafc006a1e0 0x7fafc006a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.472+0000 7fafc5c5f700 1 -- 172.21.15.28:0/4105108489 >> 172.21.15.28:0/4105108489 conn(0x7fafc006ce40 msgr2=0x7fafc00b48c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.472+0000 7fafc5c5f700 1 -- 172.21.15.28:0/4105108489 shutdown_connections 2023-11-28T11:12:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.472+0000 7fafc5c5f700 1 -- 172.21.15.28:0/4105108489 wait complete. 2023-11-28T11:12:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.473+0000 7fafc5c5f700 1 Processor -- start 2023-11-28T11:12:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.473+0000 7fafc5c5f700 1 -- start start 2023-11-28T11:12:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.474+0000 7fafc5c5f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafc006a1e0 0x7fafc01f6730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.474+0000 7fafc5c5f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafc006ab90 0x7fafc014bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.474+0000 7fafc5c5f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fafc015fa10 0x7fafc00c8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.474+0000 7fafc5c5f700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00c8710 con 0x7fafc015fa10 2023-11-28T11:12:48.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.474+0000 7fafc5c5f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc01f6c70 con 0x7fafc006a1e0 2023-11-28T11:12:48.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.474+0000 7fafc5c5f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00c7a90 con 0x7fafc006ab90 2023-11-28T11:12:48.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.474+0000 7fafbffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fafc015fa10 0x7fafc00c8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.474+0000 7fafbffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fafc015fa10 0x7fafc00c8b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52002/0 (socket says 172.21.15.28:52002) 2023-11-28T11:12:48.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.474+0000 7fafbffff700 1 -- 172.21.15.28:0/2453137543 learned_addr learned my addr 172.21.15.28:0/2453137543 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:48.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.474+0000 7fafbeffd700 1 --2- 172.21.15.28:0/2453137543 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafc006ab90 0x7fafc014bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.474+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/2453137543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafc006a1e0 0x7fafc01f6730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.475+0000 7fafbffff700 1 -- 172.21.15.28:0/2453137543 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafc006ab90 msgr2=0x7fafc014bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.475+0000 7fafbffff700 1 --2- 172.21.15.28:0/2453137543 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafc006ab90 0x7fafc014bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.475+0000 7fafbffff700 1 -- 172.21.15.28:0/2453137543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafc006a1e0 msgr2=0x7fafc01f6730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.475+0000 7fafbffff700 1 --2- 172.21.15.28:0/2453137543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafc006a1e0 0x7fafc01f6730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.475+0000 7fafbffff700 1 -- 172.21.15.28:0/2453137543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafa8009550 con 0x7fafc015fa10 2023-11-28T11:12:48.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.475+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/2453137543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafc006a1e0 0x7fafc01f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:48.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.475+0000 7fafbffff700 1 --2- 172.21.15.28:0/2453137543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fafc015fa10 0x7fafc00c8b30 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fafb800cc50 tx=0x7fafb800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.475+0000 7fafb7fff700 1 -- 172.21.15.28:0/2453137543 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb800ad80 con 0x7fafc015fa10 2023-11-28T11:12:48.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.475+0000 7fafb7fff700 1 -- 172.21.15.28:0/2453137543 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb800ce50 con 0x7fafc015fa10 2023-11-28T11:12:48.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.475+0000 7fafc5c5f700 1 -- 172.21.15.28:0/2453137543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafb00094a0 con 0x7fafc015fa10 2023-11-28T11:12:48.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.475+0000 7fafb7fff700 1 -- 172.21.15.28:0/2453137543 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8003c40 con 0x7fafc015fa10 2023-11-28T11:12:48.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.475+0000 7fafc5c5f700 1 -- 172.21.15.28:0/2453137543 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc00cfca0 con 0x7fafc015fa10 2023-11-28T11:12:48.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.476+0000 7fafb7fff700 1 -- 172.21.15.28:0/2453137543 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fafb8010050 con 0x7fafc015fa10 2023-11-28T11:12:48.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.477+0000 7fafb7fff700 1 --2- 172.21.15.28:0/2453137543 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fafa00682f0 0x7fafa006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.477+0000 7fafb7fff700 1 -- 172.21.15.28:0/2453137543 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fafb8085e60 con 0x7fafc015fa10 2023-11-28T11:12:48.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.477+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/2453137543 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fafa00682f0 0x7fafa006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.477+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/2453137543 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fafa00682f0 0x7fafa006a7a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009b90 tx=0x7fafb0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.479+0000 7fafd1de8b80 1 Processor -- start 2023-11-28T11:12:48.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.479+0000 7fafd1de8b80 1 -- start start 2023-11-28T11:12:48.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafd1de8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 0x561fe34d8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafd1de8b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35001d0 0x561fe3502c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafd1de8b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35031c0 0x561fe351e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafd1de8b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe33c2c60 con 0x561fe35031c0 2023-11-28T11:12:48.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafd1de8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe34313f0 con 0x561fe35028b0 2023-11-28T11:12:48.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafd1de8b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fe342a920 con 0x561fe35001d0 2023-11-28T11:12:48.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafbffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35031c0 0x561fe351e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafbffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35031c0 0x561fe351e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52010/0 (socket says 172.21.15.28:52010) 2023-11-28T11:12:48.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafbffff700 1 -- 172.21.15.28:0/1431358097 learned_addr learned my addr 172.21.15.28:0/1431358097 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:48.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafbeffd700 1 --2- 172.21.15.28:0/1431358097 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35001d0 0x561fe3502c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/1431358097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 0x561fe34d8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafbffff700 1 -- 172.21.15.28:0/1431358097 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35001d0 msgr2=0x561fe3502c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:48.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafbffff700 1 --2- 172.21.15.28:0/1431358097 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35001d0 0x561fe3502c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafbffff700 1 -- 172.21.15.28:0/1431358097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 msgr2=0x561fe34d8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:48.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafbffff700 1 --2- 172.21.15.28:0/1431358097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 0x561fe34d8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.480+0000 7fafbffff700 1 -- 172.21.15.28:0/1431358097 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fe32ac680 con 0x561fe35031c0 2023-11-28T11:12:48.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.481+0000 7fafbffff700 1 --2- 172.21.15.28:0/1431358097 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35031c0 0x561fe351e810 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fafb805dc70 tx=0x7fafb8055a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf4423d468422d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.481+0000 7fafb57fa700 1 -- 172.21.15.28:0/1431358097 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb806d410 con 0x561fe35031c0 2023-11-28T11:12:48.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.481+0000 7fafb57fa700 1 -- 172.21.15.28:0/1431358097 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb8054310 con 0x561fe35031c0 2023-11-28T11:12:48.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.481+0000 7fafb57fa700 1 -- 172.21.15.28:0/1431358097 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8055800 con 0x561fe35031c0 2023-11-28T11:12:48.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.481+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1431358097 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35031c0 msgr2=0x561fe351e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.481+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/1431358097 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35031c0 0x561fe351e810 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7fafb805dc70 tx=0x7fafb8055a70 comp rx=0 tx=0).stop 2023-11-28T11:12:48.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.481+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1431358097 shutdown_connections 2023-11-28T11:12:48.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.481+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/1431358097 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35031c0 0x561fe351e810 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.481+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/1431358097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 0x561fe34d8ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.482+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/1431358097 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35001d0 0x561fe3502c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.482+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1431358097 >> 172.21.15.28:0/1431358097 conn(0x561fe34d5e00 msgr2=0x561fe34d6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.482+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1431358097 shutdown_connections 2023-11-28T11:12:48.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.482+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1431358097 wait complete. 2023-11-28T11:12:48.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.482+0000 7fafd1de8b80 1 -- start start 2023-11-28T11:12:48.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.482+0000 7fafd1de8b80 1 Processor -- start 2023-11-28T11:12:48.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafd1de8b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35001d0 0x561fe35202a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafd1de8b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35028b0 0x561fe35235d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafd1de8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35031c0 0x561fe3523b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafd1de8b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe32ae970 con 0x561fe35001d0 2023-11-28T11:12:48.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafd1de8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe34f9ce0 con 0x561fe35031c0 2023-11-28T11:12:48.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafd1de8b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fe35257f0 con 0x561fe35028b0 2023-11-28T11:12:48.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafd1de8b80 10 client.0 _ll_register_callbacks cb 0x561fe32d02e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:12:48.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafbf7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35001d0 0x561fe35202a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafbf7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35001d0 0x561fe35202a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52012/0 (socket says 172.21.15.28:52012) 2023-11-28T11:12:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafbf7fe700 1 -- 172.21.15.28:0/1655532447 learned_addr learned my addr 172.21.15.28:0/1655532447 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafbeffd700 1 --2- 172.21.15.28:0/1655532447 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35028b0 0x561fe35235d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafbffff700 1 --2- 172.21.15.28:0/1655532447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35031c0 0x561fe3523b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafbf7fe700 1 -- 172.21.15.28:0/1655532447 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35028b0 msgr2=0x561fe35235d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/1655532447 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35028b0 0x561fe35235d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafbf7fe700 1 -- 172.21.15.28:0/1655532447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35031c0 msgr2=0x561fe3523b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/1655532447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35031c0 0x561fe3523b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafbf7fe700 1 -- 172.21.15.28:0/1655532447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc00cfca0 con 0x561fe35001d0 2023-11-28T11:12:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.483+0000 7fafbffff700 1 --2- 172.21.15.28:0/1655532447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35031c0 0x561fe3523b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:48.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/1655532447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35001d0 0x561fe35202a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fafc00cda60 tx=0x7fafb00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b96df5a0e02970 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7faf9e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:12:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7faf9e7fc700 1 -- 172.21.15.28:0/1655532447 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0032470 con 0x561fe35001d0 2023-11-28T11:12:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7faf9e7fc700 1 -- 172.21.15.28:0/1655532447 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fafb00325d0 con 0x561fe35001d0 2023-11-28T11:12:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1655532447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fe33f81e0 con 0x561fe35001d0 2023-11-28T11:12:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7faf9e7fc700 1 -- 172.21.15.28:0/1655532447 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafb0032790 con 0x561fe35001d0 2023-11-28T11:12:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7fafd1de8b80 10 client.6861 fetch_fsmap learned FSMap version 34 2023-11-28T11:12:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1655532447 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x561fe3258e30 con 0x561fe35001d0 2023-11-28T11:12:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7faf9e7fc700 1 -- 172.21.15.28:0/1655532447 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561fe33f81e0 con 0x561fe35001d0 2023-11-28T11:12:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7faf9e7fc700 1 -- 172.21.15.28:0/1655532447 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fafb00328f0 con 0x561fe35001d0 2023-11-28T11:12:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7fafd1de8b80 10 client.6861 fetch_fsmap finished waiting for FSMap version 34 2023-11-28T11:12:48.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7fafd1de8b80 10 client.6861 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:12:48.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7fafd1de8b80 20 client.6861 populate_metadata read hostname 'smithi028' 2023-11-28T11:12:48.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/1655532447 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x561fe34fade0 0x561fe3522970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7fafd1de8b80 4 client.6861 mds_command: new command op to 6573 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T11:12:48.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1655532447 --> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561fe32aefb0 con 0x561fe34fade0 2023-11-28T11:12:48.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.484+0000 7fafbeffd700 1 --2- 172.21.15.28:0/1655532447 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x561fe34fade0 0x561fe3522970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.485+0000 7fafbeffd700 1 --2- 172.21.15.28:0/1655532447 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x561fe34fade0 0x561fe3522970 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75d6c2d475c8d00b server_cookie=46113cd4131d5fc0 in_seq=0 out_seq=0 2023-11-28T11:12:48.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.485+0000 7faf9e7fc700 10 client.6861 ms_handle_connect on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:48.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.486+0000 7faf9e7fc700 1 -- 172.21.15.28:0/1655532447 <== mds.0 v2:172.21.15.59:6832/2406852565 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17888 (crc 0 0 0) 0x561fe32aefb0 con 0x561fe34fade0 2023-11-28T11:12:48.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.486+0000 7faf9e7fc700 10 client.6861 handle_command_reply: tid=0 2023-11-28T11:12:48.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.486+0000 7fafd1de8b80 1 client.6861 shutdown 2023-11-28T11:12:48.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.486+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1655532447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35001d0 msgr2=0x561fe35202a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.486+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/1655532447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35001d0 0x561fe35202a0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7fafc00cda60 tx=0x7fafb00047c0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.486+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1655532447 shutdown_connections 2023-11-28T11:12:48.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.486+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/1655532447 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x561fe34fade0 0x561fe3522970 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.486+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/1655532447 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35001d0 0x561fe35202a0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.486+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/1655532447 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35031c0 0x561fe3523b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.486+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/1655532447 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35028b0 0x561fe35235d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.486+0000 7faf9e7fc700 0 client.6861 ms_handle_reset on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:48.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.486+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1655532447 >> 172.21.15.28:0/1655532447 conn(0x561fe34d5e00 msgr2=0x561fe34d6790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.486+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1655532447 shutdown_connections 2023-11-28T11:12:48.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.486+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1655532447 wait complete. 2023-11-28T11:12:48.497 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.486+0000 7fafd1de8b80 20 client.6861 trim_cache size 0 max 16384 2023-11-28T11:12:48.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.499+0000 7fafd1de8b80 1 Processor -- start 2023-11-28T11:12:48.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.499+0000 7fafd1de8b80 1 -- start start 2023-11-28T11:12:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafd1de8b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35001d0 0x561fe34fa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafd1de8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 0x561fe351ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafd1de8b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35031c0 0x561fe357cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafd1de8b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe3502580 con 0x561fe35001d0 2023-11-28T11:12:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafd1de8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe34f8d90 con 0x561fe35028b0 2023-11-28T11:12:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafd1de8b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fe34d8b90 con 0x561fe35031c0 2023-11-28T11:12:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafbf7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35001d0 0x561fe34fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafbf7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35001d0 0x561fe34fa830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52020/0 (socket says 172.21.15.28:52020) 2023-11-28T11:12:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafbf7fe700 1 -- 172.21.15.28:0/1568800189 learned_addr learned my addr 172.21.15.28:0/1568800189 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafbffff700 1 --2- 172.21.15.28:0/1568800189 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35031c0 0x561fe357cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafbeffd700 1 --2- 172.21.15.28:0/1568800189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 0x561fe351ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafbf7fe700 1 -- 172.21.15.28:0/1568800189 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35031c0 msgr2=0x561fe357cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/1568800189 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35031c0 0x561fe357cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafbf7fe700 1 -- 172.21.15.28:0/1568800189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 msgr2=0x561fe351ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/1568800189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 0x561fe351ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafbf7fe700 1 -- 172.21.15.28:0/1568800189 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561fe33f11e0 con 0x561fe35001d0 2023-11-28T11:12:48.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/1568800189 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35001d0 0x561fe34fa830 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fafb0032ee0 tx=0x7fafb002b070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52ee46cbe71c36a2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafb5ffb700 1 -- 172.21.15.28:0/1568800189 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0003980 con 0x561fe35001d0 2023-11-28T11:12:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.500+0000 7fafb5ffb700 1 -- 172.21.15.28:0/1568800189 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb0032470 con 0x561fe35001d0 2023-11-28T11:12:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.501+0000 7fafb5ffb700 1 -- 172.21.15.28:0/1568800189 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb00327e0 con 0x561fe35001d0 2023-11-28T11:12:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.501+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1568800189 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35001d0 msgr2=0x561fe34fa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.501+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/1568800189 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35001d0 0x561fe34fa830 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7fafb0032ee0 tx=0x7fafb002b070 comp rx=0 tx=0).stop 2023-11-28T11:12:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.501+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1568800189 shutdown_connections 2023-11-28T11:12:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.501+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/1568800189 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35001d0 0x561fe34fa830 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.501+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/1568800189 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35031c0 0x561fe357cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.501+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/1568800189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 0x561fe351ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.501+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1568800189 >> 172.21.15.28:0/1568800189 conn(0x561fe34d5e00 msgr2=0x561fe34d9aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.501+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1568800189 shutdown_connections 2023-11-28T11:12:48.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.501+0000 7fafd1de8b80 1 -- 172.21.15.28:0/1568800189 wait complete. 2023-11-28T11:12:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.502+0000 7fafd1de8b80 1 -- start start 2023-11-28T11:12:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.502+0000 7fafd1de8b80 1 Processor -- start 2023-11-28T11:12:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafd1de8b80 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35001d0 0x561fe34d67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafd1de8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 0x561fe3513050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafd1de8b80 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35031c0 0x561fe3513590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafd1de8b80 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe34d2310 con 0x561fe35031c0 2023-11-28T11:12:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafd1de8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x561fe357a4c0 con 0x561fe35028b0 2023-11-28T11:12:48.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafd1de8b80 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x561fe34d1710 con 0x561fe35001d0 2023-11-28T11:12:48.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafd1de8b80 10 client.0 _ll_register_callbacks cb 0x561fe32d02e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T11:12:48.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafbffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35031c0 0x561fe3513590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafbffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35031c0 0x561fe3513590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52030/0 (socket says 172.21.15.28:52030) 2023-11-28T11:12:48.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafbffff700 1 -- 172.21.15.28:0/4634390 learned_addr learned my addr 172.21.15.28:0/4634390 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:48.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/4634390 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35001d0 0x561fe34d67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafbeffd700 1 --2- 172.21.15.28:0/4634390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 0x561fe3513050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafbffff700 1 -- 172.21.15.28:0/4634390 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35001d0 msgr2=0x561fe34d67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:48.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafbffff700 1 --2- 172.21.15.28:0/4634390 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35001d0 0x561fe34d67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafbffff700 1 -- 172.21.15.28:0/4634390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 msgr2=0x561fe3513050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:48.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafbffff700 1 --2- 172.21.15.28:0/4634390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 0x561fe3513050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafbffff700 1 -- 172.21.15.28:0/4634390 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561fe3258e30 con 0x561fe35031c0 2023-11-28T11:12:48.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafbeffd700 1 --2- 172.21.15.28:0/4634390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 0x561fe3513050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:48.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafbffff700 1 --2- 172.21.15.28:0/4634390 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35031c0 0x561fe3513590 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fafb8084330 tx=0x7fafb801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee9919c9507e028 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7faf9e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.28:3300/0 2023-11-28T11:12:48.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7faf9e7fc700 1 -- 172.21.15.28:0/4634390 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb805dd10 con 0x561fe35031c0 2023-11-28T11:12:48.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7faf9e7fc700 1 -- 172.21.15.28:0/4634390 <== mon.0 v2:172.21.15.28:3300/0 2 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fafb8058e30 con 0x561fe35031c0 2023-11-28T11:12:48.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafd1de8b80 1 -- 172.21.15.28:0/4634390 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561fe33e8200 con 0x561fe35031c0 2023-11-28T11:12:48.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7faf9e7fc700 1 -- 172.21.15.28:0/4634390 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7fafb805de70 con 0x561fe35031c0 2023-11-28T11:12:48.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafd1de8b80 10 client.6867 fetch_fsmap learned FSMap version 34 2023-11-28T11:12:48.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.503+0000 7fafd1de8b80 1 -- 172.21.15.28:0/4634390 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x561fe327f2a0 con 0x561fe35031c0 2023-11-28T11:12:48.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.504+0000 7faf9e7fc700 1 -- 172.21.15.28:0/4634390 <== mon.0 v2:172.21.15.28:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x561fe33e8200 con 0x561fe35031c0 2023-11-28T11:12:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.504+0000 7faf9e7fc700 1 -- 172.21.15.28:0/4634390 <== mon.0 v2:172.21.15.28:3300/0 5 ==== fsmap(e 34) v1 ==== 3106+0+0 (secure 0 0 0) 0x7fafb8051f00 con 0x561fe35031c0 2023-11-28T11:12:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.504+0000 7fafd1de8b80 10 client.6867 fetch_fsmap finished waiting for FSMap version 34 2023-11-28T11:12:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.504+0000 7fafd1de8b80 10 client.6867 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T11:12:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.504+0000 7fafd1de8b80 20 client.6867 populate_metadata read hostname 'smithi028' 2023-11-28T11:12:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.504+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/4634390 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x561fe35228a0 0x561fe3522c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.504+0000 7fafd1de8b80 4 client.6867 mds_command: new command op to 6573 tid=0[{"prefix": "damage ls", "format": "json-pretty"}] 2023-11-28T11:12:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.504+0000 7fafd1de8b80 1 -- 172.21.15.28:0/4634390 --> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] -- command(tid 0: {"prefix": "damage ls", "format": "json-pretty"}) v1 -- 0x561fe33d2210 con 0x561fe35228a0 2023-11-28T11:12:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.504+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/4634390 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x561fe35228a0 0x561fe3522c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.504+0000 7faf9e7fc700 10 client.6867 ms_handle_connect on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.504+0000 7fafbf7fe700 1 --2- 172.21.15.28:0/4634390 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x561fe35228a0 0x561fe3522c70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b56f4d5266c058a5 server_cookie=ab67d244ac9e924f in_seq=0 out_seq=0 2023-11-28T11:12:48.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7faf9e7fc700 1 -- 172.21.15.28:0/4634390 <== mds.0 v2:172.21.15.59:6832/2406852565 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+141 (crc 0 0 0) 0x561fe33d2210 con 0x561fe35228a0 2023-11-28T11:12:48.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7faf9e7fc700 10 client.6867 handle_command_reply: tid=0 2023-11-28T11:12:48.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafd1de8b80 1 client.6867 shutdown 2023-11-28T11:12:48.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafd1de8b80 1 -- 172.21.15.28:0/4634390 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35031c0 msgr2=0x561fe3513590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/4634390 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35031c0 0x561fe3513590 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7fafb8084330 tx=0x7fafb801d000 comp rx=0 tx=0).stop 2023-11-28T11:12:48.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafd1de8b80 1 -- 172.21.15.28:0/4634390 shutdown_connections 2023-11-28T11:12:48.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/4634390 >> [v2:172.21.15.59:6832/2406852565,v1:172.21.15.59:6834/2406852565] conn(0x561fe35228a0 0x561fe3522c70 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/4634390 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x561fe35031c0 0x561fe3513590 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/4634390 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x561fe35001d0 0x561fe34d67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafd1de8b80 1 --2- 172.21.15.28:0/4634390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561fe35028b0 0x561fe3513050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7faf9e7fc700 0 client.6867 ms_handle_reset on v2:172.21.15.59:6832/2406852565 2023-11-28T11:12:48.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafd1de8b80 1 -- 172.21.15.28:0/4634390 >> 172.21.15.28:0/4634390 conn(0x561fe34d5e00 msgr2=0x561fe34fa5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafd1de8b80 1 -- 172.21.15.28:0/4634390 shutdown_connections 2023-11-28T11:12:48.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafd1de8b80 1 -- 172.21.15.28:0/4634390 wait complete. 2023-11-28T11:12:48.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafd1de8b80 20 client.6867 trim_cache size 0 max 16384 2023-11-28T11:12:48.515 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:48.515 INFO:teuthology.orchestra.run.smithi028.stdout:[ 2023-11-28T11:12:48.515 INFO:teuthology.orchestra.run.smithi028.stdout: { 2023-11-28T11:12:48.515 INFO:teuthology.orchestra.run.smithi028.stdout: "damage_type": "backtrace", 2023-11-28T11:12:48.515 INFO:teuthology.orchestra.run.smithi028.stdout: "id": 2846259220, 2023-11-28T11:12:48.516 INFO:teuthology.orchestra.run.smithi028.stdout: "ino": 1099511627777, 2023-11-28T11:12:48.516 INFO:teuthology.orchestra.run.smithi028.stdout: "path": "/dir_test/file" 2023-11-28T11:12:48.516 INFO:teuthology.orchestra.run.smithi028.stdout: } 2023-11-28T11:12:48.516 INFO:teuthology.orchestra.run.smithi028.stdout:] 2023-11-28T11:12:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafc5c5f700 1 -- 172.21.15.28:0/2453137543 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fafa00682f0 msgr2=0x7fafa006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafc5c5f700 1 --2- 172.21.15.28:0/2453137543 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fafa00682f0 0x7fafa006a7a0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fafb0009b90 tx=0x7fafb0008030 comp rx=0 tx=0).stop 2023-11-28T11:12:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafc5c5f700 1 -- 172.21.15.28:0/2453137543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fafc015fa10 msgr2=0x7fafc00c8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafc5c5f700 1 --2- 172.21.15.28:0/2453137543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fafc015fa10 0x7fafc00c8b30 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fafb800cc50 tx=0x7fafb800a340 comp rx=0 tx=0).stop 2023-11-28T11:12:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafc5c5f700 1 -- 172.21.15.28:0/2453137543 shutdown_connections 2023-11-28T11:12:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafc5c5f700 1 --2- 172.21.15.28:0/2453137543 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fafa00682f0 0x7fafa006a7a0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafc5c5f700 1 --2- 172.21.15.28:0/2453137543 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fafc015fa10 0x7fafc00c8b30 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafc5c5f700 1 --2- 172.21.15.28:0/2453137543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fafc006a1e0 0x7fafc01f6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.505+0000 7fafc5c5f700 1 --2- 172.21.15.28:0/2453137543 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fafc006ab90 0x7fafc014bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.506+0000 7fafc5c5f700 1 -- 172.21.15.28:0/2453137543 >> 172.21.15.28:0/2453137543 conn(0x7fafc006ce40 msgr2=0x7fafc00ca220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.506+0000 7fafc5c5f700 1 -- 172.21.15.28:0/2453137543 shutdown_connections 2023-11-28T11:12:48.518 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.506+0000 7fafc5c5f700 1 -- 172.21.15.28:0/2453137543 wait complete. 2023-11-28T11:12:48.519 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:12:48.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.642+0000 7fb78621f700 1 Processor -- start 2023-11-28T11:12:48.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.642+0000 7fb78621f700 1 -- start start 2023-11-28T11:12:48.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.643+0000 7fb78621f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb7800aefe0 0x7fb7800b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.643+0000 7fb78621f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb7800cc100 0x7fb7800cb3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.643+0000 7fb78621f700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7800b5880 con 0x7fb7800b4a70 2023-11-28T11:12:48.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.643+0000 7fb78621f700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7800d01e0 con 0x7fb7800cc100 2023-11-28T11:12:48.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.643+0000 7fb78621f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7800d04c0 con 0x7fb7800aefe0 2023-11-28T11:12:48.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.643+0000 7fb77ffff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb7800cc100 0x7fb7800cb3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.643+0000 7fb77f7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb7800b4a70 0x7fb7800cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40092/0 (socket says 172.21.15.28:40092) 2023-11-28T11:12:48.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.643+0000 7fb77f7fe700 1 -- 172.21.15.28:0/2327470876 learned_addr learned my addr 172.21.15.28:0/2327470876 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:48.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.643+0000 7fb77effd700 1 --2- 172.21.15.28:0/2327470876 >> v2:172.21.15.59:3300/0 conn(0x7fb7800aefe0 0x7fb7800b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.643+0000 7fb77ffff700 1 -- 172.21.15.28:0/2327470876 >> v2:172.21.15.59:3300/0 conn(0x7fb7800aefe0 msgr2=0x7fb7800b4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:48.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.643+0000 7fb77ffff700 1 --2- 172.21.15.28:0/2327470876 >> v2:172.21.15.59:3300/0 conn(0x7fb7800aefe0 0x7fb7800b4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.643+0000 7fb77ffff700 1 -- 172.21.15.28:0/2327470876 >> v1:172.21.15.28:6789/0 conn(0x7fb7800b4a70 legacy=0x7fb7800cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.643+0000 7fb77ffff700 1 -- 172.21.15.28:0/2327470876 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7800d07a0 con 0x7fb7800cc100 2023-11-28T11:12:48.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.643+0000 7fb77ffff700 1 --2- 172.21.15.28:0/2327470876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7800cc100 0x7fb7800cb3c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb780070ac0 tx=0x7fb77800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec5a292bf5252f4b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.644+0000 7fb77e7fc700 1 -- 172.21.15.28:0/2327470876 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb77800bdd0 con 0x7fb7800cc100 2023-11-28T11:12:48.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.644+0000 7fb77e7fc700 1 -- 172.21.15.28:0/2327470876 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb778011460 con 0x7fb7800cc100 2023-11-28T11:12:48.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.644+0000 7fb77e7fc700 1 -- 172.21.15.28:0/2327470876 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb778011810 con 0x7fb7800cc100 2023-11-28T11:12:48.653 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.644+0000 7fb78621f700 1 -- 172.21.15.28:0/2327470876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7800cc100 msgr2=0x7fb7800cb3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.644+0000 7fb78621f700 1 --2- 172.21.15.28:0/2327470876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7800cc100 0x7fb7800cb3c0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7fb780070ac0 tx=0x7fb77800b330 comp rx=0 tx=0).stop 2023-11-28T11:12:48.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.644+0000 7fb78621f700 1 -- 172.21.15.28:0/2327470876 shutdown_connections 2023-11-28T11:12:48.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.644+0000 7fb78621f700 1 --2- 172.21.15.28:0/2327470876 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7800cc100 0x7fb7800cb3c0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.644+0000 7fb78621f700 1 --2- 172.21.15.28:0/2327470876 >> v2:172.21.15.59:3300/0 conn(0x7fb7800aefe0 0x7fb7800b4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.644+0000 7fb78621f700 1 -- 172.21.15.28:0/2327470876 >> 172.21.15.28:0/2327470876 conn(0x7fb7800b3610 msgr2=0x7fb7800cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.644+0000 7fb78621f700 1 -- 172.21.15.28:0/2327470876 shutdown_connections 2023-11-28T11:12:48.654 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.644+0000 7fb78621f700 1 -- 172.21.15.28:0/2327470876 wait complete. 2023-11-28T11:12:48.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.645+0000 7fb78621f700 1 Processor -- start 2023-11-28T11:12:48.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.646+0000 7fb78621f700 1 -- start start 2023-11-28T11:12:48.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.646+0000 7fb78621f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb7800aefe0 0x7fb7801e06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.646+0000 7fb78621f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7800b4a70 0x7fb7801e2d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.646+0000 7fb78621f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7800cc100 0x7fb7800c5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.646+0000 7fb78621f700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7801e1a60 con 0x7fb7800b4a70 2023-11-28T11:12:48.655 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.646+0000 7fb78621f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7801e28b0 con 0x7fb7800cc100 2023-11-28T11:12:48.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.646+0000 7fb78621f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7801e1010 con 0x7fb7800aefe0 2023-11-28T11:12:48.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.646+0000 7fb77effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7800b4a70 0x7fb7801e2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.646+0000 7fb77effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7800b4a70 0x7fb7801e2d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52042/0 (socket says 172.21.15.28:52042) 2023-11-28T11:12:48.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.646+0000 7fb77effd700 1 -- 172.21.15.28:0/1344119244 learned_addr learned my addr 172.21.15.28:0/1344119244 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:48.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.647+0000 7fb77f7fe700 1 --2- 172.21.15.28:0/1344119244 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb7800aefe0 0x7fb7801e06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.647+0000 7fb77ffff700 1 --2- 172.21.15.28:0/1344119244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7800cc100 0x7fb7800c5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.656 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.647+0000 7fb77effd700 1 -- 172.21.15.28:0/1344119244 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb7800aefe0 msgr2=0x7fb7801e06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.647+0000 7fb77effd700 1 --2- 172.21.15.28:0/1344119244 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb7800aefe0 0x7fb7801e06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.647+0000 7fb77effd700 1 -- 172.21.15.28:0/1344119244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7800cc100 msgr2=0x7fb7800c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.647+0000 7fb77effd700 1 --2- 172.21.15.28:0/1344119244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7800cc100 0x7fb7800c5c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.647+0000 7fb77effd700 1 -- 172.21.15.28:0/1344119244 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb778009a70 con 0x7fb7800b4a70 2023-11-28T11:12:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.647+0000 7fb77effd700 1 --2- 172.21.15.28:0/1344119244 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7800b4a70 0x7fb7801e2d40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb76800d800 tx=0x7fb76800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.647+0000 7fb777fff700 1 -- 172.21.15.28:0/1344119244 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7680125f0 con 0x7fb7800b4a70 2023-11-28T11:12:48.657 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.647+0000 7fb777fff700 1 -- 172.21.15.28:0/1344119244 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb768012750 con 0x7fb7800b4a70 2023-11-28T11:12:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.647+0000 7fb777fff700 1 -- 172.21.15.28:0/1344119244 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb768012b00 con 0x7fb7800b4a70 2023-11-28T11:12:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.647+0000 7fb78621f700 1 -- 172.21.15.28:0/1344119244 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7800aead0 con 0x7fb7800b4a70 2023-11-28T11:12:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.647+0000 7fb78621f700 1 -- 172.21.15.28:0/1344119244 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7800cdd20 con 0x7fb7800b4a70 2023-11-28T11:12:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.648+0000 7fb78621f700 1 -- 172.21.15.28:0/1344119244 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7800aec60 con 0x7fb7800b4a70 2023-11-28T11:12:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.649+0000 7fb777fff700 1 -- 172.21.15.28:0/1344119244 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb768012c60 con 0x7fb7800b4a70 2023-11-28T11:12:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.649+0000 7fb777fff700 1 --2- 172.21.15.28:0/1344119244 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb7600682f0 0x7fb76006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:48.658 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.649+0000 7fb777fff700 1 -- 172.21.15.28:0/1344119244 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fb768086090 con 0x7fb7800b4a70 2023-11-28T11:12:48.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.649+0000 7fb77f7fe700 1 --2- 172.21.15.28:0/1344119244 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb7600682f0 0x7fb76006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:48.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.649+0000 7fb77f7fe700 1 --2- 172.21.15.28:0/1344119244 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb7600682f0 0x7fb76006a7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb770005910 tx=0x7fb770009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:48.659 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.653+0000 7fb777fff700 1 -- 172.21.15.28:0/1344119244 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb768052770 con 0x7fb7800b4a70 2023-11-28T11:12:48.894 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.893+0000 7fb78621f700 1 -- 172.21.15.28:0/1344119244 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fb780067150 con 0x7fb7800b4a70 2023-11-28T11:12:48.895 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.893+0000 7fb777fff700 1 -- 172.21.15.28:0/1344119244 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7fb768055d90 con 0x7fb7800b4a70 2023-11-28T11:12:48.895 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:48.895 INFO:teuthology.orchestra.run.smithi028.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:12:48.897 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.895+0000 7fb78621f700 1 -- 172.21.15.28:0/1344119244 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb7600682f0 msgr2=0x7fb76006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.897 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.895+0000 7fb78621f700 1 --2- 172.21.15.28:0/1344119244 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb7600682f0 0x7fb76006a7a0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb770005910 tx=0x7fb770009040 comp rx=0 tx=0).stop 2023-11-28T11:12:48.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.895+0000 7fb78621f700 1 -- 172.21.15.28:0/1344119244 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7800b4a70 msgr2=0x7fb7801e2d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:48.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.895+0000 7fb78621f700 1 --2- 172.21.15.28:0/1344119244 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7800b4a70 0x7fb7801e2d40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7fb76800d800 tx=0x7fb76800dbc0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.895+0000 7fb78621f700 1 -- 172.21.15.28:0/1344119244 shutdown_connections 2023-11-28T11:12:48.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.895+0000 7fb78621f700 1 --2- 172.21.15.28:0/1344119244 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb7600682f0 0x7fb76006a7a0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.895+0000 7fb78621f700 1 --2- 172.21.15.28:0/1344119244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7800cc100 0x7fb7800c5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.895+0000 7fb78621f700 1 --2- 172.21.15.28:0/1344119244 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb7800aefe0 0x7fb7801e06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.898 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.895+0000 7fb78621f700 1 --2- 172.21.15.28:0/1344119244 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb7800b4a70 0x7fb7801e2d40 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:48.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.895+0000 7fb78621f700 1 -- 172.21.15.28:0/1344119244 >> 172.21.15.28:0/1344119244 conn(0x7fb7800b3610 msgr2=0x7fb7800cb2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:48.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.895+0000 7fb78621f700 1 -- 172.21.15.28:0/1344119244 shutdown_connections 2023-11-28T11:12:48.899 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:48.895+0000 7fb78621f700 1 -- 172.21.15.28:0/1344119244 wait complete. 2023-11-28T11:12:48.908 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:12:51.442 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:12:51.445 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:12:53.909 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:12:54.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.046+0000 7f5621c39700 1 Processor -- start 2023-11-28T11:12:54.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.046+0000 7f5621c39700 1 -- start start 2023-11-28T11:12:54.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.047+0000 7f5621c39700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f561c15c8a0 0x7f561c0b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:54.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.047+0000 7f5621c39700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f561c0b2290 0x7f561c0ac660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:54.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.047+0000 7f5621c39700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f561c0d0280 con 0x7f561c06f120 2023-11-28T11:12:54.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.047+0000 7f5621c39700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f561c0d0560 con 0x7f561c0b2290 2023-11-28T11:12:54.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.047+0000 7f5621c39700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561c0d0840 con 0x7f561c15c8a0 2023-11-28T11:12:54.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.047+0000 7f561affd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f561c0b2290 0x7f561c0ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:54.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.047+0000 7f561bfff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f561c06f120 0x7f561c0cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40110/0 (socket says 172.21.15.28:40110) 2023-11-28T11:12:54.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.047+0000 7f561bfff700 1 -- 172.21.15.28:0/1599919856 learned_addr learned my addr 172.21.15.28:0/1599919856 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:12:54.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.047+0000 7f561b7fe700 1 --2- 172.21.15.28:0/1599919856 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f561c15c8a0 0x7f561c0b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:54.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.047+0000 7f561affd700 1 -- 172.21.15.28:0/1599919856 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f561c15c8a0 msgr2=0x7f561c0b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:54.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.047+0000 7f561affd700 1 --2- 172.21.15.28:0/1599919856 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f561c15c8a0 0x7f561c0b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:54.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.047+0000 7f561affd700 1 -- 172.21.15.28:0/1599919856 >> v1:172.21.15.28:6789/0 conn(0x7f561c06f120 legacy=0x7f561c0cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:54.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.047+0000 7f561affd700 1 -- 172.21.15.28:0/1599919856 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f561c0b1550 con 0x7f561c0b2290 2023-11-28T11:12:54.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.047+0000 7f561a7fc700 1 -- 172.21.15.28:0/1599919856 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836699752 0 0) 0x7f561c0d0280 con 0x7f561c06f120 2023-11-28T11:12:54.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.047+0000 7f561affd700 1 --2- 172.21.15.28:0/1599919856 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f561c0b2290 0x7f561c0ac660 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5610009840 tx=0x7f5610009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3ca7948149ab304 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:54.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.048+0000 7f561a7fc700 1 -- 172.21.15.28:0/1599919856 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5610011830 con 0x7f561c0b2290 2023-11-28T11:12:54.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.048+0000 7f561a7fc700 1 -- 172.21.15.28:0/1599919856 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5610011990 con 0x7f561c0b2290 2023-11-28T11:12:54.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.048+0000 7f561a7fc700 1 -- 172.21.15.28:0/1599919856 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5610011d20 con 0x7f561c0b2290 2023-11-28T11:12:54.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.048+0000 7f5621c39700 1 -- 172.21.15.28:0/1599919856 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f561c0b2290 msgr2=0x7f561c0ac660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:54.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.048+0000 7f5621c39700 1 --2- 172.21.15.28:0/1599919856 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f561c0b2290 0x7f561c0ac660 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f5610009840 tx=0x7f5610009b50 comp rx=0 tx=0).stop 2023-11-28T11:12:54.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.048+0000 7f5621c39700 1 -- 172.21.15.28:0/1599919856 shutdown_connections 2023-11-28T11:12:54.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.048+0000 7f5621c39700 1 --2- 172.21.15.28:0/1599919856 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f561c15c8a0 0x7f561c0b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:54.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.048+0000 7f5621c39700 1 --2- 172.21.15.28:0/1599919856 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f561c0b2290 0x7f561c0ac660 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:54.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.048+0000 7f5621c39700 1 -- 172.21.15.28:0/1599919856 >> 172.21.15.28:0/1599919856 conn(0x7f561c0b3610 msgr2=0x7f561c0cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:54.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.048+0000 7f5621c39700 1 -- 172.21.15.28:0/1599919856 shutdown_connections 2023-11-28T11:12:54.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.048+0000 7f5621c39700 1 -- 172.21.15.28:0/1599919856 wait complete. 2023-11-28T11:12:54.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.049+0000 7f5621c39700 1 Processor -- start 2023-11-28T11:12:54.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.050+0000 7f5621c39700 1 -- start start 2023-11-28T11:12:54.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.050+0000 7f5621c39700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f561c06f120 0x7f561c0c3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:54.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.050+0000 7f5621c39700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f561c0b2290 0x7f561c0c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:54.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.050+0000 7f5621c39700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f561c15c8a0 0x7f561c14ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:54.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.050+0000 7f5621c39700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c0c99f0 con 0x7f561c15c8a0 2023-11-28T11:12:54.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.050+0000 7f5621c39700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f561c1503d0 con 0x7f561c06f120 2023-11-28T11:12:54.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.050+0000 7f5621c39700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f561c0c9710 con 0x7f561c0b2290 2023-11-28T11:12:54.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.050+0000 7f561bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f561c15c8a0 0x7f561c14ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:54.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.051+0000 7f561bfff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f561c15c8a0 0x7f561c14ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52058/0 (socket says 172.21.15.28:52058) 2023-11-28T11:12:54.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.051+0000 7f561bfff700 1 -- 172.21.15.28:0/1552755118 learned_addr learned my addr 172.21.15.28:0/1552755118 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:54.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.051+0000 7f561affd700 1 --2- 172.21.15.28:0/1552755118 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f561c0b2290 0x7f561c0c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:54.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.051+0000 7f561b7fe700 1 --2- 172.21.15.28:0/1552755118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f561c06f120 0x7f561c0c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:54.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.051+0000 7f561bfff700 1 -- 172.21.15.28:0/1552755118 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f561c0b2290 msgr2=0x7f561c0c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:54.059 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.051+0000 7f561bfff700 1 --2- 172.21.15.28:0/1552755118 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f561c0b2290 0x7f561c0c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:54.059 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.051+0000 7f561bfff700 1 -- 172.21.15.28:0/1552755118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f561c06f120 msgr2=0x7f561c0c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:54.059 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.051+0000 7f561bfff700 1 --2- 172.21.15.28:0/1552755118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f561c06f120 0x7f561c0c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:54.059 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.051+0000 7f561bfff700 1 -- 172.21.15.28:0/1552755118 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56100094a0 con 0x7f561c15c8a0 2023-11-28T11:12:54.059 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.051+0000 7f561bfff700 1 --2- 172.21.15.28:0/1552755118 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f561c15c8a0 0x7f561c14ef80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5614002230 tx=0x7f561400cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:54.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.051+0000 7f5603fff700 1 -- 172.21.15.28:0/1552755118 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f561400aa60 con 0x7f561c15c8a0 2023-11-28T11:12:54.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.051+0000 7f5603fff700 1 -- 172.21.15.28:0/1552755118 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f561400abe0 con 0x7f561c15c8a0 2023-11-28T11:12:54.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.051+0000 7f5621c39700 1 -- 172.21.15.28:0/1552755118 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f561c15cc70 con 0x7f561c15c8a0 2023-11-28T11:12:54.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.051+0000 7f5603fff700 1 -- 172.21.15.28:0/1552755118 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56140070d0 con 0x7f561c15c8a0 2023-11-28T11:12:54.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.051+0000 7f5621c39700 1 -- 172.21.15.28:0/1552755118 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f561c0b1830 con 0x7f561c15c8a0 2023-11-28T11:12:54.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.052+0000 7f5621c39700 1 -- 172.21.15.28:0/1552755118 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f561c06a6c0 con 0x7f561c15c8a0 2023-11-28T11:12:54.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.053+0000 7f5603fff700 1 -- 172.21.15.28:0/1552755118 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f561401a030 con 0x7f561c15c8a0 2023-11-28T11:12:54.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.053+0000 7f5603fff700 1 --2- 172.21.15.28:0/1552755118 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5608068320 0x7f560806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:54.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.053+0000 7f5603fff700 1 -- 172.21.15.28:0/1552755118 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f5614085aa0 con 0x7f561c15c8a0 2023-11-28T11:12:54.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.053+0000 7f561b7fe700 1 --2- 172.21.15.28:0/1552755118 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5608068320 0x7f560806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:54.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.054+0000 7f561b7fe700 1 --2- 172.21.15.28:0/1552755118 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5608068320 0x7f560806a7d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f560c009b40 tx=0x7f560c009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:54.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.057+0000 7f5603fff700 1 -- 172.21.15.28:0/1552755118 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5614052090 con 0x7f561c15c8a0 2023-11-28T11:12:54.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.294+0000 7f5621c39700 1 -- 172.21.15.28:0/1552755118 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f561c0675a0 con 0x7f561c15c8a0 2023-11-28T11:12:54.296 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.294+0000 7f5603fff700 1 -- 172.21.15.28:0/1552755118 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f5614018020 con 0x7f561c15c8a0 2023-11-28T11:12:54.296 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:54.296 INFO:teuthology.orchestra.run.smithi028.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:12:54.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.296+0000 7f5621c39700 1 -- 172.21.15.28:0/1552755118 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5608068320 msgr2=0x7f560806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:54.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.296+0000 7f5621c39700 1 --2- 172.21.15.28:0/1552755118 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5608068320 0x7f560806a7d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f560c009b40 tx=0x7f560c009210 comp rx=0 tx=0).stop 2023-11-28T11:12:54.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.296+0000 7f5621c39700 1 -- 172.21.15.28:0/1552755118 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f561c15c8a0 msgr2=0x7f561c14ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:54.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.296+0000 7f5621c39700 1 --2- 172.21.15.28:0/1552755118 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f561c15c8a0 0x7f561c14ef80 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f5614002230 tx=0x7f561400cf50 comp rx=0 tx=0).stop 2023-11-28T11:12:54.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.296+0000 7f5621c39700 1 -- 172.21.15.28:0/1552755118 shutdown_connections 2023-11-28T11:12:54.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.296+0000 7f5621c39700 1 --2- 172.21.15.28:0/1552755118 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5608068320 0x7f560806a7d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:54.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.296+0000 7f5621c39700 1 --2- 172.21.15.28:0/1552755118 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f561c15c8a0 0x7f561c14ef80 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:54.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.296+0000 7f5621c39700 1 --2- 172.21.15.28:0/1552755118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f561c06f120 0x7f561c0c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:54.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.296+0000 7f5621c39700 1 --2- 172.21.15.28:0/1552755118 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f561c0b2290 0x7f561c0c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:54.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.296+0000 7f5621c39700 1 -- 172.21.15.28:0/1552755118 >> 172.21.15.28:0/1552755118 conn(0x7f561c0b3610 msgr2=0x7f561c0ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:54.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.297+0000 7f5621c39700 1 -- 172.21.15.28:0/1552755118 shutdown_connections 2023-11-28T11:12:54.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:54.297+0000 7f5621c39700 1 -- 172.21.15.28:0/1552755118 wait complete. 2023-11-28T11:12:54.310 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:12:59.312 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:12:59.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.450+0000 7f13af2ff700 1 Processor -- start 2023-11-28T11:12:59.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.450+0000 7f13af2ff700 1 -- start start 2023-11-28T11:12:59.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.451+0000 7f13af2ff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f13a815a620 0x7f13a80cd460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:59.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.451+0000 7f13af2ff700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f13a80b2bd0 0x7f13a80b3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:59.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.451+0000 7f13af2ff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13a80b0d10 0x7f13a80b1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:59.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.451+0000 7f13af2ff700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a815ed10 con 0x7f13a815a620 2023-11-28T11:12:59.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.451+0000 7f13af2ff700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13a80b16d0 con 0x7f13a80b2bd0 2023-11-28T11:12:59.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.451+0000 7f13af2ff700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a815e090 con 0x7f13a80b0d10 2023-11-28T11:12:59.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.451+0000 7f13ad09b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f13a815a620 0x7f13a80cd460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:59.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.451+0000 7f13ad09b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f13a815a620 0x7f13a80cd460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52062/0 (socket says 172.21.15.28:52062) 2023-11-28T11:12:59.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.451+0000 7f13ad09b700 1 -- 172.21.15.28:0/2479083426 learned_addr learned my addr 172.21.15.28:0/2479083426 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:59.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.452+0000 7f13ad89c700 1 --2- 172.21.15.28:0/2479083426 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13a80b0d10 0x7f13a80b1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:59.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.452+0000 7f13ac89a700 1 --2- 172.21.15.28:0/2479083426 >> v2:172.21.15.59:3300/0 conn(0x7f13a80b2bd0 0x7f13a80b3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:59.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.452+0000 7f13ad09b700 1 -- 172.21.15.28:0/2479083426 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13a80b0d10 msgr2=0x7f13a80b1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:59.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.452+0000 7f13ad09b700 1 --2- 172.21.15.28:0/2479083426 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13a80b0d10 0x7f13a80b1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:59.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.452+0000 7f13ad09b700 1 -- 172.21.15.28:0/2479083426 >> v2:172.21.15.59:3300/0 conn(0x7f13a80b2bd0 msgr2=0x7f13a80b3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:12:59.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.452+0000 7f13ad09b700 1 --2- 172.21.15.28:0/2479083426 >> v2:172.21.15.59:3300/0 conn(0x7f13a80b2bd0 0x7f13a80b3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:59.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.452+0000 7f13ad09b700 1 -- 172.21.15.28:0/2479083426 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a815e370 con 0x7f13a815a620 2023-11-28T11:12:59.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.452+0000 7f13ad89c700 1 --2- 172.21.15.28:0/2479083426 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13a80b0d10 0x7f13a80b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:59.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.452+0000 7f13ac89a700 1 --2- 172.21.15.28:0/2479083426 >> v2:172.21.15.59:3300/0 conn(0x7f13a80b2bd0 0x7f13a80b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:12:59.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.452+0000 7f13ad09b700 1 --2- 172.21.15.28:0/2479083426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13a815a620 0x7f13a80cd460 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f13a80728c0 tx=0x7f1398009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a392702737c3cd5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:59.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.452+0000 7f139ffff700 1 -- 172.21.15.28:0/2479083426 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1398005e30 con 0x7f13a815a620 2023-11-28T11:12:59.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.452+0000 7f139ffff700 1 -- 172.21.15.28:0/2479083426 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1398004030 con 0x7f13a815a620 2023-11-28T11:12:59.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.452+0000 7f139ffff700 1 -- 172.21.15.28:0/2479083426 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13980043e0 con 0x7f13a815a620 2023-11-28T11:12:59.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.453+0000 7f13af2ff700 1 -- 172.21.15.28:0/2479083426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13a815a620 msgr2=0x7f13a80cd460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:59.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.453+0000 7f13af2ff700 1 --2- 172.21.15.28:0/2479083426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13a815a620 0x7f13a80cd460 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f13a80728c0 tx=0x7f1398009bd0 comp rx=0 tx=0).stop 2023-11-28T11:12:59.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.453+0000 7f13af2ff700 1 -- 172.21.15.28:0/2479083426 shutdown_connections 2023-11-28T11:12:59.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.453+0000 7f13af2ff700 1 --2- 172.21.15.28:0/2479083426 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13a80b0d10 0x7f13a80b1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:59.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.453+0000 7f13af2ff700 1 --2- 172.21.15.28:0/2479083426 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13a815a620 0x7f13a80cd460 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:59.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.453+0000 7f13af2ff700 1 --2- 172.21.15.28:0/2479083426 >> v2:172.21.15.59:3300/0 conn(0x7f13a80b2bd0 0x7f13a80b3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:59.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.453+0000 7f13af2ff700 1 -- 172.21.15.28:0/2479083426 >> 172.21.15.28:0/2479083426 conn(0x7f13a80cbd10 msgr2=0x7f13a80cc110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:59.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.453+0000 7f13af2ff700 1 -- 172.21.15.28:0/2479083426 shutdown_connections 2023-11-28T11:12:59.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.453+0000 7f13af2ff700 1 -- 172.21.15.28:0/2479083426 wait complete. 2023-11-28T11:12:59.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.454+0000 7f13af2ff700 1 Processor -- start 2023-11-28T11:12:59.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.454+0000 7f13af2ff700 1 -- start start 2023-11-28T11:12:59.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13af2ff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13a80b0d10 0x7f13a814c820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:59.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13af2ff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13a80b2bd0 0x7f13a80c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:59.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13af2ff700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13a80c3300 0x7f13a815c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:59.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13af2ff700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a80c4470 con 0x7f13a80b2bd0 2023-11-28T11:12:59.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13af2ff700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13a814cd60 con 0x7f13a80b0d10 2023-11-28T11:12:59.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13af2ff700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13a80c4900 con 0x7f13a80c3300 2023-11-28T11:12:59.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13ac89a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13a80b2bd0 0x7f13a80c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:59.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13ac89a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13a80b2bd0 0x7f13a80c2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52070/0 (socket says 172.21.15.28:52070) 2023-11-28T11:12:59.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13ac89a700 1 -- 172.21.15.28:0/695518926 learned_addr learned my addr 172.21.15.28:0/695518926 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:12:59.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13ad89c700 1 --2- 172.21.15.28:0/695518926 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13a80c3300 0x7f13a815c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:59.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13ad09b700 1 --2- 172.21.15.28:0/695518926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13a80b0d10 0x7f13a814c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:59.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13ac89a700 1 -- 172.21.15.28:0/695518926 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13a80c3300 msgr2=0x7f13a815c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:59.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13ac89a700 1 --2- 172.21.15.28:0/695518926 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13a80c3300 0x7f13a815c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:59.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13ac89a700 1 -- 172.21.15.28:0/695518926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13a80b0d10 msgr2=0x7f13a814c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:59.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13ac89a700 1 --2- 172.21.15.28:0/695518926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13a80b0d10 0x7f13a814c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:59.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13ac89a700 1 -- 172.21.15.28:0/695518926 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1398009500 con 0x7f13a80b2bd0 2023-11-28T11:12:59.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.455+0000 7f13ad09b700 1 --2- 172.21.15.28:0/695518926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13a80b0d10 0x7f13a814c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:12:59.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.456+0000 7f13ac89a700 1 --2- 172.21.15.28:0/695518926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13a80b2bd0 0x7f13a80c2dc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f13a400e840 tx=0x7f13a400ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:59.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.456+0000 7f139dffb700 1 -- 172.21.15.28:0/695518926 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a400c020 con 0x7f13a80b2bd0 2023-11-28T11:12:59.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.456+0000 7f139dffb700 1 -- 172.21.15.28:0/695518926 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13a4004250 con 0x7f13a80b2bd0 2023-11-28T11:12:59.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.456+0000 7f13af2ff700 1 -- 172.21.15.28:0/695518926 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13a815e930 con 0x7f13a80b2bd0 2023-11-28T11:12:59.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.456+0000 7f139dffb700 1 -- 172.21.15.28:0/695518926 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13a40036a0 con 0x7f13a80b2bd0 2023-11-28T11:12:59.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.456+0000 7f13af2ff700 1 -- 172.21.15.28:0/695518926 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13a815e650 con 0x7f13a80b2bd0 2023-11-28T11:12:59.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.457+0000 7f13af2ff700 1 -- 172.21.15.28:0/695518926 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13a80cee60 con 0x7f13a80b2bd0 2023-11-28T11:12:59.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.457+0000 7f139dffb700 1 -- 172.21.15.28:0/695518926 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f13a4003800 con 0x7f13a80b2bd0 2023-11-28T11:12:59.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.458+0000 7f139dffb700 1 --2- 172.21.15.28:0/695518926 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f138c0682f0 0x7f138c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:12:59.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.458+0000 7f139dffb700 1 -- 172.21.15.28:0/695518926 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f13a4085ae0 con 0x7f13a80b2bd0 2023-11-28T11:12:59.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.458+0000 7f13ad09b700 1 --2- 172.21.15.28:0/695518926 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f138c0682f0 0x7f138c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:12:59.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.458+0000 7f13ad09b700 1 --2- 172.21.15.28:0/695518926 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f138c0682f0 0x7f138c06a7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f13a80728c0 tx=0x7f1398008070 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:12:59.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.462+0000 7f139dffb700 1 -- 172.21.15.28:0/695518926 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f13a40521c0 con 0x7f13a80b2bd0 2023-11-28T11:12:59.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.700+0000 7f13af2ff700 1 -- 172.21.15.28:0/695518926 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f13a80c4620 con 0x7f13a80b2bd0 2023-11-28T11:12:59.701 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.700+0000 7f139dffb700 1 -- 172.21.15.28:0/695518926 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f13a400f040 con 0x7f13a80b2bd0 2023-11-28T11:12:59.702 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:12:59.702 INFO:teuthology.orchestra.run.smithi028.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:12:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.702+0000 7f13af2ff700 1 -- 172.21.15.28:0/695518926 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f138c0682f0 msgr2=0x7f138c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.702+0000 7f13af2ff700 1 --2- 172.21.15.28:0/695518926 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f138c0682f0 0x7f138c06a7a0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f13a80728c0 tx=0x7f1398008070 comp rx=0 tx=0).stop 2023-11-28T11:12:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.702+0000 7f13af2ff700 1 -- 172.21.15.28:0/695518926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13a80b2bd0 msgr2=0x7f13a80c2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:12:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.702+0000 7f13af2ff700 1 --2- 172.21.15.28:0/695518926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13a80b2bd0 0x7f13a80c2dc0 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f13a400e840 tx=0x7f13a400ec00 comp rx=0 tx=0).stop 2023-11-28T11:12:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.702+0000 7f13af2ff700 1 -- 172.21.15.28:0/695518926 shutdown_connections 2023-11-28T11:12:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.702+0000 7f13af2ff700 1 --2- 172.21.15.28:0/695518926 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f138c0682f0 0x7f138c06a7a0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:59.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.702+0000 7f13af2ff700 1 --2- 172.21.15.28:0/695518926 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f13a80c3300 0x7f13a815c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:59.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.702+0000 7f13af2ff700 1 --2- 172.21.15.28:0/695518926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13a80b0d10 0x7f13a814c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:59.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.702+0000 7f13af2ff700 1 --2- 172.21.15.28:0/695518926 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f13a80b2bd0 0x7f13a80c2dc0 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:12:59.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.702+0000 7f13af2ff700 1 -- 172.21.15.28:0/695518926 >> 172.21.15.28:0/695518926 conn(0x7f13a80cbd10 msgr2=0x7f13a80aebf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:12:59.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.702+0000 7f13af2ff700 1 -- 172.21.15.28:0/695518926 shutdown_connections 2023-11-28T11:12:59.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:12:59.702+0000 7f13af2ff700 1 -- 172.21.15.28:0/695518926 wait complete. 2023-11-28T11:12:59.715 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:13:04.717 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:13:04.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.854+0000 7f7ad669c700 1 Processor -- start 2023-11-28T11:13:04.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.854+0000 7f7ad669c700 1 -- start start 2023-11-28T11:13:04.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.854+0000 7f7ad669c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ad00b6510 0x7f7ad00cb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:04.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.854+0000 7f7ad669c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7ad0070c30 0x7f7ad00cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:04.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.854+0000 7f7ad669c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ad00ce9b0 con 0x7f7ad00ccfe0 2023-11-28T11:13:04.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.854+0000 7f7ad669c700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ad00b0d30 con 0x7f7ad0070c30 2023-11-28T11:13:04.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.854+0000 7f7ad669c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad00b0fd0 con 0x7f7ad00b6510 2023-11-28T11:13:04.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.855+0000 7f7ad4c39700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7ad0070c30 0x7f7ad00cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:04.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.855+0000 7f7ad4c39700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f7ad0070c30 0x7f7ad00cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52080/0 (socket says 172.21.15.28:52080) 2023-11-28T11:13:04.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.855+0000 7f7ad4c39700 1 -- 172.21.15.28:0/1158496255 learned_addr learned my addr 172.21.15.28:0/1158496255 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:04.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.855+0000 7f7acffff700 1 --2- 172.21.15.28:0/1158496255 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ad00b6510 0x7f7ad00cb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:04.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.855+0000 7f7ad4c39700 1 -- 172.21.15.28:0/1158496255 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ad00b6510 msgr2=0x7f7ad00cb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:04.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.855+0000 7f7ad4c39700 1 --2- 172.21.15.28:0/1158496255 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ad00b6510 0x7f7ad00cb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:04.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.855+0000 7f7ad4c39700 1 -- 172.21.15.28:0/1158496255 >> v1:172.21.15.59:6789/0 conn(0x7f7ad00ccfe0 legacy=0x7f7ad0070520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:04.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.855+0000 7f7ad4c39700 1 -- 172.21.15.28:0/1158496255 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ad00b12b0 con 0x7f7ad0070c30 2023-11-28T11:13:04.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.855+0000 7f7ad4c39700 1 --2- 172.21.15.28:0/1158496255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ad0070c30 0x7f7ad00cee40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009e50 tx=0x7f7ac800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5b23f241bd8638f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:04.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.855+0000 7f7aceffd700 1 -- 172.21.15.28:0/1158496255 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac800e070 con 0x7f7ad0070c30 2023-11-28T11:13:04.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.855+0000 7f7aceffd700 1 -- 172.21.15.28:0/1158496255 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac800b910 con 0x7f7ad0070c30 2023-11-28T11:13:04.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.855+0000 7f7aceffd700 1 -- 172.21.15.28:0/1158496255 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac800bcc0 con 0x7f7ad0070c30 2023-11-28T11:13:04.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.856+0000 7f7ad669c700 1 -- 172.21.15.28:0/1158496255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ad0070c30 msgr2=0x7f7ad00cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:04.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.856+0000 7f7ad669c700 1 --2- 172.21.15.28:0/1158496255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ad0070c30 0x7f7ad00cee40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f7ac8009e50 tx=0x7f7ac800b330 comp rx=0 tx=0).stop 2023-11-28T11:13:04.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.856+0000 7f7ad669c700 1 -- 172.21.15.28:0/1158496255 shutdown_connections 2023-11-28T11:13:04.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.856+0000 7f7ad669c700 1 --2- 172.21.15.28:0/1158496255 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ad0070c30 0x7f7ad00cee40 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:04.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.856+0000 7f7ad669c700 1 --2- 172.21.15.28:0/1158496255 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ad00b6510 0x7f7ad00cb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:04.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.856+0000 7f7ad669c700 1 -- 172.21.15.28:0/1158496255 >> 172.21.15.28:0/1158496255 conn(0x7f7ad00d0bd0 msgr2=0x7f7ad00d0fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:04.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.856+0000 7f7ad669c700 1 -- 172.21.15.28:0/1158496255 shutdown_connections 2023-11-28T11:13:04.860 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.856+0000 7f7ad669c700 1 -- 172.21.15.28:0/1158496255 wait complete. 2023-11-28T11:13:04.862 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.857+0000 7f7ad669c700 1 Processor -- start 2023-11-28T11:13:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.857+0000 7f7ad669c700 1 -- start start 2023-11-28T11:13:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.858+0000 7f7ad669c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ad0070c30 0x7f7ad01e3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.858+0000 7f7ad669c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ad00b6510 0x7f7ad015ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.858+0000 7f7ad669c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ad00ccfe0 0x7f7ad015cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.858+0000 7f7ad669c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad0146df0 con 0x7f7ad00b6510 2023-11-28T11:13:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.858+0000 7f7ad669c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad01470d0 con 0x7f7ad0070c30 2023-11-28T11:13:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.858+0000 7f7ad669c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ad01473b0 con 0x7f7ad00ccfe0 2023-11-28T11:13:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.858+0000 7f7acf7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ad00b6510 0x7f7ad015ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.858+0000 7f7acf7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ad00b6510 0x7f7ad015ca20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52084/0 (socket says 172.21.15.28:52084) 2023-11-28T11:13:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.858+0000 7f7acf7fe700 1 -- 172.21.15.28:0/3575254797 learned_addr learned my addr 172.21.15.28:0/3575254797 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.858+0000 7f7ad4c39700 1 --2- 172.21.15.28:0/3575254797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ad00ccfe0 0x7f7ad015cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.858+0000 7f7acffff700 1 --2- 172.21.15.28:0/3575254797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ad0070c30 0x7f7ad01e3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:04.863 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.858+0000 7f7acf7fe700 1 -- 172.21.15.28:0/3575254797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ad00ccfe0 msgr2=0x7f7ad015cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.859+0000 7f7acf7fe700 1 --2- 172.21.15.28:0/3575254797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ad00ccfe0 0x7f7ad015cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.859+0000 7f7acf7fe700 1 -- 172.21.15.28:0/3575254797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ad0070c30 msgr2=0x7f7ad01e3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.859+0000 7f7acf7fe700 1 --2- 172.21.15.28:0/3575254797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ad0070c30 0x7f7ad01e3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.859+0000 7f7acf7fe700 1 -- 172.21.15.28:0/3575254797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ac8009b30 con 0x7f7ad00b6510 2023-11-28T11:13:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.859+0000 7f7acffff700 1 --2- 172.21.15.28:0/3575254797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ad0070c30 0x7f7ad01e3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.859+0000 7f7acf7fe700 1 --2- 172.21.15.28:0/3575254797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ad00b6510 0x7f7ad015ca20 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400c420 tx=0x7f7ac400c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.859+0000 7f7accff9700 1 -- 172.21.15.28:0/3575254797 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac40052d0 con 0x7f7ad00b6510 2023-11-28T11:13:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.859+0000 7f7accff9700 1 -- 172.21.15.28:0/3575254797 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ac4005430 con 0x7f7ad00b6510 2023-11-28T11:13:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.859+0000 7f7ad669c700 1 -- 172.21.15.28:0/3575254797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ad00cf380 con 0x7f7ad00b6510 2023-11-28T11:13:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.859+0000 7f7accff9700 1 -- 172.21.15.28:0/3575254797 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ac40038c0 con 0x7f7ad00b6510 2023-11-28T11:13:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.859+0000 7f7ad669c700 1 -- 172.21.15.28:0/3575254797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ad00cb760 con 0x7f7ad00b6510 2023-11-28T11:13:04.864 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.860+0000 7f7ad669c700 1 -- 172.21.15.28:0/3575254797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ad00cf510 con 0x7f7ad00b6510 2023-11-28T11:13:04.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.860+0000 7f7accff9700 1 -- 172.21.15.28:0/3575254797 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7ac4019030 con 0x7f7ad00b6510 2023-11-28T11:13:04.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.861+0000 7f7accff9700 1 --2- 172.21.15.28:0/3575254797 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7abc0682f0 0x7f7abc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:04.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.861+0000 7f7accff9700 1 -- 172.21.15.28:0/3575254797 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f7ac40850f0 con 0x7f7ad00b6510 2023-11-28T11:13:04.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.861+0000 7f7acffff700 1 --2- 172.21.15.28:0/3575254797 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7abc0682f0 0x7f7abc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:04.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.861+0000 7f7acffff700 1 --2- 172.21.15.28:0/3575254797 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7abc0682f0 0x7f7abc06a7a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009c30 tx=0x7f7ac0008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:04.866 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:04.865+0000 7f7accff9700 1 -- 172.21.15.28:0/3575254797 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f7ac40517d0 con 0x7f7ad00b6510 2023-11-28T11:13:05.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:05.102+0000 7f7ad669c700 1 -- 172.21.15.28:0/3575254797 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7ad00675a0 con 0x7f7ad00b6510 2023-11-28T11:13:05.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:05.103+0000 7f7accff9700 1 -- 172.21.15.28:0/3575254797 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f7ac4054df0 con 0x7f7ad00b6510 2023-11-28T11:13:05.104 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:05.104 INFO:teuthology.orchestra.run.smithi028.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:13:05.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:05.105+0000 7f7ad669c700 1 -- 172.21.15.28:0/3575254797 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7abc0682f0 msgr2=0x7f7abc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:05.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:05.105+0000 7f7ad669c700 1 --2- 172.21.15.28:0/3575254797 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7abc0682f0 0x7f7abc06a7a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7ac0009c30 tx=0x7f7ac0008040 comp rx=0 tx=0).stop 2023-11-28T11:13:05.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:05.105+0000 7f7ad669c700 1 -- 172.21.15.28:0/3575254797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ad00b6510 msgr2=0x7f7ad015ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:05.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:05.105+0000 7f7ad669c700 1 --2- 172.21.15.28:0/3575254797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ad00b6510 0x7f7ad015ca20 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400c420 tx=0x7f7ac400c7e0 comp rx=0 tx=0).stop 2023-11-28T11:13:05.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:05.105+0000 7f7ad669c700 1 -- 172.21.15.28:0/3575254797 shutdown_connections 2023-11-28T11:13:05.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:05.105+0000 7f7ad669c700 1 --2- 172.21.15.28:0/3575254797 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7abc0682f0 0x7f7abc06a7a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:05.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:05.105+0000 7f7ad669c700 1 --2- 172.21.15.28:0/3575254797 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7ad00ccfe0 0x7f7ad015cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:05.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:05.105+0000 7f7ad669c700 1 --2- 172.21.15.28:0/3575254797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ad0070c30 0x7f7ad01e3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:05.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:05.105+0000 7f7ad669c700 1 --2- 172.21.15.28:0/3575254797 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7ad00b6510 0x7f7ad015ca20 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:05.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:05.105+0000 7f7ad669c700 1 -- 172.21.15.28:0/3575254797 >> 172.21.15.28:0/3575254797 conn(0x7f7ad00d0bd0 msgr2=0x7f7ad00b5870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:05.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:05.105+0000 7f7ad669c700 1 -- 172.21.15.28:0/3575254797 shutdown_connections 2023-11-28T11:13:05.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:05.105+0000 7f7ad669c700 1 -- 172.21.15.28:0/3575254797 wait complete. 2023-11-28T11:13:05.118 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=20 retry_count=0)... 2023-11-28T11:13:10.119 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T11:13:10.253 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.252+0000 7f5ec0637700 1 Processor -- start 2023-11-28T11:13:10.253 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.252+0000 7f5ec0637700 1 -- start start 2023-11-28T11:13:10.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.253+0000 7f5ec0637700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5eb80aefe0 0x7f5eb80b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:10.255 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.253+0000 7f5ec0637700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb80b5880 con 0x7f5eb80b4a70 2023-11-28T11:13:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.253+0000 7f5ec0637700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5eb80d01e0 con 0x7f5eb80cc100 2023-11-28T11:13:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.253+0000 7f5ec0637700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5eb80d04c0 con 0x7f5eb80aefe0 2023-11-28T11:13:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.253+0000 7f5ebdbd2700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5eb80aefe0 0x7f5eb80b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.253+0000 7f5ebe3d3700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5eb80b4a70 0x7f5eb80cd610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40148/0 (socket says 172.21.15.28:40148) 2023-11-28T11:13:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.253+0000 7f5ebe3d3700 1 -- 172.21.15.28:0/3521166060 learned_addr learned my addr 172.21.15.28:0/3521166060 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.254+0000 7f5ebdbd2700 1 -- 172.21.15.28:0/3521166060 >> v1:172.21.15.59:6789/0 conn(0x7f5eb80cc100 legacy=0x7f5eb80cb3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.254+0000 7f5ebdbd2700 1 -- 172.21.15.28:0/3521166060 >> v1:172.21.15.28:6789/0 conn(0x7f5eb80b4a70 legacy=0x7f5eb80cd610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.254+0000 7f5ebdbd2700 1 -- 172.21.15.28:0/3521166060 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb80d07a0 con 0x7f5eb80aefe0 2023-11-28T11:13:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.254+0000 7f5ebd3d1700 1 -- 172.21.15.28:0/3521166060 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1416932375 0 0) 0x7f5eb80b5880 con 0x7f5eb80b4a70 2023-11-28T11:13:10.256 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.254+0000 7f5ebdbd2700 1 --2- 172.21.15.28:0/3521166060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5eb80aefe0 0x7f5eb80b4e40 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009840 tx=0x7f5eb4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74870a62495b27bf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:10.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.254+0000 7f5ebd3d1700 1 -- 172.21.15.28:0/3521166060 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4011830 con 0x7f5eb80aefe0 2023-11-28T11:13:10.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.254+0000 7f5ebd3d1700 1 -- 172.21.15.28:0/3521166060 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5eb4011990 con 0x7f5eb80aefe0 2023-11-28T11:13:10.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.254+0000 7f5ebd3d1700 1 -- 172.21.15.28:0/3521166060 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4011d20 con 0x7f5eb80aefe0 2023-11-28T11:13:10.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.255+0000 7f5ec0637700 1 -- 172.21.15.28:0/3521166060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5eb80aefe0 msgr2=0x7f5eb80b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:10.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.255+0000 7f5ec0637700 1 --2- 172.21.15.28:0/3521166060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5eb80aefe0 0x7f5eb80b4e40 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009840 tx=0x7f5eb4009b50 comp rx=0 tx=0).stop 2023-11-28T11:13:10.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.255+0000 7f5ec0637700 1 -- 172.21.15.28:0/3521166060 shutdown_connections 2023-11-28T11:13:10.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.255+0000 7f5ec0637700 1 --2- 172.21.15.28:0/3521166060 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5eb80aefe0 0x7f5eb80b4e40 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:10.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.255+0000 7f5ec0637700 1 -- 172.21.15.28:0/3521166060 >> 172.21.15.28:0/3521166060 conn(0x7f5eb80b3610 msgr2=0x7f5eb80cfad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:10.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.255+0000 7f5ec0637700 1 -- 172.21.15.28:0/3521166060 shutdown_connections 2023-11-28T11:13:10.257 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.255+0000 7f5ec0637700 1 -- 172.21.15.28:0/3521166060 wait complete. 2023-11-28T11:13:10.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.256+0000 7f5ec0637700 1 Processor -- start 2023-11-28T11:13:10.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.256+0000 7f5ec0637700 1 -- start start 2023-11-28T11:13:10.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ec0637700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5eb80aefe0 0x7f5eb80c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:10.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ec0637700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5eb80b4a70 0x7f5eb80cb530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:10.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ec0637700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5eb80cc100 0x7f5eb80c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:10.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ec0637700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80c4670 con 0x7f5eb80aefe0 2023-11-28T11:13:10.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ec0637700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80c4e00 con 0x7f5eb80b4a70 2023-11-28T11:13:10.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ec0637700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5eb80c31e0 con 0x7f5eb80cc100 2023-11-28T11:13:10.260 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ebe3d3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5eb80aefe0 0x7f5eb80c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:10.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ebe3d3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5eb80aefe0 0x7f5eb80c48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52096/0 (socket says 172.21.15.28:52096) 2023-11-28T11:13:10.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ebe3d3700 1 -- 172.21.15.28:0/1578150887 learned_addr learned my addr 172.21.15.28:0/1578150887 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:10.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ebdbd2700 1 --2- 172.21.15.28:0/1578150887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5eb80b4a70 0x7f5eb80cb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:10.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ebebd4700 1 --2- 172.21.15.28:0/1578150887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5eb80cc100 0x7f5eb80c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:10.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ebe3d3700 1 -- 172.21.15.28:0/1578150887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5eb80cc100 msgr2=0x7f5eb80c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:10.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ebe3d3700 1 --2- 172.21.15.28:0/1578150887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5eb80cc100 0x7f5eb80c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:10.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ebe3d3700 1 -- 172.21.15.28:0/1578150887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5eb80b4a70 msgr2=0x7f5eb80cb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:10.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ebe3d3700 1 --2- 172.21.15.28:0/1578150887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5eb80b4a70 0x7f5eb80cb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:10.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ebe3d3700 1 -- 172.21.15.28:0/1578150887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5eb40094a0 con 0x7f5eb80aefe0 2023-11-28T11:13:10.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5ebe3d3700 1 --2- 172.21.15.28:0/1578150887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5eb80aefe0 0x7f5eb80c48c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c520 tx=0x7f5ea800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:10.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.257+0000 7f5eaeffd700 1 -- 172.21.15.28:0/1578150887 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8004c80 con 0x7f5eb80aefe0 2023-11-28T11:13:10.261 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.258+0000 7f5eaeffd700 1 -- 172.21.15.28:0/1578150887 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ea8004e00 con 0x7f5eb80aefe0 2023-11-28T11:13:10.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.258+0000 7f5ec0637700 1 -- 172.21.15.28:0/1578150887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5eb80aead0 con 0x7f5eb80aefe0 2023-11-28T11:13:10.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.258+0000 7f5eaeffd700 1 -- 172.21.15.28:0/1578150887 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ea8007310 con 0x7f5eb80aefe0 2023-11-28T11:13:10.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.258+0000 7f5ec0637700 1 -- 172.21.15.28:0/1578150887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5eb80cdd20 con 0x7f5eb80aefe0 2023-11-28T11:13:10.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.259+0000 7f5ec0637700 1 -- 172.21.15.28:0/1578150887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5eb80aec60 con 0x7f5eb80aefe0 2023-11-28T11:13:10.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.259+0000 7f5eaeffd700 1 -- 172.21.15.28:0/1578150887 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5ea801a020 con 0x7f5eb80aefe0 2023-11-28T11:13:10.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.259+0000 7f5eaeffd700 1 --2- 172.21.15.28:0/1578150887 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5e9c0682f0 0x7f5e9c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:10.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.260+0000 7f5ebdbd2700 1 --2- 172.21.15.28:0/1578150887 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5e9c0682f0 0x7f5e9c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:10.262 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.260+0000 7f5eaeffd700 1 -- 172.21.15.28:0/1578150887 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f5ea8085a30 con 0x7f5eb80aefe0 2023-11-28T11:13:10.263 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.260+0000 7f5ebdbd2700 1 --2- 172.21.15.28:0/1578150887 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5e9c0682f0 0x7f5e9c06a7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009840 tx=0x7f5eb401c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:10.264 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.264+0000 7f5eaeffd700 1 -- 172.21.15.28:0/1578150887 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5ea8052110 con 0x7f5eb80aefe0 2023-11-28T11:13:10.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.503+0000 7f5ec0637700 1 -- 172.21.15.28:0/1578150887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f5eb80c3510 con 0x7f5eb80aefe0 2023-11-28T11:13:10.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.503+0000 7f5eaeffd700 1 -- 172.21.15.28:0/1578150887 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+164 (secure 0 0 0) 0x7f5ea8055730 con 0x7f5eb80aefe0 2023-11-28T11:13:10.505 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:10.505 INFO:teuthology.orchestra.run.smithi028.stdout:{"status":"HEALTH_ERR","checks":{"MDS_DAMAGE":{"severity":"HEALTH_ERR","summary":{"message":"1 MDSs report damaged metadata","count":1},"muted":false}},"mutes":[]} 2023-11-28T11:13:10.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.505+0000 7f5ec0637700 1 -- 172.21.15.28:0/1578150887 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5e9c0682f0 msgr2=0x7f5e9c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:10.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.505+0000 7f5ec0637700 1 --2- 172.21.15.28:0/1578150887 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5e9c0682f0 0x7f5e9c06a7a0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4009840 tx=0x7f5eb401c040 comp rx=0 tx=0).stop 2023-11-28T11:13:10.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.505+0000 7f5ec0637700 1 -- 172.21.15.28:0/1578150887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5eb80aefe0 msgr2=0x7f5eb80c48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:10.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.505+0000 7f5ec0637700 1 --2- 172.21.15.28:0/1578150887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5eb80aefe0 0x7f5eb80c48c0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5ea800c520 tx=0x7f5ea800c8e0 comp rx=0 tx=0).stop 2023-11-28T11:13:10.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.505+0000 7f5ec0637700 1 -- 172.21.15.28:0/1578150887 shutdown_connections 2023-11-28T11:13:10.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.505+0000 7f5ec0637700 1 --2- 172.21.15.28:0/1578150887 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5e9c0682f0 0x7f5e9c06a7a0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:10.509 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.505+0000 7f5ec0637700 1 --2- 172.21.15.28:0/1578150887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5eb80aefe0 0x7f5eb80c48c0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:10.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.505+0000 7f5ec0637700 1 --2- 172.21.15.28:0/1578150887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5eb80cc100 0x7f5eb80c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:10.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.505+0000 7f5ec0637700 1 --2- 172.21.15.28:0/1578150887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5eb80b4a70 0x7f5eb80cb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:10.510 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.505+0000 7f5ec0637700 1 -- 172.21.15.28:0/1578150887 >> 172.21.15.28:0/1578150887 conn(0x7f5eb80b3610 msgr2=0x7f5eb80af520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:10.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.505+0000 7f5ec0637700 1 -- 172.21.15.28:0/1578150887 shutdown_connections 2023-11-28T11:13:10.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.505+0000 7f5ec0637700 1 -- 172.21.15.28:0/1578150887 wait complete. 2023-11-28T11:13:10.519 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T11:13:10.519 DEBUG:teuthology.orchestra.run.smithi028:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:13:10.563 DEBUG:teuthology.orchestra.run.smithi059:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T11:13:10.621 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T11:13:10.621 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:13:10.621 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:13:10.640 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:13:10.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.803+0000 7fcabc2a0700 1 Processor -- start 2023-11-28T11:13:10.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.803+0000 7fcabc2a0700 1 -- start start 2023-11-28T11:13:10.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.804+0000 7fcabc2a0700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcab415a650 0x7fcab40b2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:10.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.804+0000 7fcabc2a0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcab40b5500 0x7fcab40cc4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:10.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.804+0000 7fcabc2a0700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab40ca990 con 0x7fcab415aa20 2023-11-28T11:13:10.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.804+0000 7fcabc2a0700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab40caf50 con 0x7fcab415a650 2023-11-28T11:13:10.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.804+0000 7fcabc2a0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40aef10 con 0x7fcab40b5500 2023-11-28T11:13:10.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.804+0000 7fcaba03c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcab415a650 0x7fcab40b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:10.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.805+0000 7fcaba03c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fcab415a650 0x7fcab40b2950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52104/0 (socket says 172.21.15.28:52104) 2023-11-28T11:13:10.807 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.805+0000 7fcaba03c700 1 -- 172.21.15.28:0/3371672959 learned_addr learned my addr 172.21.15.28:0/3371672959 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:10.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.805+0000 7fcaba03c700 1 -- 172.21.15.28:0/3371672959 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcab40b5500 msgr2=0x7fcab40cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:10.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.805+0000 7fcaba03c700 1 --2- 172.21.15.28:0/3371672959 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcab40b5500 0x7fcab40cc4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:10.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.805+0000 7fcaba03c700 1 -- 172.21.15.28:0/3371672959 >> v1:172.21.15.59:6789/0 conn(0x7fcab415aa20 legacy=0x7fcab40ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:10.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.805+0000 7fcaba03c700 1 -- 172.21.15.28:0/3371672959 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab40af1f0 con 0x7fcab415a650 2023-11-28T11:13:10.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.805+0000 7fcaba03c700 1 --2- 172.21.15.28:0/3371672959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcab415a650 0x7fcab40b2950 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcab00098f0 tx=0x7fcab0009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f572ba18a868227c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:10.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.805+0000 7fcab903a700 1 -- 172.21.15.28:0/3371672959 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab0004230 con 0x7fcab415a650 2023-11-28T11:13:10.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.805+0000 7fcab903a700 1 -- 172.21.15.28:0/3371672959 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcab0004390 con 0x7fcab415a650 2023-11-28T11:13:10.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.805+0000 7fcab903a700 1 -- 172.21.15.28:0/3371672959 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcab00036a0 con 0x7fcab415a650 2023-11-28T11:13:10.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.806+0000 7fcabc2a0700 1 -- 172.21.15.28:0/3371672959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcab415a650 msgr2=0x7fcab40b2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:10.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.806+0000 7fcabc2a0700 1 --2- 172.21.15.28:0/3371672959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcab415a650 0x7fcab40b2950 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fcab00098f0 tx=0x7fcab0009c00 comp rx=0 tx=0).stop 2023-11-28T11:13:10.808 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.806+0000 7fcabc2a0700 1 -- 172.21.15.28:0/3371672959 shutdown_connections 2023-11-28T11:13:10.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.806+0000 7fcabc2a0700 1 --2- 172.21.15.28:0/3371672959 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcab415a650 0x7fcab40b2950 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:10.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.806+0000 7fcabc2a0700 1 --2- 172.21.15.28:0/3371672959 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcab40b5500 0x7fcab40cc4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:10.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.806+0000 7fcabc2a0700 1 -- 172.21.15.28:0/3371672959 >> 172.21.15.28:0/3371672959 conn(0x7fcab40b1650 msgr2=0x7fcab40b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:10.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.806+0000 7fcabc2a0700 1 -- 172.21.15.28:0/3371672959 shutdown_connections 2023-11-28T11:13:10.809 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.806+0000 7fcabc2a0700 1 -- 172.21.15.28:0/3371672959 wait complete. 2023-11-28T11:13:10.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.807+0000 7fcabc2a0700 1 Processor -- start 2023-11-28T11:13:10.812 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.808+0000 7fcabc2a0700 1 -- start start 2023-11-28T11:13:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.808+0000 7fcabc2a0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab40b5500 0x7fcab40b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.808+0000 7fcabc2a0700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcab415a650 0x7fcab41581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.808+0000 7fcabc2a0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcab415aa20 0x7fcab4158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.808+0000 7fcabc2a0700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40b2a80 con 0x7fcab415a650 2023-11-28T11:13:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.808+0000 7fcabc2a0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40b2d60 con 0x7fcab40b5500 2023-11-28T11:13:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.808+0000 7fcabc2a0700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40b3040 con 0x7fcab415aa20 2023-11-28T11:13:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcab983b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcab415a650 0x7fcab41581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcab983b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcab415a650 0x7fcab41581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52110/0 (socket says 172.21.15.28:52110) 2023-11-28T11:13:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcab983b700 1 -- 172.21.15.28:0/2073229763 learned_addr learned my addr 172.21.15.28:0/2073229763 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcaba03c700 1 --2- 172.21.15.28:0/2073229763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab40b5500 0x7fcab40b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcaba83d700 1 --2- 172.21.15.28:0/2073229763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcab415aa20 0x7fcab4158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:10.813 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcab983b700 1 -- 172.21.15.28:0/2073229763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcab415aa20 msgr2=0x7fcab4158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcab983b700 1 --2- 172.21.15.28:0/2073229763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcab415aa20 0x7fcab4158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcab983b700 1 -- 172.21.15.28:0/2073229763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab40b5500 msgr2=0x7fcab40b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcab983b700 1 --2- 172.21.15.28:0/2073229763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab40b5500 0x7fcab40b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcab983b700 1 -- 172.21.15.28:0/2073229763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab0009550 con 0x7fcab415a650 2023-11-28T11:13:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcaba83d700 1 --2- 172.21.15.28:0/2073229763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcab415aa20 0x7fcab4158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:13:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcab983b700 1 --2- 172.21.15.28:0/2073229763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcab415a650 0x7fcab41581c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800b450 tx=0x7fcaa800b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcaa6ffd700 1 -- 172.21.15.28:0/2073229763 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8011830 con 0x7fcab415a650 2023-11-28T11:13:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcaa6ffd700 1 -- 172.21.15.28:0/2073229763 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa8011990 con 0x7fcab415a650 2023-11-28T11:13:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcaa6ffd700 1 -- 172.21.15.28:0/2073229763 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa8011d20 con 0x7fcab415a650 2023-11-28T11:13:10.814 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcabc2a0700 1 -- 172.21.15.28:0/2073229763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab40cc9f0 con 0x7fcab415a650 2023-11-28T11:13:10.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.809+0000 7fcabc2a0700 1 -- 172.21.15.28:0/2073229763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab40cabe0 con 0x7fcab415a650 2023-11-28T11:13:10.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.810+0000 7fcabc2a0700 1 -- 172.21.15.28:0/2073229763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab40ccb80 con 0x7fcab415a650 2023-11-28T11:13:10.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.811+0000 7fcaa6ffd700 1 -- 172.21.15.28:0/2073229763 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fcaa8021070 con 0x7fcab415a650 2023-11-28T11:13:10.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.811+0000 7fcaa6ffd700 1 --2- 172.21.15.28:0/2073229763 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fca980682f0 0x7fca9806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:10.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.811+0000 7fcaa6ffd700 1 -- 172.21.15.28:0/2073229763 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fcaa8085ca0 con 0x7fcab415a650 2023-11-28T11:13:10.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.811+0000 7fcaba03c700 1 --2- 172.21.15.28:0/2073229763 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fca980682f0 0x7fca9806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:10.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.811+0000 7fcaba03c700 1 --2- 172.21.15.28:0/2073229763 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fca980682f0 0x7fca9806a7a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009130 tx=0x7fcab0008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:10.815 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:10.815+0000 7fcaa6ffd700 1 -- 172.21.15.28:0/2073229763 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fcaa808a070 con 0x7fcab415a650 2023-11-28T11:13:11.011 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.010+0000 7fcabc2a0700 1 -- 172.21.15.28:0/2073229763 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcab40675a0 con 0x7fcab415a650 2023-11-28T11:13:11.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.012+0000 7fcaa6ffd700 1 -- 172.21.15.28:0/2073229763 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18174 (secure 0 0 0) 0x7fcaa8052380 con 0x7fcab415a650 2023-11-28T11:13:11.012 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:11.013 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":68,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:12:35.277297+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:12:01.724396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:12:02.776612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.9","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:13:11.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.014+0000 7fcabc2a0700 1 -- 172.21.15.28:0/2073229763 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fca980682f0 msgr2=0x7fca9806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.014+0000 7fcabc2a0700 1 --2- 172.21.15.28:0/2073229763 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fca980682f0 0x7fca9806a7a0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fcab0009130 tx=0x7fcab0008030 comp rx=0 tx=0).stop 2023-11-28T11:13:11.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.014+0000 7fcabc2a0700 1 -- 172.21.15.28:0/2073229763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcab415a650 msgr2=0x7fcab41581c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.014+0000 7fcabc2a0700 1 --2- 172.21.15.28:0/2073229763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcab415a650 0x7fcab41581c0 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fcaa800b450 tx=0x7fcaa800b810 comp rx=0 tx=0).stop 2023-11-28T11:13:11.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.014+0000 7fcabc2a0700 1 -- 172.21.15.28:0/2073229763 shutdown_connections 2023-11-28T11:13:11.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.014+0000 7fcabc2a0700 1 --2- 172.21.15.28:0/2073229763 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fca980682f0 0x7fca9806a7a0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.014+0000 7fcabc2a0700 1 --2- 172.21.15.28:0/2073229763 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fcab415aa20 0x7fcab4158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.014+0000 7fcabc2a0700 1 --2- 172.21.15.28:0/2073229763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcab40b5500 0x7fcab40b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.014+0000 7fcabc2a0700 1 --2- 172.21.15.28:0/2073229763 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fcab415a650 0x7fcab41581c0 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.014+0000 7fcabc2a0700 1 -- 172.21.15.28:0/2073229763 >> 172.21.15.28:0/2073229763 conn(0x7fcab40b1650 msgr2=0x7fcab4156830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:11.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.014+0000 7fcabc2a0700 1 -- 172.21.15.28:0/2073229763 shutdown_connections 2023-11-28T11:13:11.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.014+0000 7fcabc2a0700 1 -- 172.21.15.28:0/2073229763 wait complete. 2023-11-28T11:13:11.027 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:12:01.724396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:12:02.776612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:13:11.028 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:13:11.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.154+0000 7fe2855b5700 1 Processor -- start 2023-11-28T11:13:11.155 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.154+0000 7fe2855b5700 1 -- start start 2023-11-28T11:13:11.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.155+0000 7fe2855b5700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe2800b62d0 0x7fe2800cb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.155+0000 7fe2855b5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe2800b2d90 0x7fe2800ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.155+0000 7fe2855b5700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2800cd2a0 con 0x7fe28006f8b0 2023-11-28T11:13:11.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.155+0000 7fe2855b5700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2800d0400 con 0x7fe2800b62d0 2023-11-28T11:13:11.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.155+0000 7fe2855b5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2800d06e0 con 0x7fe2800b2d90 2023-11-28T11:13:11.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.155+0000 7fe27effd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe2800b62d0 0x7fe2800cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.155+0000 7fe27effd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fe2800b62d0 0x7fe2800cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52116/0 (socket says 172.21.15.28:52116) 2023-11-28T11:13:11.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.155+0000 7fe27effd700 1 -- 172.21.15.28:0/2898225303 learned_addr learned my addr 172.21.15.28:0/2898225303 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:11.158 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.156+0000 7fe27f7fe700 1 --2- 172.21.15.28:0/2898225303 >> v2:172.21.15.59:3300/0 conn(0x7fe2800b2d90 0x7fe2800ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.156+0000 7fe27effd700 1 -- 172.21.15.28:0/2898225303 >> v2:172.21.15.59:3300/0 conn(0x7fe2800b2d90 msgr2=0x7fe2800ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:11.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.156+0000 7fe27effd700 1 --2- 172.21.15.28:0/2898225303 >> v2:172.21.15.59:3300/0 conn(0x7fe2800b2d90 0x7fe2800ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.156+0000 7fe27effd700 1 -- 172.21.15.28:0/2898225303 >> v1:172.21.15.59:6789/0 conn(0x7fe28006f8b0 legacy=0x7fe2800b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:11.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.156+0000 7fe27effd700 1 -- 172.21.15.28:0/2898225303 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2800cffb0 con 0x7fe2800b62d0 2023-11-28T11:13:11.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.156+0000 7fe27effd700 1 --2- 172.21.15.28:0/2898225303 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2800b62d0 0x7fe2800cb630 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fe28015dee0 tx=0x7fe274009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=caf8b0136e0040d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:11.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.156+0000 7fe27dffb700 1 -- 172.21.15.28:0/2898225303 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe274004230 con 0x7fe2800b62d0 2023-11-28T11:13:11.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.156+0000 7fe27dffb700 1 -- 172.21.15.28:0/2898225303 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe274004390 con 0x7fe2800b62d0 2023-11-28T11:13:11.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.156+0000 7fe27dffb700 1 -- 172.21.15.28:0/2898225303 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2740036a0 con 0x7fe2800b62d0 2023-11-28T11:13:11.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.156+0000 7fe2855b5700 1 -- 172.21.15.28:0/2898225303 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2800b62d0 msgr2=0x7fe2800cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.156+0000 7fe2855b5700 1 --2- 172.21.15.28:0/2898225303 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2800b62d0 0x7fe2800cb630 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fe28015dee0 tx=0x7fe274009bd0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.157+0000 7fe2855b5700 1 -- 172.21.15.28:0/2898225303 shutdown_connections 2023-11-28T11:13:11.159 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.157+0000 7fe2855b5700 1 --2- 172.21.15.28:0/2898225303 >> v2:172.21.15.59:3300/0 conn(0x7fe2800b2d90 0x7fe2800ce7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.160 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.157+0000 7fe2855b5700 1 --2- 172.21.15.28:0/2898225303 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2800b62d0 0x7fe2800cb630 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.160 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.157+0000 7fe2855b5700 1 -- 172.21.15.28:0/2898225303 >> 172.21.15.28:0/2898225303 conn(0x7fe2800cd680 msgr2=0x7fe2800cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:11.160 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.157+0000 7fe2855b5700 1 -- 172.21.15.28:0/2898225303 shutdown_connections 2023-11-28T11:13:11.160 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.157+0000 7fe2855b5700 1 -- 172.21.15.28:0/2898225303 wait complete. 2023-11-28T11:13:11.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.158+0000 7fe2855b5700 1 Processor -- start 2023-11-28T11:13:11.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.158+0000 7fe2855b5700 1 -- start start 2023-11-28T11:13:11.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe2855b5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe28006f8b0 0x7fe2800c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe2855b5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2800b2d90 0x7fe2800c7120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe2855b5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe2800b62d0 0x7fe2801e6050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe2855b5700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2801e4d60 con 0x7fe2800b2d90 2023-11-28T11:13:11.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe2855b5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2801e5bc0 con 0x7fe28006f8b0 2023-11-28T11:13:11.163 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe2855b5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2801e4fb0 con 0x7fe2800b62d0 2023-11-28T11:13:11.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe27e7fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2800b2d90 0x7fe2800c7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe27e7fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2800b2d90 0x7fe2800c7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52124/0 (socket says 172.21.15.28:52124) 2023-11-28T11:13:11.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe27e7fc700 1 -- 172.21.15.28:0/716144717 learned_addr learned my addr 172.21.15.28:0/716144717 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:11.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe27effd700 1 --2- 172.21.15.28:0/716144717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe28006f8b0 0x7fe2800c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe27f7fe700 1 --2- 172.21.15.28:0/716144717 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe2800b62d0 0x7fe2801e6050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe27e7fc700 1 -- 172.21.15.28:0/716144717 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe2800b62d0 msgr2=0x7fe2801e6050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe27e7fc700 1 --2- 172.21.15.28:0/716144717 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe2800b62d0 0x7fe2801e6050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe27e7fc700 1 -- 172.21.15.28:0/716144717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe28006f8b0 msgr2=0x7fe2800c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.164 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe27e7fc700 1 --2- 172.21.15.28:0/716144717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe28006f8b0 0x7fe2800c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe27e7fc700 1 -- 172.21.15.28:0/716144717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe274009550 con 0x7fe2800b2d90 2023-11-28T11:13:11.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe27e7fc700 1 --2- 172.21.15.28:0/716144717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2800b2d90 0x7fe2800c7120 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe27000b4a0 tx=0x7fe27000b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:11.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe2677fe700 1 -- 172.21.15.28:0/716144717 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270011830 con 0x7fe2800b2d90 2023-11-28T11:13:11.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.159+0000 7fe2677fe700 1 -- 172.21.15.28:0/716144717 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe270011990 con 0x7fe2800b2d90 2023-11-28T11:13:11.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.160+0000 7fe2855b5700 1 -- 172.21.15.28:0/716144717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2800b66a0 con 0x7fe2800b2d90 2023-11-28T11:13:11.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.160+0000 7fe2677fe700 1 -- 172.21.15.28:0/716144717 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe270011d40 con 0x7fe2800b2d90 2023-11-28T11:13:11.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.160+0000 7fe2855b5700 1 -- 172.21.15.28:0/716144717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe28006fd30 con 0x7fe2800b2d90 2023-11-28T11:13:11.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.161+0000 7fe2855b5700 1 -- 172.21.15.28:0/716144717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2800cce50 con 0x7fe2800b2d90 2023-11-28T11:13:11.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.161+0000 7fe2677fe700 1 -- 172.21.15.28:0/716144717 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe27000f050 con 0x7fe2800b2d90 2023-11-28T11:13:11.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.161+0000 7fe2677fe700 1 --2- 172.21.15.28:0/716144717 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe26c0682f0 0x7fe26c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.161+0000 7fe2677fe700 1 -- 172.21.15.28:0/716144717 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fe270085a00 con 0x7fe2800b2d90 2023-11-28T11:13:11.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.161+0000 7fe27effd700 1 --2- 172.21.15.28:0/716144717 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe26c0682f0 0x7fe26c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.162+0000 7fe27effd700 1 --2- 172.21.15.28:0/716144717 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe26c0682f0 0x7fe26c06a7a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe274000c00 tx=0x7fe274008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:11.166 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.165+0000 7fe2677fe700 1 -- 172.21.15.28:0/716144717 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe2700520e0 con 0x7fe2800b2d90 2023-11-28T11:13:11.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.357+0000 7fe2855b5700 1 -- 172.21.15.28:0/716144717 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe2800739f0 con 0x7fe2800b2d90 2023-11-28T11:13:11.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.358+0000 7fe2677fe700 1 -- 172.21.15.28:0/716144717 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v68) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe270015020 con 0x7fe2800b2d90 2023-11-28T11:13:11.358 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:13:11.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.359+0000 7fe2855b5700 1 -- 172.21.15.28:0/716144717 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe26c0682f0 msgr2=0x7fe26c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.359+0000 7fe2855b5700 1 --2- 172.21.15.28:0/716144717 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe26c0682f0 0x7fe26c06a7a0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe274000c00 tx=0x7fe274008030 comp rx=0 tx=0).stop 2023-11-28T11:13:11.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.359+0000 7fe2855b5700 1 -- 172.21.15.28:0/716144717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2800b2d90 msgr2=0x7fe2800c7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.359+0000 7fe2855b5700 1 --2- 172.21.15.28:0/716144717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2800b2d90 0x7fe2800c7120 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fe27000b4a0 tx=0x7fe27000b860 comp rx=0 tx=0).stop 2023-11-28T11:13:11.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.359+0000 7fe2855b5700 1 -- 172.21.15.28:0/716144717 shutdown_connections 2023-11-28T11:13:11.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.359+0000 7fe2855b5700 1 --2- 172.21.15.28:0/716144717 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe26c0682f0 0x7fe26c06a7a0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.359+0000 7fe2855b5700 1 --2- 172.21.15.28:0/716144717 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe2800b62d0 0x7fe2801e6050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.359+0000 7fe2855b5700 1 --2- 172.21.15.28:0/716144717 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe28006f8b0 0x7fe2800c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.359+0000 7fe2855b5700 1 --2- 172.21.15.28:0/716144717 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe2800b2d90 0x7fe2800c7120 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.359+0000 7fe2855b5700 1 -- 172.21.15.28:0/716144717 >> 172.21.15.28:0/716144717 conn(0x7fe2800cd680 msgr2=0x7fe2800b5340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:11.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.360+0000 7fe2855b5700 1 -- 172.21.15.28:0/716144717 shutdown_connections 2023-11-28T11:13:11.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.360+0000 7fe2855b5700 1 -- 172.21.15.28:0/716144717 wait complete. 2023-11-28T11:13:11.372 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:13:11.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.499+0000 7fb125cce700 1 Processor -- start 2023-11-28T11:13:11.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.499+0000 7fb125cce700 1 -- start start 2023-11-28T11:13:11.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb125cce700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1200cf5f0 0x7fb1200caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb125cce700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fb1200cb3e0 0x7fb12006afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb125cce700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1200ce550 con 0x7fb1200b1630 2023-11-28T11:13:11.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb125cce700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1200cfa60 con 0x7fb1200cb3e0 2023-11-28T11:13:11.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb125cce700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1200ce830 con 0x7fb1200cf5f0 2023-11-28T11:13:11.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11f7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb1200b1630 0x7fb1200b1a00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40188/0 (socket says 172.21.15.28:40188) 2023-11-28T11:13:11.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11f7fe700 1 -- 172.21.15.28:0/243368759 learned_addr learned my addr 172.21.15.28:0/243368759 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:11.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 -- 172.21.15.28:0/243368759 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3612343257 0 0) 0x7fb1200ce550 con 0x7fb1200b1630 2023-11-28T11:13:11.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 -- 172.21.15.28:0/243368759 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1080035b0 con 0x7fb1200b1630 2023-11-28T11:13:11.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11ffff700 1 --2- 172.21.15.28:0/243368759 >> v2:172.21.15.59:3300/0 conn(0x7fb1200cb3e0 0x7fb12006afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11effd700 1 --2- 172.21.15.28:0/243368759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1200cf5f0 0x7fb1200caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 -- 172.21.15.28:0/243368759 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1802490864 0 0) 0x7fb1080035b0 con 0x7fb1200b1630 2023-11-28T11:13:11.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 -- 172.21.15.28:0/243368759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1200cf5f0 msgr2=0x7fb1200caea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:11.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 --2- 172.21.15.28:0/243368759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1200cf5f0 0x7fb1200caea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 -- 172.21.15.28:0/243368759 >> v2:172.21.15.59:3300/0 conn(0x7fb1200cb3e0 msgr2=0x7fb12006afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:11.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 --2- 172.21.15.28:0/243368759 >> v2:172.21.15.59:3300/0 conn(0x7fb1200cb3e0 0x7fb12006afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 -- 172.21.15.28:0/243368759 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1200ceb10 con 0x7fb1200b1630 2023-11-28T11:13:11.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 -- 172.21.15.28:0/243368759 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fb114001670 con 0x7fb1200b1630 2023-11-28T11:13:11.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 -- 172.21.15.28:0/243368759 >> v1:172.21.15.28:6789/0 conn(0x7fb1200b1630 legacy=0x7fb1200b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 --2- 172.21.15.28:0/243368759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb108003a00 0x7fb108003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 --2- 172.21.15.28:0/243368759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb108004430 0x7fb1080048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 --2- 172.21.15.28:0/243368759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1200cf5f0 0x7fb108004fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 -- 172.21.15.28:0/243368759 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1200ce830 con 0x7fb108003a00 2023-11-28T11:13:11.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 -- 172.21.15.28:0/243368759 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10800ba10 con 0x7fb108004430 2023-11-28T11:13:11.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.500+0000 7fb11e7fc700 1 -- 172.21.15.28:0/243368759 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10800bb50 con 0x7fb1200cf5f0 2023-11-28T11:13:11.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.501+0000 7fb11f7fe700 1 --2- 172.21.15.28:0/243368759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb108003a00 0x7fb108003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.501+0000 7fb11effd700 1 --2- 172.21.15.28:0/243368759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb108004430 0x7fb1080048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.501+0000 7fb11ffff700 1 --2- 172.21.15.28:0/243368759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1200cf5f0 0x7fb108004fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.501+0000 7fb11f7fe700 1 -- 172.21.15.28:0/243368759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1200cf5f0 msgr2=0x7fb108004fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:11.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.501+0000 7fb11f7fe700 1 --2- 172.21.15.28:0/243368759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb1200cf5f0 0x7fb108004fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.501+0000 7fb11f7fe700 1 -- 172.21.15.28:0/243368759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb108004430 msgr2=0x7fb1080048f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:11.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.501+0000 7fb11f7fe700 1 --2- 172.21.15.28:0/243368759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb108004430 0x7fb1080048f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.501+0000 7fb11f7fe700 1 -- 172.21.15.28:0/243368759 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1080035b0 con 0x7fb108003a00 2023-11-28T11:13:11.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.501+0000 7fb11f7fe700 1 --2- 172.21.15.28:0/243368759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb108003a00 0x7fb108003e10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c320 tx=0x7fb108005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da816d7f647555fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:11.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.501+0000 7fb11e7fc700 1 -- 172.21.15.28:0/243368759 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11400ce60 con 0x7fb108003a00 2023-11-28T11:13:11.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.501+0000 7fb11e7fc700 1 -- 172.21.15.28:0/243368759 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb114005960 con 0x7fb108003a00 2023-11-28T11:13:11.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.501+0000 7fb11e7fc700 1 -- 172.21.15.28:0/243368759 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114005cf0 con 0x7fb108003a00 2023-11-28T11:13:11.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.502+0000 7fb125cce700 1 -- 172.21.15.28:0/243368759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb108003a00 msgr2=0x7fb108003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.502+0000 7fb125cce700 1 --2- 172.21.15.28:0/243368759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb108003a00 0x7fb108003e10 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb11400c320 tx=0x7fb108005880 comp rx=0 tx=0).stop 2023-11-28T11:13:11.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.502+0000 7fb11ffff700 1 -- 172.21.15.28:0/243368759 reap_dead start 2023-11-28T11:13:11.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.502+0000 7fb125cce700 1 -- 172.21.15.28:0/243368759 shutdown_connections 2023-11-28T11:13:11.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.502+0000 7fb125cce700 1 -- 172.21.15.28:0/243368759 >> 172.21.15.28:0/243368759 conn(0x7fb1200cc630 msgr2=0x7fb1200cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:11.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.502+0000 7fb125cce700 1 -- 172.21.15.28:0/243368759 shutdown_connections 2023-11-28T11:13:11.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.502+0000 7fb125cce700 1 -- 172.21.15.28:0/243368759 wait complete. 2023-11-28T11:13:11.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.503+0000 7fb125cce700 1 Processor -- start 2023-11-28T11:13:11.508 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.503+0000 7fb125cce700 1 -- start start 2023-11-28T11:13:11.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.504+0000 7fb125cce700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb108003a00 0x7fb12015dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.504+0000 7fb125cce700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb12014dfa0 0x7fb12015f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.504+0000 7fb125cce700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1200b5140 0x7fb1200b55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.504+0000 7fb125cce700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1200b3e90 con 0x7fb108003a00 2023-11-28T11:13:11.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.504+0000 7fb125cce700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb12014e410 con 0x7fb1200b5140 2023-11-28T11:13:11.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.504+0000 7fb125cce700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb12015fc80 con 0x7fb12014dfa0 2023-11-28T11:13:11.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.504+0000 7fb11f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb108003a00 0x7fb12015dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.504+0000 7fb11f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb108003a00 0x7fb12015dee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52142/0 (socket says 172.21.15.28:52142) 2023-11-28T11:13:11.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.504+0000 7fb11f7fe700 1 -- 172.21.15.28:0/3819028630 learned_addr learned my addr 172.21.15.28:0/3819028630 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:11.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.504+0000 7fb11f7fe700 1 -- 172.21.15.28:0/3819028630 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb12014dfa0 msgr2=0x7fb12015f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.504+0000 7fb11f7fe700 1 --2- 172.21.15.28:0/3819028630 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb12014dfa0 0x7fb12015f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.504+0000 7fb11f7fe700 1 -- 172.21.15.28:0/3819028630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1200b5140 msgr2=0x7fb1200b55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.504+0000 7fb11f7fe700 1 --2- 172.21.15.28:0/3819028630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1200b5140 0x7fb1200b55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.504+0000 7fb11f7fe700 1 -- 172.21.15.28:0/3819028630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1200cf0d0 con 0x7fb108003a00 2023-11-28T11:13:11.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.504+0000 7fb11f7fe700 1 --2- 172.21.15.28:0/3819028630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb108003a00 0x7fb12015dee0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb114000c00 tx=0x7fb114001a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:11.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.505+0000 7fb107fff700 1 -- 172.21.15.28:0/3819028630 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb114004a80 con 0x7fb108003a00 2023-11-28T11:13:11.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.505+0000 7fb107fff700 1 -- 172.21.15.28:0/3819028630 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb114004be0 con 0x7fb108003a00 2023-11-28T11:13:11.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.505+0000 7fb107fff700 1 -- 172.21.15.28:0/3819028630 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1140070d0 con 0x7fb108003a00 2023-11-28T11:13:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.505+0000 7fb125cce700 1 -- 172.21.15.28:0/3819028630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1200cedf0 con 0x7fb108003a00 2023-11-28T11:13:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.505+0000 7fb125cce700 1 -- 172.21.15.28:0/3819028630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1200ce140 con 0x7fb108003a00 2023-11-28T11:13:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.506+0000 7fb125cce700 1 -- 172.21.15.28:0/3819028630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1200b5af0 con 0x7fb108003a00 2023-11-28T11:13:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.506+0000 7fb107fff700 1 -- 172.21.15.28:0/3819028630 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb114024030 con 0x7fb108003a00 2023-11-28T11:13:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.506+0000 7fb107fff700 1 --2- 172.21.15.28:0/3819028630 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb10c0682f0 0x7fb10c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.506+0000 7fb107fff700 1 -- 172.21.15.28:0/3819028630 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fb114087230 con 0x7fb108003a00 2023-11-28T11:13:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.506+0000 7fb11effd700 1 --2- 172.21.15.28:0/3819028630 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb10c0682f0 0x7fb10c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.507+0000 7fb11effd700 1 --2- 172.21.15.28:0/3819028630 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb10c0682f0 0x7fb10c06a7a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb1200aea30 tx=0x7fb1100093c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:11.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.511+0000 7fb107fff700 1 -- 172.21.15.28:0/3819028630 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb1140539d0 con 0x7fb108003a00 2023-11-28T11:13:11.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.705+0000 7fb125cce700 1 -- 172.21.15.28:0/3819028630 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb1200739f0 con 0x7fb108003a00 2023-11-28T11:13:11.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.705+0000 7fb107fff700 1 -- 172.21.15.28:0/3819028630 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb114007da0 con 0x7fb108003a00 2023-11-28T11:13:11.706 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:13:11.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.707+0000 7fb125cce700 1 -- 172.21.15.28:0/3819028630 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb10c0682f0 msgr2=0x7fb10c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.707+0000 7fb125cce700 1 --2- 172.21.15.28:0/3819028630 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb10c0682f0 0x7fb10c06a7a0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fb1200aea30 tx=0x7fb1100093c0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.707+0000 7fb125cce700 1 -- 172.21.15.28:0/3819028630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb108003a00 msgr2=0x7fb12015dee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.707+0000 7fb125cce700 1 --2- 172.21.15.28:0/3819028630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb108003a00 0x7fb12015dee0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb114000c00 tx=0x7fb114001a00 comp rx=0 tx=0).stop 2023-11-28T11:13:11.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.707+0000 7fb125cce700 1 -- 172.21.15.28:0/3819028630 shutdown_connections 2023-11-28T11:13:11.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.707+0000 7fb125cce700 1 --2- 172.21.15.28:0/3819028630 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb10c0682f0 0x7fb10c06a7a0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.707+0000 7fb125cce700 1 --2- 172.21.15.28:0/3819028630 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb108003a00 0x7fb12015dee0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.707+0000 7fb125cce700 1 --2- 172.21.15.28:0/3819028630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1200b5140 0x7fb1200b55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.707+0000 7fb125cce700 1 --2- 172.21.15.28:0/3819028630 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb12014dfa0 0x7fb12015f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.707+0000 7fb125cce700 1 -- 172.21.15.28:0/3819028630 >> 172.21.15.28:0/3819028630 conn(0x7fb1200cc630 msgr2=0x7fb1200cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:11.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.707+0000 7fb125cce700 1 -- 172.21.15.28:0/3819028630 shutdown_connections 2023-11-28T11:13:11.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.707+0000 7fb125cce700 1 -- 172.21.15.28:0/3819028630 wait complete. 2023-11-28T11:13:11.719 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:13:11.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.846+0000 7f3cf2759700 1 Processor -- start 2023-11-28T11:13:11.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.846+0000 7f3cf2759700 1 -- start start 2023-11-28T11:13:11.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3cf2759700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f3cec159500 0x7f3cec0cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.850 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3cf2759700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cec15e040 con 0x7f3cec15dbf0 2023-11-28T11:13:11.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3cf2759700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cec0ceff0 con 0x7f3cec0b1d40 2023-11-28T11:13:11.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3cf2759700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cec0cf2d0 con 0x7f3cec159500 2023-11-28T11:13:11.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3ceb7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f3cec15dbf0 0x7f3cec0b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40210/0 (socket says 172.21.15.28:40210) 2023-11-28T11:13:11.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3ceb7fe700 1 -- 172.21.15.28:0/2897431117 learned_addr learned my addr 172.21.15.28:0/2897431117 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:11.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3ceaffd700 1 -- 172.21.15.28:0/2897431117 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 269804947 0 0) 0x7f3cec15e040 con 0x7f3cec15dbf0 2023-11-28T11:13:11.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3ceaffd700 1 -- 172.21.15.28:0/2897431117 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cd80035b0 con 0x7f3cec15dbf0 2023-11-28T11:13:11.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3cebfff700 1 --2- 172.21.15.28:0/2897431117 >> v2:172.21.15.59:3300/0 conn(0x7f3cec159500 0x7f3cec0cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.851 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3ceaffd700 1 -- 172.21.15.28:0/2897431117 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 266581323 0 0) 0x7f3cd80035b0 con 0x7f3cec15dbf0 2023-11-28T11:13:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3ceaffd700 1 -- 172.21.15.28:0/2897431117 >> v2:172.21.15.59:3300/0 conn(0x7f3cec159500 msgr2=0x7f3cec0cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3ceaffd700 1 --2- 172.21.15.28:0/2897431117 >> v2:172.21.15.59:3300/0 conn(0x7f3cec159500 0x7f3cec0cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3ceaffd700 1 -- 172.21.15.28:0/2897431117 >> v1:172.21.15.59:6789/0 conn(0x7f3cec0b1d40 legacy=0x7f3cec06bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3ceaffd700 1 -- 172.21.15.28:0/2897431117 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec15d110 con 0x7f3cec15dbf0 2023-11-28T11:13:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3ceaffd700 1 -- 172.21.15.28:0/2897431117 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f3cd4001670 con 0x7f3cec15dbf0 2023-11-28T11:13:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.847+0000 7f3ceaffd700 1 -- 172.21.15.28:0/2897431117 >> v1:172.21.15.28:6789/0 conn(0x7f3cec15dbf0 legacy=0x7f3cec0b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3ceaffd700 1 --2- 172.21.15.28:0/2897431117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cd8003a00 0x7f3cd8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3ceaffd700 1 --2- 172.21.15.28:0/2897431117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd8004850 0x7f3cd8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3ceaffd700 1 --2- 172.21.15.28:0/2897431117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd8005200 0x7f3cd800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3ceaffd700 1 -- 172.21.15.28:0/2897431117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec0cf2d0 con 0x7f3cd8003a00 2023-11-28T11:13:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3ceaffd700 1 -- 172.21.15.28:0/2897431117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd800c060 con 0x7f3cd8005200 2023-11-28T11:13:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3ceaffd700 1 -- 172.21.15.28:0/2897431117 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cd800c220 con 0x7f3cd8004850 2023-11-28T11:13:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3cebfff700 1 --2- 172.21.15.28:0/2897431117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cd8003a00 0x7f3cd8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3cf0cf6700 1 --2- 172.21.15.28:0/2897431117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd8005200 0x7f3cd800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3ceb7fe700 1 --2- 172.21.15.28:0/2897431117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd8004850 0x7f3cd8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3cebfff700 1 -- 172.21.15.28:0/2897431117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd8004850 msgr2=0x7f3cd8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3cebfff700 1 --2- 172.21.15.28:0/2897431117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd8004850 0x7f3cd8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3cebfff700 1 -- 172.21.15.28:0/2897431117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd8005200 msgr2=0x7f3cd800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3cebfff700 1 --2- 172.21.15.28:0/2897431117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd8005200 0x7f3cd800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3cebfff700 1 -- 172.21.15.28:0/2897431117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cd80035b0 con 0x7f3cd8003a00 2023-11-28T11:13:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3cf0cf6700 1 --2- 172.21.15.28:0/2897431117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cd8005200 0x7f3cd800ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:11.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3cf0cf6700 1 -- 172.21.15.28:0/2897431117 reap_dead start 2023-11-28T11:13:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3cebfff700 1 --2- 172.21.15.28:0/2897431117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cd8003a00 0x7f3cd8003e10 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009fc0 tx=0x7f3cd8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d6218ff112b68c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3ceaffd700 1 -- 172.21.15.28:0/2897431117 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc00ec30 con 0x7f3cd8003a00 2023-11-28T11:13:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3ceaffd700 1 -- 172.21.15.28:0/2897431117 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cdc00f040 con 0x7f3cd8003a00 2023-11-28T11:13:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.848+0000 7f3ceaffd700 1 -- 172.21.15.28:0/2897431117 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc014480 con 0x7f3cd8003a00 2023-11-28T11:13:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.849+0000 7f3cf2759700 1 -- 172.21.15.28:0/2897431117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cd8003a00 msgr2=0x7f3cd8003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.849+0000 7f3cf2759700 1 --2- 172.21.15.28:0/2897431117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cd8003a00 0x7f3cd8003e10 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009fc0 tx=0x7f3cd8005880 comp rx=0 tx=0).stop 2023-11-28T11:13:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.849+0000 7f3cf2759700 1 -- 172.21.15.28:0/2897431117 shutdown_connections 2023-11-28T11:13:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.849+0000 7f3cf2759700 1 --2- 172.21.15.28:0/2897431117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cd8003a00 0x7f3cd8003e10 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.849+0000 7f3cf2759700 1 -- 172.21.15.28:0/2897431117 >> 172.21.15.28:0/2897431117 conn(0x7f3cec0cc630 msgr2=0x7f3cec0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.849+0000 7f3cf2759700 1 -- 172.21.15.28:0/2897431117 shutdown_connections 2023-11-28T11:13:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.849+0000 7f3cf2759700 1 -- 172.21.15.28:0/2897431117 wait complete. 2023-11-28T11:13:11.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.850+0000 7f3cf2759700 1 Processor -- start 2023-11-28T11:13:11.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.850+0000 7f3cf2759700 1 -- start start 2023-11-28T11:13:11.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3cf2759700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd8003a00 0x7f3cec1476e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3cf2759700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cec14ac70 0x7f3cec0b48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3cf2759700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cec0b4e30 0x7f3cec0b29e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3cf2759700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec147f10 con 0x7f3cec14ac70 2023-11-28T11:13:11.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3cf2759700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec147c20 con 0x7f3cec0b4e30 2023-11-28T11:13:11.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3cf2759700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cec0c45f0 con 0x7f3cd8003a00 2023-11-28T11:13:11.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3ceb7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cec14ac70 0x7f3cec0b48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3ceb7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cec14ac70 0x7f3cec0b48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52164/0 (socket says 172.21.15.28:52164) 2023-11-28T11:13:11.857 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3ceb7fe700 1 -- 172.21.15.28:0/354481012 learned_addr learned my addr 172.21.15.28:0/354481012 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3cebfff700 1 --2- 172.21.15.28:0/354481012 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd8003a00 0x7f3cec1476e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3cf0cf6700 1 --2- 172.21.15.28:0/354481012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cec0b4e30 0x7f3cec0b29e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3ceb7fe700 1 -- 172.21.15.28:0/354481012 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd8003a00 msgr2=0x7f3cec1476e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3ceb7fe700 1 --2- 172.21.15.28:0/354481012 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd8003a00 0x7f3cec1476e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3ceb7fe700 1 -- 172.21.15.28:0/354481012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cec0b4e30 msgr2=0x7f3cec0b29e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3ceb7fe700 1 --2- 172.21.15.28:0/354481012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cec0b4e30 0x7f3cec0b29e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3ceb7fe700 1 -- 172.21.15.28:0/354481012 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec15d6d0 con 0x7f3cec14ac70 2023-11-28T11:13:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.851+0000 7f3cf0cf6700 1 --2- 172.21.15.28:0/354481012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cec0b4e30 0x7f3cec0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.852+0000 7f3ceb7fe700 1 --2- 172.21.15.28:0/354481012 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cec14ac70 0x7f3cec0b48f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400eaa0 tx=0x7f3cd400ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.852+0000 7f3ce8ff9700 1 -- 172.21.15.28:0/354481012 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd4005e50 con 0x7f3cec14ac70 2023-11-28T11:13:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.852+0000 7f3ce8ff9700 1 -- 172.21.15.28:0/354481012 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cd400c760 con 0x7f3cec14ac70 2023-11-28T11:13:11.858 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.852+0000 7f3ce8ff9700 1 -- 172.21.15.28:0/354481012 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cd400cb10 con 0x7f3cec14ac70 2023-11-28T11:13:11.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.852+0000 7f3cf2759700 1 -- 172.21.15.28:0/354481012 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cec15d3f0 con 0x7f3cec14ac70 2023-11-28T11:13:11.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.852+0000 7f3cf2759700 1 -- 172.21.15.28:0/354481012 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cec1598d0 con 0x7f3cec14ac70 2023-11-28T11:13:11.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.853+0000 7f3cf2759700 1 -- 172.21.15.28:0/354481012 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cec0739f0 con 0x7f3cec14ac70 2023-11-28T11:13:11.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.853+0000 7f3ce8ff9700 1 -- 172.21.15.28:0/354481012 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f3cd400cc70 con 0x7f3cec14ac70 2023-11-28T11:13:11.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.854+0000 7f3ce8ff9700 1 --2- 172.21.15.28:0/354481012 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ccc0682f0 0x7f3ccc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:11.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.854+0000 7f3ce8ff9700 1 -- 172.21.15.28:0/354481012 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f3cd4085db0 con 0x7f3cec14ac70 2023-11-28T11:13:11.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.854+0000 7f3cebfff700 1 --2- 172.21.15.28:0/354481012 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ccc0682f0 0x7f3ccc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:11.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.854+0000 7f3cebfff700 1 --2- 172.21.15.28:0/354481012 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ccc0682f0 0x7f3ccc06a7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc000f80 tx=0x7f3cdc013000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:11.859 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:11.858+0000 7f3ce8ff9700 1 -- 172.21.15.28:0/354481012 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f3cd4052490 con 0x7f3cec14ac70 2023-11-28T11:13:12.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.050+0000 7f3cf2759700 1 -- 172.21.15.28:0/354481012 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f3cec0ca070 con 0x7f3cec14ac70 2023-11-28T11:13:12.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.050+0000 7f3ce8ff9700 1 -- 172.21.15.28:0/354481012 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3cd4055ab0 con 0x7f3cec14ac70 2023-11-28T11:13:12.051 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:13:12.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.052+0000 7f3cf2759700 1 -- 172.21.15.28:0/354481012 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ccc0682f0 msgr2=0x7f3ccc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:12.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.052+0000 7f3cf2759700 1 --2- 172.21.15.28:0/354481012 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ccc0682f0 0x7f3ccc06a7a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc000f80 tx=0x7f3cdc013000 comp rx=0 tx=0).stop 2023-11-28T11:13:12.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.052+0000 7f3cf2759700 1 -- 172.21.15.28:0/354481012 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cec14ac70 msgr2=0x7f3cec0b48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:12.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.052+0000 7f3cf2759700 1 --2- 172.21.15.28:0/354481012 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cec14ac70 0x7f3cec0b48f0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f3cd400eaa0 tx=0x7f3cd400ee60 comp rx=0 tx=0).stop 2023-11-28T11:13:12.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.052+0000 7f3cf2759700 1 -- 172.21.15.28:0/354481012 shutdown_connections 2023-11-28T11:13:12.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.052+0000 7f3cf2759700 1 --2- 172.21.15.28:0/354481012 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f3ccc0682f0 0x7f3ccc06a7a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.052+0000 7f3cf2759700 1 --2- 172.21.15.28:0/354481012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3cec0b4e30 0x7f3cec0b29e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.052+0000 7f3cf2759700 1 --2- 172.21.15.28:0/354481012 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f3cd8003a00 0x7f3cec1476e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.052+0000 7f3cf2759700 1 --2- 172.21.15.28:0/354481012 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f3cec14ac70 0x7f3cec0b48f0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.053+0000 7f3cf2759700 1 -- 172.21.15.28:0/354481012 >> 172.21.15.28:0/354481012 conn(0x7f3cec0cc630 msgr2=0x7f3cec0af580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:12.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.053+0000 7f3cf2759700 1 -- 172.21.15.28:0/354481012 shutdown_connections 2023-11-28T11:13:12.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.053+0000 7f3cf2759700 1 -- 172.21.15.28:0/354481012 wait complete. 2023-11-28T11:13:12.065 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:13:12.193 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.193+0000 7f11eb5ac700 1 Processor -- start 2023-11-28T11:13:12.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.193+0000 7f11eb5ac700 1 -- start start 2023-11-28T11:13:12.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.193+0000 7f11eb5ac700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f11e40b2d90 0x7f11e40ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:12.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11eb5ac700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e40cd2a0 con 0x7f11e40cb630 2023-11-28T11:13:12.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11eb5ac700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11e40d0400 con 0x7f11e40b62d0 2023-11-28T11:13:12.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11eb5ac700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11e40d06e0 con 0x7f11e40b2d90 2023-11-28T11:13:12.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11e8b47700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f11e40cb630 0x7f11e40b2680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40230/0 (socket says 172.21.15.28:40230) 2023-11-28T11:13:12.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11e8b47700 1 -- 172.21.15.28:0/4132641952 learned_addr learned my addr 172.21.15.28:0/4132641952 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:12.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11d7fff700 1 -- 172.21.15.28:0/4132641952 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3053459878 0 0) 0x7f11e40cd2a0 con 0x7f11e40cb630 2023-11-28T11:13:12.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11e9b49700 1 --2- 172.21.15.28:0/4132641952 >> v2:172.21.15.59:3300/0 conn(0x7f11e40b2d90 0x7f11e40ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:12.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11d7fff700 1 -- 172.21.15.28:0/4132641952 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f11d00035b0 con 0x7f11e40cb630 2023-11-28T11:13:12.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11d7fff700 1 -- 172.21.15.28:0/4132641952 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 539250723 0 0) 0x7f11d00035b0 con 0x7f11e40cb630 2023-11-28T11:13:12.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11d7fff700 1 -- 172.21.15.28:0/4132641952 >> v2:172.21.15.59:3300/0 conn(0x7f11e40b2d90 msgr2=0x7f11e40ce7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:12.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11d7fff700 1 --2- 172.21.15.28:0/4132641952 >> v2:172.21.15.59:3300/0 conn(0x7f11e40b2d90 0x7f11e40ce7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11d7fff700 1 -- 172.21.15.28:0/4132641952 >> v1:172.21.15.59:6789/0 conn(0x7f11e40b62d0 legacy=0x7f11e406f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:12.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11d7fff700 1 -- 172.21.15.28:0/4132641952 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e40cffb0 con 0x7f11e40cb630 2023-11-28T11:13:12.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11d7fff700 1 -- 172.21.15.28:0/4132641952 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f11d8001670 con 0x7f11e40cb630 2023-11-28T11:13:12.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11d7fff700 1 -- 172.21.15.28:0/4132641952 >> v1:172.21.15.28:6789/0 conn(0x7f11e40cb630 legacy=0x7f11e40b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:12.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11d7fff700 1 --2- 172.21.15.28:0/4132641952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d0003a00 0x7f11d0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:12.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11d7fff700 1 --2- 172.21.15.28:0/4132641952 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11d0004850 0x7f11d0004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:12.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11d7fff700 1 --2- 172.21.15.28:0/4132641952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11d0005200 0x7f11d000ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:12.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11d7fff700 1 -- 172.21.15.28:0/4132641952 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40d06e0 con 0x7f11d0005200 2023-11-28T11:13:12.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11d7fff700 1 -- 172.21.15.28:0/4132641952 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11d000c080 con 0x7f11d0003a00 2023-11-28T11:13:12.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11d7fff700 1 -- 172.21.15.28:0/4132641952 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11d000c240 con 0x7f11d0004850 2023-11-28T11:13:12.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11e9b49700 1 --2- 172.21.15.28:0/4132641952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11d0005200 0x7f11d000ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:12.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11e9348700 1 --2- 172.21.15.28:0/4132641952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d0003a00 0x7f11d0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:12.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.194+0000 7f11e8b47700 1 --2- 172.21.15.28:0/4132641952 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11d0004850 0x7f11d0004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.195+0000 7f11e9b49700 1 -- 172.21.15.28:0/4132641952 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11d0004850 msgr2=0x7f11d0004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.195+0000 7f11e9b49700 1 --2- 172.21.15.28:0/4132641952 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11d0004850 0x7f11d0004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.195+0000 7f11e9b49700 1 -- 172.21.15.28:0/4132641952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d0003a00 msgr2=0x7f11d0003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.195+0000 7f11e9b49700 1 --2- 172.21.15.28:0/4132641952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d0003a00 0x7f11d0003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.195+0000 7f11e9b49700 1 -- 172.21.15.28:0/4132641952 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11d00035b0 con 0x7f11d0005200 2023-11-28T11:13:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.195+0000 7f11e8b47700 1 --2- 172.21.15.28:0/4132641952 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11d0004850 0x7f11d0004cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.195+0000 7f11e9b49700 1 -- 172.21.15.28:0/4132641952 reap_dead start 2023-11-28T11:13:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.195+0000 7f11e9b49700 1 --2- 172.21.15.28:0/4132641952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11d0005200 0x7f11d000ba10 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009f90 tx=0x7f11d0005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39c9fe2391705f56 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.195+0000 7f11d7fff700 1 -- 172.21.15.28:0/4132641952 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc00ec30 con 0x7f11d0005200 2023-11-28T11:13:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.195+0000 7f11d7fff700 1 -- 172.21.15.28:0/4132641952 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11dc00f040 con 0x7f11d0005200 2023-11-28T11:13:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.195+0000 7f11d7fff700 1 -- 172.21.15.28:0/4132641952 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc014480 con 0x7f11d0005200 2023-11-28T11:13:12.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.195+0000 7f11eb5ac700 1 -- 172.21.15.28:0/4132641952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11d0005200 msgr2=0x7f11d000ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.195+0000 7f11eb5ac700 1 --2- 172.21.15.28:0/4132641952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11d0005200 0x7f11d000ba10 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f11dc009f90 tx=0x7f11d0005880 comp rx=0 tx=0).stop 2023-11-28T11:13:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.196+0000 7f11eb5ac700 1 -- 172.21.15.28:0/4132641952 shutdown_connections 2023-11-28T11:13:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.196+0000 7f11eb5ac700 1 --2- 172.21.15.28:0/4132641952 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11d0005200 0x7f11d000ba10 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.196+0000 7f11eb5ac700 1 -- 172.21.15.28:0/4132641952 >> 172.21.15.28:0/4132641952 conn(0x7f11e40cd680 msgr2=0x7f11e40cda80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.196+0000 7f11eb5ac700 1 -- 172.21.15.28:0/4132641952 shutdown_connections 2023-11-28T11:13:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.196+0000 7f11eb5ac700 1 -- 172.21.15.28:0/4132641952 wait complete. 2023-11-28T11:13:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.197+0000 7f11eb5ac700 1 Processor -- start 2023-11-28T11:13:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.197+0000 7f11eb5ac700 1 -- start start 2023-11-28T11:13:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11eb5ac700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d0005200 0x7f11e4152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11eb5ac700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11e4152fb0 0x7f11e40c50c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:12.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11eb5ac700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11e40c6b60 0x7f11e40c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11eb5ac700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40c4d70 con 0x7f11e40c6b60 2023-11-28T11:13:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11eb5ac700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40c47b0 con 0x7f11d0005200 2023-11-28T11:13:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11eb5ac700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11e40c4a90 con 0x7f11e4152fb0 2023-11-28T11:13:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11e9b49700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11e40c6b60 0x7f11e40c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11e9b49700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11e40c6b60 0x7f11e40c6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52186/0 (socket says 172.21.15.28:52186) 2023-11-28T11:13:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11e9b49700 1 -- 172.21.15.28:0/2968622826 learned_addr learned my addr 172.21.15.28:0/2968622826 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11e8b47700 1 --2- 172.21.15.28:0/2968622826 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11e4152fb0 0x7f11e40c50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11e9348700 1 --2- 172.21.15.28:0/2968622826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d0005200 0x7f11e4152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11e9b49700 1 -- 172.21.15.28:0/2968622826 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11e4152fb0 msgr2=0x7f11e40c50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11e9b49700 1 --2- 172.21.15.28:0/2968622826 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11e4152fb0 0x7f11e40c50c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11e9b49700 1 -- 172.21.15.28:0/2968622826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d0005200 msgr2=0x7f11e4152790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:12.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11e9b49700 1 --2- 172.21.15.28:0/2968622826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d0005200 0x7f11e4152790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11e9b49700 1 -- 172.21.15.28:0/2968622826 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e40cce50 con 0x7f11e40c6b60 2023-11-28T11:13:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11e9b49700 1 --2- 172.21.15.28:0/2968622826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11e40c6b60 0x7f11e40c6fd0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f11e40d09c0 tx=0x7f11dc004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11d5ffb700 1 -- 172.21.15.28:0/2968622826 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc007000 con 0x7f11e40c6b60 2023-11-28T11:13:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11d5ffb700 1 -- 172.21.15.28:0/2968622826 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11dc003e30 con 0x7f11e40c6b60 2023-11-28T11:13:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11eb5ac700 1 -- 172.21.15.28:0/2968622826 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11e40b66a0 con 0x7f11e40c6b60 2023-11-28T11:13:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.198+0000 7f11d5ffb700 1 -- 172.21.15.28:0/2968622826 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11dc015be0 con 0x7f11e40c6b60 2023-11-28T11:13:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.199+0000 7f11eb5ac700 1 -- 172.21.15.28:0/2968622826 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11e40ccfe0 con 0x7f11e40c6b60 2023-11-28T11:13:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.200+0000 7f11eb5ac700 1 -- 172.21.15.28:0/2968622826 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11e40739f0 con 0x7f11e40c6b60 2023-11-28T11:13:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.200+0000 7f11d5ffb700 1 -- 172.21.15.28:0/2968622826 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f11dc015d40 con 0x7f11e40c6b60 2023-11-28T11:13:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.200+0000 7f11d5ffb700 1 --2- 172.21.15.28:0/2968622826 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f11c8068320 0x7f11c806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.200+0000 7f11d5ffb700 1 -- 172.21.15.28:0/2968622826 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f11dc087bc0 con 0x7f11e40c6b60 2023-11-28T11:13:12.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.201+0000 7f11e9348700 1 --2- 172.21.15.28:0/2968622826 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f11c8068320 0x7f11c806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:12.213 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.201+0000 7f11e9348700 1 --2- 172.21.15.28:0/2968622826 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f11c8068320 0x7f11c806a7d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f11e0000c00 tx=0x7f11e0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:12.213 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.205+0000 7f11d5ffb700 1 -- 172.21.15.28:0/2968622826 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f11dc0542a0 con 0x7f11e40c6b60 2023-11-28T11:13:12.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.399+0000 7f11eb5ac700 1 -- 172.21.15.28:0/2968622826 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f11e406aa40 con 0x7f11e40c6b60 2023-11-28T11:13:12.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.399+0000 7f11d5ffb700 1 -- 172.21.15.28:0/2968622826 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) v1 ==== 100+0+11 (secure 0 0 0) 0x7f11dc0180a0 con 0x7f11e40c6b60 2023-11-28T11:13:12.400 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:13:12.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.401+0000 7f11eb5ac700 1 -- 172.21.15.28:0/2968622826 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f11c8068320 msgr2=0x7f11c806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:12.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.401+0000 7f11eb5ac700 1 --2- 172.21.15.28:0/2968622826 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f11c8068320 0x7f11c806a7d0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f11e0000c00 tx=0x7f11e0009040 comp rx=0 tx=0).stop 2023-11-28T11:13:12.402 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.401+0000 7f11eb5ac700 1 -- 172.21.15.28:0/2968622826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11e40c6b60 msgr2=0x7f11e40c6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.401+0000 7f11eb5ac700 1 --2- 172.21.15.28:0/2968622826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11e40c6b60 0x7f11e40c6fd0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f11e40d09c0 tx=0x7f11dc004b40 comp rx=0 tx=0).stop 2023-11-28T11:13:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.401+0000 7f11eb5ac700 1 -- 172.21.15.28:0/2968622826 shutdown_connections 2023-11-28T11:13:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.401+0000 7f11eb5ac700 1 --2- 172.21.15.28:0/2968622826 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f11c8068320 0x7f11c806a7d0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.401+0000 7f11eb5ac700 1 --2- 172.21.15.28:0/2968622826 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f11e40c6b60 0x7f11e40c6fd0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.401+0000 7f11eb5ac700 1 --2- 172.21.15.28:0/2968622826 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f11d0005200 0x7f11e4152790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.401+0000 7f11eb5ac700 1 --2- 172.21.15.28:0/2968622826 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f11e4152fb0 0x7f11e40c50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.401+0000 7f11eb5ac700 1 -- 172.21.15.28:0/2968622826 >> 172.21.15.28:0/2968622826 conn(0x7f11e40cd680 msgr2=0x7f11e40ce640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.402+0000 7f11eb5ac700 1 -- 172.21.15.28:0/2968622826 shutdown_connections 2023-11-28T11:13:12.403 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.402+0000 7f11eb5ac700 1 -- 172.21.15.28:0/2968622826 wait complete. 2023-11-28T11:13:12.414 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-11-28T11:13:12.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.539+0000 7f5fee058700 1 Processor -- start 2023-11-28T11:13:12.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.540+0000 7f5fee058700 1 -- start start 2023-11-28T11:13:12.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.540+0000 7f5fee058700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f5fe815a230 0x7f5fe815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:12.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.540+0000 7f5fee058700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5fe80b4fe0 0x7f5fe80d0a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:12.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.540+0000 7f5fee058700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fe806abb0 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.540+0000 7f5fee058700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe815a600 con 0x7f5fe80b4fe0 2023-11-28T11:13:12.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.540+0000 7f5fee058700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe806a280 con 0x7f5fe815a230 2023-11-28T11:13:12.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.540+0000 7f5fe6ffd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f5fe80cd4a0 0x7f5fe80b48d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40250/0 (socket says 172.21.15.28:40250) 2023-11-28T11:13:12.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.541+0000 7f5fe7fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f5fe80b4fe0 0x7f5fe80d0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:12.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.541+0000 7f5fe6ffd700 1 -- 172.21.15.28:0/3242864385 learned_addr learned my addr 172.21.15.28:0/3242864385 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:12.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.541+0000 7f5fe77fe700 1 --2- 172.21.15.28:0/3242864385 >> v2:172.21.15.59:3300/0 conn(0x7f5fe815a230 0x7f5fe815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:12.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.541+0000 7f5fe67fc700 1 -- 172.21.15.28:0/3242864385 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270425901 0 0) 0x7f5fe806abb0 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.541+0000 7f5fe67fc700 1 -- 172.21.15.28:0/3242864385 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fd00035b0 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.541+0000 7f5fe7fff700 1 -- 172.21.15.28:0/3242864385 >> v2:172.21.15.59:3300/0 conn(0x7f5fe815a230 msgr2=0x7f5fe815d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:12.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.541+0000 7f5fe7fff700 1 --2- 172.21.15.28:0/3242864385 >> v2:172.21.15.59:3300/0 conn(0x7f5fe815a230 0x7f5fe815d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.541+0000 7f5fe7fff700 1 -- 172.21.15.28:0/3242864385 >> v1:172.21.15.28:6789/0 conn(0x7f5fe80cd4a0 legacy=0x7f5fe80b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:12.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.541+0000 7f5fe7fff700 1 -- 172.21.15.28:0/3242864385 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe806a560 con 0x7f5fe80b4fe0 2023-11-28T11:13:12.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.541+0000 7f5fe7fff700 1 --2- 172.21.15.28:0/3242864385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fe80b4fe0 0x7f5fe80d0a20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80cf830 tx=0x7f5fe000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd3e333408323eee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:12.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.541+0000 7f5fe67fc700 1 -- 172.21.15.28:0/3242864385 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe000e070 con 0x7f5fe80b4fe0 2023-11-28T11:13:12.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.541+0000 7f5fe67fc700 1 -- 172.21.15.28:0/3242864385 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fe000bb20 con 0x7f5fe80b4fe0 2023-11-28T11:13:12.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.541+0000 7f5fe67fc700 1 -- 172.21.15.28:0/3242864385 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fe0012430 con 0x7f5fe80b4fe0 2023-11-28T11:13:12.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.542+0000 7f5fee058700 1 -- 172.21.15.28:0/3242864385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fe80b4fe0 msgr2=0x7f5fe80d0a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:12.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.542+0000 7f5fee058700 1 --2- 172.21.15.28:0/3242864385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fe80b4fe0 0x7f5fe80d0a20 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80cf830 tx=0x7f5fe000b330 comp rx=0 tx=0).stop 2023-11-28T11:13:12.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.542+0000 7f5fee058700 1 -- 172.21.15.28:0/3242864385 shutdown_connections 2023-11-28T11:13:12.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.542+0000 7f5fee058700 1 --2- 172.21.15.28:0/3242864385 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fe80b4fe0 0x7f5fe80d0a20 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.542+0000 7f5fee058700 1 --2- 172.21.15.28:0/3242864385 >> v2:172.21.15.59:3300/0 conn(0x7f5fe815a230 0x7f5fe815d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.542+0000 7f5fee058700 1 -- 172.21.15.28:0/3242864385 >> 172.21.15.28:0/3242864385 conn(0x7f5fe80cf8d0 msgr2=0x7f5fe80cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:12.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.542+0000 7f5fee058700 1 -- 172.21.15.28:0/3242864385 shutdown_connections 2023-11-28T11:13:12.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.542+0000 7f5fee058700 1 -- 172.21.15.28:0/3242864385 wait complete. 2023-11-28T11:13:12.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.543+0000 7f5fee058700 1 Processor -- start 2023-11-28T11:13:12.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.543+0000 7f5fee058700 1 -- start start 2023-11-28T11:13:12.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.544+0000 7f5fee058700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe80b4fe0 0x7f5fe80b8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:12.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.544+0000 7f5fee058700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fe80cd4a0 0x7f5fe81528e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:12.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.544+0000 7f5fee058700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fe815a230 0x7f5fe81f29b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:12.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.544+0000 7f5fee058700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80c7170 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.544+0000 7f5fee058700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe81f2ef0 con 0x7f5fe80b4fe0 2023-11-28T11:13:12.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.544+0000 7f5fee058700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80b68f0 con 0x7f5fe815a230 2023-11-28T11:13:12.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.544+0000 7f5fe6ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fe80cd4a0 0x7f5fe81528e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:12.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.544+0000 7f5fe6ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fe80cd4a0 0x7f5fe81528e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52198/0 (socket says 172.21.15.28:52198) 2023-11-28T11:13:12.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.544+0000 7f5fe6ffd700 1 -- 172.21.15.28:0/3366597620 learned_addr learned my addr 172.21.15.28:0/3366597620 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:12.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.544+0000 7f5fe6ffd700 1 -- 172.21.15.28:0/3366597620 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fe815a230 msgr2=0x7f5fe81f29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:12.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.544+0000 7f5fe6ffd700 1 --2- 172.21.15.28:0/3366597620 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fe815a230 0x7f5fe81f29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.544+0000 7f5fe6ffd700 1 -- 172.21.15.28:0/3366597620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe80b4fe0 msgr2=0x7f5fe80b8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:12.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.544+0000 7f5fe6ffd700 1 --2- 172.21.15.28:0/3366597620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe80b4fe0 0x7f5fe80b8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.544+0000 7f5fe6ffd700 1 -- 172.21.15.28:0/3366597620 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe0009a70 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.544+0000 7f5fe6ffd700 1 --2- 172.21.15.28:0/3366597620 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fe80cd4a0 0x7f5fe81528e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c700 tx=0x7f5fdc00cac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:12.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.545+0000 7f5fcffff700 1 -- 172.21.15.28:0/3366597620 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc005d70 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.545+0000 7f5fcffff700 1 -- 172.21.15.28:0/3366597620 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fdc0070d0 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.545+0000 7f5fee058700 1 -- 172.21.15.28:0/3366597620 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe80cd920 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.545+0000 7f5fcffff700 1 -- 172.21.15.28:0/3366597620 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fdc007460 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.545+0000 7f5fee058700 1 -- 172.21.15.28:0/3366597620 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe806a840 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.546+0000 7f5fee058700 1 -- 172.21.15.28:0/3366597620 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe806b040 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.546+0000 7f5fcffff700 1 -- 172.21.15.28:0/3366597620 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f5fdc019030 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.546+0000 7f5fcffff700 1 --2- 172.21.15.28:0/3366597620 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fd40682f0 0x7f5fd406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:12.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.546+0000 7f5fcffff700 1 -- 172.21.15.28:0/3366597620 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f5fdc0852f0 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.547+0000 7f5fe77fe700 1 --2- 172.21.15.28:0/3366597620 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fd40682f0 0x7f5fd406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:12.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.547+0000 7f5fe77fe700 1 --2- 172.21.15.28:0/3366597620 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fd40682f0 0x7f5fd406a7a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8006b20 tx=0x7f5fd8006ab0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:12.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.551+0000 7f5fcffff700 1 -- 172.21.15.28:0/3366597620 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5fdc051950 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.761 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.761+0000 7f5fee058700 1 -- 172.21.15.28:0/3366597620 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5fe80675a0 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.762 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.761+0000 7f5fcffff700 1 -- 172.21.15.28:0/3366597620 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v68) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5fdc054f70 con 0x7f5fe80cd4a0 2023-11-28T11:13:12.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.763+0000 7f5fee058700 1 -- 172.21.15.28:0/3366597620 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fd40682f0 msgr2=0x7f5fd406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:12.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.763+0000 7f5fee058700 1 --2- 172.21.15.28:0/3366597620 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fd40682f0 0x7f5fd406a7a0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5fd8006b20 tx=0x7f5fd8006ab0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.763+0000 7f5fee058700 1 -- 172.21.15.28:0/3366597620 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fe80cd4a0 msgr2=0x7f5fe81528e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:12.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.763+0000 7f5fee058700 1 --2- 172.21.15.28:0/3366597620 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fe80cd4a0 0x7f5fe81528e0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f5fdc00c700 tx=0x7f5fdc00cac0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.763+0000 7f5fee058700 1 -- 172.21.15.28:0/3366597620 shutdown_connections 2023-11-28T11:13:12.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.763+0000 7f5fee058700 1 --2- 172.21.15.28:0/3366597620 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5fd40682f0 0x7f5fd406a7a0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.763+0000 7f5fee058700 1 --2- 172.21.15.28:0/3366597620 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f5fe815a230 0x7f5fe81f29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.763+0000 7f5fee058700 1 --2- 172.21.15.28:0/3366597620 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5fe80b4fe0 0x7f5fe80b8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.764 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.763+0000 7f5fee058700 1 --2- 172.21.15.28:0/3366597620 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f5fe80cd4a0 0x7f5fe81528e0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:12.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.763+0000 7f5fee058700 1 -- 172.21.15.28:0/3366597620 >> 172.21.15.28:0/3366597620 conn(0x7f5fe80cf8d0 msgr2=0x7f5fe815f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:12.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.763+0000 7f5fee058700 1 -- 172.21.15.28:0/3366597620 shutdown_connections 2023-11-28T11:13:12.765 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:12.763+0000 7f5fee058700 1 -- 172.21.15.28:0/3366597620 wait complete. 2023-11-28T11:13:12.765 INFO:teuthology.orchestra.run.smithi028.stderr:listed 0 entries 2023-11-28T11:13:12.775 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi028.front.sepia.ceph.com... 2023-11-28T11:13:12.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:13:12.776 DEBUG:teuthology.orchestra.run.smithi028:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-11-28T11:13:12.815 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi028.stderr:ceph-fuse[129555]: fuse finished with error 0 and tester_r 0 2023-11-28T11:13:12.817 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:13:12.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:13:12.817 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T11:13:12.984 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:13:12.985 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:13:13.007 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3961 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:13:13.007 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:13:13.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:13:13.008 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T11:13:13.066 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:13:13.067 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-11-28T11:13:13.067 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:13:13.067 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:13:13.085 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:13:13.086 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:13:13.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:13:13.086 DEBUG:teuthology.orchestra.run.smithi028:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T11:13:13.143 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:13:13.143 INFO:teuthology.orchestra.run.smithi028.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-11-28T11:13:13.144 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:13:13.144 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:13:13.166 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3961 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:13:13.166 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:13:13.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:13:13.166 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T11:13:13.226 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:13:13.227 INFO:teuthology.orchestra.run.smithi059.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-11-28T11:13:13.227 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:13:13.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.353+0000 7f9253df4700 1 Processor -- start 2023-11-28T11:13:13.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.354+0000 7f9253df4700 1 -- start start 2023-11-28T11:13:13.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.354+0000 7f9253df4700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f924c159590 0x7f924c06e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:13.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.354+0000 7f9253df4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f924c0ce890 0x7f924c0ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:13.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.354+0000 7f9253df4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f924c15d120 con 0x7f924c15db70 2023-11-28T11:13:13.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.354+0000 7f9253df4700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f924c0cf250 con 0x7f924c159590 2023-11-28T11:13:13.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.354+0000 7f9253df4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924c15d6e0 con 0x7f924c0ce890 2023-11-28T11:13:13.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.355+0000 7f9251b90700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f924c159590 0x7f924c06e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:13.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.355+0000 7f9251b90700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f924c159590 0x7f924c06e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52204/0 (socket says 172.21.15.28:52204) 2023-11-28T11:13:13.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.355+0000 7f9251b90700 1 -- 172.21.15.28:0/1015881841 learned_addr learned my addr 172.21.15.28:0/1015881841 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:13.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.355+0000 7f9252391700 1 --2- 172.21.15.28:0/1015881841 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f924c0ce890 0x7f924c0ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:13.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.355+0000 7f9251b90700 1 -- 172.21.15.28:0/1015881841 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f924c0ce890 msgr2=0x7f924c0ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:13.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.355+0000 7f9251b90700 1 --2- 172.21.15.28:0/1015881841 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f924c0ce890 0x7f924c0ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.355+0000 7f9251b90700 1 -- 172.21.15.28:0/1015881841 >> v1:172.21.15.59:6789/0 conn(0x7f924c15db70 legacy=0x7f924c0ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:13.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.355+0000 7f9251b90700 1 -- 172.21.15.28:0/1015881841 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c0aef60 con 0x7f924c159590 2023-11-28T11:13:13.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.355+0000 7f9251b90700 1 --2- 172.21.15.28:0/1015881841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f924c159590 0x7f924c06e400 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f924800bb90 tx=0x7f924800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4788bd292653f8d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:13.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.355+0000 7f9250b8e700 1 -- 172.21.15.28:0/1015881841 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f924800dd10 con 0x7f924c159590 2023-11-28T11:13:13.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.355+0000 7f9250b8e700 1 -- 172.21.15.28:0/1015881841 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f924800de70 con 0x7f924c159590 2023-11-28T11:13:13.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.355+0000 7f9250b8e700 1 -- 172.21.15.28:0/1015881841 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9248004230 con 0x7f924c159590 2023-11-28T11:13:13.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.356+0000 7f9253df4700 1 -- 172.21.15.28:0/1015881841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f924c159590 msgr2=0x7f924c06e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:13.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.356+0000 7f9253df4700 1 --2- 172.21.15.28:0/1015881841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f924c159590 0x7f924c06e400 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f924800bb90 tx=0x7f924800bea0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.356+0000 7f9253df4700 1 -- 172.21.15.28:0/1015881841 shutdown_connections 2023-11-28T11:13:13.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.356+0000 7f9253df4700 1 --2- 172.21.15.28:0/1015881841 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f924c159590 0x7f924c06e400 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.356+0000 7f9253df4700 1 --2- 172.21.15.28:0/1015881841 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f924c0ce890 0x7f924c0ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.356+0000 7f9253df4700 1 -- 172.21.15.28:0/1015881841 >> 172.21.15.28:0/1015881841 conn(0x7f924c0b1650 msgr2=0x7f924c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:13.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.356+0000 7f9253df4700 1 -- 172.21.15.28:0/1015881841 shutdown_connections 2023-11-28T11:13:13.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.356+0000 7f9253df4700 1 -- 172.21.15.28:0/1015881841 wait complete. 2023-11-28T11:13:13.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.357+0000 7f9253df4700 1 Processor -- start 2023-11-28T11:13:13.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.358+0000 7f9253df4700 1 -- start start 2023-11-28T11:13:13.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.358+0000 7f9253df4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f924c0ce890 0x7f924c155160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:13.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.358+0000 7f9253df4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f924c159590 0x7f924c0b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:13.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.358+0000 7f9253df4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f924c15db70 0x7f924c0b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:13.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.358+0000 7f9253df4700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c1545e0 con 0x7f924c15db70 2023-11-28T11:13:13.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f9253df4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c157870 con 0x7f924c0ce890 2023-11-28T11:13:13.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f9253df4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f924c155740 con 0x7f924c159590 2023-11-28T11:13:13.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f9252391700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f924c15db70 0x7f924c0b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:13.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f9252391700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f924c15db70 0x7f924c0b2f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52214/0 (socket says 172.21.15.28:52214) 2023-11-28T11:13:13.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f9252391700 1 -- 172.21.15.28:0/2726612476 learned_addr learned my addr 172.21.15.28:0/2726612476 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:13.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f9251b90700 1 --2- 172.21.15.28:0/2726612476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f924c0ce890 0x7f924c155160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:13.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f925138f700 1 --2- 172.21.15.28:0/2726612476 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f924c159590 0x7f924c0b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:13.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f9252391700 1 -- 172.21.15.28:0/2726612476 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f924c159590 msgr2=0x7f924c0b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:13.366 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f9252391700 1 --2- 172.21.15.28:0/2726612476 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f924c159590 0x7f924c0b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f9252391700 1 -- 172.21.15.28:0/2726612476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f924c0ce890 msgr2=0x7f924c155160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:13.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f9252391700 1 --2- 172.21.15.28:0/2726612476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f924c0ce890 0x7f924c155160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f9252391700 1 -- 172.21.15.28:0/2726612476 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924800b7f0 con 0x7f924c15db70 2023-11-28T11:13:13.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f9252391700 1 --2- 172.21.15.28:0/2726612476 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f924c15db70 0x7f924c0b2f30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f924400cd70 tx=0x7f924400a570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:13.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f923e7fc700 1 -- 172.21.15.28:0/2726612476 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244003a20 con 0x7f924c15db70 2023-11-28T11:13:13.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f923e7fc700 1 -- 172.21.15.28:0/2726612476 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9244003b80 con 0x7f924c15db70 2023-11-28T11:13:13.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f9253df4700 1 -- 172.21.15.28:0/2726612476 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924c15d370 con 0x7f924c15db70 2023-11-28T11:13:13.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f923e7fc700 1 -- 172.21.15.28:0/2726612476 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9244004020 con 0x7f924c15db70 2023-11-28T11:13:13.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.359+0000 7f9253df4700 1 -- 172.21.15.28:0/2726612476 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924c0af240 con 0x7f924c15db70 2023-11-28T11:13:13.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.360+0000 7f9253df4700 1 -- 172.21.15.28:0/2726612476 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924c15dff0 con 0x7f924c15db70 2023-11-28T11:13:13.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.361+0000 7f923e7fc700 1 -- 172.21.15.28:0/2726612476 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f924401c020 con 0x7f924c15db70 2023-11-28T11:13:13.367 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.361+0000 7f923e7fc700 1 --2- 172.21.15.28:0/2726612476 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f92300682f0 0x7f923006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:13.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.361+0000 7f923e7fc700 1 -- 172.21.15.28:0/2726612476 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f9244012070 con 0x7f924c15db70 2023-11-28T11:13:13.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.361+0000 7f9251b90700 1 --2- 172.21.15.28:0/2726612476 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f92300682f0 0x7f923006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:13.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.362+0000 7f9251b90700 1 --2- 172.21.15.28:0/2726612476 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f92300682f0 0x7f923006a7a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f924800bae0 tx=0x7f924801d040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:13.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.366+0000 7f923e7fc700 1 -- 172.21.15.28:0/2726612476 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9244052b30 con 0x7f924c15db70 2023-11-28T11:13:13.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.603+0000 7f9253df4700 1 -- 172.21.15.28:0/2726612476 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f924c0675a0 con 0x7f924c15db70 2023-11-28T11:13:13.604 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.603+0000 7f923e7fc700 1 -- 172.21.15.28:0/2726612476 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f9244056150 con 0x7f924c15db70 2023-11-28T11:13:13.604 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:13.604 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1564028052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1564028052},{"type":"v1","addr":"172.21.15.59:6835","nonce":1564028052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6558,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2509041002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2509041002},{"type":"v1","addr":"172.21.15.28:6836","nonce":2509041002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6561,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3994419079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3994419079},{"type":"v1","addr":"172.21.15.28:6837","nonce":3994419079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:12:03.800884+0000","modified":"2023-11-28T11:12:20.278498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.59:6834/2406852565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2406852565},{"type":"v1","addr":"172.21.15.59:6834","nonce":2406852565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:13:13.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.605+0000 7f9253df4700 1 -- 172.21.15.28:0/2726612476 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f92300682f0 msgr2=0x7f923006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:13.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.605+0000 7f9253df4700 1 --2- 172.21.15.28:0/2726612476 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f92300682f0 0x7f923006a7a0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f924800bae0 tx=0x7f924801d040 comp rx=0 tx=0).stop 2023-11-28T11:13:13.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.605+0000 7f9253df4700 1 -- 172.21.15.28:0/2726612476 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f924c15db70 msgr2=0x7f924c0b2f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:13.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.605+0000 7f9253df4700 1 --2- 172.21.15.28:0/2726612476 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f924c15db70 0x7f924c0b2f30 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f924400cd70 tx=0x7f924400a570 comp rx=0 tx=0).stop 2023-11-28T11:13:13.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.605+0000 7f9253df4700 1 -- 172.21.15.28:0/2726612476 shutdown_connections 2023-11-28T11:13:13.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.605+0000 7f9253df4700 1 --2- 172.21.15.28:0/2726612476 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f92300682f0 0x7f923006a7a0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.605+0000 7f9253df4700 1 --2- 172.21.15.28:0/2726612476 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f924c15db70 0x7f924c0b2f30 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.605+0000 7f9253df4700 1 --2- 172.21.15.28:0/2726612476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f924c0ce890 0x7f924c155160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.605+0000 7f9253df4700 1 --2- 172.21.15.28:0/2726612476 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f924c159590 0x7f924c0b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.606+0000 7f9253df4700 1 -- 172.21.15.28:0/2726612476 >> 172.21.15.28:0/2726612476 conn(0x7f924c0b1650 msgr2=0x7f924c0cdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:13.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.606+0000 7f9253df4700 1 -- 172.21.15.28:0/2726612476 shutdown_connections 2023-11-28T11:13:13.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.606+0000 7f9253df4700 1 -- 172.21.15.28:0/2726612476 wait complete. 2023-11-28T11:13:13.608 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 34 2023-11-28T11:13:13.618 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:13:13.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.742+0000 7f870ee2b700 1 Processor -- start 2023-11-28T11:13:13.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.742+0000 7f870ee2b700 1 -- start start 2023-11-28T11:13:13.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.743+0000 7f870ee2b700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f870815c560 0x7f870806f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:13.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.743+0000 7f870ee2b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f87080ceec0 0x7f870806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:13.750 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.743+0000 7f870ee2b700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87080b30b0 con 0x7f87080b2660 2023-11-28T11:13:13.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.743+0000 7f870ee2b700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f870806c020 con 0x7f87080ceec0 2023-11-28T11:13:13.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.743+0000 7f870ee2b700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87080d09c0 con 0x7f870815c560 2023-11-28T11:13:13.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.743+0000 7f870d3c8700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f87080ceec0 0x7f870806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:13.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.743+0000 7f8707fff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f87080b2660 0x7f87080ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40278/0 (socket says 172.21.15.28:40278) 2023-11-28T11:13:13.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.743+0000 7f870d3c8700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f87080ceec0 0x7f870806c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52222/0 (socket says 172.21.15.28:52222) 2023-11-28T11:13:13.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.743+0000 7f8707fff700 1 -- 172.21.15.28:0/380176158 learned_addr learned my addr 172.21.15.28:0/380176158 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:13.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.744+0000 7f870cbc7700 1 --2- 172.21.15.28:0/380176158 >> v2:172.21.15.59:3300/0 conn(0x7f870815c560 0x7f870806f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:13.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.744+0000 7f870d3c8700 1 -- 172.21.15.28:0/380176158 >> v2:172.21.15.59:3300/0 conn(0x7f870815c560 msgr2=0x7f870806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:13.751 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.744+0000 7f870d3c8700 1 --2- 172.21.15.28:0/380176158 >> v2:172.21.15.59:3300/0 conn(0x7f870815c560 0x7f870806f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.744+0000 7f870d3c8700 1 -- 172.21.15.28:0/380176158 >> v1:172.21.15.28:6789/0 conn(0x7f87080b2660 legacy=0x7f87080ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:13.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.744+0000 7f870d3c8700 1 -- 172.21.15.28:0/380176158 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870806fdf0 con 0x7f87080ceec0 2023-11-28T11:13:13.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.744+0000 7f87077fe700 1 -- 172.21.15.28:0/380176158 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600813450 0 0) 0x7f87080b30b0 con 0x7f87080b2660 2023-11-28T11:13:13.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.744+0000 7f870d3c8700 1 --2- 172.21.15.28:0/380176158 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87080ceec0 0x7f870806c4b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f8708066920 tx=0x7f870000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3659ab41188e7ed6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:13.752 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.744+0000 7f87077fe700 1 -- 172.21.15.28:0/380176158 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f870000e070 con 0x7f87080ceec0 2023-11-28T11:13:13.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.744+0000 7f87077fe700 1 -- 172.21.15.28:0/380176158 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f870000bbd0 con 0x7f87080ceec0 2023-11-28T11:13:13.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.744+0000 7f87077fe700 1 -- 172.21.15.28:0/380176158 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8700012430 con 0x7f87080ceec0 2023-11-28T11:13:13.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.745+0000 7f870ee2b700 1 -- 172.21.15.28:0/380176158 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87080ceec0 msgr2=0x7f870806c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:13.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.745+0000 7f870ee2b700 1 --2- 172.21.15.28:0/380176158 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87080ceec0 0x7f870806c4b0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f8708066920 tx=0x7f870000b330 comp rx=0 tx=0).stop 2023-11-28T11:13:13.753 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.745+0000 7f870ee2b700 1 -- 172.21.15.28:0/380176158 shutdown_connections 2023-11-28T11:13:13.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.745+0000 7f870ee2b700 1 --2- 172.21.15.28:0/380176158 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87080ceec0 0x7f870806c4b0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.745+0000 7f870ee2b700 1 --2- 172.21.15.28:0/380176158 >> v2:172.21.15.59:3300/0 conn(0x7f870815c560 0x7f870806f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.745+0000 7f870ee2b700 1 -- 172.21.15.28:0/380176158 >> 172.21.15.28:0/380176158 conn(0x7f87080cd660 msgr2=0x7f87080cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:13.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.745+0000 7f870ee2b700 1 -- 172.21.15.28:0/380176158 shutdown_connections 2023-11-28T11:13:13.754 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.745+0000 7f870ee2b700 1 -- 172.21.15.28:0/380176158 wait complete. 2023-11-28T11:13:13.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.746+0000 7f870ee2b700 1 Processor -- start 2023-11-28T11:13:13.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.746+0000 7f870ee2b700 1 -- start start 2023-11-28T11:13:13.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870ee2b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87080b2660 0x7f870815d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:13.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870ee2b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87080ceec0 0x7f870806c720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:13.755 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870ee2b700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f870815c560 0x7f870814f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:13.756 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870ee2b700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87080ac3d0 con 0x7f87080b2660 2023-11-28T11:13:13.756 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870ee2b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87080ac670 con 0x7f87080ceec0 2023-11-28T11:13:13.756 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870ee2b700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87080ac950 con 0x7f870815c560 2023-11-28T11:13:13.756 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870cbc7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87080b2660 0x7f870815d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:13.756 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870cbc7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87080b2660 0x7f870815d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52224/0 (socket says 172.21.15.28:52224) 2023-11-28T11:13:13.756 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870cbc7700 1 -- 172.21.15.28:0/1815158881 learned_addr learned my addr 172.21.15.28:0/1815158881 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:13.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f8707fff700 1 --2- 172.21.15.28:0/1815158881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87080ceec0 0x7f870806c720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:13.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870d3c8700 1 --2- 172.21.15.28:0/1815158881 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f870815c560 0x7f870814f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:13.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870cbc7700 1 -- 172.21.15.28:0/1815158881 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f870815c560 msgr2=0x7f870814f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:13.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870cbc7700 1 --2- 172.21.15.28:0/1815158881 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f870815c560 0x7f870814f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870cbc7700 1 -- 172.21.15.28:0/1815158881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87080ceec0 msgr2=0x7f870806c720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:13.757 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870cbc7700 1 --2- 172.21.15.28:0/1815158881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87080ceec0 0x7f870806c720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870cbc7700 1 -- 172.21.15.28:0/1815158881 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8700009a70 con 0x7f87080b2660 2023-11-28T11:13:13.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870d3c8700 1 --2- 172.21.15.28:0/1815158881 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f870815c560 0x7f870814f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:13:13.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f870cbc7700 1 --2- 172.21.15.28:0/1815158881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87080b2660 0x7f870815d2f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00d800 tx=0x7f86fc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:13.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.747+0000 7f87057fa700 1 -- 172.21.15.28:0/1815158881 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0125f0 con 0x7f87080b2660 2023-11-28T11:13:13.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.748+0000 7f87057fa700 1 -- 172.21.15.28:0/1815158881 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86fc012750 con 0x7f87080b2660 2023-11-28T11:13:13.758 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.748+0000 7f87057fa700 1 -- 172.21.15.28:0/1815158881 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86fc012b00 con 0x7f87080b2660 2023-11-28T11:13:13.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.748+0000 7f870ee2b700 1 -- 172.21.15.28:0/1815158881 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87080b2dd0 con 0x7f87080b2660 2023-11-28T11:13:13.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.748+0000 7f870ee2b700 1 -- 172.21.15.28:0/1815158881 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87080b2af0 con 0x7f87080b2660 2023-11-28T11:13:13.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.748+0000 7f870ee2b700 1 -- 172.21.15.28:0/1815158881 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f870806bcb0 con 0x7f87080b2660 2023-11-28T11:13:13.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.749+0000 7f87057fa700 1 -- 172.21.15.28:0/1815158881 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f86fc012c60 con 0x7f87080b2660 2023-11-28T11:13:13.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.749+0000 7f87057fa700 1 --2- 172.21.15.28:0/1815158881 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86ec0682c0 0x7f86ec06a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:13.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.749+0000 7f87057fa700 1 -- 172.21.15.28:0/1815158881 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f86fc086330 con 0x7f87080b2660 2023-11-28T11:13:13.759 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.750+0000 7f8707fff700 1 --2- 172.21.15.28:0/1815158881 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86ec0682c0 0x7f86ec06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:13.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.750+0000 7f8707fff700 1 --2- 172.21.15.28:0/1815158881 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86ec0682c0 0x7f86ec06a770 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f870815cf50 tx=0x7f86f4007be0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:13.760 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.754+0000 7f87057fa700 1 -- 172.21.15.28:0/1815158881 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f86fc052a10 con 0x7f87080b2660 2023-11-28T11:13:13.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.948+0000 7f870ee2b700 1 -- 172.21.15.28:0/1815158881 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87080675a0 con 0x7f87080b2660 2023-11-28T11:13:13.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.949+0000 7f87057fa700 1 -- 172.21.15.28:0/1815158881 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18174 (secure 0 0 0) 0x7f86fc056030 con 0x7f87080b2660 2023-11-28T11:13:13.950 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:13.950 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":68,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:12:35.277297+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:12:01.724396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:12:02.776612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.9","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:13:13.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.951+0000 7f870ee2b700 1 -- 172.21.15.28:0/1815158881 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86ec0682c0 msgr2=0x7f86ec06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:13.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.951+0000 7f870ee2b700 1 --2- 172.21.15.28:0/1815158881 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86ec0682c0 0x7f86ec06a770 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f870815cf50 tx=0x7f86f4007be0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.951+0000 7f870ee2b700 1 -- 172.21.15.28:0/1815158881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87080b2660 msgr2=0x7f870815d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:13.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.951+0000 7f870ee2b700 1 --2- 172.21.15.28:0/1815158881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87080b2660 0x7f870815d2f0 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00d800 tx=0x7f86fc00dbc0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.951+0000 7f870ee2b700 1 -- 172.21.15.28:0/1815158881 shutdown_connections 2023-11-28T11:13:13.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.951+0000 7f870ee2b700 1 --2- 172.21.15.28:0/1815158881 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f86ec0682c0 0x7f86ec06a770 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.951+0000 7f870ee2b700 1 --2- 172.21.15.28:0/1815158881 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f87080b2660 0x7f870815d2f0 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.951+0000 7f870ee2b700 1 --2- 172.21.15.28:0/1815158881 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f870815c560 0x7f870814f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.951+0000 7f870ee2b700 1 --2- 172.21.15.28:0/1815158881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87080ceec0 0x7f870806c720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:13.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.951+0000 7f870ee2b700 1 -- 172.21.15.28:0/1815158881 >> 172.21.15.28:0/1815158881 conn(0x7f87080cd660 msgr2=0x7f87080ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:13.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.951+0000 7f870ee2b700 1 -- 172.21.15.28:0/1815158881 shutdown_connections 2023-11-28T11:13:13.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:13.951+0000 7f870ee2b700 1 -- 172.21.15.28:0/1815158881 wait complete. 2023-11-28T11:13:13.964 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2023-11-28T11:08:38.728197+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2023-11-28T11:08:39.218586+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T11:12:01.724396+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T11:12:02.776612+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-11-28T11:13:13.965 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T11:13:14.092 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.092+0000 7fc41c570700 1 Processor -- start 2023-11-28T11:13:14.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.092+0000 7fc41c570700 1 -- start start 2023-11-28T11:13:14.100 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.093+0000 7fc41c570700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4140cd140 0x7fc41415dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.093+0000 7fc41c570700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4140ceff0 con 0x7fc414159500 2023-11-28T11:13:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.093+0000 7fc41c570700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4140cf2d0 con 0x7fc4140b1d40 2023-11-28T11:13:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.093+0000 7fc41c570700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41415d110 con 0x7fc4140cd140 2023-11-28T11:13:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.093+0000 7fc41a30c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fc414159500 0x7fc4140b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40290/0 (socket says 172.21.15.28:40290) 2023-11-28T11:13:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.093+0000 7fc41a30c700 1 -- 172.21.15.28:0/2435315019 learned_addr learned my addr 172.21.15.28:0/2435315019 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.093+0000 7fc41930a700 1 -- 172.21.15.28:0/2435315019 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 945105582 0 0) 0x7fc4140ceff0 con 0x7fc414159500 2023-11-28T11:13:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.093+0000 7fc41ab0d700 1 --2- 172.21.15.28:0/2435315019 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4140cd140 0x7fc41415dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.093+0000 7fc41930a700 1 -- 172.21.15.28:0/2435315019 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4000035b0 con 0x7fc414159500 2023-11-28T11:13:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 -- 172.21.15.28:0/2435315019 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 122449650 0 0) 0x7fc4000035b0 con 0x7fc414159500 2023-11-28T11:13:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 -- 172.21.15.28:0/2435315019 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4140cd140 msgr2=0x7fc41415dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:14.101 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 --2- 172.21.15.28:0/2435315019 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4140cd140 0x7fc41415dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 -- 172.21.15.28:0/2435315019 >> v1:172.21.15.59:6789/0 conn(0x7fc4140b1d40 legacy=0x7fc41406bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 -- 172.21.15.28:0/2435315019 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41415d3f0 con 0x7fc414159500 2023-11-28T11:13:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 -- 172.21.15.28:0/2435315019 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fc404001670 con 0x7fc414159500 2023-11-28T11:13:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 -- 172.21.15.28:0/2435315019 >> v1:172.21.15.28:6789/0 conn(0x7fc414159500 legacy=0x7fc4140b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 --2- 172.21.15.28:0/2435315019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc400003a30 0x7fc400003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 --2- 172.21.15.28:0/2435315019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4000043d0 0x7fc400004890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 --2- 172.21.15.28:0/2435315019 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4140cd140 0x7fc4000050f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 -- 172.21.15.28:0/2435315019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc41415d110 con 0x7fc400003a30 2023-11-28T11:13:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 -- 172.21.15.28:0/2435315019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40000baa0 con 0x7fc4000043d0 2023-11-28T11:13:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 -- 172.21.15.28:0/2435315019 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc40000bc60 con 0x7fc4140cd140 2023-11-28T11:13:14.102 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41a30c700 1 --2- 172.21.15.28:0/2435315019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc400003a30 0x7fc400003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc419b0b700 1 --2- 172.21.15.28:0/2435315019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4000043d0 0x7fc400004890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41ab0d700 1 --2- 172.21.15.28:0/2435315019 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4140cd140 0x7fc4000050f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41a30c700 1 -- 172.21.15.28:0/2435315019 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4140cd140 msgr2=0x7fc4000050f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41a30c700 1 --2- 172.21.15.28:0/2435315019 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4140cd140 0x7fc4000050f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41a30c700 1 -- 172.21.15.28:0/2435315019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4000043d0 msgr2=0x7fc400004890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41a30c700 1 --2- 172.21.15.28:0/2435315019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4000043d0 0x7fc400004890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41a30c700 1 -- 172.21.15.28:0/2435315019 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4000035b0 con 0x7fc400003a30 2023-11-28T11:13:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41a30c700 1 --2- 172.21.15.28:0/2435315019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc400003a30 0x7fc400003e00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc4040050f0 tx=0x7fc400005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=847c66bb7489f824 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 -- 172.21.15.28:0/2435315019 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404005b60 con 0x7fc400003a30 2023-11-28T11:13:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 -- 172.21.15.28:0/2435315019 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc404005cc0 con 0x7fc400003a30 2023-11-28T11:13:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.094+0000 7fc41930a700 1 -- 172.21.15.28:0/2435315019 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404008300 con 0x7fc400003a30 2023-11-28T11:13:14.103 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.095+0000 7fc41c570700 1 -- 172.21.15.28:0/2435315019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc400003a30 msgr2=0x7fc400003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.095+0000 7fc41c570700 1 --2- 172.21.15.28:0/2435315019 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc400003a30 0x7fc400003e00 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc4040050f0 tx=0x7fc400005880 comp rx=0 tx=0).stop 2023-11-28T11:13:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.095+0000 7fc41ab0d700 1 -- 172.21.15.28:0/2435315019 reap_dead start 2023-11-28T11:13:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.095+0000 7fc41c570700 1 -- 172.21.15.28:0/2435315019 shutdown_connections 2023-11-28T11:13:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.095+0000 7fc41c570700 1 -- 172.21.15.28:0/2435315019 >> 172.21.15.28:0/2435315019 conn(0x7fc4140cc630 msgr2=0x7fc4140cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.095+0000 7fc41c570700 1 -- 172.21.15.28:0/2435315019 shutdown_connections 2023-11-28T11:13:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.095+0000 7fc41c570700 1 -- 172.21.15.28:0/2435315019 wait complete. 2023-11-28T11:13:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.096+0000 7fc41c570700 1 Processor -- start 2023-11-28T11:13:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.096+0000 7fc41c570700 1 -- start start 2023-11-28T11:13:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.097+0000 7fc41c570700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc400003a30 0x7fc414147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.097+0000 7fc41c570700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4141577a0 0x7fc414157c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.097+0000 7fc41c570700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4140c6c80 0x7fc4140c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.097+0000 7fc41c570700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4140c8580 con 0x7fc400003a30 2023-11-28T11:13:14.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.097+0000 7fc41c570700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc414158150 con 0x7fc4140c6c80 2023-11-28T11:13:14.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.097+0000 7fc41c570700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc414148510 con 0x7fc4141577a0 2023-11-28T11:13:14.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.097+0000 7fc41a30c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc400003a30 0x7fc414147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.097+0000 7fc41a30c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc400003a30 0x7fc414147fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52244/0 (socket says 172.21.15.28:52244) 2023-11-28T11:13:14.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.097+0000 7fc41a30c700 1 -- 172.21.15.28:0/2242545304 learned_addr learned my addr 172.21.15.28:0/2242545304 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:14.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.097+0000 7fc41a30c700 1 -- 172.21.15.28:0/2242545304 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4141577a0 msgr2=0x7fc414157c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.097+0000 7fc41a30c700 1 --2- 172.21.15.28:0/2242545304 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4141577a0 0x7fc414157c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.097+0000 7fc41a30c700 1 -- 172.21.15.28:0/2242545304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4140c6c80 msgr2=0x7fc4140c70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.097+0000 7fc41ab0d700 1 --2- 172.21.15.28:0/2242545304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4140c6c80 0x7fc4140c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.105 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.097+0000 7fc41a30c700 1 --2- 172.21.15.28:0/2242545304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4140c6c80 0x7fc4140c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.097+0000 7fc41a30c700 1 -- 172.21.15.28:0/2242545304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4141598d0 con 0x7fc400003a30 2023-11-28T11:13:14.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.098+0000 7fc41a30c700 1 --2- 172.21.15.28:0/2242545304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc400003a30 0x7fc414147fd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc404004b20 tx=0x7fc404004a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:14.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.098+0000 7fc40affd700 1 -- 172.21.15.28:0/2242545304 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404005b50 con 0x7fc400003a30 2023-11-28T11:13:14.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.098+0000 7fc40affd700 1 -- 172.21.15.28:0/2242545304 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc404005e30 con 0x7fc400003a30 2023-11-28T11:13:14.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.098+0000 7fc41c570700 1 -- 172.21.15.28:0/2242545304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc41415d6d0 con 0x7fc400003a30 2023-11-28T11:13:14.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.098+0000 7fc40affd700 1 -- 172.21.15.28:0/2242545304 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc404001b10 con 0x7fc400003a30 2023-11-28T11:13:14.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.098+0000 7fc41c570700 1 -- 172.21.15.28:0/2242545304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc414159a60 con 0x7fc400003a30 2023-11-28T11:13:14.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.099+0000 7fc41c570700 1 -- 172.21.15.28:0/2242545304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4140739f0 con 0x7fc400003a30 2023-11-28T11:13:14.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.099+0000 7fc40affd700 1 -- 172.21.15.28:0/2242545304 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fc404011030 con 0x7fc400003a30 2023-11-28T11:13:14.106 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.100+0000 7fc40affd700 1 --2- 172.21.15.28:0/2242545304 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3f80682f0 0x7fc3f806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.100+0000 7fc40affd700 1 -- 172.21.15.28:0/2242545304 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fc4040870a0 con 0x7fc400003a30 2023-11-28T11:13:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.100+0000 7fc419b0b700 1 --2- 172.21.15.28:0/2242545304 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3f80682f0 0x7fc3f806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.100+0000 7fc419b0b700 1 --2- 172.21.15.28:0/2242545304 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3f80682f0 0x7fc3f806a7a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc4141571b0 tx=0x7fc410009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:14.107 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.104+0000 7fc40affd700 1 -- 172.21.15.28:0/2242545304 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fc404053700 con 0x7fc400003a30 2023-11-28T11:13:14.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.298+0000 7fc41c570700 1 -- 172.21.15.28:0/2242545304 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fc4140739f0 con 0x7fc400003a30 2023-11-28T11:13:14.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.299+0000 7fc40affd700 1 -- 172.21.15.28:0/2242545304 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v68) v1 ==== 92+0+10 (secure 0 0 0) 0x7fc404056d20 con 0x7fc400003a30 2023-11-28T11:13:14.300 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 8 2023-11-28T11:13:14.302 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.301+0000 7fc41c570700 1 -- 172.21.15.28:0/2242545304 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3f80682f0 msgr2=0x7fc3f806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.301+0000 7fc41c570700 1 --2- 172.21.15.28:0/2242545304 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3f80682f0 0x7fc3f806a7a0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fc4141571b0 tx=0x7fc410009040 comp rx=0 tx=0).stop 2023-11-28T11:13:14.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.301+0000 7fc41c570700 1 -- 172.21.15.28:0/2242545304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc400003a30 msgr2=0x7fc414147fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.301+0000 7fc41c570700 1 --2- 172.21.15.28:0/2242545304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc400003a30 0x7fc414147fd0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc404004b20 tx=0x7fc404004a90 comp rx=0 tx=0).stop 2023-11-28T11:13:14.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.301+0000 7fc41c570700 1 -- 172.21.15.28:0/2242545304 shutdown_connections 2023-11-28T11:13:14.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.301+0000 7fc41c570700 1 --2- 172.21.15.28:0/2242545304 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fc3f80682f0 0x7fc3f806a7a0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.301+0000 7fc41c570700 1 --2- 172.21.15.28:0/2242545304 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fc400003a30 0x7fc414147fd0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.301+0000 7fc41c570700 1 --2- 172.21.15.28:0/2242545304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4140c6c80 0x7fc4140c70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.301+0000 7fc41c570700 1 --2- 172.21.15.28:0/2242545304 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fc4141577a0 0x7fc414157c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.301+0000 7fc41c570700 1 -- 172.21.15.28:0/2242545304 >> 172.21.15.28:0/2242545304 conn(0x7fc4140cc630 msgr2=0x7fc4140ce100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:14.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.301+0000 7fc41c570700 1 -- 172.21.15.28:0/2242545304 shutdown_connections 2023-11-28T11:13:14.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.301+0000 7fc41c570700 1 -- 172.21.15.28:0/2242545304 wait complete. 2023-11-28T11:13:14.314 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-11-28T11:13:14.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.442+0000 7ff778333700 1 Processor -- start 2023-11-28T11:13:14.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.442+0000 7ff778333700 1 -- start start 2023-11-28T11:13:14.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff778333700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff770159500 0x7ff7700cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.451 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff778333700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff7700b1d40 0x7ff7700ce8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff778333700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff77015e040 con 0x7ff77015dbf0 2023-11-28T11:13:14.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff778333700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7700cede0 con 0x7ff7700b1d40 2023-11-28T11:13:14.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff778333700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7700ceff0 con 0x7ff770159500 2023-11-28T11:13:14.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff7768d0700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff7700b1d40 0x7ff7700ce8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff7758ce700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff77015dbf0 0x7ff7700b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40310/0 (socket says 172.21.15.28:40310) 2023-11-28T11:13:14.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff7758ce700 1 -- 172.21.15.28:0/1604429511 learned_addr learned my addr 172.21.15.28:0/1604429511 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:14.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff7768d0700 1 -- 172.21.15.28:0/1604429511 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff770159500 msgr2=0x7ff7700cd140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:14.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff7760cf700 1 --2- 172.21.15.28:0/1604429511 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff770159500 0x7ff7700cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff7750cd700 1 -- 172.21.15.28:0/1604429511 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 977801191 0 0) 0x7ff77015e040 con 0x7ff77015dbf0 2023-11-28T11:13:14.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff7768d0700 1 --2- 172.21.15.28:0/1604429511 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff770159500 0x7ff7700cd140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff7768d0700 1 -- 172.21.15.28:0/1604429511 >> v1:172.21.15.28:6789/0 conn(0x7ff77015dbf0 legacy=0x7ff7700b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff7768d0700 1 -- 172.21.15.28:0/1604429511 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7700cf2d0 con 0x7ff7700b1d40 2023-11-28T11:13:14.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff7760cf700 1 --2- 172.21.15.28:0/1604429511 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff770159500 0x7ff7700cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:13:14.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.443+0000 7ff7768d0700 1 --2- 172.21.15.28:0/1604429511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7700b1d40 0x7ff7700ce8a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ff768009d90 tx=0x7ff76800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cb989a31f11876a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:14.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.444+0000 7ff7750cd700 1 -- 172.21.15.28:0/1604429511 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff76800e070 con 0x7ff7700b1d40 2023-11-28T11:13:14.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.444+0000 7ff7750cd700 1 -- 172.21.15.28:0/1604429511 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff76800bbd0 con 0x7ff7700b1d40 2023-11-28T11:13:14.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.444+0000 7ff7750cd700 1 -- 172.21.15.28:0/1604429511 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff768012430 con 0x7ff7700b1d40 2023-11-28T11:13:14.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.444+0000 7ff778333700 1 -- 172.21.15.28:0/1604429511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7700b1d40 msgr2=0x7ff7700ce8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.444+0000 7ff778333700 1 --2- 172.21.15.28:0/1604429511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7700b1d40 0x7ff7700ce8a0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7ff768009d90 tx=0x7ff76800b330 comp rx=0 tx=0).stop 2023-11-28T11:13:14.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.444+0000 7ff778333700 1 -- 172.21.15.28:0/1604429511 shutdown_connections 2023-11-28T11:13:14.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.444+0000 7ff778333700 1 --2- 172.21.15.28:0/1604429511 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7700b1d40 0x7ff7700ce8a0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.444+0000 7ff778333700 1 --2- 172.21.15.28:0/1604429511 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff770159500 0x7ff7700cd140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.444+0000 7ff778333700 1 -- 172.21.15.28:0/1604429511 >> 172.21.15.28:0/1604429511 conn(0x7ff7700cc630 msgr2=0x7ff7700cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.444+0000 7ff778333700 1 -- 172.21.15.28:0/1604429511 shutdown_connections 2023-11-28T11:13:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.444+0000 7ff778333700 1 -- 172.21.15.28:0/1604429511 wait complete. 2023-11-28T11:13:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.445+0000 7ff778333700 1 Processor -- start 2023-11-28T11:13:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.445+0000 7ff778333700 1 -- start start 2023-11-28T11:13:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.446+0000 7ff778333700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7700b1d40 0x7ff7700ac620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.446+0000 7ff778333700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff770159500 0x7ff7700aded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.446+0000 7ff778333700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff77015dbf0 0x7ff77014f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.446+0000 7ff778333700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7700ae790 con 0x7ff770159500 2023-11-28T11:13:14.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.446+0000 7ff778333700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff770156b90 con 0x7ff77015dbf0 2023-11-28T11:13:14.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.446+0000 7ff778333700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff770156e70 con 0x7ff7700b1d40 2023-11-28T11:13:14.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.446+0000 7ff7758ce700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff770159500 0x7ff7700aded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.446+0000 7ff7758ce700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff770159500 0x7ff7700aded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52260/0 (socket says 172.21.15.28:52260) 2023-11-28T11:13:14.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.446+0000 7ff7758ce700 1 -- 172.21.15.28:0/2590979839 learned_addr learned my addr 172.21.15.28:0/2590979839 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:14.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.446+0000 7ff7768d0700 1 --2- 172.21.15.28:0/2590979839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff77015dbf0 0x7ff77014f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.446+0000 7ff7760cf700 1 --2- 172.21.15.28:0/2590979839 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7700b1d40 0x7ff7700ac620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.447+0000 7ff7758ce700 1 -- 172.21.15.28:0/2590979839 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7700b1d40 msgr2=0x7ff7700ac620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.447+0000 7ff7758ce700 1 --2- 172.21.15.28:0/2590979839 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7700b1d40 0x7ff7700ac620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.447+0000 7ff7758ce700 1 -- 172.21.15.28:0/2590979839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff77015dbf0 msgr2=0x7ff77014f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.447+0000 7ff7758ce700 1 --2- 172.21.15.28:0/2590979839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff77015dbf0 0x7ff77014f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.447+0000 7ff7758ce700 1 -- 172.21.15.28:0/2590979839 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff768009a70 con 0x7ff770159500 2023-11-28T11:13:14.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.447+0000 7ff7758ce700 1 --2- 172.21.15.28:0/2590979839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff770159500 0x7ff7700aded0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ff7600011e0 tx=0x7ff76000ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:14.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.447+0000 7ff766ffd700 1 -- 172.21.15.28:0/2590979839 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff760004c80 con 0x7ff770159500 2023-11-28T11:13:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.447+0000 7ff766ffd700 1 -- 172.21.15.28:0/2590979839 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff760004de0 con 0x7ff770159500 2023-11-28T11:13:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.447+0000 7ff778333700 1 -- 172.21.15.28:0/2590979839 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff77015d3f0 con 0x7ff770159500 2023-11-28T11:13:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.447+0000 7ff766ffd700 1 -- 172.21.15.28:0/2590979839 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff760007310 con 0x7ff770159500 2023-11-28T11:13:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.447+0000 7ff778333700 1 -- 172.21.15.28:0/2590979839 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff77015d110 con 0x7ff770159500 2023-11-28T11:13:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.448+0000 7ff778333700 1 -- 172.21.15.28:0/2590979839 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff77015d6d0 con 0x7ff770159500 2023-11-28T11:13:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.448+0000 7ff766ffd700 1 -- 172.21.15.28:0/2590979839 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff76001a020 con 0x7ff770159500 2023-11-28T11:13:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.449+0000 7ff766ffd700 1 --2- 172.21.15.28:0/2590979839 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7540682f0 0x7ff75406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.449+0000 7ff766ffd700 1 -- 172.21.15.28:0/2590979839 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff7600859d0 con 0x7ff770159500 2023-11-28T11:13:14.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.449+0000 7ff7760cf700 1 --2- 172.21.15.28:0/2590979839 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7540682f0 0x7ff75406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.449+0000 7ff7760cf700 1 --2- 172.21.15.28:0/2590979839 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7540682f0 0x7ff75406a7a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff76c006fd0 tx=0x7ff76c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:14.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.453+0000 7ff766ffd700 1 -- 172.21.15.28:0/2590979839 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff7600520b0 con 0x7ff770159500 2023-11-28T11:13:14.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.645+0000 7ff778333700 1 -- 172.21.15.28:0/2590979839 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff7700739f0 con 0x7ff770159500 2023-11-28T11:13:14.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.646+0000 7ff766ffd700 1 -- 172.21.15.28:0/2590979839 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v68) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff7600556d0 con 0x7ff770159500 2023-11-28T11:13:14.647 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 1 2023-11-28T11:13:14.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.648+0000 7ff778333700 1 -- 172.21.15.28:0/2590979839 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7540682f0 msgr2=0x7ff75406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.648+0000 7ff778333700 1 --2- 172.21.15.28:0/2590979839 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7540682f0 0x7ff75406a7a0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ff76c006fd0 tx=0x7ff76c008040 comp rx=0 tx=0).stop 2023-11-28T11:13:14.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.648+0000 7ff778333700 1 -- 172.21.15.28:0/2590979839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff770159500 msgr2=0x7ff7700aded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.648+0000 7ff778333700 1 --2- 172.21.15.28:0/2590979839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff770159500 0x7ff7700aded0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7ff7600011e0 tx=0x7ff76000ca30 comp rx=0 tx=0).stop 2023-11-28T11:13:14.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.648+0000 7ff778333700 1 -- 172.21.15.28:0/2590979839 shutdown_connections 2023-11-28T11:13:14.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.648+0000 7ff778333700 1 --2- 172.21.15.28:0/2590979839 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7540682f0 0x7ff75406a7a0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.648+0000 7ff778333700 1 --2- 172.21.15.28:0/2590979839 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff77015dbf0 0x7ff77014f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.648+0000 7ff778333700 1 --2- 172.21.15.28:0/2590979839 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7700b1d40 0x7ff7700ac620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.648+0000 7ff778333700 1 --2- 172.21.15.28:0/2590979839 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff770159500 0x7ff7700aded0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.648+0000 7ff778333700 1 -- 172.21.15.28:0/2590979839 >> 172.21.15.28:0/2590979839 conn(0x7ff7700cc630 msgr2=0x7ff7700cbd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:14.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.648+0000 7ff778333700 1 -- 172.21.15.28:0/2590979839 shutdown_connections 2023-11-28T11:13:14.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.648+0000 7ff778333700 1 -- 172.21.15.28:0/2590979839 wait complete. 2023-11-28T11:13:14.661 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T11:13:14.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.788+0000 7fa329818700 1 Processor -- start 2023-11-28T11:13:14.788 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.788+0000 7fa329818700 1 -- start start 2023-11-28T11:13:14.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.788+0000 7fa329818700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32415c640 0x7fa3240cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.788+0000 7fa329818700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3240d2820 0x7fa3240b5c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.788+0000 7fa329818700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa32406bf30 0x7fa32406c350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.788+0000 7fa329818700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32415ca10 con 0x7fa3240d2820 2023-11-28T11:13:14.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.788+0000 7fa329818700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa32406c890 con 0x7fa32406bf30 2023-11-28T11:13:14.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.788+0000 7fa329818700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240d1470 con 0x7fa32415c640 2023-11-28T11:13:14.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa3227fc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3240d2820 0x7fa3240b5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa3227fc700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa3240d2820 0x7fa3240b5c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52268/0 (socket says 172.21.15.28:52268) 2023-11-28T11:13:14.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa3227fc700 1 -- 172.21.15.28:0/1609519027 learned_addr learned my addr 172.21.15.28:0/1609519027 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:14.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa322ffd700 1 --2- 172.21.15.28:0/1609519027 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32415c640 0x7fa3240cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.791 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa3237fe700 1 --2- 172.21.15.28:0/1609519027 >> v2:172.21.15.59:3300/0 conn(0x7fa32406bf30 0x7fa32406c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa3227fc700 1 -- 172.21.15.28:0/1609519027 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32415c640 msgr2=0x7fa3240cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:14.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa3227fc700 1 --2- 172.21.15.28:0/1609519027 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32415c640 0x7fa3240cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa3227fc700 1 -- 172.21.15.28:0/1609519027 >> v2:172.21.15.59:3300/0 conn(0x7fa32406bf30 msgr2=0x7fa32406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:14.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa3227fc700 1 --2- 172.21.15.28:0/1609519027 >> v2:172.21.15.59:3300/0 conn(0x7fa32406bf30 0x7fa32406c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa3227fc700 1 -- 172.21.15.28:0/1609519027 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3240cd4e0 con 0x7fa3240d2820 2023-11-28T11:13:14.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa3237fe700 1 --2- 172.21.15.28:0/1609519027 >> v2:172.21.15.59:3300/0 conn(0x7fa32406bf30 0x7fa32406c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:14.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa322ffd700 1 --2- 172.21.15.28:0/1609519027 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32415c640 0x7fa3240cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:14.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa3227fc700 1 --2- 172.21.15.28:0/1609519027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240d2820 0x7fa3240b5c40 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa32415ffd0 tx=0x7fa314009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecbf8703f177d823 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:14.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa321ffb700 1 -- 172.21.15.28:0/1609519027 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31400b870 con 0x7fa3240d2820 2023-11-28T11:13:14.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa321ffb700 1 -- 172.21.15.28:0/1609519027 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31400b9d0 con 0x7fa3240d2820 2023-11-28T11:13:14.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.789+0000 7fa321ffb700 1 -- 172.21.15.28:0/1609519027 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31400bd60 con 0x7fa3240d2820 2023-11-28T11:13:14.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.790+0000 7fa329818700 1 -- 172.21.15.28:0/1609519027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240d2820 msgr2=0x7fa3240b5c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.792 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.790+0000 7fa329818700 1 --2- 172.21.15.28:0/1609519027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240d2820 0x7fa3240b5c40 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fa32415ffd0 tx=0x7fa314009bd0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.790+0000 7fa329818700 1 -- 172.21.15.28:0/1609519027 shutdown_connections 2023-11-28T11:13:14.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.790+0000 7fa329818700 1 --2- 172.21.15.28:0/1609519027 >> v2:172.21.15.59:3300/0 conn(0x7fa32406bf30 0x7fa32406c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.790+0000 7fa329818700 1 --2- 172.21.15.28:0/1609519027 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa32415c640 0x7fa3240cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.790+0000 7fa329818700 1 --2- 172.21.15.28:0/1609519027 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa3240d2820 0x7fa3240b5c40 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.790+0000 7fa329818700 1 -- 172.21.15.28:0/1609519027 >> 172.21.15.28:0/1609519027 conn(0x7fa3240d2df0 msgr2=0x7fa3240d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:14.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.790+0000 7fa329818700 1 -- 172.21.15.28:0/1609519027 shutdown_connections 2023-11-28T11:13:14.793 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.790+0000 7fa329818700 1 -- 172.21.15.28:0/1609519027 wait complete. 2023-11-28T11:13:14.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.791+0000 7fa329818700 1 Processor -- start 2023-11-28T11:13:14.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.791+0000 7fa329818700 1 -- start start 2023-11-28T11:13:14.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa329818700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa32406bf30 0x7fa32414f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa329818700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3240d2820 0x7fa3240b67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa329818700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa32415c640 0x7fa324159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa329818700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240c8ce0 con 0x7fa32415c640 2023-11-28T11:13:14.796 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa329818700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240b5d50 con 0x7fa32406bf30 2023-11-28T11:13:14.797 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa329818700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240c8a00 con 0x7fa3240d2820 2023-11-28T11:13:14.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa3237fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa32415c640 0x7fa324159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa3237fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa32415c640 0x7fa324159a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52276/0 (socket says 172.21.15.28:52276) 2023-11-28T11:13:14.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa3237fe700 1 -- 172.21.15.28:0/1521611187 learned_addr learned my addr 172.21.15.28:0/1521611187 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:14.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa3237fe700 1 -- 172.21.15.28:0/1521611187 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3240d2820 msgr2=0x7fa3240b67f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa3227fc700 1 --2- 172.21.15.28:0/1521611187 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3240d2820 0x7fa3240b67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa3237fe700 1 --2- 172.21.15.28:0/1521611187 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3240d2820 0x7fa3240b67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa3237fe700 1 -- 172.21.15.28:0/1521611187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa32406bf30 msgr2=0x7fa32414f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa322ffd700 1 --2- 172.21.15.28:0/1521611187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa32406bf30 0x7fa32414f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa3237fe700 1 --2- 172.21.15.28:0/1521611187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa32406bf30 0x7fa32414f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.792+0000 7fa3237fe700 1 -- 172.21.15.28:0/1521611187 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa314009550 con 0x7fa32415c640 2023-11-28T11:13:14.798 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.793+0000 7fa3237fe700 1 --2- 172.21.15.28:0/1521611187 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa32415c640 0x7fa324159a80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa31c007db0 tx=0x7fa31c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:14.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.793+0000 7fa30bfff700 1 -- 172.21.15.28:0/1521611187 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c00ddd0 con 0x7fa32415c640 2023-11-28T11:13:14.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.793+0000 7fa30bfff700 1 -- 172.21.15.28:0/1521611187 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa31c00f040 con 0x7fa32415c640 2023-11-28T11:13:14.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.793+0000 7fa329818700 1 -- 172.21.15.28:0/1521611187 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3240d1710 con 0x7fa32415c640 2023-11-28T11:13:14.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.793+0000 7fa30bfff700 1 -- 172.21.15.28:0/1521611187 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa31c003c60 con 0x7fa32415c640 2023-11-28T11:13:14.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.793+0000 7fa329818700 1 -- 172.21.15.28:0/1521611187 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3240cd7c0 con 0x7fa32415c640 2023-11-28T11:13:14.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.794+0000 7fa329818700 1 -- 172.21.15.28:0/1521611187 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3240cf580 con 0x7fa32415c640 2023-11-28T11:13:14.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.794+0000 7fa30bfff700 1 -- 172.21.15.28:0/1521611187 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa31c01c030 con 0x7fa32415c640 2023-11-28T11:13:14.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.795+0000 7fa30bfff700 1 --2- 172.21.15.28:0/1521611187 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3100682f0 0x7fa31006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:14.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.795+0000 7fa322ffd700 1 --2- 172.21.15.28:0/1521611187 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3100682f0 0x7fa31006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:14.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.795+0000 7fa30bfff700 1 -- 172.21.15.28:0/1521611187 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa31c085d90 con 0x7fa32415c640 2023-11-28T11:13:14.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.795+0000 7fa322ffd700 1 --2- 172.21.15.28:0/1521611187 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3100682f0 0x7fa31006a7a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa318009930 tx=0x7fa318008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:14.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.799+0000 7fa30bfff700 1 -- 172.21.15.28:0/1521611187 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa31c052470 con 0x7fa32415c640 2023-11-28T11:13:14.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.991+0000 7fa329818700 1 -- 172.21.15.28:0/1521611187 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa3240739f0 con 0x7fa32415c640 2023-11-28T11:13:14.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.992+0000 7fa30bfff700 1 -- 172.21.15.28:0/1521611187 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa31c01a020 con 0x7fa32415c640 2023-11-28T11:13:14.993 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 32 2023-11-28T11:13:14.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.994+0000 7fa329818700 1 -- 172.21.15.28:0/1521611187 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3100682f0 msgr2=0x7fa31006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.994+0000 7fa329818700 1 --2- 172.21.15.28:0/1521611187 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3100682f0 0x7fa31006a7a0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7fa318009930 tx=0x7fa318008040 comp rx=0 tx=0).stop 2023-11-28T11:13:14.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.994+0000 7fa329818700 1 -- 172.21.15.28:0/1521611187 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa32415c640 msgr2=0x7fa324159a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:14.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.994+0000 7fa329818700 1 --2- 172.21.15.28:0/1521611187 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa32415c640 0x7fa324159a80 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7fa31c007db0 tx=0x7fa31c00d3b0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.994+0000 7fa329818700 1 -- 172.21.15.28:0/1521611187 shutdown_connections 2023-11-28T11:13:14.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.994+0000 7fa329818700 1 --2- 172.21.15.28:0/1521611187 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa3100682f0 0x7fa31006a7a0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.994+0000 7fa329818700 1 --2- 172.21.15.28:0/1521611187 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa32415c640 0x7fa324159a80 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.994+0000 7fa329818700 1 --2- 172.21.15.28:0/1521611187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa32406bf30 0x7fa32414f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.994+0000 7fa329818700 1 --2- 172.21.15.28:0/1521611187 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa3240d2820 0x7fa3240b67f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:14.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.994+0000 7fa329818700 1 -- 172.21.15.28:0/1521611187 >> 172.21.15.28:0/1521611187 conn(0x7fa3240d2df0 msgr2=0x7fa3240b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:14.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.994+0000 7fa329818700 1 -- 172.21.15.28:0/1521611187 shutdown_connections 2023-11-28T11:13:14.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:14.994+0000 7fa329818700 1 -- 172.21.15.28:0/1521611187 wait complete. 2023-11-28T11:13:15.007 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T11:13:15.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.134+0000 7f64c42e2700 1 Processor -- start 2023-11-28T11:13:15.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.134+0000 7f64c42e2700 1 -- start start 2023-11-28T11:13:15.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.135+0000 7f64c42e2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64bc0cd140 0x7f64bc06b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.135+0000 7f64c42e2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64bc06d140 0x7f64bc06b730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.135+0000 7f64c42e2700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64bc0ae730 con 0x7f64bc15a640 2023-11-28T11:13:15.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.135+0000 7f64c42e2700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64bc0cb210 con 0x7f64bc06d140 2023-11-28T11:13:15.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.135+0000 7f64c42e2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc0c3e10 con 0x7f64bc0cd140 2023-11-28T11:13:15.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.135+0000 7f64c287f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64bc06d140 0x7f64bc06b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.135+0000 7f64c187d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64bc0cd140 0x7f64bc06b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.135+0000 7f64c207e700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f64bc15a640 0x7f64bc0c3700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:51124/0 (socket says 172.21.15.28:51124) 2023-11-28T11:13:15.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.135+0000 7f64c287f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f64bc06d140 0x7f64bc06b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.28:39574/0 (socket says 172.21.15.28:39574) 2023-11-28T11:13:15.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.135+0000 7f64c207e700 1 -- 172.21.15.28:0/3036455665 learned_addr learned my addr 172.21.15.28:0/3036455665 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:15.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.136+0000 7f64c187d700 1 -- 172.21.15.28:0/3036455665 >> v2:172.21.15.59:3300/0 conn(0x7f64bc06d140 msgr2=0x7f64bc06b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:15.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.136+0000 7f64c187d700 1 --2- 172.21.15.28:0/3036455665 >> v2:172.21.15.59:3300/0 conn(0x7f64bc06d140 0x7f64bc06b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.136+0000 7f64c187d700 1 -- 172.21.15.28:0/3036455665 >> v1:172.21.15.59:6789/0 conn(0x7f64bc15a640 legacy=0x7f64bc0c3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.136+0000 7f64c187d700 1 -- 172.21.15.28:0/3036455665 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64bc06e9e0 con 0x7f64bc0cd140 2023-11-28T11:13:15.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.136+0000 7f64c107c700 1 -- 172.21.15.28:0/3036455665 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158351809 0 0) 0x7f64bc0ae730 con 0x7f64bc15a640 2023-11-28T11:13:15.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.136+0000 7f64c287f700 1 --2- 172.21.15.28:0/3036455665 >> v2:172.21.15.59:3300/0 conn(0x7f64bc06d140 0x7f64bc06b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:13:15.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.136+0000 7f64c187d700 1 --2- 172.21.15.28:0/3036455665 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64bc0cd140 0x7f64bc06b1f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009840 tx=0x7f64b8009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d8a7102e1da9a6a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:15.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.136+0000 7f64c107c700 1 -- 172.21.15.28:0/3036455665 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8011830 con 0x7f64bc0cd140 2023-11-28T11:13:15.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.136+0000 7f64c107c700 1 -- 172.21.15.28:0/3036455665 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b8011990 con 0x7f64bc0cd140 2023-11-28T11:13:15.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.136+0000 7f64c107c700 1 -- 172.21.15.28:0/3036455665 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b8011d20 con 0x7f64bc0cd140 2023-11-28T11:13:15.144 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.137+0000 7f64c42e2700 1 -- 172.21.15.28:0/3036455665 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64bc0cd140 msgr2=0x7f64bc06b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.137+0000 7f64c42e2700 1 --2- 172.21.15.28:0/3036455665 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64bc0cd140 0x7f64bc06b1f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f64b8009840 tx=0x7f64b8009b50 comp rx=0 tx=0).stop 2023-11-28T11:13:15.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.137+0000 7f64c42e2700 1 -- 172.21.15.28:0/3036455665 shutdown_connections 2023-11-28T11:13:15.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.137+0000 7f64c42e2700 1 --2- 172.21.15.28:0/3036455665 >> v2:172.21.15.59:3300/0 conn(0x7f64bc06d140 0x7f64bc06b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.145 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.137+0000 7f64c42e2700 1 --2- 172.21.15.28:0/3036455665 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64bc0cd140 0x7f64bc06b1f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.137+0000 7f64c42e2700 1 -- 172.21.15.28:0/3036455665 >> 172.21.15.28:0/3036455665 conn(0x7f64bc0cc630 msgr2=0x7f64bc0cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:15.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.137+0000 7f64c42e2700 1 -- 172.21.15.28:0/3036455665 shutdown_connections 2023-11-28T11:13:15.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.137+0000 7f64c42e2700 1 -- 172.21.15.28:0/3036455665 wait complete. 2023-11-28T11:13:15.146 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.138+0000 7f64c42e2700 1 Processor -- start 2023-11-28T11:13:15.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.138+0000 7f64c42e2700 1 -- start start 2023-11-28T11:13:15.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c42e2700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64bc06d140 0x7f64bc15b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c42e2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc0cd140 0x7f64bc14d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c42e2700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64bc15a640 0x7f64bc0b1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.147 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c42e2700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc15cfc0 con 0x7f64bc06d140 2023-11-28T11:13:15.148 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c42e2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc15aab0 con 0x7f64bc0cd140 2023-11-28T11:13:15.148 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c42e2700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc15ad90 con 0x7f64bc15a640 2023-11-28T11:13:15.148 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c207e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64bc06d140 0x7f64bc15b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.148 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c207e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64bc06d140 0x7f64bc15b590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52286/0 (socket says 172.21.15.28:52286) 2023-11-28T11:13:15.148 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c207e700 1 -- 172.21.15.28:0/3756333409 learned_addr learned my addr 172.21.15.28:0/3756333409 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:15.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c287f700 1 --2- 172.21.15.28:0/3756333409 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64bc15a640 0x7f64bc0b1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c187d700 1 --2- 172.21.15.28:0/3756333409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc0cd140 0x7f64bc14d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c207e700 1 -- 172.21.15.28:0/3756333409 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64bc15a640 msgr2=0x7f64bc0b1380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c207e700 1 --2- 172.21.15.28:0/3756333409 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64bc15a640 0x7f64bc0b1380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c207e700 1 -- 172.21.15.28:0/3756333409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc0cd140 msgr2=0x7f64bc14d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.149 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c207e700 1 --2- 172.21.15.28:0/3756333409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc0cd140 0x7f64bc14d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c207e700 1 -- 172.21.15.28:0/3756333409 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b4009a70 con 0x7f64bc06d140 2023-11-28T11:13:15.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.139+0000 7f64c207e700 1 --2- 172.21.15.28:0/3756333409 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64bc06d140 0x7f64bc15b590 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0011e0 tx=0x7f64ac00cae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:15.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.140+0000 7f64b2ffd700 1 -- 172.21.15.28:0/3756333409 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac004c80 con 0x7f64bc06d140 2023-11-28T11:13:15.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.140+0000 7f64b2ffd700 1 -- 172.21.15.28:0/3756333409 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ac004e00 con 0x7f64bc06d140 2023-11-28T11:13:15.150 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.140+0000 7f64c42e2700 1 -- 172.21.15.28:0/3756333409 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64b80094a0 con 0x7f64bc06d140 2023-11-28T11:13:15.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.140+0000 7f64b2ffd700 1 -- 172.21.15.28:0/3756333409 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ac007310 con 0x7f64bc06d140 2023-11-28T11:13:15.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.140+0000 7f64c42e2700 1 -- 172.21.15.28:0/3756333409 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64bc0aec60 con 0x7f64bc06d140 2023-11-28T11:13:15.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.141+0000 7f64c42e2700 1 -- 172.21.15.28:0/3756333409 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64bc0ae980 con 0x7f64bc06d140 2023-11-28T11:13:15.151 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.141+0000 7f64b2ffd700 1 -- 172.21.15.28:0/3756333409 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f64ac01a020 con 0x7f64bc06d140 2023-11-28T11:13:15.152 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.141+0000 7f64b2ffd700 1 --2- 172.21.15.28:0/3756333409 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f64a00682f0 0x7f64a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.141+0000 7f64b2ffd700 1 -- 172.21.15.28:0/3756333409 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f64ac085a60 con 0x7f64bc06d140 2023-11-28T11:13:15.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.141+0000 7f64c187d700 1 --2- 172.21.15.28:0/3756333409 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f64a00682f0 0x7f64a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.142+0000 7f64c187d700 1 --2- 172.21.15.28:0/3756333409 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f64a00682f0 0x7f64a006a7a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f64b8000c00 tx=0x7f64b801c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:15.153 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.146+0000 7f64b2ffd700 1 -- 172.21.15.28:0/3756333409 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f64ac052140 con 0x7f64bc06d140 2023-11-28T11:13:15.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.341+0000 7f64c42e2700 1 -- 172.21.15.28:0/3756333409 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64bc0aef40 con 0x7f64bc06d140 2023-11-28T11:13:15.342 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.342+0000 7f64b2ffd700 1 -- 172.21.15.28:0/3756333409 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v68) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64ac055760 con 0x7f64bc06d140 2023-11-28T11:13:15.343 INFO:teuthology.orchestra.run.smithi028.stdout:pg_num: 64 2023-11-28T11:13:15.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.344+0000 7f64c42e2700 1 -- 172.21.15.28:0/3756333409 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f64a00682f0 msgr2=0x7f64a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.344+0000 7f64c42e2700 1 --2- 172.21.15.28:0/3756333409 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f64a00682f0 0x7f64a006a7a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f64b8000c00 tx=0x7f64b801c040 comp rx=0 tx=0).stop 2023-11-28T11:13:15.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.344+0000 7f64c42e2700 1 -- 172.21.15.28:0/3756333409 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64bc06d140 msgr2=0x7f64bc15b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.344+0000 7f64c42e2700 1 --2- 172.21.15.28:0/3756333409 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64bc06d140 0x7f64bc15b590 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f64ac0011e0 tx=0x7f64ac00cae0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.344+0000 7f64c42e2700 1 -- 172.21.15.28:0/3756333409 shutdown_connections 2023-11-28T11:13:15.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.344+0000 7f64c42e2700 1 --2- 172.21.15.28:0/3756333409 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f64a00682f0 0x7f64a006a7a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.344+0000 7f64c42e2700 1 --2- 172.21.15.28:0/3756333409 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f64bc06d140 0x7f64bc15b590 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.345 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.344+0000 7f64c42e2700 1 --2- 172.21.15.28:0/3756333409 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f64bc15a640 0x7f64bc0b1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.344+0000 7f64c42e2700 1 --2- 172.21.15.28:0/3756333409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64bc0cd140 0x7f64bc14d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.344+0000 7f64c42e2700 1 -- 172.21.15.28:0/3756333409 >> 172.21.15.28:0/3756333409 conn(0x7f64bc0cc630 msgr2=0x7f64bc0c4980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:15.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.344+0000 7f64c42e2700 1 -- 172.21.15.28:0/3756333409 shutdown_connections 2023-11-28T11:13:15.346 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.344+0000 7f64c42e2700 1 -- 172.21.15.28:0/3756333409 wait complete. 2023-11-28T11:13:15.357 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:13:15.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.482+0000 7fa1af3e4700 1 Processor -- start 2023-11-28T11:13:15.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.482+0000 7fa1af3e4700 1 -- start start 2023-11-28T11:13:15.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.483+0000 7fa1af3e4700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fa1a815c560 0x7fa1a806f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.483+0000 7fa1af3e4700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa1a80b2660 0x7fa1a80ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.483+0000 7fa1af3e4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a80cecf0 0x7fa1a8160200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.483+0000 7fa1af3e4700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a80cf150 con 0x7fa1a80b2660 2023-11-28T11:13:15.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.483+0000 7fa1af3e4700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1a80b30b0 con 0x7fa1a815c560 2023-11-28T11:13:15.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.483+0000 7fa1af3e4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a806c020 con 0x7fa1a80cecf0 2023-11-28T11:13:15.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.483+0000 7fa1ac97f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa1a80b2660 0x7fa1a80ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.483+0000 7fa1ac97f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fa1a80b2660 0x7fa1a80ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52294/0 (socket says 172.21.15.28:52294) 2023-11-28T11:13:15.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.483+0000 7fa1ac97f700 1 -- 172.21.15.28:0/530857091 learned_addr learned my addr 172.21.15.28:0/530857091 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:15.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.483+0000 7fa1ad981700 1 --2- 172.21.15.28:0/530857091 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a80cecf0 0x7fa1a8160200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.483+0000 7fa1ad180700 1 --2- 172.21.15.28:0/530857091 >> v2:172.21.15.59:3300/0 conn(0x7fa1a815c560 0x7fa1a806f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa1ac97f700 1 -- 172.21.15.28:0/530857091 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a80cecf0 msgr2=0x7fa1a8160200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:15.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa1ac97f700 1 --2- 172.21.15.28:0/530857091 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a80cecf0 0x7fa1a8160200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa1ac97f700 1 -- 172.21.15.28:0/530857091 >> v2:172.21.15.59:3300/0 conn(0x7fa1a815c560 msgr2=0x7fa1a806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:15.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa1ac97f700 1 --2- 172.21.15.28:0/530857091 >> v2:172.21.15.59:3300/0 conn(0x7fa1a815c560 0x7fa1a806f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa1ac97f700 1 -- 172.21.15.28:0/530857091 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a80d09c0 con 0x7fa1a80b2660 2023-11-28T11:13:15.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa1ad981700 1 --2- 172.21.15.28:0/530857091 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a80cecf0 0x7fa1a8160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:15.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa1ad180700 1 --2- 172.21.15.28:0/530857091 >> v2:172.21.15.59:3300/0 conn(0x7fa1a815c560 0x7fa1a806f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:15.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa1ac97f700 1 --2- 172.21.15.28:0/530857091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1a80b2660 0x7fa1a80ce7b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8066920 tx=0x7fa1a4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12605b529ec0838 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:15.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa19ffff700 1 -- 172.21.15.28:0/530857091 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a400b870 con 0x7fa1a80b2660 2023-11-28T11:13:15.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa19ffff700 1 -- 172.21.15.28:0/530857091 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a400b9d0 con 0x7fa1a80b2660 2023-11-28T11:13:15.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa19ffff700 1 -- 172.21.15.28:0/530857091 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a400bd60 con 0x7fa1a80b2660 2023-11-28T11:13:15.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa1af3e4700 1 -- 172.21.15.28:0/530857091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1a80b2660 msgr2=0x7fa1a80ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa1af3e4700 1 --2- 172.21.15.28:0/530857091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1a80b2660 0x7fa1a80ce7b0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7fa1a8066920 tx=0x7fa1a4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa1af3e4700 1 -- 172.21.15.28:0/530857091 shutdown_connections 2023-11-28T11:13:15.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa1af3e4700 1 --2- 172.21.15.28:0/530857091 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a80cecf0 0x7fa1a8160200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa1af3e4700 1 --2- 172.21.15.28:0/530857091 >> v2:172.21.15.59:3300/0 conn(0x7fa1a815c560 0x7fa1a806f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa1af3e4700 1 --2- 172.21.15.28:0/530857091 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1a80b2660 0x7fa1a80ce7b0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.487 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.484+0000 7fa1af3e4700 1 -- 172.21.15.28:0/530857091 >> 172.21.15.28:0/530857091 conn(0x7fa1a80cd660 msgr2=0x7fa1a80cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:15.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.485+0000 7fa1af3e4700 1 -- 172.21.15.28:0/530857091 shutdown_connections 2023-11-28T11:13:15.488 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.485+0000 7fa1af3e4700 1 -- 172.21.15.28:0/530857091 wait complete. 2023-11-28T11:13:15.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.486+0000 7fa1af3e4700 1 Processor -- start 2023-11-28T11:13:15.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.486+0000 7fa1af3e4700 1 -- start start 2023-11-28T11:13:15.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.486+0000 7fa1af3e4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a80b2660 0x7fa1a814ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.486+0000 7fa1af3e4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1a80cecf0 0x7fa1a815c930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.486+0000 7fa1af3e4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a815c560 0x7fa1a80b5d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.486+0000 7fa1af3e4700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a8157210 con 0x7fa1a80cecf0 2023-11-28T11:13:15.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.486+0000 7fa1af3e4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a815f6f0 con 0x7fa1a815c560 2023-11-28T11:13:15.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.486+0000 7fa1af3e4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1a815f9d0 con 0x7fa1a80b2660 2023-11-28T11:13:15.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa1ac97f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1a80cecf0 0x7fa1a815c930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa1ac97f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1a80cecf0 0x7fa1a815c930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52302/0 (socket says 172.21.15.28:52302) 2023-11-28T11:13:15.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa1ac97f700 1 -- 172.21.15.28:0/991501549 learned_addr learned my addr 172.21.15.28:0/991501549 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:15.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa1ad180700 1 --2- 172.21.15.28:0/991501549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a80b2660 0x7fa1a814ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa1ad981700 1 --2- 172.21.15.28:0/991501549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a815c560 0x7fa1a80b5d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa1ac97f700 1 -- 172.21.15.28:0/991501549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a80b2660 msgr2=0x7fa1a814ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa1ac97f700 1 --2- 172.21.15.28:0/991501549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a80b2660 0x7fa1a814ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa1ac97f700 1 -- 172.21.15.28:0/991501549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a815c560 msgr2=0x7fa1a80b5d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa1ac97f700 1 --2- 172.21.15.28:0/991501549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a815c560 0x7fa1a80b5d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa1ac97f700 1 -- 172.21.15.28:0/991501549 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1a4009550 con 0x7fa1a80cecf0 2023-11-28T11:13:15.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa1ac97f700 1 --2- 172.21.15.28:0/991501549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1a80cecf0 0x7fa1a815c930 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4000c00 tx=0x7fa1a4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:15.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa19dffb700 1 -- 172.21.15.28:0/991501549 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4004480 con 0x7fa1a80cecf0 2023-11-28T11:13:15.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa19dffb700 1 -- 172.21.15.28:0/991501549 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1a40045e0 con 0x7fa1a80cecf0 2023-11-28T11:13:15.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa1af3e4700 1 -- 172.21.15.28:0/991501549 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1a80b2af0 con 0x7fa1a80cecf0 2023-11-28T11:13:15.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa19dffb700 1 -- 172.21.15.28:0/991501549 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1a40038e0 con 0x7fa1a80cecf0 2023-11-28T11:13:15.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.487+0000 7fa1af3e4700 1 -- 172.21.15.28:0/991501549 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1a806fdf0 con 0x7fa1a80cecf0 2023-11-28T11:13:15.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.488+0000 7fa1af3e4700 1 -- 172.21.15.28:0/991501549 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1a80b2dd0 con 0x7fa1a80cecf0 2023-11-28T11:13:15.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.489+0000 7fa19dffb700 1 -- 172.21.15.28:0/991501549 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fa1a400f050 con 0x7fa1a80cecf0 2023-11-28T11:13:15.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.489+0000 7fa19dffb700 1 --2- 172.21.15.28:0/991501549 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa18c0682f0 0x7fa18c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.489+0000 7fa19dffb700 1 -- 172.21.15.28:0/991501549 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa1a4087590 con 0x7fa1a80cecf0 2023-11-28T11:13:15.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.489+0000 7fa1ad180700 1 --2- 172.21.15.28:0/991501549 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa18c0682f0 0x7fa18c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.490+0000 7fa1ad180700 1 --2- 172.21.15.28:0/991501549 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa18c0682f0 0x7fa18c06a7a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa19800c940 tx=0x7fa198008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:15.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.494+0000 7fa19dffb700 1 -- 172.21.15.28:0/991501549 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fa1a4053c70 con 0x7fa1a80cecf0 2023-11-28T11:13:15.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.731+0000 7fa1af3e4700 1 -- 172.21.15.28:0/991501549 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa1a80675a0 con 0x7fa1a80cecf0 2023-11-28T11:13:15.732 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.731+0000 7fa19dffb700 1 -- 172.21.15.28:0/991501549 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7fa1a4057290 con 0x7fa1a80cecf0 2023-11-28T11:13:15.732 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:15.732 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1564028052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1564028052},{"type":"v1","addr":"172.21.15.59:6835","nonce":1564028052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6558,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2509041002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2509041002},{"type":"v1","addr":"172.21.15.28:6836","nonce":2509041002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6561,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3994419079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3994419079},{"type":"v1","addr":"172.21.15.28:6837","nonce":3994419079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:12:03.800884+0000","modified":"2023-11-28T11:12:20.278498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.59:6834/2406852565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2406852565},{"type":"v1","addr":"172.21.15.59:6834","nonce":2406852565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:13:15.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.733+0000 7fa1af3e4700 1 -- 172.21.15.28:0/991501549 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa18c0682f0 msgr2=0x7fa18c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.733+0000 7fa1af3e4700 1 --2- 172.21.15.28:0/991501549 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa18c0682f0 0x7fa18c06a7a0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fa19800c940 tx=0x7fa198008040 comp rx=0 tx=0).stop 2023-11-28T11:13:15.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.733+0000 7fa1af3e4700 1 -- 172.21.15.28:0/991501549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1a80cecf0 msgr2=0x7fa1a815c930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.733+0000 7fa1af3e4700 1 --2- 172.21.15.28:0/991501549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1a80cecf0 0x7fa1a815c930 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fa1a4000c00 tx=0x7fa1a4004750 comp rx=0 tx=0).stop 2023-11-28T11:13:15.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.734+0000 7fa1af3e4700 1 -- 172.21.15.28:0/991501549 shutdown_connections 2023-11-28T11:13:15.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.734+0000 7fa1af3e4700 1 --2- 172.21.15.28:0/991501549 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fa18c0682f0 0x7fa18c06a7a0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.734+0000 7fa1af3e4700 1 --2- 172.21.15.28:0/991501549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a815c560 0x7fa1a80b5d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.734+0000 7fa1af3e4700 1 --2- 172.21.15.28:0/991501549 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fa1a80b2660 0x7fa1a814ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.734+0000 7fa1af3e4700 1 --2- 172.21.15.28:0/991501549 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fa1a80cecf0 0x7fa1a815c930 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.734+0000 7fa1af3e4700 1 -- 172.21.15.28:0/991501549 >> 172.21.15.28:0/991501549 conn(0x7fa1a80cd660 msgr2=0x7fa1a80b1c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:15.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.734+0000 7fa1af3e4700 1 -- 172.21.15.28:0/991501549 shutdown_connections 2023-11-28T11:13:15.735 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.734+0000 7fa1af3e4700 1 -- 172.21.15.28:0/991501549 wait complete. 2023-11-28T11:13:15.736 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 34 2023-11-28T11:13:15.746 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:13:15.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.873+0000 7fb391317700 1 Processor -- start 2023-11-28T11:13:15.873 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.873+0000 7fb391317700 1 -- start start 2023-11-28T11:13:15.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.873+0000 7fb391317700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb38c15a6c0 0x7fb38c0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.873+0000 7fb391317700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38c0cfb90 con 0x7fb38c0b2660 2023-11-28T11:13:15.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.873+0000 7fb391317700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38c0cb930 con 0x7fb38c0d0020 2023-11-28T11:13:15.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.873+0000 7fb391317700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0cf260 con 0x7fb38c15a6c0 2023-11-28T11:13:15.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb38b7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb38c0b2660 0x7fb38c0b3a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40370/0 (socket says 172.21.15.28:40370) 2023-11-28T11:13:15.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb38b7fe700 1 -- 172.21.15.28:0/3330701014 learned_addr learned my addr 172.21.15.28:0/3330701014 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:15.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb38affd700 1 --2- 172.21.15.28:0/3330701014 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb38c15a6c0 0x7fb38c0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb389ffb700 1 -- 172.21.15.28:0/3330701014 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2158604709 0 0) 0x7fb38c0cfb90 con 0x7fb38c0b2660 2023-11-28T11:13:15.877 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb389ffb700 1 -- 172.21.15.28:0/3330701014 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3780035b0 con 0x7fb38c0b2660 2023-11-28T11:13:15.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb389ffb700 1 -- 172.21.15.28:0/3330701014 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3256067177 0 0) 0x7fb3780035b0 con 0x7fb38c0b2660 2023-11-28T11:13:15.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb389ffb700 1 -- 172.21.15.28:0/3330701014 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb38c15a6c0 msgr2=0x7fb38c0cb230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:15.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb389ffb700 1 --2- 172.21.15.28:0/3330701014 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb38c15a6c0 0x7fb38c0cb230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb389ffb700 1 -- 172.21.15.28:0/3330701014 >> v1:172.21.15.59:6789/0 conn(0x7fb38c0d0020 legacy=0x7fb38c0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb389ffb700 1 -- 172.21.15.28:0/3330701014 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c0cf540 con 0x7fb38c0b2660 2023-11-28T11:13:15.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb389ffb700 1 -- 172.21.15.28:0/3330701014 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fb3840019f0 con 0x7fb38c0b2660 2023-11-28T11:13:15.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb389ffb700 1 -- 172.21.15.28:0/3330701014 >> v1:172.21.15.28:6789/0 conn(0x7fb38c0b2660 legacy=0x7fb38c0b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb389ffb700 1 --2- 172.21.15.28:0/3330701014 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb38c15a6c0 0x7fb378003a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb389ffb700 1 --2- 172.21.15.28:0/3330701014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb378003f70 0x7fb378004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb389ffb700 1 --2- 172.21.15.28:0/3330701014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb378004d90 0x7fb378005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb389ffb700 1 -- 172.21.15.28:0/3330701014 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c0cf260 con 0x7fb378003f70 2023-11-28T11:13:15.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb389ffb700 1 -- 172.21.15.28:0/3330701014 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb37800ba40 con 0x7fb378004d90 2023-11-28T11:13:15.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb389ffb700 1 -- 172.21.15.28:0/3330701014 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb37800bc00 con 0x7fb38c15a6c0 2023-11-28T11:13:15.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.874+0000 7fb38a7fc700 1 --2- 172.21.15.28:0/3330701014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb378003f70 0x7fb378004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.875+0000 7fb38affd700 1 --2- 172.21.15.28:0/3330701014 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb38c15a6c0 0x7fb378003a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.875+0000 7fb38b7fe700 1 --2- 172.21.15.28:0/3330701014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb378004d90 0x7fb378005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.875+0000 7fb38a7fc700 1 -- 172.21.15.28:0/3330701014 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb38c15a6c0 msgr2=0x7fb378003a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:15.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.875+0000 7fb38a7fc700 1 --2- 172.21.15.28:0/3330701014 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb38c15a6c0 0x7fb378003a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.875+0000 7fb38a7fc700 1 -- 172.21.15.28:0/3330701014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb378004d90 msgr2=0x7fb378005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:15.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.875+0000 7fb38a7fc700 1 --2- 172.21.15.28:0/3330701014 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb378004d90 0x7fb378005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.875+0000 7fb38a7fc700 1 -- 172.21.15.28:0/3330701014 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3780035b0 con 0x7fb378003f70 2023-11-28T11:13:15.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.875+0000 7fb38a7fc700 1 --2- 172.21.15.28:0/3330701014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb378003f70 0x7fb378004850 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb3740018d0 tx=0x7fb378005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53ea8c8947b858ec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:15.879 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.875+0000 7fb389ffb700 1 -- 172.21.15.28:0/3330701014 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37400c690 con 0x7fb378003f70 2023-11-28T11:13:15.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.875+0000 7fb389ffb700 1 -- 172.21.15.28:0/3330701014 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37400c7f0 con 0x7fb378003f70 2023-11-28T11:13:15.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.875+0000 7fb389ffb700 1 -- 172.21.15.28:0/3330701014 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37400cba0 con 0x7fb378003f70 2023-11-28T11:13:15.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.876+0000 7fb391317700 1 -- 172.21.15.28:0/3330701014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb378003f70 msgr2=0x7fb378004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.876+0000 7fb391317700 1 --2- 172.21.15.28:0/3330701014 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb378003f70 0x7fb378004850 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7fb3740018d0 tx=0x7fb378005880 comp rx=0 tx=0).stop 2023-11-28T11:13:15.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.876+0000 7fb38b7fe700 1 -- 172.21.15.28:0/3330701014 reap_dead start 2023-11-28T11:13:15.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.876+0000 7fb391317700 1 -- 172.21.15.28:0/3330701014 shutdown_connections 2023-11-28T11:13:15.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.876+0000 7fb391317700 1 -- 172.21.15.28:0/3330701014 >> 172.21.15.28:0/3330701014 conn(0x7fb38c0cd660 msgr2=0x7fb38c0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:15.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.876+0000 7fb391317700 1 -- 172.21.15.28:0/3330701014 shutdown_connections 2023-11-28T11:13:15.880 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.876+0000 7fb391317700 1 -- 172.21.15.28:0/3330701014 wait complete. 2023-11-28T11:13:15.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.877+0000 7fb391317700 1 Processor -- start 2023-11-28T11:13:15.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.877+0000 7fb391317700 1 -- start start 2023-11-28T11:13:15.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb391317700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb378003f70 0x7fb38c1ef590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb391317700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb38c0bb580 0x7fb38c0c7fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb391317700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb38c0c8500 0x7fb38c1574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb391317700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c156d10 con 0x7fb38c0bb580 2023-11-28T11:13:15.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb391317700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c15ab30 con 0x7fb378003f70 2023-11-28T11:13:15.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb391317700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38c15ae10 con 0x7fb38c0c8500 2023-11-28T11:13:15.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb38a7fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb38c0bb580 0x7fb38c0c7fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb38a7fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb38c0bb580 0x7fb38c0c7fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52320/0 (socket says 172.21.15.28:52320) 2023-11-28T11:13:15.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb38a7fc700 1 -- 172.21.15.28:0/1089064837 learned_addr learned my addr 172.21.15.28:0/1089064837 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:15.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb38affd700 1 --2- 172.21.15.28:0/1089064837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb378003f70 0x7fb38c1ef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb38b7fe700 1 --2- 172.21.15.28:0/1089064837 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb38c0c8500 0x7fb38c1574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb38a7fc700 1 -- 172.21.15.28:0/1089064837 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb38c0c8500 msgr2=0x7fb38c1574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb38a7fc700 1 --2- 172.21.15.28:0/1089064837 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb38c0c8500 0x7fb38c1574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb38a7fc700 1 -- 172.21.15.28:0/1089064837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb378003f70 msgr2=0x7fb38c1ef590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:15.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb38a7fc700 1 --2- 172.21.15.28:0/1089064837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb378003f70 0x7fb38c1ef590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:15.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb38a7fc700 1 -- 172.21.15.28:0/1089064837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38c0d0470 con 0x7fb38c0bb580 2023-11-28T11:13:15.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb38affd700 1 --2- 172.21.15.28:0/1089064837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb378003f70 0x7fb38c1ef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:15.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb38a7fc700 1 --2- 172.21.15.28:0/1089064837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb38c0bb580 0x7fb38c0c7fc0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fb37800ba10 tx=0x7fb374001230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:15.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.878+0000 7fb3837fe700 1 -- 172.21.15.28:0/1089064837 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37400cb60 con 0x7fb38c0bb580 2023-11-28T11:13:15.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.879+0000 7fb3837fe700 1 -- 172.21.15.28:0/1089064837 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb374021070 con 0x7fb38c0bb580 2023-11-28T11:13:15.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.879+0000 7fb391317700 1 -- 172.21.15.28:0/1089064837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb38c0cf820 con 0x7fb38c0bb580 2023-11-28T11:13:15.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.879+0000 7fb3837fe700 1 -- 172.21.15.28:0/1089064837 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb374001330 con 0x7fb38c0bb580 2023-11-28T11:13:15.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.879+0000 7fb391317700 1 -- 172.21.15.28:0/1089064837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb38c0d0600 con 0x7fb38c0bb580 2023-11-28T11:13:15.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.879+0000 7fb391317700 1 -- 172.21.15.28:0/1089064837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb38c0675a0 con 0x7fb38c0bb580 2023-11-28T11:13:15.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.880+0000 7fb3837fe700 1 -- 172.21.15.28:0/1089064837 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb374010050 con 0x7fb38c0bb580 2023-11-28T11:13:15.885 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.880+0000 7fb3837fe700 1 --2- 172.21.15.28:0/1089064837 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb36c0682f0 0x7fb36c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:15.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.880+0000 7fb3837fe700 1 -- 172.21.15.28:0/1089064837 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fb374086fe0 con 0x7fb38c0bb580 2023-11-28T11:13:15.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.880+0000 7fb38affd700 1 --2- 172.21.15.28:0/1089064837 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb36c0682f0 0x7fb36c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:15.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.881+0000 7fb38affd700 1 --2- 172.21.15.28:0/1089064837 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb36c0682f0 0x7fb36c06a7a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009c60 tx=0x7fb37c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:15.886 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:15.885+0000 7fb3837fe700 1 -- 172.21.15.28:0/1089064837 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fb3740536c0 con 0x7fb38c0bb580 2023-11-28T11:13:16.082 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.081+0000 7fb391317700 1 -- 172.21.15.28:0/1089064837 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb38c0b2ba0 con 0x7fb38c0bb580 2023-11-28T11:13:16.083 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.083+0000 7fb3837fe700 1 -- 172.21.15.28:0/1089064837 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18174 (secure 0 0 0) 0x7fb374056ce0 con 0x7fb38c0bb580 2023-11-28T11:13:16.083 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:16.084 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":68,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:12:35.277297+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:12:01.724396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:12:02.776612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.9","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:13:16.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.085+0000 7fb391317700 1 -- 172.21.15.28:0/1089064837 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb36c0682f0 msgr2=0x7fb36c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:16.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.085+0000 7fb391317700 1 --2- 172.21.15.28:0/1089064837 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb36c0682f0 0x7fb36c06a7a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fb37c009c60 tx=0x7fb37c008040 comp rx=0 tx=0).stop 2023-11-28T11:13:16.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.085+0000 7fb391317700 1 -- 172.21.15.28:0/1089064837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb38c0bb580 msgr2=0x7fb38c0c7fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:16.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.085+0000 7fb391317700 1 --2- 172.21.15.28:0/1089064837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb38c0bb580 0x7fb38c0c7fc0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7fb37800ba10 tx=0x7fb374001230 comp rx=0 tx=0).stop 2023-11-28T11:13:16.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.085+0000 7fb391317700 1 -- 172.21.15.28:0/1089064837 shutdown_connections 2023-11-28T11:13:16.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.085+0000 7fb391317700 1 --2- 172.21.15.28:0/1089064837 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb36c0682f0 0x7fb36c06a7a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.085+0000 7fb391317700 1 --2- 172.21.15.28:0/1089064837 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb38c0c8500 0x7fb38c1574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.085+0000 7fb391317700 1 --2- 172.21.15.28:0/1089064837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb378003f70 0x7fb38c1ef590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.086 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.085+0000 7fb391317700 1 --2- 172.21.15.28:0/1089064837 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb38c0bb580 0x7fb38c0c7fc0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.085+0000 7fb391317700 1 -- 172.21.15.28:0/1089064837 >> 172.21.15.28:0/1089064837 conn(0x7fb38c0cd660 msgr2=0x7fb38c0cbb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:16.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.085+0000 7fb391317700 1 -- 172.21.15.28:0/1089064837 shutdown_connections 2023-11-28T11:13:16.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.085+0000 7fb391317700 1 -- 172.21.15.28:0/1089064837 wait complete. 2023-11-28T11:13:16.098 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:13:16.099 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:13:16.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.223+0000 7f471e26c700 1 Processor -- start 2023-11-28T11:13:16.224 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.223+0000 7f471e26c700 1 -- start start 2023-11-28T11:13:16.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.224+0000 7f471e26c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f471806e400 0x7f471815db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:16.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.224+0000 7f471e26c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f47180ce890 0x7f47180ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:16.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.224+0000 7f471e26c700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471815e0b0 con 0x7f4718159590 2023-11-28T11:13:16.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.224+0000 7f471e26c700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47180cf250 con 0x7f47180ce890 2023-11-28T11:13:16.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.224+0000 7f471e26c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f471815d120 con 0x7f471806e400 2023-11-28T11:13:16.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.224+0000 7f471c809700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f47180ce890 0x7f47180ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:16.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.224+0000 7f4717fff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f4718159590 0x7f47180ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40386/0 (socket says 172.21.15.28:40386) 2023-11-28T11:13:16.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.224+0000 7f4717fff700 1 -- 172.21.15.28:0/57802512 learned_addr learned my addr 172.21.15.28:0/57802512 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:16.227 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.225+0000 7f47177fe700 1 --2- 172.21.15.28:0/57802512 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f471806e400 0x7f471815db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:16.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.225+0000 7f471c809700 1 -- 172.21.15.28:0/57802512 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f471806e400 msgr2=0x7f471815db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:16.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.225+0000 7f471c809700 1 --2- 172.21.15.28:0/57802512 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f471806e400 0x7f471815db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.225+0000 7f471c809700 1 -- 172.21.15.28:0/57802512 >> v1:172.21.15.28:6789/0 conn(0x7f4718159590 legacy=0x7f47180ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:16.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.225+0000 7f471c809700 1 -- 172.21.15.28:0/57802512 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f471815d6e0 con 0x7f47180ce890 2023-11-28T11:13:16.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.225+0000 7f4716ffd700 1 -- 172.21.15.28:0/57802512 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3758297860 0 0) 0x7f4708001110 con 0x7f4718159590 2023-11-28T11:13:16.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.225+0000 7f471c809700 1 --2- 172.21.15.28:0/57802512 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47180ce890 0x7f47180ced10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f4710009d90 tx=0x7f471000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f451425f3477dada server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:16.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.225+0000 7f4716ffd700 1 -- 172.21.15.28:0/57802512 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f471000e070 con 0x7f47180ce890 2023-11-28T11:13:16.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.225+0000 7f4716ffd700 1 -- 172.21.15.28:0/57802512 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f471000bbd0 con 0x7f47180ce890 2023-11-28T11:13:16.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.225+0000 7f4716ffd700 1 -- 172.21.15.28:0/57802512 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710012430 con 0x7f47180ce890 2023-11-28T11:13:16.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.226+0000 7f471e26c700 1 -- 172.21.15.28:0/57802512 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47180ce890 msgr2=0x7f47180ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:16.228 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.226+0000 7f471e26c700 1 --2- 172.21.15.28:0/57802512 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47180ce890 0x7f47180ced10 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f4710009d90 tx=0x7f471000b330 comp rx=0 tx=0).stop 2023-11-28T11:13:16.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.226+0000 7f471e26c700 1 -- 172.21.15.28:0/57802512 shutdown_connections 2023-11-28T11:13:16.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.226+0000 7f471e26c700 1 --2- 172.21.15.28:0/57802512 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f47180ce890 0x7f47180ced10 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.226+0000 7f471e26c700 1 --2- 172.21.15.28:0/57802512 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f471806e400 0x7f471815db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.226+0000 7f471e26c700 1 -- 172.21.15.28:0/57802512 >> 172.21.15.28:0/57802512 conn(0x7f47180b1650 msgr2=0x7f47180b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:16.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.226+0000 7f471e26c700 1 -- 172.21.15.28:0/57802512 shutdown_connections 2023-11-28T11:13:16.229 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.226+0000 7f471e26c700 1 -- 172.21.15.28:0/57802512 wait complete. 2023-11-28T11:13:16.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.227+0000 7f471e26c700 1 Processor -- start 2023-11-28T11:13:16.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.227+0000 7f471e26c700 1 -- start start 2023-11-28T11:13:16.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f471e26c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f471806e400 0x7f47180ac520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:16.231 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f471e26c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47180ce890 0x7f47180c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f471e26c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4718159590 0x7f47180ad1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f471e26c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47180ae0a0 con 0x7f4718159590 2023-11-28T11:13:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f471e26c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47180aca60 con 0x7f471806e400 2023-11-28T11:13:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f471e26c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4718158740 con 0x7f47180ce890 2023-11-28T11:13:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f471c809700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4718159590 0x7f47180ad1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f471c809700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4718159590 0x7f47180ad1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52338/0 (socket says 172.21.15.28:52338) 2023-11-28T11:13:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f471c809700 1 -- 172.21.15.28:0/4086649307 learned_addr learned my addr 172.21.15.28:0/4086649307 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f471c809700 1 -- 172.21.15.28:0/4086649307 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47180ce890 msgr2=0x7f47180c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f471c809700 1 --2- 172.21.15.28:0/4086649307 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47180ce890 0x7f47180c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f471c809700 1 -- 172.21.15.28:0/4086649307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f471806e400 msgr2=0x7f47180ac520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f471c809700 1 --2- 172.21.15.28:0/4086649307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f471806e400 0x7f47180ac520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.232 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f471c809700 1 -- 172.21.15.28:0/4086649307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4710009a70 con 0x7f4718159590 2023-11-28T11:13:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f471c809700 1 --2- 172.21.15.28:0/4086649307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4718159590 0x7f47180ad1e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f471000b370 tx=0x7f4710004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f4714ff9700 1 -- 172.21.15.28:0/4086649307 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710012430 con 0x7f4718159590 2023-11-28T11:13:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.228+0000 7f4714ff9700 1 -- 172.21.15.28:0/4086649307 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4710020070 con 0x7f4718159590 2023-11-28T11:13:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.229+0000 7f471e26c700 1 -- 172.21.15.28:0/4086649307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47180af240 con 0x7f4718159590 2023-11-28T11:13:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.229+0000 7f4714ff9700 1 -- 172.21.15.28:0/4086649307 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4710004260 con 0x7f4718159590 2023-11-28T11:13:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.229+0000 7f471e26c700 1 -- 172.21.15.28:0/4086649307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47180aef60 con 0x7f4718159590 2023-11-28T11:13:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.229+0000 7f471e26c700 1 -- 172.21.15.28:0/4086649307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471815d370 con 0x7f4718159590 2023-11-28T11:13:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.230+0000 7f4714ff9700 1 -- 172.21.15.28:0/4086649307 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f471002e070 con 0x7f4718159590 2023-11-28T11:13:16.233 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.230+0000 7f4714ff9700 1 --2- 172.21.15.28:0/4086649307 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f47040682f0 0x7f470406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:16.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.230+0000 7f4714ff9700 1 -- 172.21.15.28:0/4086649307 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f4710022070 con 0x7f4718159590 2023-11-28T11:13:16.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.230+0000 7f4717fff700 1 --2- 172.21.15.28:0/4086649307 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f47040682f0 0x7f470406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:16.234 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.231+0000 7f4717fff700 1 --2- 172.21.15.28:0/4086649307 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f47040682f0 0x7f470406a7a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4708001610 tx=0x7f47080079f0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:16.235 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.235+0000 7f4714ff9700 1 -- 172.21.15.28:0/4086649307 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f4710054390 con 0x7f4718159590 2023-11-28T11:13:16.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.473+0000 7f471e26c700 1 -- 172.21.15.28:0/4086649307 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47180ab8c0 con 0x7f4718159590 2023-11-28T11:13:16.474 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.474+0000 7f4714ff9700 1 -- 172.21.15.28:0/4086649307 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f47100579b0 con 0x7f4718159590 2023-11-28T11:13:16.474 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:16.475 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1564028052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1564028052},{"type":"v1","addr":"172.21.15.59:6835","nonce":1564028052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6558,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2509041002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2509041002},{"type":"v1","addr":"172.21.15.28:6836","nonce":2509041002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6561,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3994419079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3994419079},{"type":"v1","addr":"172.21.15.28:6837","nonce":3994419079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:12:03.800884+0000","modified":"2023-11-28T11:12:20.278498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.59:6834/2406852565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2406852565},{"type":"v1","addr":"172.21.15.59:6834","nonce":2406852565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:13:16.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.476+0000 7f471e26c700 1 -- 172.21.15.28:0/4086649307 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f47040682f0 msgr2=0x7f470406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:16.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.476+0000 7f471e26c700 1 --2- 172.21.15.28:0/4086649307 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f47040682f0 0x7f470406a7a0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f4708001610 tx=0x7f47080079f0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.476+0000 7f471e26c700 1 -- 172.21.15.28:0/4086649307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4718159590 msgr2=0x7f47180ad1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:16.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.476+0000 7f471e26c700 1 --2- 172.21.15.28:0/4086649307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4718159590 0x7f47180ad1e0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f471000b370 tx=0x7f4710004ad0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.476+0000 7f471e26c700 1 -- 172.21.15.28:0/4086649307 shutdown_connections 2023-11-28T11:13:16.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.476+0000 7f471e26c700 1 --2- 172.21.15.28:0/4086649307 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f47040682f0 0x7f470406a7a0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.476+0000 7f471e26c700 1 --2- 172.21.15.28:0/4086649307 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f4718159590 0x7f47180ad1e0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.476+0000 7f471e26c700 1 --2- 172.21.15.28:0/4086649307 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f471806e400 0x7f47180ac520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.476+0000 7f471e26c700 1 --2- 172.21.15.28:0/4086649307 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f47180ce890 0x7f47180c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.476+0000 7f471e26c700 1 -- 172.21.15.28:0/4086649307 >> 172.21.15.28:0/4086649307 conn(0x7f47180b1650 msgr2=0x7f471806d8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:16.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.476+0000 7f471e26c700 1 -- 172.21.15.28:0/4086649307 shutdown_connections 2023-11-28T11:13:16.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.476+0000 7f471e26c700 1 -- 172.21.15.28:0/4086649307 wait complete. 2023-11-28T11:13:16.478 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 34 2023-11-28T11:13:16.488 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:13:16.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.612+0000 7f2ecc81f700 1 Processor -- start 2023-11-28T11:13:16.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.612+0000 7f2ecc81f700 1 -- start start 2023-11-28T11:13:16.615 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.613+0000 7f2ecc81f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2ec80cb230 0x7f2ec80ccea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:16.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.613+0000 7f2ecc81f700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec80cfb90 con 0x7f2ec815a6c0 2023-11-28T11:13:16.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.613+0000 7f2ecc81f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec80cb680 con 0x7f2ec80b2660 2023-11-28T11:13:16.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.613+0000 7f2ecc81f700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec80cb930 con 0x7f2ec80cb230 2023-11-28T11:13:16.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.613+0000 7f2ec5d9b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f2ec80cb230 0x7f2ec80ccea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:16.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.613+0000 7f2ec659c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f2ec815a6c0 0x7f2ec80d0020 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40400/0 (socket says 172.21.15.28:40400) 2023-11-28T11:13:16.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.613+0000 7f2ec659c700 1 -- 172.21.15.28:0/2513957039 learned_addr learned my addr 172.21.15.28:0/2513957039 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:16.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.614+0000 7f2ec5d9b700 1 -- 172.21.15.28:0/2513957039 >> v1:172.21.15.59:6789/0 conn(0x7f2ec80b2660 legacy=0x7f2ec80b3a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:16.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.614+0000 7f2ec5d9b700 1 -- 172.21.15.28:0/2513957039 >> v1:172.21.15.28:6789/0 conn(0x7f2ec815a6c0 legacy=0x7f2ec80d0020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:16.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.614+0000 7f2ec559a700 1 -- 172.21.15.28:0/2513957039 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 27409271 0 0) 0x7f2ec80cfb90 con 0x7f2ec815a6c0 2023-11-28T11:13:16.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.614+0000 7f2ec5d9b700 1 -- 172.21.15.28:0/2513957039 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec80cf260 con 0x7f2ec80cb230 2023-11-28T11:13:16.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.614+0000 7f2ec5d9b700 1 --2- 172.21.15.28:0/2513957039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec80cb230 0x7f2ec80ccea0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009840 tx=0x7f2eb8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7143c31a6a97361 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:16.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.614+0000 7f2ec559a700 1 -- 172.21.15.28:0/2513957039 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8004230 con 0x7f2ec80cb230 2023-11-28T11:13:16.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.614+0000 7f2ec559a700 1 -- 172.21.15.28:0/2513957039 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb8004390 con 0x7f2ec80cb230 2023-11-28T11:13:16.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.614+0000 7f2ec559a700 1 -- 172.21.15.28:0/2513957039 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80036a0 con 0x7f2ec80cb230 2023-11-28T11:13:16.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.615+0000 7f2ecc81f700 1 -- 172.21.15.28:0/2513957039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec80cb230 msgr2=0x7f2ec80ccea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:16.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.615+0000 7f2ecc81f700 1 --2- 172.21.15.28:0/2513957039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec80cb230 0x7f2ec80ccea0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f2eb8009840 tx=0x7f2eb8009b50 comp rx=0 tx=0).stop 2023-11-28T11:13:16.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.615+0000 7f2ecc81f700 1 -- 172.21.15.28:0/2513957039 shutdown_connections 2023-11-28T11:13:16.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.615+0000 7f2ecc81f700 1 --2- 172.21.15.28:0/2513957039 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec80cb230 0x7f2ec80ccea0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.615+0000 7f2ecc81f700 1 -- 172.21.15.28:0/2513957039 >> 172.21.15.28:0/2513957039 conn(0x7f2ec80cd660 msgr2=0x7f2ec80cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:16.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.615+0000 7f2ecc81f700 1 -- 172.21.15.28:0/2513957039 shutdown_connections 2023-11-28T11:13:16.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.615+0000 7f2ecc81f700 1 -- 172.21.15.28:0/2513957039 wait complete. 2023-11-28T11:13:16.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.616+0000 7f2ecc81f700 1 Processor -- start 2023-11-28T11:13:16.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.616+0000 7f2ecc81f700 1 -- start start 2023-11-28T11:13:16.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.617+0000 7f2ecc81f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec80b2660 0x7f2ec815bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:16.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.617+0000 7f2ecc81f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ec80cb230 0x7f2ec81544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:16.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.617+0000 7f2ecc81f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ec815a6c0 0x7f2ec80c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:16.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.617+0000 7f2ecc81f700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec815aa90 con 0x7f2ec80b2660 2023-11-28T11:13:16.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.617+0000 7f2ecc81f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec815ace0 con 0x7f2ec80cb230 2023-11-28T11:13:16.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.617+0000 7f2ecc81f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec815afc0 con 0x7f2ec815a6c0 2023-11-28T11:13:16.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.617+0000 7f2ec659c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec80b2660 0x7f2ec815bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:16.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.617+0000 7f2ec659c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec80b2660 0x7f2ec815bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52348/0 (socket says 172.21.15.28:52348) 2023-11-28T11:13:16.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.617+0000 7f2ec659c700 1 -- 172.21.15.28:0/1942258912 learned_addr learned my addr 172.21.15.28:0/1942258912 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:16.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.617+0000 7f2ec659c700 1 -- 172.21.15.28:0/1942258912 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ec815a6c0 msgr2=0x7f2ec80c9cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:16.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.617+0000 7f2ec659c700 1 --2- 172.21.15.28:0/1942258912 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ec815a6c0 0x7f2ec80c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.617+0000 7f2ec659c700 1 -- 172.21.15.28:0/1942258912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ec80cb230 msgr2=0x7f2ec81544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:16.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.617+0000 7f2ec659c700 1 --2- 172.21.15.28:0/1942258912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ec80cb230 0x7f2ec81544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.617+0000 7f2ec659c700 1 -- 172.21.15.28:0/1942258912 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eb80094a0 con 0x7f2ec80b2660 2023-11-28T11:13:16.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.617+0000 7f2ec659c700 1 --2- 172.21.15.28:0/1942258912 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec80b2660 0x7f2ec815bc30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00c520 tx=0x7f2ebc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:16.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.618+0000 7f2eb6ffd700 1 -- 172.21.15.28:0/1942258912 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc004c80 con 0x7f2ec80b2660 2023-11-28T11:13:16.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.618+0000 7f2eb6ffd700 1 -- 172.21.15.28:0/1942258912 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ebc004e00 con 0x7f2ec80b2660 2023-11-28T11:13:16.621 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.618+0000 7f2ecc81f700 1 -- 172.21.15.28:0/1942258912 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec80cf820 con 0x7f2ec80b2660 2023-11-28T11:13:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.618+0000 7f2eb6ffd700 1 -- 172.21.15.28:0/1942258912 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ebc007310 con 0x7f2ec80b2660 2023-11-28T11:13:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.618+0000 7f2ecc81f700 1 -- 172.21.15.28:0/1942258912 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec80cf540 con 0x7f2ec80b2660 2023-11-28T11:13:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.619+0000 7f2ecc81f700 1 -- 172.21.15.28:0/1942258912 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec80cd3e0 con 0x7f2ec80b2660 2023-11-28T11:13:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.619+0000 7f2eb6ffd700 1 -- 172.21.15.28:0/1942258912 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f2ebc01a020 con 0x7f2ec80b2660 2023-11-28T11:13:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.619+0000 7f2eb6ffd700 1 --2- 172.21.15.28:0/1942258912 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2ea80682f0 0x7f2ea806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.620+0000 7f2eb6ffd700 1 -- 172.21.15.28:0/1942258912 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f2ebc085a20 con 0x7f2ec80b2660 2023-11-28T11:13:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.620+0000 7f2ec5d9b700 1 --2- 172.21.15.28:0/1942258912 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2ea80682f0 0x7f2ea806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:16.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.620+0000 7f2ec5d9b700 1 --2- 172.21.15.28:0/1942258912 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2ea80682f0 0x7f2ea806a7a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2ec80cef70 tx=0x7f2eb8013040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:16.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.624+0000 7f2eb6ffd700 1 -- 172.21.15.28:0/1942258912 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f2ebc052100 con 0x7f2ec80b2660 2023-11-28T11:13:16.865 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.865+0000 7f2ecc81f700 1 -- 172.21.15.28:0/1942258912 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2ec815b8b0 con 0x7f2ec80b2660 2023-11-28T11:13:16.866 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.866+0000 7f2eb6ffd700 1 -- 172.21.15.28:0/1942258912 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4650 (secure 0 0 0) 0x7f2ebc055720 con 0x7f2ec80b2660 2023-11-28T11:13:16.866 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:16.867 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":34,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1564028052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1564028052},{"type":"v1","addr":"172.21.15.59:6835","nonce":1564028052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6558,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2509041002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2509041002},{"type":"v1","addr":"172.21.15.28:6836","nonce":2509041002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6561,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3994419079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3994419079},{"type":"v1","addr":"172.21.15.28:6837","nonce":3994419079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T11:12:03.800884+0000","modified":"2023-11-28T11:12:20.278498+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6573},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6573":{"gid":6573,"name":"d","rank":0,"incarnation":29,"state":"up:active","state_seq":7,"addr":"172.21.15.59:6834/2406852565","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2406852565},{"type":"v1","addr":"172.21.15.59:6834","nonce":2406852565}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-11-28T11:13:16.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.867+0000 7f2ecc81f700 1 -- 172.21.15.28:0/1942258912 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2ea80682f0 msgr2=0x7f2ea806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:16.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.867+0000 7f2ecc81f700 1 --2- 172.21.15.28:0/1942258912 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2ea80682f0 0x7f2ea806a7a0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2ec80cef70 tx=0x7f2eb8013040 comp rx=0 tx=0).stop 2023-11-28T11:13:16.868 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.867+0000 7f2ecc81f700 1 -- 172.21.15.28:0/1942258912 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec80b2660 msgr2=0x7f2ec815bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.867+0000 7f2ecc81f700 1 --2- 172.21.15.28:0/1942258912 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec80b2660 0x7f2ec815bc30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2ebc00c520 tx=0x7f2ebc00c8e0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.868+0000 7f2ecc81f700 1 -- 172.21.15.28:0/1942258912 shutdown_connections 2023-11-28T11:13:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.868+0000 7f2ecc81f700 1 --2- 172.21.15.28:0/1942258912 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f2ea80682f0 0x7f2ea806a7a0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.868+0000 7f2ecc81f700 1 --2- 172.21.15.28:0/1942258912 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f2ec80b2660 0x7f2ec815bc30 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.868+0000 7f2ecc81f700 1 --2- 172.21.15.28:0/1942258912 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f2ec815a6c0 0x7f2ec80c9cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.868+0000 7f2ecc81f700 1 --2- 172.21.15.28:0/1942258912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2ec80cb230 0x7f2ec81544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.868+0000 7f2ecc81f700 1 -- 172.21.15.28:0/1942258912 >> 172.21.15.28:0/1942258912 conn(0x7f2ec80cd660 msgr2=0x7f2ec80b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.868+0000 7f2ecc81f700 1 -- 172.21.15.28:0/1942258912 shutdown_connections 2023-11-28T11:13:16.869 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:16.868+0000 7f2ecc81f700 1 -- 172.21.15.28:0/1942258912 wait complete. 2023-11-28T11:13:16.870 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 34 2023-11-28T11:13:16.880 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:13:17.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.005+0000 7ff7d40d5700 1 Processor -- start 2023-11-28T11:13:17.005 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.005+0000 7ff7d40d5700 1 -- start start 2023-11-28T11:13:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.006+0000 7ff7d40d5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7cc161090 0x7ff7cc0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.006+0000 7ff7d40d5700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff7cc0caeb0 0x7ff7cc0cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.006+0000 7ff7d40d5700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7cc0ce340 con 0x7ff7cc15c990 2023-11-28T11:13:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.006+0000 7ff7d40d5700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7cc0cebe0 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.006+0000 7ff7d40d5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc0cf780 con 0x7ff7cc161090 2023-11-28T11:13:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.006+0000 7ff7d1670700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7cc161090 0x7ff7cc0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:17.008 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.006+0000 7ff7d2672700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff7cc0caeb0 0x7ff7cc0cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.006+0000 7ff7d1670700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7cc161090 0x7ff7cc0ca970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3301/0 says I am v2:172.21.15.28:58116/0 (socket says 172.21.15.28:58116) 2023-11-28T11:13:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.006+0000 7ff7d1670700 1 -- 172.21.15.28:0/3619271171 learned_addr learned my addr 172.21.15.28:0/3619271171 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.006+0000 7ff7d2672700 1 -- 172.21.15.28:0/3619271171 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7cc161090 msgr2=0x7ff7cc0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.006+0000 7ff7d2672700 1 --2- 172.21.15.28:0/3619271171 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7cc161090 0x7ff7cc0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.006+0000 7ff7d2672700 1 -- 172.21.15.28:0/3619271171 >> v1:172.21.15.59:6789/0 conn(0x7ff7cc15c990 legacy=0x7ff7cc0cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.006+0000 7ff7d2672700 1 -- 172.21.15.28:0/3619271171 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7cc0cb3f0 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.007+0000 7ff7d2672700 1 --2- 172.21.15.28:0/3619271171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7cc0caeb0 0x7ff7cc0cc310 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009d90 tx=0x7ff7c400b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=41cc9cb23955ab8b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.007+0000 7ff7d0e6f700 1 -- 172.21.15.28:0/3619271171 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400e070 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.007+0000 7ff7d0e6f700 1 -- 172.21.15.28:0/3619271171 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c4003a20 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.007+0000 7ff7d0e6f700 1 -- 172.21.15.28:0/3619271171 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4003dd0 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.007+0000 7ff7d40d5700 1 -- 172.21.15.28:0/3619271171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7cc0caeb0 msgr2=0x7ff7cc0cc310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:17.009 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.007+0000 7ff7d40d5700 1 --2- 172.21.15.28:0/3619271171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7cc0caeb0 0x7ff7cc0cc310 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4009d90 tx=0x7ff7c400b330 comp rx=0 tx=0).stop 2023-11-28T11:13:17.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.007+0000 7ff7d40d5700 1 -- 172.21.15.28:0/3619271171 shutdown_connections 2023-11-28T11:13:17.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.007+0000 7ff7d40d5700 1 --2- 172.21.15.28:0/3619271171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7cc0caeb0 0x7ff7cc0cc310 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:17.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.007+0000 7ff7d40d5700 1 --2- 172.21.15.28:0/3619271171 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7cc161090 0x7ff7cc0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:17.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.007+0000 7ff7d40d5700 1 -- 172.21.15.28:0/3619271171 >> 172.21.15.28:0/3619271171 conn(0x7ff7cc0b1650 msgr2=0x7ff7cc0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:17.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.007+0000 7ff7d40d5700 1 -- 172.21.15.28:0/3619271171 shutdown_connections 2023-11-28T11:13:17.010 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.007+0000 7ff7d40d5700 1 -- 172.21.15.28:0/3619271171 wait complete. 2023-11-28T11:13:17.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.009+0000 7ff7d40d5700 1 Processor -- start 2023-11-28T11:13:17.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.009+0000 7ff7d40d5700 1 -- start start 2023-11-28T11:13:17.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.009+0000 7ff7d40d5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7cc0caeb0 0x7ff7cc0c5e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:17.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.009+0000 7ff7d40d5700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7cc15c990 0x7ff7cc1f11e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:17.012 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.009+0000 7ff7d40d5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7cc161090 0x7ff7cc1f1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.009+0000 7ff7d40d5700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc0ae960 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.009+0000 7ff7d40d5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc1e7a40 con 0x7ff7cc161090 2023-11-28T11:13:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.009+0000 7ff7d40d5700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7cc0abbf0 con 0x7ff7cc15c990 2023-11-28T11:13:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7d1e71700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7cc0caeb0 0x7ff7cc0c5e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7d1e71700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7cc0caeb0 0x7ff7cc0c5e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52362/0 (socket says 172.21.15.28:52362) 2023-11-28T11:13:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7d1e71700 1 -- 172.21.15.28:0/1333318908 learned_addr learned my addr 172.21.15.28:0/1333318908 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7d2672700 1 --2- 172.21.15.28:0/1333318908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7cc161090 0x7ff7cc1f1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7d1670700 1 --2- 172.21.15.28:0/1333318908 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7cc15c990 0x7ff7cc1f11e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7d1e71700 1 -- 172.21.15.28:0/1333318908 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7cc15c990 msgr2=0x7ff7cc1f11e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7d1e71700 1 --2- 172.21.15.28:0/1333318908 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7cc15c990 0x7ff7cc1f11e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7d1e71700 1 -- 172.21.15.28:0/1333318908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7cc161090 msgr2=0x7ff7cc1f1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:17.013 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7d1e71700 1 --2- 172.21.15.28:0/1333318908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7cc161090 0x7ff7cc1f1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7d1e71700 1 -- 172.21.15.28:0/1333318908 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c4009a70 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7d1e71700 1 --2- 172.21.15.28:0/1333318908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7cc0caeb0 0x7ff7cc0c5e80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800c780 tx=0x7ff7c800ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7c27fc700 1 -- 172.21.15.28:0/1333318908 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c80054e0 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7c27fc700 1 -- 172.21.15.28:0/1333318908 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7c8003680 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7d40d5700 1 -- 172.21.15.28:0/1333318908 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7cc0ce870 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7c27fc700 1 -- 172.21.15.28:0/1333318908 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c8003a30 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.010+0000 7ff7d40d5700 1 -- 172.21.15.28:0/1333318908 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7cc0ce590 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.011+0000 7ff7d40d5700 1 -- 172.21.15.28:0/1333318908 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7cc0cc850 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.014 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.012+0000 7ff7c27fc700 1 -- 172.21.15.28:0/1333318908 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff7c8019030 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.012+0000 7ff7c27fc700 1 --2- 172.21.15.28:0/1333318908 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7b0068340 0x7ff7b006a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:17.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.012+0000 7ff7c27fc700 1 -- 172.21.15.28:0/1333318908 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff7c8085260 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.012+0000 7ff7d1670700 1 --2- 172.21.15.28:0/1333318908 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7b0068340 0x7ff7b006a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:17.015 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.013+0000 7ff7d1670700 1 --2- 172.21.15.28:0/1333318908 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7b0068340 0x7ff7b006a7f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0aeac0 tx=0x7ff7bc0092c0 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:17.017 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.016+0000 7ff7c27fc700 1 -- 172.21.15.28:0/1333318908 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff7c80518d0 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.212 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.211+0000 7ff7d40d5700 1 -- 172.21.15.28:0/1333318908 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7cc0675a0 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.213 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.213+0000 7ff7c27fc700 1 -- 172.21.15.28:0/1333318908 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+18174 (secure 0 0 0) 0x7ff7c8054ef0 con 0x7ff7cc0caeb0 2023-11-28T11:13:17.213 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:17.214 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":68,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:12:35.277297+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-11-28T11:12:01.724396+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-11-28T11:12:02.776612+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"11.9","mappings":[{"from":6,"to":0}]},{"pgid":"11.2d","mappings":[{"from":6,"to":0}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:13:17.215 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.214+0000 7ff7d40d5700 1 -- 172.21.15.28:0/1333318908 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7b0068340 msgr2=0x7ff7b006a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:17.215 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.214+0000 7ff7d40d5700 1 --2- 172.21.15.28:0/1333318908 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7b0068340 0x7ff7b006a7f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0aeac0 tx=0x7ff7bc0092c0 comp rx=0 tx=0).stop 2023-11-28T11:13:17.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.214+0000 7ff7d40d5700 1 -- 172.21.15.28:0/1333318908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7cc0caeb0 msgr2=0x7ff7cc0c5e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:17.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.214+0000 7ff7d40d5700 1 --2- 172.21.15.28:0/1333318908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7cc0caeb0 0x7ff7cc0c5e80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7ff7c800c780 tx=0x7ff7c800ca90 comp rx=0 tx=0).stop 2023-11-28T11:13:17.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.215+0000 7ff7d40d5700 1 -- 172.21.15.28:0/1333318908 shutdown_connections 2023-11-28T11:13:17.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.215+0000 7ff7d40d5700 1 --2- 172.21.15.28:0/1333318908 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff7b0068340 0x7ff7b006a7f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:17.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.215+0000 7ff7d40d5700 1 --2- 172.21.15.28:0/1333318908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff7cc0caeb0 0x7ff7cc0c5e80 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:17.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.215+0000 7ff7d40d5700 1 --2- 172.21.15.28:0/1333318908 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7cc161090 0x7ff7cc1f1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:17.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.215+0000 7ff7d40d5700 1 --2- 172.21.15.28:0/1333318908 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff7cc15c990 0x7ff7cc1f11e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:17.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.215+0000 7ff7d40d5700 1 -- 172.21.15.28:0/1333318908 >> 172.21.15.28:0/1333318908 conn(0x7ff7cc0b1650 msgr2=0x7ff7cc06b030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:17.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.215+0000 7ff7d40d5700 1 -- 172.21.15.28:0/1333318908 shutdown_connections 2023-11-28T11:13:17.216 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.215+0000 7ff7d40d5700 1 -- 172.21.15.28:0/1333318908 wait complete. 2023-11-28T11:13:17.228 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T11:13:17.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.352+0000 7f2444036700 1 Processor -- start 2023-11-28T11:13:17.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.352+0000 7f2444036700 1 -- start start 2023-11-28T11:13:17.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.353+0000 7f2444036700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f243c15c990 0x7f243c0cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:17.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.353+0000 7f2444036700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f243c161090 0x7f243c0ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:17.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.353+0000 7f2444036700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f243c0ce340 con 0x7f243c0caeb0 2023-11-28T11:13:17.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.353+0000 7f2444036700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f243c0cebe0 con 0x7f243c15c990 2023-11-28T11:13:17.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.353+0000 7f2444036700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0cb3f0 con 0x7f243c161090 2023-11-28T11:13:17.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.353+0000 7f2441dd2700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f243c15c990 0x7f243c0cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:17.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.353+0000 7f24415d1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f243c161090 0x7f243c0ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:17.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.353+0000 7f24425d3700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f243c0caeb0 0x7f243c0cc310 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:51218/0 (socket says 172.21.15.28:51218) 2023-11-28T11:13:17.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.354+0000 7f24425d3700 1 -- 172.21.15.28:0/1934413239 learned_addr learned my addr 172.21.15.28:0/1934413239 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:17.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.354+0000 7f2441dd2700 1 -- 172.21.15.28:0/1934413239 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f243c161090 msgr2=0x7f243c0ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:17.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.354+0000 7f2441dd2700 1 --2- 172.21.15.28:0/1934413239 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f243c161090 0x7f243c0ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:17.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.354+0000 7f2441dd2700 1 -- 172.21.15.28:0/1934413239 >> v1:172.21.15.59:6789/0 conn(0x7f243c0caeb0 legacy=0x7f243c0cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:17.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.354+0000 7f2441dd2700 1 -- 172.21.15.28:0/1934413239 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f243c0ce590 con 0x7f243c15c990 2023-11-28T11:13:17.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.354+0000 7f2440dd0700 1 -- 172.21.15.28:0/1934413239 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3350729258 0 0) 0x7f243c0ce340 con 0x7f243c0caeb0 2023-11-28T11:13:17.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.354+0000 7f2441dd2700 1 --2- 172.21.15.28:0/1934413239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c15c990 0x7f243c0cf070 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f242c009840 tx=0x7f242c009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=26fc85b5fd6b73e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:17.356 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.354+0000 7f2440dd0700 1 -- 172.21.15.28:0/1934413239 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c011830 con 0x7f243c15c990 2023-11-28T11:13:17.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.354+0000 7f2440dd0700 1 -- 172.21.15.28:0/1934413239 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242c011990 con 0x7f243c15c990 2023-11-28T11:13:17.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.354+0000 7f2440dd0700 1 -- 172.21.15.28:0/1934413239 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c011d20 con 0x7f243c15c990 2023-11-28T11:13:17.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.355+0000 7f2444036700 1 -- 172.21.15.28:0/1934413239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c15c990 msgr2=0x7f243c0cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:17.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.355+0000 7f2444036700 1 --2- 172.21.15.28:0/1934413239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c15c990 0x7f243c0cf070 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f242c009840 tx=0x7f242c009b50 comp rx=0 tx=0).stop 2023-11-28T11:13:17.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.355+0000 7f2444036700 1 -- 172.21.15.28:0/1934413239 shutdown_connections 2023-11-28T11:13:17.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.355+0000 7f2444036700 1 --2- 172.21.15.28:0/1934413239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c15c990 0x7f243c0cf070 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:17.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.355+0000 7f2444036700 1 --2- 172.21.15.28:0/1934413239 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f243c161090 0x7f243c0ca970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:17.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.355+0000 7f2444036700 1 -- 172.21.15.28:0/1934413239 >> 172.21.15.28:0/1934413239 conn(0x7f243c0b1650 msgr2=0x7f243c0b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:17.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.355+0000 7f2444036700 1 -- 172.21.15.28:0/1934413239 shutdown_connections 2023-11-28T11:13:17.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.355+0000 7f2444036700 1 -- 172.21.15.28:0/1934413239 wait complete. 2023-11-28T11:13:17.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.356+0000 7f2444036700 1 Processor -- start 2023-11-28T11:13:17.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.356+0000 7f2444036700 1 -- start start 2023-11-28T11:13:17.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f2444036700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f243c0caeb0 0x7f243c15aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:17.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f2444036700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f243c15c990 0x7f243c0b3c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:17.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f2444036700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c161090 0x7f243c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:17.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f2444036700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c15e7c0 con 0x7f243c0caeb0 2023-11-28T11:13:17.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f2444036700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f243c15e930 con 0x7f243c161090 2023-11-28T11:13:17.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f2444036700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f243c0c5320 con 0x7f243c15c990 2023-11-28T11:13:17.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f2441dd2700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f243c0caeb0 0x7f243c15aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:17.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f2441dd2700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f243c0caeb0 0x7f243c15aff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52376/0 (socket says 172.21.15.28:52376) 2023-11-28T11:13:17.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f2441dd2700 1 -- 172.21.15.28:0/2532529365 learned_addr learned my addr 172.21.15.28:0/2532529365 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:17.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f24425d3700 1 --2- 172.21.15.28:0/2532529365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c161090 0x7f243c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:17.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f24415d1700 1 --2- 172.21.15.28:0/2532529365 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f243c15c990 0x7f243c0b3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:17.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f2441dd2700 1 -- 172.21.15.28:0/2532529365 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f243c15c990 msgr2=0x7f243c0b3c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:17.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f2441dd2700 1 --2- 172.21.15.28:0/2532529365 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f243c15c990 0x7f243c0b3c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:17.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f2441dd2700 1 -- 172.21.15.28:0/2532529365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c161090 msgr2=0x7f243c0b4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:17.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f2441dd2700 1 --2- 172.21.15.28:0/2532529365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c161090 0x7f243c0b4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:17.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.357+0000 7f2441dd2700 1 -- 172.21.15.28:0/2532529365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f242c0094a0 con 0x7f243c0caeb0 2023-11-28T11:13:17.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.358+0000 7f2441dd2700 1 --2- 172.21.15.28:0/2532529365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f243c0caeb0 0x7f243c15aff0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f242c000c00 tx=0x7f242c004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:17.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.358+0000 7f24327fc700 1 -- 172.21.15.28:0/2532529365 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0045d0 con 0x7f243c0caeb0 2023-11-28T11:13:17.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.358+0000 7f24327fc700 1 -- 172.21.15.28:0/2532529365 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f242c01f070 con 0x7f243c0caeb0 2023-11-28T11:13:17.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.358+0000 7f2444036700 1 -- 172.21.15.28:0/2532529365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f243c0cf5b0 con 0x7f243c0caeb0 2023-11-28T11:13:17.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.358+0000 7f24327fc700 1 -- 172.21.15.28:0/2532529365 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f242c0038c0 con 0x7f243c0caeb0 2023-11-28T11:13:17.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.358+0000 7f2444036700 1 -- 172.21.15.28:0/2532529365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f243c0ce870 con 0x7f243c0caeb0 2023-11-28T11:13:17.362 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.359+0000 7f2444036700 1 -- 172.21.15.28:0/2532529365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f243c0cf740 con 0x7f243c0caeb0 2023-11-28T11:13:17.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.359+0000 7f24327fc700 1 -- 172.21.15.28:0/2532529365 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f242c00f050 con 0x7f243c0caeb0 2023-11-28T11:13:17.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.359+0000 7f24327fc700 1 --2- 172.21.15.28:0/2532529365 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24200682f0 0x7f242006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:17.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.360+0000 7f24327fc700 1 -- 172.21.15.28:0/2532529365 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f242c021070 con 0x7f243c0caeb0 2023-11-28T11:13:17.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.360+0000 7f24415d1700 1 --2- 172.21.15.28:0/2532529365 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24200682f0 0x7f242006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:17.363 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.360+0000 7f24415d1700 1 --2- 172.21.15.28:0/2532529365 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24200682f0 0x7f242006a7a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f243c1f54a0 tx=0x7f2438009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:17.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.364+0000 7f24327fc700 1 -- 172.21.15.28:0/2532529365 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f242c053c60 con 0x7f243c0caeb0 2023-11-28T11:13:17.601 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:17.601+0000 7f2444036700 1 -- 172.21.15.28:0/2532529365 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f243c0675a0 con 0x7f243c0caeb0 2023-11-28T11:13:18.296 INFO:tasks.ceph.mon.a.smithi028.stderr:2023-11-28T11:13:18.296+0000 7fde5abc8700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T11:13:18.300 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.299+0000 7f24327fc700 1 -- 172.21.15.28:0/2532529365 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7f242c057280 con 0x7f243c0caeb0 2023-11-28T11:13:18.303 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.302+0000 7f2444036700 1 -- 172.21.15.28:0/2532529365 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24200682f0 msgr2=0x7f242006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.302+0000 7f2444036700 1 --2- 172.21.15.28:0/2532529365 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24200682f0 0x7f242006a7a0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f243c1f54a0 tx=0x7f2438009210 comp rx=0 tx=0).stop 2023-11-28T11:13:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.302+0000 7f2444036700 1 -- 172.21.15.28:0/2532529365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f243c0caeb0 msgr2=0x7f243c15aff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.302+0000 7f2444036700 1 --2- 172.21.15.28:0/2532529365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f243c0caeb0 0x7f243c15aff0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f242c000c00 tx=0x7f242c004750 comp rx=0 tx=0).stop 2023-11-28T11:13:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.303+0000 7f2444036700 1 -- 172.21.15.28:0/2532529365 shutdown_connections 2023-11-28T11:13:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.303+0000 7f2444036700 1 --2- 172.21.15.28:0/2532529365 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f24200682f0 0x7f242006a7a0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.303+0000 7f2444036700 1 --2- 172.21.15.28:0/2532529365 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f243c0caeb0 0x7f243c15aff0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.303+0000 7f2444036700 1 --2- 172.21.15.28:0/2532529365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f243c161090 0x7f243c0b4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.303+0000 7f2444036700 1 --2- 172.21.15.28:0/2532529365 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f243c15c990 0x7f243c0b3c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.303+0000 7f2444036700 1 -- 172.21.15.28:0/2532529365 >> 172.21.15.28:0/2532529365 conn(0x7f243c0b1650 msgr2=0x7f243c0c8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.303+0000 7f2444036700 1 -- 172.21.15.28:0/2532529365 shutdown_connections 2023-11-28T11:13:18.304 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.303+0000 7f2444036700 1 -- 172.21.15.28:0/2532529365 wait complete. 2023-11-28T11:13:18.305 INFO:teuthology.orchestra.run.smithi028.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T11:13:18.306 INFO:tasks.ceph.mds.d.smithi059.stderr: -5304> 2023-11-28T11:12:09.371+0000 7fa84c78e700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T11:13:18.309 INFO:tasks.ceph.mds.d.smithi059.stderr: -3706> 2023-11-28T11:12:17.566+0000 7fa848f87700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /dir_test/file auth v2 ap=1 s=0 n(v0 1=1+0) (iauth excl) (ifile excl) (ixattr excl) (iversion lock) cr={6768=0-4194304@1} caps={6768=pAsxLsXsxFsxcrwb/pAsxXsxFxwb@2},l=6768 | request=0 caps=1 dirtyparent=0 dirty=0 authpin=1 scrubqueue=0 0x565370f6db80]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(11)0x10000000001:[<0x10000000000/file v2>,<0x1/dir_test v4>]//[]","error_str":"failed to decode on-disk backtrace (26 bytes)!"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-11-28T11:13:18.319 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T11:13:18.340 INFO:tasks.ceph.mds.d.smithi059.stdout:starting mds.d at 2023-11-28T11:13:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.452+0000 7f910782f700 1 Processor -- start 2023-11-28T11:13:18.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.452+0000 7f910782f700 1 -- start start 2023-11-28T11:13:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.453+0000 7f910782f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9100159500 0x7f91000cd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.453+0000 7f910782f700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f910015e040 con 0x7f910015dbf0 2023-11-28T11:13:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.453+0000 7f910782f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91000ceff0 con 0x7f91000b1d40 2023-11-28T11:13:18.454 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.453+0000 7f910782f700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91000cf2d0 con 0x7f9100159500 2023-11-28T11:13:18.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.453+0000 7f9104dca700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f910015dbf0 0x7f91000b1630 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40444/0 (socket says 172.21.15.28:40444) 2023-11-28T11:13:18.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.453+0000 7f9104dca700 1 -- 172.21.15.28:0/3726721646 learned_addr learned my addr 172.21.15.28:0/3726721646 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:18.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.453+0000 7f91055cb700 1 --2- 172.21.15.28:0/3726721646 >> v2:172.21.15.28:3300/0 conn(0x7f9100159500 0x7f91000cd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:18.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.454+0000 7f90f3fff700 1 -- 172.21.15.28:0/3726721646 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3750860608 0 0) 0x7f910015e040 con 0x7f910015dbf0 2023-11-28T11:13:18.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.454+0000 7f90f3fff700 1 -- 172.21.15.28:0/3726721646 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90ec0035b0 con 0x7f910015dbf0 2023-11-28T11:13:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.454+0000 7f91055cb700 1 -- 172.21.15.28:0/3726721646 >> v1:172.21.15.59:6789/0 conn(0x7f91000b1d40 legacy=0x7f910006bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.454+0000 7f91055cb700 1 -- 172.21.15.28:0/3726721646 >> v1:172.21.15.28:6789/0 conn(0x7f910015dbf0 legacy=0x7f91000b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.454+0000 7f91055cb700 1 -- 172.21.15.28:0/3726721646 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f910015d110 con 0x7f9100159500 2023-11-28T11:13:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.454+0000 7f91055cb700 1 --2- 172.21.15.28:0/3726721646 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9100159500 0x7f91000cd140 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f90f4009840 tx=0x7f90f4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b5eec19dc66035 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.454+0000 7f90f3fff700 1 -- 172.21.15.28:0/3726721646 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f4011830 con 0x7f9100159500 2023-11-28T11:13:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.454+0000 7f90f3fff700 1 -- 172.21.15.28:0/3726721646 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f4011990 con 0x7f9100159500 2023-11-28T11:13:18.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.454+0000 7f90f3fff700 1 -- 172.21.15.28:0/3726721646 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f4011d20 con 0x7f9100159500 2023-11-28T11:13:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.455+0000 7f910782f700 1 -- 172.21.15.28:0/3726721646 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9100159500 msgr2=0x7f91000cd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.455+0000 7f910782f700 1 --2- 172.21.15.28:0/3726721646 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9100159500 0x7f91000cd140 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f90f4009840 tx=0x7f90f4009b50 comp rx=0 tx=0).stop 2023-11-28T11:13:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.455+0000 7f910782f700 1 -- 172.21.15.28:0/3726721646 shutdown_connections 2023-11-28T11:13:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.455+0000 7f910782f700 1 --2- 172.21.15.28:0/3726721646 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9100159500 0x7f91000cd140 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.455+0000 7f910782f700 1 -- 172.21.15.28:0/3726721646 >> 172.21.15.28:0/3726721646 conn(0x7f91000cc630 msgr2=0x7f91000cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.455+0000 7f910782f700 1 -- 172.21.15.28:0/3726721646 shutdown_connections 2023-11-28T11:13:18.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.455+0000 7f910782f700 1 -- 172.21.15.28:0/3726721646 wait complete. 2023-11-28T11:13:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.457+0000 7f910782f700 1 Processor -- start 2023-11-28T11:13:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.457+0000 7f910782f700 1 -- start start 2023-11-28T11:13:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f910782f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91000b1d40 0x7f9100159e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f910782f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9100159500 0x7f9100150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f910782f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f910015dbf0 0x7f9100151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f910782f700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91001570f0 con 0x7f91000b1d40 2023-11-28T11:13:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f910782f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91000c4390 con 0x7f9100159500 2023-11-28T11:13:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f910782f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91000c4670 con 0x7f910015dbf0 2023-11-28T11:13:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f91055cb700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91000b1d40 0x7f9100159e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f91055cb700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91000b1d40 0x7f9100159e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52390/0 (socket says 172.21.15.28:52390) 2023-11-28T11:13:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f91055cb700 1 -- 172.21.15.28:0/1696981521 learned_addr learned my addr 172.21.15.28:0/1696981521 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:18.460 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f9104dca700 1 --2- 172.21.15.28:0/1696981521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9100159500 0x7f9100150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:18.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f91055cb700 1 -- 172.21.15.28:0/1696981521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f910015dbf0 msgr2=0x7f9100151280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:18.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f9105dcc700 1 --2- 172.21.15.28:0/1696981521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f910015dbf0 0x7f9100151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:18.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f91055cb700 1 --2- 172.21.15.28:0/1696981521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f910015dbf0 0x7f9100151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:18.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f91055cb700 1 -- 172.21.15.28:0/1696981521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9100159500 msgr2=0x7f9100150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:18.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f91055cb700 1 --2- 172.21.15.28:0/1696981521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9100159500 0x7f9100150d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:18.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f91055cb700 1 -- 172.21.15.28:0/1696981521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f40094a0 con 0x7f91000b1d40 2023-11-28T11:13:18.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f9104dca700 1 --2- 172.21.15.28:0/1696981521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9100159500 0x7f9100150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:13:18.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.458+0000 7f91055cb700 1 --2- 172.21.15.28:0/1696981521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91000b1d40 0x7f9100159e80 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f90f40090b0 tx=0x7f90f4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:18.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.459+0000 7f90f1ffb700 1 -- 172.21.15.28:0/1696981521 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f40038a0 con 0x7f91000b1d40 2023-11-28T11:13:18.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.459+0000 7f90f1ffb700 1 -- 172.21.15.28:0/1696981521 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90f401f070 con 0x7f91000b1d40 2023-11-28T11:13:18.461 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.459+0000 7f910782f700 1 -- 172.21.15.28:0/1696981521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f910015d6d0 con 0x7f91000b1d40 2023-11-28T11:13:18.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.459+0000 7f90f1ffb700 1 -- 172.21.15.28:0/1696981521 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90f4003c00 con 0x7f91000b1d40 2023-11-28T11:13:18.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.459+0000 7f910782f700 1 -- 172.21.15.28:0/1696981521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f910015d3f0 con 0x7f91000b1d40 2023-11-28T11:13:18.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.460+0000 7f910782f700 1 -- 172.21.15.28:0/1696981521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91001598d0 con 0x7f91000b1d40 2023-11-28T11:13:18.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.460+0000 7f90f1ffb700 1 -- 172.21.15.28:0/1696981521 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f90f400f050 con 0x7f91000b1d40 2023-11-28T11:13:18.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.461+0000 7f90f1ffb700 1 --2- 172.21.15.28:0/1696981521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f90e40682f0 0x7f90e406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:18.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.461+0000 7f90f1ffb700 1 -- 172.21.15.28:0/1696981521 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6526+0+0 (secure 0 0 0) 0x7f90f4021070 con 0x7f91000b1d40 2023-11-28T11:13:18.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.461+0000 7f9104dca700 1 --2- 172.21.15.28:0/1696981521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f90e40682f0 0x7f90e406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:18.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.461+0000 7f9104dca700 1 --2- 172.21.15.28:0/1696981521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f90e40682f0 0x7f90e406a7a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9100169650 tx=0x7f90fc009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:18.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.465+0000 7f90f1ffb700 1 -- 172.21.15.28:0/1696981521 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f90f4053e60 con 0x7f91000b1d40 2023-11-28T11:13:18.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:18.704+0000 7f910782f700 1 -- 172.21.15.28:0/1696981521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f91000739f0 con 0x7f91000b1d40 2023-11-28T11:13:19.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.308+0000 7f90f1ffb700 1 -- 172.21.15.28:0/1696981521 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7f90f4057480 con 0x7f91000b1d40 2023-11-28T11:13:19.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.311+0000 7f910782f700 1 -- 172.21.15.28:0/1696981521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f90e40682f0 msgr2=0x7f90e406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:19.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.311+0000 7f910782f700 1 --2- 172.21.15.28:0/1696981521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f90e40682f0 0x7f90e406a7a0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f9100169650 tx=0x7f90fc009040 comp rx=0 tx=0).stop 2023-11-28T11:13:19.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.311+0000 7f910782f700 1 -- 172.21.15.28:0/1696981521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91000b1d40 msgr2=0x7f9100159e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:19.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.311+0000 7f910782f700 1 --2- 172.21.15.28:0/1696981521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91000b1d40 0x7f9100159e80 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f90f40090b0 tx=0x7f90f4004750 comp rx=0 tx=0).stop 2023-11-28T11:13:19.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.312+0000 7f910782f700 1 -- 172.21.15.28:0/1696981521 shutdown_connections 2023-11-28T11:13:19.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.312+0000 7f910782f700 1 --2- 172.21.15.28:0/1696981521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f90e40682f0 0x7f90e406a7a0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:19.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.312+0000 7f910782f700 1 --2- 172.21.15.28:0/1696981521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f91000b1d40 0x7f9100159e80 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:19.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.312+0000 7f910782f700 1 --2- 172.21.15.28:0/1696981521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f910015dbf0 0x7f9100151280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:19.313 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.312+0000 7f910782f700 1 --2- 172.21.15.28:0/1696981521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9100159500 0x7f9100150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:19.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.312+0000 7f910782f700 1 -- 172.21.15.28:0/1696981521 >> 172.21.15.28:0/1696981521 conn(0x7f91000cc630 msgr2=0x7f910006c4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:19.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.312+0000 7f910782f700 1 -- 172.21.15.28:0/1696981521 shutdown_connections 2023-11-28T11:13:19.314 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.312+0000 7f910782f700 1 -- 172.21.15.28:0/1696981521 wait complete. 2023-11-28T11:13:19.327 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T11:13:19.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.463+0000 7f80ba772700 1 Processor -- start 2023-11-28T11:13:19.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.464+0000 7f80ba772700 1 -- start start 2023-11-28T11:13:19.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.464+0000 7f80ba772700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f80b413fd90 0x7f80b40c9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:19.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.464+0000 7f80ba772700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b406f260 con 0x7f80b4071180 2023-11-28T11:13:19.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.464+0000 7f80ba772700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80b4141cc0 con 0x7f80b40c9e60 2023-11-28T11:13:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.464+0000 7f80ba772700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80b40c2af0 con 0x7f80b413fd90 2023-11-28T11:13:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.464+0000 7f80b37fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f80b413fd90 0x7f80b40c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.464+0000 7f80b3fff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f80b4071180 0x7f80b4071550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40456/0 (socket says 172.21.15.28:40456) 2023-11-28T11:13:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.464+0000 7f80b3fff700 1 -- 172.21.15.28:0/3465573969 learned_addr learned my addr 172.21.15.28:0/3465573969 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:19.468 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.465+0000 7f80b37fe700 1 -- 172.21.15.28:0/3465573969 >> v1:172.21.15.59:6789/0 conn(0x7f80b40c9e60 legacy=0x7f80b4142000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.465+0000 7f80b37fe700 1 -- 172.21.15.28:0/3465573969 >> v1:172.21.15.28:6789/0 conn(0x7f80b4071180 legacy=0x7f80b4071550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.465+0000 7f80b37fe700 1 -- 172.21.15.28:0/3465573969 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80b406e650 con 0x7f80b413fd90 2023-11-28T11:13:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.465+0000 7f80b2ffd700 1 -- 172.21.15.28:0/3465573969 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591048602 0 0) 0x7f80b406f260 con 0x7f80b4071180 2023-11-28T11:13:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.465+0000 7f80b37fe700 1 --2- 172.21.15.28:0/3465573969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80b413fd90 0x7f80b40c9920 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009840 tx=0x7f80a8009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56c1055dd98c06d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.465+0000 7f80b2ffd700 1 -- 172.21.15.28:0/3465573969 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8004230 con 0x7f80b413fd90 2023-11-28T11:13:19.469 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.465+0000 7f80b2ffd700 1 -- 172.21.15.28:0/3465573969 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a8004390 con 0x7f80b413fd90 2023-11-28T11:13:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.465+0000 7f80b2ffd700 1 -- 172.21.15.28:0/3465573969 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a80036a0 con 0x7f80b413fd90 2023-11-28T11:13:19.470 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.466+0000 7f80ba772700 1 -- 172.21.15.28:0/3465573969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80b413fd90 msgr2=0x7f80b40c9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.466+0000 7f80ba772700 1 --2- 172.21.15.28:0/3465573969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80b413fd90 0x7f80b40c9920 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f80a8009840 tx=0x7f80a8009b50 comp rx=0 tx=0).stop 2023-11-28T11:13:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.466+0000 7f80ba772700 1 -- 172.21.15.28:0/3465573969 shutdown_connections 2023-11-28T11:13:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.466+0000 7f80ba772700 1 --2- 172.21.15.28:0/3465573969 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80b413fd90 0x7f80b40c9920 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.466+0000 7f80ba772700 1 -- 172.21.15.28:0/3465573969 >> 172.21.15.28:0/3465573969 conn(0x7f80b40cfe60 msgr2=0x7f80b40b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.466+0000 7f80ba772700 1 -- 172.21.15.28:0/3465573969 shutdown_connections 2023-11-28T11:13:19.471 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.466+0000 7f80ba772700 1 -- 172.21.15.28:0/3465573969 wait complete. 2023-11-28T11:13:19.472 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.467+0000 7f80ba772700 1 Processor -- start 2023-11-28T11:13:19.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.467+0000 7f80ba772700 1 -- start start 2023-11-28T11:13:19.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80ba772700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80b4071180 0x7f80b4156060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:19.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80ba772700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80b40c9e60 0x7f80b40ac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:19.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80ba772700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80b413fd90 0x7f80b40b43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:19.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80ba772700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40c3e00 con 0x7f80b40c9e60 2023-11-28T11:13:19.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80ba772700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40c2d60 con 0x7f80b4071180 2023-11-28T11:13:19.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80ba772700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80b40c3040 con 0x7f80b413fd90 2023-11-28T11:13:19.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80b37fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80b40c9e60 0x7f80b40ac1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:19.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80b37fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80b40c9e60 0x7f80b40ac1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52406/0 (socket says 172.21.15.28:52406) 2023-11-28T11:13:19.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80b37fe700 1 -- 172.21.15.28:0/269093779 learned_addr learned my addr 172.21.15.28:0/269093779 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:19.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80b3fff700 1 --2- 172.21.15.28:0/269093779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80b4071180 0x7f80b4156060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:19.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80b8d0f700 1 --2- 172.21.15.28:0/269093779 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80b413fd90 0x7f80b40b43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:19.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80b37fe700 1 -- 172.21.15.28:0/269093779 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80b413fd90 msgr2=0x7f80b40b43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:19.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80b37fe700 1 --2- 172.21.15.28:0/269093779 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80b413fd90 0x7f80b40b43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:19.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80b37fe700 1 -- 172.21.15.28:0/269093779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80b4071180 msgr2=0x7f80b4156060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80b37fe700 1 --2- 172.21.15.28:0/269093779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80b4071180 0x7f80b4156060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80b37fe700 1 -- 172.21.15.28:0/269093779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80a80094a0 con 0x7f80b40c9e60 2023-11-28T11:13:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.468+0000 7f80b37fe700 1 --2- 172.21.15.28:0/269093779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80b40c9e60 0x7f80b40ac1d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f80a80090b0 tx=0x7f80a80047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.469+0000 7f80b0ff9700 1 -- 172.21.15.28:0/269093779 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8011630 con 0x7f80b40c9e60 2023-11-28T11:13:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.469+0000 7f80b0ff9700 1 -- 172.21.15.28:0/269093779 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80a8011790 con 0x7f80b40c9e60 2023-11-28T11:13:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.469+0000 7f80ba772700 1 -- 172.21.15.28:0/269093779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80b406ec10 con 0x7f80b40c9e60 2023-11-28T11:13:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.469+0000 7f80b0ff9700 1 -- 172.21.15.28:0/269093779 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80a8011b40 con 0x7f80b40c9e60 2023-11-28T11:13:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.469+0000 7f80ba772700 1 -- 172.21.15.28:0/269093779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80b406e930 con 0x7f80b40c9e60 2023-11-28T11:13:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.470+0000 7f80ba772700 1 -- 172.21.15.28:0/269093779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80b406eef0 con 0x7f80b40c9e60 2023-11-28T11:13:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.470+0000 7f80b0ff9700 1 -- 172.21.15.28:0/269093779 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f80a8017020 con 0x7f80b40c9e60 2023-11-28T11:13:19.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.470+0000 7f80b0ff9700 1 --2- 172.21.15.28:0/269093779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80a00682f0 0x7f80a006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.471+0000 7f80b0ff9700 1 -- 172.21.15.28:0/269093779 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6526+0+0 (secure 0 0 0) 0x7f80a8087760 con 0x7f80b40c9e60 2023-11-28T11:13:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.471+0000 7f80b3fff700 1 --2- 172.21.15.28:0/269093779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80a00682f0 0x7f80a006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.471+0000 7f80b3fff700 1 --2- 172.21.15.28:0/269093779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80a00682f0 0x7f80a006a7a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f80a4000f60 tx=0x7f80a4007c90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:19.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.475+0000 7f80b0ff9700 1 -- 172.21.15.28:0/269093779 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f80a8053d00 con 0x7f80b40c9e60 2023-11-28T11:13:19.667 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:19.666+0000 7f80ba772700 1 -- 172.21.15.28:0/269093779 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f80b40b14f0 con 0x7f80b40c9e60 2023-11-28T11:13:20.364 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.363+0000 7f80b0ff9700 1 -- 172.21.15.28:0/269093779 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v70) v1 ==== 181+0+0 (secure 0 0 0) 0x7f80a8057320 con 0x7f80b40c9e60 2023-11-28T11:13:20.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.367+0000 7f80ba772700 1 -- 172.21.15.28:0/269093779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80a00682f0 msgr2=0x7f80a006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:20.369 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.367+0000 7f80ba772700 1 --2- 172.21.15.28:0/269093779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80a00682f0 0x7f80a006a7a0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f80a4000f60 tx=0x7f80a4007c90 comp rx=0 tx=0).stop 2023-11-28T11:13:20.369 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.367+0000 7f80ba772700 1 -- 172.21.15.28:0/269093779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80b40c9e60 msgr2=0x7f80b40ac1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:20.369 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.367+0000 7f80ba772700 1 --2- 172.21.15.28:0/269093779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80b40c9e60 0x7f80b40ac1d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f80a80090b0 tx=0x7f80a80047c0 comp rx=0 tx=0).stop 2023-11-28T11:13:20.369 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.367+0000 7f80ba772700 1 -- 172.21.15.28:0/269093779 shutdown_connections 2023-11-28T11:13:20.369 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.367+0000 7f80ba772700 1 --2- 172.21.15.28:0/269093779 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80a00682f0 0x7f80a006a7a0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:20.370 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.367+0000 7f80ba772700 1 --2- 172.21.15.28:0/269093779 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80b413fd90 0x7f80b40b43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:20.370 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.367+0000 7f80ba772700 1 --2- 172.21.15.28:0/269093779 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80b4071180 0x7f80b4156060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:20.370 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.367+0000 7f80ba772700 1 --2- 172.21.15.28:0/269093779 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80b40c9e60 0x7f80b40ac1d0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:20.370 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.367+0000 7f80ba772700 1 -- 172.21.15.28:0/269093779 >> 172.21.15.28:0/269093779 conn(0x7f80b40cfe60 msgr2=0x7f80b41401b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:20.370 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.367+0000 7f80ba772700 1 -- 172.21.15.28:0/269093779 shutdown_connections 2023-11-28T11:13:20.371 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.368+0000 7f80ba772700 1 -- 172.21.15.28:0/269093779 wait complete. 2023-11-28T11:13:20.371 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_metadata' removed 2023-11-28T11:13:20.384 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T11:13:20.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.519+0000 7f6e5cd46700 1 Processor -- start 2023-11-28T11:13:20.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.519+0000 7f6e5cd46700 1 -- start start 2023-11-28T11:13:20.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e5cd46700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e5815c990 0x7f6e580cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:20.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e5cd46700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f6e580cc310 0x7f6e580cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:20.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e5cd46700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e580ce340 con 0x7f6e58161090 2023-11-28T11:13:20.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e5cd46700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e580cebe0 con 0x7f6e580cc310 2023-11-28T11:13:20.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e5cd46700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580cccf0 con 0x7f6e5815c990 2023-11-28T11:13:20.538 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e55d9b700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f6e58161090 0x7f6e580ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40472/0 (socket says 172.21.15.28:40472) 2023-11-28T11:13:20.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e55d9b700 1 -- 172.21.15.28:0/3353751726 learned_addr learned my addr 172.21.15.28:0/3353751726 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:20.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e5659c700 1 --2- 172.21.15.28:0/3353751726 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e5815c990 0x7f6e580cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:20.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e5559a700 1 -- 172.21.15.28:0/3353751726 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1801495678 0 0) 0x7f6e580ce340 con 0x7f6e58161090 2023-11-28T11:13:20.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e5559a700 1 -- 172.21.15.28:0/3353751726 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e400035b0 con 0x7f6e58161090 2023-11-28T11:13:20.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e56d9d700 1 --2- 172.21.15.28:0/3353751726 >> v2:172.21.15.59:3300/0 conn(0x7f6e580cc310 0x7f6e580cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:20.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e5559a700 1 -- 172.21.15.28:0/3353751726 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 102508902 0 0) 0x7f6e400035b0 con 0x7f6e58161090 2023-11-28T11:13:20.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e5559a700 1 -- 172.21.15.28:0/3353751726 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e5815c990 msgr2=0x7f6e580cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:20.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e5559a700 1 --2- 172.21.15.28:0/3353751726 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e5815c990 0x7f6e580cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:20.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e5559a700 1 -- 172.21.15.28:0/3353751726 >> v2:172.21.15.59:3300/0 conn(0x7f6e580cc310 msgr2=0x7f6e580cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:20.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.520+0000 7f6e5559a700 1 --2- 172.21.15.28:0/3353751726 >> v2:172.21.15.59:3300/0 conn(0x7f6e580cc310 0x7f6e580cc7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:20.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e5559a700 1 -- 172.21.15.28:0/3353751726 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e580cb3f0 con 0x7f6e58161090 2023-11-28T11:13:20.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e5559a700 1 -- 172.21.15.28:0/3353751726 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f6e4c001670 con 0x7f6e58161090 2023-11-28T11:13:20.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e5559a700 1 -- 172.21.15.28:0/3353751726 >> v1:172.21.15.28:6789/0 conn(0x7f6e58161090 legacy=0x7f6e580ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:20.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e5559a700 1 --2- 172.21.15.28:0/3353751726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e40003a00 0x7f6e40003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:20.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e5559a700 1 --2- 172.21.15.28:0/3353751726 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e5815c990 0x7f6e40004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:20.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e5559a700 1 --2- 172.21.15.28:0/3353751726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e40004d90 0x7f6e40005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:20.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e5559a700 1 -- 172.21.15.28:0/3353751726 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580cebe0 con 0x7f6e40004d90 2023-11-28T11:13:20.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e5559a700 1 -- 172.21.15.28:0/3353751726 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e580cccf0 con 0x7f6e40003a00 2023-11-28T11:13:20.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e5559a700 1 -- 172.21.15.28:0/3353751726 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e4000bb40 con 0x7f6e5815c990 2023-11-28T11:13:20.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e56d9d700 1 --2- 172.21.15.28:0/3353751726 >> v2:172.21.15.59:3300/0 conn(0x7f6e580cc310 0x7f6e580cc7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:20.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e5659c700 1 --2- 172.21.15.28:0/3353751726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e40003a00 0x7f6e40003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:20.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e55d9b700 1 --2- 172.21.15.28:0/3353751726 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e5815c990 0x7f6e40004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:20.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e56d9d700 1 --2- 172.21.15.28:0/3353751726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e40004d90 0x7f6e40005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:20.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e56d9d700 1 -- 172.21.15.28:0/3353751726 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e5815c990 msgr2=0x7f6e40004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:20.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e56d9d700 1 --2- 172.21.15.28:0/3353751726 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e5815c990 0x7f6e40004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:20.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e56d9d700 1 -- 172.21.15.28:0/3353751726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e40003a00 msgr2=0x7f6e40003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:20.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e56d9d700 1 --2- 172.21.15.28:0/3353751726 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e40003a00 0x7f6e40003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:20.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e56d9d700 1 -- 172.21.15.28:0/3353751726 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e400035b0 con 0x7f6e40004d90 2023-11-28T11:13:20.544 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e56d9d700 1 --2- 172.21.15.28:0/3353751726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e40004d90 0x7f6e40005200 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6e500073e0 tx=0x7f6e40005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d39e4147273c13d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:20.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e5559a700 1 -- 172.21.15.28:0/3353751726 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e50014400 con 0x7f6e40004d90 2023-11-28T11:13:20.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e5559a700 1 -- 172.21.15.28:0/3353751726 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e50014560 con 0x7f6e40004d90 2023-11-28T11:13:20.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.521+0000 7f6e5559a700 1 -- 172.21.15.28:0/3353751726 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e500148d0 con 0x7f6e40004d90 2023-11-28T11:13:20.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.522+0000 7f6e5cd46700 1 -- 172.21.15.28:0/3353751726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e40004d90 msgr2=0x7f6e40005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:20.545 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.522+0000 7f6e5cd46700 1 --2- 172.21.15.28:0/3353751726 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e40004d90 0x7f6e40005200 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f6e500073e0 tx=0x7f6e40005880 comp rx=0 tx=0).stop 2023-11-28T11:13:20.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.522+0000 7f6e56d9d700 1 -- 172.21.15.28:0/3353751726 reap_dead start 2023-11-28T11:13:20.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.522+0000 7f6e5cd46700 1 -- 172.21.15.28:0/3353751726 shutdown_connections 2023-11-28T11:13:20.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.522+0000 7f6e5cd46700 1 -- 172.21.15.28:0/3353751726 >> 172.21.15.28:0/3353751726 conn(0x7f6e580b1650 msgr2=0x7f6e580b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:20.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.522+0000 7f6e5cd46700 1 -- 172.21.15.28:0/3353751726 shutdown_connections 2023-11-28T11:13:20.546 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.522+0000 7f6e5cd46700 1 -- 172.21.15.28:0/3353751726 wait complete. 2023-11-28T11:13:20.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.523+0000 7f6e5cd46700 1 Processor -- start 2023-11-28T11:13:20.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.523+0000 7f6e5cd46700 1 -- start start 2023-11-28T11:13:20.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.524+0000 7f6e5cd46700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e40004d90 0x7f6e58160010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:20.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.524+0000 7f6e5cd46700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e58148850 0x7f6e5815e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:20.547 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.524+0000 7f6e5cd46700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e5815cdd0 0x7f6e5814af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:20.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.524+0000 7f6e5cd46700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e58148cc0 con 0x7f6e58148850 2023-11-28T11:13:20.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.524+0000 7f6e5cd46700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5815fb80 con 0x7f6e40004d90 2023-11-28T11:13:20.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.524+0000 7f6e5cd46700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5815ed20 con 0x7f6e5815cdd0 2023-11-28T11:13:20.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.524+0000 7f6e55d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e58148850 0x7f6e5815e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:20.548 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.524+0000 7f6e55d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e58148850 0x7f6e5815e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52426/0 (socket says 172.21.15.28:52426) 2023-11-28T11:13:20.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.524+0000 7f6e55d9b700 1 -- 172.21.15.28:0/471507584 learned_addr learned my addr 172.21.15.28:0/471507584 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:20.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.524+0000 7f6e56d9d700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e5815cdd0 0x7f6e5814af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:20.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.524+0000 7f6e5659c700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e40004d90 0x7f6e58160010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:20.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.524+0000 7f6e55d9b700 1 -- 172.21.15.28:0/471507584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e5815cdd0 msgr2=0x7f6e5814af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:20.549 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.525+0000 7f6e55d9b700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e5815cdd0 0x7f6e5814af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:20.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.525+0000 7f6e55d9b700 1 -- 172.21.15.28:0/471507584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e40004d90 msgr2=0x7f6e58160010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:20.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.525+0000 7f6e55d9b700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e40004d90 0x7f6e58160010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:20.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.525+0000 7f6e55d9b700 1 -- 172.21.15.28:0/471507584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e580ce870 con 0x7f6e58148850 2023-11-28T11:13:20.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.525+0000 7f6e56d9d700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e5815cdd0 0x7f6e5814af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:13:20.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.525+0000 7f6e5659c700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e40004d90 0x7f6e58160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:13:20.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.525+0000 7f6e55d9b700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e58148850 0x7f6e5815e7e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0017a0 tx=0x7f6e4c0076a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:20.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.525+0000 7f6e46ffd700 1 -- 172.21.15.28:0/471507584 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c0077c0 con 0x7f6e58148850 2023-11-28T11:13:20.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.525+0000 7f6e46ffd700 1 -- 172.21.15.28:0/471507584 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e4c00f870 con 0x7f6e58148850 2023-11-28T11:13:20.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.525+0000 7f6e5cd46700 1 -- 172.21.15.28:0/471507584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e580ce590 con 0x7f6e58148850 2023-11-28T11:13:20.551 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.525+0000 7f6e46ffd700 1 -- 172.21.15.28:0/471507584 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c00fc20 con 0x7f6e58148850 2023-11-28T11:13:20.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.525+0000 7f6e5cd46700 1 -- 172.21.15.28:0/471507584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e580cb080 con 0x7f6e58148850 2023-11-28T11:13:20.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.526+0000 7f6e5cd46700 1 -- 172.21.15.28:0/471507584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e580739f0 con 0x7f6e58148850 2023-11-28T11:13:20.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.526+0000 7f6e46ffd700 1 -- 172.21.15.28:0/471507584 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6e4c00fd80 con 0x7f6e58148850 2023-11-28T11:13:20.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.527+0000 7f6e46ffd700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6e38068320 0x7f6e3806a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:20.552 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.527+0000 7f6e46ffd700 1 -- 172.21.15.28:0/471507584 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6088+0+0 (secure 0 0 0) 0x7f6e4c085e50 con 0x7f6e58148850 2023-11-28T11:13:20.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.527+0000 7f6e5659c700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6e38068320 0x7f6e3806a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:20.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.527+0000 7f6e5659c700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6e38068320 0x7f6e3806a7d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009ae0 tx=0x7f6e48008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:20.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.531+0000 7f6e46ffd700 1 -- 172.21.15.28:0/471507584 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6e4c052630 con 0x7f6e58148850 2023-11-28T11:13:20.723 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:20.722+0000 7f6e5cd46700 1 -- 172.21.15.28:0/471507584 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6e5806fce0 con 0x7f6e58148850 2023-11-28T11:13:21.369 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.367+0000 7f6e46ffd700 1 -- 172.21.15.28:0/471507584 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v71) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6e4c055c50 con 0x7f6e58148850 2023-11-28T11:13:21.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.371+0000 7f6e5cd46700 1 -- 172.21.15.28:0/471507584 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6e38068320 msgr2=0x7f6e3806a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:21.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.371+0000 7f6e5cd46700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6e38068320 0x7f6e3806a7d0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009ae0 tx=0x7f6e48008040 comp rx=0 tx=0).stop 2023-11-28T11:13:21.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.371+0000 7f6e5cd46700 1 -- 172.21.15.28:0/471507584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e58148850 msgr2=0x7f6e5815e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:21.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.371+0000 7f6e5cd46700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e58148850 0x7f6e5815e7e0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c0017a0 tx=0x7f6e4c0076a0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.371+0000 7f6e5cd46700 1 -- 172.21.15.28:0/471507584 shutdown_connections 2023-11-28T11:13:21.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.371+0000 7f6e5cd46700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f6e38068320 0x7f6e3806a7d0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.371+0000 7f6e5cd46700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f6e5815cdd0 0x7f6e5814af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.371+0000 7f6e5cd46700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6e40004d90 0x7f6e58160010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.371+0000 7f6e5cd46700 1 --2- 172.21.15.28:0/471507584 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6e58148850 0x7f6e5815e7e0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.374 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.371+0000 7f6e5cd46700 1 -- 172.21.15.28:0/471507584 >> 172.21.15.28:0/471507584 conn(0x7f6e580b1650 msgr2=0x7f6e5806ce30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:21.375 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.371+0000 7f6e5cd46700 1 -- 172.21.15.28:0/471507584 shutdown_connections 2023-11-28T11:13:21.375 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.371+0000 7f6e5cd46700 1 -- 172.21.15.28:0/471507584 wait complete. 2023-11-28T11:13:21.375 INFO:teuthology.orchestra.run.smithi028.stderr:pool 'cephfs_data' removed 2023-11-28T11:13:21.387 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T11:13:21.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.519+0000 7f0059b5e700 1 Processor -- start 2023-11-28T11:13:21.521 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.519+0000 7f0059b5e700 1 -- start start 2023-11-28T11:13:21.522 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:13:21.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.520+0000 7f0059b5e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f00540cf5f0 0x7f00540caea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:21.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.520+0000 7f0059b5e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00540cb3e0 0x7f005406afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:21.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.520+0000 7f0059b5e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00540ce550 con 0x7f00540b1630 2023-11-28T11:13:21.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.520+0000 7f0059b5e700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00540ce830 con 0x7f00540cf5f0 2023-11-28T11:13:21.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.520+0000 7f0059b5e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00540ceb10 con 0x7f00540cb3e0 2023-11-28T11:13:21.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.520+0000 7f0052ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f00540cf5f0 0x7f00540caea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:21.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.520+0000 7f0052ffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f00540cf5f0 0x7f00540caea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52434/0 (socket says 172.21.15.28:52434) 2023-11-28T11:13:21.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.520+0000 7f0052ffd700 1 -- 172.21.15.28:0/4014558774 learned_addr learned my addr 172.21.15.28:0/4014558774 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:21.525 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.521+0000 7f0053fff700 1 --2- 172.21.15.28:0/4014558774 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00540cb3e0 0x7f005406afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:21.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.521+0000 7f0052ffd700 1 -- 172.21.15.28:0/4014558774 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00540cb3e0 msgr2=0x7f005406afc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:21.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.521+0000 7f0052ffd700 1 --2- 172.21.15.28:0/4014558774 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00540cb3e0 0x7f005406afc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.521+0000 7f0052ffd700 1 -- 172.21.15.28:0/4014558774 >> v1:172.21.15.59:6789/0 conn(0x7f00540b1630 legacy=0x7f00540b1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:21.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.521+0000 7f0052ffd700 1 -- 172.21.15.28:0/4014558774 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00540cedf0 con 0x7f00540cf5f0 2023-11-28T11:13:21.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.521+0000 7f0052ffd700 1 --2- 172.21.15.28:0/4014558774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00540cf5f0 0x7f00540caea0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0048009920 tx=0x7f0048009c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9b8ec02503c4beb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:21.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.521+0000 7f00527fc700 1 -- 172.21.15.28:0/4014558774 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0048004230 con 0x7f00540cf5f0 2023-11-28T11:13:21.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.521+0000 7f00527fc700 1 -- 172.21.15.28:0/4014558774 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0048004390 con 0x7f00540cf5f0 2023-11-28T11:13:21.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.521+0000 7f00527fc700 1 -- 172.21.15.28:0/4014558774 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0048009e70 con 0x7f00540cf5f0 2023-11-28T11:13:21.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.521+0000 7f0059b5e700 1 -- 172.21.15.28:0/4014558774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00540cf5f0 msgr2=0x7f00540caea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:21.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.522+0000 7f0059b5e700 1 --2- 172.21.15.28:0/4014558774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00540cf5f0 0x7f00540caea0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f0048009920 tx=0x7f0048009c30 comp rx=0 tx=0).stop 2023-11-28T11:13:21.526 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.522+0000 7f0059b5e700 1 -- 172.21.15.28:0/4014558774 shutdown_connections 2023-11-28T11:13:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.522+0000 7f0059b5e700 1 --2- 172.21.15.28:0/4014558774 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00540cb3e0 0x7f005406afc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.522+0000 7f0059b5e700 1 --2- 172.21.15.28:0/4014558774 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00540cf5f0 0x7f00540caea0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.522+0000 7f0059b5e700 1 -- 172.21.15.28:0/4014558774 >> 172.21.15.28:0/4014558774 conn(0x7f00540cc630 msgr2=0x7f00540cca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.522+0000 7f0059b5e700 1 -- 172.21.15.28:0/4014558774 shutdown_connections 2023-11-28T11:13:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.522+0000 7f0059b5e700 1 -- 172.21.15.28:0/4014558774 wait complete. 2023-11-28T11:13:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.523+0000 7f0059b5e700 1 Processor -- start 2023-11-28T11:13:21.527 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.523+0000 7f0059b5e700 1 -- start start 2023-11-28T11:13:21.528 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:13:21.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f0059b5e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00540b1630 0x7f00540b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:21.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f0059b5e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00540cb3e0 0x7f00540c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:21.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f0059b5e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00540cf5f0 0x7f00541f22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:21.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f0059b5e700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00540b53f0 con 0x7f00540b1630 2023-11-28T11:13:21.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f0059b5e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0054154350 con 0x7f00540cf5f0 2023-11-28T11:13:21.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f0059b5e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00540b3e10 con 0x7f00540cb3e0 2023-11-28T11:13:21.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f00537fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00540b1630 0x7f00540b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:21.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f00537fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00540b1630 0x7f00540b4eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52438/0 (socket says 172.21.15.28:52438) 2023-11-28T11:13:21.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f00537fe700 1 -- 172.21.15.28:0/3933983283 learned_addr learned my addr 172.21.15.28:0/3933983283 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:21.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f0052ffd700 1 --2- 172.21.15.28:0/3933983283 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00540cb3e0 0x7f00540c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:21.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f0053fff700 1 --2- 172.21.15.28:0/3933983283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00540cf5f0 0x7f00541f22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:21.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f00537fe700 1 -- 172.21.15.28:0/3933983283 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00540cb3e0 msgr2=0x7f00540c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:21.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f00537fe700 1 --2- 172.21.15.28:0/3933983283 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00540cb3e0 0x7f00540c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f00537fe700 1 -- 172.21.15.28:0/3933983283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00540cf5f0 msgr2=0x7f00541f22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:21.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f00537fe700 1 --2- 172.21.15.28:0/3933983283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00540cf5f0 0x7f00541f22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f00537fe700 1 -- 172.21.15.28:0/3933983283 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0048009550 con 0x7f00540b1630 2023-11-28T11:13:21.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.524+0000 7f00537fe700 1 --2- 172.21.15.28:0/3933983283 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00540b1630 0x7f00540b4eb0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f004400c4d0 tx=0x7f004400c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:21.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.525+0000 7f003bfff700 1 -- 172.21.15.28:0/3933983283 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044011790 con 0x7f00540b1630 2023-11-28T11:13:21.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.525+0000 7f003bfff700 1 -- 172.21.15.28:0/3933983283 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00440118f0 con 0x7f00540b1630 2023-11-28T11:13:21.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.525+0000 7f0059b5e700 1 -- 172.21.15.28:0/3933983283 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00540ce140 con 0x7f00540b1630 2023-11-28T11:13:21.532 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.525+0000 7f003bfff700 1 -- 172.21.15.28:0/3933983283 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0044011ca0 con 0x7f00540b1630 2023-11-28T11:13:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.525+0000 7f0059b5e700 1 -- 172.21.15.28:0/3933983283 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00540cf0d0 con 0x7f00540b1630 2023-11-28T11:13:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.526+0000 7f0059b5e700 1 -- 172.21.15.28:0/3933983283 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00540ce2d0 con 0x7f00540b1630 2023-11-28T11:13:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.526+0000 7f003bfff700 1 -- 172.21.15.28:0/3933983283 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0044010050 con 0x7f00540b1630 2023-11-28T11:13:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.526+0000 7f003bfff700 1 --2- 172.21.15.28:0/3933983283 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00400682f0 0x7f004006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.527+0000 7f003bfff700 1 -- 172.21.15.28:0/3933983283 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f00440857a0 con 0x7f00540b1630 2023-11-28T11:13:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.527+0000 7f0052ffd700 1 --2- 172.21.15.28:0/3933983283 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00400682f0 0x7f004006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:21.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.527+0000 7f0052ffd700 1 --2- 172.21.15.28:0/3933983283 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00400682f0 0x7f004006a7a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0048009130 tx=0x7f0048008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:21.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.531+0000 7f003bfff700 1 -- 172.21.15.28:0/3933983283 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f00440521c0 con 0x7f00540b1630 2023-11-28T11:13:21.731 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.729+0000 7f0059b5e700 1 -- 172.21.15.28:0/3933983283 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f00540739f0 con 0x7f00540b1630 2023-11-28T11:13:21.734 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.733+0000 7f003bfff700 1 -- 172.21.15.28:0/3933983283 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7f00440557e0 con 0x7f00540b1630 2023-11-28T11:13:21.736 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.735+0000 7f0059b5e700 1 -- 172.21.15.28:0/3933983283 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00400682f0 msgr2=0x7f004006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.735+0000 7f0059b5e700 1 --2- 172.21.15.28:0/3933983283 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00400682f0 0x7f004006a7a0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f0048009130 tx=0x7f0048008030 comp rx=0 tx=0).stop 2023-11-28T11:13:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.735+0000 7f0059b5e700 1 -- 172.21.15.28:0/3933983283 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00540b1630 msgr2=0x7f00540b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.735+0000 7f0059b5e700 1 --2- 172.21.15.28:0/3933983283 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00540b1630 0x7f00540b4eb0 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7f004400c4d0 tx=0x7f004400c890 comp rx=0 tx=0).stop 2023-11-28T11:13:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.735+0000 7f0059b5e700 1 -- 172.21.15.28:0/3933983283 shutdown_connections 2023-11-28T11:13:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.735+0000 7f0059b5e700 1 --2- 172.21.15.28:0/3933983283 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f00400682f0 0x7f004006a7a0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.735+0000 7f0059b5e700 1 --2- 172.21.15.28:0/3933983283 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f00540b1630 0x7f00540b4eb0 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.735+0000 7f0059b5e700 1 --2- 172.21.15.28:0/3933983283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f00540cf5f0 0x7f00541f22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.735+0000 7f0059b5e700 1 --2- 172.21.15.28:0/3933983283 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f00540cb3e0 0x7f00540c98f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.735+0000 7f0059b5e700 1 -- 172.21.15.28:0/3933983283 >> 172.21.15.28:0/3933983283 conn(0x7f00540cc630 msgr2=0x7f00540b1a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.735+0000 7f0059b5e700 1 -- 172.21.15.28:0/3933983283 shutdown_connections 2023-11-28T11:13:21.737 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.735+0000 7f0059b5e700 1 -- 172.21.15.28:0/3933983283 wait complete. 2023-11-28T11:13:21.748 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair' 2023-11-28T11:13:21.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.876+0000 7fab54e3f700 1 Processor -- start 2023-11-28T11:13:21.878 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.876+0000 7fab54e3f700 1 -- start start 2023-11-28T11:13:21.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.877+0000 7fab54e3f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fab50159590 0x7fab5006e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:21.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.877+0000 7fab54e3f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fab5015db70 0x7fab500ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:21.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.877+0000 7fab54e3f700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab5015d120 con 0x7fab500ce890 2023-11-28T11:13:21.881 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.877+0000 7fab54e3f700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab5015d6e0 con 0x7fab5015db70 2023-11-28T11:13:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.877+0000 7fab54e3f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab500aef60 con 0x7fab50159590 2023-11-28T11:13:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.878+0000 7fab4dd9b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fab5015db70 0x7fab500ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.878+0000 7fab4ed9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fab500ce890 0x7fab500caf50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40506/0 (socket says 172.21.15.28:40506) 2023-11-28T11:13:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.878+0000 7fab4ed9d700 1 -- 172.21.15.28:0/2029366370 learned_addr learned my addr 172.21.15.28:0/2029366370 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.878+0000 7fab4dd9b700 1 -- 172.21.15.28:0/2029366370 >> v2:172.21.15.59:3300/0 conn(0x7fab50159590 msgr2=0x7fab5006e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.878+0000 7fab4dd9b700 1 --2- 172.21.15.28:0/2029366370 >> v2:172.21.15.59:3300/0 conn(0x7fab50159590 0x7fab5006e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.878+0000 7fab4dd9b700 1 -- 172.21.15.28:0/2029366370 >> v1:172.21.15.28:6789/0 conn(0x7fab500ce890 legacy=0x7fab500caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.878+0000 7fab4d59a700 1 -- 172.21.15.28:0/2029366370 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163027154 0 0) 0x7fab5015d120 con 0x7fab500ce890 2023-11-28T11:13:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.878+0000 7fab4dd9b700 1 -- 172.21.15.28:0/2029366370 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab500af240 con 0x7fab5015db70 2023-11-28T11:13:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.878+0000 7fab4dd9b700 1 --2- 172.21.15.28:0/2029366370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab5015db70 0x7fab500ae730 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fab5006bd10 tx=0x7fab40009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2911da6801f09ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:21.882 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.878+0000 7fab4d59a700 1 -- 172.21.15.28:0/2029366370 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40011830 con 0x7fab5015db70 2023-11-28T11:13:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.878+0000 7fab4d59a700 1 -- 172.21.15.28:0/2029366370 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab40011990 con 0x7fab5015db70 2023-11-28T11:13:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.878+0000 7fab4d59a700 1 -- 172.21.15.28:0/2029366370 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab40011d20 con 0x7fab5015db70 2023-11-28T11:13:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.879+0000 7fab54e3f700 1 -- 172.21.15.28:0/2029366370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab5015db70 msgr2=0x7fab500ae730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.879+0000 7fab54e3f700 1 --2- 172.21.15.28:0/2029366370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab5015db70 0x7fab500ae730 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fab5006bd10 tx=0x7fab40009b20 comp rx=0 tx=0).stop 2023-11-28T11:13:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.879+0000 7fab54e3f700 1 -- 172.21.15.28:0/2029366370 shutdown_connections 2023-11-28T11:13:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.879+0000 7fab54e3f700 1 --2- 172.21.15.28:0/2029366370 >> v2:172.21.15.59:3300/0 conn(0x7fab50159590 0x7fab5006e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.879+0000 7fab54e3f700 1 --2- 172.21.15.28:0/2029366370 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab5015db70 0x7fab500ae730 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.883 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.879+0000 7fab54e3f700 1 -- 172.21.15.28:0/2029366370 >> 172.21.15.28:0/2029366370 conn(0x7fab500b1650 msgr2=0x7fab500b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:21.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.879+0000 7fab54e3f700 1 -- 172.21.15.28:0/2029366370 shutdown_connections 2023-11-28T11:13:21.884 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.879+0000 7fab54e3f700 1 -- 172.21.15.28:0/2029366370 wait complete. 2023-11-28T11:13:21.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.880+0000 7fab54e3f700 1 Processor -- start 2023-11-28T11:13:21.887 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.880+0000 7fab54e3f700 1 -- start start 2023-11-28T11:13:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.881+0000 7fab54e3f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab500ce890 0x7fab500c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.881+0000 7fab54e3f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab50159590 0x7fab500c2800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.881+0000 7fab54e3f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab5015db70 0x7fab501439e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.881+0000 7fab54e3f700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab500c7d60 con 0x7fab500ce890 2023-11-28T11:13:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.881+0000 7fab54e3f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab50145910 con 0x7fab5015db70 2023-11-28T11:13:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.881+0000 7fab54e3f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab500c7150 con 0x7fab50159590 2023-11-28T11:13:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.881+0000 7fab4e59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab500ce890 0x7fab500c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.881+0000 7fab4e59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab500ce890 0x7fab500c6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52450/0 (socket says 172.21.15.28:52450) 2023-11-28T11:13:21.888 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.881+0000 7fab4e59c700 1 -- 172.21.15.28:0/1168137634 learned_addr learned my addr 172.21.15.28:0/1168137634 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:21.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.881+0000 7fab4ed9d700 1 --2- 172.21.15.28:0/1168137634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab5015db70 0x7fab501439e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:21.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.881+0000 7fab4e59c700 1 -- 172.21.15.28:0/1168137634 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab50159590 msgr2=0x7fab500c2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:21.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.881+0000 7fab4e59c700 1 --2- 172.21.15.28:0/1168137634 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab50159590 0x7fab500c2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.881+0000 7fab4e59c700 1 -- 172.21.15.28:0/1168137634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab5015db70 msgr2=0x7fab501439e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:21.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.882+0000 7fab4e59c700 1 --2- 172.21.15.28:0/1168137634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab5015db70 0x7fab501439e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:21.889 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.882+0000 7fab4e59c700 1 -- 172.21.15.28:0/1168137634 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab400094a0 con 0x7fab500ce890 2023-11-28T11:13:21.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.882+0000 7fab4ed9d700 1 --2- 172.21.15.28:0/1168137634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab5015db70 0x7fab501439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:13:21.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.882+0000 7fab4e59c700 1 --2- 172.21.15.28:0/1168137634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab500ce890 0x7fab500c6b70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fab4400b530 tx=0x7fab4400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:21.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.882+0000 7fab3effd700 1 -- 172.21.15.28:0/1168137634 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44011830 con 0x7fab500ce890 2023-11-28T11:13:21.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.882+0000 7fab3effd700 1 -- 172.21.15.28:0/1168137634 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab440119b0 con 0x7fab500ce890 2023-11-28T11:13:21.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.882+0000 7fab54e3f700 1 -- 172.21.15.28:0/1168137634 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5015e010 con 0x7fab500ce890 2023-11-28T11:13:21.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.882+0000 7fab3effd700 1 -- 172.21.15.28:0/1168137634 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab44011d60 con 0x7fab500ce890 2023-11-28T11:13:21.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.882+0000 7fab54e3f700 1 -- 172.21.15.28:0/1168137634 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5015d370 con 0x7fab500ce890 2023-11-28T11:13:21.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.883+0000 7fab54e3f700 1 -- 172.21.15.28:0/1168137634 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab500aec70 con 0x7fab500ce890 2023-11-28T11:13:21.890 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.883+0000 7fab3effd700 1 -- 172.21.15.28:0/1168137634 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fab44021070 con 0x7fab500ce890 2023-11-28T11:13:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.884+0000 7fab3effd700 1 --2- 172.21.15.28:0/1168137634 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fab300682f0 0x7fab3006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.884+0000 7fab3effd700 1 -- 172.21.15.28:0/1168137634 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fab44085af0 con 0x7fab500ce890 2023-11-28T11:13:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.884+0000 7fab4dd9b700 1 --2- 172.21.15.28:0/1168137634 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fab300682f0 0x7fab3006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.884+0000 7fab4dd9b700 1 --2- 172.21.15.28:0/1168137634 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fab300682f0 0x7fab3006a7a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fab500c5730 tx=0x7fab40010000 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:21.891 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:21.888+0000 7fab3effd700 1 -- 172.21.15.28:0/1168137634 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fab440524b0 con 0x7fab500ce890 2023-11-28T11:13:22.165 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:22.162+0000 7fab54e3f700 1 -- 172.21.15.28:0/1168137634 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]} v 0) v1 -- 0x7fab500675a0 con 0x7fab500ce890 2023-11-28T11:13:22.368 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:22.367+0000 7fab3effd700 1 -- 172.21.15.28:0/1168137634 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_forward_scrub.TestForwardScrub.test_health_status_after_backtrace_repair"]}]=0 v243) v1 ==== 169+0+0 (secure 0 0 0) 0x7fab44015020 con 0x7fab500ce890 2023-11-28T11:13:22.371 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:22.369+0000 7fab54e3f700 1 -- 172.21.15.28:0/1168137634 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fab300682f0 msgr2=0x7fab3006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:22.371 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:22.369+0000 7fab54e3f700 1 --2- 172.21.15.28:0/1168137634 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fab300682f0 0x7fab3006a7a0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fab500c5730 tx=0x7fab40010000 comp rx=0 tx=0).stop 2023-11-28T11:13:22.371 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:22.369+0000 7fab54e3f700 1 -- 172.21.15.28:0/1168137634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab500ce890 msgr2=0x7fab500c6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:22.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:22.369+0000 7fab54e3f700 1 --2- 172.21.15.28:0/1168137634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab500ce890 0x7fab500c6b70 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fab4400b530 tx=0x7fab4400b8f0 comp rx=0 tx=0).stop 2023-11-28T11:13:22.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:22.369+0000 7fab54e3f700 1 -- 172.21.15.28:0/1168137634 shutdown_connections 2023-11-28T11:13:22.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:22.369+0000 7fab54e3f700 1 --2- 172.21.15.28:0/1168137634 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fab300682f0 0x7fab3006a7a0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:22.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:22.369+0000 7fab54e3f700 1 --2- 172.21.15.28:0/1168137634 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fab500ce890 0x7fab500c6b70 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:22.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:22.369+0000 7fab54e3f700 1 --2- 172.21.15.28:0/1168137634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fab5015db70 0x7fab501439e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:22.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:22.369+0000 7fab54e3f700 1 --2- 172.21.15.28:0/1168137634 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fab50159590 0x7fab500c2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:22.372 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:22.369+0000 7fab54e3f700 1 -- 172.21.15.28:0/1168137634 >> 172.21.15.28:0/1168137634 conn(0x7fab500b1650 msgr2=0x7fab500cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:22.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:22.369+0000 7fab54e3f700 1 -- 172.21.15.28:0/1168137634 shutdown_connections 2023-11-28T11:13:22.373 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:22.369+0000 7fab54e3f700 1 -- 172.21.15.28:0/1168137634 wait complete. 2023-11-28T11:13:22.384 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared ... ERROR 2023-11-28T11:13:22.385 INFO:tasks.cephfs_test_runner: 2023-11-28T11:13:22.386 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T11:13:22.386 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:13:22.386 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T11:13:22.386 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:13:22.386 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T11:13:22.386 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T11:13:22.386 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=20) 2023-11-28T11:13:22.386 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T11:13:22.386 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T11:13:22.386 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 20s and 0 retries 2023-11-28T11:13:22.386 INFO:tasks.cephfs_test_runner: 2023-11-28T11:13:22.387 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:13:22.387 INFO:tasks.cephfs_test_runner:Ran 3 tests in 231.033s 2023-11-28T11:13:22.387 INFO:tasks.cephfs_test_runner: 2023-11-28T11:13:22.387 INFO:tasks.cephfs_test_runner:FAILED (errors=1) 2023-11-28T11:13:22.387 INFO:tasks.cephfs_test_runner: 2023-11-28T11:13:22.387 INFO:tasks.cephfs_test_runner:====================================================================== 2023-11-28T11:13:22.387 INFO:tasks.cephfs_test_runner:ERROR: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:13:22.387 INFO:tasks.cephfs_test_runner:Test that the damage health status is cleared 2023-11-28T11:13:22.387 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T11:13:22.387 INFO:tasks.cephfs_test_runner:Traceback (most recent call last): 2023-11-28T11:13:22.387 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs/test_forward_scrub.py", line 500, in test_health_status_after_backtrace_repair 2023-11-28T11:13:22.387 INFO:tasks.cephfs_test_runner: self.wait_until_true(lambda: self._check_for_presence_of_MDS_damage(), timeout=20) 2023-11-28T11:13:22.387 INFO:tasks.cephfs_test_runner: File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/ceph_test_case.py", line 212, in wait_until_true 2023-11-28T11:13:22.388 INFO:tasks.cephfs_test_runner: raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count)) 2023-11-28T11:13:22.388 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 20s and 0 retries 2023-11-28T11:13:22.388 INFO:tasks.cephfs_test_runner: 2023-11-28T11:13:22.388 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:13:22.518 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=f9ae253d36d34e5c92804e361ee81e0e Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5ae09a5cc7064ca2bf994df19f3006b79794789a/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_neesingh-rh_ceph_e67b67438c1e5e13734a7203ff9e57e30b8e4d32/qa/tasks/cephfs_test_runner.py", line 211, in task raise RuntimeError("Test failure: {0}".format(", ".join(bad_tests))) RuntimeError: Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub) 2023-11-28T11:13:22.521 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-28T11:13:22.532 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-11-28T11:13:22.597 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-11-28T11:13:22.598 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:13:22.598 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:13:22.620 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:13:22.620 DEBUG:teuthology.orchestra.run.smithi028:> set -ex 2023-11-28T11:13:22.621 DEBUG:teuthology.orchestra.run.smithi028:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:13:22.679 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=42,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19542 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw,seclabel 0 0 2023-11-28T11:13:22.679 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:13:22.679 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:13:22.701 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3961 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:13:22.701 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2023-11-28T11:13:22.702 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2023-11-28T11:13:22.763 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=16287364k,nr_inodes=4071841,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 tmpfs /sys/fs/cgroup tmpfs ro,seclabel,nosuid,nodev,noexec,mode=755 0 0 cgroup /sys/fs/cgroup/systemd cgroup rw,seclabel,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 cgroup /sys/fs/cgroup/freezer cgroup rw,seclabel,nosuid,nodev,noexec,relatime,freezer 0 0 cgroup /sys/fs/cgroup/blkio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,blkio 0 0 cgroup /sys/fs/cgroup/net_cls,net_prio cgroup rw,seclabel,nosuid,nodev,noexec,relatime,net_cls,net_prio 0 0 cgroup /sys/fs/cgroup/cpu,cpuacct cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpu,cpuacct 0 0 cgroup /sys/fs/cgroup/pids cgroup rw,seclabel,nosuid,nodev,noexec,relatime,pids 0 0 cgroup /sys/fs/cgroup/rdma cgroup rw,seclabel,nosuid,nodev,noexec,relatime,rdma 0 0 cgroup /sys/fs/cgroup/memory cgroup rw,seclabel,nosuid,nodev,noexec,relatime,memory 0 0 cgroup /sys/fs/cgroup/hugetlb cgroup rw,seclabel,nosuid,nodev,noexec,relatime,hugetlb 0 0 cgroup /sys/fs/cgroup/cpuset cgroup rw,seclabel,nosuid,nodev,noexec,relatime,cpuset 0 0 cgroup /sys/fs/cgroup/devices cgroup rw,seclabel,nosuid,nodev,noexec,relatime,devices 0 0 cgroup /sys/fs/cgroup/perf_event cgroup rw,seclabel,nosuid,nodev,noexec,relatime,perf_event 0 0 none /sys/kernel/tracing tracefs rw,seclabel,relatime 0 0 configfs /sys/kernel/config configfs rw,relatime 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,relatime 0 0 mqueue /dev/mqueue mqueue rw,seclabel,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,relatime 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=45,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=3961 0 0 fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3261172k,mode=700,uid=1000,gid=1265 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw,seclabel 0 0 2023-11-28T11:13:22.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:13:22.763 DEBUG:teuthology.orchestra.run.smithi028:> ip netns list 2023-11-28T11:13:22.782 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T11:13:22.782 INFO:teuthology.orchestra.run.smithi028.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T11:13:22.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:13:22.783 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T11:13:22.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:13:22.863 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T11:13:22.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:13:22.960 DEBUG:teuthology.orchestra.run.smithi028:> sudo ip link delete ceph-brx 2023-11-28T11:13:23.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:13:23.060 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2023-11-28T11:13:23.081 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-28T11:13:23.081 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T11:13:23.081 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:13:23.082 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T11:13:23.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:13:23.168 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T11:13:23.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T11:13:23.256 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2023-11-28T11:13:23.354 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-28T11:13:23.363 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T11:13:23.363 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:13:23.495 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.494+0000 7fdbc3a5e700 1 Processor -- start 2023-11-28T11:13:23.496 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.494+0000 7fdbc3a5e700 1 -- start start 2023-11-28T11:13:23.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.495+0000 7fdbc3a5e700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdbbc15a230 0x7fdbbc15d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:23.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.495+0000 7fdbc3a5e700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdbbc0cd4a0 0x7fdbbc0b48d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:23.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.495+0000 7fdbc3a5e700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbbc0d1a30 con 0x7fdbbc0b4e10 2023-11-28T11:13:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.495+0000 7fdbc3a5e700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbbc15a600 con 0x7fdbbc15a230 2023-11-28T11:13:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.495+0000 7fdbc3a5e700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbbc0b5290 con 0x7fdbbc0cd4a0 2023-11-28T11:13:23.501 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.495+0000 7fdbc17fa700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdbbc15a230 0x7fdbbc15d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.495+0000 7fdbc17fa700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fdbbc15a230 0x7fdbbc15d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52458/0 (socket says 172.21.15.28:52458) 2023-11-28T11:13:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.495+0000 7fdbc17fa700 1 -- 172.21.15.28:0/1870989183 learned_addr learned my addr 172.21.15.28:0/1870989183 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:23.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.495+0000 7fdbc0ff9700 1 --2- 172.21.15.28:0/1870989183 >> v2:172.21.15.59:3300/0 conn(0x7fdbbc0cd4a0 0x7fdbbc0b48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:23.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.496+0000 7fdbc17fa700 1 -- 172.21.15.28:0/1870989183 >> v2:172.21.15.59:3300/0 conn(0x7fdbbc0cd4a0 msgr2=0x7fdbbc0b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:23.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.496+0000 7fdbc17fa700 1 --2- 172.21.15.28:0/1870989183 >> v2:172.21.15.59:3300/0 conn(0x7fdbbc0cd4a0 0x7fdbbc0b48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.496+0000 7fdbc17fa700 1 -- 172.21.15.28:0/1870989183 >> v1:172.21.15.59:6789/0 conn(0x7fdbbc0b4e10 legacy=0x7fdbbc160110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.496+0000 7fdbc17fa700 1 -- 172.21.15.28:0/1870989183 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbbc06abb0 con 0x7fdbbc15a230 2023-11-28T11:13:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.496+0000 7fdbc0ff9700 1 --2- 172.21.15.28:0/1870989183 >> v2:172.21.15.59:3300/0 conn(0x7fdbbc0cd4a0 0x7fdbbc0b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:13:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.496+0000 7fdbc17fa700 1 --2- 172.21.15.28:0/1870989183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbbc15a230 0x7fdbbc15d6b0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0d2e20 tx=0x7fdbb8009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed58db6993fa2e97 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.496+0000 7fdbb3fff700 1 -- 172.21.15.28:0/1870989183 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb8004230 con 0x7fdbbc15a230 2023-11-28T11:13:23.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.496+0000 7fdbb3fff700 1 -- 172.21.15.28:0/1870989183 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb8004390 con 0x7fdbbc15a230 2023-11-28T11:13:23.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.496+0000 7fdbb3fff700 1 -- 172.21.15.28:0/1870989183 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb80036a0 con 0x7fdbbc15a230 2023-11-28T11:13:23.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.496+0000 7fdbc3a5e700 1 -- 172.21.15.28:0/1870989183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbbc15a230 msgr2=0x7fdbbc15d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:23.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.496+0000 7fdbc3a5e700 1 --2- 172.21.15.28:0/1870989183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbbc15a230 0x7fdbbc15d6b0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0d2e20 tx=0x7fdbb8009bd0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.505 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.496+0000 7fdbc3a5e700 1 -- 172.21.15.28:0/1870989183 shutdown_connections 2023-11-28T11:13:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.496+0000 7fdbc3a5e700 1 --2- 172.21.15.28:0/1870989183 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbbc15a230 0x7fdbbc15d6b0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.496+0000 7fdbc3a5e700 1 --2- 172.21.15.28:0/1870989183 >> v2:172.21.15.59:3300/0 conn(0x7fdbbc0cd4a0 0x7fdbbc0b48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.496+0000 7fdbc3a5e700 1 -- 172.21.15.28:0/1870989183 >> 172.21.15.28:0/1870989183 conn(0x7fdbbc0cf8d0 msgr2=0x7fdbbc0cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:23.506 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.497+0000 7fdbc3a5e700 1 -- 172.21.15.28:0/1870989183 shutdown_connections 2023-11-28T11:13:23.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.497+0000 7fdbc3a5e700 1 -- 172.21.15.28:0/1870989183 wait complete. 2023-11-28T11:13:23.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.498+0000 7fdbc3a5e700 1 Processor -- start 2023-11-28T11:13:23.507 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.498+0000 7fdbc3a5e700 1 -- start start 2023-11-28T11:13:23.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.499+0000 7fdbc3a5e700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdbbc0b4e10 0x7fdbbc0c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:23.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.499+0000 7fdbc3a5e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc0cd4a0 0x7fdbbc14ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:23.511 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.499+0000 7fdbc3a5e700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbbc15a230 0x7fdbbc14b380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.499+0000 7fdbc3a5e700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc0c73a0 con 0x7fdbbc15a230 2023-11-28T11:13:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.499+0000 7fdbc3a5e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc0c7640 con 0x7fdbbc0cd4a0 2023-11-28T11:13:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.499+0000 7fdbc3a5e700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbbc0c7920 con 0x7fdbbc0b4e10 2023-11-28T11:13:23.512 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.499+0000 7fdbc1ffb700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbbc15a230 0x7fdbbc14b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.499+0000 7fdbc1ffb700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbbc15a230 0x7fdbbc14b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52466/0 (socket says 172.21.15.28:52466) 2023-11-28T11:13:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.499+0000 7fdbc1ffb700 1 -- 172.21.15.28:0/1955294722 learned_addr learned my addr 172.21.15.28:0/1955294722 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.499+0000 7fdbc0ff9700 1 --2- 172.21.15.28:0/1955294722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc0cd4a0 0x7fdbbc14ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:23.513 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.499+0000 7fdbc17fa700 1 --2- 172.21.15.28:0/1955294722 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdbbc0b4e10 0x7fdbbc0c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:23.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.500+0000 7fdbc1ffb700 1 -- 172.21.15.28:0/1955294722 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdbbc0b4e10 msgr2=0x7fdbbc0c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:23.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.500+0000 7fdbc1ffb700 1 --2- 172.21.15.28:0/1955294722 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdbbc0b4e10 0x7fdbbc0c6e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.500+0000 7fdbc1ffb700 1 -- 172.21.15.28:0/1955294722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc0cd4a0 msgr2=0x7fdbbc14ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:23.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.500+0000 7fdbc1ffb700 1 --2- 172.21.15.28:0/1955294722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc0cd4a0 0x7fdbbc14ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.514 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.500+0000 7fdbc1ffb700 1 -- 172.21.15.28:0/1955294722 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb8009550 con 0x7fdbbc15a230 2023-11-28T11:13:23.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.500+0000 7fdbc1ffb700 1 --2- 172.21.15.28:0/1955294722 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbbc15a230 0x7fdbbc14b380 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400cc10 tx=0x7fdbb400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:23.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.500+0000 7fdbb1ffb700 1 -- 172.21.15.28:0/1955294722 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb400abb0 con 0x7fdbbc15a230 2023-11-28T11:13:23.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.500+0000 7fdbb1ffb700 1 -- 172.21.15.28:0/1955294722 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbb400ad10 con 0x7fdbbc15a230 2023-11-28T11:13:23.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.500+0000 7fdbc3a5e700 1 -- 172.21.15.28:0/1955294722 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbbc06a560 con 0x7fdbbc15a230 2023-11-28T11:13:23.515 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.500+0000 7fdbb1ffb700 1 -- 172.21.15.28:0/1955294722 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbb4012660 con 0x7fdbbc15a230 2023-11-28T11:13:23.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.500+0000 7fdbc3a5e700 1 -- 172.21.15.28:0/1955294722 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbbc06a280 con 0x7fdbbc15a230 2023-11-28T11:13:23.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.501+0000 7fdbc3a5e700 1 -- 172.21.15.28:0/1955294722 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbbc06a840 con 0x7fdbbc15a230 2023-11-28T11:13:23.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.501+0000 7fdbb1ffb700 1 -- 172.21.15.28:0/1955294722 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdbb4022070 con 0x7fdbbc15a230 2023-11-28T11:13:23.516 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.502+0000 7fdbb1ffb700 1 --2- 172.21.15.28:0/1955294722 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdba00682f0 0x7fdba006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:23.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.502+0000 7fdbb1ffb700 1 -- 172.21.15.28:0/1955294722 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fdbb40861e0 con 0x7fdbbc15a230 2023-11-28T11:13:23.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.502+0000 7fdbc17fa700 1 --2- 172.21.15.28:0/1955294722 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdba00682f0 0x7fdba006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:23.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.502+0000 7fdbc17fa700 1 --2- 172.21.15.28:0/1955294722 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdba00682f0 0x7fdba006a7a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0ca270 tx=0x7fdbb8008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:23.517 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.506+0000 7fdbb1ffb700 1 -- 172.21.15.28:0/1955294722 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fdbb4052b30 con 0x7fdbbc15a230 2023-11-28T11:13:23.697 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.696+0000 7fdbc3a5e700 1 -- 172.21.15.28:0/1955294722 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdbbc0675a0 con 0x7fdba00682f0 2023-11-28T11:13:23.700 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.698+0000 7fdbb1ffb700 1 -- 172.21.15.28:0/1955294722 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72816 (secure 0 0 0) 0x7fdbbc0675a0 con 0x7fdba00682f0 2023-11-28T11:13:23.702 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:23.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.700+0000 7fdbc3a5e700 1 -- 172.21.15.28:0/1955294722 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdba00682f0 msgr2=0x7fdba006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:23.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.700+0000 7fdbc3a5e700 1 --2- 172.21.15.28:0/1955294722 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdba00682f0 0x7fdba006a7a0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fdbbc0ca270 tx=0x7fdbb8008030 comp rx=0 tx=0).stop 2023-11-28T11:13:23.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.700+0000 7fdbc3a5e700 1 -- 172.21.15.28:0/1955294722 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbbc15a230 msgr2=0x7fdbbc14b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:23.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.700+0000 7fdbc3a5e700 1 --2- 172.21.15.28:0/1955294722 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbbc15a230 0x7fdbbc14b380 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fdbb400cc10 tx=0x7fdbb400a060 comp rx=0 tx=0).stop 2023-11-28T11:13:23.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.700+0000 7fdbc3a5e700 1 -- 172.21.15.28:0/1955294722 shutdown_connections 2023-11-28T11:13:23.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.700+0000 7fdbc3a5e700 1 --2- 172.21.15.28:0/1955294722 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdba00682f0 0x7fdba006a7a0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.701+0000 7fdbc3a5e700 1 --2- 172.21.15.28:0/1955294722 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdbbc15a230 0x7fdbbc14b380 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.701+0000 7fdbc3a5e700 1 --2- 172.21.15.28:0/1955294722 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdbbc0b4e10 0x7fdbbc0c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.701+0000 7fdbc3a5e700 1 --2- 172.21.15.28:0/1955294722 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbbc0cd4a0 0x7fdbbc14ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.701+0000 7fdbc3a5e700 1 -- 172.21.15.28:0/1955294722 >> 172.21.15.28:0/1955294722 conn(0x7fdbbc0cf8d0 msgr2=0x7fdbbc15e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:23.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.701+0000 7fdbc3a5e700 1 -- 172.21.15.28:0/1955294722 shutdown_connections 2023-11-28T11:13:23.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.701+0000 7fdbc3a5e700 1 -- 172.21.15.28:0/1955294722 wait complete. 2023-11-28T11:13:23.707 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:13:23.714 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2023-11-28T11:13:23.227399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":81216,"kb_used_data":7680,"kb_used_omap":0,"kb_used_meta":73472,"kb_avail":754893504,"statfs":{"total":773094113280,"available":773010948096,"internally_reserved":0,"allocated":7864320,"data_stored":4319543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":75235328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12289,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-74,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001406"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:53.259553+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:12:53.259553+0000","last_peered":"2023-11-28T11:12:53.259553+0000","last_clean":"2023-11-28T11:12:53.259553+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:12:53.259553+0000","last_undegraded":"2023-11-28T11:12:53.259553+0000","last_fullsized":"2023-11-28T11:12:53.259553+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:35.286107+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:12:35.286107+0000","last_peered":"2023-11-28T11:12:35.286107+0000","last_clean":"2023-11-28T11:12:35.286107+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:12:35.286107+0000","last_undegraded":"2023-11-28T11:12:35.286107+0000","last_fullsized":"2023-11-28T11:12:35.286107+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:35.779428+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:12:35.779428+0000","last_peered":"2023-11-28T11:12:35.779428+0000","last_clean":"2023-11-28T11:12:35.779428+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:12:35.779428+0000","last_undegraded":"2023-11-28T11:12:35.779428+0000","last_fullsized":"2023-11-28T11:12:35.779428+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":206,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:43.258422+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:12:43.258422+0000","last_peered":"2023-11-28T11:12:43.258422+0000","last_clean":"2023-11-28T11:12:43.258422+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:12:43.258422+0000","last_undegraded":"2023-11-28T11:12:43.258422+0000","last_fullsized":"2023-11-28T11:12:43.258422+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:53.259590+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:12:53.259590+0000","last_peered":"2023-11-28T11:12:53.259590+0000","last_clean":"2023-11-28T11:12:53.259590+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:12:53.259590+0000","last_undegraded":"2023-11-28T11:12:53.259590+0000","last_fullsized":"2023-11-28T11:12:53.259590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:43.258405+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:12:43.258405+0000","last_peered":"2023-11-28T11:12:43.258405+0000","last_clean":"2023-11-28T11:12:43.258405+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:12:43.258405+0000","last_undegraded":"2023-11-28T11:12:43.258405+0000","last_fullsized":"2023-11-28T11:12:43.258405+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:53.255623+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:12:53.255623+0000","last_peered":"2023-11-28T11:12:53.255623+0000","last_clean":"2023-11-28T11:12:53.255623+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:12:53.255623+0000","last_undegraded":"2023-11-28T11:12:53.255623+0000","last_fullsized":"2023-11-28T11:12:53.255623+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":55,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:35.291076+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:12:35.291076+0000","last_peered":"2023-11-28T11:12:35.291076+0000","last_clean":"2023-11-28T11:12:35.291076+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:12:35.291076+0000","last_undegraded":"2023-11-28T11:12:35.291076+0000","last_fullsized":"2023-11-28T11:12:35.291076+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":55,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:35.291623+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:12:35.291623+0000","last_peered":"2023-11-28T11:12:35.291623+0000","last_clean":"2023-11-28T11:12:35.291623+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:12:35.291623+0000","last_undegraded":"2023-11-28T11:12:35.291623+0000","last_fullsized":"2023-11-28T11:12:35.291623+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11000,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360840,"statfs":{"total":96636764160,"available":96625500160,"internally_reserved":0,"allocated":1425408,"data_stored":979508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57099999999999995,"15min":0.63100000000000001},"min":{"1min":0.38700000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72599999999999998,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59399999999999997,"15min":0.63300000000000001},"min":{"1min":0.309,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.77000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.61299999999999999,"15min":0.67100000000000004},"min":{"1min":0.38,"5min":0.316,"15min":0.316},"max":{"1min":0.83399999999999996,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58299999999999996,"15min":0.59299999999999997},"min":{"1min":0.41199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73699999999999999,"5min":1.224,"15min":1.224},"last":0.72999999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.63100000000000001,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84999999999999998,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.69499999999999995,"15min":0.76300000000000001},"min":{"1min":0.44900000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88200000000000001,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.60399999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64400000000000002,"15min":0.68999999999999995},"min":{"1min":0.45800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76500000000000001,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.69699999999999995,"15min":0.75700000000000001},"min":{"1min":0.48499999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.78600000000000003,"5min":3.294,"15min":3.294},"last":0.74299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.373,"5min":0.312,"15min":0.312},"max":{"1min":0.77600000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.56200000000000006},"min":{"1min":0.34399999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.79500000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51200000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.63800000000000001},"min":{"1min":0.41299999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81100000000000005,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.63300000000000001,"15min":0.64500000000000002},"min":{"1min":0.434,"5min":0.373,"15min":0.373},"max":{"1min":0.86599999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.54800000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85599999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9780,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362060,"statfs":{"total":96636764160,"available":96626749440,"internally_reserved":0,"allocated":831488,"data_stored":388632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.51100000000000001,"15min":0.47099999999999997},"min":{"1min":0.376,"5min":0.252,"15min":0.252},"max":{"1min":0.79300000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.51200000000000001,"15min":0.48599999999999999},"min":{"1min":0.46200000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77400000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.39900000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55200000000000005,"15min":0.52000000000000002},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80300000000000005,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.504},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81799999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59999999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.37,"5min":0.312,"15min":0.312},"max":{"1min":0.72599999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.497},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64100000000000001,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80200000000000005,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.61399999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67100000000000004,"15min":0.71199999999999997},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84199999999999997,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67200000000000004,"15min":0.69499999999999995},"min":{"1min":0.51200000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83199999999999996,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.67000000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73999999999999999,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60199999999999998,"15min":0.622},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84999999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.623,"15min":0.629},"min":{"1min":0.437,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69699999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.45300000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68000000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82399999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10956,"kb_used_data":1412,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360884,"statfs":{"total":96636764160,"available":96625545216,"internally_reserved":0,"allocated":1445888,"data_stored":991015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.33100000000000002},"min":{"1min":0.255,"5min":0.19,"15min":0.19},"max":{"1min":0.501,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55600000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.28199999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.34100000000000003,"15min":0.32300000000000001},"min":{"1min":0.26300000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.51400000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39000000000000001,"15min":0.38300000000000001},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.36599999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.39700000000000002},"min":{"1min":0.35799999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60499999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42799999999999999,"15min":0.41399999999999998},"min":{"1min":0.32900000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.31900000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56999999999999995,"15min":0.54000000000000004},"min":{"1min":0.44900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.46400000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.014},"last":0.70499999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.47699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.627},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.57099999999999995},"min":{"1min":0.48999999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.78400000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.621,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0389999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.52100000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88200000000000001,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.76300000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63,"15min":0.60899999999999999},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63200000000000001,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.67100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9964,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361876,"statfs":{"total":96636764160,"available":96626561024,"internally_reserved":0,"allocated":823296,"data_stored":388540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33600000000000002,"15min":0.32900000000000001},"min":{"1min":0.25,"5min":0.218,"15min":0.218},"max":{"1min":0.501,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.374},"min":{"1min":0.30299999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.53800000000000003,"5min":0.622,"15min":0.622},"last":0.38}]},{"osd":2,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38100000000000001,"15min":0.38300000000000001},"min":{"1min":0.28899999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39700000000000002,"15min":0.39900000000000002},"min":{"1min":0.28899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61699999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.28199999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.372},"min":{"1min":0.30499999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.35699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.59899999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.34799999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.46400000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.74399999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.53800000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66100000000000003,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71499999999999997,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81599999999999995,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.503},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.51900000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.79200000000000004,"5min":1.016,"15min":1.016},"last":0.55100000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46700000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78700000000000003,"5min":1.105,"15min":1.105},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65300000000000002,"15min":0.64400000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80800000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.67600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9788,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362052,"statfs":{"total":96636764160,"available":96626741248,"internally_reserved":0,"allocated":839680,"data_stored":399086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35799999999999998,"15min":0.371},"min":{"1min":0.247,"5min":0.214,"15min":0.214},"max":{"1min":0.50700000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.371,"5min":0.38100000000000001,"15min":0.38700000000000001},"min":{"1min":0.22800000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.54900000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.29899999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.29599999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.54100000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.42099999999999999,"15min":0.441},"min":{"1min":0.29399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57899999999999996,"5min":1.528,"15min":1.528},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38300000000000001,"15min":0.39400000000000002},"min":{"1min":0.27700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.51400000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.45600000000000002,"15min":0.47599999999999998},"min":{"1min":0.312,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55300000000000005,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.38700000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80700000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.442},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.46500000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.50700000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.63700000000000001},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93899999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.40200000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.62,"15min":0.622},"min":{"1min":0.48499999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.432},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.66800000000000004},"min":{"1min":0.441,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85399999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.628,"15min":0.63},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.80100000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9908,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361932,"statfs":{"total":96636764160,"available":96626618368,"internally_reserved":0,"allocated":831488,"data_stored":388652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.58799999999999997,"15min":0.83799999999999997},"min":{"1min":0.22500000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.48699999999999999,"5min":11.965,"15min":11.965},"last":0.246},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.61399999999999999,"15min":0.872},"min":{"1min":0.24199999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47699999999999998,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.35399999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35199999999999998,"15min":0.34200000000000003},"min":{"1min":0.219,"5min":0.215,"15min":0.215},"max":{"1min":0.56899999999999995,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.74099999999999999,"15min":1.097},"min":{"1min":0.25800000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58299999999999996,"5min":16.988,"15min":16.988},"last":0.39400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45800000000000002,"15min":0.49299999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57099999999999995,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.38},{"interface":"front","average":{"1min":0.435,"5min":1.0229999999999999,"15min":1.6040000000000001},"min":{"1min":0.32000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61599999999999999,"5min":27.887,"15min":27.887},"last":0.33900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65800000000000003,"15min":0.69899999999999995},"min":{"1min":0.40300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.93799999999999994,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64300000000000002,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85499999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.49399999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65700000000000003,"15min":0.68500000000000005},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.87,"5min":1.98,"15min":1.98},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.878,"15min":1.1279999999999999},"min":{"1min":0.441,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83199999999999996,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.45400000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.89000000000000001,"15min":1.1379999999999999},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83699999999999997,"5min":11.888,"15min":11.888},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.67300000000000004,"15min":0.72199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88400000000000001,"5min":1.925,"15min":1.925},"last":0.48499999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69699999999999995,"15min":0.73899999999999999},"min":{"1min":0.45800000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91500000000000004,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72199999999999998,"15min":0.77100000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86899999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.59099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10032,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361808,"statfs":{"total":96636764160,"available":96626491392,"internally_reserved":0,"allocated":827392,"data_stored":388610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70899999999999996,"5min":0.996,"15min":0.996},"last":0.625},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.72699999999999998,"15min":0.90600000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74399999999999999,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.59199999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.38,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81699999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.47799999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.56299999999999994}]},{"osd":3,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.65300000000000002,"15min":0.73299999999999998},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.73999999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86499999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.64800000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82399999999999995,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.78600000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57899999999999996,"15min":0.60099999999999998},"min":{"1min":0.39900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71999999999999997,"5min":1.145,"15min":1.145},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60699999999999998,"15min":0.627},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.91800000000000004,"5min":1.085,"15min":1.085},"last":0.66700000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.623},"min":{"1min":0.36799999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81599999999999995,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.439,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83499999999999996,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.71299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9788,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362052,"statfs":{"total":96636764160,"available":96626741248,"internally_reserved":0,"allocated":839680,"data_stored":395500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51100000000000001,"15min":0.505},"min":{"1min":0.39600000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.73999999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":0.80700000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57999999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64100000000000001,"15min":0.66500000000000004},"min":{"1min":0.42599999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95299999999999996,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.66500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92100000000000004,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.39500000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79500000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.443,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78500000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.42599999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61699999999999999,"15min":0.67300000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90100000000000002,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.879,"5min":1.042,"15min":1.042},"last":0.60799999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52300000000000002,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.69199999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.625},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59999999999999998,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.80400000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.66600000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.60699999999999998},"min":{"1min":0.38,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.622,"15min":0.64200000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85999999999999999,"5min":1.036,"15min":1.036},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64600000000000002,"15min":0.68400000000000005},"min":{"1min":0.39900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.84499999999999997,"5min":1.329,"15min":1.329},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64800000000000002,"15min":0.68300000000000005},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88200000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.67900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:13:23.717 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:13:23.842 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.840+0000 7f9cffe44700 1 Processor -- start 2023-11-28T11:13:23.842 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.840+0000 7f9cffe44700 1 -- start start 2023-11-28T11:13:23.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.841+0000 7f9cffe44700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cf8159590 0x7f9cf806e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:23.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.841+0000 7f9cffe44700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9cf815db70 0x7f9cf80ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:23.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.841+0000 7f9cffe44700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9cf80ce890 0x7f9cf80ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:23.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.841+0000 7f9cffe44700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf80cf250 con 0x7f9cf80ce890 2023-11-28T11:13:23.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.841+0000 7f9cffe44700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cf815d120 con 0x7f9cf815db70 2023-11-28T11:13:23.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.841+0000 7f9cffe44700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf815d6e0 con 0x7f9cf8159590 2023-11-28T11:13:23.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.841+0000 7f9cfe3e1700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9cf80ce890 0x7f9cf80ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:23.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.841+0000 7f9cfe3e1700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f9cf80ce890 0x7f9cf80ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52474/0 (socket says 172.21.15.28:52474) 2023-11-28T11:13:23.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.841+0000 7f9cfe3e1700 1 -- 172.21.15.28:0/4090657908 learned_addr learned my addr 172.21.15.28:0/4090657908 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:23.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.841+0000 7f9cfdbe0700 1 --2- 172.21.15.28:0/4090657908 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cf8159590 0x7f9cf806e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:23.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.841+0000 7f9cfd3df700 1 --2- 172.21.15.28:0/4090657908 >> v2:172.21.15.59:3300/0 conn(0x7f9cf815db70 0x7f9cf80ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:23.846 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.842+0000 7f9cfe3e1700 1 -- 172.21.15.28:0/4090657908 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cf8159590 msgr2=0x7f9cf806e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:23.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.842+0000 7f9cfe3e1700 1 --2- 172.21.15.28:0/4090657908 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cf8159590 0x7f9cf806e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.842+0000 7f9cfe3e1700 1 -- 172.21.15.28:0/4090657908 >> v2:172.21.15.59:3300/0 conn(0x7f9cf815db70 msgr2=0x7f9cf80ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:23.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.842+0000 7f9cfe3e1700 1 --2- 172.21.15.28:0/4090657908 >> v2:172.21.15.59:3300/0 conn(0x7f9cf815db70 0x7f9cf80ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.842+0000 7f9cfe3e1700 1 -- 172.21.15.28:0/4090657908 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf80aef60 con 0x7f9cf80ce890 2023-11-28T11:13:23.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.842+0000 7f9cfe3e1700 1 --2- 172.21.15.28:0/4090657908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9cf80ce890 0x7f9cf80ced10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9cf806f840 tx=0x7f9cf000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298d605f5b3340d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:23.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.842+0000 7f9cfcbde700 1 -- 172.21.15.28:0/4090657908 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000e070 con 0x7f9cf80ce890 2023-11-28T11:13:23.847 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.842+0000 7f9cfcbde700 1 -- 172.21.15.28:0/4090657908 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf000b910 con 0x7f9cf80ce890 2023-11-28T11:13:23.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.842+0000 7f9cfcbde700 1 -- 172.21.15.28:0/4090657908 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf000bcc0 con 0x7f9cf80ce890 2023-11-28T11:13:23.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.843+0000 7f9cffe44700 1 -- 172.21.15.28:0/4090657908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9cf80ce890 msgr2=0x7f9cf80ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:23.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.843+0000 7f9cffe44700 1 --2- 172.21.15.28:0/4090657908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9cf80ce890 0x7f9cf80ced10 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f9cf806f840 tx=0x7f9cf000b330 comp rx=0 tx=0).stop 2023-11-28T11:13:23.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.843+0000 7f9cffe44700 1 -- 172.21.15.28:0/4090657908 shutdown_connections 2023-11-28T11:13:23.848 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.843+0000 7f9cffe44700 1 --2- 172.21.15.28:0/4090657908 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9cf80ce890 0x7f9cf80ced10 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.843+0000 7f9cffe44700 1 --2- 172.21.15.28:0/4090657908 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cf8159590 0x7f9cf806e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.843+0000 7f9cffe44700 1 --2- 172.21.15.28:0/4090657908 >> v2:172.21.15.59:3300/0 conn(0x7f9cf815db70 0x7f9cf80ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.843+0000 7f9cffe44700 1 -- 172.21.15.28:0/4090657908 >> 172.21.15.28:0/4090657908 conn(0x7f9cf80b1650 msgr2=0x7f9cf80b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:23.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.843+0000 7f9cffe44700 1 -- 172.21.15.28:0/4090657908 shutdown_connections 2023-11-28T11:13:23.849 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.843+0000 7f9cffe44700 1 -- 172.21.15.28:0/4090657908 wait complete. 2023-11-28T11:13:23.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.844+0000 7f9cffe44700 1 Processor -- start 2023-11-28T11:13:23.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.844+0000 7f9cffe44700 1 -- start start 2023-11-28T11:13:23.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cffe44700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cf80ce890 0x7f9cf81559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:23.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cffe44700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9cf8159590 0x7f9cf81f0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:23.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cffe44700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cf815db70 0x7f9cf8147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:23.852 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cffe44700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf81f10d0 con 0x7f9cf8159590 2023-11-28T11:13:23.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cffe44700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf81f05b0 con 0x7f9cf815db70 2023-11-28T11:13:23.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cffe44700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cf81f0830 con 0x7f9cf80ce890 2023-11-28T11:13:23.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cfd3df700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9cf8159590 0x7f9cf81f0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:23.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cfd3df700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9cf8159590 0x7f9cf81f0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52478/0 (socket says 172.21.15.28:52478) 2023-11-28T11:13:23.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cfd3df700 1 -- 172.21.15.28:0/493751073 learned_addr learned my addr 172.21.15.28:0/493751073 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:23.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cfdbe0700 1 --2- 172.21.15.28:0/493751073 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cf80ce890 0x7f9cf81559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:23.853 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cfd3df700 1 -- 172.21.15.28:0/493751073 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cf80ce890 msgr2=0x7f9cf81559f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:23.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cfe3e1700 1 --2- 172.21.15.28:0/493751073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cf815db70 0x7f9cf8147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:23.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cfd3df700 1 --2- 172.21.15.28:0/493751073 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cf80ce890 0x7f9cf81559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cfd3df700 1 -- 172.21.15.28:0/493751073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cf815db70 msgr2=0x7f9cf8147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:23.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cfd3df700 1 --2- 172.21.15.28:0/493751073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cf815db70 0x7f9cf8147e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:23.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cfd3df700 1 -- 172.21.15.28:0/493751073 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cf0009b30 con 0x7f9cf8159590 2023-11-28T11:13:23.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cfd3df700 1 --2- 172.21.15.28:0/493751073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9cf8159590 0x7f9cf81f0070 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400e870 tx=0x7f9cf400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:23.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cee7fc700 1 -- 172.21.15.28:0/493751073 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf4009590 con 0x7f9cf8159590 2023-11-28T11:13:23.854 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cee7fc700 1 -- 172.21.15.28:0/493751073 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cf400ee70 con 0x7f9cf8159590 2023-11-28T11:13:23.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.845+0000 7f9cee7fc700 1 -- 172.21.15.28:0/493751073 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cf400ca50 con 0x7f9cf8159590 2023-11-28T11:13:23.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.846+0000 7f9cffe44700 1 -- 172.21.15.28:0/493751073 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cf815d370 con 0x7f9cf8159590 2023-11-28T11:13:23.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.846+0000 7f9cffe44700 1 -- 172.21.15.28:0/493751073 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cf80af240 con 0x7f9cf8159590 2023-11-28T11:13:23.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.846+0000 7f9cffe44700 1 -- 172.21.15.28:0/493751073 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cf80aec70 con 0x7f9cf8159590 2023-11-28T11:13:23.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.847+0000 7f9cee7fc700 1 -- 172.21.15.28:0/493751073 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9cf4019030 con 0x7f9cf8159590 2023-11-28T11:13:23.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.848+0000 7f9cee7fc700 1 --2- 172.21.15.28:0/493751073 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9cdc0682f0 0x7f9cdc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:23.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.848+0000 7f9cee7fc700 1 -- 172.21.15.28:0/493751073 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f9cf4086110 con 0x7f9cf8159590 2023-11-28T11:13:23.855 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.848+0000 7f9cfdbe0700 1 --2- 172.21.15.28:0/493751073 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9cdc0682f0 0x7f9cdc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:23.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.848+0000 7f9cfdbe0700 1 --2- 172.21.15.28:0/493751073 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9cdc0682f0 0x7f9cdc06a7a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009ab0 tx=0x7f9ce8008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:23.856 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:23.851+0000 7f9cee7fc700 1 -- 172.21.15.28:0/493751073 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9cf4052ae0 con 0x7f9cf8159590 2023-11-28T11:13:24.041 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.040+0000 7f9cffe44700 1 -- 172.21.15.28:0/493751073 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9cf806c1b0 con 0x7f9cdc0682f0 2023-11-28T11:13:24.044 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.042+0000 7f9cee7fc700 1 -- 172.21.15.28:0/493751073 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72816 (secure 0 0 0) 0x7f9cf806c1b0 con 0x7f9cdc0682f0 2023-11-28T11:13:24.045 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.044+0000 7f9cffe44700 1 -- 172.21.15.28:0/493751073 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9cdc0682f0 msgr2=0x7f9cdc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:24.046 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.044+0000 7f9cffe44700 1 --2- 172.21.15.28:0/493751073 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9cdc0682f0 0x7f9cdc06a7a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f9ce8009ab0 tx=0x7f9ce8008040 comp rx=0 tx=0).stop 2023-11-28T11:13:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.044+0000 7f9cffe44700 1 -- 172.21.15.28:0/493751073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9cf8159590 msgr2=0x7f9cf81f0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.044+0000 7f9cffe44700 1 --2- 172.21.15.28:0/493751073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9cf8159590 0x7f9cf81f0070 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f9cf400e870 tx=0x7f9cf400ec30 comp rx=0 tx=0).stop 2023-11-28T11:13:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.045+0000 7f9cffe44700 1 -- 172.21.15.28:0/493751073 shutdown_connections 2023-11-28T11:13:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.045+0000 7f9cffe44700 1 --2- 172.21.15.28:0/493751073 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9cdc0682f0 0x7f9cdc06a7a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.045+0000 7f9cffe44700 1 --2- 172.21.15.28:0/493751073 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9cf815db70 0x7f9cf8147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.045+0000 7f9cffe44700 1 --2- 172.21.15.28:0/493751073 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9cf80ce890 0x7f9cf81559f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.047 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.045+0000 7f9cffe44700 1 --2- 172.21.15.28:0/493751073 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9cf8159590 0x7f9cf81f0070 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.045+0000 7f9cffe44700 1 -- 172.21.15.28:0/493751073 >> 172.21.15.28:0/493751073 conn(0x7f9cf80b1650 msgr2=0x7f9cf80c8d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:24.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.045+0000 7f9cffe44700 1 -- 172.21.15.28:0/493751073 shutdown_connections 2023-11-28T11:13:24.048 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.045+0000 7f9cffe44700 1 -- 172.21.15.28:0/493751073 wait complete. 2023-11-28T11:13:24.048 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:13:24.058 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2023-11-28T11:13:23.227399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":81216,"kb_used_data":7680,"kb_used_omap":0,"kb_used_meta":73472,"kb_avail":754893504,"statfs":{"total":773094113280,"available":773010948096,"internally_reserved":0,"allocated":7864320,"data_stored":4319543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":75235328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12289,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-74,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001406"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:53.259553+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:12:53.259553+0000","last_peered":"2023-11-28T11:12:53.259553+0000","last_clean":"2023-11-28T11:12:53.259553+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:12:53.259553+0000","last_undegraded":"2023-11-28T11:12:53.259553+0000","last_fullsized":"2023-11-28T11:12:53.259553+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:35.286107+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:12:35.286107+0000","last_peered":"2023-11-28T11:12:35.286107+0000","last_clean":"2023-11-28T11:12:35.286107+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:12:35.286107+0000","last_undegraded":"2023-11-28T11:12:35.286107+0000","last_fullsized":"2023-11-28T11:12:35.286107+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:35.779428+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:12:35.779428+0000","last_peered":"2023-11-28T11:12:35.779428+0000","last_clean":"2023-11-28T11:12:35.779428+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:12:35.779428+0000","last_undegraded":"2023-11-28T11:12:35.779428+0000","last_fullsized":"2023-11-28T11:12:35.779428+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":206,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:43.258422+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:12:43.258422+0000","last_peered":"2023-11-28T11:12:43.258422+0000","last_clean":"2023-11-28T11:12:43.258422+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:12:43.258422+0000","last_undegraded":"2023-11-28T11:12:43.258422+0000","last_fullsized":"2023-11-28T11:12:43.258422+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:53.259590+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:12:53.259590+0000","last_peered":"2023-11-28T11:12:53.259590+0000","last_clean":"2023-11-28T11:12:53.259590+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:12:53.259590+0000","last_undegraded":"2023-11-28T11:12:53.259590+0000","last_fullsized":"2023-11-28T11:12:53.259590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:43.258405+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:12:43.258405+0000","last_peered":"2023-11-28T11:12:43.258405+0000","last_clean":"2023-11-28T11:12:43.258405+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:12:43.258405+0000","last_undegraded":"2023-11-28T11:12:43.258405+0000","last_fullsized":"2023-11-28T11:12:43.258405+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:53.255623+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:12:53.255623+0000","last_peered":"2023-11-28T11:12:53.255623+0000","last_clean":"2023-11-28T11:12:53.255623+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:12:53.255623+0000","last_undegraded":"2023-11-28T11:12:53.255623+0000","last_fullsized":"2023-11-28T11:12:53.255623+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":55,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:35.291076+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:12:35.291076+0000","last_peered":"2023-11-28T11:12:35.291076+0000","last_clean":"2023-11-28T11:12:35.291076+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:12:35.291076+0000","last_undegraded":"2023-11-28T11:12:35.291076+0000","last_fullsized":"2023-11-28T11:12:35.291076+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":55,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:35.291623+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:12:35.291623+0000","last_peered":"2023-11-28T11:12:35.291623+0000","last_clean":"2023-11-28T11:12:35.291623+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:12:35.291623+0000","last_undegraded":"2023-11-28T11:12:35.291623+0000","last_fullsized":"2023-11-28T11:12:35.291623+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11000,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360840,"statfs":{"total":96636764160,"available":96625500160,"internally_reserved":0,"allocated":1425408,"data_stored":979508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57099999999999995,"15min":0.63100000000000001},"min":{"1min":0.38700000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72599999999999998,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59399999999999997,"15min":0.63300000000000001},"min":{"1min":0.309,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.77000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.61299999999999999,"15min":0.67100000000000004},"min":{"1min":0.38,"5min":0.316,"15min":0.316},"max":{"1min":0.83399999999999996,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58299999999999996,"15min":0.59299999999999997},"min":{"1min":0.41199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73699999999999999,"5min":1.224,"15min":1.224},"last":0.72999999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.63100000000000001,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84999999999999998,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.69499999999999995,"15min":0.76300000000000001},"min":{"1min":0.44900000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88200000000000001,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.60399999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64400000000000002,"15min":0.68999999999999995},"min":{"1min":0.45800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76500000000000001,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.69699999999999995,"15min":0.75700000000000001},"min":{"1min":0.48499999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.78600000000000003,"5min":3.294,"15min":3.294},"last":0.74299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.373,"5min":0.312,"15min":0.312},"max":{"1min":0.77600000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.56200000000000006},"min":{"1min":0.34399999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.79500000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51200000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.63800000000000001},"min":{"1min":0.41299999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81100000000000005,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.63300000000000001,"15min":0.64500000000000002},"min":{"1min":0.434,"5min":0.373,"15min":0.373},"max":{"1min":0.86599999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.54800000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85599999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9780,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362060,"statfs":{"total":96636764160,"available":96626749440,"internally_reserved":0,"allocated":831488,"data_stored":388632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.51100000000000001,"15min":0.47099999999999997},"min":{"1min":0.376,"5min":0.252,"15min":0.252},"max":{"1min":0.79300000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.51200000000000001,"15min":0.48599999999999999},"min":{"1min":0.46200000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77400000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.39900000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55200000000000005,"15min":0.52000000000000002},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80300000000000005,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.504},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81799999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59999999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.37,"5min":0.312,"15min":0.312},"max":{"1min":0.72599999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.497},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64100000000000001,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80200000000000005,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.61399999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67100000000000004,"15min":0.71199999999999997},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84199999999999997,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67200000000000004,"15min":0.69499999999999995},"min":{"1min":0.51200000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83199999999999996,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.67000000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73999999999999999,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60199999999999998,"15min":0.622},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84999999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.623,"15min":0.629},"min":{"1min":0.437,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69699999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.45300000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68000000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82399999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10956,"kb_used_data":1412,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360884,"statfs":{"total":96636764160,"available":96625545216,"internally_reserved":0,"allocated":1445888,"data_stored":991015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.33100000000000002},"min":{"1min":0.255,"5min":0.19,"15min":0.19},"max":{"1min":0.501,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55600000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.28199999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.34100000000000003,"15min":0.32300000000000001},"min":{"1min":0.26300000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.51400000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39000000000000001,"15min":0.38300000000000001},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.36599999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.39700000000000002},"min":{"1min":0.35799999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60499999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42799999999999999,"15min":0.41399999999999998},"min":{"1min":0.32900000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.31900000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56999999999999995,"15min":0.54000000000000004},"min":{"1min":0.44900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.46400000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.014},"last":0.70499999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.47699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.627},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.57099999999999995},"min":{"1min":0.48999999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.78400000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.621,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0389999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.52100000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88200000000000001,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.76300000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63,"15min":0.60899999999999999},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63200000000000001,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.67100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9964,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361876,"statfs":{"total":96636764160,"available":96626561024,"internally_reserved":0,"allocated":823296,"data_stored":388540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33600000000000002,"15min":0.32900000000000001},"min":{"1min":0.25,"5min":0.218,"15min":0.218},"max":{"1min":0.501,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.374},"min":{"1min":0.30299999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.53800000000000003,"5min":0.622,"15min":0.622},"last":0.38}]},{"osd":2,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38100000000000001,"15min":0.38300000000000001},"min":{"1min":0.28899999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39700000000000002,"15min":0.39900000000000002},"min":{"1min":0.28899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61699999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.28199999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.372},"min":{"1min":0.30499999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.35699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.59899999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.34799999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.46400000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.74399999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.53800000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66100000000000003,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71499999999999997,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81599999999999995,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.503},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.51900000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.79200000000000004,"5min":1.016,"15min":1.016},"last":0.55100000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46700000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78700000000000003,"5min":1.105,"15min":1.105},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65300000000000002,"15min":0.64400000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80800000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.67600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9788,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362052,"statfs":{"total":96636764160,"available":96626741248,"internally_reserved":0,"allocated":839680,"data_stored":399086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35799999999999998,"15min":0.371},"min":{"1min":0.247,"5min":0.214,"15min":0.214},"max":{"1min":0.50700000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.371,"5min":0.38100000000000001,"15min":0.38700000000000001},"min":{"1min":0.22800000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.54900000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.29899999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.29599999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.54100000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.42099999999999999,"15min":0.441},"min":{"1min":0.29399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57899999999999996,"5min":1.528,"15min":1.528},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38300000000000001,"15min":0.39400000000000002},"min":{"1min":0.27700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.51400000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.45600000000000002,"15min":0.47599999999999998},"min":{"1min":0.312,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55300000000000005,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.38700000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80700000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.442},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.46500000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.50700000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.63700000000000001},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93899999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.40200000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.62,"15min":0.622},"min":{"1min":0.48499999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.432},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.66800000000000004},"min":{"1min":0.441,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85399999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.628,"15min":0.63},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.80100000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9908,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361932,"statfs":{"total":96636764160,"available":96626618368,"internally_reserved":0,"allocated":831488,"data_stored":388652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.58799999999999997,"15min":0.83799999999999997},"min":{"1min":0.22500000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.48699999999999999,"5min":11.965,"15min":11.965},"last":0.246},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.61399999999999999,"15min":0.872},"min":{"1min":0.24199999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47699999999999998,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.35399999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35199999999999998,"15min":0.34200000000000003},"min":{"1min":0.219,"5min":0.215,"15min":0.215},"max":{"1min":0.56899999999999995,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.74099999999999999,"15min":1.097},"min":{"1min":0.25800000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58299999999999996,"5min":16.988,"15min":16.988},"last":0.39400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45800000000000002,"15min":0.49299999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57099999999999995,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.38},{"interface":"front","average":{"1min":0.435,"5min":1.0229999999999999,"15min":1.6040000000000001},"min":{"1min":0.32000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61599999999999999,"5min":27.887,"15min":27.887},"last":0.33900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65800000000000003,"15min":0.69899999999999995},"min":{"1min":0.40300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.93799999999999994,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64300000000000002,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85499999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.49399999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65700000000000003,"15min":0.68500000000000005},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.87,"5min":1.98,"15min":1.98},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.878,"15min":1.1279999999999999},"min":{"1min":0.441,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83199999999999996,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.45400000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.89000000000000001,"15min":1.1379999999999999},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83699999999999997,"5min":11.888,"15min":11.888},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.67300000000000004,"15min":0.72199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88400000000000001,"5min":1.925,"15min":1.925},"last":0.48499999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69699999999999995,"15min":0.73899999999999999},"min":{"1min":0.45800000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91500000000000004,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72199999999999998,"15min":0.77100000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86899999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.59099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10032,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361808,"statfs":{"total":96636764160,"available":96626491392,"internally_reserved":0,"allocated":827392,"data_stored":388610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70899999999999996,"5min":0.996,"15min":0.996},"last":0.625},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.72699999999999998,"15min":0.90600000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74399999999999999,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.59199999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.38,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81699999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.47799999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.56299999999999994}]},{"osd":3,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.65300000000000002,"15min":0.73299999999999998},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.73999999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86499999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.64800000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82399999999999995,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.78600000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57899999999999996,"15min":0.60099999999999998},"min":{"1min":0.39900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71999999999999997,"5min":1.145,"15min":1.145},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60699999999999998,"15min":0.627},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.91800000000000004,"5min":1.085,"15min":1.085},"last":0.66700000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.623},"min":{"1min":0.36799999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81599999999999995,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.439,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83499999999999996,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.71299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9788,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362052,"statfs":{"total":96636764160,"available":96626741248,"internally_reserved":0,"allocated":839680,"data_stored":395500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51100000000000001,"15min":0.505},"min":{"1min":0.39600000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.73999999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":0.80700000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57999999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64100000000000001,"15min":0.66500000000000004},"min":{"1min":0.42599999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95299999999999996,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.66500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92100000000000004,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.39500000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79500000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.443,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78500000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.42599999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61699999999999999,"15min":0.67300000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90100000000000002,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.879,"5min":1.042,"15min":1.042},"last":0.60799999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52300000000000002,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.69199999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.625},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59999999999999998,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.80400000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.66600000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.60699999999999998},"min":{"1min":0.38,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.622,"15min":0.64200000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85999999999999999,"5min":1.036,"15min":1.036},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64600000000000002,"15min":0.68400000000000005},"min":{"1min":0.39900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.84499999999999997,"5min":1.329,"15min":1.329},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64800000000000002,"15min":0.68300000000000005},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88200000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.67900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:13:24.061 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T11:13:24.062 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:13:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.185+0000 7f8590859700 1 Processor -- start 2023-11-28T11:13:24.187 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.185+0000 7f8590859700 1 -- start start 2023-11-28T11:13:24.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f8590859700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f858c06d9d0 0x7f858c15b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:24.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f8590859700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f858c06bc30 0x7f858c15e950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:24.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f8590859700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f858c06c3a0 con 0x7f858c06dda0 2023-11-28T11:13:24.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f8590859700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858c15ee90 con 0x7f858c06bc30 2023-11-28T11:13:24.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f8590859700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f858c06c0c0 con 0x7f858c06d9d0 2023-11-28T11:13:24.190 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f8589d9b700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f858c06bc30 0x7f858c15e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:24.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f858ad9d700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f858c06dda0 0x7f858c150eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40546/0 (socket says 172.21.15.28:40546) 2023-11-28T11:13:24.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f858ad9d700 1 -- 172.21.15.28:0/210627267 learned_addr learned my addr 172.21.15.28:0/210627267 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:24.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f858a59c700 1 --2- 172.21.15.28:0/210627267 >> v2:172.21.15.59:3300/0 conn(0x7f858c06d9d0 0x7f858c15b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:24.191 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f8589d9b700 1 -- 172.21.15.28:0/210627267 >> v2:172.21.15.59:3300/0 conn(0x7f858c06d9d0 msgr2=0x7f858c15b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:24.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f8589d9b700 1 --2- 172.21.15.28:0/210627267 >> v2:172.21.15.59:3300/0 conn(0x7f858c06d9d0 0x7f858c15b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.192 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f8589d9b700 1 -- 172.21.15.28:0/210627267 >> v1:172.21.15.28:6789/0 conn(0x7f858c06dda0 legacy=0x7f858c150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:24.193 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f8589d9b700 1 -- 172.21.15.28:0/210627267 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c15ab60 con 0x7f858c06bc30 2023-11-28T11:13:24.193 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f858959a700 1 -- 172.21.15.28:0/210627267 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238434013 0 0) 0x7f858c06c3a0 con 0x7f858c06dda0 2023-11-28T11:13:24.193 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f8589d9b700 1 --2- 172.21.15.28:0/210627267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f858c06bc30 0x7f858c15e950 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f858c0cf1a0 tx=0x7f8580009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2014cb666a18a1b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:24.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.186+0000 7f858959a700 1 -- 172.21.15.28:0/210627267 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8580011830 con 0x7f858c06bc30 2023-11-28T11:13:24.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.187+0000 7f858959a700 1 -- 172.21.15.28:0/210627267 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8580011990 con 0x7f858c06bc30 2023-11-28T11:13:24.194 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.187+0000 7f858959a700 1 -- 172.21.15.28:0/210627267 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8580011d20 con 0x7f858c06bc30 2023-11-28T11:13:24.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.187+0000 7f8590859700 1 -- 172.21.15.28:0/210627267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f858c06bc30 msgr2=0x7f858c15e950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:24.195 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.187+0000 7f8590859700 1 --2- 172.21.15.28:0/210627267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f858c06bc30 0x7f858c15e950 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f858c0cf1a0 tx=0x7f8580009b20 comp rx=0 tx=0).stop 2023-11-28T11:13:24.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.187+0000 7f8590859700 1 -- 172.21.15.28:0/210627267 shutdown_connections 2023-11-28T11:13:24.196 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.187+0000 7f8590859700 1 --2- 172.21.15.28:0/210627267 >> v2:172.21.15.59:3300/0 conn(0x7f858c06d9d0 0x7f858c15b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.187+0000 7f8590859700 1 --2- 172.21.15.28:0/210627267 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f858c06bc30 0x7f858c15e950 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.187+0000 7f8590859700 1 -- 172.21.15.28:0/210627267 >> 172.21.15.28:0/210627267 conn(0x7f858c0b0dd0 msgr2=0x7f858c0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:24.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.187+0000 7f8590859700 1 -- 172.21.15.28:0/210627267 shutdown_connections 2023-11-28T11:13:24.197 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.187+0000 7f8590859700 1 -- 172.21.15.28:0/210627267 wait complete. 2023-11-28T11:13:24.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.188+0000 7f8590859700 1 Processor -- start 2023-11-28T11:13:24.198 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.188+0000 7f8590859700 1 -- start start 2023-11-28T11:13:24.203 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.189+0000 7f8590859700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f858c06bc30 0x7f858c1e3f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.189+0000 7f8590859700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f858c06d9d0 0x7f858c1f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.189+0000 7f8590859700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f858c06dda0 0x7f858c14de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.189+0000 7f8590859700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c0c3e60 con 0x7f858c06dda0 2023-11-28T11:13:24.204 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.189+0000 7f8590859700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c0c4140 con 0x7f858c06d9d0 2023-11-28T11:13:24.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.189+0000 7f8590859700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858c1f5ba0 con 0x7f858c06bc30 2023-11-28T11:13:24.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.189+0000 7f858ad9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f858c06dda0 0x7f858c14de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:24.205 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.189+0000 7f858ad9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f858c06dda0 0x7f858c14de80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52494/0 (socket says 172.21.15.28:52494) 2023-11-28T11:13:24.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.189+0000 7f858ad9d700 1 -- 172.21.15.28:0/357701372 learned_addr learned my addr 172.21.15.28:0/357701372 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:24.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.190+0000 7f8589d9b700 1 --2- 172.21.15.28:0/357701372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f858c06d9d0 0x7f858c1f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:24.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.190+0000 7f858a59c700 1 --2- 172.21.15.28:0/357701372 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f858c06bc30 0x7f858c1e3f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:24.206 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.190+0000 7f858ad9d700 1 -- 172.21.15.28:0/357701372 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f858c06bc30 msgr2=0x7f858c1e3f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:24.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.190+0000 7f858ad9d700 1 --2- 172.21.15.28:0/357701372 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f858c06bc30 0x7f858c1e3f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.190+0000 7f858ad9d700 1 -- 172.21.15.28:0/357701372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f858c06d9d0 msgr2=0x7f858c1f5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:24.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.190+0000 7f858ad9d700 1 --2- 172.21.15.28:0/357701372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f858c06d9d0 0x7f858c1f5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.207 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.190+0000 7f858ad9d700 1 -- 172.21.15.28:0/357701372 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85800094a0 con 0x7f858c06dda0 2023-11-28T11:13:24.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.190+0000 7f8589d9b700 1 --2- 172.21.15.28:0/357701372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f858c06d9d0 0x7f858c1f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:13:24.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.190+0000 7f858ad9d700 1 --2- 172.21.15.28:0/357701372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f858c06dda0 0x7f858c14de80 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f858400cb60 tx=0x7f858400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:24.208 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.190+0000 7f857affd700 1 -- 172.21.15.28:0/357701372 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f858400ab50 con 0x7f858c06dda0 2023-11-28T11:13:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.190+0000 7f857affd700 1 -- 172.21.15.28:0/357701372 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f858400acb0 con 0x7f858c06dda0 2023-11-28T11:13:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.190+0000 7f8590859700 1 -- 172.21.15.28:0/357701372 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f858c15b120 con 0x7f858c06dda0 2023-11-28T11:13:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.190+0000 7f857affd700 1 -- 172.21.15.28:0/357701372 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8584007310 con 0x7f858c06dda0 2023-11-28T11:13:24.209 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.190+0000 7f8590859700 1 -- 172.21.15.28:0/357701372 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f858c15ae40 con 0x7f858c06dda0 2023-11-28T11:13:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.191+0000 7f8590859700 1 -- 172.21.15.28:0/357701372 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f858c06fac0 con 0x7f858c06dda0 2023-11-28T11:13:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.191+0000 7f857affd700 1 -- 172.21.15.28:0/357701372 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f858401a030 con 0x7f858c06dda0 2023-11-28T11:13:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.192+0000 7f857affd700 1 --2- 172.21.15.28:0/357701372 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f856c0682f0 0x7f856c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:24.210 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.192+0000 7f857affd700 1 -- 172.21.15.28:0/357701372 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f85840857e0 con 0x7f858c06dda0 2023-11-28T11:13:24.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.192+0000 7f858a59c700 1 --2- 172.21.15.28:0/357701372 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f856c0682f0 0x7f856c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:24.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.192+0000 7f858a59c700 1 --2- 172.21.15.28:0/357701372 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f856c0682f0 0x7f856c06a7a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f857c008850 tx=0x7f857c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:24.211 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.196+0000 7f857affd700 1 -- 172.21.15.28:0/357701372 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f85840521a0 con 0x7f858c06dda0 2023-11-28T11:13:24.386 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.384+0000 7f8590859700 1 -- 172.21.15.28:0/357701372 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f858c0675a0 con 0x7f856c0682f0 2023-11-28T11:13:24.389 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.387+0000 7f857affd700 1 -- 172.21.15.28:0/357701372 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72816 (secure 0 0 0) 0x7f858c0675a0 con 0x7f856c0682f0 2023-11-28T11:13:24.390 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:24.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.389+0000 7f8590859700 1 -- 172.21.15.28:0/357701372 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f856c0682f0 msgr2=0x7f856c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:24.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.389+0000 7f8590859700 1 --2- 172.21.15.28:0/357701372 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f856c0682f0 0x7f856c06a7a0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f857c008850 tx=0x7f857c008040 comp rx=0 tx=0).stop 2023-11-28T11:13:24.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.389+0000 7f8590859700 1 -- 172.21.15.28:0/357701372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f858c06dda0 msgr2=0x7f858c14de80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:24.392 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.389+0000 7f8590859700 1 --2- 172.21.15.28:0/357701372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f858c06dda0 0x7f858c14de80 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f858400cb60 tx=0x7f858400cf20 comp rx=0 tx=0).stop 2023-11-28T11:13:24.394 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.389+0000 7f8590859700 1 -- 172.21.15.28:0/357701372 shutdown_connections 2023-11-28T11:13:24.396 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.389+0000 7f8590859700 1 --2- 172.21.15.28:0/357701372 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f856c0682f0 0x7f856c06a7a0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.389+0000 7f8590859700 1 --2- 172.21.15.28:0/357701372 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f858c06dda0 0x7f858c14de80 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.389+0000 7f8590859700 1 --2- 172.21.15.28:0/357701372 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f858c06bc30 0x7f858c1e3f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.399 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.389+0000 7f8590859700 1 --2- 172.21.15.28:0/357701372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f858c06d9d0 0x7f858c1f5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.389+0000 7f8590859700 1 -- 172.21.15.28:0/357701372 >> 172.21.15.28:0/357701372 conn(0x7f858c0b0dd0 msgr2=0x7f858c15c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:24.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.389+0000 7f8590859700 1 -- 172.21.15.28:0/357701372 shutdown_connections 2023-11-28T11:13:24.400 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.389+0000 7f8590859700 1 -- 172.21.15.28:0/357701372 wait complete. 2023-11-28T11:13:24.400 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:13:24.403 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":207,"stamp":"2023-11-28T11:13:23.227399+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":81216,"kb_used_data":7680,"kb_used_omap":0,"kb_used_meta":73472,"kb_avail":754893504,"statfs":{"total":773094113280,"available":773010948096,"internally_reserved":0,"allocated":7864320,"data_stored":4319543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":75235328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-12289,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-6,"num_read_kb":-5,"num_write":-74,"num_write_kb":-37,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-15,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001406"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:53.259553+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:12:53.259553+0000","last_peered":"2023-11-28T11:12:53.259553+0000","last_clean":"2023-11-28T11:12:53.259553+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:12:53.259553+0000","last_undegraded":"2023-11-28T11:12:53.259553+0000","last_fullsized":"2023-11-28T11:12:53.259553+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:35.286107+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:12:35.286107+0000","last_peered":"2023-11-28T11:12:35.286107+0000","last_clean":"2023-11-28T11:12:35.286107+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:12:35.286107+0000","last_undegraded":"2023-11-28T11:12:35.286107+0000","last_fullsized":"2023-11-28T11:12:35.286107+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:35.779428+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:12:35.779428+0000","last_peered":"2023-11-28T11:12:35.779428+0000","last_clean":"2023-11-28T11:12:35.779428+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:12:35.779428+0000","last_undegraded":"2023-11-28T11:12:35.779428+0000","last_fullsized":"2023-11-28T11:12:35.779428+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:30:17.742214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":206,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:43.258422+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:12:43.258422+0000","last_peered":"2023-11-28T11:12:43.258422+0000","last_clean":"2023-11-28T11:12:43.258422+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:12:43.258422+0000","last_undegraded":"2023-11-28T11:12:43.258422+0000","last_fullsized":"2023-11-28T11:12:43.258422+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:53.259590+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:12:53.259590+0000","last_peered":"2023-11-28T11:12:53.259590+0000","last_clean":"2023-11-28T11:12:53.259590+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:12:53.259590+0000","last_undegraded":"2023-11-28T11:12:53.259590+0000","last_fullsized":"2023-11-28T11:12:53.259590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":56,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:43.258405+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:12:43.258405+0000","last_peered":"2023-11-28T11:12:43.258405+0000","last_clean":"2023-11-28T11:12:43.258405+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:12:43.258405+0000","last_undegraded":"2023-11-28T11:12:43.258405+0000","last_fullsized":"2023-11-28T11:12:43.258405+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":61,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:53.255623+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:12:53.255623+0000","last_peered":"2023-11-28T11:12:53.255623+0000","last_clean":"2023-11-28T11:12:53.255623+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:12:53.255623+0000","last_undegraded":"2023-11-28T11:12:53.255623+0000","last_fullsized":"2023-11-28T11:12:53.255623+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":55,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:35.291076+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:12:35.291076+0000","last_peered":"2023-11-28T11:12:35.291076+0000","last_clean":"2023-11-28T11:12:35.291076+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:12:35.291076+0000","last_undegraded":"2023-11-28T11:12:35.291076+0000","last_fullsized":"2023-11-28T11:12:35.291076+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":55,"reported_epoch":68,"state":"active+clean","last_fresh":"2023-11-28T11:12:35.291623+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:12:35.291623+0000","last_peered":"2023-11-28T11:12:35.291623+0000","last_clean":"2023-11-28T11:12:35.291623+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:12:35.291623+0000","last_undegraded":"2023-11-28T11:12:35.291623+0000","last_fullsized":"2023-11-28T11:12:35.291623+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574908,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11000,"kb_used_data":1392,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360840,"statfs":{"total":96636764160,"available":96625500160,"internally_reserved":0,"allocated":1425408,"data_stored":979508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57099999999999995,"15min":0.63100000000000001},"min":{"1min":0.38700000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72599999999999998,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59399999999999997,"15min":0.63300000000000001},"min":{"1min":0.309,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.77000000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.61299999999999999,"15min":0.67100000000000004},"min":{"1min":0.38,"5min":0.316,"15min":0.316},"max":{"1min":0.83399999999999996,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58299999999999996,"15min":0.59299999999999997},"min":{"1min":0.41199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73699999999999999,"5min":1.224,"15min":1.224},"last":0.72999999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.63100000000000001,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84999999999999998,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.69499999999999995,"15min":0.76300000000000001},"min":{"1min":0.44900000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88200000000000001,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.60399999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64400000000000002,"15min":0.68999999999999995},"min":{"1min":0.45800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76500000000000001,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.69699999999999995,"15min":0.75700000000000001},"min":{"1min":0.48499999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.78600000000000003,"5min":3.294,"15min":3.294},"last":0.74299999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.373,"5min":0.312,"15min":0.312},"max":{"1min":0.77600000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.56200000000000006},"min":{"1min":0.34399999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.79500000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51200000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.63800000000000001},"min":{"1min":0.41299999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81100000000000005,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.628,"5min":0.63300000000000001,"15min":0.64500000000000002},"min":{"1min":0.434,"5min":0.373,"15min":0.373},"max":{"1min":0.86599999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.54800000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85599999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574907,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9780,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362060,"statfs":{"total":96636764160,"available":96626749440,"internally_reserved":0,"allocated":831488,"data_stored":388632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.51100000000000001,"15min":0.47099999999999997},"min":{"1min":0.376,"5min":0.252,"15min":0.252},"max":{"1min":0.79300000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.51200000000000001,"15min":0.48599999999999999},"min":{"1min":0.46200000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77400000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.39900000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55200000000000005,"15min":0.52000000000000002},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80300000000000005,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.504},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81799999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59999999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.37,"5min":0.312,"15min":0.312},"max":{"1min":0.72599999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.497},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64100000000000001,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80200000000000005,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.61399999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67100000000000004,"15min":0.71199999999999997},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84199999999999997,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67200000000000004,"15min":0.69499999999999995},"min":{"1min":0.51200000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83199999999999996,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.67000000000000004}]},{"osd":4,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73999999999999999,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60199999999999998,"15min":0.622},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84999999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.623,"15min":0.629},"min":{"1min":0.437,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.69699999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.45300000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68000000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82399999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.71399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10956,"kb_used_data":1412,"kb_used_omap":0,"kb_used_meta":9536,"kb_avail":94360884,"statfs":{"total":96636764160,"available":96625545216,"internally_reserved":0,"allocated":1445888,"data_stored":991015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9764864},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.33100000000000002},"min":{"1min":0.255,"5min":0.19,"15min":0.19},"max":{"1min":0.501,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55600000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.28199999999999997}]},{"osd":2,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.34100000000000003,"15min":0.32300000000000001},"min":{"1min":0.26300000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.51400000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39000000000000001,"15min":0.38300000000000001},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.36599999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.39700000000000002},"min":{"1min":0.35799999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60499999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42799999999999999,"15min":0.41399999999999998},"min":{"1min":0.32900000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.31900000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56999999999999995,"15min":0.54000000000000004},"min":{"1min":0.44900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.46400000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.014},"last":0.70499999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.47699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.627},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.57099999999999995},"min":{"1min":0.48999999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.78400000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.621,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0389999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.52100000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88200000000000001,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.76300000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63,"15min":0.60899999999999999},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63200000000000001,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.67100000000000004}]}]},{"osd":0,"up_from":13,"seq":55834574908,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9964,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":9152,"kb_avail":94361876,"statfs":{"total":96636764160,"available":96626561024,"internally_reserved":0,"allocated":823296,"data_stored":388540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9371648},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33600000000000002,"15min":0.32900000000000001},"min":{"1min":0.25,"5min":0.218,"15min":0.218},"max":{"1min":0.501,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.374},"min":{"1min":0.30299999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.53800000000000003,"5min":0.622,"15min":0.622},"last":0.38}]},{"osd":2,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38100000000000001,"15min":0.38300000000000001},"min":{"1min":0.28899999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39700000000000002,"15min":0.39900000000000002},"min":{"1min":0.28899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61699999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.28199999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.372},"min":{"1min":0.30499999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.35699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.59899999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.34799999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.46400000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.74399999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.53800000000000003}]},{"osd":5,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66100000000000003,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71499999999999997,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81599999999999995,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.56499999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.503},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.51900000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.79200000000000004,"5min":1.016,"15min":1.016},"last":0.55100000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46700000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78700000000000003,"5min":1.105,"15min":1.105},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65300000000000002,"15min":0.64400000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80800000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.67600000000000005}]}]},{"osd":2,"up_from":13,"seq":55834574908,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9788,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362052,"statfs":{"total":96636764160,"available":96626741248,"internally_reserved":0,"allocated":839680,"data_stored":399086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35799999999999998,"15min":0.371},"min":{"1min":0.247,"5min":0.214,"15min":0.214},"max":{"1min":0.50700000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.28000000000000003},{"interface":"front","average":{"1min":0.371,"5min":0.38100000000000001,"15min":0.38700000000000001},"min":{"1min":0.22800000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.54900000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.29899999999999999}]},{"osd":1,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.29599999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.54100000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.42099999999999999,"15min":0.441},"min":{"1min":0.29399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57899999999999996,"5min":1.528,"15min":1.528},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38300000000000001,"15min":0.39400000000000002},"min":{"1min":0.27700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.51400000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.432,"5min":0.45600000000000002,"15min":0.47599999999999998},"min":{"1min":0.312,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55300000000000005,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.38700000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80700000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.442},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.46500000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.50700000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.63700000000000001},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93899999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.40200000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.62,"15min":0.622},"min":{"1min":0.48499999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.432},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.66800000000000004},"min":{"1min":0.441,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85399999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.53900000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.628,"15min":0.63},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.80100000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.48999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574907,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9908,"kb_used_data":812,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361932,"statfs":{"total":96636764160,"available":96626618368,"internally_reserved":0,"allocated":831488,"data_stored":388652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.58799999999999997,"15min":0.83799999999999997},"min":{"1min":0.22500000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.48699999999999999,"5min":11.965,"15min":11.965},"last":0.246},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.61399999999999999,"15min":0.872},"min":{"1min":0.24199999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47699999999999998,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.35399999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35199999999999998,"15min":0.34200000000000003},"min":{"1min":0.219,"5min":0.215,"15min":0.215},"max":{"1min":0.56899999999999995,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.25700000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.74099999999999999,"15min":1.097},"min":{"1min":0.25800000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58299999999999996,"5min":16.988,"15min":16.988},"last":0.39400000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45800000000000002,"15min":0.49299999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57099999999999995,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.38},{"interface":"front","average":{"1min":0.435,"5min":1.0229999999999999,"15min":1.6040000000000001},"min":{"1min":0.32000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61599999999999999,"5min":27.887,"15min":27.887},"last":0.33900000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65800000000000003,"15min":0.69899999999999995},"min":{"1min":0.40300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.93799999999999994,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64300000000000002,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85499999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.49399999999999999}]},{"osd":5,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65700000000000003,"15min":0.68500000000000005},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.87,"5min":1.98,"15min":1.98},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.878,"15min":1.1279999999999999},"min":{"1min":0.441,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83199999999999996,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.45400000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.89000000000000001,"15min":1.1379999999999999},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83699999999999997,"5min":11.888,"15min":11.888},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63,"5min":0.67300000000000004,"15min":0.72199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88400000000000001,"5min":1.925,"15min":1.925},"last":0.48499999999999999}]},{"osd":7,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69699999999999995,"15min":0.73899999999999999},"min":{"1min":0.45800000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91500000000000004,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72199999999999998,"15min":0.77100000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86899999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.59099999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574908,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10032,"kb_used_data":808,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361808,"statfs":{"total":96636764160,"available":96626491392,"internally_reserved":0,"allocated":827392,"data_stored":388610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70899999999999996,"5min":0.996,"15min":0.996},"last":0.625},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.72699999999999998,"15min":0.90600000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74399999999999999,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.59199999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.38,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81699999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.47799999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.56299999999999994}]},{"osd":3,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.65300000000000002,"15min":0.73299999999999998},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.73999999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86499999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.64800000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82399999999999995,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.78600000000000003}]},{"osd":6,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57899999999999996,"15min":0.60099999999999998},"min":{"1min":0.39900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71999999999999997,"5min":1.145,"15min":1.145},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60699999999999998,"15min":0.627},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.91800000000000004,"5min":1.085,"15min":1.085},"last":0.66700000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.623},"min":{"1min":0.36799999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81599999999999995,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.439,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83499999999999996,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.71299999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574908,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9788,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":8960,"kb_avail":94362052,"statfs":{"total":96636764160,"available":96626741248,"internally_reserved":0,"allocated":839680,"data_stored":395500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9175040},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51100000000000001,"15min":0.505},"min":{"1min":0.39600000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.73999999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":0.80700000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.57999999999999996}]},{"osd":1,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64100000000000001,"15min":0.66500000000000004},"min":{"1min":0.42599999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95299999999999996,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.66500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92100000000000004,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.39500000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79500000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.443,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78500000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.42599999999999999}]},{"osd":3,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61699999999999999,"15min":0.67300000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90100000000000002,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.879,"5min":1.042,"15min":1.042},"last":0.60799999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52300000000000002,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.69199999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.625},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59999999999999998,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.80400000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.66600000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.60699999999999998},"min":{"1min":0.38,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.622,"15min":0.64200000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85999999999999999,"5min":1.036,"15min":1.036},"last":0.59699999999999998}]},{"osd":7,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64600000000000002,"15min":0.68400000000000005},"min":{"1min":0.39900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.84499999999999997,"5min":1.329,"15min":1.329},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64800000000000002,"15min":0.68300000000000005},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88200000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.67900000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:13:24.406 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T11:13:24.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.529+0000 7f70fef4f700 1 Processor -- start 2023-11-28T11:13:24.531 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.529+0000 7f70fef4f700 1 -- start start 2023-11-28T11:13:24.533 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.530+0000 7f70fef4f700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f70f80cc0c0 0x7f70f80cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.530+0000 7f70fef4f700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f70f80cdd40 0x7f70f80ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.530+0000 7f70fef4f700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70f815c8f0 con 0x7f70f815c520 2023-11-28T11:13:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.530+0000 7f70fef4f700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f806af30 con 0x7f70f80cdd40 2023-11-28T11:13:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.530+0000 7f70fef4f700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70f80cca70 con 0x7f70f80cc0c0 2023-11-28T11:13:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.530+0000 7f70fd4ec700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f70f80cdd40 0x7f70f80ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.530+0000 7f70fd4ec700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f70f80cdd40 0x7f70f80ce1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52502/0 (socket says 172.21.15.28:52502) 2023-11-28T11:13:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.530+0000 7f70fd4ec700 1 -- 172.21.15.28:0/1385137602 learned_addr learned my addr 172.21.15.28:0/1385137602 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:24.534 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.530+0000 7f70f7fff700 1 --2- 172.21.15.28:0/1385137602 >> v2:172.21.15.59:3300/0 conn(0x7f70f80cc0c0 0x7f70f80cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.530+0000 7f70fd4ec700 1 -- 172.21.15.28:0/1385137602 >> v2:172.21.15.59:3300/0 conn(0x7f70f80cc0c0 msgr2=0x7f70f80cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.531+0000 7f70fd4ec700 1 --2- 172.21.15.28:0/1385137602 >> v2:172.21.15.59:3300/0 conn(0x7f70f80cc0c0 0x7f70f80cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.531+0000 7f70fd4ec700 1 -- 172.21.15.28:0/1385137602 >> v1:172.21.15.59:6789/0 conn(0x7f70f815c520 legacy=0x7f70f80c9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.531+0000 7f70fd4ec700 1 -- 172.21.15.28:0/1385137602 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f80caaa0 con 0x7f70f80cdd40 2023-11-28T11:13:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.531+0000 7f70f7fff700 1 --2- 172.21.15.28:0/1385137602 >> v2:172.21.15.59:3300/0 conn(0x7f70f80cc0c0 0x7f70f80cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:13:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.531+0000 7f70fd4ec700 1 --2- 172.21.15.28:0/1385137602 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70f80cdd40 0x7f70f80ce1c0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f70f80b2550 tx=0x7f70f000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eae74a47c1e1d241 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.531+0000 7f70f77fe700 1 -- 172.21.15.28:0/1385137602 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f000e070 con 0x7f70f80cdd40 2023-11-28T11:13:24.535 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.531+0000 7f70f77fe700 1 -- 172.21.15.28:0/1385137602 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70f0003a20 con 0x7f70f80cdd40 2023-11-28T11:13:24.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.531+0000 7f70f77fe700 1 -- 172.21.15.28:0/1385137602 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70f0003dd0 con 0x7f70f80cdd40 2023-11-28T11:13:24.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.531+0000 7f70fef4f700 1 -- 172.21.15.28:0/1385137602 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70f80cdd40 msgr2=0x7f70f80ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:24.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.531+0000 7f70fef4f700 1 --2- 172.21.15.28:0/1385137602 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70f80cdd40 0x7f70f80ce1c0 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f70f80b2550 tx=0x7f70f000b330 comp rx=0 tx=0).stop 2023-11-28T11:13:24.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.531+0000 7f70fef4f700 1 -- 172.21.15.28:0/1385137602 shutdown_connections 2023-11-28T11:13:24.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.531+0000 7f70fef4f700 1 --2- 172.21.15.28:0/1385137602 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70f80cdd40 0x7f70f80ce1c0 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.531+0000 7f70fef4f700 1 --2- 172.21.15.28:0/1385137602 >> v2:172.21.15.59:3300/0 conn(0x7f70f80cc0c0 0x7f70f80cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.531+0000 7f70fef4f700 1 -- 172.21.15.28:0/1385137602 >> 172.21.15.28:0/1385137602 conn(0x7f70f806dad0 msgr2=0x7f70f806dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:24.536 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.532+0000 7f70fef4f700 1 -- 172.21.15.28:0/1385137602 shutdown_connections 2023-11-28T11:13:24.537 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.532+0000 7f70fef4f700 1 -- 172.21.15.28:0/1385137602 wait complete. 2023-11-28T11:13:24.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.533+0000 7f70fef4f700 1 Processor -- start 2023-11-28T11:13:24.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.533+0000 7f70fef4f700 1 -- start start 2023-11-28T11:13:24.539 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fef4f700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70f80cc0c0 0x7f70f8149980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:24.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fef4f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70f80cdd40 0x7f70f80b27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:24.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fef4f700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f815c520 0x7f70f80b14a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:24.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fef4f700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f8149ec0 con 0x7f70f80cc0c0 2023-11-28T11:13:24.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fef4f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70f80b0c40 con 0x7f70f80cdd40 2023-11-28T11:13:24.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fef4f700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70f80b0310 con 0x7f70f815c520 2023-11-28T11:13:24.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fcceb700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70f80cc0c0 0x7f70f8149980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:24.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fcceb700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70f80cc0c0 0x7f70f8149980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52504/0 (socket says 172.21.15.28:52504) 2023-11-28T11:13:24.540 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fcceb700 1 -- 172.21.15.28:0/2846670801 learned_addr learned my addr 172.21.15.28:0/2846670801 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:24.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fd4ec700 1 --2- 172.21.15.28:0/2846670801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f815c520 0x7f70f80b14a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:24.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70f7fff700 1 --2- 172.21.15.28:0/2846670801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70f80cdd40 0x7f70f80b27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:24.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fcceb700 1 -- 172.21.15.28:0/2846670801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f815c520 msgr2=0x7f70f80b14a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:24.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fcceb700 1 --2- 172.21.15.28:0/2846670801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f815c520 0x7f70f80b14a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fcceb700 1 -- 172.21.15.28:0/2846670801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70f80cdd40 msgr2=0x7f70f80b27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:24.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fcceb700 1 --2- 172.21.15.28:0/2846670801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70f80cdd40 0x7f70f80b27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fcceb700 1 -- 172.21.15.28:0/2846670801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70f0009b30 con 0x7f70f80cc0c0 2023-11-28T11:13:24.541 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70fcceb700 1 --2- 172.21.15.28:0/2846670801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70f80cc0c0 0x7f70f8149980 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00b4a0 tx=0x7f70ec00b860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70f57fa700 1 -- 172.21.15.28:0/2846670801 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec011830 con 0x7f70f80cc0c0 2023-11-28T11:13:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70f57fa700 1 -- 172.21.15.28:0/2846670801 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70ec011990 con 0x7f70f80cc0c0 2023-11-28T11:13:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.534+0000 7f70f57fa700 1 -- 172.21.15.28:0/2846670801 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70ec011d40 con 0x7f70f80cc0c0 2023-11-28T11:13:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.535+0000 7f70fef4f700 1 -- 172.21.15.28:0/2846670801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70f80cb060 con 0x7f70f80cc0c0 2023-11-28T11:13:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.535+0000 7f70fef4f700 1 -- 172.21.15.28:0/2846670801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70f80cad80 con 0x7f70f80cc0c0 2023-11-28T11:13:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.536+0000 7f70fef4f700 1 -- 172.21.15.28:0/2846670801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70f80cd820 con 0x7f70f80cc0c0 2023-11-28T11:13:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.536+0000 7f70f57fa700 1 -- 172.21.15.28:0/2846670801 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f70ec00f050 con 0x7f70f80cc0c0 2023-11-28T11:13:24.542 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.536+0000 7f70f57fa700 1 --2- 172.21.15.28:0/2846670801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70dc0682f0 0x7f70dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:24.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.537+0000 7f70f7fff700 1 --2- 172.21.15.28:0/2846670801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70dc0682f0 0x7f70dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:24.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.537+0000 7f70f57fa700 1 -- 172.21.15.28:0/2846670801 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f70ec085650 con 0x7f70f80cc0c0 2023-11-28T11:13:24.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.537+0000 7f70f7fff700 1 --2- 172.21.15.28:0/2846670801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70dc0682f0 0x7f70dc06a7a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f70e4006fd0 tx=0x7f70e4008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:24.543 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.541+0000 7f70f57fa700 1 -- 172.21.15.28:0/2846670801 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f70ec052010 con 0x7f70f80cc0c0 2023-11-28T11:13:24.740 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.739+0000 7f70fef4f700 1 -- 172.21.15.28:0/2846670801 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70f80739f0 con 0x7f70f80cc0c0 2023-11-28T11:13:24.741 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.740+0000 7f70f57fa700 1 -- 172.21.15.28:0/2846670801 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v71) v1 ==== 74+0+14496 (secure 0 0 0) 0x7f70ec055630 con 0x7f70f80cc0c0 2023-11-28T11:13:24.741 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:24.742 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":71,"fsid":"6e299adc-7a2a-4cf1-afa2-7ce104b29a51","created":"2023-11-28T11:08:27.318241+0000","modified":"2023-11-28T11:13:21.361127+0000","last_up_change":"2023-11-28T11:08:37.353927+0000","last_in_change":"2023-11-28T11:08:29.891204+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2023-11-28T11:08:38.728197+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2023-11-28T11:08:39.218586+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"5fc20989-bed6-49f3-875f-71acbd109417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6800","nonce":103508},{"type":"v1","addr":"172.21.15.28:6801","nonce":103508}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6802","nonce":103508},{"type":"v1","addr":"172.21.15.28:6803","nonce":103508}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6806","nonce":103508},{"type":"v1","addr":"172.21.15.28:6807","nonce":103508}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6804","nonce":103508},{"type":"v1","addr":"172.21.15.28:6805","nonce":103508}]},"public_addr":"172.21.15.28:6801/103508","cluster_addr":"172.21.15.28:6803/103508","heartbeat_back_addr":"172.21.15.28:6807/103508","heartbeat_front_addr":"172.21.15.28:6805/103508","state":["exists","up"]},{"osd":1,"uuid":"7a973281-6ea0-4b40-8af3-7e7f0f7f3ca7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6824","nonce":103511},{"type":"v1","addr":"172.21.15.28:6825","nonce":103511}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6826","nonce":103511},{"type":"v1","addr":"172.21.15.28:6827","nonce":103511}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6830","nonce":103511},{"type":"v1","addr":"172.21.15.28:6831","nonce":103511}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6828","nonce":103511},{"type":"v1","addr":"172.21.15.28:6829","nonce":103511}]},"public_addr":"172.21.15.28:6825/103511","cluster_addr":"172.21.15.28:6827/103511","heartbeat_back_addr":"172.21.15.28:6831/103511","heartbeat_front_addr":"172.21.15.28:6829/103511","state":["exists","up"]},{"osd":2,"uuid":"f65f7a5b-99af-48a6-92ac-01403f42b390","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6816","nonce":103512},{"type":"v1","addr":"172.21.15.28:6817","nonce":103512}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6818","nonce":103512},{"type":"v1","addr":"172.21.15.28:6819","nonce":103512}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6822","nonce":103512},{"type":"v1","addr":"172.21.15.28:6823","nonce":103512}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6820","nonce":103512},{"type":"v1","addr":"172.21.15.28:6821","nonce":103512}]},"public_addr":"172.21.15.28:6817/103512","cluster_addr":"172.21.15.28:6819/103512","heartbeat_back_addr":"172.21.15.28:6823/103512","heartbeat_front_addr":"172.21.15.28:6821/103512","state":["exists","up"]},{"osd":3,"uuid":"33cd8375-45c3-49b6-a4bb-fdcc1b2e4511","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6808","nonce":103510},{"type":"v1","addr":"172.21.15.28:6809","nonce":103510}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6810","nonce":103510},{"type":"v1","addr":"172.21.15.28:6811","nonce":103510}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6814","nonce":103510},{"type":"v1","addr":"172.21.15.28:6815","nonce":103510}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6812","nonce":103510},{"type":"v1","addr":"172.21.15.28:6813","nonce":103510}]},"public_addr":"172.21.15.28:6809/103510","cluster_addr":"172.21.15.28:6811/103510","heartbeat_back_addr":"172.21.15.28:6815/103510","heartbeat_front_addr":"172.21.15.28:6813/103510","state":["exists","up"]},{"osd":4,"uuid":"8405f8e6-e7cf-4db6-9379-f177a3ae6acb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6801","nonce":103695},{"type":"v1","addr":"172.21.15.59:6803","nonce":103695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6805","nonce":103695},{"type":"v1","addr":"172.21.15.59:6807","nonce":103695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6813","nonce":103695},{"type":"v1","addr":"172.21.15.59:6815","nonce":103695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6809","nonce":103695},{"type":"v1","addr":"172.21.15.59:6811","nonce":103695}]},"public_addr":"172.21.15.59:6803/103695","cluster_addr":"172.21.15.59:6807/103695","heartbeat_back_addr":"172.21.15.59:6815/103695","heartbeat_front_addr":"172.21.15.59:6811/103695","state":["exists","up"]},{"osd":5,"uuid":"10082fea-1dcf-4140-b3b5-b02ce671e800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":103696},{"type":"v1","addr":"172.21.15.59:6802","nonce":103696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":103696},{"type":"v1","addr":"172.21.15.59:6806","nonce":103696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":103696},{"type":"v1","addr":"172.21.15.59:6814","nonce":103696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":103696},{"type":"v1","addr":"172.21.15.59:6810","nonce":103696}]},"public_addr":"172.21.15.59:6802/103696","cluster_addr":"172.21.15.59:6806/103696","heartbeat_back_addr":"172.21.15.59:6814/103696","heartbeat_front_addr":"172.21.15.59:6810/103696","state":["exists","up"]},{"osd":6,"uuid":"25e6b8ae-36a7-4713-8199-3916b0a4b217","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":103697},{"type":"v1","addr":"172.21.15.59:6817","nonce":103697}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":103697},{"type":"v1","addr":"172.21.15.59:6819","nonce":103697}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":103697},{"type":"v1","addr":"172.21.15.59:6823","nonce":103697}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":103697},{"type":"v1","addr":"172.21.15.59:6821","nonce":103697}]},"public_addr":"172.21.15.59:6817/103697","cluster_addr":"172.21.15.59:6819/103697","heartbeat_back_addr":"172.21.15.59:6823/103697","heartbeat_front_addr":"172.21.15.59:6821/103697","state":["exists","up"]},{"osd":7,"uuid":"12eb3ca7-d8e0-4cfd-a68d-528bb95118d9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":103734},{"type":"v1","addr":"172.21.15.59:6825","nonce":103734}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":103734},{"type":"v1","addr":"172.21.15.59:6827","nonce":103734}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":103734},{"type":"v1","addr":"172.21.15.59:6831","nonce":103734}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":103734},{"type":"v1","addr":"172.21.15.59:6829","nonce":103734}]},"public_addr":"172.21.15.59:6825/103734","cluster_addr":"172.21.15.59:6827/103734","heartbeat_back_addr":"172.21.15.59:6831/103734","heartbeat_front_addr":"172.21.15.59:6829/103734","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:34.828503+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.076675+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.165638+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.129911+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.093677+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.044126+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.070337+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T11:08:35.072141+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:6834/2406852565":"2023-11-29T11:13:17.601917+0000","172.21.15.59:6832/2406852565":"2023-11-29T11:13:17.601917+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T11:13:24.743 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.742+0000 7f70fef4f700 1 -- 172.21.15.28:0/2846670801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70dc0682f0 msgr2=0x7f70dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:24.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.742+0000 7f70fef4f700 1 --2- 172.21.15.28:0/2846670801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70dc0682f0 0x7f70dc06a7a0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f70e4006fd0 tx=0x7f70e4008040 comp rx=0 tx=0).stop 2023-11-28T11:13:24.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.742+0000 7f70fef4f700 1 -- 172.21.15.28:0/2846670801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70f80cc0c0 msgr2=0x7f70f8149980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:24.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.742+0000 7f70fef4f700 1 --2- 172.21.15.28:0/2846670801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70f80cc0c0 0x7f70f8149980 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7f70ec00b4a0 tx=0x7f70ec00b860 comp rx=0 tx=0).stop 2023-11-28T11:13:24.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.742+0000 7f70fef4f700 1 -- 172.21.15.28:0/2846670801 shutdown_connections 2023-11-28T11:13:24.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.742+0000 7f70fef4f700 1 --2- 172.21.15.28:0/2846670801 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f70dc0682f0 0x7f70dc06a7a0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.742+0000 7f70fef4f700 1 --2- 172.21.15.28:0/2846670801 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f70f80cc0c0 0x7f70f8149980 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.744 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.742+0000 7f70fef4f700 1 --2- 172.21.15.28:0/2846670801 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f70f815c520 0x7f70f80b14a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.742+0000 7f70fef4f700 1 --2- 172.21.15.28:0/2846670801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f70f80cdd40 0x7f70f80b27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:24.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.742+0000 7f70fef4f700 1 -- 172.21.15.28:0/2846670801 >> 172.21.15.28:0/2846670801 conn(0x7f70f806dad0 msgr2=0x7f70f806c2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:24.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.742+0000 7f70fef4f700 1 -- 172.21.15.28:0/2846670801 shutdown_connections 2023-11-28T11:13:24.745 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:24.742+0000 7f70fef4f700 1 -- 172.21.15.28:0/2846670801 wait complete. 2023-11-28T11:13:25.757 INFO:tasks.ceph:Scrubbing osd.0 2023-11-28T11:13:25.757 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:13:25.900 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.899+0000 7feac5f72700 1 Processor -- start 2023-11-28T11:13:25.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.899+0000 7feac5f72700 1 -- start start 2023-11-28T11:13:25.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feac5f72700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7feac006c640 0x7feac015cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:25.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feac5f72700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac0160ad0 con 0x7feac0160f60 2023-11-28T11:13:25.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feac5f72700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feac015d760 con 0x7feac015c740 2023-11-28T11:13:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feac5f72700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac015d9d0 con 0x7feac006c640 2023-11-28T11:13:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feabffff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7feac0160f60 0x7feac015d050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40574/0 (socket says 172.21.15.28:40574) 2023-11-28T11:13:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feabffff700 1 -- 172.21.15.28:0/3433620660 learned_addr learned my addr 172.21.15.28:0/3433620660 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feabeffd700 1 --2- 172.21.15.28:0/3433620660 >> v2:172.21.15.59:3300/0 conn(0x7feac006c640 0x7feac015cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:25.912 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385043905 0 0) 0x7feac0160ad0 con 0x7feac0160f60 2023-11-28T11:13:25.913 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feaa80035b0 con 0x7feac0160f60 2023-11-28T11:13:25.913 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583833629 0 0) 0x7feac015d760 con 0x7feac015c740 2023-11-28T11:13:25.913 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feac0160ad0 con 0x7feac015c740 2023-11-28T11:13:25.913 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1775218401 0 0) 0x7feaa80035b0 con 0x7feac0160f60 2023-11-28T11:13:25.913 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 >> v2:172.21.15.59:3300/0 conn(0x7feac006c640 msgr2=0x7feac015cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:25.914 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feabe7fc700 1 --2- 172.21.15.28:0/3433620660 >> v2:172.21.15.59:3300/0 conn(0x7feac006c640 0x7feac015cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:25.914 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 >> v1:172.21.15.59:6789/0 conn(0x7feac015c740 legacy=0x7feac006bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:25.914 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac0160480 con 0x7feac0160f60 2023-11-28T11:13:25.914 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7feab80019f0 con 0x7feac0160f60 2023-11-28T11:13:25.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.900+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 >> v1:172.21.15.28:6789/0 conn(0x7feac0160f60 legacy=0x7feac015d050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:25.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabe7fc700 1 --2- 172.21.15.28:0/3433620660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feaa8003b90 0x7feaa8003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:25.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabe7fc700 1 --2- 172.21.15.28:0/3433620660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feaa8004850 0x7feaa8004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:25.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabe7fc700 1 --2- 172.21.15.28:0/3433620660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feaa8005200 0x7feaa800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:25.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac015d9d0 con 0x7feaa8003b90 2023-11-28T11:13:25.915 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa800c060 con 0x7feaa8005200 2023-11-28T11:13:25.916 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa800c220 con 0x7feaa8004850 2023-11-28T11:13:25.916 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabeffd700 1 --2- 172.21.15.28:0/3433620660 >> v2:172.21.15.59:3300/0 conn(0x7feac006c640 0x7feac015cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:13:25.916 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabf7fe700 1 --2- 172.21.15.28:0/3433620660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feaa8003b90 0x7feaa8003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:25.916 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabffff700 1 --2- 172.21.15.28:0/3433620660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feaa8005200 0x7feaa800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:25.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabeffd700 1 --2- 172.21.15.28:0/3433620660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feaa8004850 0x7feaa8004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:25.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabf7fe700 1 -- 172.21.15.28:0/3433620660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feaa8004850 msgr2=0x7feaa8004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:25.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabf7fe700 1 --2- 172.21.15.28:0/3433620660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feaa8004850 0x7feaa8004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:25.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabf7fe700 1 -- 172.21.15.28:0/3433620660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feaa8005200 msgr2=0x7feaa800ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:25.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabf7fe700 1 --2- 172.21.15.28:0/3433620660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feaa8005200 0x7feaa800ba10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:25.917 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabffff700 1 -- 172.21.15.28:0/3433620660 reap_dead start 2023-11-28T11:13:25.918 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabf7fe700 1 -- 172.21.15.28:0/3433620660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa80035b0 con 0x7feaa8003b90 2023-11-28T11:13:25.918 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabf7fe700 1 --2- 172.21.15.28:0/3433620660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feaa8003b90 0x7feaa8003f80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7feab4001930 tx=0x7feaa8005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ebae4f985327f71 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:25.918 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40052d0 con 0x7feaa8003b90 2023-11-28T11:13:25.918 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab4005450 con 0x7feaa8003b90 2023-11-28T11:13:25.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.901+0000 7feabe7fc700 1 -- 172.21.15.28:0/3433620660 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40038c0 con 0x7feaa8003b90 2023-11-28T11:13:25.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.902+0000 7feac5f72700 1 -- 172.21.15.28:0/3433620660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feaa8003b90 msgr2=0x7feaa8003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:25.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.902+0000 7feac5f72700 1 --2- 172.21.15.28:0/3433620660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feaa8003b90 0x7feaa8003f80 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7feab4001930 tx=0x7feaa8005880 comp rx=0 tx=0).stop 2023-11-28T11:13:25.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.902+0000 7feac5f72700 1 -- 172.21.15.28:0/3433620660 shutdown_connections 2023-11-28T11:13:25.919 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.902+0000 7feac5f72700 1 --2- 172.21.15.28:0/3433620660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feaa8003b90 0x7feaa8003f80 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.902+0000 7feac5f72700 1 -- 172.21.15.28:0/3433620660 >> 172.21.15.28:0/3433620660 conn(0x7feac00d2e70 msgr2=0x7feac00d3270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.902+0000 7feac5f72700 1 -- 172.21.15.28:0/3433620660 shutdown_connections 2023-11-28T11:13:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.902+0000 7feac5f72700 1 -- 172.21.15.28:0/3433620660 wait complete. 2023-11-28T11:13:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.903+0000 7feac5f72700 1 Processor -- start 2023-11-28T11:13:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.903+0000 7feac5f72700 1 -- start start 2023-11-28T11:13:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.904+0000 7feac5f72700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feaa8003b90 0x7feac00b7410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:25.920 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.904+0000 7feac5f72700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feac00b5ce0 0x7feac014e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.904+0000 7feac5f72700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feac00b3c40 0x7feac00b40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.904+0000 7feac5f72700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac00b6150 con 0x7feac00b3c40 2023-11-28T11:13:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.904+0000 7feac5f72700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac00b4620 con 0x7feac00b5ce0 2023-11-28T11:13:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.904+0000 7feac5f72700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac014e660 con 0x7feaa8003b90 2023-11-28T11:13:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.904+0000 7feabffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feac00b3c40 0x7feac00b40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.904+0000 7feabffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feac00b3c40 0x7feac00b40e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52528/0 (socket says 172.21.15.28:52528) 2023-11-28T11:13:25.921 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.904+0000 7feabffff700 1 -- 172.21.15.28:0/3781166396 learned_addr learned my addr 172.21.15.28:0/3781166396 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:25.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.905+0000 7feabf7fe700 1 --2- 172.21.15.28:0/3781166396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feaa8003b90 0x7feac00b7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:25.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.905+0000 7feabeffd700 1 --2- 172.21.15.28:0/3781166396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feac00b5ce0 0x7feac014e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:25.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.905+0000 7feabffff700 1 -- 172.21.15.28:0/3781166396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feaa8003b90 msgr2=0x7feac00b7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:25.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.905+0000 7feabffff700 1 --2- 172.21.15.28:0/3781166396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feaa8003b90 0x7feac00b7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:25.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.905+0000 7feabffff700 1 -- 172.21.15.28:0/3781166396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feac00b5ce0 msgr2=0x7feac014e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:25.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.905+0000 7feabffff700 1 --2- 172.21.15.28:0/3781166396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feac00b5ce0 0x7feac014e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:25.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.905+0000 7feabffff700 1 -- 172.21.15.28:0/3781166396 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac006bf30 con 0x7feac00b3c40 2023-11-28T11:13:25.922 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.905+0000 7feabf7fe700 1 --2- 172.21.15.28:0/3781166396 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7feaa8003b90 0x7feac00b7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:25.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.905+0000 7feabffff700 1 --2- 172.21.15.28:0/3781166396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feac00b3c40 0x7feac00b40e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7feac006cdc0 tx=0x7feab800a5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:25.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.905+0000 7feaa7fff700 1 -- 172.21.15.28:0/3781166396 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab800ad10 con 0x7feac00b3c40 2023-11-28T11:13:25.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.905+0000 7feaa7fff700 1 -- 172.21.15.28:0/3781166396 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab800ae70 con 0x7feac00b3c40 2023-11-28T11:13:25.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.905+0000 7feac5f72700 1 -- 172.21.15.28:0/3781166396 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac0160760 con 0x7feac00b3c40 2023-11-28T11:13:25.923 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.905+0000 7feaa7fff700 1 -- 172.21.15.28:0/3781166396 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab8013610 con 0x7feac00b3c40 2023-11-28T11:13:25.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.905+0000 7feac5f72700 1 -- 172.21.15.28:0/3781166396 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac006c0c0 con 0x7feac00b3c40 2023-11-28T11:13:25.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.906+0000 7feac5f72700 1 -- 172.21.15.28:0/3781166396 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7feac00739f0 con 0x7feac00b3c40 2023-11-28T11:13:25.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.906+0000 7feaa7fff700 1 -- 172.21.15.28:0/3781166396 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7feab8013770 con 0x7feac00b3c40 2023-11-28T11:13:25.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.907+0000 7feaa7fff700 1 --2- 172.21.15.28:0/3781166396 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7feaac0682f0 0x7feaac06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:25.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.907+0000 7feaa7fff700 1 -- 172.21.15.28:0/3781166396 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7feab8086570 con 0x7feac00b3c40 2023-11-28T11:13:25.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.907+0000 7feabf7fe700 1 --2- 172.21.15.28:0/3781166396 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7feaac0682f0 0x7feaac06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:25.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.907+0000 7feaa7fff700 1 --2- 172.21.15.28:0/3781166396 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x7feaac06eb90 0x7feaac070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:25.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.907+0000 7feaa7fff700 1 -- 172.21.15.28:0/3781166396 --> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7feaac0716b0 con 0x7feaac06eb90 2023-11-28T11:13:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.907+0000 7feaa7fff700 1 -- 172.21.15.28:0/3781166396 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7feab8088c20 con 0x7feac00b3c40 2023-11-28T11:13:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.907+0000 7feabeffd700 1 --2- 172.21.15.28:0/3781166396 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x7feaac06eb90 0x7feaac070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.907+0000 7feabf7fe700 1 --2- 172.21.15.28:0/3781166396 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7feaac0682f0 0x7feaac06a7a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7feab4005040 tx=0x7feab4009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.908+0000 7feabeffd700 1 --2- 172.21.15.28:0/3781166396 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x7feaac06eb90 0x7feaac070fe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:25.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.908+0000 7feaa7fff700 1 -- 172.21.15.28:0/3781166396 <== osd.0 v2:172.21.15.28:6800/103508 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7feaac0716b0 con 0x7feaac06eb90 2023-11-28T11:13:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.923+0000 7feac5f72700 1 -- 172.21.15.28:0/3781166396 --> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7feac00d0a40 con 0x7feaac06eb90 2023-11-28T11:13:25.926 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.923+0000 7feaa7fff700 1 -- 172.21.15.28:0/3781166396 <== osd.0 v2:172.21.15.28:6800/103508 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7feac00d0a40 con 0x7feaac06eb90 2023-11-28T11:13:25.926 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:13:25.926 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:13:25.927 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:13:25.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.923+0000 7feac5f72700 1 -- 172.21.15.28:0/3781166396 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x7feaac06eb90 msgr2=0x7feaac070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:25.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.923+0000 7feac5f72700 1 --2- 172.21.15.28:0/3781166396 >> [v2:172.21.15.28:6800/103508,v1:172.21.15.28:6801/103508] conn(0x7feaac06eb90 0x7feaac070fe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:25.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.923+0000 7feac5f72700 1 -- 172.21.15.28:0/3781166396 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7feaac0682f0 msgr2=0x7feaac06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:25.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.923+0000 7feac5f72700 1 --2- 172.21.15.28:0/3781166396 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7feaac0682f0 0x7feaac06a7a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7feab4005040 tx=0x7feab4009030 comp rx=0 tx=0).stop 2023-11-28T11:13:25.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.923+0000 7feac5f72700 1 -- 172.21.15.28:0/3781166396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feac00b3c40 msgr2=0x7feac00b40e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:25.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.923+0000 7feac5f72700 1 --2- 172.21.15.28:0/3781166396 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7feac00b3c40 0x7feac00b40e0 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7feac006cdc0 tx=0x7feab800a5b0 comp rx=0 tx=0).stop 2023-11-28T11:13:25.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.923+0000 7feabffff700 1 -- 172.21.15.28:0/3781166396 reap_dead start 2023-11-28T11:13:25.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.923+0000 7feac5f72700 1 -- 172.21.15.28:0/3781166396 shutdown_connections 2023-11-28T11:13:25.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.923+0000 7feac5f72700 1 -- 172.21.15.28:0/3781166396 >> 172.21.15.28:0/3781166396 conn(0x7feac00d2e70 msgr2=0x7feac00b3240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:25.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.924+0000 7feac5f72700 1 -- 172.21.15.28:0/3781166396 shutdown_connections 2023-11-28T11:13:25.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:25.924+0000 7feac5f72700 1 -- 172.21.15.28:0/3781166396 wait complete. 2023-11-28T11:13:25.937 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-28T11:13:26.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.061+0000 7f9b84e84700 1 Processor -- start 2023-11-28T11:13:26.066 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.061+0000 7f9b84e84700 1 -- start start 2023-11-28T11:13:26.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b84e84700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f9b80161090 0x7f9b800ca970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b84e84700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b800caeb0 0x7f9b800cc310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b84e84700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b800ce340 con 0x7f9b8015c990 2023-11-28T11:13:26.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b84e84700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b800cebe0 con 0x7f9b80161090 2023-11-28T11:13:26.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b84e84700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b800cf780 con 0x7f9b800caeb0 2023-11-28T11:13:26.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7e59c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f9b8015c990 0x7f9b800cf070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40592/0 (socket says 172.21.15.28:40592) 2023-11-28T11:13:26.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7e59c700 1 -- 172.21.15.28:0/43454157 learned_addr learned my addr 172.21.15.28:0/43454157 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:26.067 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7ed9d700 1 --2- 172.21.15.28:0/43454157 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b800caeb0 0x7f9b800cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7dd9b700 1 --2- 172.21.15.28:0/43454157 >> v2:172.21.15.59:3300/0 conn(0x7f9b80161090 0x7f9b800ca970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7d59a700 1 -- 172.21.15.28:0/43454157 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 653221013 0 0) 0x7f9b800ce340 con 0x7f9b8015c990 2023-11-28T11:13:26.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7d59a700 1 -- 172.21.15.28:0/43454157 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9b680035b0 con 0x7f9b8015c990 2023-11-28T11:13:26.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7d59a700 1 -- 172.21.15.28:0/43454157 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1814107189 0 0) 0x7f9b680035b0 con 0x7f9b8015c990 2023-11-28T11:13:26.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7d59a700 1 -- 172.21.15.28:0/43454157 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b800caeb0 msgr2=0x7f9b800cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:26.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7d59a700 1 --2- 172.21.15.28:0/43454157 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b800caeb0 0x7f9b800cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7d59a700 1 -- 172.21.15.28:0/43454157 >> v2:172.21.15.59:3300/0 conn(0x7f9b80161090 msgr2=0x7f9b800ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:26.068 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7d59a700 1 --2- 172.21.15.28:0/43454157 >> v2:172.21.15.59:3300/0 conn(0x7f9b80161090 0x7f9b800ca970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7d59a700 1 -- 172.21.15.28:0/43454157 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b800cb3f0 con 0x7f9b8015c990 2023-11-28T11:13:26.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7d59a700 1 -- 172.21.15.28:0/43454157 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f9b70001670 con 0x7f9b8015c990 2023-11-28T11:13:26.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7d59a700 1 -- 172.21.15.28:0/43454157 >> v1:172.21.15.28:6789/0 conn(0x7f9b8015c990 legacy=0x7f9b800cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7d59a700 1 --2- 172.21.15.28:0/43454157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b68003a00 0x7f9b68003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.062+0000 7f9b7d59a700 1 --2- 172.21.15.28:0/43454157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b68004850 0x7f9b68004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.069 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7d59a700 1 --2- 172.21.15.28:0/43454157 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b800caeb0 0x7f9b68005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7d59a700 1 -- 172.21.15.28:0/43454157 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b800cebe0 con 0x7f9b68004850 2023-11-28T11:13:26.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7d59a700 1 -- 172.21.15.28:0/43454157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b800cf780 con 0x7f9b68003a00 2023-11-28T11:13:26.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7d59a700 1 -- 172.21.15.28:0/43454157 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b6800bac0 con 0x7f9b800caeb0 2023-11-28T11:13:26.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7dd9b700 1 --2- 172.21.15.28:0/43454157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b68004850 0x7f9b68004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7e59c700 1 --2- 172.21.15.28:0/43454157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b68003a00 0x7f9b68003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7ed9d700 1 --2- 172.21.15.28:0/43454157 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b800caeb0 0x7f9b68005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.070 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7dd9b700 1 -- 172.21.15.28:0/43454157 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b800caeb0 msgr2=0x7f9b68005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:26.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7dd9b700 1 --2- 172.21.15.28:0/43454157 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b800caeb0 0x7f9b68005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7dd9b700 1 -- 172.21.15.28:0/43454157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b68003a00 msgr2=0x7f9b68003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:26.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7dd9b700 1 --2- 172.21.15.28:0/43454157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b68003a00 0x7f9b68003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7dd9b700 1 -- 172.21.15.28:0/43454157 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b680035b0 con 0x7f9b68004850 2023-11-28T11:13:26.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7e59c700 1 --2- 172.21.15.28:0/43454157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b68003a00 0x7f9b68003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:26.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7dd9b700 1 --2- 172.21.15.28:0/43454157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b68004850 0x7f9b68004cc0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009b60 tx=0x7f9b68005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fed021782964bef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:26.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7d59a700 1 -- 172.21.15.28:0/43454157 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7400e6c0 con 0x7f9b68004850 2023-11-28T11:13:26.071 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7d59a700 1 -- 172.21.15.28:0/43454157 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b7400e820 con 0x7f9b68004850 2023-11-28T11:13:26.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.063+0000 7f9b7d59a700 1 -- 172.21.15.28:0/43454157 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7400eb90 con 0x7f9b68004850 2023-11-28T11:13:26.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.064+0000 7f9b84e84700 1 -- 172.21.15.28:0/43454157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b68004850 msgr2=0x7f9b68004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.064+0000 7f9b84e84700 1 --2- 172.21.15.28:0/43454157 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b68004850 0x7f9b68004cc0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f9b74009b60 tx=0x7f9b68005880 comp rx=0 tx=0).stop 2023-11-28T11:13:26.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.064+0000 7f9b7ed9d700 1 -- 172.21.15.28:0/43454157 reap_dead start 2023-11-28T11:13:26.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.064+0000 7f9b84e84700 1 -- 172.21.15.28:0/43454157 shutdown_connections 2023-11-28T11:13:26.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.064+0000 7f9b84e84700 1 -- 172.21.15.28:0/43454157 >> 172.21.15.28:0/43454157 conn(0x7f9b800b1650 msgr2=0x7f9b800b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:26.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.064+0000 7f9b84e84700 1 -- 172.21.15.28:0/43454157 shutdown_connections 2023-11-28T11:13:26.072 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.064+0000 7f9b84e84700 1 -- 172.21.15.28:0/43454157 wait complete. 2023-11-28T11:13:26.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.065+0000 7f9b84e84700 1 Processor -- start 2023-11-28T11:13:26.075 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.065+0000 7f9b84e84700 1 -- start start 2023-11-28T11:13:26.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.066+0000 7f9b84e84700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b68004850 0x7f9b801f2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.066+0000 7f9b84e84700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b801f1870 0x7f9b800b2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.066+0000 7f9b84e84700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b801f1ce0 0x7f9b800b2e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.066+0000 7f9b84e84700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b801f3090 con 0x7f9b801f1ce0 2023-11-28T11:13:26.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.066+0000 7f9b84e84700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b800b46d0 con 0x7f9b801f1870 2023-11-28T11:13:26.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.066+0000 7f9b84e84700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b800b49b0 con 0x7f9b68004850 2023-11-28T11:13:26.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.066+0000 7f9b7ed9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b801f1ce0 0x7f9b800b2e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.076 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.066+0000 7f9b7ed9d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b801f1ce0 0x7f9b800b2e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52550/0 (socket says 172.21.15.28:52550) 2023-11-28T11:13:26.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.066+0000 7f9b7ed9d700 1 -- 172.21.15.28:0/3318062470 learned_addr learned my addr 172.21.15.28:0/3318062470 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:26.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.066+0000 7f9b7dd9b700 1 --2- 172.21.15.28:0/3318062470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b801f1870 0x7f9b800b2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.066+0000 7f9b7e59c700 1 --2- 172.21.15.28:0/3318062470 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b68004850 0x7f9b801f2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.066+0000 7f9b7ed9d700 1 -- 172.21.15.28:0/3318062470 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b68004850 msgr2=0x7f9b801f2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.066+0000 7f9b7ed9d700 1 --2- 172.21.15.28:0/3318062470 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b68004850 0x7f9b801f2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.066+0000 7f9b7ed9d700 1 -- 172.21.15.28:0/3318062470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b801f1870 msgr2=0x7f9b800b2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.077 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.066+0000 7f9b7ed9d700 1 --2- 172.21.15.28:0/3318062470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b801f1870 0x7f9b800b2950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.067+0000 7f9b7ed9d700 1 -- 172.21.15.28:0/3318062470 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b800ce870 con 0x7f9b801f1ce0 2023-11-28T11:13:26.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.067+0000 7f9b7ed9d700 1 --2- 172.21.15.28:0/3318062470 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b801f1ce0 0x7f9b800b2e90 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9b80161460 tx=0x7f9b780111d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:26.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.067+0000 7f9b6effd700 1 -- 172.21.15.28:0/3318062470 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b7800e070 con 0x7f9b801f1ce0 2023-11-28T11:13:26.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.067+0000 7f9b6effd700 1 -- 172.21.15.28:0/3318062470 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b78003cf0 con 0x7f9b801f1ce0 2023-11-28T11:13:26.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.067+0000 7f9b84e84700 1 -- 172.21.15.28:0/3318062470 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b800ce590 con 0x7f9b801f1ce0 2023-11-28T11:13:26.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.067+0000 7f9b6effd700 1 -- 172.21.15.28:0/3318062470 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b780117a0 con 0x7f9b801f1ce0 2023-11-28T11:13:26.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.067+0000 7f9b84e84700 1 -- 172.21.15.28:0/3318062470 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b800cc310 con 0x7f9b801f1ce0 2023-11-28T11:13:26.078 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.068+0000 7f9b84e84700 1 -- 172.21.15.28:0/3318062470 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b800c3150 con 0x7f9b801f1ce0 2023-11-28T11:13:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.068+0000 7f9b6effd700 1 -- 172.21.15.28:0/3318062470 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f9b78011900 con 0x7f9b801f1ce0 2023-11-28T11:13:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.069+0000 7f9b6effd700 1 --2- 172.21.15.28:0/3318062470 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9b600682f0 0x7f9b6006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.069+0000 7f9b7e59c700 1 --2- 172.21.15.28:0/3318062470 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9b600682f0 0x7f9b6006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.069+0000 7f9b6effd700 1 -- 172.21.15.28:0/3318062470 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f9b78085fc0 con 0x7f9b801f1ce0 2023-11-28T11:13:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.069+0000 7f9b7e59c700 1 --2- 172.21.15.28:0/3318062470 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9b600682f0 0x7f9b6006a7a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9b70000ff0 tx=0x7f9b70007040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:26.079 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.073+0000 7f9b6effd700 1 -- 172.21.15.28:0/3318062470 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f9b800c3150 con 0x7f9b801f1ce0 2023-11-28T11:13:26.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.265+0000 7f9b84e84700 1 -- 172.21.15.28:0/3318062470 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b800c3150 con 0x7f9b600682f0 2023-11-28T11:13:26.267 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.266+0000 7f9b6effd700 1 -- 172.21.15.28:0/3318062470 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9b800c3150 con 0x7f9b600682f0 2023-11-28T11:13:26.269 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.268+0000 7f9b84e84700 1 -- 172.21.15.28:0/3318062470 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9b600682f0 msgr2=0x7f9b6006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.268+0000 7f9b84e84700 1 --2- 172.21.15.28:0/3318062470 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9b600682f0 0x7f9b6006a7a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f9b70000ff0 tx=0x7f9b70007040 comp rx=0 tx=0).stop 2023-11-28T11:13:26.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.268+0000 7f9b84e84700 1 -- 172.21.15.28:0/3318062470 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b801f1ce0 msgr2=0x7f9b800b2e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.268+0000 7f9b84e84700 1 --2- 172.21.15.28:0/3318062470 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b801f1ce0 0x7f9b800b2e90 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7f9b80161460 tx=0x7f9b780111d0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.268+0000 7f9b84e84700 1 -- 172.21.15.28:0/3318062470 shutdown_connections 2023-11-28T11:13:26.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.268+0000 7f9b84e84700 1 --2- 172.21.15.28:0/3318062470 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f9b600682f0 0x7f9b6006a7a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.270 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.268+0000 7f9b84e84700 1 --2- 172.21.15.28:0/3318062470 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f9b801f1ce0 0x7f9b800b2e90 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.268+0000 7f9b84e84700 1 --2- 172.21.15.28:0/3318062470 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f9b68004850 0x7f9b801f2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.268+0000 7f9b84e84700 1 --2- 172.21.15.28:0/3318062470 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9b801f1870 0x7f9b800b2950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.268+0000 7f9b84e84700 1 -- 172.21.15.28:0/3318062470 >> 172.21.15.28:0/3318062470 conn(0x7f9b800b1650 msgr2=0x7f9b800c91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.268+0000 7f9b84e84700 1 -- 172.21.15.28:0/3318062470 shutdown_connections 2023-11-28T11:13:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.268+0000 7f9b84e84700 1 -- 172.21.15.28:0/3318062470 wait complete. 2023-11-28T11:13:26.271 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 0 to deep-scrub 2023-11-28T11:13:26.281 INFO:tasks.ceph:Scrubbing osd.1 2023-11-28T11:13:26.282 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:13:26.411 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.410+0000 7fdb5ec59700 1 Processor -- start 2023-11-28T11:13:26.412 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.410+0000 7fdb5ec59700 1 -- start start 2023-11-28T11:13:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.411+0000 7fdb5ec59700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fdb580b4fe0 0x7fdb5815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.411+0000 7fdb5ec59700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5806abb0 con 0x7fdb5815a230 2023-11-28T11:13:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.411+0000 7fdb5ec59700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb5806a280 con 0x7fdb580cd4a0 2023-11-28T11:13:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.411+0000 7fdb5ec59700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb5815a600 con 0x7fdb580b4fe0 2023-11-28T11:13:26.415 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.411+0000 7fdb5c9f5700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fdb5815a230 0x7fdb580b48d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40612/0 (socket says 172.21.15.28:40612) 2023-11-28T11:13:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.411+0000 7fdb5c9f5700 1 -- 172.21.15.28:0/3261659564 learned_addr learned my addr 172.21.15.28:0/3261659564 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.411+0000 7fdb577fe700 1 -- 172.21.15.28:0/3261659564 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3364246405 0 0) 0x7fdb5806abb0 con 0x7fdb5815a230 2023-11-28T11:13:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.411+0000 7fdb577fe700 1 -- 172.21.15.28:0/3261659564 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdb480035b0 con 0x7fdb5815a230 2023-11-28T11:13:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.411+0000 7fdb57fff700 1 --2- 172.21.15.28:0/3261659564 >> v2:172.21.15.59:3300/0 conn(0x7fdb580b4fe0 0x7fdb5815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.411+0000 7fdb577fe700 1 -- 172.21.15.28:0/3261659564 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3243951972 0 0) 0x7fdb480035b0 con 0x7fdb5815a230 2023-11-28T11:13:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.411+0000 7fdb577fe700 1 -- 172.21.15.28:0/3261659564 >> v2:172.21.15.59:3300/0 conn(0x7fdb580b4fe0 msgr2=0x7fdb5815d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.411+0000 7fdb577fe700 1 --2- 172.21.15.28:0/3261659564 >> v2:172.21.15.59:3300/0 conn(0x7fdb580b4fe0 0x7fdb5815d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.416 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.411+0000 7fdb577fe700 1 -- 172.21.15.28:0/3261659564 >> v1:172.21.15.59:6789/0 conn(0x7fdb580cd4a0 legacy=0x7fdb580d0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.411+0000 7fdb577fe700 1 -- 172.21.15.28:0/3261659564 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb5806a560 con 0x7fdb5815a230 2023-11-28T11:13:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb577fe700 1 -- 172.21.15.28:0/3261659564 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7fdb4c001670 con 0x7fdb5815a230 2023-11-28T11:13:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb577fe700 1 -- 172.21.15.28:0/3261659564 >> v1:172.21.15.28:6789/0 conn(0x7fdb5815a230 legacy=0x7fdb580b48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb577fe700 1 --2- 172.21.15.28:0/3261659564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb48003a00 0x7fdb48003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb577fe700 1 --2- 172.21.15.28:0/3261659564 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb48004850 0x7fdb48004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb577fe700 1 --2- 172.21.15.28:0/3261659564 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb48005200 0x7fdb4800ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb577fe700 1 -- 172.21.15.28:0/3261659564 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb5815a600 con 0x7fdb48005200 2023-11-28T11:13:26.417 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb577fe700 1 -- 172.21.15.28:0/3261659564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4800c080 con 0x7fdb48003a00 2023-11-28T11:13:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb577fe700 1 -- 172.21.15.28:0/3261659564 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb4800c240 con 0x7fdb48004850 2023-11-28T11:13:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb5d1f6700 1 --2- 172.21.15.28:0/3261659564 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb48005200 0x7fdb4800ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb5c9f5700 1 --2- 172.21.15.28:0/3261659564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb48003a00 0x7fdb48003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb57fff700 1 --2- 172.21.15.28:0/3261659564 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb48004850 0x7fdb48004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb5d1f6700 1 -- 172.21.15.28:0/3261659564 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb48004850 msgr2=0x7fdb48004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb5d1f6700 1 --2- 172.21.15.28:0/3261659564 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb48004850 0x7fdb48004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb5d1f6700 1 -- 172.21.15.28:0/3261659564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb48003a00 msgr2=0x7fdb48003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:26.418 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb5d1f6700 1 --2- 172.21.15.28:0/3261659564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb48003a00 0x7fdb48003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb5d1f6700 1 -- 172.21.15.28:0/3261659564 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb480035b0 con 0x7fdb48005200 2023-11-28T11:13:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb5d1f6700 1 -- 172.21.15.28:0/3261659564 reap_dead start 2023-11-28T11:13:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb5d1f6700 1 --2- 172.21.15.28:0/3261659564 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb48005200 0x7fdb4800ba10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000c190 tx=0x7fdb48005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb992324b7433761 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb577fe700 1 -- 172.21.15.28:0/3261659564 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50001db0 con 0x7fdb48005200 2023-11-28T11:13:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb577fe700 1 -- 172.21.15.28:0/3261659564 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb5000ccf0 con 0x7fdb48005200 2023-11-28T11:13:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.412+0000 7fdb577fe700 1 -- 172.21.15.28:0/3261659564 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb50003c40 con 0x7fdb48005200 2023-11-28T11:13:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.413+0000 7fdb5ec59700 1 -- 172.21.15.28:0/3261659564 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb48005200 msgr2=0x7fdb4800ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.419 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.413+0000 7fdb5ec59700 1 --2- 172.21.15.28:0/3261659564 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb48005200 0x7fdb4800ba10 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fdb5000c190 tx=0x7fdb48005880 comp rx=0 tx=0).stop 2023-11-28T11:13:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.413+0000 7fdb5ec59700 1 -- 172.21.15.28:0/3261659564 shutdown_connections 2023-11-28T11:13:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.413+0000 7fdb5ec59700 1 --2- 172.21.15.28:0/3261659564 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb48005200 0x7fdb4800ba10 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.413+0000 7fdb5ec59700 1 -- 172.21.15.28:0/3261659564 >> 172.21.15.28:0/3261659564 conn(0x7fdb580cf8d0 msgr2=0x7fdb580cfcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.413+0000 7fdb5ec59700 1 -- 172.21.15.28:0/3261659564 shutdown_connections 2023-11-28T11:13:26.420 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.413+0000 7fdb5ec59700 1 -- 172.21.15.28:0/3261659564 wait complete. 2023-11-28T11:13:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.414+0000 7fdb5ec59700 1 Processor -- start 2023-11-28T11:13:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.414+0000 7fdb5ec59700 1 -- start start 2023-11-28T11:13:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb5ec59700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb48005200 0x7fdb580c9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb5ec59700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb580cc890 0x7fdb58156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.423 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb5ec59700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb580b6cd0 0x7fdb580b7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb5ec59700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb580cb610 con 0x7fdb48005200 2023-11-28T11:13:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb5ec59700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb580b76b0 con 0x7fdb580b6cd0 2023-11-28T11:13:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb5ec59700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb580cb7f0 con 0x7fdb580cc890 2023-11-28T11:13:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb5c9f5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb48005200 0x7fdb580c9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb5c9f5700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb48005200 0x7fdb580c9510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52566/0 (socket says 172.21.15.28:52566) 2023-11-28T11:13:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb5c9f5700 1 -- 172.21.15.28:0/934312279 learned_addr learned my addr 172.21.15.28:0/934312279 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb57fff700 1 --2- 172.21.15.28:0/934312279 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb580cc890 0x7fdb58156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.424 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb5d1f6700 1 --2- 172.21.15.28:0/934312279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb580b6cd0 0x7fdb580b7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb5c9f5700 1 -- 172.21.15.28:0/934312279 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb580cc890 msgr2=0x7fdb58156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb5c9f5700 1 --2- 172.21.15.28:0/934312279 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fdb580cc890 0x7fdb58156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb5c9f5700 1 -- 172.21.15.28:0/934312279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb580b6cd0 msgr2=0x7fdb580b7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb5c9f5700 1 --2- 172.21.15.28:0/934312279 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb580b6cd0 0x7fdb580b7170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.415+0000 7fdb5c9f5700 1 -- 172.21.15.28:0/934312279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb580cd8c0 con 0x7fdb48005200 2023-11-28T11:13:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.416+0000 7fdb5c9f5700 1 --2- 172.21.15.28:0/934312279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb48005200 0x7fdb580c9510 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0072c0 tx=0x7fdb4c007680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.416+0000 7fdb557fa700 1 -- 172.21.15.28:0/934312279 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c0077a0 con 0x7fdb48005200 2023-11-28T11:13:26.425 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.416+0000 7fdb557fa700 1 -- 172.21.15.28:0/934312279 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb4c00f8d0 con 0x7fdb48005200 2023-11-28T11:13:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.416+0000 7fdb557fa700 1 -- 172.21.15.28:0/934312279 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb4c00fc80 con 0x7fdb48005200 2023-11-28T11:13:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.416+0000 7fdb5ec59700 1 -- 172.21.15.28:0/934312279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb5806a840 con 0x7fdb48005200 2023-11-28T11:13:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.416+0000 7fdb5ec59700 1 -- 172.21.15.28:0/934312279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb5806b040 con 0x7fdb48005200 2023-11-28T11:13:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.416+0000 7fdb5ec59700 1 -- 172.21.15.28:0/934312279 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdb580739f0 con 0x7fdb48005200 2023-11-28T11:13:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.417+0000 7fdb557fa700 1 -- 172.21.15.28:0/934312279 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fdb4c01d460 con 0x7fdb48005200 2023-11-28T11:13:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.418+0000 7fdb557fa700 1 --2- 172.21.15.28:0/934312279 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdb3c0682f0 0x7fdb3c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.418+0000 7fdb557fa700 1 -- 172.21.15.28:0/934312279 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fdb4c085c70 con 0x7fdb48005200 2023-11-28T11:13:26.426 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.418+0000 7fdb57fff700 1 --2- 172.21.15.28:0/934312279 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdb3c0682f0 0x7fdb3c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.418+0000 7fdb557fa700 1 --2- 172.21.15.28:0/934312279 >> [v2:172.21.15.28:6824/103511,v1:172.21.15.28:6825/103511] conn(0x7fdb3c06eb90 0x7fdb3c070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.418+0000 7fdb557fa700 1 -- 172.21.15.28:0/934312279 --> [v2:172.21.15.28:6824/103511,v1:172.21.15.28:6825/103511] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdb3c0716b0 con 0x7fdb3c06eb90 2023-11-28T11:13:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.418+0000 7fdb557fa700 1 -- 172.21.15.28:0/934312279 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdb4c088320 con 0x7fdb48005200 2023-11-28T11:13:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.418+0000 7fdb5d1f6700 1 --2- 172.21.15.28:0/934312279 >> [v2:172.21.15.28:6824/103511,v1:172.21.15.28:6825/103511] conn(0x7fdb3c06eb90 0x7fdb3c070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.418+0000 7fdb57fff700 1 --2- 172.21.15.28:0/934312279 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdb3c0682f0 0x7fdb3c06a7a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdb580b83c0 tx=0x7fdb44008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.418+0000 7fdb5d1f6700 1 --2- 172.21.15.28:0/934312279 >> [v2:172.21.15.28:6824/103511,v1:172.21.15.28:6825/103511] conn(0x7fdb3c06eb90 0x7fdb3c070fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:26.427 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.419+0000 7fdb557fa700 1 -- 172.21.15.28:0/934312279 <== osd.1 v2:172.21.15.28:6824/103511 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdb3c0716b0 con 0x7fdb3c06eb90 2023-11-28T11:13:26.434 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.433+0000 7fdb5ec59700 1 -- 172.21.15.28:0/934312279 --> [v2:172.21.15.28:6824/103511,v1:172.21.15.28:6825/103511] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdb5806b5e0 con 0x7fdb3c06eb90 2023-11-28T11:13:26.435 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.433+0000 7fdb557fa700 1 -- 172.21.15.28:0/934312279 <== osd.1 v2:172.21.15.28:6824/103511 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fdb5806b5e0 con 0x7fdb3c06eb90 2023-11-28T11:13:26.435 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:13:26.435 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:13:26.435 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:13:26.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.434+0000 7fdb5ec59700 1 -- 172.21.15.28:0/934312279 >> [v2:172.21.15.28:6824/103511,v1:172.21.15.28:6825/103511] conn(0x7fdb3c06eb90 msgr2=0x7fdb3c070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.436 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.434+0000 7fdb5ec59700 1 --2- 172.21.15.28:0/934312279 >> [v2:172.21.15.28:6824/103511,v1:172.21.15.28:6825/103511] conn(0x7fdb3c06eb90 0x7fdb3c070fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.434+0000 7fdb5ec59700 1 -- 172.21.15.28:0/934312279 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdb3c0682f0 msgr2=0x7fdb3c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.434+0000 7fdb5ec59700 1 --2- 172.21.15.28:0/934312279 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fdb3c0682f0 0x7fdb3c06a7a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fdb580b83c0 tx=0x7fdb44008040 comp rx=0 tx=0).stop 2023-11-28T11:13:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.434+0000 7fdb5ec59700 1 -- 172.21.15.28:0/934312279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb48005200 msgr2=0x7fdb580c9510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.434+0000 7fdb5ec59700 1 --2- 172.21.15.28:0/934312279 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fdb48005200 0x7fdb580c9510 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fdb4c0072c0 tx=0x7fdb4c007680 comp rx=0 tx=0).stop 2023-11-28T11:13:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.434+0000 7fdb5d1f6700 1 -- 172.21.15.28:0/934312279 reap_dead start 2023-11-28T11:13:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.434+0000 7fdb5ec59700 1 -- 172.21.15.28:0/934312279 shutdown_connections 2023-11-28T11:13:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.434+0000 7fdb5ec59700 1 -- 172.21.15.28:0/934312279 >> 172.21.15.28:0/934312279 conn(0x7fdb580cf8d0 msgr2=0x7fdb5815eb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:26.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.434+0000 7fdb5ec59700 1 -- 172.21.15.28:0/934312279 shutdown_connections 2023-11-28T11:13:26.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.434+0000 7fdb5ec59700 1 -- 172.21.15.28:0/934312279 wait complete. 2023-11-28T11:13:26.447 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-28T11:13:26.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.571+0000 7f6012610700 1 Processor -- start 2023-11-28T11:13:26.573 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.571+0000 7f6012610700 1 -- start start 2023-11-28T11:13:26.575 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.572+0000 7f6012610700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f600c15a6c0 0x7f600c0cb230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.575 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.572+0000 7f6012610700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f600c0b2660 0x7f600c0b2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.572+0000 7f6012610700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f600c0cfb90 con 0x7f600c0d0020 2023-11-28T11:13:26.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.572+0000 7f6012610700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f600c0b3020 con 0x7f600c15a6c0 2023-11-28T11:13:26.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.572+0000 7f6012610700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c0cb930 con 0x7f600c0b2660 2023-11-28T11:13:26.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.572+0000 7f600bfff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f600c15a6c0 0x7f600c0cb230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.572+0000 7f600bfff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f600c15a6c0 0x7f600c0cb230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52576/0 (socket says 172.21.15.28:52576) 2023-11-28T11:13:26.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.572+0000 7f600bfff700 1 -- 172.21.15.28:0/1341021441 learned_addr learned my addr 172.21.15.28:0/1341021441 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:26.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.572+0000 7f600b7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f600c0d0020 0x7f600c0ccea0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40636/0 (socket says 172.21.15.28:40636) 2023-11-28T11:13:26.576 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.572+0000 7f6010bad700 1 --2- 172.21.15.28:0/1341021441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f600c0b2660 0x7f600c0b2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.572+0000 7f600bfff700 1 -- 172.21.15.28:0/1341021441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f600c0b2660 msgr2=0x7f600c0b2ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:26.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.572+0000 7f600bfff700 1 --2- 172.21.15.28:0/1341021441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f600c0b2660 0x7f600c0b2ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.572+0000 7f600bfff700 1 -- 172.21.15.28:0/1341021441 >> v1:172.21.15.28:6789/0 conn(0x7f600c0d0020 legacy=0x7f600c0ccea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.572+0000 7f600bfff700 1 -- 172.21.15.28:0/1341021441 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f600c0cf260 con 0x7f600c15a6c0 2023-11-28T11:13:26.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.572+0000 7f600affd700 1 -- 172.21.15.28:0/1341021441 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 332351444 0 0) 0x7f600c0cfb90 con 0x7f600c0d0020 2023-11-28T11:13:26.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.573+0000 7f600bfff700 1 --2- 172.21.15.28:0/1341021441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f600c15a6c0 0x7f600c0cb230 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0098f0 tx=0x7f5ffc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=148dcad77fc93f87 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:26.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.573+0000 7f600affd700 1 -- 172.21.15.28:0/1341021441 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc00b8a0 con 0x7f600c15a6c0 2023-11-28T11:13:26.577 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.573+0000 7f600affd700 1 -- 172.21.15.28:0/1341021441 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ffc00ba00 con 0x7f600c15a6c0 2023-11-28T11:13:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.573+0000 7f600affd700 1 -- 172.21.15.28:0/1341021441 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ffc00bd90 con 0x7f600c15a6c0 2023-11-28T11:13:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.573+0000 7f6012610700 1 -- 172.21.15.28:0/1341021441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f600c15a6c0 msgr2=0x7f600c0cb230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.573+0000 7f6012610700 1 --2- 172.21.15.28:0/1341021441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f600c15a6c0 0x7f600c0cb230 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc0098f0 tx=0x7f5ffc009c00 comp rx=0 tx=0).stop 2023-11-28T11:13:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.573+0000 7f6012610700 1 -- 172.21.15.28:0/1341021441 shutdown_connections 2023-11-28T11:13:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.573+0000 7f6012610700 1 --2- 172.21.15.28:0/1341021441 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f600c0b2660 0x7f600c0b2ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.573+0000 7f6012610700 1 --2- 172.21.15.28:0/1341021441 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f600c15a6c0 0x7f600c0cb230 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.578 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.573+0000 7f6012610700 1 -- 172.21.15.28:0/1341021441 >> 172.21.15.28:0/1341021441 conn(0x7f600c0cd660 msgr2=0x7f600c0cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.573+0000 7f6012610700 1 -- 172.21.15.28:0/1341021441 shutdown_connections 2023-11-28T11:13:26.579 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.573+0000 7f6012610700 1 -- 172.21.15.28:0/1341021441 wait complete. 2023-11-28T11:13:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.575+0000 7f6012610700 1 Processor -- start 2023-11-28T11:13:26.581 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.575+0000 7f6012610700 1 -- start start 2023-11-28T11:13:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.575+0000 7f6012610700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f600c0b2660 0x7f600c1f2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.575+0000 7f6012610700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600c0d0020 0x7f600c06bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.575+0000 7f6012610700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f600c0c6c90 0x7f600c157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.575+0000 7f6012610700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c06dad0 con 0x7f600c0c6c90 2023-11-28T11:13:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6012610700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f600c06c280 con 0x7f600c0d0020 2023-11-28T11:13:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6012610700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f600c06c560 con 0x7f600c0b2660 2023-11-28T11:13:26.582 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6010bad700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f600c0c6c90 0x7f600c157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6010bad700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f600c0c6c90 0x7f600c157e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52586/0 (socket says 172.21.15.28:52586) 2023-11-28T11:13:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6010bad700 1 -- 172.21.15.28:0/3143101759 learned_addr learned my addr 172.21.15.28:0/3143101759 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f600bfff700 1 --2- 172.21.15.28:0/3143101759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f600c0b2660 0x7f600c1f2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f600b7fe700 1 --2- 172.21.15.28:0/3143101759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600c0d0020 0x7f600c06bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6010bad700 1 -- 172.21.15.28:0/3143101759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f600c0b2660 msgr2=0x7f600c1f2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6010bad700 1 --2- 172.21.15.28:0/3143101759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f600c0b2660 0x7f600c1f2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6010bad700 1 -- 172.21.15.28:0/3143101759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600c0d0020 msgr2=0x7f600c06bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.583 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6010bad700 1 --2- 172.21.15.28:0/3143101759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600c0d0020 0x7f600c06bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6010bad700 1 -- 172.21.15.28:0/3143101759 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc009550 con 0x7f600c0c6c90 2023-11-28T11:13:26.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f600bfff700 1 --2- 172.21.15.28:0/3143101759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f600c0b2660 0x7f600c1f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:26.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6010bad700 1 --2- 172.21.15.28:0/3143101759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f600c0c6c90 0x7f600c157e10 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f6004007e40 tx=0x7f600400e3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:26.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6008ff9700 1 -- 172.21.15.28:0/3143101759 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f600400eea0 con 0x7f600c0c6c90 2023-11-28T11:13:26.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6008ff9700 1 -- 172.21.15.28:0/3143101759 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6004014420 con 0x7f600c0c6c90 2023-11-28T11:13:26.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6008ff9700 1 -- 172.21.15.28:0/3143101759 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60040147b0 con 0x7f600c0c6c90 2023-11-28T11:13:26.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6012610700 1 -- 172.21.15.28:0/3143101759 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f600c0cf820 con 0x7f600c0c6c90 2023-11-28T11:13:26.584 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.576+0000 7f6012610700 1 -- 172.21.15.28:0/3143101759 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f600c0cf540 con 0x7f600c0c6c90 2023-11-28T11:13:26.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.577+0000 7f6012610700 1 -- 172.21.15.28:0/3143101759 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f600c0d0470 con 0x7f600c0c6c90 2023-11-28T11:13:26.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.578+0000 7f6008ff9700 1 -- 172.21.15.28:0/3143101759 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f6004012050 con 0x7f600c0c6c90 2023-11-28T11:13:26.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.578+0000 7f6008ff9700 1 --2- 172.21.15.28:0/3143101759 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5ff80682f0 0x7f5ff806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.578+0000 7f6008ff9700 1 -- 172.21.15.28:0/3143101759 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f60040861a0 con 0x7f600c0c6c90 2023-11-28T11:13:26.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.578+0000 7f600bfff700 1 --2- 172.21.15.28:0/3143101759 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5ff80682f0 0x7f5ff806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.585 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.578+0000 7f600bfff700 1 --2- 172.21.15.28:0/3143101759 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5ff80682f0 0x7f5ff806a7a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f600c072970 tx=0x7f5ffc008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:26.586 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.582+0000 7f6008ff9700 1 -- 172.21.15.28:0/3143101759 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f6004052bc0 con 0x7f600c0c6c90 2023-11-28T11:13:26.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.779+0000 7f6012610700 1 -- 172.21.15.28:0/3143101759 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f600c0675a0 con 0x7f5ff80682f0 2023-11-28T11:13:26.781 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.780+0000 7f6008ff9700 1 -- 172.21.15.28:0/3143101759 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f600c0675a0 con 0x7f5ff80682f0 2023-11-28T11:13:26.783 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.782+0000 7f6012610700 1 -- 172.21.15.28:0/3143101759 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5ff80682f0 msgr2=0x7f5ff806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.782+0000 7f6012610700 1 --2- 172.21.15.28:0/3143101759 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5ff80682f0 0x7f5ff806a7a0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f600c072970 tx=0x7f5ffc008030 comp rx=0 tx=0).stop 2023-11-28T11:13:26.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.782+0000 7f6012610700 1 -- 172.21.15.28:0/3143101759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f600c0c6c90 msgr2=0x7f600c157e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.782+0000 7f6012610700 1 --2- 172.21.15.28:0/3143101759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f600c0c6c90 0x7f600c157e10 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f6004007e40 tx=0x7f600400e3c0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.782+0000 7f6012610700 1 -- 172.21.15.28:0/3143101759 shutdown_connections 2023-11-28T11:13:26.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.782+0000 7f6012610700 1 --2- 172.21.15.28:0/3143101759 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f5ff80682f0 0x7f5ff806a7a0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.782+0000 7f6012610700 1 --2- 172.21.15.28:0/3143101759 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f600c0c6c90 0x7f600c157e10 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.784 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.782+0000 7f6012610700 1 --2- 172.21.15.28:0/3143101759 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f600c0b2660 0x7f600c1f2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.782+0000 7f6012610700 1 --2- 172.21.15.28:0/3143101759 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f600c0d0020 0x7f600c06bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.782+0000 7f6012610700 1 -- 172.21.15.28:0/3143101759 >> 172.21.15.28:0/3143101759 conn(0x7f600c0cd660 msgr2=0x7f600c0b1d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.782+0000 7f6012610700 1 -- 172.21.15.28:0/3143101759 shutdown_connections 2023-11-28T11:13:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.782+0000 7f6012610700 1 -- 172.21.15.28:0/3143101759 wait complete. 2023-11-28T11:13:26.785 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 1 to deep-scrub 2023-11-28T11:13:26.795 INFO:tasks.ceph:Scrubbing osd.2 2023-11-28T11:13:26.796 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:13:26.924 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.923+0000 7f77f87f6700 1 Processor -- start 2023-11-28T11:13:26.925 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.923+0000 7f77f87f6700 1 -- start start 2023-11-28T11:13:26.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f87f6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77f0159590 0x7f77f006e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f87f6700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f77f00ce890 0x7f77f00ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f87f6700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f77f015d120 con 0x7f77f015db70 2023-11-28T11:13:26.927 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f87f6700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f77f00cf250 con 0x7f77f00ce890 2023-11-28T11:13:26.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f87f6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f015d6e0 con 0x7f77f0159590 2023-11-28T11:13:26.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f5d91700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f77f015db70 0x7f77f00ae730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40650/0 (socket says 172.21.15.28:40650) 2023-11-28T11:13:26.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f5d91700 1 -- 172.21.15.28:0/3700905238 learned_addr learned my addr 172.21.15.28:0/3700905238 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:26.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f6592700 1 --2- 172.21.15.28:0/3700905238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77f0159590 0x7f77f006e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f6d93700 1 --2- 172.21.15.28:0/3700905238 >> v2:172.21.15.59:3300/0 conn(0x7f77f00ce890 0x7f77f00ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f5590700 1 -- 172.21.15.28:0/3700905238 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311924511 0 0) 0x7f77f015d120 con 0x7f77f015db70 2023-11-28T11:13:26.928 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f5590700 1 -- 172.21.15.28:0/3700905238 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f77dc0035b0 con 0x7f77f015db70 2023-11-28T11:13:26.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f6592700 1 -- 172.21.15.28:0/3700905238 >> v2:172.21.15.59:3300/0 conn(0x7f77f00ce890 msgr2=0x7f77f00ced10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:26.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f6592700 1 --2- 172.21.15.28:0/3700905238 >> v2:172.21.15.59:3300/0 conn(0x7f77f00ce890 0x7f77f00ced10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f6592700 1 -- 172.21.15.28:0/3700905238 >> v1:172.21.15.28:6789/0 conn(0x7f77f015db70 legacy=0x7f77f00ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f6592700 1 -- 172.21.15.28:0/3700905238 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77f00aef60 con 0x7f77f0159590 2023-11-28T11:13:26.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f6d93700 1 --2- 172.21.15.28:0/3700905238 >> v2:172.21.15.59:3300/0 conn(0x7f77f00ce890 0x7f77f00ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:13:26.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.924+0000 7f77f6592700 1 --2- 172.21.15.28:0/3700905238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77f0159590 0x7f77f006e400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009840 tx=0x7f77ec009b50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de7aff1cc01d0a15 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:26.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.925+0000 7f77f5590700 1 -- 172.21.15.28:0/3700905238 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec011830 con 0x7f77f0159590 2023-11-28T11:13:26.929 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.925+0000 7f77f5590700 1 -- 172.21.15.28:0/3700905238 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec011990 con 0x7f77f0159590 2023-11-28T11:13:26.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.925+0000 7f77f5590700 1 -- 172.21.15.28:0/3700905238 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec011d00 con 0x7f77f0159590 2023-11-28T11:13:26.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.925+0000 7f77f87f6700 1 -- 172.21.15.28:0/3700905238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77f0159590 msgr2=0x7f77f006e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.925+0000 7f77f87f6700 1 --2- 172.21.15.28:0/3700905238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77f0159590 0x7f77f006e400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f77ec009840 tx=0x7f77ec009b50 comp rx=0 tx=0).stop 2023-11-28T11:13:26.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.925+0000 7f77f87f6700 1 -- 172.21.15.28:0/3700905238 shutdown_connections 2023-11-28T11:13:26.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.925+0000 7f77f87f6700 1 --2- 172.21.15.28:0/3700905238 >> v2:172.21.15.59:3300/0 conn(0x7f77f00ce890 0x7f77f00ced10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.925+0000 7f77f87f6700 1 --2- 172.21.15.28:0/3700905238 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77f0159590 0x7f77f006e400 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.925+0000 7f77f87f6700 1 -- 172.21.15.28:0/3700905238 >> 172.21.15.28:0/3700905238 conn(0x7f77f00b1650 msgr2=0x7f77f00b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:26.930 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.925+0000 7f77f87f6700 1 -- 172.21.15.28:0/3700905238 shutdown_connections 2023-11-28T11:13:26.931 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.925+0000 7f77f87f6700 1 -- 172.21.15.28:0/3700905238 wait complete. 2023-11-28T11:13:26.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.927+0000 7f77f87f6700 1 Processor -- start 2023-11-28T11:13:26.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.927+0000 7f77f87f6700 1 -- start start 2023-11-28T11:13:26.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.927+0000 7f77f87f6700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77f00ce890 0x7f77f0153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.927+0000 7f77f87f6700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77f0159590 0x7f77f00c8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.927+0000 7f77f87f6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77f015db70 0x7f77f00c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.927+0000 7f77f87f6700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f00b36c0 con 0x7f77f00ce890 2023-11-28T11:13:26.934 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.927+0000 7f77f87f6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77f00b39a0 con 0x7f77f015db70 2023-11-28T11:13:26.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.927+0000 7f77f87f6700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77f0154fd0 con 0x7f77f0159590 2023-11-28T11:13:26.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.927+0000 7f77f6592700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77f00ce890 0x7f77f0153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.927+0000 7f77f6592700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77f00ce890 0x7f77f0153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52596/0 (socket says 172.21.15.28:52596) 2023-11-28T11:13:26.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.928+0000 7f77f6592700 1 -- 172.21.15.28:0/3661485956 learned_addr learned my addr 172.21.15.28:0/3661485956 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:26.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.928+0000 7f77f5d91700 1 --2- 172.21.15.28:0/3661485956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77f0159590 0x7f77f00c8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.928+0000 7f77f6d93700 1 --2- 172.21.15.28:0/3661485956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77f015db70 0x7f77f00c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.928+0000 7f77f6592700 1 -- 172.21.15.28:0/3661485956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77f0159590 msgr2=0x7f77f00c8b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.935 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.928+0000 7f77f6592700 1 --2- 172.21.15.28:0/3661485956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77f0159590 0x7f77f00c8b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.928+0000 7f77f6592700 1 -- 172.21.15.28:0/3661485956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77f015db70 msgr2=0x7f77f00c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.928+0000 7f77f6592700 1 --2- 172.21.15.28:0/3661485956 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77f015db70 0x7f77f00c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.928+0000 7f77f6592700 1 -- 172.21.15.28:0/3661485956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77e8009a70 con 0x7f77f00ce890 2023-11-28T11:13:26.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.928+0000 7f77f5d91700 1 --2- 172.21.15.28:0/3661485956 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f77f0159590 0x7f77f00c8b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:26.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.928+0000 7f77f6592700 1 --2- 172.21.15.28:0/3661485956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77f00ce890 0x7f77f0153aa0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0090b0 tx=0x7f77ec004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:26.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.928+0000 7f77e6ffd700 1 -- 172.21.15.28:0/3661485956 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec0036a0 con 0x7f77f00ce890 2023-11-28T11:13:26.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.928+0000 7f77e6ffd700 1 -- 172.21.15.28:0/3661485956 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77ec01f070 con 0x7f77f00ce890 2023-11-28T11:13:26.936 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.928+0000 7f77f87f6700 1 -- 172.21.15.28:0/3661485956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77ec0094a0 con 0x7f77f00ce890 2023-11-28T11:13:26.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.928+0000 7f77e6ffd700 1 -- 172.21.15.28:0/3661485956 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77ec003a40 con 0x7f77f00ce890 2023-11-28T11:13:26.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.928+0000 7f77f87f6700 1 -- 172.21.15.28:0/3661485956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77f015d370 con 0x7f77f00ce890 2023-11-28T11:13:26.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.929+0000 7f77f87f6700 1 -- 172.21.15.28:0/3661485956 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f77f00b3b10 con 0x7f77f00ce890 2023-11-28T11:13:26.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.929+0000 7f77e6ffd700 1 -- 172.21.15.28:0/3661485956 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f77ec00f050 con 0x7f77f00ce890 2023-11-28T11:13:26.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.930+0000 7f77e6ffd700 1 --2- 172.21.15.28:0/3661485956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f77d40682f0 0x7f77d406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.930+0000 7f77e6ffd700 1 -- 172.21.15.28:0/3661485956 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f77ec021070 con 0x7f77f00ce890 2023-11-28T11:13:26.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.930+0000 7f77f5d91700 1 --2- 172.21.15.28:0/3661485956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f77d40682f0 0x7f77d406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.937 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.930+0000 7f77e6ffd700 1 --2- 172.21.15.28:0/3661485956 >> [v2:172.21.15.28:6816/103512,v1:172.21.15.28:6817/103512] conn(0x7f77d406eb90 0x7f77d4070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:26.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.930+0000 7f77e6ffd700 1 -- 172.21.15.28:0/3661485956 --> [v2:172.21.15.28:6816/103512,v1:172.21.15.28:6817/103512] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f77d40716b0 con 0x7f77d406eb90 2023-11-28T11:13:26.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.930+0000 7f77e6ffd700 1 -- 172.21.15.28:0/3661485956 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f77ec089880 con 0x7f77f00ce890 2023-11-28T11:13:26.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.930+0000 7f77f6d93700 1 --2- 172.21.15.28:0/3661485956 >> [v2:172.21.15.28:6816/103512,v1:172.21.15.28:6817/103512] conn(0x7f77d406eb90 0x7f77d4070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:26.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.930+0000 7f77f5d91700 1 --2- 172.21.15.28:0/3661485956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f77d40682f0 0x7f77d406a7a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f77f01f4540 tx=0x7f77e0009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:26.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.931+0000 7f77f6d93700 1 --2- 172.21.15.28:0/3661485956 >> [v2:172.21.15.28:6816/103512,v1:172.21.15.28:6817/103512] conn(0x7f77d406eb90 0x7f77d4070fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:26.938 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.931+0000 7f77e6ffd700 1 -- 172.21.15.28:0/3661485956 <== osd.2 v2:172.21.15.28:6816/103512 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f77d40716b0 con 0x7f77d406eb90 2023-11-28T11:13:26.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.946+0000 7f77f87f6700 1 -- 172.21.15.28:0/3661485956 --> [v2:172.21.15.28:6816/103512,v1:172.21.15.28:6817/103512] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f77f00af240 con 0x7f77d406eb90 2023-11-28T11:13:26.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.946+0000 7f77e6ffd700 1 -- 172.21.15.28:0/3661485956 <== osd.2 v2:172.21.15.28:6816/103512 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f77f00af240 con 0x7f77d406eb90 2023-11-28T11:13:26.948 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:13:26.948 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:13:26.948 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:13:26.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.946+0000 7f77f87f6700 1 -- 172.21.15.28:0/3661485956 >> [v2:172.21.15.28:6816/103512,v1:172.21.15.28:6817/103512] conn(0x7f77d406eb90 msgr2=0x7f77d4070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.946+0000 7f77f87f6700 1 --2- 172.21.15.28:0/3661485956 >> [v2:172.21.15.28:6816/103512,v1:172.21.15.28:6817/103512] conn(0x7f77d406eb90 0x7f77d4070fe0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:26.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.946+0000 7f77f87f6700 1 -- 172.21.15.28:0/3661485956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f77d40682f0 msgr2=0x7f77d406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.946+0000 7f77f87f6700 1 --2- 172.21.15.28:0/3661485956 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f77d40682f0 0x7f77d406a7a0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f77f01f4540 tx=0x7f77e0009040 comp rx=0 tx=0).stop 2023-11-28T11:13:26.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.946+0000 7f77f87f6700 1 -- 172.21.15.28:0/3661485956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77f00ce890 msgr2=0x7f77f0153aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:26.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.946+0000 7f77f87f6700 1 --2- 172.21.15.28:0/3661485956 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f77f00ce890 0x7f77f0153aa0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f77ec0090b0 tx=0x7f77ec004750 comp rx=0 tx=0).stop 2023-11-28T11:13:26.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.946+0000 7f77f6d93700 1 -- 172.21.15.28:0/3661485956 reap_dead start 2023-11-28T11:13:26.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.946+0000 7f77f87f6700 1 -- 172.21.15.28:0/3661485956 shutdown_connections 2023-11-28T11:13:26.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.946+0000 7f77f87f6700 1 -- 172.21.15.28:0/3661485956 >> 172.21.15.28:0/3661485956 conn(0x7f77f00b1650 msgr2=0x7f77f015bd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:26.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.946+0000 7f77f87f6700 1 -- 172.21.15.28:0/3661485956 shutdown_connections 2023-11-28T11:13:26.950 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:26.946+0000 7f77f87f6700 1 -- 172.21.15.28:0/3661485956 wait complete. 2023-11-28T11:13:26.959 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-28T11:13:27.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.083+0000 7f0f4be4a700 1 Processor -- start 2023-11-28T11:13:27.085 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.084+0000 7f0f4be4a700 1 -- start start 2023-11-28T11:13:27.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.084+0000 7f0f4be4a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f4406e400 0x7f0f4415db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.084+0000 7f0f4be4a700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0f440ce890 0x7f0f440ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.085+0000 7f0f4be4a700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f4415e0b0 con 0x7f0f44159590 2023-11-28T11:13:27.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.085+0000 7f0f4a3e7700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0f440ce890 0x7f0f440ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.087 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.085+0000 7f0f4be4a700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f440cf250 con 0x7f0f440ce890 2023-11-28T11:13:27.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.085+0000 7f0f4be4a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4415d120 con 0x7f0f4406e400 2023-11-28T11:13:27.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.085+0000 7f0f4a3e7700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f0f440ce890 0x7f0f440ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52610/0 (socket says 172.21.15.28:52610) 2023-11-28T11:13:27.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.085+0000 7f0f4a3e7700 1 -- 172.21.15.28:0/2648470156 learned_addr learned my addr 172.21.15.28:0/2648470156 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:27.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.085+0000 7f0f493e5700 1 --2- 172.21.15.28:0/2648470156 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f4406e400 0x7f0f4415db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.085+0000 7f0f4a3e7700 1 -- 172.21.15.28:0/2648470156 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f4406e400 msgr2=0x7f0f4415db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:27.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.085+0000 7f0f4a3e7700 1 --2- 172.21.15.28:0/2648470156 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f4406e400 0x7f0f4415db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.085+0000 7f0f4a3e7700 1 -- 172.21.15.28:0/2648470156 >> v1:172.21.15.59:6789/0 conn(0x7f0f44159590 legacy=0x7f0f440ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.088 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.085+0000 7f0f4a3e7700 1 -- 172.21.15.28:0/2648470156 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4415d6e0 con 0x7f0f440ce890 2023-11-28T11:13:27.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.085+0000 7f0f4a3e7700 1 --2- 172.21.15.28:0/2648470156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f440ce890 0x7f0f440ced10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009e50 tx=0x7f0f3c00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cdd0515f17e199e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:27.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.085+0000 7f0f48be4700 1 -- 172.21.15.28:0/2648470156 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c00e070 con 0x7f0f440ce890 2023-11-28T11:13:27.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.085+0000 7f0f48be4700 1 -- 172.21.15.28:0/2648470156 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3c00b910 con 0x7f0f440ce890 2023-11-28T11:13:27.089 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.085+0000 7f0f48be4700 1 -- 172.21.15.28:0/2648470156 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c00bcc0 con 0x7f0f440ce890 2023-11-28T11:13:27.092 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.086+0000 7f0f4be4a700 1 -- 172.21.15.28:0/2648470156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f440ce890 msgr2=0x7f0f440ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.092 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.086+0000 7f0f4be4a700 1 --2- 172.21.15.28:0/2648470156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f440ce890 0x7f0f440ced10 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f0f3c009e50 tx=0x7f0f3c00b330 comp rx=0 tx=0).stop 2023-11-28T11:13:27.092 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.086+0000 7f0f4be4a700 1 -- 172.21.15.28:0/2648470156 shutdown_connections 2023-11-28T11:13:27.092 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.086+0000 7f0f4be4a700 1 --2- 172.21.15.28:0/2648470156 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f440ce890 0x7f0f440ced10 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.092 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.086+0000 7f0f4be4a700 1 --2- 172.21.15.28:0/2648470156 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f4406e400 0x7f0f4415db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.092 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.086+0000 7f0f4be4a700 1 -- 172.21.15.28:0/2648470156 >> 172.21.15.28:0/2648470156 conn(0x7f0f440b1650 msgr2=0x7f0f440b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:27.092 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.086+0000 7f0f4be4a700 1 -- 172.21.15.28:0/2648470156 shutdown_connections 2023-11-28T11:13:27.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.086+0000 7f0f4be4a700 1 -- 172.21.15.28:0/2648470156 wait complete. 2023-11-28T11:13:27.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.087+0000 7f0f4be4a700 1 Processor -- start 2023-11-28T11:13:27.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.087+0000 7f0f4be4a700 1 -- start start 2023-11-28T11:13:27.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.088+0000 7f0f4be4a700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f4406e400 0x7f0f440ab8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.088+0000 7f0f4be4a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f440ce890 0x7f0f440b53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.088+0000 7f0f4be4a700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f44159590 0x7f0f4415ad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.088+0000 7f0f4be4a700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f44159ca0 con 0x7f0f44159590 2023-11-28T11:13:27.093 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.088+0000 7f0f4be4a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f44159f80 con 0x7f0f440ce890 2023-11-28T11:13:27.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.088+0000 7f0f4be4a700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4415a260 con 0x7f0f4406e400 2023-11-28T11:13:27.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.088+0000 7f0f4a3e7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f44159590 0x7f0f4415ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.088+0000 7f0f4a3e7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f44159590 0x7f0f4415ad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52616/0 (socket says 172.21.15.28:52616) 2023-11-28T11:13:27.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.088+0000 7f0f4a3e7700 1 -- 172.21.15.28:0/3228671521 learned_addr learned my addr 172.21.15.28:0/3228671521 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:27.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.088+0000 7f0f49be6700 1 --2- 172.21.15.28:0/3228671521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f4406e400 0x7f0f440ab8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.088+0000 7f0f493e5700 1 --2- 172.21.15.28:0/3228671521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f440ce890 0x7f0f440b53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.089+0000 7f0f4a3e7700 1 -- 172.21.15.28:0/3228671521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f4406e400 msgr2=0x7f0f440ab8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.094 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.089+0000 7f0f4a3e7700 1 --2- 172.21.15.28:0/3228671521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f4406e400 0x7f0f440ab8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.089+0000 7f0f4a3e7700 1 -- 172.21.15.28:0/3228671521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f440ce890 msgr2=0x7f0f440b53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.089+0000 7f0f4a3e7700 1 --2- 172.21.15.28:0/3228671521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f440ce890 0x7f0f440b53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.089+0000 7f0f4a3e7700 1 -- 172.21.15.28:0/3228671521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f3c009b30 con 0x7f0f44159590 2023-11-28T11:13:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.089+0000 7f0f4a3e7700 1 --2- 172.21.15.28:0/3228671521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f44159590 0x7f0f4415ad40 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f0f4406eac0 tx=0x7f0f3c004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.089+0000 7f0f367fc700 1 -- 172.21.15.28:0/3228671521 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c0047c0 con 0x7f0f44159590 2023-11-28T11:13:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.089+0000 7f0f367fc700 1 -- 172.21.15.28:0/3228671521 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f3c0092e0 con 0x7f0f44159590 2023-11-28T11:13:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.089+0000 7f0f4be4a700 1 -- 172.21.15.28:0/3228671521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f440af240 con 0x7f0f44159590 2023-11-28T11:13:27.095 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.089+0000 7f0f367fc700 1 -- 172.21.15.28:0/3228671521 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f3c01c660 con 0x7f0f44159590 2023-11-28T11:13:27.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.089+0000 7f0f4be4a700 1 -- 172.21.15.28:0/3228671521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f440aef60 con 0x7f0f44159590 2023-11-28T11:13:27.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.090+0000 7f0f4be4a700 1 -- 172.21.15.28:0/3228671521 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f4415d350 con 0x7f0f44159590 2023-11-28T11:13:27.096 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.090+0000 7f0f367fc700 1 -- 172.21.15.28:0/3228671521 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f0f3c02c070 con 0x7f0f44159590 2023-11-28T11:13:27.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.091+0000 7f0f367fc700 1 --2- 172.21.15.28:0/3228671521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0f280682f0 0x7f0f2806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.091+0000 7f0f367fc700 1 -- 172.21.15.28:0/3228671521 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f0f3c0875c0 con 0x7f0f44159590 2023-11-28T11:13:27.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.091+0000 7f0f49be6700 1 --2- 172.21.15.28:0/3228671521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0f280682f0 0x7f0f2806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.091+0000 7f0f49be6700 1 --2- 172.21.15.28:0/3228671521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0f280682f0 0x7f0f2806a7a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0f38007fd0 tx=0x7f0f38009040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:27.097 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.095+0000 7f0f367fc700 1 -- 172.21.15.28:0/3228671521 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f0f3c053fe0 con 0x7f0f44159590 2023-11-28T11:13:27.289 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.288+0000 7f0f4be4a700 1 -- 172.21.15.28:0/3228671521 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f440675a0 con 0x7f0f280682f0 2023-11-28T11:13:27.290 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.288+0000 7f0f367fc700 1 -- 172.21.15.28:0/3228671521 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0f440675a0 con 0x7f0f280682f0 2023-11-28T11:13:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.290+0000 7f0f4be4a700 1 -- 172.21.15.28:0/3228671521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0f280682f0 msgr2=0x7f0f2806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.290+0000 7f0f4be4a700 1 --2- 172.21.15.28:0/3228671521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0f280682f0 0x7f0f2806a7a0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f0f38007fd0 tx=0x7f0f38009040 comp rx=0 tx=0).stop 2023-11-28T11:13:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.290+0000 7f0f4be4a700 1 -- 172.21.15.28:0/3228671521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f44159590 msgr2=0x7f0f4415ad40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.290+0000 7f0f4be4a700 1 --2- 172.21.15.28:0/3228671521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f44159590 0x7f0f4415ad40 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f0f4406eac0 tx=0x7f0f3c004ad0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.290+0000 7f0f4be4a700 1 -- 172.21.15.28:0/3228671521 shutdown_connections 2023-11-28T11:13:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.290+0000 7f0f4be4a700 1 --2- 172.21.15.28:0/3228671521 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f0f280682f0 0x7f0f2806a7a0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.292 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.290+0000 7f0f4be4a700 1 --2- 172.21.15.28:0/3228671521 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f0f44159590 0x7f0f4415ad40 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.290+0000 7f0f4be4a700 1 --2- 172.21.15.28:0/3228671521 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f0f4406e400 0x7f0f440ab8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.290+0000 7f0f4be4a700 1 --2- 172.21.15.28:0/3228671521 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0f440ce890 0x7f0f440b53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.290+0000 7f0f4be4a700 1 -- 172.21.15.28:0/3228671521 >> 172.21.15.28:0/3228671521 conn(0x7f0f440b1650 msgr2=0x7f0f440af710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.290+0000 7f0f4be4a700 1 -- 172.21.15.28:0/3228671521 shutdown_connections 2023-11-28T11:13:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.290+0000 7f0f4be4a700 1 -- 172.21.15.28:0/3228671521 wait complete. 2023-11-28T11:13:27.293 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 2 to deep-scrub 2023-11-28T11:13:27.303 INFO:tasks.ceph:Scrubbing osd.3 2023-11-28T11:13:27.304 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:13:27.432 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.431+0000 7f7f808de700 1 Processor -- start 2023-11-28T11:13:27.433 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.431+0000 7f7f808de700 1 -- start start 2023-11-28T11:13:27.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f808de700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f7f7c15c2e0 0x7f7f7c0cf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f808de700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7c0b5870 con 0x7f7f7c15c6b0 2023-11-28T11:13:27.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f808de700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f7c0b5b50 con 0x7f7f7c0b6630 2023-11-28T11:13:27.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f808de700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f7c0b6a90 con 0x7f7f7c15c2e0 2023-11-28T11:13:27.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f79d9b700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f7f7c15c6b0 0x7f7f7c0b3930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40678/0 (socket says 172.21.15.28:40678) 2023-11-28T11:13:27.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f79d9b700 1 -- 172.21.15.28:0/3381216408 learned_addr learned my addr 172.21.15.28:0/3381216408 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:27.437 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f7a59c700 1 --2- 172.21.15.28:0/3381216408 >> v2:172.21.15.59:3300/0 conn(0x7f7f7c15c2e0 0x7f7f7c0cf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 22057335 0 0) 0x7f7f7c0b5870 con 0x7f7f7c15c6b0 2023-11-28T11:13:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f640035b0 con 0x7f7f7c15c6b0 2023-11-28T11:13:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3821976795 0 0) 0x7f7f7c0b5b50 con 0x7f7f7c0b6630 2023-11-28T11:13:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f7c0b5870 con 0x7f7f7c0b6630 2023-11-28T11:13:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1043339755 0 0) 0x7f7f640035b0 con 0x7f7f7c15c6b0 2023-11-28T11:13:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 >> v2:172.21.15.59:3300/0 conn(0x7f7f7c15c2e0 msgr2=0x7f7f7c0cf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:27.438 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f7959a700 1 --2- 172.21.15.28:0/3381216408 >> v2:172.21.15.59:3300/0 conn(0x7f7f7c15c2e0 0x7f7f7c0cf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 >> v1:172.21.15.59:6789/0 conn(0x7f7f7c0b6630 legacy=0x7f7f7c0704b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c0b5e30 con 0x7f7f7c15c6b0 2023-11-28T11:13:27.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f7f70001670 con 0x7f7f7c15c6b0 2023-11-28T11:13:27.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 >> v1:172.21.15.28:6789/0 conn(0x7f7f7c15c6b0 legacy=0x7f7f7c0b3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.432+0000 7f7f7959a700 1 --2- 172.21.15.28:0/3381216408 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7f64003b90 0x7f7f64003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7959a700 1 --2- 172.21.15.28:0/3381216408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f64004850 0x7f7f64004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.439 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7959a700 1 --2- 172.21.15.28:0/3381216408 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f64005200 0x7f7f6400ba10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6400c0e0 con 0x7f7f64005200 2023-11-28T11:13:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6400c250 con 0x7f7f64004850 2023-11-28T11:13:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f6400c410 con 0x7f7f64003b90 2023-11-28T11:13:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7ad9d700 1 --2- 172.21.15.28:0/3381216408 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f64005200 0x7f7f6400ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7a59c700 1 --2- 172.21.15.28:0/3381216408 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7f64003b90 0x7f7f64003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f79d9b700 1 --2- 172.21.15.28:0/3381216408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f64004850 0x7f7f64004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7ad9d700 1 -- 172.21.15.28:0/3381216408 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7f64003b90 msgr2=0x7f7f64003f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:27.440 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7ad9d700 1 --2- 172.21.15.28:0/3381216408 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7f64003b90 0x7f7f64003f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7ad9d700 1 -- 172.21.15.28:0/3381216408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f64004850 msgr2=0x7f7f64004cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7ad9d700 1 --2- 172.21.15.28:0/3381216408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f64004850 0x7f7f64004cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7ad9d700 1 -- 172.21.15.28:0/3381216408 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f640035b0 con 0x7f7f64005200 2023-11-28T11:13:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7ad9d700 1 -- 172.21.15.28:0/3381216408 reap_dead start 2023-11-28T11:13:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7ad9d700 1 --2- 172.21.15.28:0/3381216408 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f64005200 0x7f7f6400ba10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7f74002230 tx=0x7f7f64005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6e8dfb029cb1565 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74001f30 con 0x7f7f64005200 2023-11-28T11:13:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f7400ce80 con 0x7f7f64005200 2023-11-28T11:13:27.441 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.433+0000 7f7f7959a700 1 -- 172.21.15.28:0/3381216408 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74003c40 con 0x7f7f64005200 2023-11-28T11:13:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.434+0000 7f7f808de700 1 -- 172.21.15.28:0/3381216408 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f64005200 msgr2=0x7f7f6400ba10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.434+0000 7f7f808de700 1 --2- 172.21.15.28:0/3381216408 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f64005200 0x7f7f6400ba10 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f7f74002230 tx=0x7f7f64005880 comp rx=0 tx=0).stop 2023-11-28T11:13:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.434+0000 7f7f808de700 1 -- 172.21.15.28:0/3381216408 shutdown_connections 2023-11-28T11:13:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.434+0000 7f7f808de700 1 --2- 172.21.15.28:0/3381216408 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f64005200 0x7f7f6400ba10 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.434+0000 7f7f808de700 1 -- 172.21.15.28:0/3381216408 >> 172.21.15.28:0/3381216408 conn(0x7f7f7c0d0ae0 msgr2=0x7f7f7c0d0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.434+0000 7f7f808de700 1 -- 172.21.15.28:0/3381216408 shutdown_connections 2023-11-28T11:13:27.442 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.434+0000 7f7f808de700 1 -- 172.21.15.28:0/3381216408 wait complete. 2023-11-28T11:13:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.435+0000 7f7f808de700 1 Processor -- start 2023-11-28T11:13:27.443 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.435+0000 7f7f808de700 1 -- start start 2023-11-28T11:13:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.436+0000 7f7f808de700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f64005200 0x7f7f7c1f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.436+0000 7f7f808de700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f7c14db90 0x7f7f7c0b1060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.436+0000 7f7f808de700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7f7c151c80 0x7f7f7c0b1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.436+0000 7f7f808de700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c158af0 con 0x7f7f64005200 2023-11-28T11:13:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.436+0000 7f7f808de700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c1f5840 con 0x7f7f7c14db90 2023-11-28T11:13:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.436+0000 7f7f808de700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7c152660 con 0x7f7f7c151c80 2023-11-28T11:13:27.446 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.436+0000 7f7f7a59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f64005200 0x7f7f7c1f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.436+0000 7f7f7a59c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f64005200 0x7f7f7c1f6290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52632/0 (socket says 172.21.15.28:52632) 2023-11-28T11:13:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.436+0000 7f7f7a59c700 1 -- 172.21.15.28:0/238707754 learned_addr learned my addr 172.21.15.28:0/238707754 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.436+0000 7f7f7ad9d700 1 --2- 172.21.15.28:0/238707754 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7f7c151c80 0x7f7f7c0b1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.436+0000 7f7f79d9b700 1 --2- 172.21.15.28:0/238707754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f7c14db90 0x7f7f7c0b1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.436+0000 7f7f7a59c700 1 -- 172.21.15.28:0/238707754 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7f7c151c80 msgr2=0x7f7f7c0b1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.436+0000 7f7f7a59c700 1 --2- 172.21.15.28:0/238707754 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f7f7c151c80 0x7f7f7c0b1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.447 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.436+0000 7f7f7a59c700 1 -- 172.21.15.28:0/238707754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f7c14db90 msgr2=0x7f7f7c0b1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.436+0000 7f7f7a59c700 1 --2- 172.21.15.28:0/238707754 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7f7c14db90 0x7f7f7c0b1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.437+0000 7f7f7a59c700 1 -- 172.21.15.28:0/238707754 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f7c0b5520 con 0x7f7f64005200 2023-11-28T11:13:27.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.437+0000 7f7f7a59c700 1 --2- 172.21.15.28:0/238707754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f64005200 0x7f7f7c1f6290 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0119a0 tx=0x7f7f6c011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:27.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.437+0000 7f7f6affd700 1 -- 172.21.15.28:0/238707754 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00f780 con 0x7f7f64005200 2023-11-28T11:13:27.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.437+0000 7f7f6affd700 1 -- 172.21.15.28:0/238707754 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f6c00f8e0 con 0x7f7f64005200 2023-11-28T11:13:27.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.437+0000 7f7f808de700 1 -- 172.21.15.28:0/238707754 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f7c0b6110 con 0x7f7f64005200 2023-11-28T11:13:27.448 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.437+0000 7f7f6affd700 1 -- 172.21.15.28:0/238707754 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f6c00fc70 con 0x7f7f64005200 2023-11-28T11:13:27.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.437+0000 7f7f808de700 1 -- 172.21.15.28:0/238707754 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f7c0cf2c0 con 0x7f7f64005200 2023-11-28T11:13:27.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.438+0000 7f7f808de700 1 -- 172.21.15.28:0/238707754 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7f7c0739f0 con 0x7f7f64005200 2023-11-28T11:13:27.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.438+0000 7f7f6affd700 1 -- 172.21.15.28:0/238707754 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f7f6c014440 con 0x7f7f64005200 2023-11-28T11:13:27.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.439+0000 7f7f6affd700 1 --2- 172.21.15.28:0/238707754 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7f5c0682f0 0x7f7f5c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.439+0000 7f7f6affd700 1 -- 172.21.15.28:0/238707754 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f7f6c0861f0 con 0x7f7f64005200 2023-11-28T11:13:27.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.439+0000 7f7f79d9b700 1 --2- 172.21.15.28:0/238707754 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7f5c0682f0 0x7f7f5c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.439+0000 7f7f6affd700 1 --2- 172.21.15.28:0/238707754 >> [v2:172.21.15.28:6808/103510,v1:172.21.15.28:6809/103510] conn(0x7f7f5c06eb90 0x7f7f5c070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.449 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.439+0000 7f7f6affd700 1 -- 172.21.15.28:0/238707754 --> [v2:172.21.15.28:6808/103510,v1:172.21.15.28:6809/103510] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7f5c0716b0 con 0x7f7f5c06eb90 2023-11-28T11:13:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.439+0000 7f7f6affd700 1 -- 172.21.15.28:0/238707754 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f6c0888a0 con 0x7f7f64005200 2023-11-28T11:13:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.439+0000 7f7f7ad9d700 1 --2- 172.21.15.28:0/238707754 >> [v2:172.21.15.28:6808/103510,v1:172.21.15.28:6809/103510] conn(0x7f7f5c06eb90 0x7f7f5c070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.439+0000 7f7f79d9b700 1 --2- 172.21.15.28:0/238707754 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7f5c0682f0 0x7f7f5c06a7a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c14df60 tx=0x7f7f70001e00 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.439+0000 7f7f7ad9d700 1 --2- 172.21.15.28:0/238707754 >> [v2:172.21.15.28:6808/103510,v1:172.21.15.28:6809/103510] conn(0x7f7f5c06eb90 0x7f7f5c070fe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:27.450 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.440+0000 7f7f6affd700 1 -- 172.21.15.28:0/238707754 <== osd.3 v2:172.21.15.28:6808/103510 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7f5c0716b0 con 0x7f7f5c06eb90 2023-11-28T11:13:27.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.454+0000 7f7f808de700 1 -- 172.21.15.28:0/238707754 --> [v2:172.21.15.28:6808/103510,v1:172.21.15.28:6809/103510] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7f7c0d0180 con 0x7f7f5c06eb90 2023-11-28T11:13:27.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.454+0000 7f7f6affd700 1 -- 172.21.15.28:0/238707754 <== osd.3 v2:172.21.15.28:6808/103510 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7f7c0d0180 con 0x7f7f5c06eb90 2023-11-28T11:13:27.456 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:13:27.456 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:13:27.456 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:13:27.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.455+0000 7f7f808de700 1 -- 172.21.15.28:0/238707754 >> [v2:172.21.15.28:6808/103510,v1:172.21.15.28:6809/103510] conn(0x7f7f5c06eb90 msgr2=0x7f7f5c070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.455+0000 7f7f808de700 1 --2- 172.21.15.28:0/238707754 >> [v2:172.21.15.28:6808/103510,v1:172.21.15.28:6809/103510] conn(0x7f7f5c06eb90 0x7f7f5c070fe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.455+0000 7f7f808de700 1 -- 172.21.15.28:0/238707754 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7f5c0682f0 msgr2=0x7f7f5c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.455+0000 7f7f808de700 1 --2- 172.21.15.28:0/238707754 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f7f5c0682f0 0x7f7f5c06a7a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7f7c14df60 tx=0x7f7f70001e00 comp rx=0 tx=0).stop 2023-11-28T11:13:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.455+0000 7f7f808de700 1 -- 172.21.15.28:0/238707754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f64005200 msgr2=0x7f7f7c1f6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.455+0000 7f7f808de700 1 --2- 172.21.15.28:0/238707754 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f7f64005200 0x7f7f7c1f6290 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f7f6c0119a0 tx=0x7f7f6c011cb0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.455+0000 7f7f7ad9d700 1 -- 172.21.15.28:0/238707754 reap_dead start 2023-11-28T11:13:27.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.455+0000 7f7f808de700 1 -- 172.21.15.28:0/238707754 shutdown_connections 2023-11-28T11:13:27.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.455+0000 7f7f808de700 1 -- 172.21.15.28:0/238707754 >> 172.21.15.28:0/238707754 conn(0x7f7f7c0d0ae0 msgr2=0x7f7f7c0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:27.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.455+0000 7f7f808de700 1 -- 172.21.15.28:0/238707754 shutdown_connections 2023-11-28T11:13:27.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.455+0000 7f7f808de700 1 -- 172.21.15.28:0/238707754 wait complete. 2023-11-28T11:13:27.468 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-28T11:13:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.592+0000 7f515ed24700 1 Processor -- start 2023-11-28T11:13:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.592+0000 7f515ed24700 1 -- start start 2023-11-28T11:13:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.592+0000 7f515ed24700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f515815c990 0x7f51580cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.592+0000 7f515ed24700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f51580cc310 0x7f51580cc7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.592+0000 7f515ed24700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51580ce340 con 0x7f5158161090 2023-11-28T11:13:27.601 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.592+0000 7f515ed24700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51580cebe0 con 0x7f51580cc310 2023-11-28T11:13:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.592+0000 7f515ed24700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51580cccf0 con 0x7f515815c990 2023-11-28T11:13:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.593+0000 7f515cac0700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f515815c990 0x7f51580cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.593+0000 7f515d2c1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f51580cc310 0x7f51580cc7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.593+0000 7f5157fff700 1 --1- >> v1:172.21.15.59:6789/0 conn(0x7f5158161090 0x7f51580ca970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.59:6789/0 says I am v1:172.21.15.28:51486/0 (socket says 172.21.15.28:51486) 2023-11-28T11:13:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.593+0000 7f5157fff700 1 -- 172.21.15.28:0/593444222 learned_addr learned my addr 172.21.15.28:0/593444222 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.593+0000 7f515d2c1700 1 -- 172.21.15.28:0/593444222 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f515815c990 msgr2=0x7f51580cf070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.593+0000 7f515d2c1700 1 --2- 172.21.15.28:0/593444222 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f515815c990 0x7f51580cf070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.602 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.593+0000 7f515d2c1700 1 -- 172.21.15.28:0/593444222 >> v1:172.21.15.59:6789/0 conn(0x7f5158161090 legacy=0x7f51580ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.593+0000 7f515d2c1700 1 -- 172.21.15.28:0/593444222 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51580cb3f0 con 0x7f51580cc310 2023-11-28T11:13:27.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.593+0000 7f51577fe700 1 -- 172.21.15.28:0/593444222 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1584422878 0 0) 0x7f51580ce340 con 0x7f5158161090 2023-11-28T11:13:27.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.593+0000 7f515cac0700 1 --2- 172.21.15.28:0/593444222 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f515815c990 0x7f51580cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:13:27.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.593+0000 7f515d2c1700 1 --2- 172.21.15.28:0/593444222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51580cc310 0x7f51580cc7b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5150009d90 tx=0x7f515000b330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=704db9af3c7df506 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:27.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.594+0000 7f51577fe700 1 -- 172.21.15.28:0/593444222 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515000e070 con 0x7f51580cc310 2023-11-28T11:13:27.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.594+0000 7f51577fe700 1 -- 172.21.15.28:0/593444222 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515000bb20 con 0x7f51580cc310 2023-11-28T11:13:27.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.594+0000 7f51577fe700 1 -- 172.21.15.28:0/593444222 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5150012430 con 0x7f51580cc310 2023-11-28T11:13:27.603 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.594+0000 7f515ed24700 1 -- 172.21.15.28:0/593444222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51580cc310 msgr2=0x7f51580cc7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.604 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.594+0000 7f515ed24700 1 --2- 172.21.15.28:0/593444222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51580cc310 0x7f51580cc7b0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f5150009d90 tx=0x7f515000b330 comp rx=0 tx=0).stop 2023-11-28T11:13:27.604 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.594+0000 7f515ed24700 1 -- 172.21.15.28:0/593444222 shutdown_connections 2023-11-28T11:13:27.604 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.594+0000 7f515ed24700 1 --2- 172.21.15.28:0/593444222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51580cc310 0x7f51580cc7b0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.604 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.594+0000 7f515ed24700 1 --2- 172.21.15.28:0/593444222 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f515815c990 0x7f51580cf070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.604 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.594+0000 7f515ed24700 1 -- 172.21.15.28:0/593444222 >> 172.21.15.28:0/593444222 conn(0x7f51580b1650 msgr2=0x7f51580b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:27.604 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.594+0000 7f515ed24700 1 -- 172.21.15.28:0/593444222 shutdown_connections 2023-11-28T11:13:27.604 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.594+0000 7f515ed24700 1 -- 172.21.15.28:0/593444222 wait complete. 2023-11-28T11:13:27.604 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.595+0000 7f515ed24700 1 Processor -- start 2023-11-28T11:13:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.595+0000 7f515ed24700 1 -- start start 2023-11-28T11:13:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.596+0000 7f515ed24700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f51580cc310 0x7f51580b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.596+0000 7f515ed24700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f515815c990 0x7f515806b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.596+0000 7f515ed24700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5158161090 0x7f515806b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.596+0000 7f515ed24700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515814abe0 con 0x7f515815c990 2023-11-28T11:13:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.596+0000 7f515ed24700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f515814d1e0 con 0x7f5158161090 2023-11-28T11:13:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.596+0000 7f515ed24700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f515814d4c0 con 0x7f51580cc310 2023-11-28T11:13:27.605 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.596+0000 7f5157fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f515815c990 0x7f515806b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.596+0000 7f5157fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f515815c990 0x7f515806b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52650/0 (socket says 172.21.15.28:52650) 2023-11-28T11:13:27.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.596+0000 7f5157fff700 1 -- 172.21.15.28:0/1746946819 learned_addr learned my addr 172.21.15.28:0/1746946819 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:27.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.597+0000 7f515cac0700 1 --2- 172.21.15.28:0/1746946819 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f51580cc310 0x7f51580b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.597+0000 7f515d2c1700 1 --2- 172.21.15.28:0/1746946819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5158161090 0x7f515806b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.597+0000 7f5157fff700 1 -- 172.21.15.28:0/1746946819 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f51580cc310 msgr2=0x7f51580b3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.597+0000 7f5157fff700 1 --2- 172.21.15.28:0/1746946819 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f51580cc310 0x7f51580b3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.597+0000 7f5157fff700 1 -- 172.21.15.28:0/1746946819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5158161090 msgr2=0x7f515806b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.606 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.597+0000 7f5157fff700 1 --2- 172.21.15.28:0/1746946819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5158161090 0x7f515806b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.597+0000 7f5157fff700 1 -- 172.21.15.28:0/1746946819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f514c0094a0 con 0x7f515815c990 2023-11-28T11:13:27.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.597+0000 7f5157fff700 1 --2- 172.21.15.28:0/1746946819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f515815c990 0x7f515806b3c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f514400c540 tx=0x7f514400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:27.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.597+0000 7f51557fa700 1 -- 172.21.15.28:0/1746946819 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144004c80 con 0x7f515815c990 2023-11-28T11:13:27.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.597+0000 7f51557fa700 1 -- 172.21.15.28:0/1746946819 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5144004de0 con 0x7f515815c990 2023-11-28T11:13:27.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.597+0000 7f51557fa700 1 -- 172.21.15.28:0/1746946819 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5144007310 con 0x7f515815c990 2023-11-28T11:13:27.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.597+0000 7f515ed24700 1 -- 172.21.15.28:0/1746946819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5150009a70 con 0x7f515815c990 2023-11-28T11:13:27.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.597+0000 7f515ed24700 1 -- 172.21.15.28:0/1746946819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51580ce870 con 0x7f515815c990 2023-11-28T11:13:27.607 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.598+0000 7f515ed24700 1 -- 172.21.15.28:0/1746946819 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51580ce590 con 0x7f515815c990 2023-11-28T11:13:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.599+0000 7f51557fa700 1 -- 172.21.15.28:0/1746946819 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f514401a020 con 0x7f515815c990 2023-11-28T11:13:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.599+0000 7f51557fa700 1 --2- 172.21.15.28:0/1746946819 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f513c068320 0x7f513c06a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.599+0000 7f51557fa700 1 -- 172.21.15.28:0/1746946819 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f51440855d0 con 0x7f515815c990 2023-11-28T11:13:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.599+0000 7f515cac0700 1 --2- 172.21.15.28:0/1746946819 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f513c068320 0x7f513c06a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.608 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.600+0000 7f515cac0700 1 --2- 172.21.15.28:0/1746946819 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f513c068320 0x7f513c06a7d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f514c009d70 tx=0x7f514c009210 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:27.609 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.604+0000 7f51557fa700 1 -- 172.21.15.28:0/1746946819 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f5144051f60 con 0x7f515815c990 2023-11-28T11:13:27.799 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.798+0000 7f515ed24700 1 -- 172.21.15.28:0/1746946819 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f51580cb080 con 0x7f513c068320 2023-11-28T11:13:27.800 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.798+0000 7f51557fa700 1 -- 172.21.15.28:0/1746946819 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f51580cb080 con 0x7f513c068320 2023-11-28T11:13:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.800+0000 7f515ed24700 1 -- 172.21.15.28:0/1746946819 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f513c068320 msgr2=0x7f513c06a7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.800+0000 7f515ed24700 1 --2- 172.21.15.28:0/1746946819 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f513c068320 0x7f513c06a7d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f514c009d70 tx=0x7f514c009210 comp rx=0 tx=0).stop 2023-11-28T11:13:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.800+0000 7f515ed24700 1 -- 172.21.15.28:0/1746946819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f515815c990 msgr2=0x7f515806b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.800+0000 7f515ed24700 1 --2- 172.21.15.28:0/1746946819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f515815c990 0x7f515806b3c0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f514400c540 tx=0x7f514400c850 comp rx=0 tx=0).stop 2023-11-28T11:13:27.802 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.800+0000 7f515ed24700 1 -- 172.21.15.28:0/1746946819 shutdown_connections 2023-11-28T11:13:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.800+0000 7f515ed24700 1 --2- 172.21.15.28:0/1746946819 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f513c068320 0x7f513c06a7d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.800+0000 7f515ed24700 1 --2- 172.21.15.28:0/1746946819 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5158161090 0x7f515806b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.800+0000 7f515ed24700 1 --2- 172.21.15.28:0/1746946819 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f51580cc310 0x7f51580b3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.800+0000 7f515ed24700 1 --2- 172.21.15.28:0/1746946819 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f515815c990 0x7f515806b3c0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.800+0000 7f515ed24700 1 -- 172.21.15.28:0/1746946819 >> 172.21.15.28:0/1746946819 conn(0x7f51580b1650 msgr2=0x7f51580cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:27.803 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.800+0000 7f515ed24700 1 -- 172.21.15.28:0/1746946819 shutdown_connections 2023-11-28T11:13:27.804 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.800+0000 7f515ed24700 1 -- 172.21.15.28:0/1746946819 wait complete. 2023-11-28T11:13:27.804 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 3 to deep-scrub 2023-11-28T11:13:27.814 INFO:tasks.ceph:Scrubbing osd.4 2023-11-28T11:13:27.814 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:13:27.942 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.941+0000 7f5995d6d700 1 Processor -- start 2023-11-28T11:13:27.943 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.941+0000 7f5995d6d700 1 -- start start 2023-11-28T11:13:27.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f5995d6d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f599015a5a0 0x7f599006b930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f5995d6d700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59900cb570 con 0x7f59900ca280 2023-11-28T11:13:27.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f5995d6d700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59900ca6f0 con 0x7f59900cc4b0 2023-11-28T11:13:27.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f5995d6d700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59900cc110 con 0x7f599015a5a0 2023-11-28T11:13:27.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f598f7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f599015a5a0 0x7f599006b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f598effd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f59900ca280 0x7f59900cba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40716/0 (socket says 172.21.15.28:40716) 2023-11-28T11:13:27.946 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f598f7fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f599015a5a0 0x7f599006b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52656/0 (socket says 172.21.15.28:52656) 2023-11-28T11:13:27.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f598effd700 1 -- 172.21.15.28:0/2826780637 learned_addr learned my addr 172.21.15.28:0/2826780637 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:27.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f598f7fe700 1 -- 172.21.15.28:0/2826780637 >> v1:172.21.15.59:6789/0 conn(0x7f59900cc4b0 legacy=0x7f59900c35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f598f7fe700 1 -- 172.21.15.28:0/2826780637 >> v1:172.21.15.28:6789/0 conn(0x7f59900ca280 legacy=0x7f59900cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f598e7fc700 1 -- 172.21.15.28:0/2826780637 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378113547 0 0) 0x7f59900cb570 con 0x7f59900ca280 2023-11-28T11:13:27.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f598f7fe700 1 -- 172.21.15.28:0/2826780637 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59900ca960 con 0x7f599015a5a0 2023-11-28T11:13:27.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f598f7fe700 1 --2- 172.21.15.28:0/2826780637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f599015a5a0 0x7f599006b930 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5980009840 tx=0x7f5980009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec175319ae47d62d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:27.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f598e7fc700 1 -- 172.21.15.28:0/2826780637 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980011830 con 0x7f599015a5a0 2023-11-28T11:13:27.947 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f598e7fc700 1 -- 172.21.15.28:0/2826780637 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5980011990 con 0x7f599015a5a0 2023-11-28T11:13:27.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.942+0000 7f598e7fc700 1 -- 172.21.15.28:0/2826780637 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5980011d20 con 0x7f599015a5a0 2023-11-28T11:13:27.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.943+0000 7f5995d6d700 1 -- 172.21.15.28:0/2826780637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f599015a5a0 msgr2=0x7f599006b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.943+0000 7f5995d6d700 1 --2- 172.21.15.28:0/2826780637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f599015a5a0 0x7f599006b930 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f5980009840 tx=0x7f5980009b50 comp rx=0 tx=0).stop 2023-11-28T11:13:27.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.943+0000 7f5995d6d700 1 -- 172.21.15.28:0/2826780637 shutdown_connections 2023-11-28T11:13:27.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.943+0000 7f5995d6d700 1 --2- 172.21.15.28:0/2826780637 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f599015a5a0 0x7f599006b930 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.943+0000 7f5995d6d700 1 -- 172.21.15.28:0/2826780637 >> 172.21.15.28:0/2826780637 conn(0x7f59900b1650 msgr2=0x7f59900b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:27.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.943+0000 7f5995d6d700 1 -- 172.21.15.28:0/2826780637 shutdown_connections 2023-11-28T11:13:27.948 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.943+0000 7f5995d6d700 1 -- 172.21.15.28:0/2826780637 wait complete. 2023-11-28T11:13:27.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.944+0000 7f5995d6d700 1 Processor -- start 2023-11-28T11:13:27.949 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.944+0000 7f5995d6d700 1 -- start start 2023-11-28T11:13:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.945+0000 7f5995d6d700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59900ca280 0x7f59900cf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.945+0000 7f5995d6d700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59900cc4b0 0x7f599014d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.945+0000 7f5995d6d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f599015a5a0 0x7f59901e3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.945+0000 7f5995d6d700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59900cd670 con 0x7f59900cc4b0 2023-11-28T11:13:27.952 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.945+0000 7f5995d6d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f599014d700 con 0x7f599015a5a0 2023-11-28T11:13:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.945+0000 7f5995d6d700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59900b1250 con 0x7f59900ca280 2023-11-28T11:13:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.945+0000 7f598effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59900cc4b0 0x7f599014d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.945+0000 7f598effd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59900cc4b0 0x7f599014d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52664/0 (socket says 172.21.15.28:52664) 2023-11-28T11:13:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.945+0000 7f598effd700 1 -- 172.21.15.28:0/3042107631 learned_addr learned my addr 172.21.15.28:0/3042107631 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.945+0000 7f598ffff700 1 --2- 172.21.15.28:0/3042107631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f599015a5a0 0x7f59901e3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.945+0000 7f598f7fe700 1 --2- 172.21.15.28:0/3042107631 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59900ca280 0x7f59900cf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.946+0000 7f598effd700 1 -- 172.21.15.28:0/3042107631 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59900ca280 msgr2=0x7f59900cf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.953 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.946+0000 7f598effd700 1 --2- 172.21.15.28:0/3042107631 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59900ca280 0x7f59900cf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.946+0000 7f598effd700 1 -- 172.21.15.28:0/3042107631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f599015a5a0 msgr2=0x7f59901e3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.946+0000 7f598effd700 1 --2- 172.21.15.28:0/3042107631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f599015a5a0 0x7f59901e3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.946+0000 7f598effd700 1 -- 172.21.15.28:0/3042107631 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59800094a0 con 0x7f59900cc4b0 2023-11-28T11:13:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.946+0000 7f598f7fe700 1 --2- 172.21.15.28:0/3042107631 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f59900ca280 0x7f59900cf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.946+0000 7f598effd700 1 --2- 172.21.15.28:0/3042107631 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59900cc4b0 0x7f599014d1c0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f597800c520 tx=0x7f597800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.946+0000 7f5987fff700 1 -- 172.21.15.28:0/3042107631 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5978004c80 con 0x7f59900cc4b0 2023-11-28T11:13:27.954 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.946+0000 7f5987fff700 1 -- 172.21.15.28:0/3042107631 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5978004de0 con 0x7f59900cc4b0 2023-11-28T11:13:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.946+0000 7f5995d6d700 1 -- 172.21.15.28:0/3042107631 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59900caf20 con 0x7f59900cc4b0 2023-11-28T11:13:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.946+0000 7f5987fff700 1 -- 172.21.15.28:0/3042107631 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5978007310 con 0x7f59900cc4b0 2023-11-28T11:13:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.946+0000 7f5995d6d700 1 -- 172.21.15.28:0/3042107631 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59900cac40 con 0x7f59900cc4b0 2023-11-28T11:13:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.947+0000 7f5995d6d700 1 -- 172.21.15.28:0/3042107631 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f59900cb200 con 0x7f59900cc4b0 2023-11-28T11:13:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.947+0000 7f5987fff700 1 -- 172.21.15.28:0/3042107631 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f597801a020 con 0x7f59900cc4b0 2023-11-28T11:13:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.948+0000 7f5987fff700 1 --2- 172.21.15.28:0/3042107631 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f59700682f0 0x7f597006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.948+0000 7f5987fff700 1 -- 172.21.15.28:0/3042107631 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f5978085640 con 0x7f59900cc4b0 2023-11-28T11:13:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.948+0000 7f598f7fe700 1 --2- 172.21.15.28:0/3042107631 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f59700682f0 0x7f597006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.955 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.948+0000 7f5987fff700 1 --2- 172.21.15.28:0/3042107631 >> [v2:172.21.15.59:6801/103695,v1:172.21.15.59:6803/103695] conn(0x7f597006eb90 0x7f5970070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:27.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.948+0000 7f5987fff700 1 -- 172.21.15.28:0/3042107631 --> [v2:172.21.15.59:6801/103695,v1:172.21.15.59:6803/103695] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59700716b0 con 0x7f597006eb90 2023-11-28T11:13:27.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.948+0000 7f5987fff700 1 -- 172.21.15.28:0/3042107631 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5978087cf0 con 0x7f59900cc4b0 2023-11-28T11:13:27.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.948+0000 7f598ffff700 1 --2- 172.21.15.28:0/3042107631 >> [v2:172.21.15.59:6801/103695,v1:172.21.15.59:6803/103695] conn(0x7f597006eb90 0x7f5970070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:27.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.948+0000 7f598f7fe700 1 --2- 172.21.15.28:0/3042107631 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f59700682f0 0x7f597006a7a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5980000c00 tx=0x7f598001c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:27.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.949+0000 7f598ffff700 1 --2- 172.21.15.28:0/3042107631 >> [v2:172.21.15.59:6801/103695,v1:172.21.15.59:6803/103695] conn(0x7f597006eb90 0x7f5970070fe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:27.956 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.950+0000 7f5987fff700 1 -- 172.21.15.28:0/3042107631 <== osd.4 v2:172.21.15.59:6801/103695 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f59700716b0 con 0x7f597006eb90 2023-11-28T11:13:27.965 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.964+0000 7f5995d6d700 1 -- 172.21.15.28:0/3042107631 --> [v2:172.21.15.59:6801/103695,v1:172.21.15.59:6803/103695] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f59900739f0 con 0x7f597006eb90 2023-11-28T11:13:27.966 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.964+0000 7f5987fff700 1 -- 172.21.15.28:0/3042107631 <== osd.4 v2:172.21.15.59:6801/103695 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f59900739f0 con 0x7f597006eb90 2023-11-28T11:13:27.966 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:13:27.966 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:13:27.967 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:13:27.967 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.965+0000 7f5995d6d700 1 -- 172.21.15.28:0/3042107631 >> [v2:172.21.15.59:6801/103695,v1:172.21.15.59:6803/103695] conn(0x7f597006eb90 msgr2=0x7f5970070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.965+0000 7f5995d6d700 1 --2- 172.21.15.28:0/3042107631 >> [v2:172.21.15.59:6801/103695,v1:172.21.15.59:6803/103695] conn(0x7f597006eb90 0x7f5970070fe0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:27.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.965+0000 7f5995d6d700 1 -- 172.21.15.28:0/3042107631 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f59700682f0 msgr2=0x7f597006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.965+0000 7f5995d6d700 1 --2- 172.21.15.28:0/3042107631 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f59700682f0 0x7f597006a7a0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f5980000c00 tx=0x7f598001c040 comp rx=0 tx=0).stop 2023-11-28T11:13:27.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.965+0000 7f5995d6d700 1 -- 172.21.15.28:0/3042107631 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59900cc4b0 msgr2=0x7f599014d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:27.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.965+0000 7f5995d6d700 1 --2- 172.21.15.28:0/3042107631 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f59900cc4b0 0x7f599014d1c0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f597800c520 tx=0x7f597800c830 comp rx=0 tx=0).stop 2023-11-28T11:13:27.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.965+0000 7f598ffff700 1 -- 172.21.15.28:0/3042107631 reap_dead start 2023-11-28T11:13:27.968 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.965+0000 7f5995d6d700 1 -- 172.21.15.28:0/3042107631 shutdown_connections 2023-11-28T11:13:27.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.965+0000 7f5995d6d700 1 -- 172.21.15.28:0/3042107631 >> 172.21.15.28:0/3042107631 conn(0x7f59900b1650 msgr2=0x7f599015aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:27.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.965+0000 7f5995d6d700 1 -- 172.21.15.28:0/3042107631 shutdown_connections 2023-11-28T11:13:27.969 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:27.965+0000 7f5995d6d700 1 -- 172.21.15.28:0/3042107631 wait complete. 2023-11-28T11:13:27.978 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-28T11:13:28.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.102+0000 7f80f9873700 1 Processor -- start 2023-11-28T11:13:28.104 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.102+0000 7f80f9873700 1 -- start start 2023-11-28T11:13:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.103+0000 7f80f9873700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80f415c640 0x7f80f40cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.103+0000 7f80f9873700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f40b6350 con 0x7f80f406bf30 2023-11-28T11:13:28.108 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.103+0000 7f80f9873700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80f40d1470 con 0x7f80f40d2820 2023-11-28T11:13:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.103+0000 7f80f9873700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f415ca10 con 0x7f80f415c640 2023-11-28T11:13:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.103+0000 7f80f37fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f80f406bf30 0x7f80f406c350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40732/0 (socket says 172.21.15.28:40732) 2023-11-28T11:13:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.103+0000 7f80f37fe700 1 -- 172.21.15.28:0/958383887 learned_addr learned my addr 172.21.15.28:0/958383887 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.103+0000 7f80f2ffd700 1 --2- 172.21.15.28:0/958383887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80f415c640 0x7f80f40cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.103+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870996206 0 0) 0x7f80f40b6350 con 0x7f80f406bf30 2023-11-28T11:13:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.103+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80dc0035b0 con 0x7f80f406bf30 2023-11-28T11:13:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2115678423 0 0) 0x7f80f40d1470 con 0x7f80f40d2820 2023-11-28T11:13:28.109 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f80f40b6350 con 0x7f80f40d2820 2023-11-28T11:13:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1413778095 0 0) 0x7f80dc0035b0 con 0x7f80f406bf30 2023-11-28T11:13:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80f415c640 msgr2=0x7f80f40cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 --2- 172.21.15.28:0/958383887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80f415c640 0x7f80f40cf040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 >> v1:172.21.15.59:6789/0 conn(0x7f80f40d2820 legacy=0x7f80f40b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f40cd4e0 con 0x7f80f406bf30 2023-11-28T11:13:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f80ec0019f0 con 0x7f80f406bf30 2023-11-28T11:13:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f2ffd700 1 --2- 172.21.15.28:0/958383887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80f415c640 0x7f80f40cf040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:13:28.110 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 >> v1:172.21.15.28:6789/0 conn(0x7f80f406bf30 legacy=0x7f80f406c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 --2- 172.21.15.28:0/958383887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80dc003bc0 0x7f80dc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 --2- 172.21.15.28:0/958383887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80dc004850 0x7f80dc004cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 --2- 172.21.15.28:0/958383887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80f415c640 0x7f80dc005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f415ca10 con 0x7f80dc004850 2023-11-28T11:13:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc00bac0 con 0x7f80dc003bc0 2023-11-28T11:13:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80dc00bc50 con 0x7f80f415c640 2023-11-28T11:13:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f27fc700 1 --2- 172.21.15.28:0/958383887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80dc004850 0x7f80dc004cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.111 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f2ffd700 1 --2- 172.21.15.28:0/958383887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80dc003bc0 0x7f80dc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f37fe700 1 --2- 172.21.15.28:0/958383887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80f415c640 0x7f80dc005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f27fc700 1 -- 172.21.15.28:0/958383887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80f415c640 msgr2=0x7f80dc005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f27fc700 1 --2- 172.21.15.28:0/958383887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80f415c640 0x7f80dc005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f27fc700 1 -- 172.21.15.28:0/958383887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80dc003bc0 msgr2=0x7f80dc003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f27fc700 1 --2- 172.21.15.28:0/958383887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80dc003bc0 0x7f80dc003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f27fc700 1 -- 172.21.15.28:0/958383887 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80e80094a0 con 0x7f80dc004850 2023-11-28T11:13:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f37fe700 1 --2- 172.21.15.28:0/958383887 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80f415c640 0x7f80dc005200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:28.112 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f27fc700 1 --2- 172.21.15.28:0/958383887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80dc004850 0x7f80dc004cc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f80e4001970 tx=0x7f80dc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be74386806789842 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4001430 con 0x7f80dc004850 2023-11-28T11:13:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e40052d0 con 0x7f80dc004850 2023-11-28T11:13:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.104+0000 7f80f1ffb700 1 -- 172.21.15.28:0/958383887 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e4003680 con 0x7f80dc004850 2023-11-28T11:13:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.105+0000 7f80f9873700 1 -- 172.21.15.28:0/958383887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80dc004850 msgr2=0x7f80dc004cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.105+0000 7f80f9873700 1 --2- 172.21.15.28:0/958383887 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80dc004850 0x7f80dc004cc0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f80e4001970 tx=0x7f80dc005880 comp rx=0 tx=0).stop 2023-11-28T11:13:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.105+0000 7f80f37fe700 1 -- 172.21.15.28:0/958383887 reap_dead start 2023-11-28T11:13:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.105+0000 7f80f9873700 1 -- 172.21.15.28:0/958383887 shutdown_connections 2023-11-28T11:13:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.105+0000 7f80f9873700 1 -- 172.21.15.28:0/958383887 >> 172.21.15.28:0/958383887 conn(0x7f80f40d2df0 msgr2=0x7f80f40d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:28.113 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.105+0000 7f80f9873700 1 -- 172.21.15.28:0/958383887 shutdown_connections 2023-11-28T11:13:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.105+0000 7f80f9873700 1 -- 172.21.15.28:0/958383887 wait complete. 2023-11-28T11:13:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.106+0000 7f80f9873700 1 Processor -- start 2023-11-28T11:13:28.114 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.106+0000 7f80f9873700 1 -- start start 2023-11-28T11:13:28.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.107+0000 7f80f9873700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80dc004850 0x7f80f4158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.107+0000 7f80f9873700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80f40c9ec0 0x7f80f4158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.107+0000 7f80f9873700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f40c8d60 0x7f80f40c91d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.116 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.107+0000 7f80f9873700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f40c9710 con 0x7f80dc004850 2023-11-28T11:13:28.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.107+0000 7f80f9873700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80f41593a0 con 0x7f80f40c8d60 2023-11-28T11:13:28.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.107+0000 7f80f9873700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80f4158b80 con 0x7f80f40c9ec0 2023-11-28T11:13:28.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.107+0000 7f80f2ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80dc004850 0x7f80f4158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.107+0000 7f80f2ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80dc004850 0x7f80f4158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52682/0 (socket says 172.21.15.28:52682) 2023-11-28T11:13:28.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.107+0000 7f80f2ffd700 1 -- 172.21.15.28:0/979998480 learned_addr learned my addr 172.21.15.28:0/979998480 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:28.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.108+0000 7f80f2ffd700 1 -- 172.21.15.28:0/979998480 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80f40c9ec0 msgr2=0x7f80f4158330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.108+0000 7f80f2ffd700 1 --2- 172.21.15.28:0/979998480 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80f40c9ec0 0x7f80f4158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.117 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.108+0000 7f80f37fe700 1 --2- 172.21.15.28:0/979998480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f40c8d60 0x7f80f40c91d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.108+0000 7f80f2ffd700 1 -- 172.21.15.28:0/979998480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f40c8d60 msgr2=0x7f80f40c91d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.108+0000 7f80f2ffd700 1 --2- 172.21.15.28:0/979998480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f40c8d60 0x7f80f40c91d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.108+0000 7f80f2ffd700 1 -- 172.21.15.28:0/979998480 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80f40d1710 con 0x7f80dc004850 2023-11-28T11:13:28.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.108+0000 7f80f2ffd700 1 --2- 172.21.15.28:0/979998480 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80dc004850 0x7f80f4158e60 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f80e8011cd0 tx=0x7f80e800f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:28.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.108+0000 7f80dbfff700 1 -- 172.21.15.28:0/979998480 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e8011e70 con 0x7f80dc004850 2023-11-28T11:13:28.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.108+0000 7f80dbfff700 1 -- 172.21.15.28:0/979998480 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80e800fc60 con 0x7f80dc004850 2023-11-28T11:13:28.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.108+0000 7f80f9873700 1 -- 172.21.15.28:0/979998480 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80f40cd7c0 con 0x7f80dc004850 2023-11-28T11:13:28.118 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.108+0000 7f80dbfff700 1 -- 172.21.15.28:0/979998480 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80e8004020 con 0x7f80dc004850 2023-11-28T11:13:28.119 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.108+0000 7f80f9873700 1 -- 172.21.15.28:0/979998480 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80f40cdaa0 con 0x7f80dc004850 2023-11-28T11:13:28.119 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.109+0000 7f80f9873700 1 -- 172.21.15.28:0/979998480 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80f4068570 con 0x7f80dc004850 2023-11-28T11:13:28.119 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.109+0000 7f80dbfff700 1 -- 172.21.15.28:0/979998480 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f80e8024070 con 0x7f80dc004850 2023-11-28T11:13:28.119 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.110+0000 7f80dbfff700 1 --2- 172.21.15.28:0/979998480 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80e00682f0 0x7f80e006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.119 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.110+0000 7f80dbfff700 1 -- 172.21.15.28:0/979998480 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f80e8085f00 con 0x7f80dc004850 2023-11-28T11:13:28.119 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.110+0000 7f80f27fc700 1 --2- 172.21.15.28:0/979998480 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80e00682f0 0x7f80e006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.119 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.110+0000 7f80f27fc700 1 --2- 172.21.15.28:0/979998480 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80e00682f0 0x7f80e006a7a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f80f40c6f60 tx=0x7f80e4009030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:28.119 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.114+0000 7f80dbfff700 1 -- 172.21.15.28:0/979998480 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f80e8052920 con 0x7f80dc004850 2023-11-28T11:13:28.308 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.307+0000 7f80f9873700 1 -- 172.21.15.28:0/979998480 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f80f406a840 con 0x7f80e00682f0 2023-11-28T11:13:28.309 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.307+0000 7f80dbfff700 1 -- 172.21.15.28:0/979998480 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f80f406a840 con 0x7f80e00682f0 2023-11-28T11:13:28.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.309+0000 7f80f9873700 1 -- 172.21.15.28:0/979998480 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80e00682f0 msgr2=0x7f80e006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.309+0000 7f80f9873700 1 --2- 172.21.15.28:0/979998480 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80e00682f0 0x7f80e006a7a0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f80f40c6f60 tx=0x7f80e4009030 comp rx=0 tx=0).stop 2023-11-28T11:13:28.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.309+0000 7f80f9873700 1 -- 172.21.15.28:0/979998480 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80dc004850 msgr2=0x7f80f4158e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.311 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.309+0000 7f80f9873700 1 --2- 172.21.15.28:0/979998480 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80dc004850 0x7f80f4158e60 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f80e8011cd0 tx=0x7f80e800f520 comp rx=0 tx=0).stop 2023-11-28T11:13:28.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.309+0000 7f80f9873700 1 -- 172.21.15.28:0/979998480 shutdown_connections 2023-11-28T11:13:28.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.309+0000 7f80f9873700 1 --2- 172.21.15.28:0/979998480 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f80e00682f0 0x7f80e006a7a0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.309+0000 7f80f9873700 1 --2- 172.21.15.28:0/979998480 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f80dc004850 0x7f80f4158e60 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.309+0000 7f80f9873700 1 --2- 172.21.15.28:0/979998480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f80f40c8d60 0x7f80f40c91d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.309+0000 7f80f9873700 1 --2- 172.21.15.28:0/979998480 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f80f40c9ec0 0x7f80f4158330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.309+0000 7f80f9873700 1 -- 172.21.15.28:0/979998480 >> 172.21.15.28:0/979998480 conn(0x7f80f40d2df0 msgr2=0x7f80f415a020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:28.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.309+0000 7f80f9873700 1 -- 172.21.15.28:0/979998480 shutdown_connections 2023-11-28T11:13:28.312 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.309+0000 7f80f9873700 1 -- 172.21.15.28:0/979998480 wait complete. 2023-11-28T11:13:28.313 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 4 to deep-scrub 2023-11-28T11:13:28.323 INFO:tasks.ceph:Scrubbing osd.5 2023-11-28T11:13:28.323 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:13:28.452 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.451+0000 7ff15d9bd700 1 Processor -- start 2023-11-28T11:13:28.453 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.451+0000 7ff15d9bd700 1 -- start start 2023-11-28T11:13:28.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff15d9bd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff158159590 0x7ff15806e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff15d9bd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7ff15815db70 0x7ff1580ae730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.455 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff15d9bd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff1580ce890 0x7ff1580ced10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff15d9bd700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1580cf250 con 0x7ff1580ce890 2023-11-28T11:13:28.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff15d9bd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15815d120 con 0x7ff15815db70 2023-11-28T11:13:28.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff15d9bd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15815d6e0 con 0x7ff158159590 2023-11-28T11:13:28.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff1577fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff1580ce890 0x7ff1580ced10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff1577fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff1580ce890 0x7ff1580ced10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52694/0 (socket says 172.21.15.28:52694) 2023-11-28T11:13:28.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff1577fe700 1 -- 172.21.15.28:0/3688543432 learned_addr learned my addr 172.21.15.28:0/3688543432 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:28.456 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff156ffd700 1 --2- 172.21.15.28:0/3688543432 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff158159590 0x7ff15806e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff1567fc700 1 --2- 172.21.15.28:0/3688543432 >> v2:172.21.15.59:3300/0 conn(0x7ff15815db70 0x7ff1580ae730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff1577fe700 1 -- 172.21.15.28:0/3688543432 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff158159590 msgr2=0x7ff15806e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:28.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff1577fe700 1 --2- 172.21.15.28:0/3688543432 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff158159590 0x7ff15806e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff1577fe700 1 -- 172.21.15.28:0/3688543432 >> v2:172.21.15.59:3300/0 conn(0x7ff15815db70 msgr2=0x7ff1580ae730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:28.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff1577fe700 1 --2- 172.21.15.28:0/3688543432 >> v2:172.21.15.59:3300/0 conn(0x7ff15815db70 0x7ff1580ae730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff1577fe700 1 -- 172.21.15.28:0/3688543432 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1580aef60 con 0x7ff1580ce890 2023-11-28T11:13:28.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.452+0000 7ff1567fc700 1 --2- 172.21.15.28:0/3688543432 >> v2:172.21.15.59:3300/0 conn(0x7ff15815db70 0x7ff1580ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:13:28.457 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.453+0000 7ff1577fe700 1 --2- 172.21.15.28:0/3688543432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1580ce890 0x7ff1580ced10 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7ff15806f840 tx=0x7ff15000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6edf47e64e57761a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.453+0000 7ff155ffb700 1 -- 172.21.15.28:0/3688543432 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15000e070 con 0x7ff1580ce890 2023-11-28T11:13:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.453+0000 7ff155ffb700 1 -- 172.21.15.28:0/3688543432 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff15000b9c0 con 0x7ff1580ce890 2023-11-28T11:13:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.453+0000 7ff155ffb700 1 -- 172.21.15.28:0/3688543432 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff15000bd70 con 0x7ff1580ce890 2023-11-28T11:13:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.453+0000 7ff15d9bd700 1 -- 172.21.15.28:0/3688543432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1580ce890 msgr2=0x7ff1580ced10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.453+0000 7ff15d9bd700 1 --2- 172.21.15.28:0/3688543432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1580ce890 0x7ff1580ced10 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7ff15806f840 tx=0x7ff15000b330 comp rx=0 tx=0).stop 2023-11-28T11:13:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.453+0000 7ff15d9bd700 1 -- 172.21.15.28:0/3688543432 shutdown_connections 2023-11-28T11:13:28.458 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.453+0000 7ff15d9bd700 1 --2- 172.21.15.28:0/3688543432 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1580ce890 0x7ff1580ced10 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.453+0000 7ff15d9bd700 1 --2- 172.21.15.28:0/3688543432 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff158159590 0x7ff15806e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.453+0000 7ff15d9bd700 1 --2- 172.21.15.28:0/3688543432 >> v2:172.21.15.59:3300/0 conn(0x7ff15815db70 0x7ff1580ae730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.453+0000 7ff15d9bd700 1 -- 172.21.15.28:0/3688543432 >> 172.21.15.28:0/3688543432 conn(0x7ff1580b1650 msgr2=0x7ff1580b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:28.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.453+0000 7ff15d9bd700 1 -- 172.21.15.28:0/3688543432 shutdown_connections 2023-11-28T11:13:28.459 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.453+0000 7ff15d9bd700 1 -- 172.21.15.28:0/3688543432 wait complete. 2023-11-28T11:13:28.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.454+0000 7ff15d9bd700 1 Processor -- start 2023-11-28T11:13:28.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.455+0000 7ff15d9bd700 1 -- start start 2023-11-28T11:13:28.462 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.455+0000 7ff15d9bd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1580ce890 0x7ff15814f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.455+0000 7ff15d9bd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff158159590 0x7ff15815a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.455+0000 7ff15d9bd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff15815db70 0x7ff15815aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.455+0000 7ff15d9bd700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580ac1d0 con 0x7ff1580ce890 2023-11-28T11:13:28.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.455+0000 7ff15d9bd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580ab8c0 con 0x7ff158159590 2023-11-28T11:13:28.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.455+0000 7ff15d9bd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1580c8a80 con 0x7ff15815db70 2023-11-28T11:13:28.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.455+0000 7ff156ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1580ce890 0x7ff15814f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.463 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.456+0000 7ff156ffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1580ce890 0x7ff15814f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52696/0 (socket says 172.21.15.28:52696) 2023-11-28T11:13:28.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.456+0000 7ff156ffd700 1 -- 172.21.15.28:0/118272859 learned_addr learned my addr 172.21.15.28:0/118272859 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:28.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.456+0000 7ff1577fe700 1 --2- 172.21.15.28:0/118272859 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff15815db70 0x7ff15815aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.456+0000 7ff1567fc700 1 --2- 172.21.15.28:0/118272859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff158159590 0x7ff15815a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.456+0000 7ff156ffd700 1 -- 172.21.15.28:0/118272859 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff15815db70 msgr2=0x7ff15815aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.456+0000 7ff156ffd700 1 --2- 172.21.15.28:0/118272859 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff15815db70 0x7ff15815aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.456+0000 7ff156ffd700 1 -- 172.21.15.28:0/118272859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff158159590 msgr2=0x7ff15815a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.456+0000 7ff156ffd700 1 --2- 172.21.15.28:0/118272859 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff158159590 0x7ff15815a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.464 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.456+0000 7ff156ffd700 1 -- 172.21.15.28:0/118272859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff150009b30 con 0x7ff1580ce890 2023-11-28T11:13:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.456+0000 7ff156ffd700 1 --2- 172.21.15.28:0/118272859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1580ce890 0x7ff15814f4f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff14800e810 tx=0x7ff14800eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.456+0000 7ff13ffff700 1 -- 172.21.15.28:0/118272859 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14800c070 con 0x7ff1580ce890 2023-11-28T11:13:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.456+0000 7ff13ffff700 1 -- 172.21.15.28:0/118272859 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff148013610 con 0x7ff1580ce890 2023-11-28T11:13:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.456+0000 7ff15d9bd700 1 -- 172.21.15.28:0/118272859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15815d370 con 0x7ff1580ce890 2023-11-28T11:13:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.456+0000 7ff13ffff700 1 -- 172.21.15.28:0/118272859 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1480139c0 con 0x7ff1580ce890 2023-11-28T11:13:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.456+0000 7ff15d9bd700 1 -- 172.21.15.28:0/118272859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1580af240 con 0x7ff1580ce890 2023-11-28T11:13:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.457+0000 7ff15d9bd700 1 -- 172.21.15.28:0/118272859 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff1580aec70 con 0x7ff1580ce890 2023-11-28T11:13:28.465 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.458+0000 7ff13ffff700 1 -- 172.21.15.28:0/118272859 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff148023070 con 0x7ff1580ce890 2023-11-28T11:13:28.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.458+0000 7ff13ffff700 1 --2- 172.21.15.28:0/118272859 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1440682f0 0x7ff14406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.458+0000 7ff13ffff700 1 -- 172.21.15.28:0/118272859 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7ff148085830 con 0x7ff1580ce890 2023-11-28T11:13:28.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.458+0000 7ff1567fc700 1 --2- 172.21.15.28:0/118272859 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1440682f0 0x7ff14406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.458+0000 7ff13ffff700 1 --2- 172.21.15.28:0/118272859 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x7ff14406eb90 0x7ff144070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.458+0000 7ff13ffff700 1 -- 172.21.15.28:0/118272859 --> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff1440716b0 con 0x7ff14406eb90 2023-11-28T11:13:28.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.458+0000 7ff13ffff700 1 -- 172.21.15.28:0/118272859 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff14808a2a0 con 0x7ff1580ce890 2023-11-28T11:13:28.466 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.458+0000 7ff1577fe700 1 --2- 172.21.15.28:0/118272859 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x7ff14406eb90 0x7ff144070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.458+0000 7ff1567fc700 1 --2- 172.21.15.28:0/118272859 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1440682f0 0x7ff14406a7a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff1581509f0 tx=0x7ff14c008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:28.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.459+0000 7ff1577fe700 1 --2- 172.21.15.28:0/118272859 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x7ff14406eb90 0x7ff144070fe0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:28.467 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.460+0000 7ff13ffff700 1 -- 172.21.15.28:0/118272859 <== osd.5 v2:172.21.15.59:6800/103696 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7ff1440716b0 con 0x7ff14406eb90 2023-11-28T11:13:28.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.474+0000 7ff15d9bd700 1 -- 172.21.15.28:0/118272859 --> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff1580ac310 con 0x7ff14406eb90 2023-11-28T11:13:28.476 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.475+0000 7ff13ffff700 1 -- 172.21.15.28:0/118272859 <== osd.5 v2:172.21.15.59:6800/103696 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff1500099b0 con 0x7ff14406eb90 2023-11-28T11:13:28.476 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:13:28.477 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:13:28.477 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:13:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.475+0000 7ff15d9bd700 1 -- 172.21.15.28:0/118272859 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x7ff14406eb90 msgr2=0x7ff144070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.475+0000 7ff15d9bd700 1 --2- 172.21.15.28:0/118272859 >> [v2:172.21.15.59:6800/103696,v1:172.21.15.59:6802/103696] conn(0x7ff14406eb90 0x7ff144070fe0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.475+0000 7ff15d9bd700 1 -- 172.21.15.28:0/118272859 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1440682f0 msgr2=0x7ff14406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.475+0000 7ff15d9bd700 1 --2- 172.21.15.28:0/118272859 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff1440682f0 0x7ff14406a7a0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7ff1581509f0 tx=0x7ff14c008040 comp rx=0 tx=0).stop 2023-11-28T11:13:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.475+0000 7ff15d9bd700 1 -- 172.21.15.28:0/118272859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1580ce890 msgr2=0x7ff15814f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.475+0000 7ff15d9bd700 1 --2- 172.21.15.28:0/118272859 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff1580ce890 0x7ff15814f4f0 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7ff14800e810 tx=0x7ff14800eb20 comp rx=0 tx=0).stop 2023-11-28T11:13:28.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.475+0000 7ff1577fe700 1 -- 172.21.15.28:0/118272859 reap_dead start 2023-11-28T11:13:28.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.475+0000 7ff15d9bd700 1 -- 172.21.15.28:0/118272859 shutdown_connections 2023-11-28T11:13:28.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.475+0000 7ff15d9bd700 1 -- 172.21.15.28:0/118272859 >> 172.21.15.28:0/118272859 conn(0x7ff1580b1650 msgr2=0x7ff1580af870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:28.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.475+0000 7ff15d9bd700 1 -- 172.21.15.28:0/118272859 shutdown_connections 2023-11-28T11:13:28.479 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.475+0000 7ff15d9bd700 1 -- 172.21.15.28:0/118272859 wait complete. 2023-11-28T11:13:28.488 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-28T11:13:28.614 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.612+0000 7ff7464c7700 1 Processor -- start 2023-11-28T11:13:28.614 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.612+0000 7ff7464c7700 1 -- start start 2023-11-28T11:13:28.616 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.613+0000 7ff7464c7700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff74015d1e0 0x7ff74015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.613+0000 7ff7464c7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff74006aa00 0x7ff7400cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.613+0000 7ff7464c7700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff74006ae80 con 0x7ff74006a560 2023-11-28T11:13:28.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.613+0000 7ff7464c7700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7400cdec0 con 0x7ff74015d1e0 2023-11-28T11:13:28.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.613+0000 7ff7464c7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7400c9cd0 con 0x7ff74006aa00 2023-11-28T11:13:28.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.613+0000 7ff73ffff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7ff74015d1e0 0x7ff74015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.613+0000 7ff73f7fe700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7ff74006a560 0x7ff7400cceb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40764/0 (socket says 172.21.15.28:40764) 2023-11-28T11:13:28.617 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.613+0000 7ff73f7fe700 1 -- 172.21.15.28:0/1273551317 learned_addr learned my addr 172.21.15.28:0/1273551317 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:28.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.613+0000 7ff744a64700 1 --2- 172.21.15.28:0/1273551317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff74006aa00 0x7ff7400cd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.614+0000 7ff73ffff700 1 -- 172.21.15.28:0/1273551317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff74006aa00 msgr2=0x7ff7400cd5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:28.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.614+0000 7ff73ffff700 1 --2- 172.21.15.28:0/1273551317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff74006aa00 0x7ff7400cd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.614+0000 7ff73ffff700 1 -- 172.21.15.28:0/1273551317 >> v1:172.21.15.28:6789/0 conn(0x7ff74006a560 legacy=0x7ff7400cceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.614+0000 7ff73ffff700 1 -- 172.21.15.28:0/1273551317 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7400c9fb0 con 0x7ff74015d1e0 2023-11-28T11:13:28.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.614+0000 7ff73effd700 1 -- 172.21.15.28:0/1273551317 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2180437960 0 0) 0x7ff74006ae80 con 0x7ff74006a560 2023-11-28T11:13:28.618 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.614+0000 7ff73ffff700 1 --2- 172.21.15.28:0/1273551317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74015d1e0 0x7ff74015d5b0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7ff730009840 tx=0x7ff730009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c32e278b09ac726 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:28.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.614+0000 7ff73effd700 1 -- 172.21.15.28:0/1273551317 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730011830 con 0x7ff74015d1e0 2023-11-28T11:13:28.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.614+0000 7ff73effd700 1 -- 172.21.15.28:0/1273551317 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff730011990 con 0x7ff74015d1e0 2023-11-28T11:13:28.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.614+0000 7ff73effd700 1 -- 172.21.15.28:0/1273551317 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730011d20 con 0x7ff74015d1e0 2023-11-28T11:13:28.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.614+0000 7ff7464c7700 1 -- 172.21.15.28:0/1273551317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74015d1e0 msgr2=0x7ff74015d5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.614+0000 7ff7464c7700 1 --2- 172.21.15.28:0/1273551317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74015d1e0 0x7ff74015d5b0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7ff730009840 tx=0x7ff730009b50 comp rx=0 tx=0).stop 2023-11-28T11:13:28.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.614+0000 7ff7464c7700 1 -- 172.21.15.28:0/1273551317 shutdown_connections 2023-11-28T11:13:28.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.614+0000 7ff7464c7700 1 --2- 172.21.15.28:0/1273551317 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff74006aa00 0x7ff7400cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.619 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.614+0000 7ff7464c7700 1 --2- 172.21.15.28:0/1273551317 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74015d1e0 0x7ff74015d5b0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.614+0000 7ff7464c7700 1 -- 172.21.15.28:0/1273551317 >> 172.21.15.28:0/1273551317 conn(0x7ff7400b0dd0 msgr2=0x7ff7400b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:28.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.615+0000 7ff7464c7700 1 -- 172.21.15.28:0/1273551317 shutdown_connections 2023-11-28T11:13:28.620 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.615+0000 7ff7464c7700 1 -- 172.21.15.28:0/1273551317 wait complete. 2023-11-28T11:13:28.622 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.616+0000 7ff7464c7700 1 Processor -- start 2023-11-28T11:13:28.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.616+0000 7ff7464c7700 1 -- start start 2023-11-28T11:13:28.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff7464c7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74006a560 0x7ff7401f18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff7464c7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff74006aa00 0x7ff7400b3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff7464c7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff74015d1e0 0x7ff7400b3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff7464c7700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7400b4370 con 0x7ff74006a560 2023-11-28T11:13:28.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff7464c7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7400ac7b0 con 0x7ff74006aa00 2023-11-28T11:13:28.623 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff7464c7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7400ab9f0 con 0x7ff74015d1e0 2023-11-28T11:13:28.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff73ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74006a560 0x7ff7401f18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff73ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74006a560 0x7ff7401f18e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52712/0 (socket says 172.21.15.28:52712) 2023-11-28T11:13:28.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff73ffff700 1 -- 172.21.15.28:0/2033404886 learned_addr learned my addr 172.21.15.28:0/2033404886 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:28.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff744a64700 1 --2- 172.21.15.28:0/2033404886 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff74015d1e0 0x7ff7400b3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff73f7fe700 1 --2- 172.21.15.28:0/2033404886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff74006aa00 0x7ff7400b3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff73ffff700 1 -- 172.21.15.28:0/2033404886 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff74015d1e0 msgr2=0x7ff7400b3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff73ffff700 1 --2- 172.21.15.28:0/2033404886 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff74015d1e0 0x7ff7400b3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.624 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff73ffff700 1 -- 172.21.15.28:0/2033404886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff74006aa00 msgr2=0x7ff7400b3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff73ffff700 1 --2- 172.21.15.28:0/2033404886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff74006aa00 0x7ff7400b3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff73ffff700 1 -- 172.21.15.28:0/2033404886 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7300094a0 con 0x7ff74006a560 2023-11-28T11:13:28.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff73ffff700 1 --2- 172.21.15.28:0/2033404886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74006a560 0x7ff7401f18e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ff730000c00 tx=0x7ff730004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:28.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff73cff9700 1 -- 172.21.15.28:0/2033404886 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7300045f0 con 0x7ff74006a560 2023-11-28T11:13:28.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff73cff9700 1 -- 172.21.15.28:0/2033404886 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff73001f070 con 0x7ff74006a560 2023-11-28T11:13:28.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff7464c7700 1 -- 172.21.15.28:0/2033404886 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7400ca570 con 0x7ff74006a560 2023-11-28T11:13:28.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff73cff9700 1 -- 172.21.15.28:0/2033404886 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff730003ae0 con 0x7ff74006a560 2023-11-28T11:13:28.625 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.617+0000 7ff7464c7700 1 -- 172.21.15.28:0/2033404886 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7400ca290 con 0x7ff74006a560 2023-11-28T11:13:28.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.618+0000 7ff7464c7700 1 -- 172.21.15.28:0/2033404886 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7400ca850 con 0x7ff74006a560 2023-11-28T11:13:28.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.619+0000 7ff73cff9700 1 -- 172.21.15.28:0/2033404886 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7ff73000f050 con 0x7ff74006a560 2023-11-28T11:13:28.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.619+0000 7ff73cff9700 1 --2- 172.21.15.28:0/2033404886 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff72c0682f0 0x7ff72c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.619+0000 7ff73cff9700 1 -- 172.21.15.28:0/2033404886 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7ff730021070 con 0x7ff74006a560 2023-11-28T11:13:28.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.619+0000 7ff73f7fe700 1 --2- 172.21.15.28:0/2033404886 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff72c0682f0 0x7ff72c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.626 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.620+0000 7ff73f7fe700 1 --2- 172.21.15.28:0/2033404886 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff72c0682f0 0x7ff72c06a7a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff7401f55c0 tx=0x7ff734007c90 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:28.627 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.623+0000 7ff73cff9700 1 -- 172.21.15.28:0/2033404886 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7ff730053fb0 con 0x7ff74006a560 2023-11-28T11:13:28.817 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.815+0000 7ff7464c7700 1 -- 172.21.15.28:0/2033404886 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7400675a0 con 0x7ff72c0682f0 2023-11-28T11:13:28.817 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.816+0000 7ff73cff9700 1 -- 172.21.15.28:0/2033404886 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7ff7400675a0 con 0x7ff72c0682f0 2023-11-28T11:13:28.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.817+0000 7ff7464c7700 1 -- 172.21.15.28:0/2033404886 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff72c0682f0 msgr2=0x7ff72c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.817+0000 7ff7464c7700 1 --2- 172.21.15.28:0/2033404886 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff72c0682f0 0x7ff72c06a7a0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7ff7401f55c0 tx=0x7ff734007c90 comp rx=0 tx=0).stop 2023-11-28T11:13:28.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.817+0000 7ff7464c7700 1 -- 172.21.15.28:0/2033404886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74006a560 msgr2=0x7ff7401f18e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.817+0000 7ff7464c7700 1 --2- 172.21.15.28:0/2033404886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74006a560 0x7ff7401f18e0 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7ff730000c00 tx=0x7ff730004750 comp rx=0 tx=0).stop 2023-11-28T11:13:28.820 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.818+0000 7ff7464c7700 1 -- 172.21.15.28:0/2033404886 shutdown_connections 2023-11-28T11:13:28.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.818+0000 7ff7464c7700 1 --2- 172.21.15.28:0/2033404886 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7ff72c0682f0 0x7ff72c06a7a0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.818+0000 7ff7464c7700 1 --2- 172.21.15.28:0/2033404886 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7ff74006a560 0x7ff7401f18e0 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.818+0000 7ff7464c7700 1 --2- 172.21.15.28:0/2033404886 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7ff74015d1e0 0x7ff7400b3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.818+0000 7ff7464c7700 1 --2- 172.21.15.28:0/2033404886 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff74006aa00 0x7ff7400b3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.818+0000 7ff7464c7700 1 -- 172.21.15.28:0/2033404886 >> 172.21.15.28:0/2033404886 conn(0x7ff7400b0dd0 msgr2=0x7ff7400b11a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:28.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.818+0000 7ff7464c7700 1 -- 172.21.15.28:0/2033404886 shutdown_connections 2023-11-28T11:13:28.821 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.818+0000 7ff7464c7700 1 -- 172.21.15.28:0/2033404886 wait complete. 2023-11-28T11:13:28.822 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 5 to deep-scrub 2023-11-28T11:13:28.831 INFO:tasks.ceph:Scrubbing osd.6 2023-11-28T11:13:28.832 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:13:28.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.959+0000 7f8b7cffd700 1 Processor -- start 2023-11-28T11:13:28.961 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.959+0000 7f8b7cffd700 1 -- start start 2023-11-28T11:13:28.974 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.960+0000 7f8b7cffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8b7806b930 0x7f8b780ca280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.974 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.960+0000 7f8b7cffd700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f8b780cc4b0 0x7f8b780cc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.960+0000 7f8b7cffd700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b780cb570 con 0x7f8b7815a5a0 2023-11-28T11:13:28.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.960+0000 7f8b7cffd700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b7806bda0 con 0x7f8b780cc4b0 2023-11-28T11:13:28.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.960+0000 7f8b7cffd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b780cce70 con 0x7f8b7806b930 2023-11-28T11:13:28.975 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.960+0000 7f8b7659c700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f8b7815a5a0 0x7f8b780cba00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40778/0 (socket says 172.21.15.28:40778) 2023-11-28T11:13:28.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.960+0000 7f8b7659c700 1 -- 172.21.15.28:0/3904162117 learned_addr learned my addr 172.21.15.28:0/3904162117 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:28.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.960+0000 7f8b7559a700 1 -- 172.21.15.28:0/3904162117 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2988910908 0 0) 0x7f8b780cb570 con 0x7f8b7815a5a0 2023-11-28T11:13:28.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.960+0000 7f8b75d9b700 1 --2- 172.21.15.28:0/3904162117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8b7806b930 0x7f8b780ca280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.976 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.960+0000 7f8b76d9d700 1 --2- 172.21.15.28:0/3904162117 >> v2:172.21.15.59:3300/0 conn(0x7f8b780cc4b0 0x7f8b780cc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7559a700 1 -- 172.21.15.28:0/3904162117 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b600035b0 con 0x7f8b7815a5a0 2023-11-28T11:13:28.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7559a700 1 -- 172.21.15.28:0/3904162117 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2603268302 0 0) 0x7f8b600035b0 con 0x7f8b7815a5a0 2023-11-28T11:13:28.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7559a700 1 -- 172.21.15.28:0/3904162117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8b7806b930 msgr2=0x7f8b780ca280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:28.977 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7559a700 1 --2- 172.21.15.28:0/3904162117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8b7806b930 0x7f8b780ca280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7559a700 1 -- 172.21.15.28:0/3904162117 >> v2:172.21.15.59:3300/0 conn(0x7f8b780cc4b0 msgr2=0x7f8b780cc930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:28.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7559a700 1 --2- 172.21.15.28:0/3904162117 >> v2:172.21.15.59:3300/0 conn(0x7f8b780cc4b0 0x7f8b780cc930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7559a700 1 -- 172.21.15.28:0/3904162117 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b780cc110 con 0x7f8b7815a5a0 2023-11-28T11:13:28.978 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7559a700 1 -- 172.21.15.28:0/3904162117 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f8b6c001670 con 0x7f8b7815a5a0 2023-11-28T11:13:28.979 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7559a700 1 -- 172.21.15.28:0/3904162117 >> v1:172.21.15.28:6789/0 conn(0x7f8b7815a5a0 legacy=0x7f8b780cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.979 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7559a700 1 --2- 172.21.15.28:0/3904162117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8b7806b930 0x7f8b60003a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.979 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7559a700 1 --2- 172.21.15.28:0/3904162117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b60003f80 0x7f8b60004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.979 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7559a700 1 --2- 172.21.15.28:0/3904162117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b60004d90 0x7f8b60005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7559a700 1 -- 172.21.15.28:0/3904162117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b7806bda0 con 0x7f8b60004d90 2023-11-28T11:13:28.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7559a700 1 -- 172.21.15.28:0/3904162117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b780cce70 con 0x7f8b60003f80 2023-11-28T11:13:28.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7559a700 1 -- 172.21.15.28:0/3904162117 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6000ba10 con 0x7f8b7806b930 2023-11-28T11:13:28.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b76d9d700 1 --2- 172.21.15.28:0/3904162117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b60004d90 0x7f8b60005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b75d9b700 1 --2- 172.21.15.28:0/3904162117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b60003f80 0x7f8b60004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b7659c700 1 --2- 172.21.15.28:0/3904162117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8b7806b930 0x7f8b60003a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b76d9d700 1 -- 172.21.15.28:0/3904162117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8b7806b930 msgr2=0x7f8b60003a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:28.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b76d9d700 1 --2- 172.21.15.28:0/3904162117 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8b7806b930 0x7f8b60003a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b76d9d700 1 -- 172.21.15.28:0/3904162117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b60003f80 msgr2=0x7f8b60004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:28.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b76d9d700 1 --2- 172.21.15.28:0/3904162117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b60003f80 0x7f8b60004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.982 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.961+0000 7f8b76d9d700 1 -- 172.21.15.28:0/3904162117 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b600035b0 con 0x7f8b60004d90 2023-11-28T11:13:28.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.962+0000 7f8b76d9d700 1 --2- 172.21.15.28:0/3904162117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b60004d90 0x7f8b60005200 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009fc0 tx=0x7f8b60005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f19b8025698a054e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:28.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.962+0000 7f8b7559a700 1 -- 172.21.15.28:0/3904162117 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7000ec30 con 0x7f8b60004d90 2023-11-28T11:13:28.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.962+0000 7f8b7559a700 1 -- 172.21.15.28:0/3904162117 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7000f040 con 0x7f8b60004d90 2023-11-28T11:13:28.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.962+0000 7f8b7559a700 1 -- 172.21.15.28:0/3904162117 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70014480 con 0x7f8b60004d90 2023-11-28T11:13:28.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.962+0000 7f8b7cffd700 1 -- 172.21.15.28:0/3904162117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b60004d90 msgr2=0x7f8b60005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.962+0000 7f8b7cffd700 1 --2- 172.21.15.28:0/3904162117 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b60004d90 0x7f8b60005200 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f8b70009fc0 tx=0x7f8b60005880 comp rx=0 tx=0).stop 2023-11-28T11:13:28.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.962+0000 7f8b76d9d700 1 -- 172.21.15.28:0/3904162117 reap_dead start 2023-11-28T11:13:28.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.962+0000 7f8b7cffd700 1 -- 172.21.15.28:0/3904162117 shutdown_connections 2023-11-28T11:13:28.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.962+0000 7f8b7cffd700 1 -- 172.21.15.28:0/3904162117 >> 172.21.15.28:0/3904162117 conn(0x7f8b780b1650 msgr2=0x7f8b780b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:28.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.962+0000 7f8b7cffd700 1 -- 172.21.15.28:0/3904162117 shutdown_connections 2023-11-28T11:13:28.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.962+0000 7f8b7cffd700 1 -- 172.21.15.28:0/3904162117 wait complete. 2023-11-28T11:13:28.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.964+0000 7f8b7cffd700 1 Processor -- start 2023-11-28T11:13:28.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.964+0000 7f8b7cffd700 1 -- start start 2023-11-28T11:13:28.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.964+0000 7f8b7cffd700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8b60004d90 0x7f8b781efbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.964+0000 7f8b7cffd700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b780b5520 0x7f8b780b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.964+0000 7f8b7cffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b781e36e0 0x7f8b780b09d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.964+0000 7f8b7cffd700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b781e4900 con 0x7f8b780b5520 2023-11-28T11:13:28.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.964+0000 7f8b7cffd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b78149810 con 0x7f8b781e36e0 2023-11-28T11:13:28.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.964+0000 7f8b7cffd700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b781e3c80 con 0x7f8b60004d90 2023-11-28T11:13:28.987 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b75d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b780b5520 0x7f8b780b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b7659c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8b60004d90 0x7f8b781efbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b75d9b700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b780b5520 0x7f8b780b0490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52734/0 (socket says 172.21.15.28:52734) 2023-11-28T11:13:28.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b75d9b700 1 -- 172.21.15.28:0/2775121899 learned_addr learned my addr 172.21.15.28:0/2775121899 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:28.988 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b76d9d700 1 --2- 172.21.15.28:0/2775121899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b781e36e0 0x7f8b780b09d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b75d9b700 1 -- 172.21.15.28:0/2775121899 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8b60004d90 msgr2=0x7f8b781efbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b75d9b700 1 --2- 172.21.15.28:0/2775121899 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f8b60004d90 0x7f8b781efbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b75d9b700 1 -- 172.21.15.28:0/2775121899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b781e36e0 msgr2=0x7f8b780b09d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:28.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b75d9b700 1 --2- 172.21.15.28:0/2775121899 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8b781e36e0 0x7f8b780b09d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:28.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b75d9b700 1 -- 172.21.15.28:0/2775121899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b780cac40 con 0x7f8b780b5520 2023-11-28T11:13:28.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b75d9b700 1 --2- 172.21.15.28:0/2775121899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b780b5520 0x7f8b780b0490 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8b680119d0 tx=0x7f8b68011ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:28.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b66ffd700 1 -- 172.21.15.28:0/2775121899 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b68011e70 con 0x7f8b780b5520 2023-11-28T11:13:28.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b66ffd700 1 -- 172.21.15.28:0/2775121899 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b6800f8e0 con 0x7f8b780b5520 2023-11-28T11:13:28.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b7cffd700 1 -- 172.21.15.28:0/2775121899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b780ca960 con 0x7f8b780b5520 2023-11-28T11:13:28.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b66ffd700 1 -- 172.21.15.28:0/2775121899 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b6800fc70 con 0x7f8b780b5520 2023-11-28T11:13:28.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.965+0000 7f8b7cffd700 1 -- 172.21.15.28:0/2775121899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b780caf20 con 0x7f8b780b5520 2023-11-28T11:13:28.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.966+0000 7f8b7cffd700 1 -- 172.21.15.28:0/2775121899 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8b780739f0 con 0x7f8b780b5520 2023-11-28T11:13:28.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.967+0000 7f8b66ffd700 1 -- 172.21.15.28:0/2775121899 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8b6801d400 con 0x7f8b780b5520 2023-11-28T11:13:28.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.967+0000 7f8b66ffd700 1 --2- 172.21.15.28:0/2775121899 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8b580682f0 0x7f8b5806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.967+0000 7f8b66ffd700 1 -- 172.21.15.28:0/2775121899 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f8b68086980 con 0x7f8b780b5520 2023-11-28T11:13:28.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.967+0000 7f8b7659c700 1 --2- 172.21.15.28:0/2775121899 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8b580682f0 0x7f8b5806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.967+0000 7f8b66ffd700 1 --2- 172.21.15.28:0/2775121899 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x7f8b5806eb90 0x7f8b58070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:28.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.967+0000 7f8b66ffd700 1 -- 172.21.15.28:0/2775121899 --> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b580716b0 con 0x7f8b5806eb90 2023-11-28T11:13:28.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.967+0000 7f8b66ffd700 1 -- 172.21.15.28:0/2775121899 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b68086dc0 con 0x7f8b780b5520 2023-11-28T11:13:28.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.967+0000 7f8b76d9d700 1 --2- 172.21.15.28:0/2775121899 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x7f8b5806eb90 0x7f8b58070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:28.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.968+0000 7f8b7659c700 1 --2- 172.21.15.28:0/2775121899 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8b580682f0 0x7f8b5806a7a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c001cf0 tx=0x7f8b6c007980 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:28.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.968+0000 7f8b76d9d700 1 --2- 172.21.15.28:0/2775121899 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x7f8b5806eb90 0x7f8b58070fe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:28.994 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.969+0000 7f8b66ffd700 1 -- 172.21.15.28:0/2775121899 <== osd.6 v2:172.21.15.59:6816/103697 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8b580716b0 con 0x7f8b5806eb90 2023-11-28T11:13:28.995 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.984+0000 7f8b7cffd700 1 -- 172.21.15.28:0/2775121899 --> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f8b781e4a40 con 0x7f8b5806eb90 2023-11-28T11:13:28.996 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.985+0000 7f8b66ffd700 1 -- 172.21.15.28:0/2775121899 <== osd.6 v2:172.21.15.59:6816/103697 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f8b781e4a40 con 0x7f8b5806eb90 2023-11-28T11:13:28.996 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:13:28.996 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:13:28.997 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:13:29.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.985+0000 7f8b7cffd700 1 -- 172.21.15.28:0/2775121899 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x7f8b5806eb90 msgr2=0x7f8b58070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.985+0000 7f8b7cffd700 1 --2- 172.21.15.28:0/2775121899 >> [v2:172.21.15.59:6816/103697,v1:172.21.15.59:6817/103697] conn(0x7f8b5806eb90 0x7f8b58070fe0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.985+0000 7f8b7cffd700 1 -- 172.21.15.28:0/2775121899 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8b580682f0 msgr2=0x7f8b5806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.000 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.985+0000 7f8b7cffd700 1 --2- 172.21.15.28:0/2775121899 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f8b580682f0 0x7f8b5806a7a0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f8b6c001cf0 tx=0x7f8b6c007980 comp rx=0 tx=0).stop 2023-11-28T11:13:29.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.985+0000 7f8b7cffd700 1 -- 172.21.15.28:0/2775121899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b780b5520 msgr2=0x7f8b780b0490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.985+0000 7f8b7cffd700 1 --2- 172.21.15.28:0/2775121899 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f8b780b5520 0x7f8b780b0490 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f8b680119d0 tx=0x7f8b68011ce0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.985+0000 7f8b76d9d700 1 -- 172.21.15.28:0/2775121899 reap_dead start 2023-11-28T11:13:29.001 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.985+0000 7f8b7cffd700 1 -- 172.21.15.28:0/2775121899 shutdown_connections 2023-11-28T11:13:29.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.985+0000 7f8b7cffd700 1 -- 172.21.15.28:0/2775121899 >> 172.21.15.28:0/2775121899 conn(0x7f8b780b1650 msgr2=0x7f8b78156960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:29.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.985+0000 7f8b7cffd700 1 -- 172.21.15.28:0/2775121899 shutdown_connections 2023-11-28T11:13:29.002 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:28.985+0000 7f8b7cffd700 1 -- 172.21.15.28:0/2775121899 wait complete. 2023-11-28T11:13:29.003 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-28T11:13:29.132 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.127+0000 7fe096ae3700 1 Processor -- start 2023-11-28T11:13:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.128+0000 7fe096ae3700 1 -- start start 2023-11-28T11:13:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.128+0000 7fe096ae3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fe09015c560 0x7fe09006f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.128+0000 7fe096ae3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0900ceec0 0x7fe09006c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.128+0000 7fe096ae3700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0900b30b0 con 0x7fe0900b2660 2023-11-28T11:13:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.128+0000 7fe096ae3700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe09006c020 con 0x7fe09015c560 2023-11-28T11:13:29.133 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.128+0000 7fe096ae3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0900d09c0 con 0x7fe0900ceec0 2023-11-28T11:13:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.128+0000 7fe08ffff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fe0900b2660 0x7fe0900ce7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40802/0 (socket says 172.21.15.28:40802) 2023-11-28T11:13:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.128+0000 7fe08ffff700 1 -- 172.21.15.28:0/94789403 learned_addr learned my addr 172.21.15.28:0/94789403 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.129+0000 7fe09487f700 1 --2- 172.21.15.28:0/94789403 >> v2:172.21.15.59:3300/0 conn(0x7fe09015c560 0x7fe09006f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.129+0000 7fe095080700 1 --2- 172.21.15.28:0/94789403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0900ceec0 0x7fe09006c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.129+0000 7fe08f7fe700 1 -- 172.21.15.28:0/94789403 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2599362446 0 0) 0x7fe0900b30b0 con 0x7fe0900b2660 2023-11-28T11:13:29.134 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.129+0000 7fe08f7fe700 1 -- 172.21.15.28:0/94789403 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe0800035b0 con 0x7fe0900b2660 2023-11-28T11:13:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.129+0000 7fe095080700 1 -- 172.21.15.28:0/94789403 >> v2:172.21.15.59:3300/0 conn(0x7fe09015c560 msgr2=0x7fe09006f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.129+0000 7fe095080700 1 --2- 172.21.15.28:0/94789403 >> v2:172.21.15.59:3300/0 conn(0x7fe09015c560 0x7fe09006f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.129+0000 7fe095080700 1 -- 172.21.15.28:0/94789403 >> v1:172.21.15.28:6789/0 conn(0x7fe0900b2660 legacy=0x7fe0900ce7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.129+0000 7fe095080700 1 -- 172.21.15.28:0/94789403 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09006fdf0 con 0x7fe0900ceec0 2023-11-28T11:13:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.129+0000 7fe09487f700 1 --2- 172.21.15.28:0/94789403 >> v2:172.21.15.59:3300/0 conn(0x7fe09015c560 0x7fe09006f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:13:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.129+0000 7fe095080700 1 --2- 172.21.15.28:0/94789403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0900ceec0 0x7fe09006c4b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe088009d90 tx=0x7fe08800b330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80b6337e7106abfd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:29.135 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.129+0000 7fe08f7fe700 1 -- 172.21.15.28:0/94789403 <== mon.2 v2:172.21.15.59:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08800e070 con 0x7fe0900ceec0 2023-11-28T11:13:29.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.129+0000 7fe08f7fe700 1 -- 172.21.15.28:0/94789403 <== mon.2 v2:172.21.15.59:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08800bb20 con 0x7fe0900ceec0 2023-11-28T11:13:29.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.129+0000 7fe08f7fe700 1 -- 172.21.15.28:0/94789403 <== mon.2 v2:172.21.15.59:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08800be90 con 0x7fe0900ceec0 2023-11-28T11:13:29.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.130+0000 7fe096ae3700 1 -- 172.21.15.28:0/94789403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0900ceec0 msgr2=0x7fe09006c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.130+0000 7fe096ae3700 1 --2- 172.21.15.28:0/94789403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0900ceec0 0x7fe09006c4b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fe088009d90 tx=0x7fe08800b330 comp rx=0 tx=0).stop 2023-11-28T11:13:29.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.130+0000 7fe096ae3700 1 -- 172.21.15.28:0/94789403 shutdown_connections 2023-11-28T11:13:29.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.130+0000 7fe096ae3700 1 --2- 172.21.15.28:0/94789403 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe0900ceec0 0x7fe09006c4b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.130+0000 7fe096ae3700 1 --2- 172.21.15.28:0/94789403 >> v2:172.21.15.59:3300/0 conn(0x7fe09015c560 0x7fe09006f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.136 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.130+0000 7fe096ae3700 1 -- 172.21.15.28:0/94789403 >> 172.21.15.28:0/94789403 conn(0x7fe0900cd660 msgr2=0x7fe0900cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.130+0000 7fe096ae3700 1 -- 172.21.15.28:0/94789403 shutdown_connections 2023-11-28T11:13:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.130+0000 7fe096ae3700 1 -- 172.21.15.28:0/94789403 wait complete. 2023-11-28T11:13:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.131+0000 7fe096ae3700 1 Processor -- start 2023-11-28T11:13:29.137 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.131+0000 7fe096ae3700 1 -- start start 2023-11-28T11:13:29.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.132+0000 7fe096ae3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0900b2660 0x7fe0900acfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.132+0000 7fe096ae3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0900ceec0 0x7fe090154c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.139 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.132+0000 7fe096ae3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09015c560 0x7fe0901564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.132+0000 7fe096ae3700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0900c53f0 con 0x7fe0900ceec0 2023-11-28T11:13:29.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.132+0000 7fe096ae3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0901572f0 con 0x7fe0900b2660 2023-11-28T11:13:29.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.132+0000 7fe096ae3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0901551b0 con 0x7fe09015c560 2023-11-28T11:13:29.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.132+0000 7fe08ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0900ceec0 0x7fe090154c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.132+0000 7fe08ffff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0900ceec0 0x7fe090154c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52750/0 (socket says 172.21.15.28:52750) 2023-11-28T11:13:29.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.132+0000 7fe08ffff700 1 -- 172.21.15.28:0/1017381660 learned_addr learned my addr 172.21.15.28:0/1017381660 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:29.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.132+0000 7fe09487f700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0900b2660 0x7fe0900acfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.140 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.132+0000 7fe08ffff700 1 -- 172.21.15.28:0/1017381660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09015c560 msgr2=0x7fe0901564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.132+0000 7fe095080700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09015c560 0x7fe0901564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.133+0000 7fe08ffff700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09015c560 0x7fe0901564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.133+0000 7fe08ffff700 1 -- 172.21.15.28:0/1017381660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0900b2660 msgr2=0x7fe0900acfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.133+0000 7fe08ffff700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0900b2660 0x7fe0900acfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.133+0000 7fe08ffff700 1 -- 172.21.15.28:0/1017381660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0840094a0 con 0x7fe0900ceec0 2023-11-28T11:13:29.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.133+0000 7fe09487f700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0900b2660 0x7fe0900acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:13:29.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.133+0000 7fe095080700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09015c560 0x7fe0901564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-11-28T11:13:29.141 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.133+0000 7fe08ffff700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0900ceec0 0x7fe090154c70 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00c500 tx=0x7fe07c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:29.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.133+0000 7fe08d7fa700 1 -- 172.21.15.28:0/1017381660 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c005d70 con 0x7fe0900ceec0 2023-11-28T11:13:29.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.133+0000 7fe08d7fa700 1 -- 172.21.15.28:0/1017381660 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c0070d0 con 0x7fe0900ceec0 2023-11-28T11:13:29.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.133+0000 7fe096ae3700 1 -- 172.21.15.28:0/1017381660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe088009a70 con 0x7fe0900ceec0 2023-11-28T11:13:29.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.133+0000 7fe08d7fa700 1 -- 172.21.15.28:0/1017381660 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c007440 con 0x7fe0900ceec0 2023-11-28T11:13:29.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.133+0000 7fe096ae3700 1 -- 172.21.15.28:0/1017381660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0900b2dd0 con 0x7fe0900ceec0 2023-11-28T11:13:29.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.134+0000 7fe096ae3700 1 -- 172.21.15.28:0/1017381660 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0900b2af0 con 0x7fe0900ceec0 2023-11-28T11:13:29.142 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.134+0000 7fe08d7fa700 1 -- 172.21.15.28:0/1017381660 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fe07c019030 con 0x7fe0900ceec0 2023-11-28T11:13:29.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.135+0000 7fe08d7fa700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0740682f0 0x7fe07406a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.135+0000 7fe08d7fa700 1 -- 172.21.15.28:0/1017381660 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fe07c084da0 con 0x7fe0900ceec0 2023-11-28T11:13:29.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.135+0000 7fe09487f700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0740682f0 0x7fe07406a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.135+0000 7fe09487f700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0740682f0 0x7fe07406a7a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe084009c50 tx=0x7fe084008040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:29.143 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.139+0000 7fe08d7fa700 1 -- 172.21.15.28:0/1017381660 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fe07c04b5e0 con 0x7fe0900ceec0 2023-11-28T11:13:29.333 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.332+0000 7fe096ae3700 1 -- 172.21.15.28:0/1017381660 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fe09006bcb0 con 0x7fe0740682f0 2023-11-28T11:13:29.334 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.332+0000 7fe08d7fa700 1 -- 172.21.15.28:0/1017381660 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe09006bcb0 con 0x7fe0740682f0 2023-11-28T11:13:29.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.334+0000 7fe096ae3700 1 -- 172.21.15.28:0/1017381660 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0740682f0 msgr2=0x7fe07406a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.334+0000 7fe096ae3700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0740682f0 0x7fe07406a7a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe084009c50 tx=0x7fe084008040 comp rx=0 tx=0).stop 2023-11-28T11:13:29.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.334+0000 7fe096ae3700 1 -- 172.21.15.28:0/1017381660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0900ceec0 msgr2=0x7fe090154c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.336 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.334+0000 7fe096ae3700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0900ceec0 0x7fe090154c70 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fe07c00c500 tx=0x7fe07c00c8c0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.334+0000 7fe096ae3700 1 -- 172.21.15.28:0/1017381660 shutdown_connections 2023-11-28T11:13:29.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.334+0000 7fe096ae3700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fe0740682f0 0x7fe07406a7a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.334+0000 7fe096ae3700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fe09015c560 0x7fe0901564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.334+0000 7fe096ae3700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe0900b2660 0x7fe0900acfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.334+0000 7fe096ae3700 1 --2- 172.21.15.28:0/1017381660 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fe0900ceec0 0x7fe090154c70 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.334+0000 7fe096ae3700 1 -- 172.21.15.28:0/1017381660 >> 172.21.15.28:0/1017381660 conn(0x7fe0900cd660 msgr2=0x7fe090066010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:29.337 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.334+0000 7fe096ae3700 1 -- 172.21.15.28:0/1017381660 shutdown_connections 2023-11-28T11:13:29.338 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.334+0000 7fe096ae3700 1 -- 172.21.15.28:0/1017381660 wait complete. 2023-11-28T11:13:29.338 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 6 to deep-scrub 2023-11-28T11:13:29.348 INFO:tasks.ceph:Scrubbing osd.7 2023-11-28T11:13:29.348 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-28T11:13:29.477 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.476+0000 7fb0d58b9700 1 Processor -- start 2023-11-28T11:13:29.478 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.476+0000 7fb0d58b9700 1 -- start start 2023-11-28T11:13:29.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.477+0000 7fb0d58b9700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb0d006d9d0 0x7fb0d015b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.477+0000 7fb0d58b9700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d006c3a0 con 0x7fb0d006bc30 2023-11-28T11:13:29.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.477+0000 7fb0d58b9700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0d006c0c0 con 0x7fb0d006dda0 2023-11-28T11:13:29.480 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.477+0000 7fb0d58b9700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0d015ab60 con 0x7fb0d006d9d0 2023-11-28T11:13:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.477+0000 7fb0ceffd700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fb0d006d9d0 0x7fb0d015b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.477+0000 7fb0ce7fc700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7fb0d006bc30 0x7fb0d0150eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40814/0 (socket says 172.21.15.28:40814) 2023-11-28T11:13:29.481 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.477+0000 7fb0ce7fc700 1 -- 172.21.15.28:0/3378792609 learned_addr learned my addr 172.21.15.28:0/3378792609 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.477+0000 7fb0ceffd700 1 -- 172.21.15.28:0/3378792609 >> v1:172.21.15.59:6789/0 conn(0x7fb0d006dda0 legacy=0x7fb0d00c9930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:29.482 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.477+0000 7fb0cdffb700 1 -- 172.21.15.28:0/3378792609 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4013695439 0 0) 0x7fb0d006c3a0 con 0x7fb0d006bc30 2023-11-28T11:13:29.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.477+0000 7fb0ceffd700 1 -- 172.21.15.28:0/3378792609 >> v1:172.21.15.28:6789/0 conn(0x7fb0d006bc30 legacy=0x7fb0d0150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.477+0000 7fb0ceffd700 1 -- 172.21.15.28:0/3378792609 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d015ae40 con 0x7fb0d006d9d0 2023-11-28T11:13:29.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.477+0000 7fb0ceffd700 1 --2- 172.21.15.28:0/3378792609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0d006d9d0 0x7fb0d015b640 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009840 tx=0x7fb0c4009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9947a391f7a9344 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:29.483 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.478+0000 7fb0cdffb700 1 -- 172.21.15.28:0/3378792609 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4011830 con 0x7fb0d006d9d0 2023-11-28T11:13:29.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.478+0000 7fb0cdffb700 1 -- 172.21.15.28:0/3378792609 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c4011990 con 0x7fb0d006d9d0 2023-11-28T11:13:29.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.478+0000 7fb0cdffb700 1 -- 172.21.15.28:0/3378792609 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c4011d20 con 0x7fb0d006d9d0 2023-11-28T11:13:29.484 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.478+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3378792609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0d006d9d0 msgr2=0x7fb0d015b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.478+0000 7fb0d58b9700 1 --2- 172.21.15.28:0/3378792609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0d006d9d0 0x7fb0d015b640 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fb0c4009840 tx=0x7fb0c4009b50 comp rx=0 tx=0).stop 2023-11-28T11:13:29.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.478+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3378792609 shutdown_connections 2023-11-28T11:13:29.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.478+0000 7fb0d58b9700 1 --2- 172.21.15.28:0/3378792609 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0d006d9d0 0x7fb0d015b640 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.485 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.478+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3378792609 >> 172.21.15.28:0/3378792609 conn(0x7fb0d00b0dd0 msgr2=0x7fb0d00b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:29.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.478+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3378792609 shutdown_connections 2023-11-28T11:13:29.486 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.478+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3378792609 wait complete. 2023-11-28T11:13:29.489 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.479+0000 7fb0d58b9700 1 Processor -- start 2023-11-28T11:13:29.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.479+0000 7fb0d58b9700 1 -- start start 2023-11-28T11:13:29.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.480+0000 7fb0d58b9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d006bc30 0x7fb0d0157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.480+0000 7fb0d58b9700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0d006d9d0 0x7fb0d00ce210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.480+0000 7fb0d58b9700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0d006dda0 0x7fb0d00ca7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.480+0000 7fb0d58b9700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d00ced30 con 0x7fb0d006d9d0 2023-11-28T11:13:29.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.480+0000 7fb0d58b9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d00c3160 con 0x7fb0d006bc30 2023-11-28T11:13:29.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.480+0000 7fb0d58b9700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0d00c3440 con 0x7fb0d006dda0 2023-11-28T11:13:29.490 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.480+0000 7fb0ce7fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0d006d9d0 0x7fb0d00ce210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.480+0000 7fb0ce7fc700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0d006d9d0 0x7fb0d00ce210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52764/0 (socket says 172.21.15.28:52764) 2023-11-28T11:13:29.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.480+0000 7fb0ce7fc700 1 -- 172.21.15.28:0/3622431172 learned_addr learned my addr 172.21.15.28:0/3622431172 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:29.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.480+0000 7fb0cf7fe700 1 --2- 172.21.15.28:0/3622431172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0d006dda0 0x7fb0d00ca7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.480+0000 7fb0ceffd700 1 --2- 172.21.15.28:0/3622431172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d006bc30 0x7fb0d0157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.481+0000 7fb0ce7fc700 1 -- 172.21.15.28:0/3622431172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0d006dda0 msgr2=0x7fb0d00ca7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.481+0000 7fb0ce7fc700 1 --2- 172.21.15.28:0/3622431172 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fb0d006dda0 0x7fb0d00ca7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.481+0000 7fb0ce7fc700 1 -- 172.21.15.28:0/3622431172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d006bc30 msgr2=0x7fb0d0157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.491 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.481+0000 7fb0ce7fc700 1 --2- 172.21.15.28:0/3622431172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d006bc30 0x7fb0d0157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.481+0000 7fb0ce7fc700 1 -- 172.21.15.28:0/3622431172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0c40094a0 con 0x7fb0d006d9d0 2023-11-28T11:13:29.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.481+0000 7fb0ceffd700 1 --2- 172.21.15.28:0/3622431172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb0d006bc30 0x7fb0d0157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:29.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.481+0000 7fb0ce7fc700 1 --2- 172.21.15.28:0/3622431172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0d006d9d0 0x7fb0d00ce210 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000c550 tx=0x7fb0c000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:29.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.481+0000 7fb0b7fff700 1 -- 172.21.15.28:0/3622431172 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0004c80 con 0x7fb0d006d9d0 2023-11-28T11:13:29.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.481+0000 7fb0b7fff700 1 -- 172.21.15.28:0/3622431172 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0c0004de0 con 0x7fb0d006d9d0 2023-11-28T11:13:29.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.481+0000 7fb0b7fff700 1 -- 172.21.15.28:0/3622431172 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0c0007310 con 0x7fb0d006d9d0 2023-11-28T11:13:29.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.481+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3622431172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0d006fac0 con 0x7fb0d006d9d0 2023-11-28T11:13:29.492 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.481+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3622431172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0d015b120 con 0x7fb0d006d9d0 2023-11-28T11:13:29.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.482+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3622431172 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb0d01515c0 con 0x7fb0d006d9d0 2023-11-28T11:13:29.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.482+0000 7fb0b7fff700 1 -- 172.21.15.28:0/3622431172 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fb0c001a020 con 0x7fb0d006d9d0 2023-11-28T11:13:29.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.483+0000 7fb0b7fff700 1 --2- 172.21.15.28:0/3622431172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb0bc0682f0 0x7fb0bc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.483+0000 7fb0b7fff700 1 -- 172.21.15.28:0/3622431172 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fb0c0085660 con 0x7fb0d006d9d0 2023-11-28T11:13:29.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.483+0000 7fb0ceffd700 1 --2- 172.21.15.28:0/3622431172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb0bc0682f0 0x7fb0bc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.483+0000 7fb0b7fff700 1 --2- 172.21.15.28:0/3622431172 >> [v2:172.21.15.59:6824/103734,v1:172.21.15.59:6825/103734] conn(0x7fb0bc06eb90 0x7fb0bc070fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.483+0000 7fb0b7fff700 1 -- 172.21.15.28:0/3622431172 --> [v2:172.21.15.59:6824/103734,v1:172.21.15.59:6825/103734] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb0bc0716b0 con 0x7fb0bc06eb90 2023-11-28T11:13:29.493 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.483+0000 7fb0b7fff700 1 -- 172.21.15.28:0/3622431172 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb0c0087d10 con 0x7fb0d006d9d0 2023-11-28T11:13:29.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.483+0000 7fb0cf7fe700 1 --2- 172.21.15.28:0/3622431172 >> [v2:172.21.15.59:6824/103734,v1:172.21.15.59:6825/103734] conn(0x7fb0bc06eb90 0x7fb0bc070fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.483+0000 7fb0ceffd700 1 --2- 172.21.15.28:0/3622431172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb0bc0682f0 0x7fb0bc06a7a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40090b0 tx=0x7fb0c4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:29.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.484+0000 7fb0cf7fe700 1 --2- 172.21.15.28:0/3622431172 >> [v2:172.21.15.59:6824/103734,v1:172.21.15.59:6825/103734] conn(0x7fb0bc06eb90 0x7fb0bc070fe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:29.494 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.484+0000 7fb0b7fff700 1 -- 172.21.15.28:0/3622431172 <== osd.7 v2:172.21.15.59:6824/103734 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fb0bc0716b0 con 0x7fb0bc06eb90 2023-11-28T11:13:29.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.499+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3622431172 --> [v2:172.21.15.59:6824/103734,v1:172.21.15.59:6825/103734] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb0d015dac0 con 0x7fb0bc06eb90 2023-11-28T11:13:29.500 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.499+0000 7fb0b7fff700 1 -- 172.21.15.28:0/3622431172 <== osd.7 v2:172.21.15.59:6824/103734 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb0d015dac0 con 0x7fb0bc06eb90 2023-11-28T11:13:29.501 INFO:teuthology.orchestra.run.smithi028.stdout:{ 2023-11-28T11:13:29.501 INFO:teuthology.orchestra.run.smithi028.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T11:13:29.501 INFO:teuthology.orchestra.run.smithi028.stdout:} 2023-11-28T11:13:29.502 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.500+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3622431172 >> [v2:172.21.15.59:6824/103734,v1:172.21.15.59:6825/103734] conn(0x7fb0bc06eb90 msgr2=0x7fb0bc070fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.500+0000 7fb0d58b9700 1 --2- 172.21.15.28:0/3622431172 >> [v2:172.21.15.59:6824/103734,v1:172.21.15.59:6825/103734] conn(0x7fb0bc06eb90 0x7fb0bc070fe0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.500+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3622431172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb0bc0682f0 msgr2=0x7fb0bc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.500+0000 7fb0d58b9700 1 --2- 172.21.15.28:0/3622431172 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fb0bc0682f0 0x7fb0bc06a7a0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fb0c40090b0 tx=0x7fb0c4008030 comp rx=0 tx=0).stop 2023-11-28T11:13:29.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.500+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3622431172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0d006d9d0 msgr2=0x7fb0d00ce210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.500+0000 7fb0d58b9700 1 --2- 172.21.15.28:0/3622431172 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fb0d006d9d0 0x7fb0d00ce210 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fb0c000c550 tx=0x7fb0c000c910 comp rx=0 tx=0).stop 2023-11-28T11:13:29.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.500+0000 7fb0cf7fe700 1 -- 172.21.15.28:0/3622431172 reap_dead start 2023-11-28T11:13:29.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.500+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3622431172 shutdown_connections 2023-11-28T11:13:29.503 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.500+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3622431172 >> 172.21.15.28:0/3622431172 conn(0x7fb0d00b0dd0 msgr2=0x7fb0d014fb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:29.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.500+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3622431172 shutdown_connections 2023-11-28T11:13:29.504 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.500+0000 7fb0d58b9700 1 -- 172.21.15.28:0/3622431172 wait complete. 2023-11-28T11:13:29.513 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-28T11:13:29.638 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.637+0000 7f677a1b7700 1 Processor -- start 2023-11-28T11:13:29.639 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.637+0000 7f677a1b7700 1 -- start start 2023-11-28T11:13:29.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.638+0000 7f677a1b7700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f677415c990 0x7f67740cf070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.638+0000 7f677a1b7700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67740ce340 con 0x7f67740cc310 2023-11-28T11:13:29.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.638+0000 7f677a1b7700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67740cebe0 con 0x7f6774161090 2023-11-28T11:13:29.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.638+0000 7f677a1b7700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67740cb3f0 con 0x7f677415c990 2023-11-28T11:13:29.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.638+0000 7f67737fe700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f677415c990 0x7f67740cf070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.641 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.638+0000 7f6773fff700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f67740cc310 0x7f67740cc770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40834/0 (socket says 172.21.15.28:40834) 2023-11-28T11:13:29.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.638+0000 7f6773fff700 1 -- 172.21.15.28:0/4168967361 learned_addr learned my addr 172.21.15.28:0/4168967361 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:29.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.638+0000 7f67737fe700 1 -- 172.21.15.28:0/4168967361 >> v1:172.21.15.59:6789/0 conn(0x7f6774161090 legacy=0x7f67740ca970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:29.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.638+0000 7f67737fe700 1 -- 172.21.15.28:0/4168967361 >> v1:172.21.15.28:6789/0 conn(0x7f67740cc310 legacy=0x7f67740cc770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.638+0000 7f67727fc700 1 -- 172.21.15.28:0/4168967361 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 542851844 0 0) 0x7f67740ce340 con 0x7f67740cc310 2023-11-28T11:13:29.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.638+0000 7f67737fe700 1 -- 172.21.15.28:0/4168967361 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67740ce590 con 0x7f677415c990 2023-11-28T11:13:29.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.638+0000 7f67737fe700 1 --2- 172.21.15.28:0/4168967361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f677415c990 0x7f67740cf070 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6764009840 tx=0x7f6764009b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9423641bee77a797 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:29.642 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.639+0000 7f67727fc700 1 -- 172.21.15.28:0/4168967361 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6764004230 con 0x7f677415c990 2023-11-28T11:13:29.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.639+0000 7f67727fc700 1 -- 172.21.15.28:0/4168967361 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6764004390 con 0x7f677415c990 2023-11-28T11:13:29.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.639+0000 7f67727fc700 1 -- 172.21.15.28:0/4168967361 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67640036a0 con 0x7f677415c990 2023-11-28T11:13:29.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.639+0000 7f677a1b7700 1 -- 172.21.15.28:0/4168967361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f677415c990 msgr2=0x7f67740cf070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.639+0000 7f677a1b7700 1 --2- 172.21.15.28:0/4168967361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f677415c990 0x7f67740cf070 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f6764009840 tx=0x7f6764009b50 comp rx=0 tx=0).stop 2023-11-28T11:13:29.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.639+0000 7f677a1b7700 1 -- 172.21.15.28:0/4168967361 shutdown_connections 2023-11-28T11:13:29.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.639+0000 7f677a1b7700 1 --2- 172.21.15.28:0/4168967361 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f677415c990 0x7f67740cf070 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.639+0000 7f677a1b7700 1 -- 172.21.15.28:0/4168967361 >> 172.21.15.28:0/4168967361 conn(0x7f67740b1650 msgr2=0x7f67740b1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:29.643 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.639+0000 7f677a1b7700 1 -- 172.21.15.28:0/4168967361 shutdown_connections 2023-11-28T11:13:29.644 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.639+0000 7f677a1b7700 1 -- 172.21.15.28:0/4168967361 wait complete. 2023-11-28T11:13:29.646 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.640+0000 7f677a1b7700 1 Processor -- start 2023-11-28T11:13:29.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.640+0000 7f677a1b7700 1 -- start start 2023-11-28T11:13:29.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.641+0000 7f677a1b7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67740cc310 0x7f677415bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.641+0000 7f677a1b7700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f677415c990 0x7f677415e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.641+0000 7f677a1b7700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6774161090 0x7f67741f20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.641+0000 7f677a1b7700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67741f2a00 con 0x7f6774161090 2023-11-28T11:13:29.647 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.641+0000 7f677a1b7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67741f2ce0 con 0x7f67740cc310 2023-11-28T11:13:29.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.641+0000 7f677a1b7700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67741f2fc0 con 0x7f677415c990 2023-11-28T11:13:29.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.641+0000 7f6773fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6774161090 0x7f67741f20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.648 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.641+0000 7f6773fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6774161090 0x7f67741f20a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52782/0 (socket says 172.21.15.28:52782) 2023-11-28T11:13:29.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.641+0000 7f6773fff700 1 -- 172.21.15.28:0/2017125489 learned_addr learned my addr 172.21.15.28:0/2017125489 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:29.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.641+0000 7f67737fe700 1 --2- 172.21.15.28:0/2017125489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67740cc310 0x7f677415bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.641+0000 7f6772ffd700 1 --2- 172.21.15.28:0/2017125489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f677415c990 0x7f677415e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.642+0000 7f6773fff700 1 -- 172.21.15.28:0/2017125489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f677415c990 msgr2=0x7f677415e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.642+0000 7f6773fff700 1 --2- 172.21.15.28:0/2017125489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f677415c990 0x7f677415e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.642+0000 7f6773fff700 1 -- 172.21.15.28:0/2017125489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67740cc310 msgr2=0x7f677415bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.642+0000 7f6773fff700 1 --2- 172.21.15.28:0/2017125489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67740cc310 0x7f677415bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.649 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.642+0000 7f6773fff700 1 -- 172.21.15.28:0/2017125489 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67640094a0 con 0x7f6774161090 2023-11-28T11:13:29.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.642+0000 7f67737fe700 1 --2- 172.21.15.28:0/2017125489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67740cc310 0x7f677415bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:29.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.642+0000 7f6773fff700 1 --2- 172.21.15.28:0/2017125489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6774161090 0x7f67741f20a0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f676c00cb60 tx=0x7f676c00ce70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:29.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.642+0000 7f675bfff700 1 -- 172.21.15.28:0/2017125489 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c00aaa0 con 0x7f6774161090 2023-11-28T11:13:29.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.642+0000 7f675bfff700 1 -- 172.21.15.28:0/2017125489 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676c00ac00 con 0x7f6774161090 2023-11-28T11:13:29.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.642+0000 7f677a1b7700 1 -- 172.21.15.28:0/2017125489 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67740cb080 con 0x7f6774161090 2023-11-28T11:13:29.650 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.642+0000 7f675bfff700 1 -- 172.21.15.28:0/2017125489 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c0070d0 con 0x7f6774161090 2023-11-28T11:13:29.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.642+0000 7f677a1b7700 1 -- 172.21.15.28:0/2017125489 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67740ce870 con 0x7f6774161090 2023-11-28T11:13:29.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.643+0000 7f677a1b7700 1 -- 172.21.15.28:0/2017125489 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67740cf5b0 con 0x7f6774161090 2023-11-28T11:13:29.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.644+0000 7f675bfff700 1 -- 172.21.15.28:0/2017125489 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f676c01a030 con 0x7f6774161090 2023-11-28T11:13:29.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.644+0000 7f675bfff700 1 --2- 172.21.15.28:0/2017125489 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f67600682f0 0x7f676006a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.644+0000 7f675bfff700 1 -- 172.21.15.28:0/2017125489 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f676c0856b0 con 0x7f6774161090 2023-11-28T11:13:29.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.644+0000 7f67737fe700 1 --2- 172.21.15.28:0/2017125489 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f67600682f0 0x7f676006a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.651 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.644+0000 7f67737fe700 1 --2- 172.21.15.28:0/2017125489 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f67600682f0 0x7f676006a7a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f67640090b0 tx=0x7f6764008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:29.652 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.648+0000 7f675bfff700 1 -- 172.21.15.28:0/2017125489 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f676c0520d0 con 0x7f6774161090 2023-11-28T11:13:29.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.839+0000 7f677a1b7700 1 -- 172.21.15.28:0/2017125489 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f67741f2e50 con 0x7f67600682f0 2023-11-28T11:13:29.841 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.840+0000 7f675bfff700 1 -- 172.21.15.28:0/2017125489 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f67741f2e50 con 0x7f67600682f0 2023-11-28T11:13:29.844 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.842+0000 7f677a1b7700 1 -- 172.21.15.28:0/2017125489 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f67600682f0 msgr2=0x7f676006a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.844 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.842+0000 7f677a1b7700 1 --2- 172.21.15.28:0/2017125489 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f67600682f0 0x7f676006a7a0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f67640090b0 tx=0x7f6764008030 comp rx=0 tx=0).stop 2023-11-28T11:13:29.844 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.842+0000 7f677a1b7700 1 -- 172.21.15.28:0/2017125489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6774161090 msgr2=0x7f67741f20a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.844 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.842+0000 7f677a1b7700 1 --2- 172.21.15.28:0/2017125489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6774161090 0x7f67741f20a0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7f676c00cb60 tx=0x7f676c00ce70 comp rx=0 tx=0).stop 2023-11-28T11:13:29.844 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.842+0000 7f677a1b7700 1 -- 172.21.15.28:0/2017125489 shutdown_connections 2023-11-28T11:13:29.844 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.842+0000 7f677a1b7700 1 --2- 172.21.15.28:0/2017125489 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f67600682f0 0x7f676006a7a0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.842+0000 7f677a1b7700 1 --2- 172.21.15.28:0/2017125489 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f6774161090 0x7f67741f20a0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.842+0000 7f677a1b7700 1 --2- 172.21.15.28:0/2017125489 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67740cc310 0x7f677415bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.842+0000 7f677a1b7700 1 --2- 172.21.15.28:0/2017125489 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f677415c990 0x7f677415e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.842+0000 7f677a1b7700 1 -- 172.21.15.28:0/2017125489 >> 172.21.15.28:0/2017125489 conn(0x7f67740b1650 msgr2=0x7f67740aedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:29.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.842+0000 7f677a1b7700 1 -- 172.21.15.28:0/2017125489 shutdown_connections 2023-11-28T11:13:29.845 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.842+0000 7f677a1b7700 1 -- 172.21.15.28:0/2017125489 wait complete. 2023-11-28T11:13:29.845 INFO:teuthology.orchestra.run.smithi028.stderr:instructed osd(s) 7 to deep-scrub 2023-11-28T11:13:29.856 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:13:29.980 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.979+0000 7f32fef3c700 1 Processor -- start 2023-11-28T11:13:29.981 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.979+0000 7f32fef3c700 1 -- start start 2023-11-28T11:13:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32fef3c700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32f80b2660 0x7f32f80ce7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32fef3c700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f32f80cecf0 0x7f32f806c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32fef3c700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32f80b30b0 con 0x7f32f815c560 2023-11-28T11:13:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32fef3c700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f806c020 con 0x7f32f80b2660 2023-11-28T11:13:29.983 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32fef3c700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32f80cf110 con 0x7f32f80cecf0 2023-11-28T11:13:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32f7fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32f80b2660 0x7f32f80ce7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32f7fff700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7f32f80b2660 0x7f32f80ce7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52788/0 (socket says 172.21.15.28:52788) 2023-11-28T11:13:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32f7fff700 1 -- 172.21.15.28:0/1934815750 learned_addr learned my addr 172.21.15.28:0/1934815750 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32fd4d9700 1 --2- 172.21.15.28:0/1934815750 >> v2:172.21.15.59:3300/0 conn(0x7f32f80cecf0 0x7f32f806c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32f7fff700 1 -- 172.21.15.28:0/1934815750 >> v2:172.21.15.59:3300/0 conn(0x7f32f80cecf0 msgr2=0x7f32f806c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32f7fff700 1 --2- 172.21.15.28:0/1934815750 >> v2:172.21.15.59:3300/0 conn(0x7f32f80cecf0 0x7f32f806c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.984 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32f7fff700 1 -- 172.21.15.28:0/1934815750 >> v1:172.21.15.59:6789/0 conn(0x7f32f815c560 legacy=0x7f32f806f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32f7fff700 1 -- 172.21.15.28:0/1934815750 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32f80d09c0 con 0x7f32f80b2660 2023-11-28T11:13:29.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32f7fff700 1 --2- 172.21.15.28:0/1934815750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32f80b2660 0x7f32f80ce7b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f32f80ce210 tx=0x7f32e4009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ad1081b57c100c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:29.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32f77fe700 1 -- 172.21.15.28:0/1934815750 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e400b870 con 0x7f32f80b2660 2023-11-28T11:13:29.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32f77fe700 1 -- 172.21.15.28:0/1934815750 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32e400b9d0 con 0x7f32f80b2660 2023-11-28T11:13:29.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.980+0000 7f32f77fe700 1 -- 172.21.15.28:0/1934815750 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e400bd60 con 0x7f32f80b2660 2023-11-28T11:13:29.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.981+0000 7f32fef3c700 1 -- 172.21.15.28:0/1934815750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32f80b2660 msgr2=0x7f32f80ce7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.981+0000 7f32fef3c700 1 --2- 172.21.15.28:0/1934815750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32f80b2660 0x7f32f80ce7b0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7f32f80ce210 tx=0x7f32e4009bd0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.985 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.981+0000 7f32fef3c700 1 -- 172.21.15.28:0/1934815750 shutdown_connections 2023-11-28T11:13:29.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.981+0000 7f32fef3c700 1 --2- 172.21.15.28:0/1934815750 >> v2:172.21.15.59:3300/0 conn(0x7f32f80cecf0 0x7f32f806c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.981+0000 7f32fef3c700 1 --2- 172.21.15.28:0/1934815750 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32f80b2660 0x7f32f80ce7b0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.981+0000 7f32fef3c700 1 -- 172.21.15.28:0/1934815750 >> 172.21.15.28:0/1934815750 conn(0x7f32f80cd660 msgr2=0x7f32f80cda60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:29.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.981+0000 7f32fef3c700 1 -- 172.21.15.28:0/1934815750 shutdown_connections 2023-11-28T11:13:29.986 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.981+0000 7f32fef3c700 1 -- 172.21.15.28:0/1934815750 wait complete. 2023-11-28T11:13:29.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.982+0000 7f32fef3c700 1 Processor -- start 2023-11-28T11:13:29.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.982+0000 7f32fef3c700 1 -- start start 2023-11-28T11:13:29.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.983+0000 7f32fef3c700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32f80b2660 0x7f32f8158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.983+0000 7f32fef3c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32f80cecf0 0x7f32f81f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.989 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.983+0000 7f32fef3c700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32f815c560 0x7f32f81f3150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.983+0000 7f32fef3c700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f815c930 con 0x7f32f80b2660 2023-11-28T11:13:29.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.983+0000 7f32fef3c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f815ceb0 con 0x7f32f80cecf0 2023-11-28T11:13:29.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.983+0000 7f32fef3c700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32f81f3740 con 0x7f32f815c560 2023-11-28T11:13:29.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.983+0000 7f32fccd8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32f80b2660 0x7f32f8158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.983+0000 7f32fccd8700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32f80b2660 0x7f32f8158180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52792/0 (socket says 172.21.15.28:52792) 2023-11-28T11:13:29.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.983+0000 7f32fccd8700 1 -- 172.21.15.28:0/786681165 learned_addr learned my addr 172.21.15.28:0/786681165 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:29.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.983+0000 7f32f7fff700 1 --2- 172.21.15.28:0/786681165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32f80cecf0 0x7f32f81f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.990 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.983+0000 7f32fd4d9700 1 --2- 172.21.15.28:0/786681165 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32f815c560 0x7f32f81f3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.984+0000 7f32fccd8700 1 -- 172.21.15.28:0/786681165 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32f815c560 msgr2=0x7f32f81f3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.984+0000 7f32fccd8700 1 --2- 172.21.15.28:0/786681165 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32f815c560 0x7f32f81f3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.984+0000 7f32fccd8700 1 -- 172.21.15.28:0/786681165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32f80cecf0 msgr2=0x7f32f81f2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:29.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.984+0000 7f32fccd8700 1 --2- 172.21.15.28:0/786681165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32f80cecf0 0x7f32f81f2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:29.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.984+0000 7f32fccd8700 1 -- 172.21.15.28:0/786681165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32e4009550 con 0x7f32f80b2660 2023-11-28T11:13:29.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.984+0000 7f32fccd8700 1 --2- 172.21.15.28:0/786681165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32f80b2660 0x7f32f8158180 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00c450 tx=0x7f32ec00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:29.991 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.984+0000 7f32f57fa700 1 -- 172.21.15.28:0/786681165 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec0052d0 con 0x7f32f80b2660 2023-11-28T11:13:29.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.984+0000 7f32f57fa700 1 -- 172.21.15.28:0/786681165 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32ec005430 con 0x7f32f80b2660 2023-11-28T11:13:29.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.984+0000 7f32fef3c700 1 -- 172.21.15.28:0/786681165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f80b2dd0 con 0x7f32f80b2660 2023-11-28T11:13:29.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.984+0000 7f32f57fa700 1 -- 172.21.15.28:0/786681165 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32ec0038c0 con 0x7f32f80b2660 2023-11-28T11:13:29.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.984+0000 7f32fef3c700 1 -- 172.21.15.28:0/786681165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f80b2af0 con 0x7f32f80b2660 2023-11-28T11:13:29.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.985+0000 7f32fef3c700 1 -- 172.21.15.28:0/786681165 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f806bcb0 con 0x7f32f80b2660 2023-11-28T11:13:29.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.985+0000 7f32f57fa700 1 -- 172.21.15.28:0/786681165 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f32ec019030 con 0x7f32f80b2660 2023-11-28T11:13:29.992 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.986+0000 7f32f57fa700 1 --2- 172.21.15.28:0/786681165 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32dc0682f0 0x7f32dc06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:29.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.986+0000 7f32f57fa700 1 -- 172.21.15.28:0/786681165 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f32ec084d90 con 0x7f32f80b2660 2023-11-28T11:13:29.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.986+0000 7f32f7fff700 1 --2- 172.21.15.28:0/786681165 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32dc0682f0 0x7f32dc06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:29.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.986+0000 7f32f7fff700 1 --2- 172.21.15.28:0/786681165 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32dc0682f0 0x7f32dc06a7a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000c00 tx=0x7f32e4008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:29.993 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:29.990+0000 7f32f57fa700 1 -- 172.21.15.28:0/786681165 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f32ec0516e0 con 0x7f32f80b2660 2023-11-28T11:13:30.178 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:30.177+0000 7f32fef3c700 1 -- 172.21.15.28:0/786681165 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32f815cff0 con 0x7f32dc0682f0 2023-11-28T11:13:30.181 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:30.180+0000 7f32f57fa700 1 -- 172.21.15.28:0/786681165 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72629 (secure 0 0 0) 0x7f32e400bb40 con 0x7f32dc0682f0 2023-11-28T11:13:30.182 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:30.183 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:30.182+0000 7f32fef3c700 1 -- 172.21.15.28:0/786681165 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32dc0682f0 msgr2=0x7f32dc06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:30.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:30.182+0000 7f32fef3c700 1 --2- 172.21.15.28:0/786681165 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32dc0682f0 0x7f32dc06a7a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f32e4000c00 tx=0x7f32e4008030 comp rx=0 tx=0).stop 2023-11-28T11:13:30.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:30.182+0000 7f32fef3c700 1 -- 172.21.15.28:0/786681165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32f80b2660 msgr2=0x7f32f8158180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:30.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:30.182+0000 7f32fef3c700 1 --2- 172.21.15.28:0/786681165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32f80b2660 0x7f32f8158180 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7f32ec00c450 tx=0x7f32ec00c810 comp rx=0 tx=0).stop 2023-11-28T11:13:30.184 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:30.182+0000 7f32fef3c700 1 -- 172.21.15.28:0/786681165 shutdown_connections 2023-11-28T11:13:30.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:30.182+0000 7f32fef3c700 1 --2- 172.21.15.28:0/786681165 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f32dc0682f0 0x7f32dc06a7a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:30.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:30.182+0000 7f32fef3c700 1 --2- 172.21.15.28:0/786681165 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f32f80b2660 0x7f32f8158180 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:30.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:30.182+0000 7f32fef3c700 1 --2- 172.21.15.28:0/786681165 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f32f815c560 0x7f32f81f3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:30.185 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:30.182+0000 7f32fef3c700 1 --2- 172.21.15.28:0/786681165 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f32f80cecf0 0x7f32f81f2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:30.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:30.182+0000 7f32fef3c700 1 -- 172.21.15.28:0/786681165 >> 172.21.15.28:0/786681165 conn(0x7f32f80cd660 msgr2=0x7f32f80ccf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:30.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:30.182+0000 7f32fef3c700 1 -- 172.21.15.28:0/786681165 shutdown_connections 2023-11-28T11:13:30.186 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:30.182+0000 7f32fef3c700 1 -- 172.21.15.28:0/786681165 wait complete. 2023-11-28T11:13:30.187 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:13:30.196 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":210,"stamp":"2023-11-28T11:13:29.228360+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":82180,"kb_used_data":7940,"kb_used_omap":0,"kb_used_meta":74176,"kb_avail":754892540,"statfs":{"total":773094113280,"available":773009960960,"internally_reserved":0,"allocated":8130560,"data_stored":4447567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":75956224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":-1,"num_object_clones":0,"num_object_copies":-2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-1,"num_whiteouts":0,"num_read":-2,"num_read_kb":-2,"num_write":-3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.867161"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:13:21.353730+0000","last_change":"2023-11-28T11:08:40.522601+0000","last_active":"2023-11-28T11:13:21.353730+0000","last_peered":"2023-11-28T11:13:21.353730+0000","last_clean":"2023-11-28T11:13:21.353730+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:13:21.353730+0000","last_undegraded":"2023-11-28T11:13:21.353730+0000","last_fullsized":"2023-11-28T11:13:21.353730+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":60,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:13:21.561436+0000","last_change":"2023-11-28T11:08:40.384837+0000","last_active":"2023-11-28T11:13:21.561436+0000","last_peered":"2023-11-28T11:13:21.561436+0000","last_clean":"2023-11-28T11:13:21.561436+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:13:21.561436+0000","last_undegraded":"2023-11-28T11:13:21.561436+0000","last_fullsized":"2023-11-28T11:13:21.561436+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":60,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:13:21.367420+0000","last_change":"2023-11-28T11:08:40.388436+0000","last_active":"2023-11-28T11:13:21.367420+0000","last_peered":"2023-11-28T11:13:21.367420+0000","last_clean":"2023-11-28T11:13:21.367420+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:13:21.367420+0000","last_undegraded":"2023-11-28T11:13:21.367420+0000","last_fullsized":"2023-11-28T11:13:21.367420+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":208,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:13:20.422426+0000","last_change":"2023-11-28T11:08:40.388673+0000","last_active":"2023-11-28T11:13:20.422426+0000","last_peered":"2023-11-28T11:13:20.422426+0000","last_clean":"2023-11-28T11:13:20.422426+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:13:20.422426+0000","last_undegraded":"2023-11-28T11:13:20.422426+0000","last_fullsized":"2023-11-28T11:13:20.422426+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:51:11.497525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:13:20.422884+0000","last_change":"2023-11-28T11:08:40.832135+0000","last_active":"2023-11-28T11:13:20.422884+0000","last_peered":"2023-11-28T11:13:20.422884+0000","last_clean":"2023-11-28T11:13:20.422884+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:13:20.422884+0000","last_undegraded":"2023-11-28T11:13:20.422884+0000","last_fullsized":"2023-11-28T11:13:20.422884+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:24:57.048483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:13:20.422752+0000","last_change":"2023-11-28T11:08:40.831989+0000","last_active":"2023-11-28T11:13:20.422752+0000","last_peered":"2023-11-28T11:13:20.422752+0000","last_clean":"2023-11-28T11:13:20.422752+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:13:20.422752+0000","last_undegraded":"2023-11-28T11:13:20.422752+0000","last_fullsized":"2023-11-28T11:13:20.422752+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T11:50:16.341995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":63,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:13:20.364783+0000","last_change":"2023-11-28T11:08:40.832320+0000","last_active":"2023-11-28T11:13:20.364783+0000","last_peered":"2023-11-28T11:13:20.364783+0000","last_clean":"2023-11-28T11:13:20.364783+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:13:20.364783+0000","last_undegraded":"2023-11-28T11:13:20.364783+0000","last_fullsized":"2023-11-28T11:13:20.364783+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:20:32.519309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:13:20.413818+0000","last_change":"2023-11-28T11:08:40.832013+0000","last_active":"2023-11-28T11:13:20.413818+0000","last_peered":"2023-11-28T11:13:20.413818+0000","last_clean":"2023-11-28T11:13:20.413818+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:13:20.413818+0000","last_undegraded":"2023-11-28T11:13:20.413818+0000","last_fullsized":"2023-11-28T11:13:20.413818+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T14:13:17.886098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":58,"reported_epoch":70,"state":"active+clean","last_fresh":"2023-11-28T11:13:20.414463+0000","last_change":"2023-11-28T11:08:40.387795+0000","last_active":"2023-11-28T11:13:20.414463+0000","last_peered":"2023-11-28T11:13:20.414463+0000","last_clean":"2023-11-28T11:13:20.414463+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:13:20.414463+0000","last_undegraded":"2023-11-28T11:13:20.414463+0000","last_fullsized":"2023-11-28T11:13:20.414463+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:08:39.371027+0000","last_clean_scrub_stamp":"2023-11-28T11:08:39.371027+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T18:05:21.247860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574910,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11100,"kb_used_data":1428,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360740,"statfs":{"total":96636764160,"available":96625397760,"internally_reserved":0,"allocated":1462272,"data_stored":998842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.57099999999999995,"15min":0.63100000000000001},"min":{"1min":0.38700000000000001,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.72599999999999998,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.59399999999999997,"15min":0.63300000000000001},"min":{"1min":0.309,"5min":0.307,"15min":0.307},"max":{"1min":0.753,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.40699999999999997}]},{"osd":1,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.61299999999999999,"15min":0.67100000000000004},"min":{"1min":0.38,"5min":0.316,"15min":0.316},"max":{"1min":0.83399999999999996,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58299999999999996,"15min":0.59299999999999997},"min":{"1min":0.41199999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.73699999999999999,"5min":1.224,"15min":1.224},"last":0.56299999999999994}]},{"osd":2,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.63100000000000001,"15min":0.67900000000000005},"min":{"1min":0.38100000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.84999999999999998,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.69499999999999995,"15min":0.76300000000000001},"min":{"1min":0.44900000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.88200000000000001,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.58899999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64400000000000002,"15min":0.68999999999999995},"min":{"1min":0.45800000000000002,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.76500000000000001,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.69699999999999995,"15min":0.75700000000000001},"min":{"1min":0.48499999999999999,"5min":0.439,"15min":0.439},"max":{"1min":0.78600000000000003,"5min":3.294,"15min":3.294},"last":0.69299999999999995}]},{"osd":4,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.53400000000000003,"15min":0.54700000000000004},"min":{"1min":0.373,"5min":0.312,"15min":0.312},"max":{"1min":0.77600000000000002,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.497},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.55700000000000005,"15min":0.56200000000000006},"min":{"1min":0.34399999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.79500000000000004,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.63800000000000001},"min":{"1min":0.41299999999999998,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.81100000000000005,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.628,"5min":0.63300000000000001,"15min":0.64500000000000002},"min":{"1min":0.434,"5min":0.373,"15min":0.373},"max":{"1min":0.86599999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65700000000000003,"15min":0.67300000000000004},"min":{"1min":0.47099999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.85599999999999998,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.626},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.40000000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.66500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574909,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9944,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361896,"statfs":{"total":96636764160,"available":96626581504,"internally_reserved":0,"allocated":868352,"data_stored":408474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.51100000000000001,"15min":0.47099999999999997},"min":{"1min":0.376,"5min":0.252,"15min":0.252},"max":{"1min":0.79300000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.51200000000000001,"15min":0.48599999999999999},"min":{"1min":0.46200000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77400000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.65100000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55200000000000005,"15min":0.52000000000000002},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80300000000000005,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.504},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81799999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.622}]},{"osd":2,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.37,"5min":0.312,"15min":0.312},"max":{"1min":0.72599999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64100000000000001,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80200000000000005,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.71499999999999997}]},{"osd":3,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67100000000000004,"15min":0.71199999999999997},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84199999999999997,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67200000000000004,"15min":0.69499999999999995},"min":{"1min":0.51200000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83199999999999996,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.72999999999999998}]},{"osd":4,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73999999999999999,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60199999999999998,"15min":0.622},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84999999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.44600000000000001}]},{"osd":5,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.623,"15min":0.629},"min":{"1min":0.437,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.55100000000000005}]},{"osd":7,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.45300000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68000000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82399999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.57199999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574909,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11036,"kb_used_data":1428,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360804,"statfs":{"total":96636764160,"available":96625463296,"internally_reserved":0,"allocated":1462272,"data_stored":998842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.33100000000000002},"min":{"1min":0.255,"5min":0.19,"15min":0.19},"max":{"1min":0.501,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55600000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.42899999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.34100000000000003,"15min":0.32300000000000001},"min":{"1min":0.26300000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.51400000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39000000000000001,"15min":0.38300000000000001},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.45300000000000001}]},{"osd":3,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.39700000000000002},"min":{"1min":0.35799999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60499999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42799999999999999,"15min":0.41399999999999998},"min":{"1min":0.32900000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.48899999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56999999999999995,"15min":0.54000000000000004},"min":{"1min":0.44900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.46400000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.014},"last":0.60499999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.47699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.57099999999999995},"min":{"1min":0.48999999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.61699999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.621,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0389999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.52100000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88200000000000001,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.64200000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63,"15min":0.60899999999999999},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.63},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63200000000000001,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.59299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574910,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10136,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361704,"statfs":{"total":96636764160,"available":96626384896,"internally_reserved":0,"allocated":868352,"data_stored":408474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33600000000000002,"15min":0.32900000000000001},"min":{"1min":0.25,"5min":0.218,"15min":0.218},"max":{"1min":0.501,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.374},"min":{"1min":0.30299999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.53800000000000003,"5min":0.622,"15min":0.622},"last":0.29799999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38100000000000001,"15min":0.38300000000000001},"min":{"1min":0.28899999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39700000000000002,"15min":0.39900000000000002},"min":{"1min":0.28899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61699999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.41299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.372},"min":{"1min":0.30499999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.35699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.59899999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.43099999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.46400000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.74399999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.55400000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66100000000000003,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71499999999999997,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81599999999999995,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.82299999999999995}]},{"osd":6,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.51900000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.79200000000000004,"5min":1.016,"15min":1.016},"last":0.68999999999999995}]},{"osd":7,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46700000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78700000000000003,"5min":1.105,"15min":1.105},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65300000000000002,"15min":0.64400000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80800000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.70799999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574910,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9876,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361964,"statfs":{"total":96636764160,"available":96626651136,"internally_reserved":0,"allocated":864256,"data_stored":407513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35799999999999998,"15min":0.371},"min":{"1min":0.247,"5min":0.214,"15min":0.214},"max":{"1min":0.50700000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.371,"5min":0.38100000000000001,"15min":0.38700000000000001},"min":{"1min":0.22800000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.54900000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.27200000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.29599999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.54100000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.42099999999999999,"15min":0.441},"min":{"1min":0.29399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57899999999999996,"5min":1.528,"15min":1.528},"last":0.34899999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38300000000000001,"15min":0.39400000000000002},"min":{"1min":0.27700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.51400000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.315},{"interface":"front","average":{"1min":0.432,"5min":0.45600000000000002,"15min":0.47599999999999998},"min":{"1min":0.312,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55300000000000005,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.376}]},{"osd":4,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80700000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.46500000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.55800000000000005}]},{"osd":5,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.63700000000000001},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93899999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.48199999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.62,"15min":0.622},"min":{"1min":0.48499999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.433},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.66800000000000004},"min":{"1min":0.441,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85399999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.628,"15min":0.63},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.80100000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.45300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574909,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10072,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361768,"statfs":{"total":96636764160,"available":96626450432,"internally_reserved":0,"allocated":868352,"data_stored":408474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33400000000000002,"5min":0.58799999999999997,"15min":0.83799999999999997},"min":{"1min":0.22500000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.48699999999999999,"5min":11.965,"15min":11.965},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.61399999999999999,"15min":0.872},"min":{"1min":0.24199999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.47699999999999998,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.248}]},{"osd":1,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36399999999999999,"5min":0.35199999999999998,"15min":0.34200000000000003},"min":{"1min":0.219,"5min":0.215,"15min":0.215},"max":{"1min":0.56899999999999995,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.379,"5min":0.74099999999999999,"15min":1.097},"min":{"1min":0.25800000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.58299999999999996,"5min":16.988,"15min":16.988},"last":0.28799999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.45800000000000002,"15min":0.49299999999999999},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.57099999999999995,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.435,"5min":1.0229999999999999,"15min":1.6040000000000001},"min":{"1min":0.32000000000000001,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61599999999999999,"5min":27.887,"15min":27.887},"last":0.33100000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.65800000000000003,"15min":0.69899999999999995},"min":{"1min":0.40300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.93799999999999994,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.64300000000000002,"15min":0.66400000000000003},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.85499999999999998,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.505}]},{"osd":5,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65700000000000003,"15min":0.68500000000000005},"min":{"1min":0.44800000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.87,"5min":1.98,"15min":1.98},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.878,"15min":1.1279999999999999},"min":{"1min":0.441,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83199999999999996,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.51500000000000001}]},{"osd":6,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.89000000000000001,"15min":1.1379999999999999},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.83699999999999997,"5min":11.888,"15min":11.888},"last":0.496},{"interface":"front","average":{"1min":0.63,"5min":0.67300000000000004,"15min":0.72199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.88400000000000001,"5min":1.925,"15min":1.925},"last":0.628}]},{"osd":7,"last update":"Tue Nov 28 11:12:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.69699999999999995,"15min":0.73899999999999999},"min":{"1min":0.45800000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.91500000000000004,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.72199999999999998,"15min":0.77100000000000002},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86899999999999999,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.59499999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574910,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10136,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361704,"statfs":{"total":96636764160,"available":96626384896,"internally_reserved":0,"allocated":868352,"data_stored":408474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70899999999999996,"5min":0.996,"15min":0.996},"last":0.441},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.72699999999999998,"15min":0.90600000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.74399999999999999,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.66200000000000003}]},{"osd":1,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.53700000000000003,"15min":0.53200000000000003},"min":{"1min":0.38,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.81699999999999995,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.64900000000000002}]},{"osd":2,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.84199999999999997,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.58699999999999997},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.83699999999999997,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.68600000000000005}]},{"osd":3,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.65300000000000002,"15min":0.73299999999999998},"min":{"1min":0.39500000000000002,"5min":0.372,"15min":0.372},"max":{"1min":0.73999999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.63100000000000001,"15min":0.63700000000000001},"min":{"1min":0.42499999999999999,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.86499999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.58699999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.54400000000000004,"15min":0.55000000000000004},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.55900000000000005,"15min":0.56899999999999995},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.82399999999999995,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.48899999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57899999999999996,"15min":0.60099999999999998},"min":{"1min":0.39900000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71999999999999997,"5min":1.145,"15min":1.145},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60699999999999998,"15min":0.627},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.91800000000000004,"5min":1.085,"15min":1.085},"last":0.52800000000000002}]},{"osd":7,"last update":"Tue Nov 28 11:12:42 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.59699999999999998,"15min":0.623},"min":{"1min":0.36799999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.81599999999999995,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.627},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65600000000000003,"15min":0.67500000000000004},"min":{"1min":0.439,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.83499999999999996,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.61799999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574910,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9880,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361960,"statfs":{"total":96636764160,"available":96626647040,"internally_reserved":0,"allocated":868352,"data_stored":408474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51100000000000001,"15min":0.505},"min":{"1min":0.39600000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.73999999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":0.80700000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.56499999999999995}]},{"osd":1,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64100000000000001,"15min":0.66500000000000004},"min":{"1min":0.42599999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95299999999999996,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.621},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.66500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92100000000000004,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.60799999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79500000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.443,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78500000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.47499999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61699999999999999,"15min":0.67300000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90100000000000002,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.879,"5min":1.042,"15min":1.042},"last":0.55400000000000005}]},{"osd":4,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52300000000000002,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.69199999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59999999999999998,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.80400000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.747}]},{"osd":6,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.60699999999999998},"min":{"1min":0.38,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.622,"15min":0.64200000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85999999999999999,"5min":1.036,"15min":1.036},"last":0.71699999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64600000000000002,"15min":0.68400000000000005},"min":{"1min":0.39900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.84499999999999997,"5min":1.329,"15min":1.329},"last":0.77500000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64800000000000002,"15min":0.68300000000000005},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88200000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:13:30.202 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2023-11-28T11:08:39.371027+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=39, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:13:30.202 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2023-11-28T11:08:39.371027+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=39, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:13:30.203 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2023-11-28T11:08:39.371027+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=39, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:13:30.203 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-28T11:08:39.371027+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=39, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:13:30.203 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2023-11-28T11:08:39.371027+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=39, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:13:30.204 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-28T11:08:39.371027+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=39, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:13:30.204 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2023-11-28T11:08:39.371027+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=39, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:13:30.205 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2023-11-28T11:08:39.371027+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=39, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:13:30.205 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2023-11-28T11:08:39.371027+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=8, tm_sec=39, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=11, tm_min=13, tm_sec=24, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T11:13:30.206 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T11:13:50.207 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T11:13:50.348 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.348+0000 7f87457c3700 1 Processor -- start 2023-11-28T11:13:50.349 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.348+0000 7f87457c3700 1 -- start start 2023-11-28T11:13:50.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.348+0000 7f87457c3700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7f874015c640 0x7f87400cf040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:50.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.348+0000 7f87457c3700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87400b6350 con 0x7f87400d2820 2023-11-28T11:13:50.351 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.348+0000 7f87457c3700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87400d1470 con 0x7f874006bf30 2023-11-28T11:13:50.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.348+0000 7f87457c3700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f874015ca10 con 0x7f874015c640 2023-11-28T11:13:50.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.349+0000 7f873e7fc700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f87400d2820 0x7f87400b5c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40858/0 (socket says 172.21.15.28:40858) 2023-11-28T11:13:50.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.349+0000 7f873e7fc700 1 -- 172.21.15.28:0/158340012 learned_addr learned my addr 172.21.15.28:0/158340012 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:50.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.349+0000 7f873effd700 1 --2- 172.21.15.28:0/158340012 >> v2:172.21.15.59:3300/0 conn(0x7f874015c640 0x7f87400cf040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:50.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.349+0000 7f873dffb700 1 -- 172.21.15.28:0/158340012 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1398192715 0 0) 0x7f87400b6350 con 0x7f87400d2820 2023-11-28T11:13:50.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.349+0000 7f873dffb700 1 -- 172.21.15.28:0/158340012 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87280035b0 con 0x7f87400d2820 2023-11-28T11:13:50.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.349+0000 7f873dffb700 1 -- 172.21.15.28:0/158340012 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1748647505 0 0) 0x7f87400d1470 con 0x7f874006bf30 2023-11-28T11:13:50.352 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.349+0000 7f873dffb700 1 -- 172.21.15.28:0/158340012 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f87400b6350 con 0x7f874006bf30 2023-11-28T11:13:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.349+0000 7f873effd700 1 -- 172.21.15.28:0/158340012 >> v1:172.21.15.59:6789/0 conn(0x7f874006bf30 legacy=0x7f874006c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.349+0000 7f873effd700 1 -- 172.21.15.28:0/158340012 >> v1:172.21.15.28:6789/0 conn(0x7f87400d2820 legacy=0x7f87400b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.349+0000 7f873effd700 1 -- 172.21.15.28:0/158340012 --> v2:172.21.15.59:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87400cd4e0 con 0x7f874015c640 2023-11-28T11:13:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.349+0000 7f873effd700 1 --2- 172.21.15.28:0/158340012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f874015c640 0x7f87400cf040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8730009840 tx=0x7f8730009b50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3563530a42351a99 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.349+0000 7f873dffb700 1 -- 172.21.15.28:0/158340012 <== mon.1 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730011830 con 0x7f874015c640 2023-11-28T11:13:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.350+0000 7f873dffb700 1 -- 172.21.15.28:0/158340012 <== mon.1 v2:172.21.15.59:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8730011990 con 0x7f874015c640 2023-11-28T11:13:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.350+0000 7f873dffb700 1 -- 172.21.15.28:0/158340012 <== mon.1 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8730011d20 con 0x7f874015c640 2023-11-28T11:13:50.353 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.350+0000 7f87457c3700 1 -- 172.21.15.28:0/158340012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f874015c640 msgr2=0x7f87400cf040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.350+0000 7f87457c3700 1 --2- 172.21.15.28:0/158340012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f874015c640 0x7f87400cf040 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8730009840 tx=0x7f8730009b50 comp rx=0 tx=0).stop 2023-11-28T11:13:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.350+0000 7f87457c3700 1 -- 172.21.15.28:0/158340012 shutdown_connections 2023-11-28T11:13:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.350+0000 7f87457c3700 1 --2- 172.21.15.28:0/158340012 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f874015c640 0x7f87400cf040 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.350+0000 7f87457c3700 1 -- 172.21.15.28:0/158340012 >> 172.21.15.28:0/158340012 conn(0x7f87400d2df0 msgr2=0x7f87400d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.350+0000 7f87457c3700 1 -- 172.21.15.28:0/158340012 shutdown_connections 2023-11-28T11:13:50.354 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.350+0000 7f87457c3700 1 -- 172.21.15.28:0/158340012 wait complete. 2023-11-28T11:13:50.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.351+0000 7f87457c3700 1 Processor -- start 2023-11-28T11:13:50.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.352+0000 7f87457c3700 1 -- start start 2023-11-28T11:13:50.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.352+0000 7f87457c3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f874006bf30 0x7f87401f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:50.357 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.352+0000 7f87457c3700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87400d2820 0x7f8740158c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:50.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.352+0000 7f87457c3700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f874015c640 0x7f87400a9740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:50.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.352+0000 7f87457c3700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f874015e5c0 con 0x7f874015c640 2023-11-28T11:13:50.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.352+0000 7f87457c3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87401f2f00 con 0x7f874006bf30 2023-11-28T11:13:50.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.352+0000 7f87457c3700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87401591b0 con 0x7f87400d2820 2023-11-28T11:13:50.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.352+0000 7f873f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f874015c640 0x7f87400a9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:50.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.352+0000 7f873f7fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f874015c640 0x7f87400a9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52808/0 (socket says 172.21.15.28:52808) 2023-11-28T11:13:50.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.352+0000 7f873f7fe700 1 -- 172.21.15.28:0/2111357670 learned_addr learned my addr 172.21.15.28:0/2111357670 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:50.358 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.353+0000 7f873effd700 1 --2- 172.21.15.28:0/2111357670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f874006bf30 0x7f87401f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:50.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.353+0000 7f873e7fc700 1 --2- 172.21.15.28:0/2111357670 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87400d2820 0x7f8740158c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:50.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.353+0000 7f873f7fe700 1 -- 172.21.15.28:0/2111357670 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87400d2820 msgr2=0x7f8740158c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:50.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.353+0000 7f873f7fe700 1 --2- 172.21.15.28:0/2111357670 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87400d2820 0x7f8740158c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.353+0000 7f873f7fe700 1 -- 172.21.15.28:0/2111357670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f874006bf30 msgr2=0x7f87401f29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:50.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.353+0000 7f873f7fe700 1 --2- 172.21.15.28:0/2111357670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f874006bf30 0x7f87401f29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.353+0000 7f873f7fe700 1 -- 172.21.15.28:0/2111357670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87300094a0 con 0x7f874015c640 2023-11-28T11:13:50.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.353+0000 7f873e7fc700 1 --2- 172.21.15.28:0/2111357670 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87400d2820 0x7f8740158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T11:13:50.359 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.353+0000 7f873f7fe700 1 --2- 172.21.15.28:0/2111357670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f874015c640 0x7f87400a9740 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f873800cc20 tx=0x7f873800cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.353+0000 7f87277fe700 1 -- 172.21.15.28:0/2111357670 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8738003c00 con 0x7f874015c640 2023-11-28T11:13:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.353+0000 7f87277fe700 1 -- 172.21.15.28:0/2111357670 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8738003d60 con 0x7f874015c640 2023-11-28T11:13:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.353+0000 7f87457c3700 1 -- 172.21.15.28:0/2111357670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87400d1710 con 0x7f874015c640 2023-11-28T11:13:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.353+0000 7f87277fe700 1 -- 172.21.15.28:0/2111357670 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f873800ab90 con 0x7f874015c640 2023-11-28T11:13:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.353+0000 7f87457c3700 1 -- 172.21.15.28:0/2111357670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87400cd7c0 con 0x7f874015c640 2023-11-28T11:13:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.354+0000 7f87457c3700 1 -- 172.21.15.28:0/2111357670 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87400cf580 con 0x7f874015c640 2023-11-28T11:13:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.355+0000 7f87277fe700 1 -- 172.21.15.28:0/2111357670 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f8738010050 con 0x7f874015c640 2023-11-28T11:13:50.360 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.355+0000 7f87277fe700 1 --2- 172.21.15.28:0/2111357670 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f872c0682f0 0x7f872c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.355+0000 7f87277fe700 1 -- 172.21.15.28:0/2111357670 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f8738085a40 con 0x7f874015c640 2023-11-28T11:13:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.355+0000 7f873effd700 1 --2- 172.21.15.28:0/2111357670 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f872c0682f0 0x7f872c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.356+0000 7f873effd700 1 --2- 172.21.15.28:0/2111357670 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f872c0682f0 0x7f872c06a7a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8730009fd0 tx=0x7f873001c040 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:50.361 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.360+0000 7f87277fe700 1 -- 172.21.15.28:0/2111357670 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f8738052400 con 0x7f874015c640 2023-11-28T11:13:50.550 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.550+0000 7f87457c3700 1 -- 172.21.15.28:0/2111357670 --> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f874015e700 con 0x7f872c0682f0 2023-11-28T11:13:50.553 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.552+0000 7f87277fe700 1 -- 172.21.15.28:0/2111357670 <== mgr.4098 v2:172.21.15.28:6832/103239 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72803 (secure 0 0 0) 0x7f874015e700 con 0x7f872c0682f0 2023-11-28T11:13:50.554 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:50.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.554+0000 7f87457c3700 1 -- 172.21.15.28:0/2111357670 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f872c0682f0 msgr2=0x7f872c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:50.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.554+0000 7f87457c3700 1 --2- 172.21.15.28:0/2111357670 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f872c0682f0 0x7f872c06a7a0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f8730009fd0 tx=0x7f873001c040 comp rx=0 tx=0).stop 2023-11-28T11:13:50.555 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.554+0000 7f87457c3700 1 -- 172.21.15.28:0/2111357670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f874015c640 msgr2=0x7f87400a9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.554+0000 7f87457c3700 1 --2- 172.21.15.28:0/2111357670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f874015c640 0x7f87400a9740 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f873800cc20 tx=0x7f873800cf30 comp rx=0 tx=0).stop 2023-11-28T11:13:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.554+0000 7f87457c3700 1 -- 172.21.15.28:0/2111357670 shutdown_connections 2023-11-28T11:13:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.554+0000 7f87457c3700 1 --2- 172.21.15.28:0/2111357670 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f872c0682f0 0x7f872c06a7a0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.556 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.554+0000 7f87457c3700 1 --2- 172.21.15.28:0/2111357670 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f874015c640 0x7f87400a9740 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.554+0000 7f87457c3700 1 --2- 172.21.15.28:0/2111357670 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f874006bf30 0x7f87401f29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.554+0000 7f87457c3700 1 --2- 172.21.15.28:0/2111357670 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f87400d2820 0x7f8740158c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.554+0000 7f87457c3700 1 -- 172.21.15.28:0/2111357670 >> 172.21.15.28:0/2111357670 conn(0x7f87400d2df0 msgr2=0x7f87400b1b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:50.557 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.554+0000 7f87457c3700 1 -- 172.21.15.28:0/2111357670 shutdown_connections 2023-11-28T11:13:50.558 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.554+0000 7f87457c3700 1 -- 172.21.15.28:0/2111357670 wait complete. 2023-11-28T11:13:50.558 INFO:teuthology.orchestra.run.smithi028.stderr:dumped all 2023-11-28T11:13:50.567 INFO:teuthology.orchestra.run.smithi028.stdout:{"pg_ready":true,"pg_map":{"version":220,"stamp":"2023-11-28T11:13:49.232720+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":82180,"kb_used_data":7940,"kb_used_omap":0,"kb_used_meta":74176,"kb_avail":754892540,"statfs":{"total":773094113280,"available":773009960960,"internally_reserved":0,"allocated":8130560,"data_stored":4447567,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":75956224},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002568"},"pg_stats":[{"pgid":"1.5","version":"0'0","reported_seq":73,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:13:32.373065+0000","last_change":"2023-11-28T11:13:32.373015+0000","last_active":"2023-11-28T11:13:32.373065+0000","last_peered":"2023-11-28T11:13:32.373065+0000","last_clean":"2023-11-28T11:13:32.373065+0000","last_became_active":"2023-11-28T11:08:40.522371+0000","last_became_peered":"2023-11-28T11:08:40.522371+0000","last_unstale":"2023-11-28T11:13:32.373065+0000","last_undegraded":"2023-11-28T11:13:32.373065+0000","last_fullsized":"2023-11-28T11:13:32.373065+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:13:32.372904+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:13:32.372904+0000","last_clean_scrub_stamp":"2023-11-28T11:13:32.372904+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:38.374414+0000","scrub_duration":0.0039731519999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":74,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:13:34.785437+0000","last_change":"2023-11-28T11:13:34.785406+0000","last_active":"2023-11-28T11:13:34.785437+0000","last_peered":"2023-11-28T11:13:34.785437+0000","last_clean":"2023-11-28T11:13:34.785437+0000","last_became_active":"2023-11-28T11:08:40.384484+0000","last_became_peered":"2023-11-28T11:08:40.384484+0000","last_unstale":"2023-11-28T11:13:34.785437+0000","last_undegraded":"2023-11-28T11:13:34.785437+0000","last_fullsized":"2023-11-28T11:13:34.785437+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:13:34.785294+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:13:34.785294+0000","last_clean_scrub_stamp":"2023-11-28T11:13:34.785294+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:22:46.856442+0000","scrub_duration":0.003908679,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":74,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:13:28.977179+0000","last_change":"2023-11-28T11:13:28.977158+0000","last_active":"2023-11-28T11:13:28.977179+0000","last_peered":"2023-11-28T11:13:28.977179+0000","last_clean":"2023-11-28T11:13:28.977179+0000","last_became_active":"2023-11-28T11:08:40.388096+0000","last_became_peered":"2023-11-28T11:08:40.388096+0000","last_unstale":"2023-11-28T11:13:28.977179+0000","last_undegraded":"2023-11-28T11:13:28.977179+0000","last_fullsized":"2023-11-28T11:13:28.977179+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:13:28.977088+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:13:28.977088+0000","last_clean_scrub_stamp":"2023-11-28T11:13:28.977088+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T19:29:52.745889+0000","scrub_duration":0.003901519,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"71'119","reported_seq":225,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:13:34.511172+0000","last_change":"2023-11-28T11:13:34.511123+0000","last_active":"2023-11-28T11:13:34.511172+0000","last_peered":"2023-11-28T11:13:34.511172+0000","last_clean":"2023-11-28T11:13:34.511172+0000","last_became_active":"2023-11-28T11:08:40.388367+0000","last_became_peered":"2023-11-28T11:08:40.388367+0000","last_unstale":"2023-11-28T11:13:34.511172+0000","last_undegraded":"2023-11-28T11:13:34.511172+0000","last_fullsized":"2023-11-28T11:13:34.511172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"71'119","last_scrub_stamp":"2023-11-28T11:13:34.511029+0000","last_deep_scrub":"71'119","last_deep_scrub_stamp":"2023-11-28T11:13:34.511029+0000","last_clean_scrub_stamp":"2023-11-28T11:13:34.511029+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:53:48.669945+0000","scrub_duration":0.0068078280000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":73,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:13:33.535020+0000","last_change":"2023-11-28T11:13:33.534987+0000","last_active":"2023-11-28T11:13:33.535020+0000","last_peered":"2023-11-28T11:13:33.535020+0000","last_clean":"2023-11-28T11:13:33.535020+0000","last_became_active":"2023-11-28T11:08:40.831735+0000","last_became_peered":"2023-11-28T11:08:40.831735+0000","last_unstale":"2023-11-28T11:13:33.535020+0000","last_undegraded":"2023-11-28T11:13:33.535020+0000","last_fullsized":"2023-11-28T11:13:33.535020+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:13:33.534875+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:13:33.534875+0000","last_clean_scrub_stamp":"2023-11-28T11:13:33.534875+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:07:44.982261+0000","scrub_duration":0.0042801460000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":73,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:13:32.534422+0000","last_change":"2023-11-28T11:13:32.534391+0000","last_active":"2023-11-28T11:13:32.534422+0000","last_peered":"2023-11-28T11:13:32.534422+0000","last_clean":"2023-11-28T11:13:32.534422+0000","last_became_active":"2023-11-28T11:08:40.831691+0000","last_became_peered":"2023-11-28T11:08:40.831691+0000","last_unstale":"2023-11-28T11:13:32.534422+0000","last_undegraded":"2023-11-28T11:13:32.534422+0000","last_fullsized":"2023-11-28T11:13:32.534422+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:13:32.534138+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:13:32.534138+0000","last_clean_scrub_stamp":"2023-11-28T11:13:32.534138+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T20:19:38.535648+0000","scrub_duration":0.0041520740000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":78,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:13:31.038963+0000","last_change":"2023-11-28T11:13:31.038914+0000","last_active":"2023-11-28T11:13:31.038963+0000","last_peered":"2023-11-28T11:13:31.038963+0000","last_clean":"2023-11-28T11:13:31.038963+0000","last_became_active":"2023-11-28T11:08:40.831940+0000","last_became_peered":"2023-11-28T11:08:40.831940+0000","last_unstale":"2023-11-28T11:13:31.038963+0000","last_undegraded":"2023-11-28T11:13:31.038963+0000","last_fullsized":"2023-11-28T11:13:31.038963+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:13:31.038758+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:13:31.038758+0000","last_clean_scrub_stamp":"2023-11-28T11:13:31.038758+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:36:55.178640+0000","scrub_duration":0.0039922480000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":73,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:13:34.376457+0000","last_change":"2023-11-28T11:13:34.376428+0000","last_active":"2023-11-28T11:13:34.376457+0000","last_peered":"2023-11-28T11:13:34.376457+0000","last_clean":"2023-11-28T11:13:34.376457+0000","last_became_active":"2023-11-28T11:08:40.831701+0000","last_became_peered":"2023-11-28T11:08:40.831701+0000","last_unstale":"2023-11-28T11:13:34.376457+0000","last_undegraded":"2023-11-28T11:13:34.376457+0000","last_fullsized":"2023-11-28T11:13:34.376457+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:13:34.376322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:13:34.376322+0000","last_clean_scrub_stamp":"2023-11-28T11:13:34.376322+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T22:20:22.422239+0000","scrub_duration":0.0039044240000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":73,"reported_epoch":71,"state":"active+clean","last_fresh":"2023-11-28T11:13:38.371464+0000","last_change":"2023-11-28T11:13:38.371437+0000","last_active":"2023-11-28T11:13:38.371464+0000","last_peered":"2023-11-28T11:13:38.371464+0000","last_clean":"2023-11-28T11:13:38.371464+0000","last_became_active":"2023-11-28T11:08:40.387486+0000","last_became_peered":"2023-11-28T11:08:40.387486+0000","last_unstale":"2023-11-28T11:13:38.371464+0000","last_undegraded":"2023-11-28T11:13:38.371464+0000","last_fullsized":"2023-11-28T11:13:38.371464+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T11:13:38.371336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T11:13:38.371336+0000","last_clean_scrub_stamp":"2023-11-28T11:13:38.371336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-11-29T16:37:02.511218+0000","scrub_duration":0.0042230849999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574914,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11100,"kb_used_data":1428,"kb_used_omap":0,"kb_used_meta":9664,"kb_avail":94360740,"statfs":{"total":96636764160,"available":96625397760,"internally_reserved":0,"allocated":1462272,"data_stored":998842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9895936},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.56299999999999994,"15min":0.628},"min":{"1min":0.42799999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.80500000000000005,"5min":3.2189999999999999,"15min":3.2189999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.629},"min":{"1min":0.40699999999999997,"5min":0.307,"15min":0.307},"max":{"1min":0.77000000000000002,"5min":3.2000000000000002,"15min":3.2000000000000002},"last":0.66600000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59299999999999997,"15min":0.66400000000000003},"min":{"1min":0.46999999999999997,"5min":0.316,"15min":0.316},"max":{"1min":0.74199999999999999,"5min":3.1360000000000001,"15min":3.1360000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.59399999999999997,"15min":0.59599999999999997},"min":{"1min":0.497,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78700000000000003,"5min":1.224,"15min":1.224},"last":0.755}]},{"osd":2,"last update":"Tue Nov 28 11:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.622,"15min":0.67600000000000005},"min":{"1min":0.45300000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.81699999999999995,"5min":3.3820000000000001,"15min":3.3820000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67100000000000004,"15min":0.755},"min":{"1min":0.51000000000000001,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.79400000000000004,"5min":3.3969999999999998,"15min":3.3969999999999998},"last":0.77500000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.63200000000000001,"15min":0.68600000000000005},"min":{"1min":0.43099999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.79400000000000004,"5min":3.3069999999999999,"15min":3.3069999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.67700000000000005,"15min":0.75},"min":{"1min":0.55300000000000005,"5min":0.439,"15min":0.439},"max":{"1min":0.80500000000000005,"5min":3.294,"15min":3.294},"last":0.73599999999999999}]},{"osd":4,"last update":"Tue Nov 28 11:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53300000000000003,"15min":0.54700000000000004},"min":{"1min":0.35999999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.78500000000000003,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55500000000000005,"15min":0.56100000000000005},"min":{"1min":0.38800000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.72699999999999998,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.58599999999999997}]},{"osd":5,"last update":"Tue Nov 28 11:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.59799999999999998,"15min":0.63200000000000001},"min":{"1min":0.34300000000000003,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.79000000000000004,"5min":1.2509999999999999,"15min":1.2509999999999999},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63,"15min":0.64400000000000002},"min":{"1min":0.40999999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.79700000000000004,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.72099999999999997}]},{"osd":6,"last update":"Tue Nov 28 11:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65000000000000002,"15min":0.67100000000000004},"min":{"1min":0.45700000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83999999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.627,"15min":0.63600000000000001},"min":{"1min":0.432,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.76600000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.70699999999999996}]}]},{"osd":6,"up_from":13,"seq":55834574913,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9944,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9088,"kb_avail":94361896,"statfs":{"total":96636764160,"available":96626581504,"internally_reserved":0,"allocated":868352,"data_stored":408474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9306112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.51100000000000001,"15min":0.47099999999999997},"min":{"1min":0.376,"5min":0.252,"15min":0.252},"max":{"1min":0.79300000000000004,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.51200000000000001,"15min":0.48599999999999999},"min":{"1min":0.46200000000000002,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.77400000000000002,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.56100000000000005}]},{"osd":1,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.55200000000000005,"15min":0.52000000000000002},"min":{"1min":0.38200000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80300000000000005,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.53200000000000003,"15min":0.504},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81799999999999995,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.73699999999999999}]},{"osd":2,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.52400000000000002,"15min":0.50600000000000001},"min":{"1min":0.37,"5min":0.312,"15min":0.312},"max":{"1min":0.72599999999999998,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.64100000000000001,"15min":0.67500000000000004},"min":{"1min":0.435,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.80200000000000005,"5min":3.7429999999999999,"15min":3.7429999999999999},"last":0.66100000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67100000000000004,"15min":0.71199999999999997},"min":{"1min":0.40200000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.84199999999999997,"5min":3.7450000000000001,"15min":3.7450000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.67200000000000004,"15min":0.69499999999999995},"min":{"1min":0.51200000000000001,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.83199999999999996,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.58199999999999996}]},{"osd":4,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"min":{"1min":0.36699999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.73999999999999999,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.60199999999999998,"15min":0.622},"min":{"1min":0.35199999999999998,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.84999999999999998,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.41399999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.623,"15min":0.629},"min":{"1min":0.437,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.873,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.61199999999999999,"15min":0.61299999999999999},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.78800000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.44}]},{"osd":7,"last update":"Tue Nov 28 11:12:44 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.56699999999999995,"15min":0.58099999999999996},"min":{"1min":0.45300000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.68000000000000005,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.66800000000000004,"15min":0.67600000000000005},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.82399999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.48799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":11036,"kb_used_data":1428,"kb_used_omap":0,"kb_used_meta":9600,"kb_avail":94360804,"statfs":{"total":96636764160,"available":96625463296,"internally_reserved":0,"allocated":1462272,"data_stored":998842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9830400},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.34000000000000002,"15min":0.33100000000000002},"min":{"1min":0.255,"5min":0.19,"15min":0.19},"max":{"1min":0.501,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.35599999999999998,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.251,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55600000000000005,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.38400000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.34100000000000003,"15min":0.32300000000000001},"min":{"1min":0.26300000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.51400000000000001,"5min":0.73899999999999999,"15min":0.73899999999999999},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.39000000000000001,"15min":0.38300000000000001},"min":{"1min":0.28999999999999998,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.66100000000000003,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.41299999999999998}]},{"osd":3,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.42099999999999999,"15min":0.39700000000000002},"min":{"1min":0.35799999999999998,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.60499999999999998,"5min":0.64700000000000002,"15min":0.64700000000000002},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42799999999999999,"15min":0.41399999999999998},"min":{"1min":0.32900000000000001,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.44600000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.56999999999999995,"15min":0.54000000000000004},"min":{"1min":0.44900000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.80500000000000005,"5min":1.0940000000000001,"15min":1.0940000000000001},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59399999999999997,"15min":0.57399999999999995},"min":{"1min":0.46400000000000002,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.91500000000000004,"5min":1.014,"15min":1.014},"last":0.52100000000000002}]},{"osd":5,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.60599999999999998,"15min":0.56200000000000006},"min":{"1min":0.47699999999999998,"5min":0.374,"15min":0.374},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.60299999999999998,"15min":0.57099999999999995},"min":{"1min":0.48999999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.621,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.0389999999999999,"5min":1.3220000000000001,"15min":1.3220000000000001},"last":0.504},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.52100000000000002,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.88200000000000001,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.621}]},{"osd":7,"last update":"Tue Nov 28 11:12:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.63,"15min":0.60899999999999999},"min":{"1min":0.49299999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86599999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.63200000000000001,"15min":0.60399999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":1.073,"5min":1.073,"15min":1.073},"last":0.59999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574914,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10136,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361704,"statfs":{"total":96636764160,"available":96626384896,"internally_reserved":0,"allocated":868352,"data_stored":408474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.33600000000000002,"15min":0.32900000000000001},"min":{"1min":0.25,"5min":0.218,"15min":0.218},"max":{"1min":0.501,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.38400000000000001,"15min":0.374},"min":{"1min":0.30299999999999999,"5min":0.246,"15min":0.246},"max":{"1min":0.53800000000000003,"5min":0.622,"15min":0.622},"last":0.38100000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.38200000000000001,"5min":0.38100000000000001,"15min":0.38300000000000001},"min":{"1min":0.28899999999999998,"5min":0.246,"15min":0.246},"max":{"1min":0.56499999999999995,"5min":0.60299999999999998,"15min":0.60299999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39700000000000002,"15min":0.39900000000000002},"min":{"1min":0.28899999999999998,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61699999999999999,"5min":0.68400000000000005,"15min":0.68400000000000005},"last":0.31}]},{"osd":3,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.372},"min":{"1min":0.30499999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.437,"15min":0.43099999999999999},"min":{"1min":0.35699999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.59899999999999998,"5min":0.83599999999999997,"15min":0.83599999999999997},"last":0.51300000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.59099999999999997,"15min":0.58499999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.75700000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.46400000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.74399999999999999,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.56599999999999995}]},{"osd":5,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.66100000000000003,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.71499999999999997,"5min":5.4269999999999996,"15min":5.4269999999999996},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.61499999999999999,"15min":0.621},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.81599999999999995,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.59899999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.621,"15min":0.622},"min":{"1min":0.41299999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":1.3759999999999999,"15min":1.3759999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63400000000000001},"min":{"1min":0.51900000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.79200000000000004,"5min":1.016,"15min":1.016},"last":0.53000000000000003}]},{"osd":7,"last update":"Tue Nov 28 11:12:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"min":{"1min":0.46700000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.78700000000000003,"5min":1.105,"15min":1.105},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.65300000000000002,"15min":0.64400000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.80800000000000005,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.73799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9876,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361964,"statfs":{"total":96636764160,"available":96626651136,"internally_reserved":0,"allocated":864256,"data_stored":407513,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.35799999999999998,"15min":0.371},"min":{"1min":0.247,"5min":0.214,"15min":0.214},"max":{"1min":0.50700000000000001,"5min":0.56200000000000006,"15min":0.56200000000000006},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.371,"5min":0.38100000000000001,"15min":0.38700000000000001},"min":{"1min":0.22800000000000001,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.54900000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.31}]},{"osd":1,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.39200000000000002,"15min":0.39400000000000002},"min":{"1min":0.29599999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.54100000000000004,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.42099999999999999,"15min":0.441},"min":{"1min":0.29399999999999998,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.57899999999999996,"5min":1.528,"15min":1.528},"last":0.40000000000000002}]},{"osd":3,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.38300000000000001,"15min":0.39400000000000002},"min":{"1min":0.27700000000000002,"5min":0.252,"15min":0.252},"max":{"1min":0.51400000000000001,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.45600000000000002,"15min":0.47599999999999998},"min":{"1min":0.312,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.55300000000000005,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.378}]},{"osd":4,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60799999999999998,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.80700000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.60799999999999998,"15min":0.60699999999999998},"min":{"1min":0.46500000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.60599999999999998}]},{"osd":5,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60799999999999998,"15min":0.61099999999999999},"min":{"1min":0.44700000000000001,"5min":0.438,"15min":0.438},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.63800000000000001,"15min":0.63700000000000001},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93899999999999995,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.54400000000000004}]},{"osd":6,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.62,"15min":0.622},"min":{"1min":0.48499999999999999,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.875,"5min":0.875,"15min":0.875},"last":0.625},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.65400000000000003,"15min":0.66800000000000004},"min":{"1min":0.441,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85399999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.64000000000000001}]},{"osd":7,"last update":"Tue Nov 28 11:12:48 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.64800000000000002,"15min":0.65000000000000002},"min":{"1min":0.45500000000000002,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.628,"15min":0.63},"min":{"1min":0.44400000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.80100000000000005,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.67700000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574913,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10072,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9216,"kb_avail":94361768,"statfs":{"total":96636764160,"available":96626450432,"internally_reserved":0,"allocated":868352,"data_stored":408474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9437184},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.46300000000000002,"15min":0.79600000000000004},"min":{"1min":0.246,"5min":0.216,"15min":0.216},"max":{"1min":0.438,"5min":11.965,"15min":11.965},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.376,"5min":0.48799999999999999,"15min":0.82999999999999996},"min":{"1min":0.23599999999999999,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.54900000000000004,"5min":12.888999999999999,"15min":12.888999999999999},"last":0.39500000000000002}]},{"osd":1,"last update":"Tue Nov 28 11:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.35799999999999998,"15min":0.34399999999999997},"min":{"1min":0.25700000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.56399999999999995,"15min":1.038},"min":{"1min":0.25,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.53200000000000003,"5min":16.988,"15min":16.988},"last":0.32900000000000001}]},{"osd":2,"last update":"Tue Nov 28 11:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.442,"15min":0.48799999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.57299999999999995,"5min":2.0609999999999999,"15min":2.0609999999999999},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.73399999999999999,"15min":1.508},"min":{"1min":0.33100000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.58999999999999997,"5min":27.887,"15min":27.887},"last":0.46000000000000002}]},{"osd":4,"last update":"Tue Nov 28 11:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.64000000000000001,"15min":0.69299999999999995},"min":{"1min":0.44800000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.77200000000000002,"5min":1.8899999999999999,"15min":1.8899999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.628,"15min":0.65900000000000003},"min":{"1min":0.41699999999999998,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.754,"5min":2.0019999999999998,"15min":2.0019999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Tue Nov 28 11:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.64400000000000002,"15min":0.68000000000000005},"min":{"1min":0.44600000000000001,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.81399999999999995,"5min":1.98,"15min":1.98},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.749,"15min":1.085},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.82299999999999995,"5min":11.933999999999999,"15min":11.933999999999999},"last":0.61399999999999999}]},{"osd":6,"last update":"Tue Nov 28 11:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.76400000000000001,"15min":1.0960000000000001},"min":{"1min":0.42699999999999999,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.80600000000000005,"5min":11.888,"15min":11.888},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.64900000000000002,"15min":0.71399999999999997},"min":{"1min":0.42599999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.80100000000000005,"5min":1.925,"15min":1.925},"last":0.71499999999999997}]},{"osd":7,"last update":"Tue Nov 28 11:13:43 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.67500000000000004,"15min":0.73099999999999998},"min":{"1min":0.47499999999999998,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80400000000000005,"5min":1.9490000000000001,"15min":1.9490000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.70099999999999996,"15min":0.76400000000000001},"min":{"1min":0.56999999999999995,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83799999999999997,"5min":2.0310000000000001,"15min":2.0310000000000001},"last":0.60599999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574914,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":10136,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9280,"kb_avail":94361704,"statfs":{"total":96636764160,"available":96626384896,"internally_reserved":0,"allocated":868352,"data_stored":408474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9502720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.54600000000000004,"15min":0.53800000000000003},"min":{"1min":0.38500000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.87,"5min":0.996,"15min":0.996},"last":0.87},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.65000000000000002,"15min":0.88},"min":{"1min":0.49199999999999999,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77100000000000002,"5min":10.045999999999999,"15min":10.045999999999999},"last":0.66900000000000004}]},{"osd":1,"last update":"Tue Nov 28 11:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.55200000000000005,"15min":0.53700000000000003},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.84899999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.59599999999999997,"15min":0.58499999999999996},"min":{"1min":0.43099999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.85199999999999998,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.82699999999999996}]},{"osd":2,"last update":"Tue Nov 28 11:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58299999999999996,"15min":0.57899999999999996},"min":{"1min":0.40999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.89700000000000002,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.89700000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.58399999999999996,"15min":0.58799999999999997},"min":{"1min":0.436,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.91200000000000003,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.91200000000000003}]},{"osd":3,"last update":"Tue Nov 28 11:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.62,"15min":0.72199999999999998},"min":{"1min":0.42799999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.80900000000000005,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.63,"15min":0.63700000000000001},"min":{"1min":0.46000000000000002,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0249999999999999,"15min":1.0249999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Tue Nov 28 11:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.54700000000000004,"15min":0.55100000000000005},"min":{"1min":0.38500000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.81799999999999995,"5min":0.877,"15min":0.877},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55700000000000005,"15min":0.56799999999999995},"min":{"1min":0.36599999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.78600000000000003,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.60099999999999998}]},{"osd":6,"last update":"Tue Nov 28 11:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.59799999999999998},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.83499999999999996,"5min":1.145,"15min":1.145},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.59399999999999997,"15min":0.623},"min":{"1min":0.38,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.82299999999999995,"5min":1.085,"15min":1.085},"last":0.68799999999999994}]},{"osd":7,"last update":"Tue Nov 28 11:13:45 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59199999999999997,"15min":0.622},"min":{"1min":0.47999999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.76200000000000001,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64800000000000002,"15min":0.67200000000000004},"min":{"1min":0.46600000000000003,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.88300000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.73399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574914,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":9880,"kb_used_data":848,"kb_used_omap":0,"kb_used_meta":9024,"kb_avail":94361960,"statfs":{"total":96636764160,"available":96626647040,"internally_reserved":0,"allocated":868352,"data_stored":408474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":9240576},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.51100000000000001,"15min":0.505},"min":{"1min":0.39600000000000002,"5min":0.317,"15min":0.317},"max":{"1min":0.73999999999999999,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.437},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.375,"5min":0.318,"15min":0.318},"max":{"1min":0.80700000000000005,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.40999999999999998}]},{"osd":1,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.64100000000000001,"15min":0.66500000000000004},"min":{"1min":0.42599999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.95299999999999996,"5min":2.8719999999999999,"15min":2.8719999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.621,"5min":0.627,"15min":0.66500000000000004},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.92100000000000004,"5min":2.8849999999999998,"15min":2.8849999999999998},"last":0.48099999999999998}]},{"osd":2,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.55000000000000004,"15min":0.54700000000000004},"min":{"1min":0.40400000000000003,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.79500000000000004,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.54500000000000004,"15min":0.53300000000000003},"min":{"1min":0.443,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.78500000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.496}]},{"osd":3,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61699999999999999,"15min":0.67300000000000004},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.90100000000000002,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.626,"5min":0.59399999999999997,"15min":0.59199999999999997},"min":{"1min":0.46000000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.879,"5min":1.042,"15min":1.042},"last":0.63100000000000001}]},{"osd":4,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52300000000000002,"15min":0.54100000000000004},"min":{"1min":0.35199999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.69199999999999995,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59999999999999998,"15min":0.626},"min":{"1min":0.38500000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.80400000000000005,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.55300000000000005}]},{"osd":6,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58999999999999997,"15min":0.60699999999999998},"min":{"1min":0.38,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.76100000000000001,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.622,"15min":0.64200000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.85999999999999999,"5min":1.036,"15min":1.036},"last":0.67100000000000004}]},{"osd":7,"last update":"Tue Nov 28 11:12:49 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.64600000000000002,"15min":0.68400000000000005},"min":{"1min":0.39900000000000002,"5min":0.311,"15min":0.311},"max":{"1min":0.84499999999999997,"5min":1.329,"15min":1.329},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.64800000000000002,"15min":0.68300000000000005},"min":{"1min":0.40200000000000002,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.88200000000000001,"5min":1.3839999999999999,"15min":1.3839999999999999},"last":0.71699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T11:13:50.571 DEBUG:teuthology.orchestra.run.smithi028:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-28T11:13:50.695 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.695+0000 7fef9edd1700 1 Processor -- start 2023-11-28T11:13:50.696 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.695+0000 7fef9edd1700 1 -- start start 2023-11-28T11:13:50.704 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef9edd1700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fef9815c950 0x7fef9806a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:50.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef9edd1700 1 --2- >> v2:172.21.15.59:3300/0 conn(0x7fef980cae50 0x7fef980cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:50.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef9edd1700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef980caab0 con 0x7fef980ca360 2023-11-28T11:13:50.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef9edd1700 1 -- --> v2:172.21.15.28:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef9815cd20 con 0x7fef9815c950 2023-11-28T11:13:50.705 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef9edd1700 1 -- --> v2:172.21.15.59:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef980cb7b0 con 0x7fef980cae50 2023-11-28T11:13:50.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef9cb6d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fef9815c950 0x7fef9806a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:50.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef9cb6d700 1 --2- >> v2:172.21.15.28:3300/0 conn(0x7fef9815c950 0x7fef9806a1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52812/0 (socket says 172.21.15.28:52812) 2023-11-28T11:13:50.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef9cb6d700 1 -- 172.21.15.28:0/2407093975 learned_addr learned my addr 172.21.15.28:0/2407093975 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:50.706 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef9d36e700 1 --2- 172.21.15.28:0/2407093975 >> v2:172.21.15.59:3300/0 conn(0x7fef980cae50 0x7fef980cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:50.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef9cb6d700 1 -- 172.21.15.28:0/2407093975 >> v2:172.21.15.59:3300/0 conn(0x7fef980cae50 msgr2=0x7fef980cb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:50.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef9cb6d700 1 --2- 172.21.15.28:0/2407093975 >> v2:172.21.15.59:3300/0 conn(0x7fef980cae50 0x7fef980cb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef9cb6d700 1 -- 172.21.15.28:0/2407093975 >> v1:172.21.15.59:6789/0 conn(0x7fef980ca360 legacy=0x7fef9806f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:50.707 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef9cb6d700 1 -- 172.21.15.28:0/2407093975 --> v2:172.21.15.28:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef9806fc00 con 0x7fef9815c950 2023-11-28T11:13:50.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef9cb6d700 1 --2- 172.21.15.28:0/2407093975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef9815c950 0x7fef9806a1e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fef980b5230 tx=0x7fef8c009bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9931aa4e3283cee0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:50.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef977fe700 1 -- 172.21.15.28:0/2407093975 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c00b870 con 0x7fef9815c950 2023-11-28T11:13:50.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef977fe700 1 -- 172.21.15.28:0/2407093975 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8c00b9d0 con 0x7fef9815c950 2023-11-28T11:13:50.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.696+0000 7fef977fe700 1 -- 172.21.15.28:0/2407093975 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8c00bd60 con 0x7fef9815c950 2023-11-28T11:13:50.708 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.697+0000 7fef9edd1700 1 -- 172.21.15.28:0/2407093975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef9815c950 msgr2=0x7fef9806a1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:50.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.697+0000 7fef9edd1700 1 --2- 172.21.15.28:0/2407093975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef9815c950 0x7fef9806a1e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fef980b5230 tx=0x7fef8c009bd0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.697+0000 7fef9edd1700 1 -- 172.21.15.28:0/2407093975 shutdown_connections 2023-11-28T11:13:50.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.697+0000 7fef9edd1700 1 --2- 172.21.15.28:0/2407093975 >> v2:172.21.15.59:3300/0 conn(0x7fef980cae50 0x7fef980cb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.709 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.697+0000 7fef9edd1700 1 --2- 172.21.15.28:0/2407093975 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef9815c950 0x7fef9806a1e0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.697+0000 7fef9edd1700 1 -- 172.21.15.28:0/2407093975 >> 172.21.15.28:0/2407093975 conn(0x7fef980b0df0 msgr2=0x7fef980b11f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:50.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.697+0000 7fef9edd1700 1 -- 172.21.15.28:0/2407093975 shutdown_connections 2023-11-28T11:13:50.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.697+0000 7fef9edd1700 1 -- 172.21.15.28:0/2407093975 wait complete. 2023-11-28T11:13:50.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.698+0000 7fef9edd1700 1 Processor -- start 2023-11-28T11:13:50.710 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.698+0000 7fef9edd1700 1 -- start start 2023-11-28T11:13:50.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.699+0000 7fef9edd1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef980ca360 0x7fef980c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:50.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.699+0000 7fef9edd1700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef980cae50 0x7fef98152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:50.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.699+0000 7fef9edd1700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef9815c950 0x7fef980aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:50.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.699+0000 7fef9edd1700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef980c4610 con 0x7fef980cae50 2023-11-28T11:13:50.711 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.699+0000 7fef9edd1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef98152c30 con 0x7fef980ca360 2023-11-28T11:13:50.712 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.699+0000 7fef9edd1700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef980c90b0 con 0x7fef9815c950 2023-11-28T11:13:50.712 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.699+0000 7fef97fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef980cae50 0x7fef98152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:50.712 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.699+0000 7fef97fff700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef980cae50 0x7fef98152130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52820/0 (socket says 172.21.15.28:52820) 2023-11-28T11:13:50.712 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.699+0000 7fef97fff700 1 -- 172.21.15.28:0/2099261985 learned_addr learned my addr 172.21.15.28:0/2099261985 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:50.712 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.700+0000 7fef9cb6d700 1 --2- 172.21.15.28:0/2099261985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef980ca360 0x7fef980c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:50.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.700+0000 7fef9d36e700 1 --2- 172.21.15.28:0/2099261985 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef9815c950 0x7fef980aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:50.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.700+0000 7fef97fff700 1 -- 172.21.15.28:0/2099261985 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef9815c950 msgr2=0x7fef980aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:50.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.700+0000 7fef97fff700 1 --2- 172.21.15.28:0/2099261985 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef9815c950 0x7fef980aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.700+0000 7fef97fff700 1 -- 172.21.15.28:0/2099261985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef980ca360 msgr2=0x7fef980c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:50.713 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.700+0000 7fef97fff700 1 --2- 172.21.15.28:0/2099261985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef980ca360 0x7fef980c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.700+0000 7fef97fff700 1 -- 172.21.15.28:0/2099261985 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef8c009550 con 0x7fef980cae50 2023-11-28T11:13:50.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.700+0000 7fef97fff700 1 --2- 172.21.15.28:0/2099261985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef980cae50 0x7fef98152130 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fef8400c4e0 tx=0x7fef8400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:50.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.700+0000 7fef957fa700 1 -- 172.21.15.28:0/2099261985 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840052d0 con 0x7fef980cae50 2023-11-28T11:13:50.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.700+0000 7fef957fa700 1 -- 172.21.15.28:0/2099261985 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef84005430 con 0x7fef980cae50 2023-11-28T11:13:50.714 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.700+0000 7fef957fa700 1 -- 172.21.15.28:0/2099261985 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef840038c0 con 0x7fef980cae50 2023-11-28T11:13:50.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.700+0000 7fef9edd1700 1 -- 172.21.15.28:0/2099261985 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef980ca7d0 con 0x7fef980cae50 2023-11-28T11:13:50.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.700+0000 7fef9edd1700 1 -- 172.21.15.28:0/2099261985 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef9806e770 con 0x7fef980cae50 2023-11-28T11:13:50.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.701+0000 7fef9edd1700 1 -- 172.21.15.28:0/2099261985 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9806ea10 con 0x7fef980cae50 2023-11-28T11:13:50.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.701+0000 7fef957fa700 1 -- 172.21.15.28:0/2099261985 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7fef84019030 con 0x7fef980cae50 2023-11-28T11:13:50.715 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.702+0000 7fef957fa700 1 --2- 172.21.15.28:0/2099261985 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fef7c0682f0 0x7fef7c06a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:50.716 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.702+0000 7fef957fa700 1 -- 172.21.15.28:0/2099261985 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7fef84084e80 con 0x7fef980cae50 2023-11-28T11:13:50.716 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.702+0000 7fef9cb6d700 1 --2- 172.21.15.28:0/2099261985 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fef7c0682f0 0x7fef7c06a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:50.716 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.702+0000 7fef9cb6d700 1 --2- 172.21.15.28:0/2099261985 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fef7c0682f0 0x7fef7c06a7a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fef8c014040 tx=0x7fef8c008030 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:50.717 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.706+0000 7fef957fa700 1 -- 172.21.15.28:0/2099261985 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7fef840518a0 con 0x7fef980cae50 2023-11-28T11:13:50.901 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.900+0000 7fef9edd1700 1 -- 172.21.15.28:0/2099261985 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fef980739f0 con 0x7fef980cae50 2023-11-28T11:13:50.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.905+0000 7fef957fa700 1 -- 172.21.15.28:0/2099261985 <== mon.0 v2:172.21.15.28:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fef84001240 con 0x7fef980cae50 2023-11-28T11:13:50.905 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.905+0000 7fef957fa700 1 -- 172.21.15.28:0/2099261985 <== mon.0 v2:172.21.15.28:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v16)=0 v16) v1 ==== 123+0+0 (secure 0 0 0) 0x7fef84054ec0 con 0x7fef980cae50 2023-11-28T11:13:50.908 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.907+0000 7fef9edd1700 1 -- 172.21.15.28:0/2099261985 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fef7c0682f0 msgr2=0x7fef7c06a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:50.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.907+0000 7fef9edd1700 1 --2- 172.21.15.28:0/2099261985 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fef7c0682f0 0x7fef7c06a7a0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fef8c014040 tx=0x7fef8c008030 comp rx=0 tx=0).stop 2023-11-28T11:13:50.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.907+0000 7fef9edd1700 1 -- 172.21.15.28:0/2099261985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef980cae50 msgr2=0x7fef98152130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:50.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.907+0000 7fef9edd1700 1 --2- 172.21.15.28:0/2099261985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef980cae50 0x7fef98152130 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fef8400c4e0 tx=0x7fef8400c8a0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.907+0000 7fef9edd1700 1 -- 172.21.15.28:0/2099261985 shutdown_connections 2023-11-28T11:13:50.909 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.907+0000 7fef9edd1700 1 --2- 172.21.15.28:0/2099261985 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7fef7c0682f0 0x7fef7c06a7a0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.907+0000 7fef9edd1700 1 --2- 172.21.15.28:0/2099261985 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7fef9815c950 0x7fef980aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.907+0000 7fef9edd1700 1 --2- 172.21.15.28:0/2099261985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fef980ca360 0x7fef980c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.907+0000 7fef9edd1700 1 --2- 172.21.15.28:0/2099261985 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7fef980cae50 0x7fef98152130 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:50.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.907+0000 7fef9edd1700 1 -- 172.21.15.28:0/2099261985 >> 172.21.15.28:0/2099261985 conn(0x7fef980b0df0 msgr2=0x7fef980b4140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:50.910 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.907+0000 7fef9edd1700 1 -- 172.21.15.28:0/2099261985 shutdown_connections 2023-11-28T11:13:50.911 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:50.907+0000 7fef9edd1700 1 -- 172.21.15.28:0/2099261985 wait complete. 2023-11-28T11:13:50.920 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T11:13:50.920 DEBUG:teuthology.orchestra.run.smithi028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T11:13:51.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.044+0000 7f25ed9a4700 1 Processor -- start 2023-11-28T11:13:51.045 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.045+0000 7f25ed9a4700 1 -- start start 2023-11-28T11:13:51.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.045+0000 7f25ed9a4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f25e806bf30 0x7f25e806c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:51.049 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.045+0000 7f25ed9a4700 1 -- --> v1:172.21.15.28:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e80b6350 con 0x7f25e815c640 2023-11-28T11:13:51.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.045+0000 7f25ed9a4700 1 -- --> v1:172.21.15.59:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25e806c870 con 0x7f25e80d2820 2023-11-28T11:13:51.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.045+0000 7f25ed9a4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e815ca10 con 0x7f25e806bf30 2023-11-28T11:13:51.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e6ffd700 1 --1- >> v1:172.21.15.28:6789/0 conn(0x7f25e815c640 0x7f25e80cf040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.28:6789/0 says I am v1:172.21.15.28:40884/0 (socket says 172.21.15.28:40884) 2023-11-28T11:13:51.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e6ffd700 1 -- 172.21.15.28:0/1929875703 learned_addr learned my addr 172.21.15.28:0/1929875703 (peer_addr_for_me v1:172.21.15.28:0/0) 2023-11-28T11:13:51.050 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e77fe700 1 --2- 172.21.15.28:0/1929875703 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f25e806bf30 0x7f25e806c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:51.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 <== mon.0 v1:172.21.15.28:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2007469073 0 0) 0x7f25e80b6350 con 0x7f25e815c640 2023-11-28T11:13:51.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 --> v1:172.21.15.28:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25d40035b0 con 0x7f25e815c640 2023-11-28T11:13:51.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 <== mon.1 v1:172.21.15.59:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249761852 0 0) 0x7f25e806c870 con 0x7f25e80d2820 2023-11-28T11:13:51.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 --> v1:172.21.15.59:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25e80b6350 con 0x7f25e80d2820 2023-11-28T11:13:51.051 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 <== mon.0 v1:172.21.15.28:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 30540862 0 0) 0x7f25d40035b0 con 0x7f25e815c640 2023-11-28T11:13:51.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f25e806bf30 msgr2=0x7f25e806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:51.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 --2- 172.21.15.28:0/1929875703 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f25e806bf30 0x7f25e806c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:51.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 >> v1:172.21.15.59:6789/0 conn(0x7f25e80d2820 legacy=0x7f25e80b5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:51.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 --> v1:172.21.15.28:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e80d1470 con 0x7f25e815c640 2023-11-28T11:13:51.052 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 <== mon.0 v1:172.21.15.28:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 4209547227 0 0) 0x7f25d8001670 con 0x7f25e815c640 2023-11-28T11:13:51.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e77fe700 1 --2- 172.21.15.28:0/1929875703 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f25e806bf30 0x7f25e806c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-11-28T11:13:51.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 >> v1:172.21.15.28:6789/0 conn(0x7f25e815c640 legacy=0x7f25e80cf040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:51.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 --2- 172.21.15.28:0/1929875703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25d4003bc0 0x7f25d4003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:51.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 --2- 172.21.15.28:0/1929875703 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f25e806bf30 0x7f25d4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:51.053 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 --2- 172.21.15.28:0/1929875703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25d4004d90 0x7f25d4005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:51.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e815ca10 con 0x7f25d4004d90 2023-11-28T11:13:51.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d400bac0 con 0x7f25d4003bc0 2023-11-28T11:13:51.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d400bc30 con 0x7f25e806bf30 2023-11-28T11:13:51.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e77fe700 1 --2- 172.21.15.28:0/1929875703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25d4004d90 0x7f25d4005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:51.054 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e6ffd700 1 --2- 172.21.15.28:0/1929875703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25d4003bc0 0x7f25d4003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:51.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.046+0000 7f25e67fc700 1 --2- 172.21.15.28:0/1929875703 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f25e806bf30 0x7f25d4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:51.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.047+0000 7f25e77fe700 1 -- 172.21.15.28:0/1929875703 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f25e806bf30 msgr2=0x7f25d4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:51.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.047+0000 7f25e77fe700 1 --2- 172.21.15.28:0/1929875703 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f25e806bf30 0x7f25d4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:51.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.047+0000 7f25e77fe700 1 -- 172.21.15.28:0/1929875703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25d4003bc0 msgr2=0x7f25d4003fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T11:13:51.055 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.047+0000 7f25e77fe700 1 --2- 172.21.15.28:0/1929875703 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25d4003bc0 0x7f25d4003fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:51.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.047+0000 7f25e77fe700 1 -- 172.21.15.28:0/1929875703 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e0009a70 con 0x7f25d4004d90 2023-11-28T11:13:51.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.047+0000 7f25e77fe700 1 --2- 172.21.15.28:0/1929875703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25d4004d90 0x7f25d4005200 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f25e000e350 tx=0x7f25d4005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66a795156dbcca42 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:51.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.047+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e000ed90 con 0x7f25d4004d90 2023-11-28T11:13:51.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.047+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f25e000f040 con 0x7f25d4004d90 2023-11-28T11:13:51.056 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.047+0000 7f25e5ffb700 1 -- 172.21.15.28:0/1929875703 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e00146c0 con 0x7f25d4004d90 2023-11-28T11:13:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.047+0000 7f25ed9a4700 1 -- 172.21.15.28:0/1929875703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25d4004d90 msgr2=0x7f25d4005200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.047+0000 7f25ed9a4700 1 --2- 172.21.15.28:0/1929875703 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25d4004d90 0x7f25d4005200 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f25e000e350 tx=0x7f25d4005880 comp rx=0 tx=0).stop 2023-11-28T11:13:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.047+0000 7f25e77fe700 1 -- 172.21.15.28:0/1929875703 reap_dead start 2023-11-28T11:13:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.048+0000 7f25ed9a4700 1 -- 172.21.15.28:0/1929875703 shutdown_connections 2023-11-28T11:13:51.057 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.048+0000 7f25ed9a4700 1 -- 172.21.15.28:0/1929875703 >> 172.21.15.28:0/1929875703 conn(0x7f25e80d2df0 msgr2=0x7f25e80d31f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:51.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.048+0000 7f25ed9a4700 1 -- 172.21.15.28:0/1929875703 shutdown_connections 2023-11-28T11:13:51.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.048+0000 7f25ed9a4700 1 -- 172.21.15.28:0/1929875703 wait complete. 2023-11-28T11:13:51.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.049+0000 7f25ed9a4700 1 Processor -- start 2023-11-28T11:13:51.058 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.049+0000 7f25ed9a4700 1 -- start start 2023-11-28T11:13:51.060 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.050+0000 7f25ed9a4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25e815f210 0x7f25e80cac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.050+0000 7f25ed9a4700 1 --2- >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f25e815f5e0 0x7f25e81553f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.050+0000 7f25ed9a4700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25e81f5f00 0x7f25e814ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.050+0000 7f25ed9a4700 1 -- --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e8155930 con 0x7f25e81f5f00 2023-11-28T11:13:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.050+0000 7f25ed9a4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25e81f66b0 con 0x7f25e815f210 2023-11-28T11:13:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.050+0000 7f25ed9a4700 1 -- --> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25e815cc30 con 0x7f25e815f5e0 2023-11-28T11:13:51.061 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.050+0000 7f25e77fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25e81f5f00 0x7f25e814ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:51.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.050+0000 7f25e77fe700 1 --2- >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25e81f5f00 0x7f25e814ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.28:3300/0 says I am v2:172.21.15.28:52842/0 (socket says 172.21.15.28:52842) 2023-11-28T11:13:51.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.050+0000 7f25e77fe700 1 -- 172.21.15.28:0/155572305 learned_addr learned my addr 172.21.15.28:0/155572305 (peer_addr_for_me v2:172.21.15.28:0/0) 2023-11-28T11:13:51.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.050+0000 7f25e77fe700 1 -- 172.21.15.28:0/155572305 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f25e815f5e0 msgr2=0x7f25e81553f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:51.062 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.050+0000 7f25e77fe700 1 --2- 172.21.15.28:0/155572305 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f25e815f5e0 0x7f25e81553f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:51.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.050+0000 7f25e77fe700 1 -- 172.21.15.28:0/155572305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25e815f210 msgr2=0x7f25e80cac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:51.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.050+0000 7f25e77fe700 1 --2- 172.21.15.28:0/155572305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25e815f210 0x7f25e80cac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:51.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.050+0000 7f25e77fe700 1 -- 172.21.15.28:0/155572305 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25e80cd7c0 con 0x7f25e81f5f00 2023-11-28T11:13:51.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.050+0000 7f25e77fe700 1 --2- 172.21.15.28:0/155572305 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25e81f5f00 0x7f25e814ad70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f25e0016000 tx=0x7f25e0009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:51.063 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.051+0000 7f25dffff700 1 -- 172.21.15.28:0/155572305 <== mon.0 v2:172.21.15.28:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e0009410 con 0x7f25e81f5f00 2023-11-28T11:13:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.051+0000 7f25dffff700 1 -- 172.21.15.28:0/155572305 <== mon.0 v2:172.21.15.28:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f25e0014d70 con 0x7f25e81f5f00 2023-11-28T11:13:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.051+0000 7f25dffff700 1 -- 172.21.15.28:0/155572305 <== mon.0 v2:172.21.15.28:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25e001d680 con 0x7f25e81f5f00 2023-11-28T11:13:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.051+0000 7f25ed9a4700 1 -- 172.21.15.28:0/155572305 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25e80cd4e0 con 0x7f25e81f5f00 2023-11-28T11:13:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.051+0000 7f25ed9a4700 1 -- 172.21.15.28:0/155572305 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25e80d1710 con 0x7f25e81f5f00 2023-11-28T11:13:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.052+0000 7f25ed9a4700 1 -- 172.21.15.28:0/155572305 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25e8068570 con 0x7f25e81f5f00 2023-11-28T11:13:51.064 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.052+0000 7f25dffff700 1 -- 172.21.15.28:0/155572305 <== mon.0 v2:172.21.15.28:3300/0 4 ==== mgrmap(e 5) v1 ==== 86415+0+0 (secure 0 0 0) 0x7f25e001d7e0 con 0x7f25e81f5f00 2023-11-28T11:13:51.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.052+0000 7f25dffff700 1 --2- 172.21.15.28:0/155572305 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f25c80682f0 0x7f25c806a7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-11-28T11:13:51.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.052+0000 7f25dffff700 1 -- 172.21.15.28:0/155572305 <== mon.0 v2:172.21.15.28:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 5616+0+0 (secure 0 0 0) 0x7f25e0087210 con 0x7f25e81f5f00 2023-11-28T11:13:51.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.052+0000 7f25e6ffd700 1 --2- 172.21.15.28:0/155572305 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f25c80682f0 0x7f25c806a7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-11-28T11:13:51.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.053+0000 7f25e6ffd700 1 --2- 172.21.15.28:0/155572305 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f25c80682f0 0x7f25c806a7a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f25e8157f80 tx=0x7f25d8004870 comp rx=0 tx=0).ready entity=mgr.4098 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T11:13:51.065 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.057+0000 7f25dffff700 1 -- 172.21.15.28:0/155572305 <== mon.0 v2:172.21.15.28:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173508 (secure 0 0 0) 0x7f25e0053c30 con 0x7f25e81f5f00 2023-11-28T11:13:51.294 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.294+0000 7f25ed9a4700 1 -- 172.21.15.28:0/155572305 --> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f25e80b4650 con 0x7f25e81f5f00 2023-11-28T11:13:51.295 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.294+0000 7f25dffff700 1 -- 172.21.15.28:0/155572305 <== mon.0 v2:172.21.15.28:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f25e0018de0 con 0x7f25e81f5f00 2023-11-28T11:13:51.295 INFO:teuthology.orchestra.run.smithi028.stdout: 2023-11-28T11:13:51.296 INFO:teuthology.orchestra.run.smithi028.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5750,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6834/2653591133","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":2653591133},{"type":"v1","addr":"172.21.15.59:6834","nonce":2653591133}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":36},{"gid":5860,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/1564028052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6833","nonce":1564028052},{"type":"v1","addr":"172.21.15.59:6835","nonce":1564028052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6558,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6836/2509041002","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6834","nonce":2509041002},{"type":"v1","addr":"172.21.15.28:6836","nonce":2509041002}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28},{"gid":6561,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.28:6837/3994419079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.28:6835","nonce":3994419079},{"type":"v1","addr":"172.21.15.28:6837","nonce":3994419079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":28}],"filesystems":[]} 2023-11-28T11:13:51.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.296+0000 7f25ed9a4700 1 -- 172.21.15.28:0/155572305 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f25c80682f0 msgr2=0x7f25c806a7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:51.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.296+0000 7f25ed9a4700 1 --2- 172.21.15.28:0/155572305 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f25c80682f0 0x7f25c806a7a0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f25e8157f80 tx=0x7f25d8004870 comp rx=0 tx=0).stop 2023-11-28T11:13:51.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.296+0000 7f25ed9a4700 1 -- 172.21.15.28:0/155572305 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25e81f5f00 msgr2=0x7f25e814ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T11:13:51.297 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.296+0000 7f25ed9a4700 1 --2- 172.21.15.28:0/155572305 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25e81f5f00 0x7f25e814ad70 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f25e0016000 tx=0x7f25e0009750 comp rx=0 tx=0).stop 2023-11-28T11:13:51.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.296+0000 7f25ed9a4700 1 -- 172.21.15.28:0/155572305 shutdown_connections 2023-11-28T11:13:51.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.296+0000 7f25ed9a4700 1 --2- 172.21.15.28:0/155572305 >> [v2:172.21.15.28:6832/103239,v1:172.21.15.28:6833/103239] conn(0x7f25c80682f0 0x7f25c806a7a0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:51.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.296+0000 7f25ed9a4700 1 --2- 172.21.15.28:0/155572305 >> [v2:172.21.15.28:3300/0,v1:172.21.15.28:6789/0] conn(0x7f25e81f5f00 0x7f25e814ad70 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:51.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.296+0000 7f25ed9a4700 1 --2- 172.21.15.28:0/155572305 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25e815f210 0x7f25e80cac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:51.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.296+0000 7f25ed9a4700 1 --2- 172.21.15.28:0/155572305 >> [v2:172.21.15.59:3301/0,v1:172.21.15.59:6790/0] conn(0x7f25e815f5e0 0x7f25e81553f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-11-28T11:13:51.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.296+0000 7f25ed9a4700 1 -- 172.21.15.28:0/155572305 >> 172.21.15.28:0/155572305 conn(0x7f25e80d2df0 msgr2=0x7f25e80b5c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T11:13:51.298 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.297+0000 7f25ed9a4700 1 -- 172.21.15.28:0/155572305 shutdown_connections 2023-11-28T11:13:51.299 INFO:teuthology.orchestra.run.smithi028.stderr:2023-11-28T11:13:51.297+0000 7f25ed9a4700 1 -- 172.21.15.28:0/155572305 wait complete. 2023-11-28T11:13:51.299 INFO:teuthology.orchestra.run.smithi028.stderr:dumped fsmap epoch 36 2023-11-28T11:13:51.309 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-28T11:13:51.309 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-28T11:13:51.310 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T11:13:51.310 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:51.318 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T11:13:51.319 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T11:13:51.319 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:51.367 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T11:13:51.367 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T11:13:51.367 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:51.379 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T11:13:51.379 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T11:13:51.379 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:51.431 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T11:13:51.431 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-28T11:13:51.431 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-28T11:13:51.431 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:51.469 INFO:tasks.ceph.osd.0:Stopped 2023-11-28T11:13:51.469 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-28T11:13:51.469 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:51.540 INFO:tasks.ceph.osd.1:Stopped 2023-11-28T11:13:51.540 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-28T11:13:51.540 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:51.587 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:13:51.589 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:13:51.611 INFO:tasks.ceph.osd.2:Stopped 2023-11-28T11:13:51.611 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-28T11:13:51.611 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:51.672 INFO:tasks.ceph.osd.3:Stopped 2023-11-28T11:13:51.672 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-28T11:13:51.672 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:51.711 INFO:tasks.ceph.osd.4:Stopped 2023-11-28T11:13:51.711 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-28T11:13:51.711 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:51.794 INFO:tasks.ceph.osd.5:Stopped 2023-11-28T11:13:51.794 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-28T11:13:51.794 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:51.833 INFO:tasks.ceph.osd.6:Stopped 2023-11-28T11:13:51.833 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-28T11:13:51.833 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:51.907 INFO:tasks.ceph.osd.7:Stopped 2023-11-28T11:13:51.908 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-28T11:13:51.908 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-28T11:13:51.908 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:51.934 INFO:tasks.ceph.mgr.y:Stopped 2023-11-28T11:13:51.934 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-28T11:13:51.934 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:51.972 INFO:tasks.ceph.mgr.x:Stopped 2023-11-28T11:13:51.973 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-28T11:13:51.973 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-28T11:13:51.973 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:51.986 INFO:tasks.ceph.mon.a:Stopped 2023-11-28T11:13:51.986 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-28T11:13:51.986 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:52.025 INFO:tasks.ceph.mon.b:Stopped 2023-11-28T11:13:52.025 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-28T11:13:52.025 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T11:13:52.077 INFO:tasks.ceph.mon.c:Stopped 2023-11-28T11:13:52.078 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-28T11:13:52.078 DEBUG:teuthology.orchestra.run.smithi028:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'inode wrongly marked free' | egrep -v 'bad backtrace on inode' | egrep -v 'inode table repaired for inode' | egrep -v 'Scrub error on inode' | egrep -v 'Scrub error on dir' | egrep -v 'Metadata damage detected' | head -n 1 2023-11-28T11:13:52.113 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:13:52.113 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-28T11:13:52.403 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:13:52.403 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-28T11:13:52.544 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:13:52.544 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-28T11:13:52.631 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:13:52.632 DEBUG:teuthology.orchestra.run.smithi028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-28T11:13:52.738 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:13:52.739 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-28T11:13:52.925 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:13:52.925 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-28T11:13:53.023 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:13:53.023 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-28T11:13:53.104 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:13:53.105 DEBUG:teuthology.orchestra.run.smithi059:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-28T11:13:53.205 INFO:tasks.ceph:Archiving mon data... 2023-11-28T11:13:53.206 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957/data/mon.a.tgz 2023-11-28T11:13:53.206 DEBUG:teuthology.orchestra.run.smithi028:> mktemp 2023-11-28T11:13:53.224 INFO:teuthology.orchestra.run.smithi028.stdout:/tmp/tmp.Etm3A44Ptj 2023-11-28T11:13:53.225 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Etm3A44Ptj 2023-11-28T11:13:53.390 DEBUG:teuthology.orchestra.run.smithi028:> sudo chmod 0666 /tmp/tmp.Etm3A44Ptj 2023-11-28T11:13:53.477 DEBUG:teuthology.orchestra.remote:smithi028:/tmp/tmp.Etm3A44Ptj is 415KB 2023-11-28T11:13:53.574 DEBUG:teuthology.orchestra.run.smithi028:> rm -fr /tmp/tmp.Etm3A44Ptj 2023-11-28T11:13:53.591 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957/data/mon.b.tgz 2023-11-28T11:13:53.591 DEBUG:teuthology.orchestra.run.smithi059:> mktemp 2023-11-28T11:13:53.609 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.AbIltH9F8J 2023-11-28T11:13:53.610 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.AbIltH9F8J 2023-11-28T11:13:53.819 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.AbIltH9F8J 2023-11-28T11:13:53.906 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.AbIltH9F8J is 539KB 2023-11-28T11:13:54.005 DEBUG:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.AbIltH9F8J 2023-11-28T11:13:54.022 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957/data/mon.c.tgz 2023-11-28T11:13:54.022 DEBUG:teuthology.orchestra.run.smithi059:> mktemp 2023-11-28T11:13:54.078 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.1VluuuL6Ep 2023-11-28T11:13:54.078 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.1VluuuL6Ep 2023-11-28T11:13:54.252 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.1VluuuL6Ep 2023-11-28T11:13:54.340 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.1VluuuL6Ep is 539KB 2023-11-28T11:13:54.439 DEBUG:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.1VluuuL6Ep 2023-11-28T11:13:54.456 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-28T11:13:54.457 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T11:13:54.459 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T11:13:54.523 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-28T11:13:54.524 DEBUG:teuthology.orchestra.run.smithi028:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T11:13:54.533 DEBUG:teuthology.orchestra.run.smithi059:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T11:13:54.570 INFO:teuthology.orchestra.run.smithi028.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T11:13:54.600 INFO:teuthology.orchestra.run.smithi059.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T11:13:54.606 INFO:tasks.ceph:Archiving crash dumps... 2023-11-28T11:13:54.607 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957/remote/smithi028/crash 2023-11-28T11:13:54.608 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T11:13:54.647 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/crash to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957/remote/smithi059/crash 2023-11-28T11:13:54.648 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T11:13:54.675 INFO:tasks.ceph:Shutting down logrotate 2023-11-28T11:13:54.675 DEBUG:teuthology.orchestra.run.smithi028:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:13:54.689 DEBUG:teuthology.orchestra.run.smithi059:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T11:13:54.787 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T11:13:54.816 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T11:13:54.843 INFO:tasks.ceph:Compressing logs... 2023-11-28T11:13:54.843 DEBUG:teuthology.orchestra.run.smithi028:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:13:54.859 DEBUG:teuthology.orchestra.run.smithi059:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:13:59.544 INFO:tasks.ceph:Archiving logs... 2023-11-28T11:13:59.545 DEBUG:teuthology.misc:Transferring archived files from smithi028:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957/remote/smithi028/log 2023-11-28T11:13:59.546 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T11:14:00.412 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/log/ceph to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957/remote/smithi059/log 2023-11-28T11:14:00.413 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T11:14:00.949 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-28T11:14:00.958 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-28T11:14:00.958 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T11:14:00.961 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T11:14:01.061 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-28T11:14:01.070 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-28T11:14:01.070 DEBUG:teuthology.orchestra.run.smithi028:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:14:01.073 DEBUG:teuthology.orchestra.run.smithi059:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T11:14:01.090 INFO:teuthology.orchestra.run.smithi028.stderr:bash: ntpq: command not found 2023-11-28T11:14:01.092 INFO:teuthology.orchestra.run.smithi059.stderr:bash: ntpq: command not found 2023-11-28T11:14:01.095 INFO:teuthology.orchestra.run.smithi028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:14:01.098 INFO:teuthology.orchestra.run.smithi028.stdout:=============================================================================== 2023-11-28T11:14:01.098 INFO:teuthology.orchestra.run.smithi028.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 5 +835us[ +831us] +/- 81ms 2023-11-28T11:14:01.098 INFO:teuthology.orchestra.run.smithi028.stdout:^* hv02.front.sepia.ceph.com 3 6 377 4 -664us[ -668us] +/- 55ms 2023-11-28T11:14:01.098 INFO:teuthology.orchestra.run.smithi028.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 6 +175us[ +171us] +/- 74ms 2023-11-28T11:14:01.099 INFO:teuthology.orchestra.run.smithi028.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:14:01.100 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T11:14:01.100 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2023-11-28T11:14:01.101 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 4 +826us[ +850us] +/- 81ms 2023-11-28T11:14:01.101 INFO:teuthology.orchestra.run.smithi059.stdout:^* hv02.front.sepia.ceph.com 3 6 377 3 -662us[ -637us] +/- 55ms 2023-11-28T11:14:01.101 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +190us[ +214us] +/- 74ms 2023-11-28T11:14:01.101 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T11:14:01.101 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-28T11:14:01.111 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-28T11:14:01.111 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-28T11:14:01.144 DEBUG:teuthology.orchestra.run.smithi028:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T11:14:01.261 DEBUG:teuthology.orchestra.run.smithi059:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T11:14:01.367 DEBUG:teuthology.orchestra.run.smithi028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:14:01.402 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:14:01.402 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T11:14:01.436 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:14:01.437 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-28T11:14:01.447 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-28T11:14:01.469 INFO:teuthology.task.internal:Duration was 1118.985502 seconds 2023-11-28T11:14:01.469 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-28T11:14:01.519 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-28T11:14:01.520 DEBUG:teuthology.orchestra.run.smithi028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T11:14:01.522 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T11:14:01.570 INFO:teuthology.orchestra.run.smithi028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:14:01.572 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T11:14:02.041 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-28T11:14:02.041 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi028.front.sepia.ceph.com 2023-11-28T11:14:02.041 DEBUG:teuthology.orchestra.run.smithi028:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T11:14:02.065 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi059.front.sepia.ceph.com 2023-11-28T11:14:02.065 DEBUG:teuthology.orchestra.run.smithi059:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T11:14:02.089 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-28T11:14:02.089 DEBUG:teuthology.orchestra.run.smithi028:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:14:02.108 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T11:14:02.173 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-28T11:14:02.173 DEBUG:teuthology.orchestra.run.smithi028:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T11:14:02.176 DEBUG:teuthology.orchestra.run.smithi059:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T11:14:02.296 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-28T11:14:02.307 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-28T11:14:02.307 DEBUG:teuthology.orchestra.run.smithi028:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T11:14:02.335 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T11:14:02.364 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-28T11:14:02.399 DEBUG:teuthology.orchestra.run.smithi028:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:14:02.401 DEBUG:teuthology.orchestra.run.smithi059:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:14:02.426 INFO:teuthology.orchestra.run.smithi028.stdout:kernel.core_pattern = core 2023-11-28T11:14:02.430 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = core 2023-11-28T11:14:02.444 DEBUG:teuthology.orchestra.run.smithi028:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:14:02.496 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:14:02.497 DEBUG:teuthology.orchestra.run.smithi059:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T11:14:02.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T11:14:02.514 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-28T11:14:02.527 INFO:teuthology.task.internal:Transferring archived files... 2023-11-28T11:14:02.528 DEBUG:teuthology.misc:Transferring archived files from smithi028:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957/remote/smithi028 2023-11-28T11:14:02.528 DEBUG:teuthology.orchestra.run.smithi028:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T11:14:02.579 DEBUG:teuthology.misc:Transferring archived files from smithi059:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957/remote/smithi059 2023-11-28T11:14:02.579 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T11:14:02.618 INFO:teuthology.task.internal:Removing archive directory... 2023-11-28T11:14:02.618 DEBUG:teuthology.orchestra.run.smithi028:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T11:14:02.621 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T11:14:02.669 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-28T11:14:02.680 INFO:teuthology.task.internal:Not uploading archives. 2023-11-28T11:14:02.680 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-28T11:14:02.713 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-28T11:14:02.713 DEBUG:teuthology.orchestra.run.smithi028:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T11:14:02.716 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T11:14:02.732 INFO:teuthology.orchestra.run.smithi028.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 11:14 /home/ubuntu/cephtest 2023-11-28T11:14:02.733 INFO:teuthology.orchestra.run.smithi059.stdout: 262307 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 11:14 /home/ubuntu/cephtest 2023-11-28T11:14:02.734 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-28T11:14:02.750 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-28T11:14:02.815 INFO:teuthology.nuke:Checking targets against current locks 2023-11-28T11:14:02.843 DEBUG:teuthology.nuke:shortname: smithi028 2023-11-28T11:14:02.844 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T11:14:02.868 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi028.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:50:16.898860', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T11:14:02.872 DEBUG:teuthology.nuke:shortname: smithi059 2023-11-28T11:14:02.872 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T11:14:02.897 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/neesingh-2023-11-28_10:48:03-fs-wip-neesingh-testing-231123-distro-default-smithi/7469957', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 10:50:16.900710', 'locked_by': 'scheduled_neesingh@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLVvN3+LmzPZ4ixiwyCkS3KZLZ9R6MOPm+kPtaJbl8fExpzbk8TudXCcJWHBL7DGPWFvye2jT6sei9eiX0EUQYs='} 2023-11-28T11:14:02.927 INFO:teuthology.orchestra.console.smithi028:Power off 2023-11-28T11:14:02.928 DEBUG:teuthology.orchestra.console.smithi028:pexpect command: ipmitool -H smithi028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T11:14:02.951 INFO:teuthology.orchestra.console.smithi059:Power off 2023-11-28T11:14:02.952 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T11:14:02.967 DEBUG:teuthology.orchestra.console.smithi028:power off output: Chassis Power Control: Down/Off 2023-11-28T11:14:02.968 DEBUG:teuthology.orchestra.console.smithi028:pexpect command: ipmitool -H smithi028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:14:02.977 DEBUG:teuthology.orchestra.console.smithi059:power off output: Chassis Power Control: Down/Off 2023-11-28T11:14:02.977 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:14:02.990 DEBUG:teuthology.orchestra.console.smithi028:check power output: Chassis Power is on 2023-11-28T11:14:02.999 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2023-11-28T11:14:06.992 DEBUG:teuthology.orchestra.console.smithi028:pexpect command: ipmitool -H smithi028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:14:07.001 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:14:07.103 DEBUG:teuthology.orchestra.console.smithi028:check power output: Chassis Power is on 2023-11-28T11:14:07.111 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is on 2023-11-28T11:14:11.106 DEBUG:teuthology.orchestra.console.smithi028:pexpect command: ipmitool -H smithi028.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:14:11.115 DEBUG:teuthology.orchestra.console.smithi059:pexpect command: ipmitool -H smithi059.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T11:14:11.217 DEBUG:teuthology.orchestra.console.smithi028:check power output: Chassis Power is off 2023-11-28T11:14:11.224 DEBUG:teuthology.orchestra.console.smithi059:check power output: Chassis Power is off 2023-11-28T11:14:11.318 INFO:teuthology.orchestra.console.smithi028:Power off completed 2023-11-28T11:14:11.325 INFO:teuthology.orchestra.console.smithi059:Power off completed 2023-11-28T11:14:11.430 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/fuse objectstore/bluestore-ec-root overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/forward-scrub} duration: 1118.985502243042 failure_reason: 'Test failure: test_health_status_after_backtrace_repair (tasks.cephfs.test_forward_scrub.TestForwardScrub)' flavor: default owner: scheduled_neesingh@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=f9ae253d36d34e5c92804e361ee81e0e status: fail success: false 2023-11-28T11:14:11.431 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T11:14:11.518 INFO:teuthology.run:FAIL